Positioning method, device, electronic device and storage medium

Abstract

The present disclosure provides a positioning method, device, electronic device and storage medium. The method includes: collecting an initial image of the robot's activity area based on an infrared camera module, the robot deployed in the robot's activity area is provided with a position dot matrix consisting of multiple positions including a vertex position, two bottom corner positions and at least one coding position, the position dot matrix represents different robot numbers through different luminous states at each coding position, the spacing formed by the vertex position and each bottom corner position is known and is the maximum two spacings formed by multiple positions; determining each luminous position contained in the initial image, and determining the position dot matrix to which each luminous position contained in the initial image belongs; based on the luminous condition of each position in each position dot matrix contained in the initial image, determining the robot number represented by the position dot matrix, and determining the position of the robot corresponding to the robot number according to the position of the position dot matrix in the initial image.

Description

Technical Field

[0001] The present disclosure relates to the technical field of robot dance formation positioning, and in particular to a positioning method, device, electronic equipment and storage medium. Background Art

[0002] In the field of legged robot dance formation control, as the cost of legged robots decreases, application scenarios such as multi-robot formation movement and group dance control have emerged one after another. In these scenarios, the main control device usually collects the position of each robot in the robot dance formation and makes adaptive command adjustments in real time according to the positions of these robots.

[0003] Related technologies often paste QR codes on robots and rely on high-resolution cameras for image recognition to complete positioning. However, this positioning method is not only extremely costly, but also requires a reserved area for QR code pasting on the robot body. It is also easily affected by common negative factors such as limited camera field of view, QR code morphological distortion and motion blur caused by robot movement, resulting in poor positioning results. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a positioning method, device, electronic device and storage medium.

[0005] A first aspect of the present disclosure provides a positioning method, the method comprising:

[0006] Acquire an initial image of a robot activity area, wherein at least one robot is deployed in the robot activity area, and the robot is provided with a position matrix, wherein the position matrix includes a vertex position, two bottom corner positions, and at least one coding position, and the position matrix is ​​used to represent different robot numbers through different luminous states at each coding position, and the spacings between the vertex position and the two bottom corner positions are respectively a first spacing and a second spacing, and the first spacing and the second spacing are both greater than the spacings between any other two positions in the position matrix, and during the positioning process, the vertex position and the two bottom corner positions are in a constantly lit state;

[0007] Determine each light-emitting position contained in the initial image, and determine a position matrix to which each light-emitting position contained in the initial image belongs according to the first spacing and the second spacing;

[0008] Based on the luminescence of each position in each position matrix contained in the initial image, the robot number represented by the position matrix is ​​determined, and according to the position of the position matrix in the initial image, the position of the robot corresponding to the robot number is determined.

[0009] Optionally, determining each light emitting position contained in the initial image includes:

[0010] Clustering pixels in the initial image whose grayscale values ​​are greater than a grayscale threshold according to the imaging size of the light source provided on the robot in the initial image, so as to determine a pending light emitting position contained in the initial image;

[0011] The pending light-emitting positions are screened according to the imaging size and / or imaging shape of the light source arranged on the robot in the initial image to determine each light-emitting position contained in the initial image.

[0012] Optionally, before determining the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix included in the initial image, the method further includes:

[0013] For each position dot matrix contained in the initial image, a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold is determined in at least one previous frame of the initial image, and based on the luminescence condition of each position in the historical position dot matrix, the luminescence condition of each position in the position dot matrix is ​​corrected.

[0014] Optionally, before determining the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix included in the initial image, the method further includes:

[0015] For each position lattice included in the initial image, determine a historical position lattice whose spacing from the position lattice is less than a displacement threshold in at least one previous frame of the initial image, and correct the position lattice through rotation and translation transformation based on the robot posture reported by the robot corresponding to the robot number represented by the historical position lattice;

[0016] And / or, based on the posture of the infrared camera module, each position dot matrix included in the initial image is corrected through rotation and translation transformation, wherein the initial image is acquired through the infrared camera module.

[0017] Optionally, the method further includes:

[0018] For each position matrix included in the initial image, determine the two largest spacings among the spacings formed by the light-emitting positions included in the position matrix, and discard the position matrix when the difference between the two largest spacings is greater than an error threshold, wherein the first spacing and the second spacing are equal; and / or,

[0019] For each position matrix contained in the initial image, a historical position matrix whose spacing with the position matrix is ​​less than a displacement threshold is determined in at least one previous frame of the initial image; when the robot number represented by the position matrix determined based on the position matrix is ​​inconsistent with the robot number represented by the historical position matrix, the position matrix is ​​discarded.

[0020] Optionally, the method further includes:

[0021] For each position dot matrix contained in the initial image, the vertex luminous positions and the bottom corner luminous positions of the position dot matrix are determined according to the two largest intervals among the intervals formed by the luminous positions included in the position dot matrix, and the direction of the position dot matrix and / or the direction of the robot corresponding to the robot number represented by the position dot matrix are determined according to the vertex luminous positions and the bottom corner luminous positions of the position dot matrix.

[0022] Optionally, determining the position of the robot corresponding to the robot number according to the position of the position matrix in the initial image includes:

[0023] According to the internal parameters of the infrared camera module and the rotation matrix and translation vector of the infrared camera module relative to the world coordinate system, the position of the position matrix in the initial image is transformed to obtain the position of the robot corresponding to the robot number in the world coordinate system, wherein the initial image is acquired by the infrared camera module.

[0024] Optionally, the method further includes:

[0025] The position of the robot is sent to the robot so that the robot updates the current position, wherein the robot is used to execute the current action instruction according to the current position.

[0026] Optionally, the robot is provided with a local odometer, and the robot is used to determine the current position based on the local odometer;

[0027] The sending the position of the robot to the robot so that the robot updates the current position includes:

[0028] The position of the robot is sent to the robot, so that the robot performs complementary filtering on the received position and the position determined based on the local odometer, and updates the current position.

[0029] Optionally, the method further includes:

[0030] Determine, according to the position of the robot, a current action instruction and at least one backup action instruction corresponding to the robot;

[0031] Sending the current action instruction and the at least one backup action instruction to the robot so that the robot executes the current action instruction, and updates the backup action instruction set according to the received at least one backup action instruction;

[0032] Among them, the backup action instruction has a timestamp used to characterize the target execution time of the backup action instruction, and the robot is used to execute the current action instruction when the current action instruction is received within the waiting time, and execute the backup action instruction in the backup action instruction set when the current action instruction is not received within the waiting time.

[0033] Optionally, determining, according to the position of the robot, a current action instruction and at least one backup action instruction corresponding to the robot includes:

[0034] According to the position of the at least one robot, a current action instruction and at least one backup action instruction corresponding to each robot are determined by a conflict search-based method and an A-star path planning algorithm carrying a time dimension.

[0035] Optionally, determining, according to the position of the robot, a current action instruction and at least one backup action instruction corresponding to the robot includes:

[0036] According to the position of the robot and the current posture reported by the robot, a current action instruction and at least one backup action instruction corresponding to the robot are determined.

[0037] Optionally, each robot is connected to the main control device based on a local area network, and the method further includes:

[0038] Receive a feedback frame sent by the robot, determine the communication delay between the main control device and the robot according to the feedback frame, and send the communication delay to the robot so that the robot determines the execution time of the current action instruction according to the communication delay, wherein the feedback frame is sent by the robot when receiving the trigger frame sent by the main control device, and the feedback frame includes the sending time of the trigger frame.

[0039] Optionally, the initial image is acquired by collecting an infrared camera module, and the infrared camera module includes a filter, and the filter is used to filter out light of a frequency at which infrared rays emitted by the light source exist.

[0040] A second aspect of the present disclosure provides a positioning device, the device comprising:

[0041] An image acquisition module, used to obtain an initial image of a robot activity area, wherein at least one robot is deployed in the robot activity area, and the robot is provided with a position matrix, wherein the position matrix includes a vertex position, two bottom corner positions and at least one coding position, and the position matrix is ​​used to represent different robot numbers by different luminous states at each coding position, and the spacings between the vertex position and the two bottom corner positions are respectively a first spacing and a second spacing, and the first spacing and the second spacing are both greater than the spacings between any other two positions in the position matrix, and during the positioning process, the vertex position and the two bottom corner positions are in a constantly lit state;

[0042] a dot matrix determination module, used to determine each light-emitting position contained in the initial image, and determine a position dot matrix to which each light-emitting position contained in the initial image belongs according to the first spacing and the second spacing;

[0043] A position determination module is used to determine the robot number represented by the position dot matrix based on the luminescence of each position in each position dot matrix contained in the initial image, and to determine the position of the robot corresponding to the robot number according to the position of the position dot matrix in the initial image.

[0044] Optionally, when the dot matrix determination module is used to determine each light-emitting position contained in the initial image, it is specifically used to:

[0045] Clustering pixels in the initial image whose grayscale values ​​are greater than a grayscale threshold according to the imaging size of the light source provided on the robot in the initial image, so as to determine a pending light emitting position contained in the initial image;

[0046] The pending light-emitting positions are screened according to the imaging size and / or imaging shape of the light source arranged on the robot in the initial image to determine each light-emitting position contained in the initial image.

[0047] Optionally, the device further comprises:

[0048] A first correction module is used to determine, for each position dot matrix contained in the initial image, a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold in at least one previous frame of the initial image before determining the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix contained in the initial image, and correct the luminescence condition of each position in the position dot matrix based on the luminescence condition of each position in the historical position dot matrix.

[0049] Optionally, the device further includes a second correction module, which is used to perform the following steps before determining the robot number represented by the position dot matrix based on the luminous condition of each position in each position dot matrix contained in the initial image:

[0050] For each position dot matrix included in the initial image, determine a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold in at least one previous frame of the initial image, and based on the robot posture reported by the robot corresponding to the robot number represented by the historical position dot matrix, correct the position dot matrix through rotation and translation transformation; and / or,

[0051] Based on the posture of the infrared camera module, each position dot matrix included in the initial image is corrected through rotation and translation transformation, wherein the initial image is acquired through the infrared camera module.

[0052] Optionally, the device further comprises a lattice filtering module, configured to perform the following steps:

[0053] For each position matrix included in the initial image, determine the two largest spacings among the spacings formed by the light-emitting positions included in the position matrix, and discard the position matrix when the difference between the two largest spacings is greater than an error threshold, wherein the first spacing and the second spacing are equal; and / or,

[0054] For each position matrix contained in the initial image, a historical position matrix whose spacing with the position matrix is ​​less than a displacement threshold is determined in at least one previous frame of the initial image; when the robot number represented by the position matrix determined based on the position matrix is ​​inconsistent with the robot number represented by the historical position matrix, the position matrix is ​​discarded.

[0055] Optionally, the device further comprises:

[0056] A direction determination module is used to determine, for each position dot matrix contained in the initial image, the vertex luminous positions and the bottom corner luminous positions of the position dot matrix according to the two largest intervals among the intervals formed by the luminous positions included in the position dot matrix, and to determine, based on the vertex luminous positions and the bottom corner luminous positions of the position dot matrix, the direction of the position dot matrix and / or the direction of the robot corresponding to the robot number represented by the position dot matrix.

[0057] Optionally, when the position determination module is used to determine the position of the robot corresponding to the robot number according to the position of the position matrix in the initial image, it is specifically used to:

[0058] According to the internal parameters of the infrared camera module and the rotation matrix and translation vector of the infrared camera module relative to the world coordinate system, the position of the position matrix in the initial image is transformed to obtain the position of the robot corresponding to the robot number in the world coordinate system, wherein the initial image is acquired by the infrared camera module.

[0059] Optionally, the device further comprises:

[0060] The position sending module is used to send the position of the robot to the robot so that the robot updates the current position, wherein the robot is used to execute the current action instruction according to the current position.

[0061] Optionally, the robot is provided with a local odometer, and the robot is used to determine the current position based on the local odometer;

[0062] The position sending module is used to send the position of the robot to the robot so that the robot updates the current position, and is specifically used to:

[0063] The position of the robot is sent to the robot, so that the robot performs complementary filtering on the received position and the position determined based on the local odometer, and updates the current position.

[0064] Optionally, the device further comprises:

[0065] An action determination module, used for determining a current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot;

[0066] An action sending module, used for sending the current action instruction and the at least one backup action instruction to the robot, so that the robot executes the current action instruction, and updates the backup action instruction set according to the received at least one backup action instruction;

[0067] Among them, the backup action instruction has a timestamp used to characterize the target execution time of the backup action instruction, and the robot is used to execute the current action instruction when the current action instruction is received within the waiting time, and execute the backup action instruction in the backup action instruction set when the current action instruction is not received within the waiting time.

[0068] Optionally, when the action determination module is used to determine the current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot, it is specifically used to:

[0069] According to the position of the at least one robot, a current action instruction and at least one backup action instruction corresponding to each robot are determined by a conflict search-based device and an A-star path planning algorithm carrying a time dimension.

[0070] Optionally, when the action determination module is used to determine the current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot, it is specifically used to:

[0071] According to the position of the robot and the current posture reported by the robot, a current action instruction and at least one backup action instruction corresponding to the robot are determined.

[0072] Optionally, each robot is connected to a main control device based on a local area network, and the device further comprises:

[0073] A delay determination module is used to receive a feedback frame sent by the robot, determine the communication delay between the main control device and the robot according to the feedback frame, and send the communication delay to the robot so that the robot determines the execution time of the current action instruction according to the communication delay, wherein the feedback frame is sent by the robot when receiving the trigger frame sent by the main control device, and the feedback frame includes the sending time of the trigger frame.

[0074] Optionally, the initial image is acquired by collecting an infrared camera module, and the infrared camera module includes a filter, and the filter is used to filter out light of a frequency at which infrared rays emitted by the light source exist.

[0075] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0076] A fourth aspect of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0077] A fifth aspect of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in the first aspect.

[0078] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0079] In the disclosed embodiments, first, the robots deployed in the activity area are encoded by the luminescence of different positions in the preset position dot matrix, which allows the size of the position dot matrix and the specific setting method of the multiple positions contained therein to have a large degree of freedom, and is no longer restricted to the form specified in the QR code or other forms of standard coding, so that adaptive corrections can be made according to the particularity of the robot's appearance. For example, the above-mentioned multiple positions are dispersed in the area on the robot body that does not affect the visual effect of the viewing, which not only improves the aesthetics of the robot's appearance, but also reduces the cost of making the robot's coding. Second, the main control system can determine the position dot matrix contained in the image based on the vertex position and the two bottom corner positions presented in the captured image, and then determine the robot's coding, position and direction based on other luminous positions, which greatly reduces the demand for the camera's image acquisition quality, further reduces costs and improves usability.

[0080] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0082] Figure 1 It is an application scenario diagram of a positioning method shown in some exemplary embodiments.

[0083] Figure 2 The figure is a flow chart of a positioning method shown in some exemplary embodiments.

[0084] Figure 3 is a schematic diagram of a position matrix shown in some exemplary embodiments.

[0085] Figure 4 is a schematic diagram of a light-emitting pixel shown in some exemplary embodiments.

[0086] Figure 5 is a flow chart of another positioning method shown in some exemplary embodiments.

[0087] Figure 6 Detailed description of the drawings is given in conjunction with some exemplary embodiments.

[0088] Figure 7 The present invention is a multi-party instruction interaction diagram of a positioning method shown in some exemplary embodiments.

[0089] Figure 8 is a schematic diagram of yet another positioning method shown in some exemplary embodiments.

[0090] Fig. 9 is a block diagram of a positioning device shown in some exemplary embodiments.

[0091] Fig.10 It is a hardware structure diagram of an electronic device shown in some exemplary embodiments. DETAILED DESCRIPTION

[0092] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0093] Robot dance programs are quite common at science and technology exhibitions and the opening ceremonies of some events. For example, dozens of robots are arranged in a certain formation and perform dance moves to music. During the dance, the robots must not only maintain their own stability, but also coordinate with other robots to avoid collisions and correct their own wrong movements in time. This requires a central control system (master control device) to monitor and adjust the position and movement of each robot in real time. Through wireless communication technology, the central control system can send instructions in real time to ensure that all robots can perform according to the predetermined trajectory and rhythm.

[0094] Generally speaking, these robots are controlled through relatively precise programming and instructions. Each robot can be equipped with multiple sensors and high-precision motion control systems, enabling them to maintain balance and accurately perform various movements during the dance. The robot's current posture, error information and other data can be sent to the main control system. The main control system can combine the information reported by one or more robots, as well as the current positions of these robots, preset dance tasks and preset accident handling mechanisms (such as algorithms for handling robot falls) and other logic to determine the action instructions that the robot should execute within a certain period of time at present or in the future, and send these action instructions to the robot to control the robot's movement.

[0095] Therefore, the accurate positioning of the robot by the main control device will directly affect the effect of this performance. In some related technologies, a positioning module will be deployed in the body of each robot, and the positioning results of the positioning module will be reported to the main control system through wireless communication technology, but this method is extremely costly and extremely dependent on the communication quality between the main control system and the robot and the positioning accuracy of the positioning module. Because in the above scenario, there are usually many spectators gathered in the exhibition hall or the opening ceremony seats (close to the robot formation), the electromagnetic signals generated by the mobile phones and other electronic devices carried by the audience (such as cellular networks, Bluetooth networks, and local area network related signals) will greatly affect the signal quality in the robot's activity area, and even cause signal paralysis. Therefore, this solution is almost not feasible.

[0096] For another solution, see Figure 1 , Figure 1 The component numbered 3 is a high-resolution camera, and the component numbered 1 is a robot (exemplarily, Figure 1 There are four robots deployed in total), and the component numbered 2 is a QR code printed or pasted on the robot body. This method relies on vision rather than network communication for positioning, but as described in the background technology, this positioning method is not only extremely costly (because in this solution, the high-resolution camera must have the ability to recognize the QR code at a long distance, and usually needs to be a color camera), and requires a QR code pasting area to be reserved on the robot body, but is also easily affected by common negative factors such as the limited camera field of view, QR code morphological distortion and motion blur caused by robot movement, resulting in poor positioning results.

[0097] In view of this, the present disclosure provides a positioning method, device, electronic device and storage medium. Next, the embodiments of the present disclosure are described in detail.

[0098] In a first aspect, the present disclosure provides a positioning method that can be applied to a master control device (also called a master control device or a central control system). Figure 2 , which may include steps S201 to S203.

[0099] Step S201, obtaining an initial image of the robot activity area, wherein at least one robot is deployed in the robot activity area, and the robot is provided with a position matrix, wherein the position matrix includes a vertex position, two bottom corner positions and at least one coding position, and the position matrix is ​​used to represent different robot numbers by different luminous states at each coding position, and the spacings between the vertex position and the two bottom corner positions are respectively a first spacing and a second spacing, and the first spacing and the second spacing are both greater than the spacings between any other two positions in the position matrix, and during the positioning process, the vertex position and the two bottom corner positions are in a normally bright state.

[0100] In one example of the present disclosure, the robot is provided with at least three light sources to illuminate at least three positions in the position matrix including the vertex position and the two bottom corner positions.

[0101] See also Figure 3 , which exemplarily shows a position matrix with 8 positions, wherein each dot represents a position in the position matrix, and in addition to the vertex position and two bottom corner positions that have been marked, P1 to P5 are coding positions, and each coding position can have multiple lighting states, such as lighting and non-lighting, so that each coding position can represent a bit of the robot code in a certain base. Preferably, in order to facilitate the representation of the direction of the position matrix and provide a larger error tolerance, the first spacing and the second spacing can be equal (of course, this equality can have a certain error). In the following text, the implementation method will be described by taking the case where the first spacing and the second spacing are equal and both are equal to the preset length (also called waist length) as an example, and more details about this will be explained later.

[0102] For example, each coding position can have two luminous states, luminous and non-luminous, so each coding position can correspond to a bit in the binary code, that is, the two luminous states of luminous and non-luminous represent the value of a coding bit as 1 and 0 respectively. At this time, the position matrix can represent a total of 2 5 If each coding position has a distinction in luminous brightness, luminous band (frequency) or other dimensions, then a coding position can represent a bit of the robot code in a higher base, but this method can be regarded as an extension and reuse of the above binary representation method, so the following text introduces the luminous state as including luminous and non-luminous.

[0103] The correspondence between each coding position and the bit in the robot coding can be arbitrarily preset, for example Figure 3 In the example shown, P1 to P5 can be made to correspond to each bit from the highest bit to the lowest bit in the 5-bit binary robot code. In this case, when the three positions P1, P3, and P4 are illuminated (for example, light sources are deployed here), and the two positions P2 and P5 are not illuminated (for example, infrared light sources are not deployed here, or light sources are deployed here but not turned on, or are covered with obstructions such as tape stickers), the position dot matrix can represent the robot code "10110", which is the decimal code "22".

[0104] It should be understood that the luminescence in the present disclosure may refer to the emission of infrared light or other light that is invisible to the naked eye. For example, when the initial image is obtained by collecting the infrared camera module, the "luminescence" only needs to be visible to the infrared camera module. In order to more accurately filter out the infrared light deployed on the robot, the infrared camera module may include a filter, which is used to filter out the light of the frequency of the infrared light emitted by the light source, that is, the filter is used to filter out the light of the frequency of the collected infrared light. For example, the filter can be set at a closer distance in the camera module lens or outside the lens (the concept of "closer" here refers to the distance between the infrared camera module and the robot), and the filtered band range can be narrower, for example, only used to filter out 850 nanometers or 940 nanometers (of course, this value will have a certain error range, and the concepts of equality, consistency, equal, and both provided in the present disclosure can also have a certain error range). The band corresponds to the infrared light emitted by the infrared light source deployed on the robot, so as to obtain an initial image that is as pure as possible.

[0105] The above method uses visual recognition based on infrared rays that are invisible to the naked eye to complete the positioning of the robot. Therefore, it will not be interfered by network delays and electromagnetic signals emitted by surrounding audiences, nor will it be easily affected by natural light and other light-emitting components in the scene. It can ensure a high degree of robustness without affecting the normal appearance of the robot.

[0106] Optionally, the main control device can collect images of the robot's activity area based on the infrared camera module. In order to enable the main control device to directly determine the position of the position matrix deployed on each robot in the image based on the image collected by the infrared camera module (in order to avoid confusion in the name, this "position" will be referred to as the "frame" of the position matrix in the following text, because this can be understood as the process of determining which frame area in the initial image is the imaging of the position matrix), the above-mentioned position matrix needs to have a special position that serves as an identifier (of course, this "special" is special in logical understanding, not necessarily special in technical implementation). In order to accurately determine the frame of each position matrix in the initial image and determine the direction (orientation) of the position matrix, there can be at least three special positions in the position matrix, and these three special positions can form an isosceles triangle (its vertices are vertex positions, and the vertices of its two bottom angles are bottom angle positions), and the waist of the isosceles triangle is longer than all the spacings formed by multiple positions of the position matrix, except for the two waists of the isosceles triangle.

[0107] It should be understood that the meaning of "the spacing formed by multiple positions" in the present disclosure is the spacing formed by each combination of all position combinations generated by taking any two different positions among these positions, for example Figure 3The position matrix shown has 8 positions, which can be determined according to the combination number formula. Different spacings (the different spacings here refer to the spacings formed by different position combinations, and the values ​​of these spacings can be the same); similar expressions such as "the spacing formed by the luminous positions included in the position matrix" in the following text are also applicable to the above principle and will not be repeated in the following text.

[0108] Thus, the waist length of the isosceles triangle becomes the maximum value among the spacings formed by multiple positions (when the shape formed by the vertex position and the two bottom angle positions is not an isosceles triangle, it does not affect the principle of this step), then the main control device can relatively easily divide the suspected luminous positions in the initial image (that is, the collected luminous points, which can also be understood as the imaging of the light source in the initial image), and can accurately identify illegal position arrays. It should be understood that the position lattice may include not only the above three special positions, but also multiple vertex positions or more bottom angle positions. Preferably, the figure enclosed by the bottom angle position and the vertex angle position can be a figure without rotational symmetry, that is, it only coincides with the original figure when it is rotated 360° and its multiples around the center point, thereby making multiple special positions have the function of representing direction.

[0109] Step S202, determining each light-emitting position contained in the initial image, and determining the position matrix to which each light-emitting position contained in the initial image belongs according to the first spacing and the second spacing.

[0110] Among them, the determination method of the light-emitting position is comprehensively determined by reference dimensions such as the camera resolution of the infrared camera module (hereinafter referred to as the camera or camera), the distance between the infrared camera module and the plane of the robot's activity area (the height of the robot can also be subtracted from this), and the imaging effect of the light source in the field of view of the infrared camera module. It can also be determined by simply screening the highlight pixels in the initial image. The imaging effect may include imaging size, imaging shape, etc., and a relatively simple way to determine the imaging effect is to conduct experiments based on the various states that the light source on the robot may present during the dance (execution of action instructions) to determine a rough imaging size range (the same applies to the imaging shape), and determine the parameters that are dependent in S202 based on the imaging size range. Of course, in addition to this, after obtaining the robot posture, the infrared camera module posture, and the distance between the infrared camera module and the plane of the robot's activity area, a reasonable imaging effect can be specifically determined by rotation and translation transformation. In addition, the above-mentioned imaging effect can also be determined based on the average number of pixels, average size, or most common shape of the determined pending light-emitting position, so as to adapt to the current shooting state of the infrared camera module to the greatest extent.

[0111] After the light-emitting positions in the initial image are determined, it is possible to determine which position arrays these light-emitting positions actually correspond to based on the conditions satisfied by the position arrays described above.

[0112] In the above process, optionally, determining each light emitting position contained in the initial image may include:

[0113] Clustering pixels in the initial image whose grayscale values ​​are greater than a grayscale threshold according to the imaging size of the light source provided on the robot in the initial image, so as to determine a pending light emitting position contained in the initial image;

[0114] The pending light-emitting positions are screened according to the imaging size and / or imaging shape of the light source arranged on the robot in the initial image to determine each light-emitting position contained in the initial image.

[0115] For example, see Figure 4 , which shows a part of an initial image. Each small square represents a pixel. The squares without numbers represent pixels whose grayscale values ​​are less than the grayscale threshold in the initial image, and the squares filled with numbers (i.e., numbers 1 to 13) represent pixels whose grayscale values ​​are greater than the grayscale threshold (hereinafter referred to as luminous pixels, because these pixels may correspond to external light sources). If the imaging shape should be circular and the imaging size should be 3 pixels in diameter (due to the limitation of the number of pixels, it is Figure 4 A cross will appear in the image, that is, the imaging shape is a cross with a length and width of 3). Then, after binarizing the initial image (that is, determining the pixels in the initial image whose grayscale values ​​are greater than the grayscale threshold), we can first determine the possible luminous positions contained therein (that is, the images formed by the light sources at these positions in the initial image).

[0116] For example, the luminous pixels may be read one by one, and then it is determined whether the pixels around them are included in the imaging size. Continuing with the above example, this step may be to determine whether the distance between a certain pixel and another pixel is less than or equal to the imaging diameter. Figure 4 In the example, after reading pixel 1, pixels 2, 3, 6, and 7 will be assigned to the pending luminous position corresponding to pixel 1 (this is because it is not clear at which position pixel 1 is located in the imaging shape at this time, so all possible situations can be taken into account, such as the situation where pixel 1 is located at the leftmost, middlemost, or rightmost of the imaging shape), and when pixel 2 is read, pixel 4 will be assigned to the pending luminous position corresponding to pixel 1, and when pixel 3 is read, pixel 5 will be assigned to the pending luminous position corresponding to pixel 1, and so on. Finally, Figure 4 Two pending light-emitting positions are determined in the initial image shown, namely, pending light-emitting position 1 composed of pixels 1 to 8 and pending light-emitting position 2 composed of pixels 9 to 13 .

[0117] Next, the pending light positions can be further screened according to parameters in dimensions such as imaging size and imaging shape. For example, the pending light position 1 in the above example obviously does not meet the imaging size and imaging shape, so it can be discarded and only the pending light position 2 is retained. Based on this, the light position in the initial image can be determined by a very simple algorithm, which is very fast and has a good effect in filtering interference factors such as object reflection and imaging phantom.

[0118] Then, we can determine the positional matrix contained in the initial image based on each luminous position in the initial image. Figure 5 Since the waist length is the maximum possible spacing between different positions in the position matrix, the positions can be read one by one, and the luminous positions within the waist length radius can be drawn into the position matrix described by the position. For example, if the luminous position is Figure 5 The arrangement shown (where the black dot is a light-emitting position) can be divided into two position dot arrays, position dot array 1 and position dot array 2, according to the waist length. Of course, the direction of the position dot array can also be determined at the same time, that is, the method can also include:

[0119] For each position dot matrix contained in the initial image, the vertex luminous positions and the bottom corner luminous positions of the position dot matrix are determined according to the two largest intervals among the intervals formed by the luminous positions included in the position dot matrix, and the direction of the position dot matrix and / or the direction of the robot corresponding to the robot number represented by the position dot matrix are determined according to the vertex luminous positions and the bottom corner luminous positions of the position dot matrix.

[0120] The vertex luminous position is the image formed by the vertex position of the luminous object, and the bottom angle luminous position is the image formed by the bottom angle position of the luminous object. It should be understood that in this process, a certain degree of tolerance can be given according to factors such as the camera error and the posture change caused by the robot movement process. For example, the luminous positions within the radius slightly larger than the waist length are divided into the same position matrix; in addition, in order to minimize the impact on the accuracy of the scheme, the position matrixes set by different robots can be made to have a spacing of 1.5 times the waist length or even more than 2 times the waist length, and this condition continues to be maintained during the robot control process.

[0121] Step S203, based on the luminescence of each position in each position matrix contained in the initial image, determine the robot number represented by the position matrix, and determine the position of the robot corresponding to the robot number according to the position of the position matrix in the initial image.

[0122] As described above, after determining the frame of the position matrix, the frame can be divided into several subframes, each of which corresponds to a position of the position matrix. Next, the value of the encoding bit corresponding to the position can be determined based on whether there is a light-emitting position in each of the subframes. Figure 3 Some of the examples given are Figure 5 The position matrix 1 and the position matrix 2 represent the two codes "11111" and "10110" respectively.

[0123] In the above steps, determining the position of the robot corresponding to the robot number according to the position of the position matrix in the initial image may include:

[0124] According to the internal parameters of the infrared camera module and the rotation matrix and translation vector of the infrared camera module relative to the world coordinate system, the position of the position matrix in the initial image is transformed to obtain the position of the robot corresponding to the robot number in the world coordinate system, wherein the initial image is acquired by the infrared camera module.

[0125] Specifically, in order to ensure the uniformity of robot dance control, the action instructions sent by the master control device to the robot often carry the position in the world coordinate system, such as controlling the speed at which the robot moves to a certain position in the world coordinate system; therefore, after determining the position in the initial image (the position at this time is usually a two-dimensional coordinate), the coordinate can be converted based on the camera intrinsic parameters, the distance between the camera and the robot, and other parameters to obtain the three-dimensional coordinate in the world coordinate system. For example, the mutual conversion between two-dimensional coordinates and three-dimensional coordinates can be completed based on the following formula:

[0126] ;

[0127] In the above formula (hereinafter referred to as the rotation and translation transformation formula), is the coordinate of the original image taken by the camera in the two-dimensional coordinate system, is the coordinate in the three-dimensional coordinate system (world coordinate system); Z is the scale factor, which indicates the depth value of the three-dimensional point from the center of the camera; and the four matrices on the right side of the equal sign from left to right are the affine transformation intrinsic parameter matrix, the projection perspective matrix, the camera rigid body transformation matrix, and the robot's rotation matrix in the world coordinate system (which can be used for perspective posture compensation). The first two (affine transformation intrinsic parameter matrix and projection perspective matrix) are related to the camera intrinsic parameters. Their multiplication can obtain the camera intrinsic parameter matrix (of course, these matrices can have other writing methods, and different writing methods of the camera intrinsic parameter matrix do not affect their effects); the parameters in the affine transformation intrinsic parameter matrix and the projection perspective matrix can be obtained through camera calibration: f is the focal length, d is the focal length, and d is the camera's rotation matrix. x ,d yRespectively represent the physical length of a pixel on the camera photosensitive plate in the X and Y directions (i.e., the directions of the two coordinate axes) of the two-dimensional coordinate system, that is, how many millimeters a pixel is on the photosensitive plate, u0, v0 respectively represent the coordinates of the center of the camera photosensitive plate in the pixel coordinate system, θ represents the angle between the horizontal and vertical edges of the photosensitive plate (90° means no error); the constant 1 is used to maintain the homogeneous coordinate form of the matrix; R camera is the rotation matrix of the camera, which can usually be a 3×3 matrix used to describe the rotation of the camera posture relative to the world coordinate system; T camera is the camera's translation vector, which can usually be a 3×1 vector used to describe the camera's translation, which can define the camera's position in the world coordinate system; R robotw is the rotation matrix of the robot, which can usually be a 3×3 matrix used to describe the posture of the robot (such as the rotation relative to the world coordinate system).

[0128] It should be understood that the above rotation and translation transformation formulas not only exist in the above forms. When the rotation transformation process (of course, this rotation transformation process is not limited to two-dimensional coordinates and three-dimensional coordinates, but can also be used between two-dimensional coordinates and two-dimensional coordinates, or between three-dimensional coordinates and three-dimensional coordinates) does not involve some of the parameters, some of the parameters can be adaptively removed (for example, these parameters are set to 1), or some matrices can be removed. For example, when the camera intrinsic parameters are not involved, the affine transformation intrinsic parameter matrix and the projection perspective matrix can be removed. When the camera posture is not involved and only a part of the initial image is corrected by relying on the robot posture, the camera rigid body transformation matrix can be removed. The same applies to other scenarios.

[0129] In the disclosed embodiments, first, the robots deployed in the activity area are encoded by the luminescence of different positions in the preset position dot matrix, which allows the size of the position dot matrix and the specific setting method of the multiple positions contained therein to have a large degree of freedom, and is no longer restricted to the form specified in the QR code or other forms of standard coding, so that adaptive corrections can be made according to the particularity of the robot's appearance. For example, the above-mentioned multiple positions are dispersed in the area on the robot body that does not affect the visual effect of the viewing, which not only improves the aesthetics of the robot's appearance, but also reduces the cost of making the robot's coding. Second, the main control system can determine the position dot matrix contained in the image based on the vertex position and the two bottom corner positions presented in the captured image, and then determine the robot's coding, position and direction based on other luminous positions, which greatly reduces the demand for the camera's image acquisition quality, further reduces costs and improves usability.

[0130] In the method provided by the present disclosure, after determining the position matrix included in the original image, a series of optional processing steps may be performed, such as correcting and screening the position matrix. This will be described in detail below.

[0131] Optionally, before determining the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix included in the initial image, the method may further include:

[0132] For each position dot matrix contained in the initial image, a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold is determined in at least one previous frame of the initial image, and based on the luminescence condition of each position in the historical position dot matrix, the luminescence condition of each position in the position dot matrix is ​​corrected.

[0133] Specifically, the method provided by the present disclosure can be executed periodically, and the interval between each execution can be very short, for example, the positioning method can be executed at a frequency of tens or hundreds of hertz; during this period, the robot can also report the posture information collected by itself to the main control device (this process can also be periodic). The above steps can be executed for non-first frames. In the image collected by at least one frame in the past, the main control device has determined the position dot matrix contained therein, and the robot number corresponding to these dot matrixes (the robot number corresponds to the robot one by one). In the case of periodic execution of the method, it can be considered that the displacement of the robot during the most recent execution of the method is predictable. In other words, it can be considered that the position of the robot in the most recent frames has not changed much.

[0134] Therefore, in the initial image collected in the past, a historical position lattice whose spacing with the position lattice is less than the displacement threshold can be found (this search process can be from near to far and have a cutoff threshold, for example, starting from the previous first frame, if not found, then advancing to the previous second frame, until advancing to the previous Nth frame or finding the corresponding historical position lattice, where N is a positive integer), and the position can be corrected based on the historical position lattice. For example, if the historical position lattice contains 5 luminous positions, and the position lattice currently being processed contains 4 luminous positions, then 1 luminous position can be added to the current position lattice, and the specific size and appearance position of the added luminous position can be determined based on a preset algorithm and according to the morphology of the luminous position in the historical position lattice, and the present disclosure does not limit this. Based on this, the above positioning method adapts to the phenomenon of luminous light source occlusion that may be caused by certain special actions of the robot, and improves the robustness of the method.

[0135] Optionally, before determining the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix included in the initial image, the method may further include:

[0136] For each position dot matrix included in the initial image, determine a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold in at least one previous frame of the initial image, and based on the robot posture reported by the robot corresponding to the robot number represented by the historical position dot matrix, correct the position dot matrix through rotation and translation transformation; and / or,

[0137] Based on the posture of the infrared camera module, each position dot matrix included in the initial image is corrected through rotation and translation transformation, wherein the initial image is acquired through the infrared camera module.

[0138] In other words, the position matrix in the initial image can be corrected by combining the camera posture and the robot posture. For example, the optical axis of the camera may not be completely perpendicular to the ground, so the image captured by the camera may have a certain deformation, and the robot may also produce a variety of tilted postures during the dance. Based on this, the above method can adapt to the image deformation problem caused by the different movements of the robot during the dance and the actual configuration angle of the camera, which greatly increases the robustness of the method and the accuracy of positioning.

[0139] Optionally, the method may further include:

[0140] For each position matrix included in the initial image, determine the two largest spacings among the spacings formed by the light-emitting positions included in the position matrix, and discard the position matrix when the difference between the two largest spacings is greater than an error threshold, wherein the first spacing and the second spacing are equal; and / or,

[0141] For each position matrix contained in the initial image, a historical position matrix whose spacing with the position matrix is ​​less than a displacement threshold is determined in at least one previous frame of the initial image; when the robot number represented by the position matrix determined based on the position matrix is ​​inconsistent with the robot number represented by the historical position matrix, the position matrix is ​​discarded.

[0142] The above steps can also be understood as discarding this position matrix in the initial image of the current frame and the luminous position in the position matrix (or, the current frame can also be discarded) when a possible abnormality is found in the current position matrix. This abnormality can be that this position matrix does not meet the conditions of the aforementioned position matrix (that is, there is no isosceles triangle with a waist length of the maximum spacing, and this judgment can also allow a certain error, that is, the difference between the two largest spacings is less than or equal to the error threshold), or it can be that the recognition result of the current frame does not match the historical position matrix at this position (these concepts have been introduced above and will not be repeated here). Based on this, the robustness of the method is further improved to avoid possible errors from having too much impact on the positioning results.

[0143] The following combination Figure 6 The selection branch diagram shown in FIG. 1 further explains the processing and recognition process of the initial image described above. The steps involved are all introduced above. Figure 6 The explanations made can be combined with the above embodiments and complement each other for the parts not described in detail. Figure 6 In the related descriptions thereof, the luminous point and the luminous position mentioned above can be regarded as the same concept, and the concepts of label and label point and the position dot matrix mentioned above can be regarded as the same concept.

[0144] After the main control device reads the (initial) image, it can first perform a binary filter on the initial image to filter out the luminous pixels, and then traverse the luminous pixels, for example, find a pixel A, and determine whether the distance between another pixel B and pixel A is less than or equal to the preset imaging size (such as the diameter of the luminous point). If so, the pixel B is classified into the luminous point where the pixel A is located. If not, a new luminous point is created (in addition, the first luminous pixel read in naturally belongs to a new luminous point). The judgment process here can also be understood as determining whether the distance between pixel B and the position of the currently existing luminous pixel (or luminous point) is greater than the imaging size. The position of the luminous point here can be determined based on any preset standard, such as using the geometric center or vertex position of the luminous point as the position of the position lattice. When the traversal is completed, the statistics of all luminous points are completed.

[0145] Next, based on preset imaging size, imaging shape and other parameters, luminous points whose pixel number is much larger than the average pixel value of existing luminous points due to factors such as reflective planes can be filtered out to determine more reasonable luminous points included in the initial image.

[0146] Subsequently, the luminous points can be traversed, and several position dot matrices (labels) can be divided according to the waist length. For example, after reading a luminous point A, another luminous point B is read, and it is determined whether the distance between luminous point B and luminous point A is greater than the waist length. If so, luminous point B is classified into a new position dot matrix (in addition, the first luminous point read naturally belongs to a new position dot matrix). If not, luminous point B is classified into the position dot matrix where luminous point A is located (the overall logic can be consistent with the above-mentioned process of dividing luminous pixels). The judgment process here can also be understood as judging whether the distance between luminous point B and the position of the existing luminous point (or position dot matrix) is greater than the waist length. The position of the position dot matrix here can be determined based on any preset standard, such as using the geometric center or vertex position of the position dot matrix as the position of the position dot matrix. After completing the traversal, the position dot matrix contained in the initial image of the current frame is obtained.

[0147] Next, each position matrix can be traversed, and the historical position matrix corresponding to the current position matrix can be obtained in the initial image collected in the past. When there is a difference in the number of luminous points included in the current position matrix and the historical position matrix, the current position matrix can be corrected according to the historical position matrix, such as adding or deleting luminous points at certain positions to overcome the inter-frame fluctuation problem caused by factors such as robot movement occlusion or robot body reflection (that is, errors in some frames). Of course, when this difference is too large (greater than a certain threshold), you can choose not to make corrections or directly discard the current position matrix.

[0148] Next, according to the posture feedback from the robot and the posture of the camera, the perspective of each position dot matrix can be corrected (for example, it can be uniformly mapped to a top-down plane image, that is, the image presented by the horizontal plane observed from above), so as to eliminate the deviation in perspective (here, it can be corrected based on the rotation and translation transformation formula above). The posture feedback from the robot here can be inferred, or calculated based on the dance task and action execution, or it can be reported by the robot at a previous time (at this time, the robot number can be based on the robot number corresponding to the historical position dot matrix, that is, the robot number represented by the historical position dot matrix corresponding to the current position dot matrix is ​​used as the robot number of the current position dot matrix, and the robot is determined based on this, so as to further determine the posture information reported by the robot).

[0149] Next, the two intervals with the largest distance between each other in the current position lattice can be selected (and the difference between the two intervals is less than the error threshold), and the vertex position and bottom corner position of the current position lattice can be determined based on the endpoints of the two intervals, and then the specific framework and direction of the current position lattice can be determined. If the above steps fail to execute, that is, no interval that meets the conditions is found or the fixed point position and bottom corner position cannot be determined, the current position lattice can be considered abnormal, and the abnormal elimination process is performed (discarding the current position lattice).

[0150] Next, the robot number represented by the position dot matrix can be determined based on the frame and orientation of the position dot matrix that has been determined. At this time, if the robot number determined by the current frame is different from the robot number corresponding to the historical position dot matrix, the robot number determined by the current frame can be used as the standard, or the current frame can be considered to be wrong, thereby discarding the robot number determined by the current frame (that is, discarding the current position dot matrix). Finally, the position of the robot corresponding to each robot number in the world coordinate system can be determined according to the rotation and translation transformation formula described above.

[0151] Next, some optional further processing of the master control device after determining the position of the robot is described.

[0152] First, the robot can be used to execute the current action instruction according to its recorded position (as mentioned above, this position usually needs to be the position in the world coordinate system), then the method may also include: sending the robot's position to the robot so that the robot updates the current position, wherein the robot is used to execute the current action instruction according to the current position.

[0153] The "current position" can be understood as a parameter recorded inside the robot, which represents the current position of the robot in the world coordinate system. The main control device can determine the position of the robot in each frame and send this position to the robot, so that the robot can update its recorded current position and then continue to execute the current action instruction according to the current position. Specifically, the current action instruction may be a target-like instruction, such as "move to a (a is a three-dimensional coordinate)", and specific actions such as taking a step and swinging an arm can be automatically completed by the robot, and this automatic completion process often requires the current position as a parameter.

[0154] Due to the particularity of the performance scene of the robot dance formation (see above for details), the connection between the robot and the main control device may be interrupted, resulting in the loss of one or more communication frames (communication frames can refer to any data packets between the robot and the main control device). At this time, the current position recorded inside the robot can be temporarily kept unchanged and the current action can be stopped to wait for the communication to be restored; or, preferably, the robot can be provided with a local odometer, and the robot is used to determine the current position based on the local odometer.

[0155] The local odometer is a module used to estimate the position movement of the robot. It can calculate the displacement of the robot in the world coordinate system based on the sensor data. At this time, the robot can make a relative calculation of the current position based on the position sent by the master device received in a previous frame and combined with the data of the local odometer. After the robot receives the position sent by the master device, it can directly update the position sent by the master device as the current position; or, sending the position of the robot to the robot so that the robot updates the current position can include: sending the position of the robot to the robot so that the robot performs complementary filtering on the received position and the position determined based on the local odometer, and updates the current position.

[0156] Complementary filtering reduces the system error of a single sensor by modeling and compensating for the errors of different sensor data, thereby improving the accuracy and reliability of the current position. Next, the position closed-loop control capability can be provided based on proportional-differential control (i.e. PD control) (which can be triggered when entering the formation movement or alignment link). When the communication delay is low, it is controlled by global control (i.e. complementary filtering above); when the communication delay fluctuates, the position closed loop is closed by the robot for local and global positioning information (i.e. the robot is calculated based on the last received position and local odometer), reducing the high dependence of positioning accuracy on communication frequency.

[0157] Optionally, the method may further include:

[0158] Determine, according to the position of the robot, a current action instruction and at least one backup action instruction corresponding to the robot;

[0159] Sending the current action instruction and the at least one backup action instruction to the robot so that the robot executes the current action instruction, and updates the backup action instruction set according to the received at least one backup action instruction;

[0160] Among them, the backup action instruction has a timestamp used to characterize the target execution time of the backup action instruction, and the robot is used to execute the current action instruction when the current action instruction is received within the waiting time, and execute the backup action instruction in the backup action instruction set when the current action instruction is not received within the waiting time.

[0161] Specifically, the robot can store a current action instruction and a set of backup action instructions, where the current action instruction is the highest priority instruction, such as a dance action that needs to be performed currently, or a standing action that needs to be performed urgently after falling down. The backup action instruction set can contain at least one backup action instruction, and each backup action instruction has a corresponding execution time (for example, represented by a timestamp). The robot can execute the current action instruction first and wait for the next instruction from the master control device, that is, receive the "current action instruction" when the master control device sends it next time, and update the current action instruction recorded by the robot according to the received "current action instruction".

[0162] However, due to network fluctuations and other reasons, the robot may not receive the next "current action instruction" issued by the main control device after executing the current action instruction. At this time, the robot can wait according to the preset waiting time (the waiting time can generally be set to tens of milliseconds, such as about 50 milliseconds, so as to take into account common communication delays and the audience's perception of action continuity). During the waiting period, the robot can temporarily perform transitional actions such as stillness and swaying left and right. After the waiting period, in order to avoid freezing in the dance process, the robot can determine the appropriate backup action instruction in the backup action instruction set based on the current waiting time (this is because the backup action instruction has a preset execution time), and then execute the determined backup action instruction as the current action instruction. In this way, the adaptability of the robot dance to network delays and the perception given to the audience are taken into account to the greatest extent.

[0163] However, if a current action instruction is received from the main control device during the execution of a backup action instruction in a backup action instruction set, the current action instruction can be comprehensively determined and executed based on the action control method configured by the robot itself, and based on factors such as whether the backup action instruction being executed conflicts with the most recently issued current action instruction and whether they have a continuous execution relationship. The present disclosure does not limit this.

[0164] In summary, please see Figure 7 , the robot and the main control device need to maintain communication at a certain frame rate. After determining the robot's position, current action instructions, and backup action instructions, the main control device can send the robot's position, current action instructions, and backup action instructions to the corresponding robot (the frame carrying these contents is in Figure 7It is represented as an instruction frame, but it should be understood that Figure 7 The various frames in the diagram are just a logical concept, not a limitation on the data combination method or data packet combination method). After receiving the instruction frame sent by the master device and executing the current action instruction according to the instruction frame (of course, there may be other sending opportunities, such as periodic sending or sending immediately after receiving the instruction frame), the robot can send a status frame containing the current actual action (which can be understood as the target task currently being executed, or the execution progress of the current action instruction), error information, posture information, local odometer information and other information to the master device. It should be understood that the various information included in the above frames are not all necessary, but can be added or deleted according to actual conditions.

[0165] For example, determining the current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot may include: determining the current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot and the current posture reported by the robot; at this time, the robot posture can not only be used to correct the deformation caused by the position matrix, but also can be used to assist the main control device in determining the action instruction (error information, current actual action and other information can also be used in this way).

[0166] Since in the application scenario of robot dance formation, the communication connection quality between the robot and the main control device and the public network is very poor or even unavailable, each robot can be connected to the main control device based on a local area network. The method can also include: receiving a feedback frame sent by the robot, and determining the communication delay between the main control device and the robot based on the feedback frame, and sending the communication delay to the robot so that the robot determines the execution time of the current action instruction based on the communication delay, wherein the feedback frame is sent by the robot when receiving the trigger frame sent by the main control device, and the feedback frame includes the sending time of the trigger frame.

[0167] Since the master control device has a relatively convenient and average connection with each robot, and the master control device usually has a more powerful hardware configuration, in an independent system composed of a master control device and multiple robots, the robot can calibrate its internally recorded time based on the time of the master control device. First, the master control device can send a trigger frame with the current timestamp (i.e., the sending timestamp of the trigger frame) and the current communication delay to multiple robots. The combination of the current timestamp and the current communication delay can be used to correct the time recorded internally by the robot. After receiving the trigger frame, the robot can carry the sending timestamp of the trigger frame in the feedback frame (at this time, the current timestamp recorded internally by the robot can also be carried), so that after the master control device receives the feedback frame, based on the receiving time of the feedback frame and the sending time of the trigger frame contained in the feedback frame, the round-trip delay of the data packet between the robot and the master control device can be determined, thereby calculating the communication delay between the robot and the master control device; then, this communication delay can be recorded in the next trigger frame, thereby completing the closed loop of data.

[0168] As for the method for the robot to determine the execution time of the current action instruction according to the communication delay, illustratively, the current action instruction may have a preset transition time, such as a 20 millisecond pause before each action. If the communication delay is 15 milliseconds, the action may be executed directly after 5 milliseconds instead of waiting for 20 milliseconds. In another exemplary embodiment, the above method may be executed when the robot performs a transition action and used to determine the end time of the transition action (execute the current action instruction after the end).

[0169] Furthermore, regarding the clock synchronization problem between the master device and each robot, the device network clock synchronization can be activated in the activity preparation stage (when the network condition is usually good), and the clock synchronization accuracy is guaranteed. Then it can be turned off after the performance begins, and the UDP (User Datagram Protocol) broadcast frame with timestamp is used to simulate the NTP (Network Time Protocol) method periodically (i.e. the interaction process of trigger frame-feedback frame mentioned above), that is, it is assumed that the upstream and downstream network delays in the LAN are equal, and the upper limit threshold of the delay can be set, and calibration is abandoned when the delay fluctuates. The LAN networking WIFI can be configured with 5GHz channels first, using high channel bandwidth settings, and 2.4GHz is only used when the WIFI channel is good. Through the above strategy, even if there are many wireless devices in the audience and the 2.4GHz or even 5GHz channels are severely congested, in combination with the robot body clock, the clock synchronization error of each device in the LAN can still be comprehensively guaranteed to be controlled within 20ms. The impact of this error on the uniformity of dance movements is controllable and difficult for the human eye to detect.

[0170] Optionally, determining the current action instruction corresponding to the robot and at least one backup action instruction according to the position of the robot may include: determining the current action instruction corresponding to each robot and at least one backup action instruction according to the position of the at least one robot by a conflict search-based method and an A-star path planning algorithm carrying a time dimension.

[0171] By using a conflict-based search (CBS) method combined with the A* (A-Star) path planning algorithm in the time dimension, it is possible to realize the formation of multi-robot patterns and the rapid and flexible changes of formations. There is no need to preset and edit fixed path trajectories, and rapid deployment can be achieved to solve the single mode of on-site performances. In addition, by considering potential conflicts and time constraints between robots, the above steps can also ensure the effectiveness and safety of path planning, thereby achieving coordination and efficient operation of multi-robot systems. The CBS method supports dynamic modification of formation patterns during the presentation process. The A* path planning algorithm is used to generate fixed sequences in the dance choreography stage, preset expected patterns, and store discrete points of each robot's position during the transformation process, which can effectively avoid the timeout problem of the CBS solution process and ensure the success rate. For example, see Figure 8 In the aforementioned application scenario, the robot may be involved in a process of changing from formation 1 to formation 2, and the successful completion of this process depends on the overall control of the main control device. The above optional implementation method further improves the effect and safety of the overall control. When the number of robots is large (for example, dozens of robots), this effect is more obvious.

[0172] Corresponding to the embodiments of the aforementioned method, the present disclosure also provides embodiments of a device and a terminal to which the device is applied.

[0173] The second aspect of the present disclosure provides a positioning device, see Fig. 9 , the device comprises:

[0174] The image acquisition module 901 is used to obtain an initial image of the robot activity area, wherein at least one robot is deployed in the robot activity area, and the robot is provided with a position matrix, wherein the position matrix includes a vertex position, two bottom corner positions and at least one coding position, and the position matrix is ​​used to represent different robot numbers by different luminous states at each coding position, and the spacing between the vertex position and the two bottom corner positions is respectively a first spacing and a second spacing, and the first spacing and the second spacing are both greater than the spacing between any other two positions in the position matrix, and during the positioning process, the vertex position and the two bottom corner positions are in a normally bright state.

[0175] Optionally, the robot is provided with at least three light sources to illuminate at least three positions in the position matrix including the vertex position and the two bottom corner positions;

[0176] A dot matrix determining module 902 is used to determine each light-emitting position contained in the initial image, and determine a position dot matrix to which each light-emitting position contained in the initial image belongs according to the first spacing and the second spacing;

[0177] The position determination module 903 is used to determine the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix contained in the initial image, and determine the position of the robot corresponding to the robot number according to the position of the position dot matrix in the initial image.

[0178] Optionally, when the dot matrix determination module is used to determine each light-emitting position contained in the initial image, it is specifically used to:

[0179] Clustering pixels in the initial image whose grayscale values ​​are greater than a grayscale threshold according to the imaging size of the light source provided on the robot in the initial image, so as to determine a pending light emitting position contained in the initial image;

[0180] The pending light-emitting positions are screened according to the imaging size and / or imaging shape of the light source arranged on the robot in the initial image to determine each light-emitting position contained in the initial image.

[0181] Optionally, the device further comprises:

[0182] A first correction module is used to determine, for each position dot matrix contained in the initial image, a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold in at least one previous frame of the initial image before determining the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix contained in the initial image, and correct the luminescence condition of each position in the position dot matrix based on the luminescence condition of each position in the historical position dot matrix.

[0183] Optionally, the device further includes a second correction module, which is used to perform the following steps before determining the robot number represented by the position dot matrix based on the luminous condition of each position in each position dot matrix contained in the initial image:

[0184] For each position dot matrix included in the initial image, determine a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold in at least one previous frame of the initial image, and based on the robot posture reported by the robot corresponding to the robot number represented by the historical position dot matrix, correct the position dot matrix through rotation and translation transformation; and / or,

[0185] Based on the posture of the infrared camera module, each position dot matrix included in the initial image is corrected through rotation and translation transformation, wherein the initial image is acquired through the infrared camera module.

[0186] Optionally, the device further comprises a lattice filtering module, configured to perform the following steps:

[0187] For each position matrix included in the initial image, determine the two largest spacings among the spacings formed by the light-emitting positions included in the position matrix, and discard the position matrix when the difference between the two largest spacings is greater than an error threshold, wherein the first spacing and the second spacing are equal; and / or,

[0188] For each position matrix contained in the initial image, a historical position matrix whose spacing with the position matrix is ​​less than a displacement threshold is determined in at least one previous frame of the initial image; when the robot number represented by the position matrix determined based on the position matrix is ​​inconsistent with the robot number represented by the historical position matrix, the position matrix is ​​discarded.

[0189] Optionally, the device further comprises:

[0190] A direction determination module is used to determine, for each position dot matrix contained in the initial image, the vertex luminous positions and the bottom corner luminous positions of the position dot matrix according to the two largest intervals among the intervals formed by the luminous positions included in the position dot matrix, and to determine, based on the vertex luminous positions and the bottom corner luminous positions of the position dot matrix, the direction of the position dot matrix and / or the direction of the robot corresponding to the robot number represented by the position dot matrix.

[0191] Optionally, when the position determination module is used to determine the position of the robot corresponding to the robot number according to the position of the position matrix in the initial image, it is specifically used to:

[0192] According to the internal parameters of the infrared camera module and the rotation matrix and translation vector of the infrared camera module relative to the world coordinate system, the position of the position matrix in the initial image is transformed to obtain the position of the robot corresponding to the robot number in the world coordinate system, wherein the initial image is acquired by the infrared camera module.

[0193] Optionally, the device further comprises:

[0194] The position sending module is used to send the position of the robot to the robot so that the robot updates the current position, wherein the robot is used to execute the current action instruction according to the current position.

[0195] Optionally, the robot is provided with a local odometer, and the robot is used to determine the current position based on the local odometer;

[0196] The position sending module is used to send the position of the robot to the robot so that the robot updates the current position, and is specifically used to:

[0197] The position of the robot is sent to the robot, so that the robot performs complementary filtering on the received position and the position determined based on the local odometer, and updates the current position.

[0198] Optionally, the device further comprises:

[0199] An action determination module, used for determining a current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot;

[0200] An action sending module, used for sending the current action instruction and the at least one backup action instruction to the robot, so that the robot executes the current action instruction, and updates the backup action instruction set according to the received at least one backup action instruction;

[0201] Among them, the backup action instruction has a timestamp used to characterize the target execution time of the backup action instruction, and the robot is used to execute the current action instruction when the current action instruction is received within the waiting time, and execute the backup action instruction in the backup action instruction set when the current action instruction is not received within the waiting time.

[0202] Optionally, when the action determination module is used to determine the current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot, it is specifically used to:

[0203] According to the position of the at least one robot, a current action instruction and at least one backup action instruction corresponding to each robot are determined by a conflict search-based device and an A-star path planning algorithm carrying a time dimension.

[0204] Optionally, when the action determination module is used to determine the current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot, it is specifically used to:

[0205] According to the position of the robot and the current posture reported by the robot, a current action instruction and at least one backup action instruction corresponding to the robot are determined.

[0206] Optionally, each robot is connected to a main control device based on a local area network, and the device further comprises:

[0207] A delay determination module is used to receive a feedback frame sent by the robot, determine the communication delay between the main control device and the robot according to the feedback frame, and send the communication delay to the robot so that the robot determines the execution time of the current action instruction according to the communication delay, wherein the feedback frame is sent by the robot when receiving the trigger frame sent by the main control device, and the feedback frame includes the sending time of the trigger frame.

[0208] Optionally, the initial image is acquired by collecting an infrared camera module, and the infrared camera module includes a filter, and the filter is used to filter out light of a frequency at which infrared rays emitted by the light source exist.

[0209] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0210] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0211] For the device embodiments and computer program product embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. In addition, the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0212] In a fourth aspect, the embodiments of the positioning device provided by the present disclosure can be applied to electronic devices. Fig.10 , which exemplarily shows a hardware schematic diagram of an electronic device. For example, the device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0213] Device 1000 may include one or more of the following components: a processing component 1001 , a memory 1002 , a power component 1003 , a multimedia component 1004 , an audio component 1005 , an input / output (I / O) interface 1006 , a sensor component 1007 , and a communication component 1008 .

[0214] The processing component 1001 generally controls the overall operation of the device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1001 may include one or more processors 1009 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1001 may include one or more modules to facilitate the interaction between the processing component 1001 and other components. For example, the processing component 1001 may include a multimedia module to facilitate the interaction between the multimedia component 1004 and the processing component 1001.

[0215] The memory 1002 is configured to store various types of data to support operations on the device 1000. Examples of such data include instructions for any application or method operating on the device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1002 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0216] The power component 1003 provides power to the various components of the device 1000. The power component 1003 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1000.

[0217] The multimedia component 1004 includes a screen that provides an output interface between the device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1004 includes a front camera and / or a rear camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0218] The audio component 1005 is configured to output and / or input audio signals. For example, the audio component 1005 includes a microphone (MIC), and when the device 1000 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1002 or sent via the communication component 1008. In some embodiments, the audio component 1005 also includes a speaker for outputting audio signals.

[0219] I / O interface 1006 provides an interface between processing component 1001 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0220] The sensor assembly 1007 includes one or more sensors for providing various aspects of status assessment for the device 1000. For example, the sensor assembly 1007 can detect the open / closed state of the device 1000, the relative positioning of components, such as the display and keypad of the device 1000, and the sensor assembly 1007 can also detect the position change of the device 1000 or a component of the device 1000, the presence or absence of user contact with the device 1000, the orientation or acceleration / deceleration of the device 1000, and the temperature change of the device 1000. The sensor assembly 1007 can also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1007 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1007 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0221] The communication component 1008 is configured to facilitate wired or wireless communication between the device 1000 and other devices. The device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 1008 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1008 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0222] In an exemplary embodiment, the device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the positioning method of the above-mentioned electronic device.

[0223] In a fifth aspect, the present disclosure, in an exemplary embodiment, further provides a non-transitory computer-readable storage medium including instructions, such as a memory 1002 including instructions, and the instructions can be executed by a processor 1009 of the device 1000 to complete the above-mentioned electronic device positioning method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0224] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0225] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the inventions claimed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not claimed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0226] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0227] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A positioning method, characterized in that: The method comprises: Acquire an initial image of a robot activity area, wherein at least one robot is deployed in the robot activity area, and the robot is provided with a position matrix, wherein the position matrix includes a vertex position, two bottom corner positions, and at least one coding position, and the position matrix is ​​used to represent different robot numbers through different luminous states at each coding position, and the spacings between the vertex position and the two bottom corner positions are respectively a first spacing and a second spacing, and the first spacing and the second spacing are both greater than the spacings between any other two positions in the position matrix, and during the positioning process, the vertex position and the two bottom corner positions are in a constantly lit state; Determine each light-emitting position contained in the initial image, and determine a position matrix to which each light-emitting position contained in the initial image belongs according to the first spacing and the second spacing; Based on the luminescence of each position in each position matrix contained in the initial image, the robot number represented by the position matrix is ​​determined, and according to the position of the position matrix in the initial image, the position of the robot corresponding to the robot number is determined.

2. The positioning method according to claim 1, characterized in that: The determining each light emitting position contained in the initial image comprises: Clustering pixels in the initial image whose grayscale values ​​are greater than a grayscale threshold according to the imaging size of the light source provided on the robot in the initial image, so as to determine a pending light emitting position contained in the initial image; The pending light-emitting positions are screened according to the imaging size and / or imaging shape of the light source arranged on the robot in the initial image to determine each light-emitting position contained in the initial image.

3. The positioning method according to claim 1, characterized in that: Before determining the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix contained in the initial image, the method further includes: For each position dot matrix contained in the initial image, a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold is determined in at least one previous frame of the initial image, and based on the luminescence condition of each position in the historical position dot matrix, the luminescence condition of each position in the position dot matrix is ​​corrected.

4. The positioning method according to claim 1, characterized in that: Before determining the robot number represented by the position dot matrix based on the luminescence condition of each position in each position dot matrix contained in the initial image, the method further includes: For each position dot matrix included in the initial image, determine a historical position dot matrix whose spacing with the position dot matrix is ​​less than a displacement threshold in at least one previous frame of the initial image, and based on the robot posture reported by the robot corresponding to the robot number represented by the historical position dot matrix, correct the position dot matrix through rotation and translation transformation; and / or, Based on the posture of the infrared camera module, each position dot matrix included in the initial image is corrected through rotation and translation transformation, wherein the initial image is acquired through the infrared camera module.

5. The positioning method according to claim 1, characterized in that: The method further comprises: For each position matrix included in the initial image, determine the two largest spacings among the spacings formed by the light-emitting positions included in the position matrix, and discard the position matrix when the difference between the two largest spacings is greater than an error threshold, wherein the first spacing and the second spacing are equal; and / or, For each position matrix contained in the initial image, a historical position matrix whose spacing with the position matrix is ​​less than a displacement threshold is determined in at least one previous frame of the initial image; when the robot number represented by the position matrix determined based on the position matrix is ​​inconsistent with the robot number represented by the historical position matrix, the position matrix is ​​discarded.

6. The positioning method according to claim 1, characterized in that: The method further comprises: For each position dot matrix contained in the initial image, the vertex luminous positions and the bottom corner luminous positions of the position dot matrix are determined according to the two largest intervals among the intervals formed by the luminous positions included in the position dot matrix, and the direction of the position dot matrix and / or the direction of the robot corresponding to the robot number represented by the position dot matrix are determined according to the vertex luminous positions and the bottom corner luminous positions of the position dot matrix.

7. The positioning method according to claim 1, characterized in that: The step of determining the position of the robot corresponding to the robot number according to the position of the position matrix in the initial image includes: According to the internal parameters of the infrared camera module and the rotation matrix and translation vector of the infrared camera module relative to the world coordinate system, the position of the position matrix in the initial image is transformed to obtain the position of the robot corresponding to the robot number in the world coordinate system, wherein the initial image is acquired by the infrared camera module.

8. The positioning method according to claim 1, characterized in that: The method further comprises: The position of the robot is sent to the robot so that the robot updates the current position, wherein the robot is used to execute the current action instruction according to the current position.

9. The positioning method according to claim 8, characterized in that: The robot is provided with a local odometer, and the robot is used to determine the current position based on the local odometer; The sending the position of the robot to the robot so that the robot updates the current position includes: The position of the robot is sent to the robot, so that the robot performs complementary filtering on the received position and the position determined based on the local odometer, and updates the current position.

10. The positioning method according to claim 1, characterized in that: The method further comprises: Determine, according to the position of the robot, a current action instruction and at least one backup action instruction corresponding to the robot; Sending the current action instruction and the at least one backup action instruction to the robot so that the robot executes the current action instruction, and updates the backup action instruction set according to the received at least one backup action instruction; Among them, the backup action instruction has a timestamp used to characterize the target execution time of the backup action instruction, and the robot is used to execute the current action instruction when the current action instruction is received within the waiting time, and execute the backup action instruction in the backup action instruction set when the current action instruction is not received within the waiting time.

11. The positioning method according to claim 10, characterized in that: The step of determining the current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot comprises: According to the position of the at least one robot, a current action instruction and at least one backup action instruction corresponding to each robot are determined by a conflict search-based method and an A-star path planning algorithm carrying a time dimension.

12. The positioning method according to claim 10, characterized in that: The step of determining the current action instruction and at least one backup action instruction corresponding to the robot according to the position of the robot comprises: According to the position of the robot and the current posture reported by the robot, a current action instruction and at least one backup action instruction corresponding to the robot are determined.

13. The positioning method according to claim 1, characterized in that: Each robot is connected to a main control device based on a local area network, and the method further includes: Receive a feedback frame sent by the robot, determine the communication delay between the main control device and the robot according to the feedback frame, and send the communication delay to the robot so that the robot determines the execution time of the current action instruction according to the communication delay, wherein the feedback frame is sent by the robot when receiving the trigger frame sent by the main control device, and the feedback frame includes the sending time of the trigger frame.

14. The positioning method according to claim 1, characterized in that: The initial image is acquired through an infrared camera module, and the infrared camera module includes a filter, and the filter is used to filter out the light of the frequency where the acquired infrared light is located.

15. A positioning device, characterized in that: The device comprises: An image acquisition module, used to obtain an initial image of a robot activity area, wherein at least one robot is deployed in the robot activity area, and the robot is provided with a position matrix, wherein the position matrix includes a vertex position, two bottom corner positions and at least one coding position, and the position matrix is ​​used to represent different robot numbers by different luminous states at each coding position, and the spacings between the vertex position and the two bottom corner positions are respectively a first spacing and a second spacing, and the first spacing and the second spacing are both greater than the spacings between any other two positions in the position matrix, and during the positioning process, the vertex position and the two bottom corner positions are in a constantly lit state; a dot matrix determination module, used to determine each light-emitting position contained in the initial image, and determine a position dot matrix to which each light-emitting position contained in the initial image belongs according to the first spacing and the second spacing; A position determination module is used to determine the robot number represented by the position dot matrix based on the luminescence of each position in each position dot matrix contained in the initial image, and to determine the position of the robot corresponding to the robot number according to the position of the position dot matrix in the initial image.

16. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 14 is implemented.

17. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 14 is implemented.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images