A self-positioning method for robot visual environment perception

Abstract

The present invention discloses a self-positioning method for robot visual environment perception, which relates to the field of robot technology. The method comprises: selecting a reference object and configuring a reference reference object set by acquiring multiple position coordinates and associated robot identification codes. Activating the reference robot to collect environmental images, and after acquiring environmental image data, positioning the follower robot through the reference positioning result. The follower robot collects internal environment images according to the reference positioning result to generate a single environmental image set. Finally, the image data is parsed in combination with the position coordinates to achieve self-positioning of the follower robot. The technical problem of insufficient positioning accuracy of existing multi-robots in the positioning process is solved. By combining the positioning reference of the reference robot and the precise analysis of the environmental image data of the follower robot, the technical effect of improving the self-positioning accuracy and stability of the multi-robot system is achieved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a self-positioning method for robot visual environment perception. Background Art

[0002] With the widespread application of robotics, especially in the fields of automation, unmanned driving and industrial robots, robot self-positioning technology has become increasingly important. In complex and dynamic environments, existing self-positioning methods face many challenges, such as the impact of environmental factors on positioning accuracy, the accumulation of sensor errors and the difficulty of position synchronization between multiple robots. Traditional positioning methods rely on single sensor data or preset maps, and often have difficulty maintaining high accuracy in complex environments. In addition, when multiple robots work together, the accuracy and stability of positioning are affected as position errors accumulate. To solve these problems, robots need a technology that can achieve precise and adaptive positioning in dynamic environments.

[0003] In the current related technologies, there is a technical problem of insufficient positioning accuracy of multiple robots during the positioning process. Summary of the invention

[0004] The present application provides a self-positioning method for robot visual environment perception, which adopts the method of obtaining multiple position coordinates and associated robot identification codes, selecting a benchmark object and configuring a benchmark reference object set. The benchmark robot is activated to collect environmental images, and after obtaining environmental image data, the follower robot is positioned according to the benchmark positioning result. The follower robot collects internal environment images based on the benchmark positioning result to generate a single environmental image set. Finally, the image data is parsed in combination with the position coordinates to achieve the self-positioning of the follower robot, and the technical effect of improving the self-positioning accuracy and stability of the multi-robot system is achieved by combining the positioning reference of the benchmark robot and the accurate analysis of the environmental image data of the follower robot.

[0005] The present application provides a robot visual environment perception self-positioning method, comprising:

[0006] Acquire target position information, wherein the target position information includes multiple position coordinates and multiple robot identification codes stored in association; select a benchmark object using the target position information as a selection space, and configure a benchmark reference object set, wherein the benchmark reference object set includes multiple reference positions and multiple corresponding benchmark robots; activate multiple benchmark robots to collect environmental images, obtain environmental image data, and perform benchmark positioning of multiple benchmark robots and multiple follower robots based on the environmental image data; based on the benchmark positioning results, call multiple follower robots to collect internal environment images and obtain internal environment image data, wherein the internal environment image data includes a single environmental image set of multiple follower robots; construct an ideal position model in combination with multiple position coordinates, extract internal environment reference information, parse the single environmental image set, obtain follow-up comparison information, and compare the internal environment reference information with the follow-up comparison information to obtain real-time follow-up deviation to perform self-positioning of multiple follower robots.

[0007] The present application proposes a method for self-positioning of robot visual environment perception. First, by acquiring multiple position coordinates and associated robot identification codes, a reference object is selected and a reference reference object set is configured. The reference robot is activated to collect environmental images. After obtaining environmental image data, the follower robot is positioned according to the reference positioning result. The follower robot collects internal environment images based on the reference positioning result to generate a single environmental image set. Finally, the image data is parsed in combination with the position coordinates to achieve self-positioning of the follower robot. By combining the positioning reference of the reference robot and the precise analysis of the environmental image data of the follower robot, the technical effect of improving the self-positioning accuracy and stability of the multi-robot system is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0009] Figure 1 A schematic diagram of a process flow of a robot visual environment perception self-positioning method provided in an embodiment of the present application;

[0010] Figure 2 A schematic diagram of a process flow of configuring a reference parameter object set for a robot visual environment perception self-positioning method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0012] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0013] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0014] The present application embodiment provides a robot visual environment perception self-positioning method, such as Figure 1 As shown, the method includes:

[0015] Step S100, obtaining target location information, wherein the target location information includes multiple location coordinates and multiple robot identification codes stored in association. Specifically, when obtaining the target location information, first determine the data source channel. The location coordinates can be obtained outdoors through the global positioning system (GPS), and the location can be determined indoors using indoor positioning systems such as ultra-wideband (UWB), radio frequency identification (RFID) or Bluetooth beacons. Fixed measurement points can also be set in the working area to measure with laser rangefinders, cameras and other equipment, and a unique identification code is assigned to the robot at the same time. In the process of obtaining the location coordinates, a GPS receiver is used outdoors to receive satellite signals to calculate longitude and latitude, and signal parameters are used indoors to calculate distance and locate according to different technologies. After that, the location coordinates are calibrated and verified. A known location reference point can be set to compare the difference between the measurement and the actual location, and the accuracy can be improved by adjusting parameters, adding measurement points or optimizing algorithms. After obtaining the coordinates, use a database, data file or memory data structure to store them in association with the identification code, and finally check the data integrity and consistency to check whether there are missing values ​​and wrong associations. If there are any problems, repair them in time or re-collect them.

[0016] Step S200, using the target position information as the selection space to select the reference object, and configuring the reference reference object set, wherein the reference reference object set includes multiple reference positions and multiple corresponding reference robots. Specifically, firstly, the position coordinate distribution, density, mutual relationship and correspondence with the robot identification code in the target position information are deeply analyzed, and then no less than 4 Z position coordinates are randomly selected as the reference reference position candidates in the selection space formed by the target position information, and the coverage of the working area is fully considered. Then, the mutual distance variance of the Z reference positions is calculated as the first reference quality coefficient to measure the degree of discreteness, and the position coverage rate is calculated based on the preset neighborhood radius as the second reference quality coefficient. If the first reference quality coefficient is less than or equal to the first reference quality threshold and the second reference quality coefficient is greater than or equal to the second reference quality threshold, the corresponding robot is used as the reference robot and the position coordinates to form a reference reference object set, and then its effectiveness is verified by simulated positioning or small-scale testing. If there is a deviation or instability problem, the selection process is reviewed, the coefficient calculation and threshold setting are checked and adjusted as needed, and the optimization of the set provides a data basis for subsequent steps.

[0017] In one possible implementation, Figure 2As shown, the target position information is used as the selection space for the reference object selection, and the reference reference object set is configured, wherein the reference reference object set includes multiple reference positions and multiple corresponding reference robots, and step S200 further includes step S210, randomly selecting Z position coordinates in the target position information to obtain Z reference reference positions, wherein Z is greater than or equal to 4, and Z is a positive integer. Specifically, first, a random selection is performed among all the position coordinates covered by the target position information, and the number of selected position coordinates is Z (Z is greater than or equal to 4 and Z is a positive integer), and the selected position coordinates will become the reference reference position. The randomness is to avoid the influence of human bias or specific mode in the selection process, so that the selection result can cover the target working area more widely. For example, if the target position information contains multiple position coordinates in a larger rectangular working area, the random selection process is like randomly scattering points in this rectangular area, and each position coordinate has the same probability of being selected, thereby ensuring the fairness and comprehensiveness of the selection, so as to obtain Z reference reference positions.

[0018] Step S220, obtain the mutual distances between the Z reference positions, and calculate the variance of the mutual distances as the first reference quality coefficient. Specifically, for the Z reference reference positions obtained, it is necessary to calculate the mutual distances between them, and calculate the distance between each two reference reference positions through a specific distance calculation formula (Euclidean distance formula, if the position coordinates are two-dimensional and distance; if they are three-dimensional coordinates, they are expanded accordingly), and the mutual distances are combined into a data set, and the variance of this data set is calculated and used as the first reference quality coefficient. The variance reflects the degree of discreteness of these reference reference positions in space. If the variance is large, it indicates that the distance difference between the reference reference positions is large, and the position distribution is uneven, which may lead to too much information in some areas during the positioning process, while insufficient information in other areas. This coefficient is crucial to measuring the quality of the reference reference positions, because appropriate uniformity can ensure that the reference reference positions are reasonably distributed in space, providing a stable and comprehensive reference for subsequent positioning work.

[0019] Step S230, based on the preset neighborhood radius, statistically obtain the position coverage rate of the Z reference positions as the second reference quality coefficient. Specifically, based on the preset neighborhood radius, the position coverage rate of the Z reference reference positions is statistically obtained, and it is used as the second reference quality coefficient. For each reference reference position, an area is delineated with the preset neighborhood radius as the center. For example, if it is on a two-dimensional plane, this area may be a circle (three-dimensional may be a sphere, etc.), and the ratio of the total area covered by the statistical area to the total area of ​​the target working area is obtained to obtain the position coverage rate. The higher the coverage rate, the more the distribution of the reference reference positions in space can fully cover the target working area, and more position information can be used in the subsequent positioning process, which is conducive to improving the accuracy and stability of positioning.

[0020] Step S240, if the first reference quality coefficient is less than or equal to the first reference quality threshold, and the second reference quality coefficient is greater than or equal to the second reference quality threshold, then the Z reference positions are used as Z reference robots, and the reference reference object set is configured. Specifically, a conditional judgment is performed: if the calculated first reference quality coefficient is less than or equal to the first reference quality threshold, and the second reference quality coefficient is greater than or equal to the second reference quality threshold, it means that the discrete degree and coverage degree of the Z reference reference positions in the spatial distribution meet the requirements, the robots corresponding to the Z reference reference positions are determined as Z reference robots, and the reference robots and their position information are configured together as a reference reference object set, and the object set will provide a stable and reliable reference basis for subsequent environmental image acquisition and positioning based on machine vision and other operations, ensuring the accuracy and effectiveness of the entire robot positioning system.

[0021] Step S300, activate multiple reference robots to collect environmental images, obtain environmental image data, and perform reference positioning on multiple reference robots and multiple follower robots according to the environmental image data. Specifically, an activation command is sent to multiple reference robots to start their environmental image acquisition systems, and the high-definition cameras and depth sensors equipped on these robots start working. The cameras collect visual image data from different perspectives, and the depth sensors obtain depth image data. During the collection, the robots adjust according to preset strategies, such as changing the shooting angle according to time intervals or according to their own motion states. Then, the visual image data is aligned and fused into panoramic image data by a registration algorithm based on feature points or regions, and then features such as texture, shape, color, depth gradient, and normal direction are extracted from the environmental image data (panoramic image and depth image). Based on the principle of machine vision, these features are used to identify N (N ≥ 3 and N is a positive integer) external reference objects in the positioning environment image, and then the real-time position of the reference robot is calculated based on their relative relationship with the reference robot (through geometric calculation methods such as triangulation principles). Finally, based on the relationship between the reference robot and the follower robot (such as relative distance and angle information, which can be obtained through communication or other measurement methods), the position of the follower robot is inferred by combining interpolation or model prediction methods to complete the benchmark positioning of the reference robot and the follower robot.

[0022] In a possible implementation, multiple reference robots are activated to collect environmental images, obtain environmental image data, and perform reference positioning on multiple reference robots and multiple follower robots according to the environmental image data. Step S300 further includes step S310, controlling multiple reference robots and multiple follower robots to perform position initialization according to the target position information. Specifically, multiple reference robots and multiple follower robots are controlled to perform position initialization according to the target position information. The target position information contains the position coordinates of each robot and the corresponding identification code. The coordinates provide a reference for the initial position of the robot. For the reference robot, according to its coordinates in the target position information, the robot is driven to move to the vicinity of the corresponding position through the robot's motion control system, involving multiple navigation methods. For example, if it is in an indoor environment, an indoor positioning system (such as based on Wi-Fi, Bluetooth or UWB technology) can be used to guide the robot to the initial position; in an outdoor environment, a global positioning system such as GPS can be used. For the follower robot, the position is initialized according to the target position information and a predetermined deployment strategy, so that it is roughly in the expected initial position, and is ready for subsequent precise positioning.

[0023] Step S320, collecting IMU data of the plurality of reference robots and the plurality of follower robots in real time to perform preliminary position estimation, and correcting the position initialization result based on the preliminary position estimation result. Specifically, while the robot moves to the initialization position, the inertial measurement unit (IMU) data of multiple reference robots and multiple follower robots are collected in real time. The IMU usually includes an accelerometer and a gyroscope. The accelerometer can measure the acceleration information of the robot in all directions. Acceleration data is the key to preliminary position estimation. Since the change of the robot's position is related to the acceleration during the movement, the velocity information can be obtained by integrating the acceleration over time, and the position information can be obtained by integrating the velocity. The acceleration data collected by the accelerometer in the IMU is integrated according to the integration principle for each robot in three-dimensional space to obtain the velocity and position information, thereby realizing preliminary position estimation. For multiple reference robots and multiple follower robots, this process is carried out at the same time to obtain their respective preliminary position estimation results. It is necessary to consider the influence of the sampling frequency, noise and other factors of the IMU data on the estimation results. In order to reduce the interference of noise, data processing methods such as filtering can be used. For example, a Kalman filter is used to filter the acceleration data to improve the data quality, thereby improving the accuracy of the preliminary position estimation, and the preliminary position estimation result is compared with the initial position. The initialization results are compared with the initialization results. Since the position initialization is only a rough position setting based on the target position information, there may be certain errors. The initial position estimation obtains more real-time and accurate position information through the integral calculation of the IMU data (although there are certain errors, it is relatively more accurate). The coordinate deviations between the two in various directions are compared to analyze the causes of the deviations. The deviations are caused by many factors, such as the mechanical error of the robot during the initialization movement, the motion deviation caused by environmental interference (such as the influence of airflow and uneven ground on the robot's movement), or the measurement error of the IMU itself. According to the results of the deviation analysis, the position initialization results are corrected. If the deviation in a certain direction is large, the robot's motion control system can be adjusted to make the robot move in the direction of reducing the deviation. For example, if the initial position estimation shows that the robot's position in the x direction is a certain distance to the right of the initialization position, then the robot is controlled to move to the left by the corresponding distance to correct the position. At the same time, for multiple robots, they can be corrected according to their respective deviations to make the positions of all robots more accurate, further reduce the errors in the subsequent positioning process, and lay the foundation for more accurate positioning.

[0024] In a possible implementation, multiple reference robots are activated to collect environmental images, obtain environmental image data, and perform reference positioning on multiple reference robots and multiple follower robots according to the environmental image data. Step S300 further includes step S330, traversing multiple reference robots, collecting multi-dimensional image data through image acquisition devices, and obtaining visual image data and depth image data. Specifically, multiple reference robots are traversed, each reference robot is equipped with a special image acquisition device, which can obtain multi-dimensional image data, start the image acquisition device, and start the acquisition process. During the acquisition process, the image acquisition device shoots the surrounding environment from different angles and dimensions. For example, the camera in the image acquisition device can rotate horizontally and vertically to cover a wider field of view, and collects visual image data containing rich information. The data records the appearance characteristics of objects in the environment, such as color, texture, shape, etc. The depth image data is collected using a depth sensor and other equipment. Each pixel value in the depth image data represents the distance information from the image acquisition point to the corresponding point in the environment, which provides a key basis for subsequent spatial positioning.

[0025] Step S340, registering and fusing the visual image data, acquiring environmental image data, and establishing a mapping relationship between the environmental image data and the depth image data, wherein the environmental image data is panoramic image data. Specifically, the collected visual image data is registered and fused. Since visual images from different perspectives have differences in position and angle when they are shot, they need to be aligned through a registration algorithm. Common registration methods are based on feature point matching. Feature points, such as corner points and edge points, are first extracted from each visual image. Matching feature point pairs are found by comparing feature descriptors of feature points in different images. Then, the transformation relationship between images is calculated based on the matched feature points, including translation, rotation, scaling and other transformations. The transformation relationship is used to fuse visual images from different perspectives into a unified coordinate system to obtain panoramic image data as environmental image data. Panoramic images can more comprehensively display the visual information of the surrounding environment. , avoiding the limitations of single-view images, and establishing a mapping relationship between the environmental image data (panoramic image data) and the depth image data while acquiring the environmental image data (panoramic image data). Since the visual image data and the depth image data are acquired simultaneously, they correspond in space. Through time synchronization and spatial calibration during the image acquisition process, the correspondence between each pixel in the visual image and the corresponding pixel in the depth image can be determined. For example, when the internal and external parameters of the image acquisition device are known, this mapping can be established based on the geometric projection relationship. When processing a certain pixel in the environmental image data, its corresponding depth information can be easily obtained, providing support for subsequent object positioning and distance calculation.

[0026] Step S350, based on the principle of machine vision, identify and locate N external reference objects in the environmental image data for reference positioning, wherein N is greater than or equal to 3 and is a positive integer. Specifically, based on the principle of machine vision, identify and locate N external reference objects in the environmental image data. N is greater than or equal to 3 and is a positive integer. This is to ensure that there are enough reference objects to accurately determine the position of the robot, and perform feature analysis on the external reference objects. The reference objects have specific visual features, such as unique shapes, colors, textures, or a combination thereof. In the environmental image data, the target detection algorithm is used to search for areas with these features. For example, if the external reference object is a red circular marker, the algorithm will look for areas with red circular features in the panoramic image data. It can be based on a machine learning model, such as a convolutional neural network (CNN). After being trained with a large amount of training data, it can accurately identify objects with specific features. After identifying the objects, the algorithm can detect the areas with red circular features. Based on the external reference object, it is positioned. According to the mapping relationship between the environmental image data and the depth image data, combined with the parameters of the image acquisition device (such as focal length, optical center position, etc.) and the principle of geometric projection, the position of the external reference object in the three-dimensional space is calculated. For example, the pixel coordinates of the reference object detected in the image, combined with the depth information, can be used to calculate its coordinates in the actual space using the triangulation principle. By identifying and positioning N external reference objects, an accurate spatial reference point is provided for the reference robot, thereby achieving reference positioning, determining the exact position of the reference robot in the environment, and providing an important basis for the positioning of the subsequent follower robot.

[0027] In one possible implementation, based on the principle of machine vision, N external reference objects in the environmental image data are identified and located for reference positioning, wherein N is greater than or equal to 3 and is a positive integer. Step S350 further includes step S351, respectively obtaining the relative azimuths of the multiple reference robots and the N external reference objects, and calculating the real-time positions of the multiple reference robots based on the relative azimuths. Specifically, first, for each reference robot, the relative azimuth angles with N external reference objects are obtained respectively through the sensors carried by it (such as visual sensors, laser radars, etc.), which involves complex sensor data processing. Taking visual sensors as an example, the position of the external reference object in the image is identified by machine vision algorithms, and the relative azimuth angle is calculated through geometric relationships in combination with the installation position and posture information of the sensor. If it is a laser radar, the direction information of the external reference object is determined according to the reflected laser point cloud data, and then the relative azimuth angle is obtained. The real-time positions of multiple reference robots are calculated according to the obtained relative azimuth angles. Assuming that the absolute positions of the external reference objects are known (this can be predetermined in the environment setting stage), the triangulation principle is used for calculation. In a two-dimensional plane, if there are two external reference objects A and B, the relative azimuth angles of the reference robot R with them are respectively, and the distance between A and B is known to be d, then the position coordinates of R can be solved by listing equations through trigonometric function relationships. In three-dimensional space, at least three external reference objects are required to determine the position of the reference robot. The (x, y, z) coordinates of the reference robot are obtained through more complex spatial geometric calculations to obtain its real-time position.

[0028] Step S352, compare the real-time position with the target position information to obtain the real-time reference deviations of the multiple reference robots. Specifically, the calculated real-time position of the reference robot is compared with the target position information, the target position information is the preset ideal position where the reference robot should be, the comparison process is performed in the same coordinate system, the coordinate value of the real-time position is subtracted from the coordinate value of the target position one by one, for example, the real-time position coordinate is, the target position coordinate is, then the deviation in direction is, and the same is true for direction and direction, and the real-time reference deviations of multiple reference robots in each direction are obtained.

[0029] Step S353, according to the Euclidean distances between the multiple reference robots and the multiple follower robots and the real-time reference deviations, determine the real-time estimated deviations of the multiple follower robots by an interpolation method. Specifically, the Euclidean distances between the multiple reference robots and the multiple follower robots are calculated to obtain the distance information between each reference robot and each follower robot. According to the calculated Euclidean distances and the real-time reference deviations of the reference robots, the real-time estimated deviations of the multiple follower robots are determined by an interpolation method. The interpolation method can select linear interpolation. If the Euclidean distance between the reference robot and the follower robot is , the Euclidean distance between the reference robot and the follower robot is , the real-time reference deviation is , and the real-time reference deviation is , then the real-time estimated deviation of the follower robot can be calculated by linear interpolation according to the distance weight, that is, the real-time estimated deviation of each follower robot is obtained.

[0030] Step S354, based on the real-time reference deviation and the real-time estimated deviation, the reference positioning of the multiple reference robots and the multiple follower robots is performed. Specifically, based on the real-time reference deviation and the real-time estimated deviation, the reference positioning of the multiple reference robots and the multiple follower robots is performed. For the reference robot, its real-time reference deviation has been obtained, and the deviation is reduced by adjusting its own position or updating the position estimate to make it closer to the target position. For the follower robot, its position is estimated and adjusted according to the real-time estimated deviation. For example, if the real-time estimated deviation is a positive value in a certain direction, it means that the follower robot has deviated from the expected position in this direction. According to the deviation size and the robot's movement ability, the movement direction and distance of the follower robot are adjusted through the control algorithm to achieve reference positioning, so that the position of the entire robot system is more accurate and stable.

[0031] Step S400, based on the benchmark positioning result, calling multiple follower robots to collect internal environment images and obtain internal environment image data, wherein the internal environment image data includes a single environment image set of multiple follower robots. Specifically, based on the benchmark positioning results of the benchmark robot and the follower robot, calling multiple follower robots to collect internal environment images. The benchmark positioning result provides the follower robot with initial position information, so that it can collect in a targeted manner. The image acquisition device (such as a camera) equipped with the follower robot starts working, and the camera converts light into image data through a lens and an image sensor. During the collection, the follower robot adjusts the camera angle and parameters according to its own posture and position to obtain a clear and comprehensive image, and also collects multi-view original internal environment images from different perspectives (including horizontal rotation and vertical pitch shooting), and then performs preliminary processing (such as median filtering, histogram equalization) on the original image to improve the quality and organize the numbering, and finally classifies the processed image according to the follower robot to form a single environment image set containing its surrounding environment information (such as obstacles, the appearance and relative position of other robots).

[0032] In a possible implementation, based on the reference positioning result, multiple follower robots are called to collect internal environment images to obtain internal environment image data, wherein the internal environment image data includes a single environment image set of multiple follower robots, and step S400 further includes step S410, wherein the image acquisition devices of the multiple follower robots obtain multiple groups of original internal environment images, wherein the original internal environment images include original visual images and original depth images. Specifically, the image acquisition devices of the multiple follower robots start working to obtain multiple groups of original internal environment images. The image acquisition devices generally include a visual sensor and a depth sensor. The visual sensor is responsible for collecting the original visual image. It captures the light in the environment through a lens and converts the light information into a digital signal to form an original visual image. The image presents the appearance characteristics of the objects in the environment, such as color, texture and shape, etc. The depth sensor collects the original depth image. The principle is to use a specific technology (such as time of flight method, structured light method, etc.) to measure the distance from the sensor to the surface of the object in the environment. The value of each pixel represents the distance information corresponding to the point. These original depth images provide a basis for the subsequent analysis of the spatial structure of the environment.

[0033] Step S420, register multiple groups of the original internal environment images, and obtain multiple groups of registered visual images and registered depth images. Specifically, the multiple groups of original internal environment images obtained are registered to obtain multiple groups of registered visual images and registered depth images. Since the follower robot may be in different postures and positions during the image acquisition process, even the images collected from the same object or area may have deviations in space. Image registration is to find the geometric transformation relationship between these images so that they are aligned in the same coordinate system. Commonly used registration methods include feature-based registration and region-based registration. Feature-based registration will first extract feature points (such as corner points, edge points, etc.) from the original visual image and the original depth image, and determine the transformation relationship between them such as translation, rotation and scaling by matching feature points in different images; the region-based registration principle is to directly use the regional information in the image for matching. Through the registration method, the original internal environment images are registered to obtain the registered visual image and depth image, so that they are consistent in space.

[0034] Step S430, calling the registered depth image to perform image segmentation based on the depth screening threshold, and based on the depth image segmentation result, correspondingly extracting the registered visual image to obtain the visual image segmentation result. Specifically, the determination of the depth screening threshold is crucial for focusing only on the environmental information inside the robot group. The threshold can be determined according to the target position information. For example, if the target position information defines a circumscribed circle radius, then the depth information within this radius is considered to be related to the internal environment of the robot group; or the neighborhood radius is determined according to the acquisition distance of the depth image acquisition device, and the depth information within this neighborhood radius is taken as the focus of attention. The threshold-based screening method can effectively exclude information in the environment that is irrelevant to the robot group, such as distant background objects, etc. The registered depth image is called for image segmentation based on the depth screening threshold. According to the set depth screening threshold, the pixels in the depth image are divided into different categories. For example, the pixels with depth values ​​within the threshold range are marked as pixels related to the robot group, and the pixels with depth values ​​exceeding the threshold are regarded as external background pixels. In this way, the segmentation of the depth image is achieved, and the environmental structure information inside the robot group is highlighted.

[0035] Step S440, output multiple groups of the visual image segmentation results and multiple groups of the depth image segmentation results as the internal environment image data. Specifically, based on the depth image segmentation results, the corresponding registered visual images are extracted to obtain the visual image segmentation results. Since the depth image and the visual image are registered, there is a one-to-one corresponding pixel relationship between them. When the depth image is segmented, according to this corresponding relationship, the pixel area related to the inside of the robot group is extracted from the registered visual image to form the visual image segmentation results. The visual image segmentation results and the depth image segmentation results match each other, and both only reflect the environmental information inside the robot group. The multiple groups of visual image segmentation results and the multiple groups of depth image segmentation results are output as the internal environment image data. The internal environment image data provides accurate and targeted information for subsequent robot self-positioning, environmental perception and decision-making operations, so that the robot can better understand its internal environment and lay the foundation for further task execution.

[0036] Step S500, combining multiple position coordinates to build an ideal position model, extracting internal environment reference information, parsing the single environment image set, obtaining follow-up comparison information, and comparing the internal environment reference information with the follow-up comparison information to obtain real-time follow-up deviation to perform self-positioning of multiple follower robots. Specifically, firstly, multiple position coordinates and single environment image sets are analyzed. The position coordinates are part of the target position information and can clarify the potential position of the robot in the workspace. The single environment image set is the internal environment image data collected by the follower robot, which contains rich visual information. Then, the position coordinates are mapped to a coordinate system related to the single environment image set, and image features are extracted. Based on these features, objects are identified and their relative positions are determined. The spatial relationship between the follower robot and other objects is inferred by combining with the position coordinate information. Then, the relative position information and position coordinates of the objects are comprehensively utilized, and self-positioning calculations are performed through complex geometric calculations and data fusion algorithms, such as triangulation principles or probability-based positioning methods. Finally, the positioning results are verified and optimized, compared with other auxiliary information, and if there is a deviation, the error source is checked and corrected to determine an accurate self-positioning result.

[0037] In a possible implementation, an ideal position model is constructed by combining multiple position coordinates, extracting internal environment reference information, parsing the single environment image set, obtaining tracking comparison information, and comparing the internal environment reference information with the tracking comparison information to obtain real-time tracking deviation to perform self-positioning of multiple tracking robots. Step S500 further includes step S510, mapping multiple position coordinates to a predefined multidimensional coordinate system, and configuring an ideal position model with multiple robot identification codes as object identifiers. Specifically, first, multiple position coordinates are mapped to a predefined multidimensional coordinate system, which is determined in advance according to the working environment and positioning requirements of the robot, for example, in a three-dimensional space, it can be a Cartesian coordinate system. In the mapping process, each position coordinate is accurately placed at the corresponding position of the multidimensional coordinate system, and multiple robot identification codes are used as object identifiers to associate the position information of each robot with its identification code, and the position information of all robots is integrated to configure an ideal position model. The model describes the position distribution of all robots in the multidimensional space under ideal conditions without errors and interference, and provides a reference framework for subsequent positioning calculations.

[0038] Step S520, traverse the ideal position model to perform random selection and determine the first follower robot. Specifically, perform a traversal operation on the configured ideal position model. During the traversal process, the first follower robot is determined by random selection. The random selection is performed among all follower robots to ensure that each follower robot has the same probability of being selected. The purpose of random selection is to introduce randomness and avoid deviations that may be caused by selecting a specific order or pattern, so that the reference coordinate system established subsequently is more general and representative.

[0039] Step S530, define the position coordinates of the first follower robot as the first reference coordinate origin, and construct a first reference coordinate system, wherein the first reference coordinate system has the same coordinate dimension and coordinate axis as the multidimensional coordinate system. Specifically, define the position coordinates of the selected first follower robot as the first reference coordinate origin, and construct the first reference coordinate system with the origin as the reference. The first reference coordinate system has the same coordinate dimension and coordinate axis as the previous multidimensional coordinate system. For example, if the multidimensional coordinate system is a three-dimensional Cartesian coordinate system, then the first reference coordinate system is also a three-dimensional Cartesian coordinate system, and the direction and unit of its coordinate axis are consistent with the multidimensional coordinate system. The constructed reference coordinate system provides a unified and relative reference frame for the subsequent calculation of relative positions, which is convenient for analyzing the position relationship between different robots.

[0040] Step S540, calculate and obtain the first relative position coordinates of the plurality of follower robots and the plurality of reference robots in the first reference coordinate system, store them as the first follower reference information, and iteratively obtain multiple sets of follower reference information, and output them as the internal environment reference information. Specifically, in the first reference coordinate system, the first relative position coordinates of the plurality of follower robots and the plurality of reference robots are calculated, and the calculation process utilizes the geometric relationship and coordinate transformation principle. For example, for a follower robot and a reference robot, their relative position coordinates in the first reference coordinate system are calculated through the coordinate relationship between their coordinates in the multidimensional coordinate system and the coordinate of the origin of the first reference coordinate system, and the calculated first relative position coordinates are stored as the first follower reference information. By iterating the above process, the above calculation is repeated for different follower robots in the ideal position model to obtain multiple sets of follower reference information, and each iteration reselects a follower robot as a new reference to calculate the corresponding relative position coordinates. Finally, these multiple groups of follow-up reference information are output to form internal environment reference information, which comprehensively reflects the relative position relationship between the target follow-up robot / the first follow-up robot and other robots under different reference conditions.

[0041] In a possible implementation, an ideal position model is constructed by combining multiple position coordinates, extracting internal environment reference information, parsing the single environment image set, obtaining follow-up comparison information, and comparing the internal environment reference information with the follow-up comparison information to obtain real-time follow-up deviation to perform self-positioning of multiple follower robots. Step S500 further includes step S550, randomly selecting from the single environment image set to extract a first single environment image set, wherein the first single environment image set is marked with a first robot identification code. Specifically, a random selection operation is performed in the single environment image set, and the single environment image set contains image information of its surrounding environment collected by each follower robot. A group of images is selected from this set through a random algorithm. This group of images is the first single environment image set. Since it is randomly selected, each single environment image set has the same probability of being selected, which ensures the randomness and fairness of the selection process. The first single environment image set is marked with a first robot identification code, and the identification code is used to uniquely identify the follower robot associated with the image set, so as to perform accurate information matching and processing later.

[0042] Step S560: perform target recognition based on the first monomer environment image set, locate the plurality of follower robots and the plurality of reference robots, and calculate the corresponding first follower comparison information, wherein the first follower comparison information includes the real-time position coordinates of the plurality of follower robots and the plurality of reference robots. Specifically, perform target recognition based on the first monomer environment image set, and use advanced image recognition technology, such as a target detection algorithm based on a convolutional neural network (CNN), to recognize objects in the image. In this scenario, the focus is on recognizing the plurality of follower robots and the plurality of reference robots. The algorithm can automatically extract the features of the objects in the image by learning a large amount of annotated data, and match them with the known robot feature model, so as to determine the position of the robot in the image. For each recognized robot, further determine its coordinate position in the image, and combine the relevant parameters during image acquisition (such as the internal and external parameters of the camera, the resolution of the image, etc.) and the pre-established coordinate system conversion relationship to calculate their real-time position coordinates in the actual environment. The real-time position coordinate information constitutes the first follower comparison information, which accurately reflects the position of the follower robot and the reference robot in the current first monomer environment image set perspective.

[0043] Step S570: According to the first robot identification code, the corresponding internal environment reference information and the first follow-up comparison information are called to perform deviation calculation to obtain a first real-time follow-up deviation. Specifically, according to the first robot identification code, the corresponding internal environment reference information is called, and the internal environment reference information is previously calculated by building a reference coordinate system and other methods, and includes the relative position relationship between the target follow-up robot / the first follow-up robot and other robots. The internal environment reference information and the first follow-up comparison information are used to perform deviation calculation. For example, the deviation sum (vector sum) of the deviations (vectors) of the target monomer (which can be understood here as the robot associated with the first robot identification code) relative to multiple robots is calculated. The specific calculation process may involve vector subtraction operations, that is, subtracting the real-time position coordinate vector in the first follow-up comparison information from the corresponding position coordinate vector in the internal environment reference information, and then summing these deviation vectors and other operations, so as to obtain the first real-time follow-up deviation, and the deviation value reflects the difference between the current actual position and the reference position.

[0044] Step S580, traverse the monomer environment image set, iteratively update and select the first monomer environment image set, obtain multiple real-time follow-up deviations of multiple follow-up robots, and perform self-positioning. Specifically, traverse the monomer environment image set, iteratively update and select the first monomer environment image set, and repeat the target recognition, first follow-up comparison information calculation and deviation calculation steps after each selection of a new first monomer environment image set, obtain multiple real-time follow-up deviations of multiple follow-up robots, the deviation value reflects the position change of the follow-up robot from different perspectives and time points, and perform self-positioning according to the obtained multiple real-time follow-up deviations. Use the deviation information to estimate and adjust the position of the follow-up robot through a suitable positioning algorithm (such as least squares method, Kalman filtering, etc.). For example, if the real-time follow-up deviation shows that the robot deviates from the reference position in a certain direction, then according to the size and direction of the deviation, the position estimation of the robot is correspondingly corrected, thereby achieving accurate self-positioning of the follow-up robot, so that the robot can determine its own accurate position in a complex environment.

[0045] The embodiment of the present application adopts the method of obtaining multiple position coordinates and associated robot identification codes, selecting a reference object and configuring a reference reference object set. The reference robot is activated to collect environmental images, and after obtaining environmental image data, the follower robot is positioned according to the reference positioning result. The follower robot collects internal environmental images according to the reference positioning result to generate a single environmental image set. Finally, the image data is parsed in combination with the position coordinates to achieve the self-positioning of the follower robot, achieving the technical effect of improving the self-positioning accuracy and stability of the multi-robot system by combining the reference robot positioning reference and the accurate analysis of the follower robot environmental image data.

[0046] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A robot visual environment perception self-positioning method, characterized in that: The method comprises: Acquiring target position information, wherein the target position information includes a plurality of position coordinates and a plurality of robot identification codes stored in association; Using the target position information as a selection space to select a reference object, and configuring a reference reference object set, wherein the reference reference object set includes a plurality of reference positions and a plurality of corresponding reference robots; Activating the plurality of reference robots to collect environmental images, acquiring environmental image data, and performing reference positioning of the plurality of reference robots and the plurality of follower robots according to the environmental image data; Based on the reference positioning result, calling a plurality of the follower robots to collect internal environment images to obtain internal environment image data, wherein the internal environment image data includes a single environment image set of the plurality of the follower robots; Combining the plurality of position coordinates to construct an ideal position model, extracting internal environment reference information, parsing the single environment image set, obtaining follow-up comparison information, and comparing the internal environment reference information with the follow-up comparison information to obtain real-time follow-up deviation to perform self-positioning of the plurality of follow-up robots; Combining the plurality of position coordinates to construct an ideal position model, extracting internal environment reference information, parsing the single environment image set, obtaining follow-up comparison information, and comparing the internal environment reference information with the follow-up comparison information to obtain real-time follow-up deviation to perform self-positioning of the plurality of follow-up robots, including: Mapping the plurality of position coordinates to a predefined multi-dimensional coordinate system, and configuring an ideal position model using the plurality of robot identification codes as object identifiers; Traversing the ideal position model to randomly select and determine a first follower robot; Define the position coordinates of the first follower robot as a first reference coordinate origin, and construct a first reference coordinate system, wherein the first reference coordinate system and the multi-dimensional coordinate system have the same coordinate dimension and coordinate axis; Calculate and obtain first relative position coordinates of the plurality of follower robots and the plurality of reference robots in the first reference coordinate system, store them as first follower reference information, and iteratively obtain multiple sets of follower reference information, and output them as the internal environment reference information; Combining the plurality of position coordinates to construct an ideal position model, extracting internal environment reference information, parsing the single environment image set, obtaining follow-up comparison information, and comparing the internal environment reference information with the follow-up comparison information to obtain real-time follow-up deviation to perform self-positioning of the plurality of follow-up robots, further comprising: Randomly selecting from the monomer environment image set, extracting a first monomer environment image set, wherein the first monomer environment image set is marked with a first robot identification code; Performing target recognition according to the first monomer environment image set, positioning the plurality of follower robots and the plurality of reference robots, and calculating corresponding first follower comparison information, wherein the first follower comparison information includes real-time position coordinates of the plurality of follower robots and the plurality of reference robots; According to the first robot identification code, calling the corresponding internal environment reference information and the first follow-up comparison information to perform deviation calculation to obtain a first real-time follow-up deviation; The single body environment image set is traversed, the first single body environment image set is selected by iterative updating, a plurality of real-time follow-up deviations of a plurality of the follow-up robots are obtained, and self-positioning is performed.

2. A robot visual environment perception self-positioning method as claimed in claim 1, characterized in that: The target position information is used as a selection space to select a reference object, and a reference reference object set is configured, including: Randomly select Z position coordinates from the target position information to obtain Z benchmark reference positions, where Z is greater than or equal to 4 and is a positive integer; Obtaining mutual distances between Z reference positions, and calculating a variance of the mutual distances as a first reference quality coefficient; Based on a preset neighborhood radius, obtaining statistically the position coverage of the Z reference positions as a second reference quality coefficient; If the first reference quality coefficient is less than or equal to a first reference quality threshold, and the second reference quality coefficient is greater than or equal to a second reference quality threshold, the reference object set is configured with Z reference reference positions as Z reference robots.

3. A robot visual environment perception self-positioning method as claimed in claim 2, characterized in that: Before activating the plurality of reference robots to collect environmental images, acquiring environmental image data, and performing reference positioning of the plurality of reference robots and the plurality of follower robots according to the environmental image data, the method comprises: According to the target position information, controlling the plurality of reference robots and the plurality of follower robots to perform position initialization; The IMU data of the plurality of reference robots and the plurality of follower robots are collected in real time to perform preliminary position estimation, and the position initialization result is corrected based on the preliminary position estimation result.

4. A robot visual environment perception self-positioning method as claimed in claim 3, characterized in that: Activating the plurality of reference robots to collect environmental images, acquiring environmental image data, and performing reference positioning of the plurality of reference robots and the plurality of follower robots according to the environmental image data, including: Traversing the plurality of reference robots, collecting multi-dimensional image data through an image acquisition device, and acquiring visual image data and depth image data; Performing registration and fusion on the visual image data to obtain environmental image data, and establishing a mapping relationship between the environmental image data and the depth image data, wherein the environmental image data is panoramic image data; Based on the principle of machine vision, N external reference objects in the environmental image data are identified and located for reference positioning, wherein N is greater than or equal to 3 and is a positive integer.

5. A robot visual environment perception self-positioning method as claimed in claim 4, characterized in that: Based on the principle of machine vision, N external reference objects in the environmental image data are identified and located for reference positioning, including: Respectively obtaining relative azimuths of the plurality of reference robots and the N external reference objects, and calculating real-time positions of the plurality of reference robots according to the relative azimuths; Comparing the real-time position with the target position information to obtain real-time reference deviations of a plurality of reference robots; Determine the real-time estimated deviations of the plurality of follower robots by an interpolation method according to the Euclidean distances between the plurality of reference robots and the plurality of follower robots and the real-time reference deviations; The reference positioning of the plurality of reference robots and the plurality of follower robots is performed based on the real-time reference deviation and the real-time estimated deviation.

6. A robot visual environment perception self-positioning method as claimed in claim 5, characterized in that: Based on the reference positioning result, calling a plurality of the follower robots to collect images of the internal environment to obtain image data of the internal environment includes: The image acquisition devices of the plurality of follower robots are used to obtain a plurality of groups of original internal environment images, wherein the original internal environment images include original visual images and original depth images; Registering a plurality of sets of the original internal environment images to obtain a plurality of sets of registered visual images and registered depth images; Calling the registered depth image to perform image segmentation based on a depth screening threshold, and based on the depth image segmentation result, correspondingly extracting the registered visual image to obtain a visual image segmentation result; Output multiple groups of the visual image segmentation results and multiple groups of the depth image segmentation results as the internal environment image data.

Images