AGV visual navigation method and system
Through the multi-sensor fusion technology of the AGV visual navigation system, using a monocular camera and laser sensor, the problem of AGV navigation and obstacle avoidance in complex environments is solved, and the rapid response and accurate motion state control is achieved, which improves the navigation efficiency of AGV and Security.
Patent Information
- Application Number
- CN202010878329.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-08-27
AI Technical Summary
The existing AGV navigation system is difficult to achieve efficient autonomous navigation and obstacle avoidance in the logistics environment, especially in complex scenarios, where the response time and motion state determination time are long.
Through the multi-sensor fusion of monocular cameras and laser sensors, image data around AGV is acquired and processed, and images within 300° are captured using a wide-angle camera, and the motion displacement of the target object is determined based on the combination of feature point matching and edge pixel comparison. Motion displacement controls the motion state and path of AGV.
The rapid response and accurate motion state determination of AGV are realized, which improves the navigation efficiency and safety of AGV in complex environments, and has the advantages of short response time and fast motion state determination.
Smart Images

Figure CN111998853B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robot navigation, and in particular to an AGV visual navigation method and system. Background Art
[0002] As a commonly used handling robot in logistics warehouses, the AGV (Automated Guided Vehicle, equipped with electromagnetic or optical automatic navigation devices) system has been widely used in tobacco, banking, papermaking and other industries due to its flexibility, reliability, low cost and high safety in logistics.
[0003] With the continuous development of machine vision technology and computer technology, AGV navigation technology based on machine vision has gradually become an important research direction in the field of intelligent robots.
[0004] In view of this, it is necessary to provide an AGV visual navigation method and system to solve the above problems. Summary of the invention
[0005] The present invention provides an AGV visual navigation method and system, which realizes map construction and real-time positioning through multi-sensor fusion of a monocular camera and a laser sensor, realizes the robot's obstacle avoidance function, and provides more feasible navigation for the robot's autonomous walking.
[0006] The technical solution for achieving the purpose of the present invention is as follows:
[0007] An AGV visual navigation method, the method comprising:
[0008] Get the surrounding image of the current location of the AGV robot.
[0009] Retrieving a previous surrounding image of the current location, wherein there is an interval between the previous surrounding image and the surrounding image, and the interval is at least 2 seconds;
[0010] Determine the motion displacement of the target object in the previous surrounding image and the surrounding image at the interval time;
[0011] The motion state and motion route of the AGV robot are controlled according to the motion displacement.
[0012] As a further improvement of the present invention, the step of obtaining the surrounding image of the current position of the AGV robot includes:
[0013] A wide-angle camera is installed on the chassis of the AGV robot;
[0014] The wide-angle camera captures the surrounding image of the current position of the AGV robot, and the surrounding image is a wide-angle image within a range of 300° in front of the AGV robot;
[0015] The target object of the wide-angle image is calibrated, and the target object includes one or more of a charging pile, other AGV robots, fixed objects, walls, or moving objects.
[0016] As a further improvement of the present invention, the step of retrieving a previous surrounding image of the current location includes:
[0017] Process the real-time surrounding images collected by the AGV robot, and mark the coordinate data of the AGV robot and the position coordinate points of the target object relative to the AGV robot in each frame of the surrounding image according to the known coordinate system;
[0018] calibrating the target object of the real-time surrounding image according to different coordinate data, and storing the real-time surrounding image in an image database according to the classification of the coordinate data;
[0019] When the AGV robot moves to the corresponding coordinate data, the interval frame surrounding image of the AGV robot at the coordinate data is retrieved from the image database, and the interval time between the interval frame surrounding image and the real-time surrounding image is at least 2 seconds.
[0020] As a further improvement of the present invention, determining the motion displacement of the target object in the previous surrounding image and the surrounding image at the interval time includes:
[0021] The previous surrounding images first select the stored image target object before storing;
[0022] After receiving the surrounding image, selecting the real-time image target object of the surrounding image, the stored image target object and the real-time image target object are the same target object;
[0023] The edge pixels of the stored image target object are compared with the edge pixels of the real-time image target object to obtain the motion displacement of the real-time image target object relative to the stored image target object.
[0024] As a further improvement of the present invention, the step of comparing edge pixels of the stored image target object with edge pixels of the real-time image target object to obtain the motion displacement of the real-time image target object relative to the stored image target object includes:
[0025] Determining in the surrounding image a second feature region corresponding to the first feature region in the previous surrounding image;
[0026] Retrieving the previous surrounding image, wherein the previous surrounding image is marked with at least one first feature area;
[0027] Downsampling the surrounding image to obtain candidate feature regions, wherein the number of the candidate feature regions is greater than or equal to the first feature regions;
[0028] Matching a second feature region corresponding to the first feature region in the candidate feature region according to edge pixels;
[0029] The first feature area contains a stored image target object, and the second feature area contains a real-time image target object.
[0030] Taking the camera coordinates of the wide-angle camera as a reference, determining the coordinate points of the stored image target object in the first feature area and the coordinate points of the real-time image target object in the second feature area;
[0031] The motion displacement of the real-time image target object relative to the stored image target object is obtained according to the difference between the two coordinate points.
[0032] As a further improvement of the present invention, controlling the motion state and motion route of the AGV robot according to the motion displacement includes:
[0033] If the motion displacement is 0, it is determined that the target object is stationary, and the AGV robot is controlled to bypass the target object and continue to move along the original planned path;
[0034] If the motion displacement is a negative value, the original motion state and the original motion path of the AGV robot corresponding to the previous surrounding image are retrieved, and the AGV robot is controlled to move according to the original motion state and the original motion path;
[0035] If the motion displacement is a positive value, the AGV robot is controlled to stop moving, and the latest surrounding images of the AGV robot's current position are continuously acquired. The edge pixels of the target object in the stored image are compared with the edge pixels of the target object in the latest image until the target object disappears. The AGV robot is controlled to continue moving along the original planned path.
[0036] As a further improvement of the present invention, determining the motion displacement of the target object in the previous surrounding image and the surrounding image at the interval time includes:
[0037] Retrieving the previous surrounding image, where the previous surrounding image and the surrounding image have an interval of more than 2 seconds;
[0038] Selecting a first target object in the previous surrounding image and a second target object in the surrounding image to obtain motion displacements of the first target object and the second target object at the interval time, wherein the first target and the second target are different states of the same target object at the interval time;
[0039] Identify a first target in a first feature area and a second target in a second feature area, and superimpose the first feature area and the second feature area to obtain a superimposed area; detect a position change relationship of the second target relative to the first target in the superimposed area, and determine a motion state of the second target based on the position change relationship, wherein the motion state is a stationary target or a moving target;
[0040] Determine the motion condition of the target object based on the motion displacement and the motion state;
[0041] The motion state and motion route of the AGV robot are controlled according to the motion situation.
[0042] An AGV visual navigation system, comprising:
[0043] Image acquisition module, to obtain the surrounding images of the current location of the AGV robot;
[0044] The navigation chip retrieves a previous surrounding image of the current location, wherein the previous surrounding image and the surrounding image have an interval of at least 2 seconds; and determines the motion displacement of the target object in the previous surrounding image and the surrounding image during the interval;
[0045] A control module controls the motion state and motion route of the AGV robot according to the motion displacement.
[0046] As a further improvement of the present invention, the above-mentioned AGV visual navigation method is run on the navigation chip.
[0047] Compared with the prior art, the present invention has the following beneficial effects:
[0048] 1. The present invention retrieves previous surrounding images at different time intervals, compares the previous surrounding images with the real-time surrounding images of the target object, and thereby determines the next motion state of the AGV robot. The present invention has the advantages of short response time and fast motion state determination time.
[0049] 2. The present invention stores previous surrounding images according to different position classifications and calibrates the position data of the target objects in the previous surrounding images, which has the advantage of fast retrieval. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 It is a flow chart of the AGV visual navigation method;
[0051] Figure 2 for Figure 1 Flow chart of step S101;
[0052] Figure 3 for Figure 1 Flow chart of step S102;
[0053] Figure 4 for Figure 1 Flow chart of step S103;
[0054] Figure 5 for Figure 4 Flow chart of step S103c in FIG.
[0055] Figure 6 This is the principle block diagram of the AGV visual navigation system;
[0056] Figure 7 It is a detailed flow chart of the AGV visual navigation method;
[0057] Figure 8 Flowchart for FAST corner extraction. DETAILED DESCRIPTION
[0058] The present invention is described in detail below in conjunction with the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in the field based on these embodiments are all within the scope of protection of the present invention.
[0059] Unless the context requires otherwise, throughout the specification and claims, the term "comprise" and other forms thereof, such as the third person singular form "comprises" and the present participle form "comprising", are to be interpreted as open, inclusive, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific example" or "some examples" and the like are intended to indicate that specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner.
[0060] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.
[0061] “At least one of A, B, and C” has the same meaning as “at least one of A, B, or C” and both include the following combinations of A, B, and C: A only, B only, C only, the combination of A and B, the combination of A and C, the combination of B and C, and the combination of A, B, and C.
[0062] “A and / or B” includes the following three combinations: A only, B only, and a combination of A and B.
[0063] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that" or "in response to determining that" or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.
[0064] The use of "adapted to" or "configured to" herein is meant to be open and inclusive language that does not exclude devices adapted or configured to perform additional tasks or steps.
[0065] Additionally, the use of “based on” is meant to be open and inclusive, as a process, step, calculation, or other action “based on” one or more stated conditions or values may, in practice, be based on additional conditions or values beyond those stated.
[0066] As used herein, "about" or "approximately" includes the stated value and the average value that is within an acceptable range of deviation from the particular value, where the acceptable range of deviation is determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).
[0067] See also Figure 1 The present invention discloses an AGV visual navigation method, comprising:
[0068] S101, obtaining the surrounding image of the current position of the AGV robot.
[0069] S102: Retrieve a previous surrounding image of the current location, where there is an interval between the previous surrounding image and the surrounding image, and the interval is at least 2 seconds.
[0070] S103: Determine the motion displacement of the target object in the previous surrounding image and the surrounding image at the interval time.
[0071] S104, controlling the motion state and motion route of the AGV robot according to the motion displacement.
[0072] Exemplarily, the current position in step S101 refers to the real-time positioning coordinates of the AGV robot in the coordinate system. The surrounding images in step S101 are not completely limited to the images around the AGV robot, but are based on the image acquisition device installed on the AGV robot, and the images within the range of 180° to 320° are collected according to the actual situation.
[0073] In some embodiments, see Figure 2 Step S101 can be specifically implemented through the following steps S101a to S101c.
[0074] S101a, AGV robot has a wide-angle camera installed on its chassis.
[0075] S101b. The wide-angle camera captures the surrounding image of the current position of the AGV robot. The surrounding image is a wide-angle image within a range of 300° in front of the AGV robot.
[0076] S101c, calibrating the target object of the wide-angle image, where the target object includes one or more of a charging pile, other AGV robots, fixed objects, walls, or moving objects.
[0077] In actual application, the preferred image acquisition device is a wide-angle camera. In the surrounding images captured by the wide-angle camera, possible target objects include: fixed buildings (walls, doors, load-bearing columns, elevators, steps, slopes), office items (tables, chairs, charging piles, cargo), and moving objects (people, other AGV robots, transport vehicles, elevators).
[0078] Using the surrounding images captured by the wide-angle camera, the surrounding images are time-aligned, matched, and trusted to fuse, thereby completing the perception of the surrounding environment of the current location of the AGV robot. Based on the perception information. Specifically, (1) Based on the surrounding images captured by the wide-angle camera, the detection algorithm can locate the specific position of the target object in front (fixed buildings, office objects, moving objects) and provide its approximate outline. The error range at this stage is still large. (2) Based on the position and outline of the target object in front, the classification algorithm accurately identifies the target object in front, and the regression algorithm locates its feature points, thereby obtaining the compact outline information of the target. (3) Using the content of continuous frame images and eliminating the interference of cross-relationships and the surrounding environment, the features of key target objects are extracted.
[0079] The preferred wide-angle camera also uses CMOS selection and ISP parameter optimization to ensure image clarity under low light conditions. In the identification of similar objects, such as the different heights of large and small charging piles, a special data group is set up to collect samples of similar target objects, and the identification of target objects is quickly completed according to specific scenarios.
[0080] In some embodiments, the previous surrounding image of step S102 is stored in a database. When the previous surrounding image is stored in the database, a vector storage method is adopted. This requires that the surrounding image (bitmap) previously collected by the acquisition device is converted into a vector image and then stored. The purpose of converting the bitmap to the vector image is to facilitate the determination of the edge pixels of the target object and the accurate position information of the target object. The vector image is composed of various vector objects, which is a graphic composed of various vector objects combined using mathematical principles. When the previous surrounding image is stored in the database, the vector storage is also marked with an index, and the index includes a data matrix and a palette matrix, and the data matrix is composed of colored rectangular cells. The value of the rectangular cell is the color value of the palette matrix.
[0081] The edge pixels of the target object are determined by a fuzzy edge detection method. The obstacle edge information described by the surrounding image in the form of a mathematical dot matrix is required to obtain the maximum and minimum radius of the target object, as well as the distance between a point outside the target object and the obstacle center and other parameters when performing AGV robot path planning or operation planning. The embodiment of the present invention uses edge information to find the center of the obstacle. The embodiment of the present invention also obtains the center of gravity of the target object by decomposing obstacles of arbitrary shapes into simple shapes. The target object is decomposed into several regions of simple shapes, and the centers of the regions of simple shapes are known. In the original coordinate system, the geometric center of the target object of arbitrary shape can be obtained by graphic decomposition and coordinate transformation. For example, the target object is an arbitrary fixed building, and the fixed building is a convex surface. The decomposition of the convex surface is composed of multiple arcs and triangles. Since the geometric centers of triangles and arcs are easy to obtain, the geometric center of the fixed building can be obtained by appropriate coordinate transformation.
[0082] After decomposing the target object into several regions with simple shapes, feature points are extracted from the regions. The feature point extraction process is as follows: establish a scale space pyramid, mesh the pyramid of each layer of the image to increase the spatial scale information, and add grid row and column limits; when detecting feature points, use the improved FAST corner point extraction, set the region of interest, and improve the accuracy of feature point extraction; consider the corner point neighborhood gradient when calculating the Harris corner point response; when removing overlapping feature points, use the non-maximum suppression method to suppress the low threshold feature point output of the Harris corner point response.
[0083] See also Figure 8In the improved FAST corner point extraction, an image scale pyramid is constructed, all parameters used in this part are initialized, and each layer of the Gaussian pyramid is gridded. Add grid row restrictions and grid column restrictions. In the FAST corner point extraction process, the image needs to be processed at the boundary and a region of interest is added; the initial threshold is used to extract the feature points in the region of interest, and the method of lowering the threshold and then detecting is adopted for the grid part where the feature points that are difficult to extract are located. At the same time, a minimum threshold is set for the case where the feature points are empty to achieve improved FAST corner point extraction; all extracted feature points are accumulated, and the positions of the feature points extracted in the region of interest are restored to the divided grid area.
[0084] When the feature points are extracted and output, the non-maximum suppression method is used to improve the quality of the feature point output and reduce the clustering and overlapping of feature points in certain areas. The essence of this algorithm is to filter the data according to the set target conditions when there are multiple candidate result values, and select the most suitable experimental results. The top layer of the scale space pyramid established with the image is the first layer, and the feature points ai, ai∈A, A={a1, a2,…, ai}, ai=(xi, yi, εi) that appear on each layer of the pyramid are selected as existing feature points. For candidate feature points bj (not appearing in each layer), bj∈B, B={b1, b2,…, bj}, bj={xj, yj, εj}. The process of removing overlapping feature points is:
[0085] Step 1: When outputting feature points, calculate the distance between a grid area on the pyramid layer where the existing candidate feature point bj in the image is located and the coordinates of the existing feature point ai, and set the critical threshold of the distance to l;
[0086] Step 2: Calculate the distance between the coordinates of the candidate feature point bj = {xj, yj, εj} and the feature point A = {a1, a2, ..., ai}, and select the candidate feature point bj′ = {xj, yj, εj} with a distance ≤ l;
[0087] Step 3: Calculate the Harris corner response threshold εj′ of the candidate feature point bj′={xj,yj,εj} whose distance value does not exceed l in Step 2, and arrange these candidate feature points in descending order according to the intensity of the response threshold of the candidate feature point b={xj,yj,εj}, that is, the value of εj′;
[0088] Step 4: According to the principle of non-maximum suppression algorithm, the candidate feature point bj′={xj, yj, εj} where the Harris corner point response threshold εj′ is at its maximum is retained, and the low-threshold neighboring candidate feature points are eliminated to obtain the neighboring feature points at the best position of the feature point, thereby achieving the purpose of reducing the output of overlapping feature points.
[0089] The embodiment of the present invention also extracts similar objects after feature point extraction, and uses the uniformity of image feature point distribution to find similar objects, mainly to estimate whether the number of feature points is the same in the surrounding images in different areas. If the number of feature points is the same or similar, it means that there are similar objects in the surrounding images; if the number of feature points is much different, it means that there are no similar objects in the surrounding images.
[0090] The uniformity of distribution of image feature points is used to find similar objects. Specifically, first, the surrounding image of the feature point to be detected is divided into regions according to five directions: vertical, horizontal, center and surrounding, upper left to lower right, and lower left to upper right, and 10 directions of image regions are obtained, namely, upper, lower, left, right, lower left, upper right, upper left, lower right, center and surrounding; then, the number of feature points in each region of the image to be detected is counted according to the above division directions, and the data values of the feature points in 10 directions are obtained; then, the average value of the number of feature points in the 10 divided regions of the surrounding image to be detected is calculated; and the variance value of the number of feature points in the surrounding image to be detected in the 10 divided regions is calculated. From the relationship between the obtained variance values of the number of feature points, it can be inferred that the larger the variance value of the number of feature points, the more obvious the floating of the data is, and whether there are similar objects in the surrounding image is indirectly determined.
[0091] Exemplarily, the interval time in step S102 is at least 2 seconds. In the known technology, the previous frame of the surrounding image of the current frame is usually retrieved to determine the motion displacement of the target object. A frame is a single image screen that is the smallest unit in an image animation. We usually say the number of frames. Simply put, it is the number of frames of pictures transmitted in 1 second. The more frames per second (fps), the smoother the displayed action will be. The surrounding images are composed of continuous pictures, and each picture is a frame. The PAL format has 25 frames per second (25 surrounding images are taken per second), and the NTSC format has 30 frames per second (30 surrounding images are taken per second).
[0092] However, in the embodiment of the present invention, calling the previous surrounding images with an interval of at least 2 seconds can be understood as: calling the previous surrounding images of the current motion trajectory of the AGV robot. It can also be understood as: calling the previous surrounding images of the previous motion trajectory when the AGV robot moves to this position. Specifically, the previous surrounding images of the previous motion trajectory when the AGV robot moves to this position are preferentially called, so that the real-time target object of the real-time surrounding image is first identified, and the real-time target object is compared with the previous target object of the previous surrounding image of the previous position. If the real-time target object is consistent with the previous target object, step S103 is skipped, and the motion displacement of the previous motion trajectory is directly used as a reference to continue driving according to the previous motion trajectory. If the real-time target object is inconsistent with the previous target object, the previous surrounding images of the current motion trajectory 2 seconds ago are called, and the real-time target object is compared with the previous target object of this trip to determine the motion displacement of the previous target object of this trip and the real-time target object in the interval time (greater than 2 seconds). If the motion displacement is 0, it is determined that the target object is in a stationary state, and the AGV robot is controlled to bypass the target object and continue to move according to the original planned path. If the motion displacement is a negative value, the original motion state and original motion path of the AGV robot corresponding to the previous surrounding image are retrieved, and the AGV robot is controlled to move in the original motion state and original motion path. If the motion displacement is a positive value, the AGV robot is controlled to stop moving, and the previous frame of the surrounding image of the current position of the AGV robot is continuously obtained. The edge pixels of the target object in the stored image are compared with the edge pixels of the target object in the previous frame image until the target object disappears, and the AGV robot is controlled to continue moving according to the original planned path.
[0093] In an exemplary embodiment, determining the motion displacement of a target object in a previous surrounding image and a surrounding image at an interval time includes:
[0094] (1) Retrieve the previous surrounding image, where the previous surrounding image and the surrounding image have an interval of more than 2 seconds; (2) Select the first target object in the previous surrounding image and the second target object in the surrounding image to obtain the motion displacement of the first target object and the second target object in the interval time, where the first target and the second target are different states of the same target object in the interval time; (3) Identify the first target in the first feature area and the second target in the second feature area, superimpose the first feature area and the second feature area to obtain a superimposed area; detect the position change relationship of the second target relative to the first target in the superimposed area, and judge the motion state of the second target based on the position change relationship, where the motion state is a stationary target or a moving target; (4) Determine the motion state of the target object based on the motion displacement and the motion state; (5) Control the motion state and motion route of the AGV robot based on the motion state.
[0095] The real-time target object of the embodiment of the present invention is matched with the previous target object, specifically as follows:
[0096] After the feature points of the image are extracted using the ORB algorithm, the extracted feature points need to be matched to determine their corresponding positional relationships.
[0097] Assume that the binary feature point descriptor extracted from the two frames of images by the ORB algorithm is expressed as shown in formula (1):
[0098]
[0099] Use formula (2) to perform XOR processing on the feature point descriptors K1 and K2:
[0100]
[0101] The larger the value of the calculated result d(K1, K2), the lower the similarity. The calculated results are sorted according to the size of the values, among which the point with the smallest value, that is, the point with the closest distance, is the best matching point. The matching results of the feature point descriptors calculated by the XOR method are very efficient. In practical applications, brute force matching always completes all matches as much as possible to find the best matching pair. However, there will be some mismatches. Incorrect feature point matching will interfere with the robot's visual navigation, and the obtained motion estimation results will be inaccurate. At this time, RANSAC is needed.
[0102] The random sample consensus algorithm further screens the feature points.
[0103] In some embodiments, see Figure 3 Step S102 can be specifically implemented through the following steps S102a to S102c.
[0104] S102a, processing the real-time surrounding images collected by the AGV robot, and marking the coordinate data of the AGV robot and the position coordinate point of the target object relative to the AGV robot in each frame of the surrounding images according to a known coordinate system.
[0105] S102b, calibrating the target object of the real-time surrounding image according to different coordinate data, and storing the real-time surrounding image in an image database according to the classification of the coordinate data.
[0106] S102c, when the AGV robot moves to the corresponding coordinate data, retrieve the interval frame surrounding image of the AGV robot at the coordinate data from the image database, and the interval time between the interval frame surrounding image and the real-time surrounding image is at least 2 seconds.
[0107] In addition to coordinate data, the AGV visual navigation of the embodiment of the present invention also combines laser technology. The laser is mainly a single-line laser radar. Specifically, the fusion process of laser data obtained by the single-line laser radar and visual information data obtained by Kinect. In the update stage of the map, the Baye estimation method is used. Baye estimation is a statistical data fusion algorithm based on the conditional or posterior probability of Baye's theorem. It is suitable for uncertain information with Gaussian noise. It can estimate the n-dimensional vector in the unknown state through the known vector Z. Assuming that the relevant environmental information vector required to complete the given task is F, and the data information obtained by the sensor is represented by vector d, then the task of data fusion is to estimate the environmental vector F through the derivation of the data vector d. Assuming that P(f, d) is the joint probability distribution density of random vectors f and d, the Bayesian formula is as follows:
[0108] P(f,d)=P(f{ld)·P(d)=P(f|d).P(f)
[0109] Where: P(d) and P(O) represent the marginal distribution density functions of d and f respectively.
[0110] This sensor obtains data from different coordinate systems and uses Bayesian estimation to perform data fusion in an indirect way.
[0111] In some embodiments, see Figure 4 Step S103 can be specifically implemented through the following steps S103a to S103c.
[0112] S103a, before storing the surrounding images, select the storage image target object.
[0113] S103b: After receiving the surrounding images, select the real-time image target object of the surrounding images, and store the image target object and the real-time image target object as the same target object.
[0114] S103c: Compare edge pixels of the stored image target object with edge pixels of the real-time image target object to obtain a motion displacement of the real-time image target object relative to the stored image target object.
[0115] In some embodiments, see Figure 5 Step S103c can be specifically implemented through the following steps S103c1 to S103c7.
[0116] S103c1. Determine in the surrounding image a second feature region corresponding to the first feature region in the previous surrounding image.
[0117] S103c2, retrieve a previous surrounding image, where the previous surrounding image is marked with at least one first feature area.
[0118] S103c3. Down-sample the surrounding image to obtain candidate feature regions, where the number of the candidate feature regions is greater than or equal to the first feature region.
[0119] S103c4. Match a second feature region corresponding to the first feature region in the candidate feature regions according to edge pixels.
[0120] S103c5: The first feature area includes a stored image target object, and the second feature area includes a real-time image target object.
[0121] S103c6, using the camera coordinates of the wide-angle camera as a reference, determining the coordinate points of the stored image target object in the first feature area and the coordinate points of the real-time image target object in the second feature area.
[0122] S103c7. Obtain the motion displacement of the real-time image target object relative to the stored image target object according to the difference between the two coordinate points.
[0123] In an exemplary embodiment, step S104 may specifically determine the motion state and motion route in the following manner.
[0124] If the motion displacement is 0, the target object is determined to be stationary, and the AGV robot is controlled to bypass the target object and continue to move along the original planned path.
[0125] If the motion displacement is a negative value, the original motion state and original motion path of the AGV robot corresponding to the previous surrounding image are retrieved, and the AGV robot is controlled to move according to the original motion state and original motion path.
[0126] If the motion displacement is positive, the AGV robot is controlled to stop moving, and the latest surrounding images of the AGV robot's current position are continuously obtained. The edge pixels of the target object in the stored image are compared with the edge pixels of the target object in the latest image until the target object disappears. The AGV robot is controlled to continue moving along the original planned path.
[0127] See also Figure 6 The present invention discloses an AGV visual navigation system, including an image acquisition module, a navigation chip and a control module, wherein the image acquisition module obtains the surrounding image of the current position of the AGV robot; the navigation chip retrieves the previous surrounding image of the current position, and the previous surrounding image and the surrounding image have an interval time of at least 2 seconds; and determines the motion displacement of the target object in the previous surrounding image and the surrounding image during the interval time; the control module controls the motion state and motion route of the AGV robot according to the motion displacement.
[0128] The present invention discloses an AGV visual navigation system, and the AGV visual navigation method is run on the navigation chip of the AGV visual navigation system. Figure 7 ,AGV visual navigation method, including the following steps:
[0129] S701,A wide-angle camera is installed on the chassis of the AGV robot;
[0130] S702, the wide-angle camera captures the surrounding image of the current position of the AGV robot, where the surrounding image is a wide-angle image within a range of 300° in front of the AGV robot;
[0131] S703, calibrating the target object of the wide-angle image, where the target object includes one or more of a charging pile, other AGV robots, fixed objects, walls, or moving objects;
[0132] S704, processing the real-time surrounding images collected by the AGV robot, and marking the coordinate data of the AGV robot and the position coordinate point of the target object relative to the AGV robot in each frame of the surrounding images according to a known coordinate system;
[0133] S705, calibrating the target object of the real-time surrounding image according to different coordinate data, and storing the real-time surrounding image in an image database according to the classification of the coordinate data;
[0134] S706, when the AGV robot moves to the corresponding coordinate data, retrieve the surrounding images of the AGV robot at the interval frame of the coordinate data from the image database, and the interval time between the surrounding images of the interval frame and the real-time surrounding images is at least 2 seconds;
[0135] S707, selecting a target object of the stored image before storing the surrounding images;
[0136] S708, after receiving the surrounding images, selecting the real-time image target object of the surrounding images, and storing the image target object and the real-time image target object as the same target object;
[0137] S709, comparing edge pixels of the stored image target object with edge pixels of the real-time image target object to obtain a motion displacement of the real-time image target object relative to the stored image target object;
[0138] S710: Control the motion state and motion route of the AGV robot according to the motion displacement.
[0139] The present invention retrieves previous surrounding images at different intervals, compares the previous surrounding images with the real-time surrounding images for target objects, and thereby determines the next motion state of the AGV robot. The present invention has the advantages of short response time and fast motion state determination time. The present invention stores previous surrounding images according to different position classifications, and calibrates the position data of the target objects in the previous surrounding images, which has the advantage of fast retrieval.
[0140] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.
[0141] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
[0142] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. An AGV visual navigation method, characterized in that: The method comprises: Get the surrounding image of the current location of the AGV robot; Retrieving a previous surrounding image of the current location, wherein there is an interval between the previous surrounding image and the surrounding image, and the interval is at least 2 seconds; Determine the motion displacement of the target object in the previous surrounding image and the surrounding image at the interval time; Control the motion state and motion route of the AGV robot according to the motion displacement; A wide-angle camera is installed on the chassis of the AGV robot; The wide-angle camera captures the surrounding image of the current position of the AGV robot, and the surrounding image is a wide-angle image within a range of 300° in front of the AGV robot; Calibrate the target object of the wide-angle image, where the target object includes one or more of a charging pile, other AGV robots, fixed objects, walls, or moving objects; Determining the motion displacement of the target object in the previous surrounding image and the surrounding image at the interval time includes: The previous surrounding images first select the stored image target object before storing; After receiving the surrounding image, selecting the real-time image target object of the surrounding image, the stored image target object and the real-time image target object are the same target object; Comparing edge pixels of the stored image target object with edge pixels of the real-time image target object to obtain a motion displacement of the real-time image target object relative to the stored image target object; The step of comparing edge pixels of the stored image target object with edge pixels of the real-time image target object to obtain a motion displacement of the real-time image target object relative to the stored image target object includes: Determining in the surrounding image a second feature region corresponding to the first feature region in the previous surrounding image; Retrieving the previous surrounding image, wherein the previous surrounding image is marked with at least one first feature area; Downsampling the surrounding image to obtain candidate feature regions, wherein the number of the candidate feature regions is greater than or equal to the first feature regions; Matching a second feature region corresponding to the first feature region in the candidate feature region according to edge pixels; The first feature area contains a stored image target object, and the second feature area contains a real-time image target object; Taking the camera coordinates of the wide-angle camera as a reference, determining the coordinate points of the stored image target object in the first feature area and the coordinate points of the real-time image target object in the second feature area; Obtaining the motion displacement of the real-time image target object relative to the stored image target object according to the difference between the two coordinate points; The retrieving the previous surrounding image of the current location includes: Process the real-time surrounding images collected by the AGV robot, and mark the coordinate data of the AGV robot and the position coordinate points of the target object relative to the AGV robot in each frame of the surrounding image according to the known coordinate system; calibrating the target object of the real-time surrounding image according to different coordinate data, and storing the real-time surrounding image in an image database according to the classification of the coordinate data; When the AGV robot moves to the corresponding coordinate data, the interval frame surrounding image of the AGV robot at the coordinate data is retrieved from the image database, and the interval time between the interval frame surrounding image and the real-time surrounding image is at least 2 seconds.
2. The AGV visual navigation method according to claim 1, characterized in that: Controlling the motion state and motion route of the AGV robot according to the motion displacement includes: If the motion displacement is 0, it is determined that the target object is stationary, and the AGV robot is controlled to bypass the target object and continue to move along the original planned path; If the motion displacement is a negative value, the original motion state and the original motion path of the AGV robot corresponding to the previous surrounding image are retrieved, and the AGV robot is controlled to move according to the original motion state and the original motion path; If the motion displacement is a positive value, the AGV robot is controlled to stop moving, and the latest surrounding images of the AGV robot's current position are continuously acquired. The edge pixels of the target object in the stored image are compared with the edge pixels of the target object in the latest image until the target object disappears. The AGV robot is controlled to continue moving along the original planned path.
3. An AGV visual navigation system, characterized in that: Any one of the AGV visual navigation methods of claims 1-2 is run on a navigation chip, and the system comprises: Image acquisition module, to obtain the surrounding images of the current location of the AGV robot; The navigation chip retrieves a previous surrounding image of the current location, wherein the previous surrounding image and the surrounding image have an interval of at least 2 seconds; and determines a motion displacement of a target object in the previous surrounding image and the surrounding image during the interval; A control module controls the motion state and motion route of the AGV robot according to the motion displacement.
