Intelligent control method and system for obstacle avoidance sudden stop of AMR robot and medium

Abstract

The invention provides an intelligent control method and system for obstacle avoidance sudden stop of an AMR robot and a medium, and the method comprises the steps: determining the position coordinates of an obstacle based on the original image data and point cloud data of the AMR robot in a target scene; the linear distance between the obstacle and the AMR robot is determined on the basis of the position coordinates in combination with real-time positioning information of the AMR robot, and the movement speed and the movement direction of the obstacle are determined on the basis of position time sequence changes of the position coordinates of the obstacle in continuous time; determining the shortest predicted meeting time between the obstacle and the AMR robot based on the linear distance, the movement speed and the movement direction in combination with the driving speed and the driving direction of the AMR robot; and if the shortest predicted meeting time is smaller than the corresponding dynamic safety threshold value, generating a sudden stop control instruction and sending the sudden stop control instruction to a motion control system so as to control the AMR robot to execute sudden stop operation. According to the invention, balance of safety and efficiency of the AMR in a dynamic environment is realized.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an intelligent control method, system and medium for obstacle avoidance and emergency stopping of an AMR robot. Background Technology

[0002] With the rapid development of industrial automation and intelligent warehousing, automated mobile robots (AMRs) are increasingly widely used in logistics handling, workshop material transportation, and other scenarios. In complex and dynamic environments, AMRs need to perceive surrounding obstacles in real time and make obstacle avoidance decisions to ensure operational safety and task efficiency. In existing technologies, grid map obstacle avoidance methods based on LiDAR are widely used. These methods acquire environmental point cloud data using LiDAR, divide the environment into multiple grid cells and mark obstacle information, and then generate obstacle avoidance paths based on preset safety distance thresholds.

[0003] However, existing methods rely solely on preset fixed safety distance thresholds for obstacle avoidance decisions, without considering the obstacle's motion state (such as speed and direction) and the AMR's own motion parameters (such as current speed and turning angle). When facing dynamic obstacles (such as suddenly appearing personnel or moving forklifts), two extreme situations are likely to occur: either the conservative safety threshold leads to premature emergency stops, affecting task execution efficiency; or the lenient safety threshold results in untimely obstacle avoidance, causing collision risks and leading to an imbalance between safety and efficiency. Summary of the Invention

[0004] This invention provides an intelligent control method, system, and medium for obstacle avoidance and emergency stopping of an AMR robot, in order to achieve a balance between safety and efficiency of AMR in dynamic environments.

[0005] In a first aspect, the present invention provides an intelligent control method for obstacle avoidance and emergency stopping of an AMR robot, comprising:

[0006] Based on the original image data and point cloud data of the AMR robot in the target scene, the position coordinates of the obstacles in the target scene are determined.

[0007] Based on the location coordinates and the real-time positioning information of the AMR robot, the straight-line distance between the obstacle and the AMR robot is determined, and the speed and direction of movement of the obstacle are determined based on the temporal changes of the obstacle's location coordinates over continuous time.

[0008] Based on the straight-line distance, the movement speed, and the movement direction, combined with the AMR robot's travel speed and travel direction, the shortest predicted encounter time between the obstacle and the AMR robot is determined.

[0009] If the shortest predicted encounter time is less than the corresponding dynamic safety threshold, an emergency stop control command is generated and sent to the motion control system of the AMR robot to control the AMR robot to perform an emergency stop operation.

[0010] Secondly, the present invention also provides an intelligent control system for obstacle avoidance and emergency stopping of an AMR robot, applied to the intelligent control method for obstacle avoidance and emergency stopping of an AMR robot as described in the first aspect; the intelligent control system for obstacle avoidance and emergency stopping of the AMR robot includes:

[0011] The positioning module is used to determine the position coordinates of obstacles in the target scene based on the original image data and point cloud data of the AMR robot in the target scene;

[0012] The operation monitoring module is used to determine the straight-line distance between the obstacle and the AMR robot based on the position coordinates and the real-time positioning information of the AMR robot, and to determine the speed and direction of the obstacle's movement based on the temporal changes of the obstacle's position coordinates over continuous time.

[0013] The time prediction module is used to determine the shortest predicted encounter time between the obstacle and the AMR robot based on the straight-line distance, the movement speed, and the movement direction, combined with the AMR robot's driving speed and driving direction.

[0014] The obstacle avoidance intelligent control module is used to generate an emergency stop control command and send it to the motion control system of the AMR robot if the shortest predicted encounter time is less than the corresponding dynamic safety threshold, so as to control the AMR robot to perform an emergency stop operation.

[0015] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing computer software programs; and a processor for reading and executing the computer software programs, thereby realizing the intelligent control method for obstacle avoidance and emergency stopping of the AMR robot as described above.

[0016] Fourthly, the present invention also provides a non-transitory computer-readable storage medium storing a computer software program, which, when executed by a processor, implements the intelligent control method for obstacle avoidance and emergency stopping of an AMR robot as described above.

[0017] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the intelligent control method for obstacle avoidance and emergency stopping of an AMR robot as described above.

[0018] The intelligent control method for obstacle avoidance and emergency stopping of an AMR robot provided in this invention determines the straight-line distance based on the obstacle's position coordinates and the AMR's real-time positioning information, clarifies the distance relationship between the obstacle and the AMR robot in the current space, obtains the obstacle's speed and direction of motion based on the temporal changes of the obstacle's position coordinates, captures the movement trend of the dynamic obstacle, and then obtains the shortest predicted encounter time based on the straight-line distance and the motion parameters of both parties, quantifies the risk of future collision between the obstacle and the AMR robot, and finally triggers an emergency stop when the risk exceeds the threshold based on the comparison result of the shortest predicted encounter time and the corresponding dynamic safety threshold. Therefore, by collaboratively integrating environmental perception, dynamic parameter calculation, and risk prediction, the emergency stop decision not only relies on static distance but also combines the movement trends of both parties, solving the problems of premature emergency stopping or untimely obstacle avoidance, and achieving a balance between safety and efficiency of the AMR in a dynamic environment. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the intelligent control method for obstacle avoidance and emergency stop of an AMR robot provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of the intelligent control system for obstacle avoidance and emergency stop of the AMR robot provided in an embodiment of the present invention;

[0021] Figure 3 An embodiment diagram of the electronic device provided in this invention;

[0022] Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0025] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0026] Optional, see below Figure 1 , Figure 1 This is a flowchart illustrating the intelligent control method for obstacle avoidance and emergency stopping of an AMR robot provided by the present invention. In this embodiment of the invention, the executing entity of the intelligent control method for obstacle avoidance and emergency stopping of an AMR robot is an intelligent control system. Therefore, the intelligent control method for obstacle avoidance and emergency stopping of an AMR robot includes:

[0027] Step 10: Based on the original image data and point cloud data of the AMR robot in the target scene, determine the position coordinates of obstacles in the target scene.

[0028] Optionally, the intelligent control system collects raw image data and point cloud data collected by the AMR robot in the target scene. The raw image data is acquired by the visual sensor (such as an RGB camera) on the robot and contains two-dimensional visual information of objects in the scene. The point cloud data is collected by LiDAR or depth camera and presents the spatial structure of the scene in the form of three-dimensional coordinate points.

[0029] Furthermore, the intelligent control system preprocesses the original image data, including image denoising, enhancement, and distortion correction, to improve image quality. Subsequently, object detection algorithms (such as the deep learning-based YOLO algorithm and Faster R-CNN algorithm) are used to analyze the preprocessed image, identifying foreground and background pixels. Foreground pixels are those that may represent obstacles, while background pixels are those representing the environmental background.

[0030] Furthermore, the intelligent control system preprocesses the acquired point cloud data, including point cloud denoising (removing noise points caused by sensor errors or environmental interference), filtering (such as voxel filtering to reduce the number of point clouds and improve processing efficiency), and registration (unifying point cloud data collected at different times to the same coordinate system), to obtain processed point cloud data.

[0031] Furthermore, the intelligent control system performs correlation analysis based on the foreground and background pixels in the image, combined with the processed point cloud data, to obtain the position coordinates of obstacles in the target scene, as described in steps 101 to 104.

[0032] Step 20: Based on the position coordinates and the real-time positioning information of the AMR robot, determine the straight-line distance between the obstacle and the AMR robot, and based on the temporal changes of the obstacle's position coordinates over continuous time, determine the obstacle's speed and direction of movement.

[0033] Furthermore, the intelligent control system has acquired the position coordinates of the obstacle and the real-time positioning information of the AMR robot (from the robot's positioning system). Based on the distance formula between two points, the straight-line distance between the obstacle and the AMR robot is calculated using the obstacle's position coordinates (x1, y1, z1) and the robot's real-time positioning coordinates (x2, y2, z2). The specific distance formula between two points will not be elaborated here.

[0034] Furthermore, the intelligent control system is based on continuous time series (e.g., t1, t2, t3…t…). n Obstacle position coordinates (p1, p2, p3...p) obtained within the range n To analyze this, we first calculate the change in the obstacle's position between two adjacent moments, i.e., Δp = p n -p n-1 The corresponding change in time is Δt = t n -t n-1 The magnitude of the obstacle's velocity, v, is the ratio of the magnitude of the change in position to the change in time, i.e., v = |Δp| / Δt. The direction of motion is determined by the direction vector of the change in position, which is a unit vector of Δp, and can be represented as (Δx / |Δp|, Δy / |Δp|) in a planar scene. To improve accuracy, position data from multiple consecutive moments can be used for fitting calculations; for example, linear fitting can be used to obtain more stable velocity and direction estimates.

[0035] In one embodiment, the known position coordinates of the obstacle (shelf) at time t1 are (5 m, 2 m), and the real-time positioning coordinates of the AMR robot at time t1 are (2 m, 2 m). According to the formula for calculating straight-line distance, the straight-line distance d between the obstacle and the robot is 3 m. Assume that at the subsequent time t2 (t2 = t1 + 1 second), the intelligent control system obtains the position coordinates of the obstacle as (5.5 m, 2 m); and at time t3 (t3 = t2 + 1 second), the position coordinates are (6 m, 2 m). Calculate the position change Δp1 from time t1 to t2 as Δp1 = (5.5 - 5, 2 - 2) = (0.5 m, 0 m), the time change Δt1 = 1 second, the velocity v1 = 0.5 m / s, and the motion direction vector as (0.5 / 0.5, 0 / 0.5) = (1, 0), i.e., along the positive X-axis direction. Calculate the position change Δp2 = (6-5.5, 2-2) = (0.5 m, 0 m) from time t2 to t3, the time change Δt2 = 1 second, the velocity v2 = 0.5 m / s, and the direction vector of motion is also (1, 0). Based on the data from multiple times, determine the obstacle's velocity to be 0.5 m / s, and the direction of motion to be along the positive X-axis.

[0036] Step 30: Based on the straight-line distance, movement speed, and movement direction, combined with the AMR robot's driving speed and driving direction, determine the shortest predicted encounter time between the obstacle and the AMR robot.

[0037] Furthermore, the intelligent control system determines the shortest predicted encounter time between the obstacle and the AMR robot based on the straight-line distance, movement speed, and movement direction, combined with the AMR robot's driving speed and driving direction, as described in steps 301 to 304.

[0038] Step 40: If the shortest predicted encounter time is less than the corresponding dynamic safety threshold, an emergency stop control command is generated and sent to the motion control system of the AMR robot to control the AMR robot to perform an emergency stop operation.

[0039] Furthermore, the intelligent control system compares the shortest predicted encounter time with a preset dynamic safety threshold. The dynamic safety threshold is a pre-set time value based on factors such as the AMR robot's speed, braking performance, obstacle motion characteristics, and scene safety requirements. For example, in a warehouse scenario, the dynamic safety threshold might be set to 2 seconds. If the shortest predicted encounter time is less than the dynamic safety threshold, it indicates a potential collision risk between the obstacle and the robot, and insufficient time for the robot to brake. In this case, the intelligent control system immediately generates an emergency stop control command. This command contains explicit control signals (such as electrical signals, digital instructions, etc.) to instruct the robot's motion control system to execute an emergency stop operation.

[0040] Optionally, the generated emergency stop control command is sent to the AMR robot's motion control system via the robot's internal communication bus (such as CAN bus, Ethernet, etc.). Upon receiving the emergency stop control command, the motion control system triggers the braking mechanism (such as cutting off the power to the drive motor, activating the mechanical brake, etc.) to bring the AMR robot to a rapid stop to avoid collisions with obstacles.

[0041] Continuing with the above embodiment, assume that the dynamic safety threshold for the AMR robot in this scenario is set to 2 seconds. The shortest predicted encounter time calculated in step 30 is 6 seconds. Since 6 seconds is greater than 2 seconds, the intelligent control system does not generate an emergency stop control command, and the robot continues to drive normally.

[0042] In another scenario, if the shortest predicted encounter time between the obstacle and the AMR robot is calculated to be 1.5 seconds, which is less than the dynamic safety threshold of 2 seconds, the intelligent control system determines that a collision risk exists and immediately generates an emergency stop control command: "Immediately cut off the power to the drive motor and activate the mechanical brake." This command is sent to the robot's motion control system via the CAN bus. Upon receiving the command, the motion control system quickly cuts off the power to the drive motor and activates the mechanical brake, causing the AMR robot to stop moving within a short time, thus avoiding a collision.

[0043] This invention determines the straight-line distance based on the obstacle's position coordinates and the AMR's real-time positioning information, clarifying the distance relationship between the obstacle and the AMR robot in the current space. It obtains the obstacle's speed and direction of motion based on the temporal changes in the obstacle's position coordinates, capturing the dynamic movement trend of the obstacle. Then, based on the straight-line distance and the motion parameters of both parties, it obtains the shortest predicted encounter time, quantifying the risk of future collisions between the obstacle and the AMR robot. Finally, based on the comparison between the shortest predicted encounter time and the corresponding dynamic safety threshold, it triggers an emergency stop when the risk exceeds the threshold. Therefore, by collaboratively integrating environmental perception, dynamic parameter calculation, and risk prediction, the emergency stop decision not only relies on static distance but also incorporates the movement trends of both parties, achieving a balance between safety and efficiency for the AMR in a dynamic environment.

[0044] In one embodiment, steps 101 to 104 include:

[0045] Step 101: Determine the pixel feature difference value of each pixel based on the pixel feature value of each pixel in the original image data and the preset background feature benchmark value.

[0046] Optionally, the raw image data is presented in the form of a pixel matrix, where each pixel contains corresponding pixel feature information. The preset background feature baseline value is obtained by analyzing and calculating the background image acquired when there are no obstacles in the target scene, including pixel grayscale baseline value and pixel texture entropy baseline value.

[0047] Therefore, for each pixel in the original image, the intelligent control system extracts its pixel grayscale value and pixel texture entropy value. The pixel grayscale value represents the brightness of the pixel, while the pixel texture entropy value is a feature value obtained by calculating the randomness of the texture distribution in a local region (e.g., a 3×3 neighborhood) surrounding the pixel, used to characterize the texture complexity of the pixel. Further, the intelligent control system calculates the difference between the pixel grayscale value and the pixel grayscale reference value for each pixel, obtaining the grayscale difference value; it also calculates the difference between the pixel texture entropy value and the pixel texture entropy reference value for each pixel, obtaining the texture entropy difference value. Therefore, it can be understood that the grayscale difference value and the texture entropy difference value together constitute the pixel feature difference value for that pixel.

[0048] In one embodiment, assuming a warehouse scene, the original image captured by the AMR robot has a resolution of 640×480, and there is a shelf obstacle in the image. In the preset background feature baseline values, the pixel grayscale baseline value corresponding to a certain pixel (x=100, y=100) is 128, and the pixel texture entropy baseline value is 3.5.

[0049] The intelligent control system extracts the pixel grayscale value of pixel (100, 100) from the original image as 150, and the pixel texture entropy value as 4.2. The grayscale difference value of this pixel is calculated as 150-128=22, and the texture entropy difference value is 4.2-3.5=0.7. Therefore, the pixel feature difference value of this pixel is (22, 0.7).

[0050] Step 102: Determine the foreground and background pixels based on the pixel feature difference values ​​of the pixels.

[0051] Furthermore, a preset pixel feature difference threshold is provided, which includes a grayscale difference threshold and a texture entropy difference threshold. This threshold is set based on the environmental characteristics and obstacle features of the target scene.

[0052] Therefore, for each pixel, the intelligent control system compares its grayscale difference value with a grayscale difference threshold, and simultaneously compares its texture entropy difference value with a texture entropy difference threshold. If both the grayscale difference value and texture entropy difference value of the pixel are greater than the grayscale difference threshold, the pixel is determined to be a foreground pixel, meaning it may belong to an obstacle area; otherwise, it is determined to be a background pixel, meaning it belongs to the scene background area.

[0053] Continuing with the above embodiment, the grayscale difference threshold is 15, and the texture entropy difference threshold is 0.5. For pixel (100, 100), its grayscale difference value is 22, which is greater than the grayscale difference threshold of 15; its texture entropy difference value is 0.7, which is greater than the texture entropy difference threshold of 0.5. Therefore, this pixel is determined to be a foreground pixel. Taking another pixel (200, 200) from the original image, its grayscale difference value is 10, which is less than the grayscale difference threshold of 15; its texture entropy difference value is 0.4, which is less than the texture entropy difference threshold of 0.5. Therefore, this pixel is determined to be a background pixel.

[0054] Step 103: Traverse the foreground pixels in a preset order and determine the first target pixel that satisfies the contour edge point condition. The condition for satisfying the contour edge point condition is: there is at least one background pixel in its eight neighborhoods.

[0055] Furthermore, the intelligent control system traverses all foreground pixels determined in step 102 according to a preset traversal order (such as a row-by-row traversal order from left to right and from top to bottom). During the traversal, for each foreground pixel, the intelligent control system checks the pixel types within its eight neighboring regions, where the eight neighboring regions refer to the eight adjacent pixels around the pixel (pixels in the top, bottom, left, right, and four diagonal directions). If at least one background pixel exists within the eight neighboring regions of the foreground pixel, the foreground pixel is determined to be a contour edge point. When the first foreground pixel that meets the contour edge point condition is encountered, the intelligent control system identifies that pixel as the first target pixel and stops the traversal.

[0056] Continuing with the above embodiment, the intelligent control system traverses the foreground pixels in a left-to-right, top-to-bottom order. When pixel (120, 110) is reached, its eight neighboring pixels are checked. It is found that the pixel above this pixel (120, 109) is a background pixel, satisfying the condition that at least one background pixel exists within its eight neighboring pixels. Therefore, pixel (120, 110) is determined to be a contour edge point and identified as the first target pixel.

[0057] Step 104: Perform correlation analysis based on the first target pixel and point cloud data to determine the position coordinates of obstacles in the target scene.

[0058] Furthermore, the intelligent control system performs correlation analysis based on the first target pixel and point cloud data to determine the position coordinates of obstacles in the target scene, as described in steps 1041 to 1044.

[0059] This invention effectively distinguishes the feature differences between pixel regions that may contain obstacles and background regions by calculating pixel feature difference values. Then, based on the difference values, it accurately divides the foreground and background pixels, initially locates the area where the obstacle is located, and finally combines point cloud data for correlation analysis, thereby accurately locating the position coordinates of the obstacle in the target scene.

[0060] In one embodiment, steps 1041 to 1044 include:

[0061] Step 1041: Starting from the first target pixel, traverse the foreground pixels in its corresponding eight neighborhood according to its corresponding position index to determine the next second target pixel that satisfies the contour edge point and has not been marked as a contour edge point.

[0062] Optionally, the intelligent control system uses the first target pixel determined in step 103 as the starting point and obtains the position index (i.e., pixel coordinates (x, y)) of the pixel in the original image. Following a preset eight-neighbor traversal order (e.g., clockwise or counter-clockwise, for example, starting from the pixel directly above, traversing the pixels to the upper right, right, lower right, bottom, lower left, left, and upper left in sequence), the system checks the pixels within the eight-neighborhood of the first target pixel. During the traversal, only pixels belonging to the foreground are considered, and it is determined whether the foreground pixel meets the condition of a contour edge point (i.e., at least one background pixel exists within its eight-neighborhood). Simultaneously, it checks whether the pixel has not been marked as a contour edge point. When the first pixel that simultaneously meets both conditions is found, the intelligent control system identifies this pixel as the second target pixel and marks it as a contour edge point.

[0063] Continuing with the above embodiment, the first target pixel is (120, 110), which has been marked as a contour edge point. The intelligent control system uses this point as a starting point and traverses the foreground pixels within its eight neighboring areas in a clockwise order. The traversal finds pixel (121, 110) to be a foreground pixel. Checking its eight neighboring areas, the system finds the pixel to its right (122, 110) to be a background pixel, satisfying the contour edge point condition, and this pixel has not been marked as a contour edge point. Therefore, the intelligent control system determines pixel (121, 110) as the second target pixel and marks it as a contour edge point.

[0064] Step 1042: Update the tracking position based on the second target pixel, and traverse the foreground pixels based on the tracking position and the conditions of the contour edge points until the traversal returns to the starting point, thus obtaining the outer contour boundary of the foreground region.

[0065] Furthermore, the intelligent control system updates the position of the second target pixel to the current tracking position, and uses this tracking position as the new starting point. Following the same eight-neighbor traversal order as in step 1041, it traverses the foreground pixels within the eight neighborhoods to find the next pixel that satisfies the contour edge point condition but has not been marked as a contour edge point. After finding this pixel, it is marked as a contour edge point, and the tracking position is updated to the position of this pixel. The above process is repeated, continuously traversing, marking, and updating the tracking position until the tracking position returns to the initial first target pixel (starting point). At this point, all the marked contour edge points are connected in traversal order, and the resulting closed curve is the outer contour boundary of the foreground region.

[0066] Continuing with the embodiment of step 1041, the initial tracking position is the second target pixel (121, 110). The intelligent control system traverses the foreground pixels in its eight neighborhoods in a clockwise order, finds the next contour edge point (121, 111) that meets the conditions, marks the point, and updates the tracking position to (121, 111).

[0067] Continuing the above process, the outline edge points (120, 111), (119, 111), ... are obtained in sequence. The markings are continuously traversed until the tracking position returns to the starting point (120, 110). At this point, all the marked outline edge points are connected in sequence to form the outer outline boundary of the foreground area corresponding to the shelf obstacle. This boundary presents the approximate outline of the shelf.

[0068] Step 1043: Perform point cloud matching in the point cloud data based on the outer contour boundary of the foreground region to obtain a subset of point cloud data.

[0069] Furthermore, the intelligent control system acquires the two-dimensional pixel coordinates of all pixels contained within the outer contour boundary of the foreground region. These pixels constitute the two-dimensional contour range of the obstacle in the image. Based on the camera's intrinsic parameter matrix K and extrinsic parameter matrix [R|t], the two-dimensional pixel coordinates are converted into a set of three-dimensional spatial rays in the camera coordinate system, with each ray corresponding to the spatial pointing direction of a contour pixel. The intelligent control system performs range filtering on the point cloud data collected by the AMR robot, selecting point cloud points located within the spatial cone range formed by all three-dimensional spatial rays. Simultaneously, combined with the approximate depth range of the foreground region in the image (which can be obtained based on prior scene knowledge or statistical analysis of point cloud data), point cloud points that are too far or too close are further filtered out, ultimately obtaining a subset of point cloud data corresponding to the outer contour boundary of the foreground region.

[0070] Continuing with the embodiment of step 1042, the set of pixel coordinates contained in the outer contour boundary of the foreground region is {(120, 110), (121, 110), (121, 111), ..., (120, 110)}. The intelligent control system converts these pixel coordinates into a set of three-dimensional spatial rays in the camera coordinate system based on the camera intrinsic parameter matrix K and extrinsic parameter matrix [R|t]. The point cloud data of the warehouse scene is filtered, and point cloud points located within the spatial cone formed by these rays and between 4.5 meters and 6 meters from the robot are selected to obtain a subset of point cloud data. This subset contains point cloud information of the shelf obstacle surface.

[0071] Step 1044: Perform correlation analysis based on foreground pixels and a subset of point cloud data to determine the position coordinates of obstacles in the target scene.

[0072] Furthermore, the intelligent control system performs correlation analysis based on the foreground pixels and a subset of point cloud data to determine the position coordinates of obstacles in the target scene, as described in steps 10441 to 10444.

[0073] This invention, through continuous traversal tracking, accurately delineates the outer contour boundary of obstacles in the image, filters out a subset of point cloud data based on the contour boundary to reduce interference from irrelevant point clouds, and finally fuses the two-dimensional image contour information with the three-dimensional point cloud data through coordinate association to obtain the accurate position coordinates of the obstacle.

[0074] In one embodiment, the process of steps 10441 to 10444 includes:

[0075] Step 10441 involves performing a correlation analysis based on the color features of each foreground pixel and the reflection intensity features of each point cloud in the point cloud data subset to determine the feature matching result between each foreground pixel and each point cloud. The reflection intensity feature characterizes the ability of the point cloud to reflect signals emitted by the acquisition device.

[0076] Optionally, the intelligent control system extracts the color features of each foreground pixel within the outer contour boundary of the foreground region. The color features can be represented by the pixel values ​​(R, G, B) of the red, green, and blue channels in the RGB color space, with each channel ranging from 0 to 255. Simultaneously, it acquires the reflection intensity features of each point cloud in the point cloud data subset obtained in step 1043. The reflection intensity feature is a quantitative indicator of the point cloud's ability to reflect signals emitted by acquisition devices such as lidar, and is typically expressed as a relative intensity value. The value range varies depending on the device model, for example, from 0 to 255.

[0077] Furthermore, the intelligent control system establishes a feature matching model to calculate the matching degree between the color features of each foreground pixel and the reflection intensity features of each point cloud. For the correlation analysis between color features and reflection intensity features, this embodiment of the invention employs a feature similarity measurement method, for example, calculating the cosine similarity between the color feature vector and the reflection intensity feature vector.

[0078] Therefore, when the similarity value is greater than the preset feature matching threshold, it is determined that the feature matching between the foreground pixel and the corresponding point cloud is successful; otherwise, it is determined that the feature matching between the foreground pixel and the corresponding point cloud fails, and finally the feature matching result between each foreground pixel and each point cloud is obtained.

[0079] Continuing with the above embodiment, the foreground pixels include (120, 110), (121, 110), etc., and the point cloud in the point cloud data subset includes point cloud 1 (5.1m, 2m, 0m), point cloud 2 (5.2m, 2.1m, 0m), etc. The intelligent control system extracts the color feature of the foreground pixel (120, 110) as (100, 100, 100), the reflection intensity feature of point cloud 1 is 80, and the reflection intensity feature of point cloud 2 is 85. The preset feature matching threshold is 0.7. The cosine similarity between the color feature of the foreground pixel and the reflection intensity feature of point cloud 1 is calculated to be 0.85, which is greater than the matching threshold, so the match is determined to be successful; the cosine similarity with the reflection intensity feature of point cloud 2 is 0.65, which is less than the matching threshold, so the match is determined to be unsuccessful. Similarly, the matching analysis of other foreground pixels and point clouds is completed to obtain the feature matching results.

[0080] Step 10442: Based on the feature matching results, determine the point clouds to be clustered in the subset of point cloud data, and select a preset number of first point clouds as the initial cluster centers.

[0081] Furthermore, based on the feature matching results obtained in step 10441, the intelligent control system filters out point clouds that successfully match at least one foreground pixel, and determines the point clouds to be clustered in the subset of point cloud data.

[0082] Furthermore, the intelligent control system sets a preset number of initial cluster centers based on the number of obstacles or preset number of clusters that may exist in the target scene. For example, the preset number is 1 (indicating that there may be a major obstacle in the scene). The system randomly selects a preset number of point clouds from the point cloud to be clustered or selects them as the first point cloud according to the principle of uniform distribution. The three-dimensional coordinates of the first point cloud are then determined as the initial cluster centers.

[0083] Continuing with the above embodiment 10441, the feature matching results show that point cloud 1, point cloud 2, point cloud 3, and point cloud 4 in the point cloud data subset all successfully match at least one foreground pixel. Therefore, these point clouds are identified as point clouds to be clustered. The preset initial cluster center number is 1. The intelligent control system selects point cloud 2 (5.2m, 2.1m, 0m) from the point clouds to be clustered as the first point cloud, and determines its three-dimensional coordinates (5.2m, 2.1m, 0m) as the initial cluster center.

[0084] Step 10443: Cluster each second point cloud based on the clustering distance between each second point cloud and each initial cluster center to obtain point cloud clusters. Update the cluster center based on the point cloud coordinates of each point cloud in each point cloud cluster to obtain the target cluster center of each point cloud cluster. Continue until the offset between the target cluster center and the initial cluster center of each point cloud cluster is less than or equal to a preset offset threshold to obtain multiple obstacle point cloud clusters.

[0085] Furthermore, the intelligent control system defines the point clouds other than the initial cluster center in the point cloud to be clustered as the second point cloud. Therefore, the clustering distance is calculated by using the Euclidean distance algorithm combined with the three-dimensional coordinates of each second point cloud and the three-dimensional coordinates of each initial cluster center. The Euclidean distance used for clustering will not be elaborated further.

[0086] Furthermore, the intelligent control system assigns each second point cloud to the cluster containing the nearest initial cluster center based on the clustering distance, forming point cloud clusters. Further, the intelligent control system calculates the geometric center of the 3D coordinates of all point clouds in each point cloud cluster and updates this geometric center as the new target cluster center for that cluster. This process of calculating the clustering distance between each second point cloud and the new target cluster center, reallocating point clouds to clusters, and updating the target cluster center is repeated until the offset (i.e., the Euclidean distance) between the target cluster center of each point cloud cluster and the initial cluster center of the previous iteration is less than or equal to a preset offset threshold (e.g., 0.1 meters). The resulting stable point cloud clusters are then considered multiple obstacle point cloud clusters.

[0087] Continuing with the embodiment of step 10442, the second point clouds in the point clouds to be clustered are point cloud 1 (5.1m, 2m, 0m), point cloud 3 (5.3m, 2m, 0m), and point cloud 4 (5.2m, 1.9m, 0m), with the initial cluster center being (5.2m, 2.1m, 0m). The clustering distance between each second point cloud and the initial cluster center is calculated: the distance for point cloud 1 is 0.14m; the distance for point cloud 3 is 0.14m; and the distance for point cloud 4 is 0.2m. All second point clouds are assigned to the cluster containing the initial cluster center, forming point cloud clusters.

[0088] Furthermore, the intelligent control system calculates the target cluster center: the average X-coordinate is (5.1+5.2+5.3+5.2) / 4 = 5.2 meters, the average Y-coordinate is (2+2.1+2+1.9) / 4 = 2 meters, and the average Z-coordinate is 0 meters, meaning the target cluster center is (5.2 meters, 2 meters, 0 meters). The offset between the target cluster center and the initial cluster center is calculated to be 0.1 meters. The preset offset threshold is 0.1 meters, satisfying the condition that the offset is less than or equal to the threshold, thus obtaining an obstacle point cloud cluster.

[0089] Step 10444: Based on the minimum bounding cube boundary of each obstacle point cloud cluster, determine the range of three-dimensional boundary coordinates, and based on the mean of the geometric center coordinates of the range of three-dimensional boundary coordinates of each obstacle point cloud cluster, determine the position coordinates of the obstacle.

[0090] Furthermore, for each obstacle point cloud cluster, the intelligent control system calculates its minimum bounding cube boundary, that is, finds the minimum and maximum values ​​of all point clouds in the cluster in the three directions of X-axis, Y-axis and Z-axis. The minimum and maximum values ​​of X-axis, Y-axis and Z-axis constitute the three-dimensional boundary coordinate range of the cluster.

[0091] For each obstacle point cloud cluster, the intelligent control system calculates the geometric center coordinates of its three-dimensional boundary coordinate range using the formulas: X = (minimum value of X-axis + maximum value of X-axis) / 2, Y = (minimum value of Y-axis + maximum value of Y-axis) / 2, and Z = (minimum value of Z-axis and maximum value of Z-axis) / 2. If multiple obstacle point cloud clusters exist, the intelligent control system calculates the average of the geometric center coordinates of all clusters and uses this average as the position coordinates of the obstacles in the target scene.

[0092] Continuing with the embodiment of step 10443, the obstacle point cloud cluster includes point cloud 1, point cloud 2, point cloud 3, and point cloud 4. Calculate the three-dimensional coordinate range of its smallest bounding cube boundary: the minimum value of the X-axis is 5.1 meters, and the maximum value of the X-axis is 5.3 meters; the minimum value of the Y-axis is 1.9 meters, and the maximum value of the Y-axis is 2.1 meters; the minimum value of the Z-axis is 0 meters, and the maximum value of the Z-axis is 0 meters. Calculate the geometric center coordinates of the three-dimensional boundary coordinate range of this cluster: X = (5.1 + 5.3) / 2 = 5.2 meters, Y = (1.9 + 2.1) / 2 = 2 meters, Z = (0 + 0) / 2 = 0 meters. Since there is only one obstacle point cloud cluster, its geometric center coordinates are the position coordinates of the obstacle. Therefore, the position coordinates of the obstacle in the target scene are determined to be (5.2 meters, 2 meters, 0 meters).

[0093] The embodiments of the present invention effectively improve the robustness and accuracy of obstacle location determination through multi-feature association and cluster analysis. Even in complex scenes with point cloud noise or partial occlusion, it can reliably output the spatial location information of obstacles, providing a decision basis for obstacle avoidance control of AMR robots.

[0094] In one embodiment, steps 301 to 304 include:

[0095] Step 301: Determine the relative motion direction angle between the obstacle and the AMR robot based on the directional difference between the obstacle's motion direction and the AMR robot's travel direction.

[0096] Optionally, the intelligent control system determines the relative motion direction angle between the obstacle and the AMR robot based on the directional difference between their motion direction and the AMR robot's travel direction. The relative motion direction angle is the angle between the obstacle and the positive X-axis of the coordinate system, and its value ranges from 0 to 180 degrees (or 0 to π radians), i.e., the relative motion direction angle θ. rel = Obstacle movement direction angle θ obs -AMR robot driving direction angle θ rob .

[0097] Continuing with the above embodiment, the direction of the obstacle's movement is known to be (1, 0), and the angle between the direction of movement and the positive X-axis (direction angle θ) is... obs The angle θ is 0 degrees. The AMR robot's direction of travel is also (1, 0), and its direction of travel angle θ is 0 degrees. rob The angle is 0 degrees. Therefore, the intelligent control system calculates the relative motion direction angle θ. rel =0 degrees - 0 degrees = 0 degrees, meaning the obstacle and the AMR robot are moving in the same direction, with a relative motion angle of 0 degrees.

[0098] Step 302: Based on the obstacle's movement speed and relative motion direction angle, determine the first velocity component of the obstacle in the relative motion direction, and based on the AMR robot's travel speed and relative motion direction angle, determine the second velocity component of the AMR robot in the relative motion direction.

[0099] Furthermore, the intelligent control system is based on the relative motion direction angle θ obtained in step 301. rel The velocity of the obstacle and the speed of the AMR robot are decomposed to obtain the velocity components in the relative motion direction.

[0100] Therefore, for an obstacle, its first velocity component v obs-rel The magnitude of the obstacle's speed, v obs With relative motion direction angle θ relThe product of the cosine values ​​is calculated using the formula v. obs-rel =v obs ×cos(θ rel The first velocity component represents the magnitude of the obstacle's velocity in the direction of relative motion.

[0101] For an AMR robot, its second velocity component v rob-rel The speed v of the AMR robot rob With relative motion direction angle θ rel The product of the cosine values ​​is calculated using the formula v. rob-rel =v rob ×cos(θ rel The second velocity component characterizes the magnitude of the AMR robot's velocity in the direction of relative motion.

[0102] Continuing with the above embodiment, the magnitude of the obstacle's moving speed v obs =0.5 m / s, the magnitude of the AMR robot's speed v rob = 1 m / s, relative motion direction angle θ rel = 0 degrees. Therefore, the first velocity component v obs-rel =0.5×cos(0)=0.5 m / s, second velocity component v rob-rel =1×cos(0)=1 m / s.

[0103] Step 303: Based on the relative motion direction angle, the first velocity component, and the second velocity component, determine the relative approach velocity between the obstacle and the AMR robot. The relative approach velocity characterizes the relative approach speed between the obstacle and the AMR robot in the relative motion direction.

[0104] Furthermore, the intelligent control system determines the motion direction relationship between the obstacle and the AMR robot based on the relative motion direction angle, and determines the relative approach speed between the obstacle and the AMR robot based on the motion direction relationship between the obstacle and the AMR robot, combined with the first velocity component and the second velocity component, as in steps 3031 to 3033. The relative approach speed represents the relative approach speed between the obstacle and the AMR robot in the relative motion direction.

[0105] Step 304: Based on the relative approach velocity and straight-line distance, determine the shortest predicted encounter time between the obstacle and the AMR robot.

[0106] Furthermore, the intelligent control system determines the shortest predicted encounter time between the obstacle and the AMR robot based on the relative approach speed and straight-line distance, as described in steps 3041 to 3043.

[0107] The embodiments of the present invention fully consider the differences in motion direction and speed between obstacles and AMR robots, and can accurately predict potential encounter times. Even in complex relative motion scenarios (such as same-direction, opposite-direction, and cross-movement), it can reliably output the shortest predicted encounter time, achieving a balance between safety and efficiency of AMRs in dynamic environments, and effectively improving the operational safety of robots in dynamic environments.

[0108] In one embodiment, steps 3031 to 3033 include:

[0109] Step 3031: If the relative motion direction angle is greater than or equal to the first motion direction angle threshold and less than the second motion direction angle threshold, then the motion direction relationship between the obstacle and the AMR robot is determined to be equilateral approaching motion. If the relative motion direction angle is greater than or equal to the second motion direction angle threshold and less than or equal to the third motion direction angle threshold, then the motion direction relationship between the obstacle and the AMR robot is determined to be slightly equilateral approaching motion.

[0110] Optionally, three motion direction angle thresholds are preset: a first motion direction angle threshold, a second motion direction angle threshold, and a third motion direction angle threshold. The motion direction angle thresholds are set based on the relationship between the motion direction of obstacles and the AMR robot in the scene. For example, the first motion direction angle threshold is 0 degrees, the second motion direction angle threshold is 30 degrees, and the third motion direction angle threshold is 180 degrees (the threshold unit is degrees, and the value range is between 0 and 180 degrees, and the first motion direction angle threshold is less than the second motion direction angle threshold, and the second motion direction angle threshold is less than the third motion direction angle threshold).

[0111] Furthermore, the intelligent control system will control the relative motion direction angle θ rel It is compared with three preset motion direction angle thresholds. If the relative motion direction angle θ rel If the relative motion direction angle θ is greater than or equal to the first motion direction angle threshold and less than the second motion direction angle threshold, the intelligent control system determines that the motion direction relationship between the obstacle and the AMR robot is equal-direction approaching motion, that is, the two motion directions are basically the same and there is a tendency to approach each other in the same direction; if the relative motion direction angle θ rel If the motion direction angle is greater than or equal to the second motion direction angle threshold and less than or equal to the third motion direction angle threshold, the intelligent control system determines that the motion direction relationship between the two is a slightly similar approaching motion, that is, there is a certain deviation between the motion directions of the two, but they are still approximately similar approaching motions.

[0112] Continuing with the above embodiment, the first motion direction angle threshold is 0 degrees, the second motion direction angle threshold is 30 degrees, and the third motion direction angle threshold is 180 degrees. In the first scenario, the relative motion direction angle θ rel=0 degrees, this angle is greater than or equal to the first motion direction angle threshold of 0 degrees and less than the second motion direction angle threshold of 30 degrees. Therefore, the intelligent control system determines that the motion direction relationship between the obstacle and the AMR robot is equal-direction approaching motion. In the second scenario, if the relative motion direction angle θ rel = 45 degrees. This angle is greater than or equal to the second motion direction angle threshold of 30 degrees and less than or equal to the third motion direction angle threshold of 180 degrees. Therefore, the intelligent control system determines that the motion direction relationship between the two is a slightly similar approach motion.

[0113] Step 3032: If the direction of motion is equal approaching motion, then the sum of the velocities between the first velocity component and the second velocity component is determined as the relative approaching velocity.

[0114] Furthermore, when the direction of motion is determined to be an equilateral approaching motion, the intelligent control system calculates the first velocity component v. obs-rel With the second velocity component v rob-rel The sum of the velocities between the obstacle and the AMR robot is used to determine the relative approach velocity v between them. rel Therefore, v rel =v rob-rel +v obs-rel In equilateral approaching motion, the sum of velocities reflects the rate at which the two objects approach each other while moving in the same direction.

[0115] Continuing from the first scene of step 3031, the motion directions are equal and approaching each other. The first velocity component v is known. obs-rel = 0.5 m / s, second velocity component v rob-rel = 1 m / s, the sum of the calculated velocities = 1 m / s + 0.5 m / s = 1.5 m / s, therefore, the relative approaching velocity v rel = 1.5 m / s.

[0116] Step 3033: If the direction of motion is a slightly similar approaching motion, then the absolute value of the velocity difference between the first velocity component and the second velocity component is determined as the relative approaching velocity.

[0117] Furthermore, when the direction of motion is determined to be a slightly similar approaching motion, the intelligent control system calculates the first velocity component v. obs-rel With the second velocity component v rob-rel The speed difference between them, i.e., Δv = v rob-rel -v obs-rel Then, the absolute value of the velocity difference is taken and determined as the relative approach velocity v between the obstacle and the AMR robot. rel Therefore, the relative approach velocity v rel It can be represented as |v rob-rel -v obs-relIn a near-uniform approaching motion, since there is a certain deviation in the direction of motion of the two objects, their relative approach speed is reflected by the absolute value of the difference between their velocity components in the relative direction of motion.

[0118] Continuing with the second scenario in step 3031, the motion direction relationship is a slightly similar approaching motion. Assume the first velocity component v obs-rel = 0.6 m / s, second velocity component v rob-rel =0.8 m / s, calculate Δv = 0.8 m / s - 0.6 m / s = 0.2 m / s, take the absolute value, therefore, the relative approach velocity v rel = 0.2 m / s.

[0119] The embodiments of the present invention fully consider the influence of directional deviation on the relative approach rate in unidirectional motion, and can adaptively select and calculate the relative approach speed according to the actual motion direction relationship. Even in near unidirectional motion scenarios with a certain directional deviation, the relative approach speed between the obstacle and the AMR robot can be reliably quantified, providing accurate speed parameters for the calculation of the shortest predicted encounter time and improving the safety of the AMR robot's dynamic obstacle avoidance decision.

[0120] In one embodiment, steps 3041 to 3043 include:

[0121] Step 3041: Determine the correction coefficient for the velocity direction's effect on distance change based on the relative motion direction angle, and calculate the effective distance by multiplying the straight-line distance and the correction coefficient. The correction coefficient is used to correct the deviation between the actual distance change and the straight-line distance change caused by the motion direction angle.

[0122] Optionally, the intelligent control system is based on the relative motion direction angle θ rel The correction coefficient k for the velocity direction's change in distance is determined using a preset correction coefficient function. This correction coefficient function is pre-set based on the relationship between the deviation between the motion direction angle and the actual distance change. For example, a cosine function is used to establish the mapping relationship between the correction coefficient and the relative motion direction angle, as the cosine function effectively reflects the influence of the angle on the projected distance. The calculation formula is k = cos(θ). rel ), where k ranges from -1 to 1. When the relative motion direction angle is 0 degrees, the correction coefficient is 1, indicating that the actual distance change is not deviated from the straight-line distance change; when the angle increases, the correction coefficient decreases, reflecting the distance change deviation caused by the direction deviation.

[0123] Furthermore, the intelligent control system calculates the effective distance d by multiplying the straight-line distance d with the correction coefficient. eff Therefore, the specific calculation formula is d. eff=d×k. The effective distance corrects the discrepancy between the actual distance change and the straight-line distance change caused by deviations in the direction angle of motion, and more closely reflects the true distance change during the motion process.

[0124] Continuing with the above embodiments, in a scenario of equilateral approaching motion, the relative motion direction angle θ rel =0 degrees, straight-line distance d = 3 meters, according to the correction factor formula, the correction factor k = cos(0) = 1, therefore, the effective distance d is calculated. eff =3 × 1 = 3 meters. In a scenario of slightly similar approaching motion, the relative motion direction angle θ rel The straight-line distance d = 4 meters. The correction factor k = cos(45) ≈ 0.707, therefore the effective distance d eff =4 × 0.707 ≈ 2.83 meters.

[0125] Step 3042: If the relative approaching speed is greater than a preset speed threshold, then it is determined that the obstacle and the AMR robot are moving closer to each other. If the relative approaching speed is less than or equal to the preset speed threshold, then it is determined that the obstacle and the AMR robot are moving away from each other.

[0126] Furthermore, a preset speed threshold is established, based on the motion characteristics of obstacles and the AMR robot in the scene, to distinguish between moving closer and moving further apart. For example, the preset speed threshold is 0.1 m / s. Therefore, the intelligent control system will adjust the relative approach speed v. rel Compare with a preset speed threshold. If the relative approach speed v rel If the speed exceeds a preset threshold, the intelligent control system determines that the obstacle and the AMR robot are moving closer to each other, meaning the distance between them is constantly decreasing; if the relative approaching speed v rel If the speed is less than or equal to the preset speed threshold, the intelligent control system determines that the two are moving away from each other, that is, the distance between them is constantly increasing or remaining stable.

[0127] Continuing with the same-direction approaching motion scenario in step 3041, the relative approaching velocity v rel = 1.5 m / s, with a preset speed threshold of 0.1 m / s. Since 1.5 m / s is greater than 0.1 m / s, it is determined that the obstacle and the AMR robot are moving closer to each other. In another scenario, if the relative approaching speed v rel =0.5 m / s, which is less than the preset speed threshold of 0.1 m / s, therefore it is determined that the two are moving away from each other.

[0128] Step 3043: If the objects are moving closer to each other, the relative approach speed is determined as the effective relative speed, and the shortest predicted encounter time between the obstacle and the AMR robot is determined based on the ratio between the effective distance and the effective relative speed.

[0129] Furthermore, when it is determined that the obstacle and the AMR robot are moving closer to each other, the intelligent control system will adjust the relative approach speed v. rel Determined as effective relative velocity v efl The ratio between the effective distance and the effective relative velocity is calculated, and this ratio is determined as the shortest predicted encounter time t between the obstacle and the AMR robot. meet The calculation formula is t meet =d eff / v efl This time value reflects the shortest time required for the two objects to meet from their current positions under the current trend of motion.

[0130] Continuing with step 3042, the equivalent direction approach motion scene, effective distance d eff = 3 meters, effective relative speed v efl =1.5 m / s, calculate the shortest predicted meeting time t meet = 2 seconds, meaning the obstacle and the AMR robot are expected to meet in 2 seconds. In a micro-unidirectional approach motion scenario, if the effective distance d... eff = 2.83 meters, effective relative speed v efl =0.2 m / s, then the shortest predicted meeting time t meet ≈14.15 seconds.

[0131] The embodiments of the present invention fully consider the influence of motion direction deviation and velocity characteristics on the meeting time, and can reliably predict the shortest meeting time in complex motion scenarios. This provides time parameters for the dynamic obstacle avoidance decision of AMR robots, enabling the robots to take corresponding obstacle avoidance measures in advance based on the meeting time, effectively improving the operational safety and reliability of the robots in dynamic environments.

[0132] Furthermore, the intelligent control system for obstacle avoidance and emergency stop of the AMR robot provided by the present invention will be described below. The intelligent control system for obstacle avoidance and emergency stop of the AMR robot described below can be referred to in correspondence with the intelligent control method for obstacle avoidance and emergency stop of the AMR robot described above.

[0133] Optional, refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of the intelligent control system for obstacle avoidance and emergency stop of the AMR robot provided by the present invention. The intelligent control system for obstacle avoidance and emergency stop of the AMR robot includes...

[0134] The positioning module 210 is used to determine the position coordinates of obstacles in the target scene based on the original image data and point cloud data of the AMR robot in the target scene;

[0135] The operation monitoring module 220 is used to determine the straight-line distance between the obstacle and the AMR robot based on the position coordinates and the real-time positioning information of the AMR robot, and to determine the speed and direction of the obstacle based on the temporal changes of the position coordinates of the obstacle over continuous time.

[0136] The time prediction module 230 is used to determine the shortest predicted encounter time between the obstacle and the AMR robot based on the straight-line distance, movement speed and movement direction, combined with the AMR robot's driving speed and driving direction.

[0137] The obstacle avoidance intelligent control module 240 is used to generate an emergency stop control command and send it to the motion control system of the AMR robot if the shortest predicted encounter time is less than the corresponding dynamic safety threshold, so as to control the AMR robot to perform an emergency stop operation.

[0138] This invention determines the straight-line distance based on the obstacle's position coordinates and the AMR's real-time positioning information, clarifying the distance relationship between the obstacle and the AMR robot in the current space. It obtains the obstacle's speed and direction of motion based on the temporal changes in the obstacle's position coordinates, capturing the dynamic movement trend of the obstacle. Then, based on the straight-line distance and the motion parameters of both parties, it obtains the shortest predicted encounter time, quantifying the risk of future collisions between the obstacle and the AMR robot. Finally, based on the comparison between the shortest predicted encounter time and the corresponding dynamic safety threshold, it triggers an emergency stop when the risk exceeds the threshold. Therefore, by collaboratively integrating environmental perception, dynamic parameter calculation, and risk prediction, the emergency stop decision not only relies on static distance but also incorporates the movement trends of both parties, achieving a balance between safety and efficiency for the AMR in a dynamic environment.

[0139] Please see Figure 3 , Figure 3 An embodiment diagram of an electronic device provided in accordance with the present invention. For example... Figure 3 As shown, this embodiment of the invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it performs the following steps:

[0140] Based on the original image data and point cloud data of the AMR robot in the target scene, the position coordinates of obstacles in the target scene are determined.

[0141] Based on the location coordinates combined with the real-time positioning information of the AMR robot, the straight-line distance between the obstacle and the AMR robot is determined, and the speed and direction of movement of the obstacle are determined based on the temporal changes of the obstacle's location coordinates over continuous time.

[0142] Based on straight-line distance, speed, and direction of movement, combined with the AMR robot's speed and direction of movement, the shortest predicted encounter time between the obstacle and the AMR robot is determined.

[0143] If the shortest predicted encounter time is less than the corresponding dynamic safety threshold, an emergency stop control command is generated and sent to the motion control system of the AMR robot to control the AMR robot to perform an emergency stop operation.

[0144] Please see Figure 4 , Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with an embodiment of the present invention is shown. Figure 4 As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 311 is stored. When the computer program 311 is executed by a processor, it performs the following steps:

[0145] Based on the original image data and point cloud data of the AMR robot in the target scene, the position coordinates of obstacles in the target scene are determined.

[0146] Based on the location coordinates combined with the real-time positioning information of the AMR robot, the straight-line distance between the obstacle and the AMR robot is determined, and the speed and direction of movement of the obstacle are determined based on the temporal changes of the obstacle's location coordinates over continuous time.

[0147] Based on straight-line distance, speed, and direction of movement, combined with the AMR robot's speed and direction of movement, the shortest predicted encounter time between the obstacle and the AMR robot is determined.

[0148] If the shortest predicted encounter time is less than the corresponding dynamic safety threshold, an emergency stop control command is generated and sent to the motion control system of the AMR robot to control the AMR robot to perform an emergency stop operation.

[0149] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the intelligent control method for obstacle avoidance and emergency stop of the AMR robot provided by the above methods. The method includes:

[0150] Based on the original image data and point cloud data of the AMR robot in the target scene, the position coordinates of obstacles in the target scene are determined.

[0151] Based on the location coordinates combined with the real-time positioning information of the AMR robot, the straight-line distance between the obstacle and the AMR robot is determined, and the speed and direction of movement of the obstacle are determined based on the temporal changes of the obstacle's location coordinates over continuous time.

[0152] Based on straight-line distance, speed, and direction of movement, combined with the AMR robot's speed and direction of movement, the shortest predicted encounter time between the obstacle and the AMR robot is determined.

[0153] If the shortest predicted encounter time is less than the corresponding dynamic safety threshold, an emergency stop control command is generated and sent to the motion control system of the AMR robot to control the AMR robot to perform an emergency stop operation.

[0154] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent control method for obstacle avoidance and emergency stopping of an AMR robot, characterized in that, include: Based on the original image data and point cloud data of the AMR robot in the target scene, the position coordinates of the obstacles in the target scene are determined. Based on the location coordinates and the real-time positioning information of the AMR robot, the straight-line distance between the obstacle and the AMR robot is determined, and the speed and direction of movement of the obstacle are determined based on the temporal changes of the obstacle's location coordinates over continuous time. Based on the straight-line distance, the movement speed, and the movement direction, combined with the AMR robot's travel speed and travel direction, the shortest predicted encounter time between the obstacle and the AMR robot is determined. If the shortest predicted encounter time is less than the corresponding dynamic safety threshold, an emergency stop control command is generated and sent to the motion control system of the AMR robot to control the AMR robot to perform an emergency stop operation.

2. The intelligent control method for obstacle avoidance and emergency stop of an AMR robot according to claim 1, characterized in that, The determination of the shortest predicted encounter time between the obstacle and the AMR robot based on the straight-line distance, the movement speed, and the movement direction, combined with the AMR robot's travel speed and travel direction, includes: The relative motion direction angle between the obstacle and the AMR robot is determined based on the directional difference between the obstacle's motion direction and the AMR robot's travel direction. Based on the obstacle's speed and the relative motion direction angle, a first velocity component of the obstacle in the relative motion direction is determined, and based on the AMR robot's speed and the relative motion direction angle, a second velocity component of the AMR robot in the relative motion direction is determined. Based on the relative motion direction angle, the first velocity component, and the second velocity component, the relative approach speed between the obstacle and the AMR robot is determined; the relative approach speed characterizes the relative approach speed between the obstacle and the AMR robot in the relative motion direction. Based on the relative approach velocity and the straight-line distance, the shortest predicted encounter time between the obstacle and the AMR robot is determined.

3. The intelligent control method for obstacle avoidance and emergency stop of an AMR robot according to claim 2, characterized in that, The determination of the shortest predicted encounter time between the obstacle and the AMR robot based on the relative approach velocity and the straight-line distance includes: The correction coefficient for the change in distance due to the velocity direction is determined based on the relative motion direction angle, and the effective distance is obtained by multiplying the straight-line distance and the correction coefficient; the correction coefficient is used to correct the deviation between the actual distance change and the straight-line distance change caused by the motion direction angle. If the relative approach speed is greater than a preset speed threshold, it is determined that the obstacle and the AMR robot are moving closer to each other; if the relative approach speed is less than or equal to the preset speed threshold, it is determined that the obstacle and the AMR robot are moving further away from each other. If the objects are moving closer to each other, the relative approach speed is determined as the effective relative speed, and the shortest predicted encounter time between the obstacle and the AMR robot is determined based on the ratio between the effective distance and the effective relative speed.

4. The intelligent control method for obstacle avoidance and emergency stop of an AMR robot according to claim 2, characterized in that, Determining the relative approach speed between the obstacle and the AMR robot based on the relative motion direction angle, the first velocity component, and the second velocity component includes: If the relative motion direction angle is greater than or equal to the first motion direction angle threshold and less than the second motion direction angle threshold, the motion direction relationship between the obstacle and the AMR robot is determined to be equilateral approaching motion; if the relative motion direction angle is greater than or equal to the second motion direction angle threshold and less than or equal to the third motion direction angle threshold, the motion direction relationship between the obstacle and the AMR robot is determined to be slightly equilateral approaching motion. If the direction of motion is equal and approaching, then the sum of the velocities between the first velocity component and the second velocity component is determined as the relative approaching velocity. If the direction of motion is a slightly similar approaching motion, then the absolute value of the velocity difference between the first velocity component and the second velocity component is determined as the relative approaching velocity.

5. The intelligent control method for obstacle avoidance and emergency stop of an AMR robot according to any one of claims 1 to 4, characterized in that, The determination of the position coordinates of obstacles in the target scene based on the original image data and point cloud data of the AMR robot in the target scene includes: Based on the pixel feature value of each pixel in the original image data and the preset background feature benchmark value, the pixel feature difference value of each pixel is determined; the background feature benchmark value includes the pixel grayscale benchmark value and the pixel texture entropy benchmark value. Foreground and background pixels are determined based on the pixel feature difference value of each pixel; Based on a preset order, the foreground pixels are traversed to determine the first target pixel that satisfies the contour edge point; the condition for satisfying the contour edge point is that there is at least one background pixel in its eight neighborhoods. Based on the correlation analysis of the first target pixel and the point cloud data, the position coordinates of the obstacle in the target scene are determined.

6. The intelligent control method for obstacle avoidance and emergency stop of an AMR robot according to claim 5, characterized in that, The step of determining the position coordinates of obstacles in the target scene by performing correlation analysis based on the first target pixel and the point cloud data includes: Starting from the first target pixel, the foreground pixels are traversed in their corresponding eight neighborhoods according to their corresponding position index to determine the next second target pixel that satisfies the contour edge point but is not marked as a contour edge point. The tracking position is updated based on the second target pixel, and the foreground pixels are traversed based on the tracking position and the conditions of the contour edge points until the traversal returns to the starting point, thus obtaining the outer contour boundary of the foreground region. Point cloud matching is performed on the outer contour boundary of the foreground region to obtain a subset of point cloud data. Based on the correlation analysis of the foreground pixels and the subset of point cloud data, the position coordinates of obstacles in the target scene are determined.

7. The intelligent control method for obstacle avoidance and emergency stop of an AMR robot according to claim 6, characterized in that, The step of determining the position coordinates of obstacles in the target scene by performing correlation analysis based on the foreground pixels and the subset of point cloud data includes: Correlation analysis is performed based on the color features of each foreground pixel and the reflection intensity features of each point cloud in the point cloud data subset to determine the feature matching result between each foreground pixel and each point cloud; the reflection intensity features characterize the ability of the point cloud to reflect signals emitted by the acquisition device. Based on the feature matching results, the point clouds to be clustered in the subset of point cloud data are determined, and a preset number of the first point clouds in the point clouds to be clustered are selected as the initial cluster centers. Each second point cloud is clustered based on the clustering distance between each second point cloud and each initial cluster center to obtain point cloud clusters. The cluster centers are then updated based on the point cloud coordinates of each point cloud in each point cloud cluster to obtain the target cluster center of each point cloud cluster. This process continues until the offset between the target cluster center and the initial cluster center of each point cloud cluster is less than or equal to a preset offset threshold, resulting in multiple obstacle point cloud clusters. Based on the minimum bounding cube boundary of each obstacle point cloud cluster, the range of three-dimensional boundary coordinates is determined, and the position coordinates of the obstacle are determined based on the mean of the geometric center coordinates of the range of three-dimensional boundary coordinates of each obstacle point cloud cluster.

8. An intelligent control system for obstacle avoidance and emergency stopping of an AMR robot, characterized in that, An intelligent control method for obstacle avoidance and emergency stopping of an AMR robot as described in any one of claims 1 to 7; the intelligent control system for obstacle avoidance and emergency stopping of the AMR robot includes: The positioning module is used to determine the position coordinates of obstacles in the target scene based on the original image data and point cloud data of the AMR robot in the target scene; The operation monitoring module is used to determine the straight-line distance between the obstacle and the AMR robot based on the position coordinates and the real-time positioning information of the AMR robot, and to determine the speed and direction of the obstacle's movement based on the temporal changes of the obstacle's position coordinates over continuous time. The time prediction module is used to determine the shortest predicted encounter time between the obstacle and the AMR robot based on the straight-line distance, the movement speed, and the movement direction, combined with the AMR robot's driving speed and driving direction. The obstacle avoidance intelligent control module is used to generate an emergency stop control command and send it to the motion control system of the AMR robot if the shortest predicted encounter time is less than the corresponding dynamic safety threshold, so as to control the AMR robot to perform an emergency stop operation.

9. An electronic device, comprising: Memory, used to store computer software programs; A processor for reading and executing the computer software program, characterized in that, when the processor executes the computer software program, it implements the intelligent control method for obstacle avoidance and emergency stop of an AMR robot as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, wherein a computer software program is stored therein, characterized in that, When the computer software program is executed by the processor, it implements the intelligent control method for obstacle avoidance and emergency stop of the AMR robot as described in any one of claims 1 to 7.

Images