Vision-assisted industrial robot path planning control system
Through multi-sensor data fusion and real-time motion prediction, dynamic adjustment of path planning is solved, and the problem of improper path planning in industrial robots in complex environments is improved, the system's adaptability and collaborative work ability is improved, and safe and efficient operation is ensured.
Patent Information
- Application Number
- CN202411064837.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing industrial robot path planning systems are difficult to accurately predict robot motion in complex and dynamic environments, and have limited adaptability and flexibility, especially in multi-robot environments, which leads to delay in decision making and improper path planning.
Using vision-assisted industrial robot path planning control system, a three-dimensional model is constructed through multi-sensor data fusion (visual sensors, lidar, infrared sensors), monitoring and predicting the motion state of other robots in real time, dynamically adjusting their own path planning, and combining adaptive exposure Control and image enhancement modules to improve perception and decision-making accuracy.
It realizes more accurate obstacle identification and path planning in complex environments, avoids collisions, improves the adaptability and collaborative working ability of the robot system, and ensures safe and efficient operation.
Smart Images

Figure CN118952192A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of industrial robot path planning and control, and in particular, relates to a vision-assisted industrial robot path planning and control system. Background Art
[0002] In the field of modern industrial automation and intelligent robots, environmental perception and path planning are key technologies for achieving efficient and safe operation. Traditional single-sensor systems have difficulty providing accurate and comprehensive information in complex and changing environments due to their limited perception capabilities. Existing motion prediction models may not be sufficient to accurately capture robot behavior in complex environments. Many models assume that the robot's motion is linear or follows simple physical laws. However, in practical applications, the robot's motion state may be affected by many factors, such as irregular motion patterns, sudden changes, etc., resulting in the inability of existing models to predict the robot's future position with high accuracy. In a complex multi-robot environment, the behavior of other robots will continue to change, which requires the system to have efficient adaptability. However, the adaptability and flexibility of existing technologies in dynamic environments are limited. Especially when the environment changes suddenly or multiple robots move at the same time, it is difficult for the system to quickly adjust and optimize the path, which can easily lead to decision delays and improper path planning.
[0003] In view of this, the present invention is proposed. Summary of the invention
[0004] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a vision-assisted industrial robot path planning control system, which solves the problems raised in the above-mentioned background technology.
[0005] In order to solve the above technical problems, the basic concept of the technical solution adopted by the present invention is:
[0006] A vision-assisted industrial robot path planning control system includes: a data acquisition module, the data acquisition module is used to acquire site data, visual sensor data, laser radar data, and infrared sensor data;
[0007] An adaptive exposure control module, which is used to automatically adjust the exposure parameters of the camera according to real-time lighting conditions;
[0008] Image enhancement module, the image enhancement module is used to enhance the features of obstacles in the image;
[0009] The light source sensor fusion module is used to combine the data of the visual sensor, laser radar, and infrared sensor to make a comprehensive judgment;
[0010] 3D reconstruction module: The 3D reconstruction module builds a three-dimensional model of the environment through multi-sensor data fusion and accurately identifies the location and shape of obstacles;
[0011] A path planning unit, the path planning unit is used to plan a cruising path of the industrial robot in the passage area and send the cruising path to the industrial robot;
[0012] Motion prediction module, when there are multiple robots, the motion prediction module analyzes the motion status of other robots, predicts their future positions, and thus adjusts its own path planning.
[0013] Optionally, the steps of the data acquisition module for acquiring site data, visual sensor data, laser radar data, and infrared sensor data are as follows:
[0014] The basic information of the current site is obtained through the site data acquisition device to form a basic environmental framework, providing a reference for the data fusion of other sensors;
[0015] The camera captures the site image in real time and uses the laser radar to emit laser pulses, receive the reflected pulse signals, and measure the distance and angle;
[0016] Use infrared sensors to emit infrared light, receive reflected infrared signals, and measure the temperature and shape of objects;
[0017] The data acquired from each sensor are synchronized in time and aligned in space.
[0018] Optionally, the steps of the adaptive exposure control module for automatically adjusting the exposure parameters of the camera according to the real-time lighting conditions are:
[0019] Collect real-time lighting data, detect current lighting conditions and analyze lighting intensity to determine current lighting intensity and distribution;
[0020] According to the data of the light sensor, the formula Calculate the required exposure parameter adjustment value and apply the calculated exposure parameter to the camera in real time, where E(t) is the exposure parameter at time t, k is the exposure parameter adjustment coefficient, and L is the exposure parameter adjustment coefficient. target is the target light intensity, L(t) is the total light intensity at time t, L adj (x, y, t) is the image data after adjusting the exposure parameters, g(I adj (x, y, t)) is the image quality assessment function;
[0021] Capturing the adjusted image, evaluating the image quality, and further optimizing the exposure parameters based on the image quality evaluation result;
[0022] If a change in lighting conditions is detected, the above steps are repeated to adjust the exposure parameters in real time.
[0023] Optionally, the steps of the image enhancement module for enhancing the features of obstacles in the image are:
[0024] Receive and store real-time image data from visual sensors, perform multi-level edge detection on images, highlight the outline of obstacles, and perform gradient calculation to enhance edge features;
[0025] Use dilation and erosion operations to remove noise and highlight the structural features of obstacles, and connect disconnected edges through morphological closing operations;
[0026] Perform multi-scale image pyramid processing to extract detailed features of obstacles at different scales and use Laplacian pyramid to fuse information at different scales to enhance the detail performance of obstacles.
[0027] The obstacles are separated from the background, and the texture features of the segmentation results are enhanced using local binary patterns; key points are extracted by combining scale-invariant feature transformation to describe the shape features of the segmentation results.
[0028] Optionally, the light source sensor fusion module is used to combine the data of the visual sensor with the laser radar and the infrared sensor to make a comprehensive judgment in the following steps: receiving data from each sensor, including a visual image, a laser radar point cloud and an infrared image;
[0029] By the formula F(x, y, t) = α·V sync (x, y, t)+β·L sync (θ,φ,t)+γ·R sync (x, y, c) fuses the visual, lidar and infrared data to generate a comprehensive environmental perception result, where α, β, γ are the fusion weight coefficients, V clean (x, y, t) is the visual image data, L calib (θ, φ, t) is the laser radar data, R sync (x, y, t) is the infrared image data;
[0030] Analyze the fused data, make a comprehensive judgment and output the results of the comprehensive judgment for subsequent decision-making and path planning.
[0031] Optionally, the 3D reconstruction module constructs a 3D model of the environment through multi-sensor data fusion, and the steps for accurately identifying the location and shape of obstacles are as follows:
[0032] The data acquisition module receives data from the visual sensor, the lidar and the infrared sensor and aligns the data of different sensors into the same coordinate system;
[0033] Extract feature points from visual images and point cloud data and match them;
[0034] Fusing visual images, lidar point clouds, and infrared sensor data to generate a comprehensive dataset;
[0035] Generate 3D point cloud using the fused data and extract surface information from the 3D point cloud to perform 3D surface reconstruction;
[0036] The position and shape of obstacles are identified and marked in the 3D model, and the constructed 3D model and obstacle information are output to the path planning unit.
[0037] Optionally, the path planning unit is used to plan a cruising path of the industrial robot in the passage area, and the step of sending the cruising path to the industrial robot is:
[0038] Receive the generated 3D environment model and obstacle information from the 3D reconstruction module and set the robot's target position or target area according to task requirements;
[0039] The 3D environment model is converted into a raster map and the formula P opt =PathPlanning(G(x,y),T start , T goal ) Search the map for the optimal path P from the starting point to the target location opt , PathPlanning is the path planning algorithm, G(x, y) is the grid map, T start is the starting position, T goal is the target location;
[0040] The planned path is converted into motion instructions that can be executed by the robot and the generated path instructions are sent to the industrial robot through the communication module.
[0041] Optionally, when there are multiple robots, the motion prediction module analyzes the motion status of other robots and predicts their future positions, thereby adjusting its own path planning steps as follows:
[0042] Receive the motion state data of other robots from the data acquisition module, including position, speed, and acceleration information;
[0043] Monitor the motion status of other robots in real time, record their motion trajectories, and build motion models of other robots based on the acquired data and motion laws;
[0044] Analyze the current motion state of other robots, identify their motion patterns and use formulas Predict the positions of other robots at future times, where P i(t+Δt) is the predicted position of the ith robot at time t+Δt, P i (t) is the position of the ith robot at time t, V i (t) is the speed of the ith robot at time t, A i (t) is the acceleration time step Δt of the ith robot at time t, β k The weight coefficient of the sport mode, M ik (t) is the kth motion mode of the ith robot at time t;
[0045] By formula Detect potential conflicts between its own planned path and other robots’ predicted positions and adjust its own path planning based on the prediction results and conflict detection, where C onflict (t) is the conflict set at time t, P self (t) is the path planned by itself at time t, P i (t+Δt) is the predicted position of the ith robot at the moment, m is the number of other robots, and ∩ is the path intersection operation;
[0046] The adjusted path planning is converted into new motion instructions and sent to the robot for execution.
[0047] After adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art. Of course, any product implementing the present invention does not necessarily need to achieve all the advantages described below at the same time:
[0048] 1. Analyze the motion status of other robots and predict their future positions, thereby adjusting their own path planning. Specifically, this module receives the motion status data of other robots, such as position, speed, and acceleration, in real time, uses these data to build accurate motion models, and uses these models to predict the motion trajectories of other robots at future moments. Based on these prediction results, the system can identify potential conflict points in advance and dynamically adjust its own path planning to avoid collisions and interference.
[0049] 2. This application can automatically adjust the camera's exposure parameters in real-time lighting conditions to ensure that the camera can capture high-quality images under various lighting conditions. Through adaptive exposure control, the system can always obtain clear visual information in an environment with changing lighting, reduce image quality problems caused by unstable lighting, and thus improve the accuracy of image processing and recognition.
[0050] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] The drawings described below are only some embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0052] In the figure:
[0053] Figure 1 Flowchart of the control system for industrial robot path planning.
[0054] It should be noted that these drawings and textual descriptions are not intended to limit the conceptual scope of the present invention in any way, but are intended to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0055] The present invention will now be described in further detail with reference to the accompanying drawings.
[0056] See also Figure 1 As shown, in this embodiment, a vision-assisted industrial robot path planning control system is provided, including a data acquisition module, which is used to acquire site data, visual sensor data, laser radar data, and infrared sensor data; by integrating different types of sensor data, the system can obtain more accurate and diverse environmental information. For example, the visual sensor provides high-resolution image data that can identify subtle object features; the laser radar can provide accurate distance measurement data, which helps to build a three-dimensional space model; and the infrared sensor can detect temperature changes and objects hidden in the visual blind spot. Combining these data, the robot can better understand and adapt to its working environment, and improve the safety and efficiency of navigation and operation.
[0057] Adaptive exposure control module, the adaptive exposure control module is used to automatically adjust the exposure parameters of the camera according to the real-time lighting conditions; the adaptive exposure control module can dynamically adapt to changes in ambient lighting to ensure that the camera always captures high-quality images. Ambient lighting conditions may change significantly in a short period of time, such as from indoors to outdoors, or from shadow areas to direct sunlight areas. Through adaptive exposure control, the camera can quickly adjust its exposure settings to avoid overexposure or underexposure, ensuring image clarity and detail. At the same time, this function also reduces the burden of image post-processing because the acquired images are of higher quality with less noise and artifacts.
[0058] Image enhancement module, the image enhancement module is used to enhance the features of obstacles in the image; the image enhancement module improves the visibility and recognition of key features in the image. Clear identification of obstacles is crucial for safe obstacle avoidance and path planning of the robot. Image enhancement technology can include edge detection, contrast adjustment, and noise filtering, making obstacles more prominent in the image, which is convenient for subsequent algorithms to identify and process. By enhancing the features of obstacles in the image, the robot can perceive the surrounding environment more accurately, detect and avoid potential dangerous objects in a timely manner, and ensure safe operation. In addition, the enhanced image also provides clearer input data for the 3D reconstruction module and the path planning module, improving the efficiency and accuracy of the overall system.
[0059] Light source sensor fusion module: The light source sensor fusion module is used to combine the data of visual sensors, laser radars, and infrared sensors for comprehensive judgment; the light source sensor fusion module uses multi-sensor fusion technology to make up for the limitations of a single sensor and provide more comprehensive and accurate environmental perception. Visual sensors can capture detailed images of the environment, but their effects are limited in cases of insufficient or excessive light; laser radars provide accurate distance measurements, but cannot recognize colors and textures; infrared sensors can detect temperature changes in low-light environments. By fusing these data, the system can generate a more reliable environmental model and improve its understanding and judgment of complex environments.
[0060] 3D reconstruction module: The 3D reconstruction module builds a 3D model of the environment through multi-sensor data fusion, accurately identifying the location and shape of obstacles; the 3D reconstruction module can provide the robot with a detailed 3D environmental model to help it achieve accurate navigation and operation in complex environments. By fusing data from visual sensors, lidar and infrared sensors, the system can generate high-precision 3D point clouds and surface models that fully reflect the spatial structure and object characteristics of the surrounding environment. 3D reconstruction not only improves the accuracy of obstacle recognition, but also can identify the height, volume and shape of objects, making it easier for the robot to make more detailed path planning and obstacle avoidance decisions.
[0061] A path planning unit, the path planning unit is used to plan a cruising path of the industrial robot in the passage area and send the cruising path to the industrial robot;
[0062] Motion prediction module: When there are multiple robots, the motion prediction module analyzes the motion status of other robots and predicts their future positions, thereby adjusting its own path planning. The motion prediction module can improve the collaborative working ability of multi-robot systems and avoid mutual interference and collisions. By analyzing the current motion status of other robots and applying motion models, the system can predict their future positions and trajectories and identify potential conflict points in advance. Based on these prediction results, its own path planning can be dynamically adjusted to select the optimal path to avoid conflict areas and ensure safe and efficient driving. This prediction and adjustment mechanism not only improves the operating efficiency and safety of a single robot,
[0063] According to the vision-assisted industrial robot path planning control system of claim 1, it is characterized in that the steps of the data acquisition module for acquiring site data, visual sensor data, laser radar data, and infrared sensor data are as follows:
[0064] The basic information of the current site is obtained through the site data acquisition device to form a basic environmental framework, which provides a reference for the data fusion of other sensors; through the site data acquisition device, the robot can obtain the basic structure and layout information of the site, including the location of fixed buildings, roads, obstacles, etc. This information constitutes a static reference frame to help align and fuse the data of other sensors in the same spatial reference system. Such a framework helps to improve the accuracy of data fusion because the data collected by each sensor can be accurately mapped to the same environmental model. In addition, the environmental framework also provides an overall perspective, allowing the system to better understand and analyze when processing dynamic information. For example, when planning paths and identifying obstacles, a fixed environmental framework can provide reference points to help the system make decisions faster and more accurately.
[0065] The camera captures the site image in real time and uses the laser radar to emit laser pulses, receive the reflected pulse signals, and measure the distance and angle;
[0066] Use infrared sensors to emit infrared light, receive reflected infrared signals, and measure the temperature and shape of objects;
[0067] The data obtained from each sensor is synchronized in time and aligned in space. Through precise time synchronization and spatial alignment, the system can generate a comprehensive, multi-dimensional environmental model to improve the accuracy and reliability of perception. This alignment technology not only reduces data redundancy and errors, but also enhances the system's real-time response capabilities in dynamic environments.
[0068] The vision-assisted industrial robot path planning and control system according to claim 1 is characterized in that the step of the adaptive exposure control module for automatically adjusting the exposure parameters of the camera according to the real-time lighting conditions is as follows:
[0069] Collect real-time lighting data, detect current lighting conditions and analyze lighting intensity to determine current lighting intensity and distribution;
[0070] According to the data of the light sensor, the formula Calculate the required exposure parameter adjustment value and apply the calculated exposure parameter to the camera in real time, where E(t) is the exposure parameter at time t, k is the exposure parameter adjustment coefficient, and L is the exposure parameter adjustment coefficient. target is the target light intensity, L(t) is the total light intensity at time t, I adj (x, y, t) is the image data after adjusting the exposure parameters, g(I adj (x, y, t)) is the image quality assessment function;
[0071] The adjusted image is captured and the image quality is evaluated. Based on the image quality evaluation results, the exposure parameters are further optimized. The dynamic optimization process ensures that the camera can adapt to various environmental lighting changes and always capture high-quality images. This not only improves the visual effect of the image, but also improves the accuracy of subsequent image processing and analysis, and enhances the robot's navigation and operation capabilities in complex environments.
[0072] If a change in lighting conditions is detected, the above steps are repeated to adjust the exposure parameters in real time.
[0073] The vision-assisted industrial robot path planning and control system according to claim 1 is characterized in that the image enhancement module is used to enhance the characteristics of obstacles in the image by:
[0074] Receive and store real-time image data from the visual sensor, perform multi-level edge detection on the image, highlight the outline of the obstacle, and perform gradient calculation to enhance the edge features; through multi-level edge detection, the system can gradually enhance the edge features in the image, highlight the outline of the obstacle, and make it more obvious and easy to identify in subsequent processing. Gradient calculation further enhances these edge features by calculating the rate of change of pixel values to highlight the boundaries of objects in the image. This process not only helps to separate obstacles from the background, but also improves the accuracy of the subsequent algorithm's judgment of the obstacle's position and shape. High-quality edge detection and enhancement enable the system to more accurately identify and avoid obstacles in complex environments, improving the safety and efficiency of overall navigation and operation.
[0075] Dilation and erosion operations are used to eliminate noise and highlight the structural features of obstacles. The morphological closing operation is used to connect the disconnected edges. The dilation operation can fill small holes in the image to make the boundary of the obstacle more continuous and clear. The erosion operation can remove isolated noise points and further purify the image. The combination of these operations can effectively highlight the structural features of obstacles, making them more obvious and easy to identify in the image. The morphological closing operation further connects the disconnected edges to ensure the complete outline of the obstacle. These processing steps not only improve the accuracy of obstacle detection, but also reduce the false detection rate, allowing the system to work stably under various environmental conditions. By eliminating noise and enhancing structural features, the system's image processing capabilities are significantly improved, providing more reliable input data for subsequent path planning and navigation.
[0076] Perform multi-scale image pyramid processing, extract the detailed features of obstacles at different scales, and use the Laplacian pyramid to fuse information at different scales to enhance the detailed performance of obstacles. The Laplacian pyramid further fuses information at different scales and enhances the detailed performance of obstacles by fusing features at each scale. This method not only improves the clarity of the obstacle edge, but also enhances its texture and shape features, making the obstacles more three-dimensional and obvious in the image.
[0077] The obstacles are separated from the background, and the texture features of the segmentation results are enhanced using local binary patterns; key points are extracted by combining scale-invariant feature transformation to describe the shape features of the segmentation results.
[0078] According to claim 1, a vision-assisted industrial robot path planning control system is characterized in that the light source sensor fusion module is used to combine the data of the visual sensor with the laser radar and the infrared sensor to make a comprehensive judgment, and the steps are: receiving data from each sensor, including a visual image, a laser radar point cloud and an infrared image;
[0079] By the formula F(x, y, t) = α·V stnc (x, y, t)+β·L sync (θ,φ,t)+γ·R stnc (x, y, t) fuses the visual, lidar and infrared data to generate a comprehensive environmental perception result, where α, β, γ are the fusion weight coefficients, V clean (x, y, t) is the visual image data, L calib (θ, φ, t) is the laser radar data, R sync(x, y, t) is infrared image data; the benefit of data fusion is that it can integrate the information provided by each sensor to make up for the shortcomings of a single sensor. For example, visual sensors provide rich color and detail information, but perform poorly in low-light conditions; LiDAR provides accurate distance information, but cannot capture color and texture; infrared sensors can work effectively at night and in low-light conditions.
[0080] Analyze the fused data, make a comprehensive judgment and output the results of the comprehensive judgment for subsequent decision-making and path planning. Analyzing the fused data can ensure that the system responds intelligently, quickly and accurately in complex environments. By analyzing the fused data, the system can identify key features and potential obstacles in the environment and accurately judge their positions and shapes. This comprehensive judgment is based on data from multiple sensors, reducing the errors that may be caused by a single sensor and improving the reliability and accuracy of the judgment. For example, in a complex indoor environment, visual sensors can help identify the color and shape of furniture and walls, lidar provides their precise position and size, and infrared sensors detect the heat source of people or equipment. Combining this information, the system can fully understand the layout and dynamic changes of the environment. After outputting the results of the comprehensive judgment, the system can plan the path accordingly, select the optimal driving route, avoid obstacles and dangerous areas, and ensure that robots or automated equipment can perform tasks efficiently and safely.
[0081] The vision-assisted industrial robot path planning and control system according to claim 1 is characterized in that the 3D reconstruction module constructs a three-dimensional model of the environment through multi-sensor data fusion, and the steps of accurately identifying the position and shape of obstacles are:
[0082] The data acquisition module receives data from the visual sensor, the lidar and the infrared sensor and aligns the data of different sensors into the same coordinate system;
[0083] Extract feature points from visual images and point cloud data and match them;
[0084] Fusing visual images, lidar point clouds, and infrared sensor data to generate a comprehensive dataset;
[0085] Generate 3D point cloud using the fused data and extract surface information from the 3D point cloud to perform 3D surface reconstruction;
[0086] The position and shape of obstacles are identified and marked in the 3D model, and the constructed 3D model and obstacle information are output to the path planning unit. This alignment process eliminates the deviation of different sensor data in time and space, so that multi-source information can be accurately integrated to form a complete environmental model. The aligned data not only improves the system's ability to understand the environment, but also enhances the accuracy and reliability of data processing, providing the necessary guarantee for the robot's accurate navigation and operation in complex environments. Through precise feature point matching, the system can better identify and locate objects in the environment and determine their relative position and shape. This not only helps to improve the accuracy of obstacle detection, but also provides a reliable data basis for 3D reconstruction. The accurate matching and fusion of feature points make the system more adaptable in complex and dynamic environments, and improve the overall navigation and obstacle avoidance performance.
[0087] According to a vision-assisted industrial robot path planning and control system according to claim 1, it is characterized in that the path planning unit is used to plan the cruising path of the industrial robot in the passage area, and the steps of sending the cruising path to the industrial robot are:
[0088] Receive the generated 3D environment model and obstacle information from the 3D reconstruction module and set the robot's target position or target area according to task requirements;
[0089] The 3D environment model is converted into a raster map and the formula P opt =PathPlanning(G(x,y),T start , T goal ) Search the map for the optimal path P from the starting point to the target location opt , PathPlanning is the path planning algorithm, G(x, y) is the grid map, T start is the starting position, T goal The advantage of searching for the optimal path on the map is that this process simplifies complex three-dimensional information, making path planning more intuitive and efficient. The grid map converts three-dimensional environmental information into a grid on a two-dimensional plane, and each grid cell represents a specific spatial area and indicates whether it is passable. This representation method not only simplifies the computational complexity, but also enables the path search algorithm to run efficiently.
[0090] The planned path is converted into motion instructions that can be executed by the robot and the generated path instructions are sent to the industrial robot through the communication module. The advantage of sending it to the industrial robot through the communication module is that this process realizes a seamless connection from high-level path planning to low-level execution control, ensuring that the robot can accurately and in real time follow the planned route. The path generated by the path planning algorithm needs to be converted into specific motion instructions, such as going straight, turning, accelerating or decelerating, so that the robot control system can execute it. This conversion process ensures that the path planning results can be actually applied and avoids execution errors caused by mismatched instructions. Sending motion instructions through the communication module realizes real-time communication and instruction transmission between the path planning unit and the robot control unit, ensuring that the robot can adjust and drive according to the latest path planning results.
[0091] The visually assisted industrial robot path planning control system according to claim 1 is characterized in that, when there are multiple robots, the motion prediction module analyzes the motion states of other robots, predicts their future positions, and thus adjusts its own path planning in the following steps:
[0092] The motion state data of other robots, including position, speed, and acceleration information, are received from the data acquisition module; by obtaining the motion state of other robots, the system can fully understand the dynamic information of all robots in the current environment, providing key input for subsequent analysis and prediction. These data enable the system to monitor the behavior of other robots in real time, identify their position, speed changes, and acceleration, and thus fully grasp the motion trajectory of each robot. This real-time data acquisition ensures that the system can respond quickly to environmental changes, adjust its own strategies in a timely manner, and avoid collisions or interference with other robots. In addition, accurate motion state data provides a basis for establishing motion models of other robots, enabling the system to accurately predict and plan in complex environments, and improve the collaborative work efficiency and safety of the entire multi-robot system.
[0093] Monitor the motion status of other robots in real time, record their motion trajectories, and build motion models of other robots based on the acquired data and motion laws; by continuously monitoring and recording the motion status, the system can obtain rich trajectory data and provide comprehensive information for analysis and modeling. Using this data, the system can identify the motion laws and patterns of other robots, such as constant speed linear motion, accelerated motion, or complex path changes. Based on these analysis results, the system can build an accurate motion model to predict the position and state of other robots at future moments. This model not only improves the accuracy of the prediction, but also enhances the system's adaptability in dynamic environments. By understanding the behavior patterns of other robots, the system can respond in advance, optimize its own path planning, avoid potential conflicts, and improve the safety and efficiency of overall operations.
[0094] Analyze the current motion state of other robots, identify their motion patterns and use formulas
[0095] Predict the positions of other robots at future times, where P i (t+Δt) is the predicted position of the ith robot at time t+Δt, P i (t) is the position of the ith robot at time t, V i (t) is the speed of the ith robot at time t, A i (t) is the acceleration time step Δt of the ith robot at time t, β k The weight coefficient of the sport mode, M ik (t) is the kth motion mode of the ith robot at time t; through analysis and prediction, the system can not only improve the safety and efficiency of path planning, but also achieve more efficient collaborative work in multi-robot systems. In short, accurate motion prediction capabilities enable robots to better adapt to dynamic environments, optimize path planning, and ensure the smooth execution of tasks.
[0096] By formula Detect potential conflicts between its own planned path and other robots’ predicted positions and adjust its own path planning based on the prediction results and conflict detection, where C conflict (t) is the conflict set at time t, P self (t) is the path planned by itself at time t, P i (t+Δt) is the predicted position of the ith robot at the moment, m is the number of other robots, and ∩ is the path intersection operation; this system can detect the intersection of its own path with the predicted positions of other robots and identify potential conflict points. Based on the conflict detection results, the system can adjust its own path planning in time, avoid potential collision areas, and choose a safer driving route. This dynamic adjustment capability improves the flexibility and real-time performance of path planning, allowing the robot to maintain efficient operation in complex and changing environments. By avoiding conflicts with other robots, the system not only improves the safety of the overall operation, but also optimizes the path and improves the efficiency of task execution.
[0097] The adjusted path planning is converted into new motion instructions and sent to the robot for execution.
[0098] The present invention is not limited to the above-mentioned embodiments. Anyone should be aware that any structural changes made under the enlightenment of the present invention, and any technical solutions that are the same or similar to the present invention, fall within the protection scope of the present invention. The technology, shape, and structural parts not described in detail in the present invention are all well-known technologies.
Claims
1. A vision-assisted industrial robot path planning control system, characterized in that: include: Data acquisition module, which is used to acquire site data, visual sensor data, lidar data, and infrared sensor data; An adaptive exposure control module, which is used to automatically adjust the exposure parameters of the camera according to real-time lighting conditions; Image enhancement module, the image enhancement module is used to enhance the features of obstacles in the image; The light source sensor fusion module is used to combine the data of the visual sensor, laser radar, and infrared sensor to make a comprehensive judgment; 3D reconstruction module: The 3D reconstruction module builds a three-dimensional model of the environment through multi-sensor data fusion and accurately identifies the location and shape of obstacles; A path planning unit, the path planning unit is used to plan a cruising path of the industrial robot in the passage area and send the cruising path to the industrial robot; Motion prediction module, when there are multiple robots, the motion prediction module analyzes the motion status of other robots, predicts their future positions, and thus adjusts its own path planning.
2. A vision-assisted industrial robot path planning control system according to claim 1, characterized in that: The steps of the data acquisition module for acquiring site data, visual sensor data, lidar data, and infrared sensor data are as follows: The basic information of the current site is obtained through the site data acquisition device to form a basic environmental framework, providing a reference for the data fusion of other sensors; The camera captures the site image in real time and uses the laser radar to emit laser pulses, receive the reflected pulse signals, and measure the distance and angle; Use infrared sensors to emit infrared light, receive reflected infrared signals, and measure the temperature and shape of objects; The data acquired from each sensor are synchronized in time and aligned in space.
3. A vision-assisted industrial robot path planning control system according to claim 1, characterized in that: The steps of the adaptive exposure control module to automatically adjust the exposure parameters of the camera according to the real-time lighting conditions are as follows: Collect real-time lighting data, detect current lighting conditions and analyze lighting intensity to determine current lighting intensity and distribution; According to the data of the light sensor, the formula Calculate the required exposure parameter adjustment value and apply the calculated exposure parameter to the camera in real time, where E(t) is the exposure parameter at time t, k is the exposure parameter adjustment coefficient, and L is the exposure parameter adjustment coefficient. target is the target light intensity, L(t) is the total light intensity at time t, I adj (x, y, t) is the image data after adjusting the exposure parameters, g(I adj (x, y, t)) is the image quality assessment function; Capturing the adjusted image, evaluating the image quality, and further optimizing the exposure parameters based on the image quality evaluation result; If a change in lighting conditions is detected, the above steps are repeated to adjust the exposure parameters in real time.
4. A vision-assisted industrial robot path planning control system according to claim 1, characterized in that: The steps of the image enhancement module to enhance the features of obstacles in the image are: Receive and store real-time image data from visual sensors, perform multi-level edge detection on images, highlight the outline of obstacles, and perform gradient calculation to enhance edge features; Use dilation and erosion operations to remove noise and highlight the structural features of obstacles, and connect disconnected edges through morphological closing operations; Perform multi-scale image pyramid processing to extract detailed features of obstacles at different scales and use Laplacian pyramid to fuse information at different scales to enhance the detail performance of obstacles; The obstacles are separated from the background, and the texture features of the segmentation results are enhanced using local binary patterns; key points are extracted by combining scale-invariant feature transformation to describe the shape features of the segmentation results.
5. The vision-assisted industrial robot path planning control system according to claim 1, characterized in that: The light source sensor fusion module is used to combine the data of the visual sensor with the lidar and infrared sensors to make a comprehensive judgment in the following steps: receiving data from each sensor, including visual images, lidar point clouds and infrared images; By the formula F(x, y, t) = α·V sync (x, y, t)+β·L sync (θ,φ,t)+γ·R sync (x, y, t) fuses the visual, lidar and infrared data to generate a comprehensive environmental perception result, where α, β, γ are the fusion weight coefficients, V clean (x, y, t) is the visual image data, L calib (θ, φ, t) is the laser radar data, R sync (x, y, t) is the infrared image data; Analyze the fused data, make a comprehensive judgment and output the results of the comprehensive judgment for subsequent decision-making and path planning.
6. A vision-assisted industrial robot path planning control system according to claim 1, characterized in that: The 3D reconstruction module builds a 3D model of the environment through multi-sensor data fusion and accurately identifies the location and shape of obstacles in the following steps: The data acquisition module receives data from the visual sensor, the lidar and the infrared sensor and aligns the data of different sensors into the same coordinate system; Extract feature points from visual images and point cloud data and match them; Fusing visual images, lidar point clouds, and infrared sensor data to generate a comprehensive dataset; Generate 3D point cloud using the fused data and extract surface information from the 3D point cloud to perform 3D surface reconstruction; The position and shape of obstacles are identified and marked in the 3D model, and the constructed 3D model and obstacle information are output to the path planning unit.
7. A vision-assisted industrial robot path planning control system according to claim 1, characterized in that: The path planning unit is used to plan a cruising path of the industrial robot in the passage area, and the steps of sending the cruising path to the industrial robot are as follows: Receive the generated 3D environment model and obstacle information from the 3D reconstruction module and set the robot's target position or target area according to task requirements; The 3D environment model is converted into a grid map and the formula P opt =PathPlanning(G(x,y),T start , T goal ) Search the map for the optimal path P from the starting point to the target location opt , PathPlanning is the path planning algorithm, G(x, y) is the grid map, T start is the starting position, T goal is the target location; The planned path is converted into motion instructions that can be executed by the robot and the generated path instructions are sent to the industrial robot through the communication module.
8. The vision-assisted industrial robot path planning control system according to claim 1, characterized in that: When there are multiple robots, the motion prediction module analyzes the motion status of other robots and predicts their future positions, thereby adjusting its own path planning steps as follows: Receive the motion state data of other robots from the data acquisition module, including position, speed, and acceleration information; Monitor the motion status of other robots in real time, record their motion trajectories, and build motion models of other robots based on the acquired data and motion laws; Analyze the current motion state of other robots, identify their motion patterns and use formulas Predict the positions of other robots at future times, where P i (t+Δt) is the predicted position of the ith robot at time t+Δt, P i (t) is the position of the ith robot at time t, V i (t) is the speed of the ith robot at time t, A i (t) is the acceleration time step Δt of the ith robot at time t, β k The weight coefficient of the sport mode, M ik (t) is the kth motion mode of the ith robot at time t; By formula Detect potential conflicts between its own planned path and other robots’ predicted positions and adjust its own path planning based on the prediction results and conflict detection, where C conflict (t) is the conflict set at time t, P self (t) is the path planned at time t, P i (t+Δt) is the predicted position of the ith robot at the moment, m is the number of other robots, and ∩ is the path intersection operation; The adjusted path planning is converted into new motion instructions and sent to the robot for execution.
Citation Information
Cited By
Chip mounter modular control system and method in monitoring mode, medium and product
CN119677088A
Working parameter automatic deployment method and device of industrial robot and industrial robot
CN119974015A
Automatic deployment method and device for working parameters of industrial robots and industrial robots
CN119974015B
Additive manufacturing multi-robot path planning method and device based on collision prediction and medium
CN120244953A
Cooperative control method and system for non-inductive assembly of equipotential insulator string W pin
CN120527799A
