Intelligent robot feeding and discharging method and system

Abstract

The invention relates to an intelligent robot feeding and discharging method and system, and the method comprises the steps: obtaining the image information of a target material, carrying out the recognition and analysis of the image information of the target material, and obtaining an analysis result which comprises the position information and posture information of the target material; inputting the analysis result into a pre-trained operation decision model for analysis, and generating an action planning instruction corresponding to the target material; task scheduling operation is executed based on the current task state and the resource information, a task scheduling result is obtained, and the scheduling operation comprises task distribution optimization and path generation control; according to the task scheduling result and the action planning instruction, the target robot is scheduled to complete feeding and discharging operation; and in the loading and unloading execution process, the operation state is monitored in real time, the operation state is detected and analyzed, an interference detection result is obtained, and the action planning instruction is dynamically adjusted based on the interference detection result. The production line has the effect of improving the efficiency of the production line.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent manufacturing, and in particular to an intelligent robot loading and unloading method and system. Background Art

[0002] Currently, robots are widely used in industrial production to perform material handling and loading and unloading operations, improving production efficiency and reducing labor costs. With the development of computer vision and artificial intelligence, some intelligent robotic systems are now capable of performing target positioning and path planning based on image recognition, replacing traditional manual loading and unloading tasks. However, in practical application, many challenges remain.

[0003] Most existing intelligent loading and unloading robots rely on preset paths and fixed rules to perform tasks. They lack the ability to dynamically adapt to changes in task status, resource allocation, and the external environment. They struggle to cope with issues such as multi-task collaborative scheduling, changing target material postures, and real-time interference processing in complex production scenarios. For example, if emergencies such as material displacement, blocked paths, or human proximity occur during an operation, existing systems often fail to identify and adjust them in a timely manner, leading to task interruptions or potential safety risks. Furthermore, task scheduling is often based on static priority settings and fails to dynamically optimize allocation based on real-time operation status and resource load conditions. Overall operational efficiency and flexible intelligence still need to be improved.

[0004] The above-mentioned existing technical solutions have the following defects: the existing loading and unloading methods are difficult to achieve real-time and robust operation planning in a multi-task dynamic environment, and are prone to problems such as low resource utilization, frequent path conflicts and task interruptions, thereby affecting the stability and efficiency of the overall loading and unloading operations. Therefore, there is room for improvement. Summary of the Invention

[0005] In order to improve the efficiency of the production line, the present application provides an intelligent robot loading and unloading method and system.

[0006] The above-mentioned invention objective of this application is achieved through the following technical solutions: An intelligent robot loading and unloading method, the intelligent robot loading and unloading method comprising: Acquire image information of a target material, perform recognition analysis on the image information of the target material, and obtain an analysis result, wherein the analysis result includes position information and posture information of the target material; Inputting the analysis results into a pre-trained operation decision model for analysis to generate action planning instructions corresponding to the target material; Performing task scheduling operations based on current task status and resource information to obtain task scheduling results, wherein the scheduling operations include task allocation optimization and path generation control; Scheduling the target robot to complete loading and unloading operations according to the task scheduling results and the action planning instructions; During the loading and unloading process, the operation status is monitored in real time, the operation status is detected and analyzed, interference detection results are obtained, and the action planning instructions are dynamically adjusted based on the interference detection results; During the robot operation, the robot recognizes the approaching behavior of people based on the environmental perception information, and performs safety control operations according to the recognition results, and the safety control operations include switching the robot's motion mode.

[0007] By adopting the above technical solution, by acquiring the image information of the target material and extracting its position information and posture information, it is possible to accurately grasp the three-dimensional spatial state of the target material, thereby providing a precise input basis for subsequent action planning; by inputting the analysis results into a pre-trained job decision model to generate action planning instructions, it is possible to infer task operation strategies based on multiple factors in complex scenarios, thereby improving the rationality of action planning and the accuracy of loading and unloading execution; by performing task scheduling operations based on the current task status and resource information, and generating scheduling results including task allocation and path control, it is possible to dynamically coordinate multi-robot resources and reasonably allocate task loads, thereby improving overall operation efficiency and scheduling flexibility; by monitoring the operation status in real time during the loading and unloading process and dynamically adjusting the action instructions according to the interference situation, it is possible to effectively respond to sudden interference situations during the operation process, thereby ensuring the continuity and robustness of the loading and unloading process; by identifying human approach behavior and switching the robot motion mode to perform safety control, it is possible to reduce safety risks in the human-computer interaction process, thereby improving the safety of the working environment.

[0008] In one example, the present application may be further configured as follows: the image information of the target material is identified and analyzed, and the analysis result obtained includes: Performing image preprocessing on the image information to extract image features of the target material in the image, wherein the image preprocessing includes image segmentation and feature extraction, and the image features include edge features, texture features, and color distribution information; Identifying the image features based on a target detection algorithm to identify the target material area in the image information; Based on the target material area, combined with a space reconstruction method, the position information and posture information of the target material are extracted.

[0009] By adopting the above technical solution, by performing image preprocessing and feature extraction on the target material image, the robustness of image recognition and the distinguishability of material features can be enhanced, thereby improving the accuracy of target detection; by identifying the material area based on the target detection algorithm and combining the spatial reconstruction method to extract position information and posture information, the precise positioning of the material in three-dimensional space can be achieved, thereby providing high-precision input support for the robot to perform loading and unloading actions.

[0010] In one example, the present application may be further configured as follows: before inputting the analysis result into a pre-trained operation decision model for analysis, the intelligent robot loading and unloading method further includes: Collect multiple sets of sample data containing image features, position information, and posture information, and construct label data corresponding to the target action sequence; Inputting the sample data and label data into a constructed deep neural network model for training, wherein the deep neural network model includes an encoder structure and an action decoding structure; The deep neural network model is optimized by minimizing the action prediction error as a loss function until the model converges, thereby obtaining the pre-trained task decision model.

[0011] By adopting the above technical solution, by collecting multiple sets of sample data containing image features and posture information and constructing action sequence labels, it is possible to establish a correspondence between training data and target tasks, thereby improving the learning effectiveness of the model; by inputting samples into a deep neural network model containing an encoder and action decoding structure and performing training, it is possible to improve the model's generalization ability for complex work scenarios, thereby enhancing the intelligence and reliability of subsequent action planning.

[0012] In one example, the present application may be further configured as follows: inputting the analysis result into a pre-trained operation decision model for analysis, and generating an action planning instruction corresponding to the target material includes: Performing structured coding on the analysis results, wherein the structured coding includes converting the position information and posture information of the target material into a unified data vector format; Inputting the structured code into the operation decision model for forward reasoning to obtain operation execution information matching the loading and unloading tasks; Based on the operation execution information, action planning instructions are generated to drive the target robot to perform corresponding loading and unloading operations.

[0013] By adopting the above technical solution, by structured encoding the analysis results and inputting them into the operation decision model for forward reasoning, it is possible to standardize the data input format and activate the model's operation logic judgment ability, thereby improving the model's reasoning efficiency and motion planning accuracy; by generating motion planning instructions based on the reasoning results, it is possible to ensure that the robot accurately performs loading and unloading operations according to task requirements, thereby improving the consistency and accuracy of the overall task execution.

[0014] In one example, the present application may be further configured as follows: the task allocation optimization includes: Obtain the operation status and available resource information of each target robot as robot status information; Extract task queue, priority information and resource occupancy information as task information; Establishing a matching relationship between the task and the robot based on the task information and the robot status information, wherein the matching relationship comprehensively considers the urgency of the task, the robot's idleness, and historical execution results; A task allocation result is generated based on the matching relationship, and the task allocation result is used to guide robot scheduling execution.

[0015] By adopting the above technical solution, by obtaining the operating status and available resource information of each target robot as robot status information, it is possible to fully grasp the robot's execution capability and resource distribution, thereby providing data support for task matching; by extracting task queues, priorities and resource occupancy to construct task information, it is possible to reflect the urgency of task processing and resource dependency, thereby improving the rationality of task scheduling; by establishing a task-robot matching relationship that comprehensively considers the task urgency, idleness and historical performance, it is possible to accurately match robots and tasks, thereby improving the fairness and efficiency of scheduling; by generating task allocation results based on the matching relationship and guiding robot scheduling execution, it is possible to achieve the optimal configuration of tasks and resources, thereby improving overall operating efficiency.

[0016] In one example, the present application may be further configured as follows: the path generation control includes: Based on the workshop layout and real-time environmental perception data, a path search space is constructed from the target robot's current position to the mission target point; Generating multiple feasible paths in the path search space, and performing a cost evaluation operation on the feasible paths to obtain the cost evaluation result, wherein the cost evaluation includes path length, number of obstacle avoidance times, and execution delay; According to the cost evaluation result, the target path is selected as the final path generation control result, which is used to drive the target robot to perform task migration.

[0017] By adopting the above technical solution, by constructing a path search space based on the workshop layout diagram and real-time environmental perception data, it is possible to dynamically reflect the on-site spatial structure and real-time obstacle distribution, thereby providing effective environmental modeling for path planning; by generating multiple feasible paths and performing cost evaluation based on path length, number of obstacle avoidances and execution delay, a multi-objective optimization path screening strategy can be implemented, thereby improving the overall performance of path selection; by selecting the path with the optimal cost as the final control result to drive robot migration, it is possible to reduce operation delays and obstacle avoidance risks, thereby improving the efficiency and safety of robot movement.

[0018] In one example, the present application may be further configured as follows: detecting and analyzing the operation status to obtain an interference detection result, and dynamically adjusting the action planning instruction based on the interference detection result includes: Obtain the execution feedback data and task process status information of the target robot as job status data; Detecting whether there are abnormal events based on the operation status data, wherein the abnormal events include target material position deviation, grasping failure, path obstruction or execution delay abnormality; When an abnormal event is detected, a corresponding interference detection result is generated, and the current action planning instruction is adjusted based on the interference detection result. The adjustment includes replanning the grasping path, switching to a backup task, or delaying the task execution time.

[0019] By adopting the above technical solution, by obtaining robot execution feedback and task process status to construct job status data, it is possible to fully perceive the operating status of the robot execution process, thereby providing an accurate basis for interference detection; by detecting abnormal events such as position offset, grasping failure, path obstruction or delay, it is possible to quickly identify the key factors that lead to job interruption, thereby improving the real-time performance of abnormal response; by dynamically adjusting action planning instructions based on detection results, including replanning paths or switching tasks, it is possible to effectively restore the operation process and reduce the probability of downtime, thereby improving the stability of the system and the continuity of task execution.

[0020] The second object of the present invention is achieved through the following technical solutions: An intelligent robot loading and unloading system, comprising: An image recognition module is used to obtain image information of a target material, perform recognition analysis on the image information of the target material, and obtain an analysis result, wherein the analysis result includes position information and posture information of the target material; An operation decision module is used to input the analysis results into a pre-trained operation decision model for analysis, and generate an action planning instruction corresponding to the target material; A scheduling decision module is used to perform task scheduling operations based on current task status and resource information to obtain task scheduling results. The scheduling operations include task allocation optimization and path generation control. An action execution module is used to schedule the target robot to complete loading and unloading operations according to the task scheduling results and the action planning instructions; An interference monitoring module is used to monitor the operation status in real time during the loading and unloading process, detect and analyze the operation status, obtain interference detection results, and dynamically adjust the action planning instructions based on the interference detection results; The safety perception module is used to identify human approach behavior based on environmental perception information during the robot operation process, and perform safety control operations according to the recognition results. The safety control operations include switching the robot's motion mode.

[0021] By adopting the above technical solution, by acquiring the image information of the target material and extracting its position information and posture information, it is possible to accurately grasp the three-dimensional spatial state of the target material, thereby providing a precise input basis for subsequent action planning; by inputting the analysis results into a pre-trained job decision model to generate action planning instructions, it is possible to infer task operation strategies based on multiple factors in complex scenarios, thereby improving the rationality of action planning and the accuracy of loading and unloading execution; by performing task scheduling operations based on the current task status and resource information, and generating scheduling results including task allocation and path control, it is possible to dynamically coordinate multi-robot resources and reasonably allocate task loads, thereby improving overall operation efficiency and scheduling flexibility; by monitoring the operation status in real time during the loading and unloading process and dynamically adjusting the action instructions according to the interference situation, it is possible to effectively respond to sudden interference situations during the operation process, thereby ensuring the continuity and robustness of the loading and unloading process; by identifying human approach behavior and switching the robot motion mode to perform safety control, it is possible to reduce safety risks in the human-computer interaction process, thereby improving the safety of the working environment.

[0022] In summary, this application has the following beneficial technical effects: 1. By acquiring the image information of the target material and extracting its position and posture information, the 3D spatial state of the target material can be accurately grasped, thus providing a precise input basis for subsequent action planning. By inputting the analysis results into a pre-trained operation decision model to generate action planning instructions, task operation strategy reasoning can be performed based on multiple factors in complex scenarios, thereby improving the rationality of action planning and the accuracy of loading and unloading execution. 2. By executing task scheduling operations based on the current task status and resource information, and generating scheduling results that include task allocation and path control, it can dynamically coordinate multi-robot resources and rationally allocate task loads, thereby improving overall operation efficiency and scheduling flexibility; by monitoring the operation status in real time during the loading and unloading process and dynamically adjusting the action instructions according to the interference situation, it can effectively respond to sudden interference during the operation, thereby ensuring the continuity and robustness of the loading and unloading process. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of an intelligent robot loading and unloading method in one embodiment of the present application; Figure 2 This is a flowchart for implementing step S10 in an intelligent robot loading and unloading method in one embodiment of the present application; Figure 3 This is another implementation flow chart of an intelligent robot loading and unloading method in one embodiment of the present application; Figure 4 This is another implementation flow chart of step S20 in an intelligent robot loading and unloading method in one embodiment of the present application; Figure 5 This is a flowchart for implementing step S30 in an intelligent robot loading and unloading method in one embodiment of the present application; Figure 6 This is another implementation flow chart of step S30 in an intelligent robot loading and unloading method in one embodiment of the present application; Figure 7 This is a flowchart for implementing step S50 in an intelligent robot loading and unloading method in one embodiment of the present application; Figure 8 This is a principle block diagram of an intelligent robot loading and unloading system in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The present application is further described in detail below with reference to the accompanying drawings.

[0025] In one embodiment, if Figure 1 As shown, the present application discloses an intelligent robot loading and unloading method, which specifically includes the following steps: S10: Acquire image information of the target material, perform recognition analysis on the image information of the target material, and obtain analysis results, which include position information and posture information of the target material.

[0026] Specifically, by calling the industrial vision acquisition interface, the image frame of the area where the target material is located is obtained, and image preprocessing operations are performed, including denoising enhancement, brightness normalization and edge filtering, and then the key material area image in the target area is extracted. Then, combined with the shape template matching and feature point extraction algorithm, the contour boundary and center position of the material are located, and then the spatial mapping operation is performed to convert the image coordinates into three-dimensional coordinate information, and the material orientation angle is extracted as posture information. For example, when processing a tilted metal part, the image recognition module can calculate the X, Y, and Z coordinates of its center position in the worktable coordinate system and its rotation angle around the Z axis, thereby fully obtaining the position information and posture information of the target material.

[0027] S20: The analysis results are input into a pre-trained operation decision model for analysis, and an action planning instruction corresponding to the target material is generated.

[0028] Specifically, by assembling the aforementioned material position information and posture information into a structured input vector and inputting it into a pre-trained job decision model, the model is built based on a deep neural network structure. It can predict the optimal job action combination based on the target position and angle information, and infer multiple action parameters for grasping, carrying and placing. For example, when it is identified that the target material is facing left and close to the edge of the workbench, the model will output an action sequence of first adjusting the gripper angle and then grasping it sideways, and further generate the timestamp and execution sequence information of each action, thereby forming a complete action planning instruction for the robot control module to directly call and execute.

[0029] S30: Execute task scheduling operations based on the current task status and resource information to obtain task scheduling results. The scheduling operations include task allocation optimization and path generation control.

[0030] Specifically, after the action planning is completed, the priority, execution progress and robot resource distribution of each current task are read in combination with the task status record table and the resource scheduling table, and it is analyzed whether there is currently a task conflict, resource competition or robot idle waiting situation, and the task allocation matrix is ​​calculated based on the scheduling optimization rules. For example, when the system detects that two robots are idle at the same time and a material handling task is in a waiting state, the scheduling module will compare and allocate the two robots based on the last execution efficiency and the current position of the two robots, and at the same time construct a path grid from the current position of the robot to the location of the material, perform the shortest cost path search in the path search graph, output the path point list and the corresponding movement instructions, and finally generate a complete task scheduling result for use by the subsequent execution module.

[0031] S40: Schedule the target robot to complete the loading and unloading operations according to the task scheduling results and the motion planning instructions.

[0032] Specifically, based on the generated task scheduling results and action planning instructions, the scheduling execution control module is called to bind the target task and action sequence to the specified robot ID, and send it to the robot control unit through the control bus. The control unit parses the instructions one by one and drives each actuator to perform collaborative operations. For example, when the scheduling result indicates that robot A should perform the task of transporting material X from workstation 1 to workstation 2, the robot will complete the gripper closing, lifting, moving, positioning and placement actions in sequence after reading the target position and path points. During the whole process, each action is controlled and executed according to the preset time constraints and path points in the action instructions, and after execution, the feedback signal is used to confirm whether the corresponding action is completed.

[0033] S50: During the loading and unloading process, the operation status is monitored in real time, the operation status is detected and analyzed, interference detection results are obtained, and the action planning instructions are dynamically adjusted based on the interference detection results.

[0034] Specifically, during the process of the robot performing loading and unloading tasks, it continuously collects robot feedback data and operation status information, including posture feedback, grasping success flag, path point passing status and time-consuming statistics, etc. These data are then compared with the task expectations to determine whether there are abnormal actions, path deviations or long execution delays. For example, if it is detected that the grasping flag is not set after the grasping action is executed, it is judged as a grasping failure. The dynamic adjustment mechanism is then triggered based on this interference event to recalculate the grasping path or switch the grasping posture, update the current task action instructions and continue to execute in the next cycle to ensure that the task completion rate and execution continuity can be maximized even in the case of interference.

[0035] S60: During the robot operation, the robot identifies the approach behavior of a person based on the environmental perception information, and performs a safety control operation according to the identification result. The safety control operation includes switching the robot's motion mode.

[0036] Specifically, while the robot is performing its tasks, it periodically acquires human behavior perception data in the surrounding environment, including infrared sensing data, depth camera images, and proximity sensor outputs. It uses a human detection algorithm to determine whether a person has entered the robot's working radius, and further identifies the person's movement trend and distance threshold. When it is determined that a person is approaching the working area, the robot control logic is triggered to switch to safety mode. For example, if a staff member is detected appearing 0.8 meters in front of the robot and staying for more than 1 second, the current action is immediately paused and the robot enters a low-speed standby state. At the same time, the warning light is turned on and an alarm message is sent to the upper management end, thereby effectively ensuring operational safety and timely response in a human-machine collaborative environment.

[0037] By adopting the above technical solution, by acquiring the image information of the target material and extracting its position information and posture information, it is possible to accurately grasp the three-dimensional spatial state of the target material, thereby providing a precise input basis for subsequent action planning; by inputting the analysis results into a pre-trained job decision model to generate action planning instructions, it is possible to infer task operation strategies based on multiple factors in complex scenarios, thereby improving the rationality of action planning and the accuracy of loading and unloading execution; by performing task scheduling operations based on the current task status and resource information, and generating scheduling results including task allocation and path control, it is possible to dynamically coordinate multi-robot resources and reasonably allocate task loads, thereby improving overall operation efficiency and scheduling flexibility; by monitoring the operation status in real time during the loading and unloading process and dynamically adjusting the action instructions according to the interference situation, it is possible to effectively respond to sudden interference situations during the operation process, thereby ensuring the continuity and robustness of the loading and unloading process; by identifying human approach behavior and switching the robot motion mode to perform safety control, it is possible to reduce safety risks in the human-computer interaction process, thereby improving the safety of the working environment.

[0038] In one embodiment, if Figure 2 As shown, in step S10, the image information of the target material is identified and analyzed to obtain the analysis results, which specifically include: S11: Perform image preprocessing on the image information to extract the image features of the target material in the image. The image preprocessing includes image segmentation and feature extraction. The image features include edge features, texture features and color distribution information.

[0039] Specifically, by calling the image preprocessing module, a series of preprocessing operations are performed on the acquired original image data. First, image denoising and sharpening are performed to improve the clarity of the image boundary. Secondly, image enhancement is performed to improve the contrast and color saturation. Then, the image segmentation stage is entered. Preliminary region division is performed based on the grayscale value gradient, and the foreground and background are separated in combination with dynamic threshold adjustment. The gradient direction histogram and texture direction response function are further used to extract the edge features and texture features of the target material. At the same time, the main color distribution in the image area is statistically analyzed to obtain color information. For example, when detecting a red plastic part, its curved edge, surface stripe structure and main red pixel aggregation area can be extracted. Finally, a feature vector containing the above three types of image features is output for subsequent target recognition analysis.

[0040] S12: Identify image features based on a target detection algorithm to identify target material areas in the image information.

[0041] Specifically, by calling the target detection algorithm, a feature map is constructed based on the image feature input obtained in the previous stage, and the position of potential target materials is judged through the high-response area in the feature map. A multi-scale convolution kernel detection strategy is executed to enhance the recognition ability of targets of different sizes. In combination with the anchor box mechanism, multiple candidate regions are generated in the image. The probability of the existence of the target material is determined by the confidence scoring function in the candidate region, and the region with the highest confidence is screened out as the final detection result. For example, when there are multiple materials of similar colors in the image at the same time, the detection algorithm can accurately locate the target material region with edge irregularities and texture uniqueness and mark the bounding box, thereby providing an accurate area range for subsequent spatial pose extraction.

[0042] S13: Based on the target material area, combined with the spatial reconstruction method, the position information and posture information of the target material are extracted.

[0043] Specifically, after identifying the target material area, a three-dimensional space reconstruction operation is performed based on the pixel coordinates of the area in the image through the mapping relationship between the image and the physical space. First, the calibration matrix and the camera intrinsic parameters are extracted for perspective transformation. Then, stereo vision or structured light depth data is used to obtain the actual coordinate information of the area in the three-dimensional space, and by fitting the relative position between its edge contour and the reference plane, the rotation angle and tilt direction of the target material are calculated. For example, when identifying a tilted gear part, the position of its center point in the workbench coordinate system and the angle information of its rotation around the Z axis can be obtained, thereby accurately outputting the position information and posture information of the target material for use in the motion planning model.

[0044] In one embodiment, if Figure 3 As shown, before step S20, that is, before the analysis result is input into the pre-trained operation decision model for analysis, the intelligent robot loading and unloading method further includes: S201: Collect multiple sets of sample data including image features, position information and posture information, and construct label data corresponding to the target action sequence.

[0045] Specifically, the sampling module records data on the process of loading and unloading operations on multiple groups of target materials under different scenarios and task conditions. During the acquisition process, each group of data contains image feature information, the position coordinates of the target material in the work platform coordinate system, the Euler angle representation or quaternion representation of the target posture, and the corresponding loading and unloading action sequence, where the action sequence is represented in the form of multi-step instructions including parameters such as grasping point selection, end effector orientation, trajectory path points, etc. At the same time, combined with manual annotation or existing motion execution records, label information matching each group of data is extracted to form a sample-label pair. For example, when collecting a group of operation data for transporting cylindrical parts, its contour features in the visual image, three-dimensional position and tilt angle in the workspace are recorded, and an action label sequence containing six-degree-of-freedom operation targets is generated, finally forming a data set that meets the requirements of supervised training.

[0046] S202: Input the sample data and label data into the constructed deep neural network model for training. The deep neural network model includes an encoder structure and an action decoding structure.

[0047] Specifically, the sample data collected above is sent as the input tensor into the constructed deep neural network model. Before the input layer, the encoder structure is called to uniformly format the image features, position information and posture information, and map them into low-dimensional potential representation vectors. After that, the key semantic features are extracted layer by layer through a series of hidden layers and passed into the decoder structure for action sequence prediction. The action decoding structure uses a recurrent neural network or a variant structure such as LSTM, GRU, etc. to model the sequence output, thereby realizing the learning of the complex mapping relationship between the input state and the target action. For example, after inputting a set of typical images of corner and special-shaped parts and their posture parameters, it can predict the adsorption point grasping method and the action sequence of the corresponding path points that should be used for the part, thereby establishing a learning bridge between training data and task behavior.

[0048] S203: Optimize the deep neural network model by minimizing the action prediction error as the loss function until the model converges to obtain a pre-trained task decision model.

[0049] Specifically, during the training process, the difference between the sample action label and the model output is used as the prediction error, and the error is measured by the mean square error, cross entropy or sequence loss function. The error value is used as the objective function for backpropagation to optimize the network parameters. After each round of iteration, the average loss change on the training set and the validation set is calculated. When the loss converges for multiple consecutive rounds or the rate of change is less than the set threshold, the training process is terminated and the model weight parameters are solidified to obtain the final job decision model. For example, in the early stages of training, the model may not be able to accurately predict path instructions, but as the number of iterations increases, the path sequence output by the model gradually approaches the actual action trajectory, and finally it can stably generate action instructions that match the actual operation, with good generalization performance and deployability.

[0050] In one embodiment, if Figure 4 As shown, in step S20, the analysis results are input into a pre-trained operation decision model for analysis to generate action planning instructions corresponding to the target material, specifically including: S21: Performing structured coding on the analysis results. The structured coding includes converting the position information and posture information of the target material into a unified data vector format.

[0051] Specifically, after completing image recognition and three-dimensional reconstruction, structured coding operations are performed on the position information and posture information of the target material, including standardizing the three-dimensional position coordinates, uniformly mapping them to a normalized space with the robot base coordinate system as a reference, and representing the posture information in the form of quaternions to eliminate singularity problems that may be caused by Euler angles. The position vector and posture vector are then spliced ​​into a unified feature vector format, which serves as the basic data structure for the input of the operation model. For example, when the target material is located at the edge of the pallet and has a large posture inclination angle, the Z-axis rotation component reflected in its structured vector will deviate significantly from the standard orientation, making it easier for subsequent models to perceive the particularity of the operation scenario.

[0052] S22: Input the structured code into the job decision model for forward reasoning to obtain job execution information that matches the loading and unloading tasks.

[0053] Specifically, after obtaining the structured encoding result, it is sent as an input tensor to the pre-trained job decision model to perform the forward reasoning process. The model activates multiple network layers to extract deep semantic features based on the feature dimensions corresponding to the encoding representation, and generates execution information representing the intention of the loading and unloading task at the output layer. The execution information includes semantic descriptions such as the operation stage identification, the end action type, and the key control point estimation results. For example, in a certain execution process, when the structured vector reflects that the target material is at the boundary position and the grasping surface is skewed, the model reasoning output result includes labels such as "lateral grasping" and "end rotation", indicating that the robot should adjust its posture to adapt to the target characteristics.

[0054] S23: Generate action planning instructions based on the job execution information to drive the target robot to perform corresponding loading and unloading operations.

[0055] Specifically, according to the semantic element combination rules in the job execution information, the action type, path structure and target parameters are deconstructed and reorganized item by item to generate action planning instructions that meet the requirements of the robot execution interface. The action planning instructions include information such as the robot's starting state, grasping point position, end orientation and path interpolation strategy, and are output in a preset control protocol format. For example, for cylindrical parts that need to be transported, the action planning instructions will specify the robot to grasp the upper center point by adsorption, and insert obstacle avoidance interpolation points in the path planning to bypass surrounding obstacles, ensuring that the subsequent execution module can smoothly drive the robot to complete the loading and unloading process according to the instruction flow.

[0056] In one embodiment, if Figure 5 As shown, in step S30, i.e., task allocation optimization, specifically includes: S31: Acquire the operation status and available resource information of each target robot as robot status information.

[0057] Specifically, after receiving the action planning request, the current operation status information of each target robot is collected, including the current task execution status, geographical location, motion control status and available status of the end effector. At the same time, the available resource information associated with it is also collected, such as the type of gripping tool, current remaining power, operation path constraints and network connection stability indicators. These data are uniformly encapsulated into a structured robot state vector. For example, when a robot is in a high-load state, but there is still a certain task window available, and its actuator is within the standard working condition range, the load weight field in its state vector can reduce the probability of it being scheduled first, thereby providing support for overall resource optimization.

[0058] S32: Extracting task queue, priority information and resource occupancy information as task information.

[0059] Specifically, on the task management side, the task queue information of the tasks to be executed is extracted, including basic parameters such as task number, operation type, required completion time window, target material identification and corresponding spatial coordinates. At the same time, the priority identification of each task is extracted. The priority is determined by factors such as the source of the task, whether the task is a periodic task or an alarm task, and whether there is a waiting dependency in the upstream process. The gripper resources, channel resources and path overlap information that may be involved in this batch of tasks are further integrated to form a resource occupancy parameter table. For example, a batch of thermal parts tasks are marked as high priority in the task queue and involve high-temperature fixtures. This information can be directly involved in scheduling decisions.

[0060] S33: Based on the task information and the robot status information, a matching relationship between the task and the robot is established. The matching relationship comprehensively considers the urgency of the task, the idleness of the robot, and the historical execution effect.

[0061] Specifically, based on the acquired task information and robot status information, the matching operation between the task and the robot is performed. During the matching process, the urgency of the task, the current idleness of the robot, the success rate of historical tasks and the execution time are comprehensively considered. A weighted scoring strategy is used to calculate the adaptation score of each task and the available robots, and a candidate list is generated from high to low according to the matching priority. For example, if a task is a high-priority task and the target is located in an area close to robot A, and robot A is currently idle and the success rate of the last three similar tasks is 100%, then its matching score is significantly higher than that of other robots, and the task can be scheduled to robot A for execution.

[0062] S34: Generate a task allocation result based on the matching relationship, and use the task allocation result to guide robot scheduling execution.

[0063] Specifically, based on the list of candidate robots obtained by the matching operation, the optimal robot is selected as the execution carrier of the task, and a task allocation result including the task number, assigned robot identifier, expected start and end time, execution posture requirements, etc. is generated. The task allocation result will be passed into the robot control interface as the basic parameter for the scheduler to subsequently execute control instructions. For example, in the multi-station scheduling process in the workshop, the scheduling module instructs robot B to go to the designated station to grab materials and complete the transportation according to the task allocation result. During this period, the scheduling process is ensured to be continuous, smooth and conflict-free based on the assigned control window and operation posture information.

[0064] In one embodiment, if Figure 6 As shown, in step S30, i.e., path generation control, specifically includes: S35: Based on the workshop layout map and real-time environmental perception data, a path search space is constructed from the current position of the target robot to the mission target point.

[0065] Specifically, after completing the robot scheduling and allocation of the target task, the initial position information of the current target robot and the coordinates of the operation target point of the corresponding task are obtained, and the static spatial obstacle information is extracted according to the preset workshop space layout diagram. At the same time, the dynamic obstacle distribution status is obtained by combining the environmental perception sensors deployed on site (such as lidar, visual cameras, etc.). The static obstacle map and dynamic perception data are used to construct a complete feasible path space. The path search space uses a rasterized spatial model or a continuous surface graph structure to express the range of the robot's passable area in a two-dimensional or three-dimensional scene. For example, when the robot is currently at workstation A and the target point is at workstation B, and there is an operating manual collaborative equipment near workstation B, the area will be marked as a high-risk grid to limit the possibility of crossing during the path generation process.

[0066] S36: Generate multiple feasible paths in the path search space, and perform cost evaluation operations on the feasible paths to obtain cost evaluation results, where the cost evaluation includes path length, number of obstacle avoidance times, and execution delay.

[0067] Specifically, in the constructed path search space, a heuristic search algorithm such as the A algorithm or the improved D algorithm is used to generate multiple candidate paths to the target point, and a multi-dimensional cost calculation is performed on each path. The cost calculation includes quantitative indicators such as the basic path length indicator, the estimated execution time required for the path, the number of obstacle avoidance actions expected to occur in the path, and path continuity and complexity. All paths will be uniformly evaluated according to the set cost function formula. For example, when path 1 is shorter but has more obstacle avoidance points and more intersections, while path 2 is slightly longer but has a smoother path and less obstacle avoidance intervention, path 2 may obtain a better score in the comprehensive cost evaluation and be used for subsequent optimal target path.

[0068] S37: Based on the cost evaluation result, the target path is selected as the final path generation control result, which is used to drive the target robot to perform task migration.

[0069] Specifically, after completing the cost evaluation of all feasible paths, the path with the lowest total cost or the highest safety factor is selected from the candidate paths as the target path according to the preset cost sorting strategy. The key control point sequence of the target path and the corresponding action instructions are combined into a path generation control result, which is output to the motion control module of the target robot to guide it to perform task migration operations along the path. For example, when the target path contains multiple turning nodes with large rotation angles, the path control result will include fine speed adjustment instructions and obstacle avoidance buffer adjustment parameters to ensure the continuity and safety of the robot's movements during the task migration process.

[0070] In one embodiment, if Figure 7As shown, in step S50, the operation status is detected and analyzed to obtain interference detection results, and the action planning instructions are dynamically adjusted based on the interference detection results, specifically including: S51: Obtain the execution feedback data and task process status information of the target robot as operation status data.

[0071] Specifically, during the robot's loading and unloading operations, the execution feedback data of the target robot is periodically collected, including the execution instruction confirmation status, the robotic arm position change data, the grasping feedback signal of the end effector, and the contact status returned by the visual or force sensor in real time. At the same time, the task process status information is synchronously recorded. The task process status information includes the current task stage identifier, the completed action mark, and the action queue that has not yet been executed. By fusing the above multi-dimensional data to form the operation status data, for example, during a grasping process, when it is detected that the end effector has not closed to the expected position, and the position sensor shows that the current grasping point deviates from the original mark, it can be determined that there is a deviation in the current state and further judgment is needed whether it is abnormal.

[0072] S52: Detecting whether there are abnormal events based on the operation status data. Abnormal events include target material position deviation, grasping failure, path obstruction, or abnormal execution delay.

[0073] Specifically, the collected operation status data is analyzed and processed, and rule matching logic or anomaly detection model is used to determine whether there are predefined abnormal events. The abnormal events include the offset of the target material between the actual position and the initial identification position exceeding the set threshold, the robot end failing to detect a valid grasping signal and determining a grasping failure, the presence of dynamic obstacles in the task path causing the path to be blocked, the task execution time being significantly higher than the average task time, etc. Automatic abnormality identification can be achieved by comparing various status indicators with historical task models or safety boundaries. For example, if no position confirmation feedback is received within 5 seconds after the task is executed, and the grasping signal is in a failed state, it can be determined that the current abnormality is a "grasping failure" type.

[0074] S53: When an abnormal event is detected, a corresponding interference detection result is generated, and the current action planning instruction is adjusted based on the interference detection result. The adjustment includes replanning the grasping path, switching to a backup task, or delaying the task execution time.

[0075] Specifically, when any of the above-mentioned abnormal events is detected in the operation status, a corresponding interference detection result is immediately generated. The interference detection result marks the current abnormality type, location coordinates and related context information, and calls the action adjustment strategy according to the abnormality type. The adjustment strategy includes re-executing the target material identification and generating a new grasping path to replace the current action, temporarily switching to a backup task or a low-priority task to maintain resource utilization when the path is continuously blocked, or delaying the task execution time and dynamically adjusting the task scheduling queue priority when the equipment resources are temporarily unavailable. For example, when manual interference is detected in the front during the execution path and cannot be cleared immediately, the scheduling mechanism will suspend the current task and temporarily assign the robot to the side to perform a low-priority material handling task, and then roll back the original task execution path after the obstacle is removed, thereby ensuring the stability and robustness of the operation process.

[0076] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0077] In one embodiment, an intelligent robot loading and unloading system is provided, and the intelligent robot loading and unloading system corresponds to an intelligent robot loading and unloading method in the above embodiment. Figure 8 As shown, the intelligent robot loading and unloading system includes an image recognition module, an operation decision module, a scheduling decision module, an action execution module, an interference monitoring module, and a safety perception module. The functional modules are described in detail as follows: An image recognition module is used to obtain image information of a target material, perform recognition and analysis on the image information of the target material, and obtain analysis results, which include position information and posture information of the target material; The operation decision module is used to input the analysis results into the pre-trained operation decision model for analysis and generate action planning instructions corresponding to the target material; The scheduling decision module is used to perform task scheduling operations based on the current task status and resource information to obtain task scheduling results. The scheduling operations include task allocation optimization and path generation control; The action execution module is used to schedule the target robot to complete loading and unloading operations based on the task scheduling results and action planning instructions; The interference monitoring module is used to monitor the operation status in real time during the loading and unloading process, detect and analyze the operation status, obtain interference detection results, and dynamically adjust the action planning instructions based on the interference detection results; The safety perception module is used to identify human approach behavior based on environmental perception information during the robot's operation and perform safety control operations based on the recognition results. The safety control operations include switching the robot's motion mode.

[0078] Optionally, the image recognition module includes: The image preprocessing submodule is used to preprocess the image information and extract the image features of the target material in the image. The image preprocessing includes image segmentation and feature extraction. The image features include edge features, texture features and color distribution information. The target detection submodule is used to identify image features based on the target detection algorithm and identify the target material area in the image information; The posture analysis submodule is used to extract the position information and posture information of the target material based on the target material area and combined with the spatial reconstruction method.

[0079] Optionally, the intelligent robot loading and unloading system further includes: The sample construction module is used to collect multiple sets of sample data containing image features, position information and posture information, and construct label data corresponding to the target action sequence; The model training module is used to input sample data and label data into the constructed deep neural network model for training. The deep neural network model includes an encoder structure and an action decoding structure; The model optimization module is used to optimize the deep neural network model by minimizing the action prediction error as the loss function until the model converges and obtains a pre-trained job decision model.

[0080] Optionally, the job decision module includes: The structured coding submodule is used to perform structured coding on the analysis results. Structured coding includes converting the position information and posture information of the target material into a unified data vector format; The model reasoning submodule is used to input the structured code into the job decision model for forward reasoning to obtain job execution information that matches the loading and unloading tasks; The instruction generation submodule is used to generate action planning instructions based on the job execution information, which are used to drive the target robot to perform corresponding loading and unloading operations.

[0081] Optionally, the scheduling decision module includes: The robot status acquisition submodule is used to obtain the operation status and available resource information of each target robot as the robot status information; The task information extraction submodule is used to extract task queue, priority information and resource occupancy information as task information; The task matching relationship construction submodule is used to establish a matching relationship between tasks and robots based on task information and robot status information. The matching relationship comprehensively considers the task urgency, robot idleness and historical execution results; The task allocation submodule is used to generate task allocation results based on the matching relationship and use the task allocation results to guide robot scheduling execution; The path search space construction submodule is used to construct the path search space from the current position of the target robot to the mission target point based on the workshop layout map and real-time environmental perception data; The path cost evaluation submodule is used to generate multiple feasible paths in the path search space and perform cost evaluation operations on the feasible paths to obtain cost evaluation results. The cost evaluation includes path length, number of obstacle avoidance times and execution delay. The path selection control submodule is used to select the target path as the final path generation control result based on the cost evaluation result, which is used to drive the target robot to perform task migration.

[0082] Optionally, the interference monitoring module includes: The state acquisition submodule is used to obtain the execution feedback data and task process status information of the target robot as the operation status data; The anomaly detection submodule is used to detect abnormal events based on the job status data. Abnormal events include target material position deviation, grasping failure, path obstruction, or abnormal execution delay; The instruction adjustment submodule is used to generate corresponding interference detection results when an abnormal event is detected, and adjust the current action planning instructions based on the interference detection results. The adjustments include replanning the grasping path, switching to backup tasks, or delaying the task execution time.

[0083] The specific definition of an intelligent robot loading and unloading system can be found in the definition of an intelligent robot loading and unloading method above, and will not be repeated here. Each module in the above-mentioned intelligent robot loading and unloading system can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0084] Those skilled in the art will clearly understand that for the sake of convenience and brevity in description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0085] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An intelligent robot loading and unloading method, characterized in that: The intelligent robot loading and unloading method comprises: Acquire image information of a target material, perform recognition analysis on the image information of the target material, and obtain an analysis result, wherein the analysis result includes position information and posture information of the target material; Inputting the analysis results into a pre-trained operation decision model for analysis to generate action planning instructions corresponding to the target material; Perform task scheduling operations based on current task status and resource information to obtain task scheduling results, wherein the scheduling operations include task allocation optimization and path generation control; Scheduling the target robot to complete loading and unloading operations according to the task scheduling results and the action planning instructions; During the loading and unloading process, the operation status is monitored in real time, the operation status is detected and analyzed, interference detection results are obtained, and the action planning instructions are dynamically adjusted based on the interference detection results; During the robot operation, the robot recognizes the approaching behavior of people based on the environmental perception information, and performs safety control operations according to the recognition results, and the safety control operations include switching the robot's motion mode.

2. The intelligent robot loading and unloading method according to claim 1, characterized in that: The identifying and analyzing the image information of the target material to obtain the analysis result includes: Performing image preprocessing on the image information to extract image features of the target material in the image, wherein the image preprocessing includes image segmentation and feature extraction, and the image features include edge features, texture features, and color distribution information; Identifying the image features based on a target detection algorithm to identify the target material area in the image information; Based on the target material area, combined with a space reconstruction method, the position information and posture information of the target material are extracted.

3. The intelligent robot loading and unloading method according to claim 1, characterized in that: Before inputting the analysis results into a pre-trained operation decision model for analysis, the intelligent robot loading and unloading method further includes: Collect multiple sets of sample data containing image features, position information, and posture information, and construct label data corresponding to the target action sequence; Inputting the sample data and label data into a constructed deep neural network model for training, wherein the deep neural network model includes an encoder structure and an action decoding structure; The deep neural network model is optimized by minimizing the action prediction error as a loss function until the model converges, thereby obtaining the pre-trained task decision model.

4. The intelligent robot loading and unloading method according to claim 1, characterized in that: Inputting the analysis results into a pre-trained operation decision model for analysis to generate an action planning instruction corresponding to the target material includes: Performing structured coding on the analysis results, wherein the structured coding includes converting the position information and posture information of the target material into a unified data vector format; Inputting the structured code into the operation decision model for forward reasoning to obtain operation execution information matching the loading and unloading tasks; Based on the operation execution information, action planning instructions are generated to drive the target robot to perform corresponding loading and unloading operations.

5. The intelligent robot loading and unloading method according to claim 1, characterized in that: The task allocation optimization includes: Obtain the operation status and available resource information of each target robot as robot status information; Extract task queue, priority information and resource occupancy information as task information; Establishing a matching relationship between the task and the robot based on the task information and the robot status information, wherein the matching relationship comprehensively considers the urgency of the task, the robot's idleness, and historical execution results; A task allocation result is generated based on the matching relationship, and the task allocation result is used to guide robot scheduling execution.

6. The intelligent robot loading and unloading method according to claim 1, characterized in that: The path generation control includes: Based on the workshop layout and real-time environmental perception data, a path search space is constructed from the target robot's current position to the mission target point; Generating multiple feasible paths in the path search space, and performing a cost evaluation operation on the feasible paths to obtain the cost evaluation result, wherein the cost evaluation includes path length, number of obstacle avoidance times, and execution delay; According to the cost evaluation result, the target path is selected as the final path generation control result, which is used to drive the target robot to perform task migration.

7. The intelligent robot loading and unloading method according to claim 1, characterized in that: The detecting and analyzing the operation status to obtain an interference detection result, and dynamically adjusting the action planning instruction based on the interference detection result includes: Obtain the execution feedback data and task process status information of the target robot as job status data; Detecting whether there are abnormal events based on the operation status data, wherein the abnormal events include target material position deviation, grasping failure, path obstruction or execution delay abnormality; When an abnormal event is detected, a corresponding interference detection result is generated, and the current action planning instruction is adjusted based on the interference detection result. The adjustment includes replanning the grasping path, switching to a backup task, or delaying the task execution time.

8. An intelligent robot loading and unloading system, characterized in that: The intelligent robot loading and unloading system comprises: An image recognition module is used to obtain image information of a target material, perform recognition analysis on the image information of the target material, and obtain an analysis result, wherein the analysis result includes position information and posture information of the target material; An operation decision module is used to input the analysis results into a pre-trained operation decision model for analysis, and generate an action planning instruction corresponding to the target material; A scheduling decision module is used to perform task scheduling operations based on current task status and resource information to obtain task scheduling results. The scheduling operations include task allocation optimization and path generation control. An action execution module is used to schedule the target robot to complete loading and unloading operations according to the task scheduling results and the action planning instructions; An interference monitoring module is used to monitor the operation status in real time during the loading and unloading process, detect and analyze the operation status, obtain interference detection results, and dynamically adjust the action planning instructions based on the interference detection results; The safety perception module is used to identify human approach behavior based on environmental perception information during the robot operation process, and perform safety control operations according to the recognition results. The safety control operations include switching the robot's motion mode.

Images