A robotic arm control system for spline injection molding

Abstract

The present invention relates to a robotic arm control system for spline injection molding. The control system batch-acquires historical robotic arm operating data, including position information of each robotic arm joint and end effector, robotic arm force, and appearance images of injection-molded spline products. A random forest model is trained using the robotic arm's position information and force as input variables and the corresponding working steps as output labels. The trained random forest model is used to determine the current robotic arm's working step. Based on the current robotic arm's working step and combined with the detection results of the spline product's appearance image by a machine vision model, the next working step is determined. Finally, a control instruction is generated based on the information about the next working step to control the robotic arm's motion state. The present invention solves the problem in existing technologies of difficulty in determining the next working step based on the robotic arm's operating data, resulting in inaccurate control of spline injection molding.

Description

Technical Field

[0001] The invention belongs to the technical field of injection molding control and relates to a robotic arm control system for spline injection molding. Background Art

[0002] In the field of injection molding production, traditional manual operation methods are gradually being replaced by robotic arm technology due to their low efficiency, prone to fatigue errors and insufficient precision control, making it difficult to meet the increasingly high product precision requirements and expanded production scale.

[0003] However, the existing technology has the problem of difficulty in automatically identifying the current working link based on the working data of the robot arm and judging the next working link based on the detection information of the spline product, resulting in insufficient automation and inaccuracy in the spline injection molding control. Summary of the Invention

[0004] The present invention provides a robotic arm control system for spline injection molding. The aim is to improve the relevant performance of the robotic arm body unit, drive unit, and sensor unit in the hardware module, optimize the functions of various parts such as the motion control unit, human-computer interaction unit, and data management unit in the software module, and add a reliable safety control unit. This allows the robotic arm to accurately complete various action tasks such as taking and placing splines, handling, and product testing during the spline injection molding production process, reduce the defective rate, avoid equipment damage and safety accidents, and better adapt to different injection molding tasks and production scenarios.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] The present application provides a robotic arm control system for spline injection molding, comprising a hardware module and a software module; the hardware module comprises a robotic arm body unit, a drive unit and a sensor unit,

[0007] The software module includes a motion control unit, a human-computer interaction unit, and a data management unit that are communicatively connected; the data management unit includes a data acquisition subunit, a data analysis subunit, and a data storage subunit; the data acquisition subunit is used to collect working data during the operation of the sensor unit robot arm, the working data including position information of each joint and end effector of the robot arm, the robot arm force, and the appearance image of the injection molded spline product; the robot arm force includes the gripping force of the end effector grasping the injection molded spline and the contact force between the robot arm and surrounding objects;

[0008] The data analysis subunit is used to analyze the working state of the robot arm according to the collected working data, and generate a control instruction according to the working state and send it to the motion control unit to control the motion state of the robot arm.

[0009] Furthermore, the sensor unit includes a position sensor, a force sensor and a vision sensor. The position sensor is used to provide real-time feedback on the position information of each joint and the end effector of the robotic arm. The force sensor is used to collect the gripping force of the end effector in grasping the spline and the contact force during the interaction with the surrounding environment. The vision sensor is used to obtain the appearance image of the spline product and the position of the injection mold.

[0010] Furthermore, the motion control unit includes a trajectory planning subunit, a speed control subunit and a motion coordination subunit. The trajectory planning subunit is used to plan the motion trajectory of the robot arm joint space or Cartesian space according to the injection molding process requirements; the speed control subunit is used to control the speed and acceleration of the robot arm during the movement process; the motion coordination subunit is used to coordinate the movement sequence and time rhythm between each robot arm or the robot arm and the injection molding machine when multiple robot arms work together or the robot arm and the injection molding machine cooperate with each other.

[0011] Furthermore, the robot body unit is used to realize the transformation of spatial position and posture to complete the actions of picking up, placing and transporting splines during the injection molding process, and the drive unit is used to provide power for the movement of the robot joints; the human-computer interaction unit includes an operation interface subunit and a programming interface subunit, and the operation interface subunit provides a visual operation interface for setting parameters, sending action instructions and displaying the movement status of the robot; the programming interface subunit is used by technicians to program and modify the action flow of the robot.

[0012] Furthermore, analyzing the working status of the robot arm according to the collected working data specifically includes the following steps:

[0013] S1. Batch obtain the historical period of robot arm working data from the data storage subunit;

[0014] S2. Using the position information of each joint and end effector of the robotic arm and the force of the robotic arm as input variables and the corresponding working links as output labels, train a random forest model; the working links include pick-and-place links, handling links, and collaboration links;

[0015] S3. Use the trained random forest model to determine the current working link of the robotic arm;

[0016] S4. Determine the next working link based on the current working link of the robotic arm and the detection result of the spline product appearance image by the machine vision model.

[0017] Furthermore, the training of the random forest model includes the following steps:

[0018] Convert the position information of each joint and end effector of the robotic arm into three-dimensional coordinate data;

[0019] Convert the force data of the robotic arm into two-dimensional data of magnitude and direction;

[0020] Randomly extract N transformed input variables and build a decision tree with the output labels;

[0021] Repeat the process of building decision trees until all input variables are traversed and a random forest is generated;

[0022] Based on the predictive power of each decision tree for the output label, determine the key input variables that have significant judgment power for the work process;

[0023] Optimize the goodness of fit of the random forest based on key input variables.

[0024] Furthermore, the machine vision model is configured as a convolutional neural network model, including the following construction steps:

[0025] Collect historical spline product appearance image data samples;

[0026] The images are marked separately according to whether the spline is qualified;

[0027] Construct a convolutional network structure to extract image features from spline product appearance images;

[0028] The convolutional neural network model is trained with image features as independent variables and the corresponding annotation information as dependent variables.

[0029] Furthermore, the detection result of the spline product appearance image by the combination of the machine vision model is used to determine the next working step, which is specifically:

[0030] When the spline product test result is determined to be qualified, the next work link will be carried out according to the normal working order;

[0031] When the test result of the spline product is determined to be unqualified, the next step is to recycle the spline product.

[0032] Furthermore, the step of generating a control instruction according to the working state and sending it to a motion control unit to control the motion state of the robotic arm specifically includes the following steps:

[0033] Get information about the next work step;

[0034] Generate control instructions based on the information, including the motion trajectory, speed, acceleration of the robotic arm, and the sequence and timing of the movements of each robotic arm or between the robotic arm and the injection molding machine;

[0035] Send control instructions to the motion control unit;

[0036] The motion control unit controls the motion state of the robotic arm according to the command instructions.

[0037] Furthermore, the software module also includes a safety control unit for monitoring in real time whether the robot arm body collides with the injection molding machine and the surrounding environment, and stopping or adjusting the action in time once a collision is detected.

[0038] Beneficial effects of the present invention:

[0039] (1) The working data of the robot arm in historical periods are acquired in batches, including the position information of each joint and end effector of the robot arm, the force of the robot arm, and the appearance image of the injection molded spline product; the position information and force of the robot arm are used as input variables, and the corresponding working link is used as the output label to train a random forest model; the trained random forest model is used to judge the working link of the current robot arm; the next working link is determined based on the current working link of the robot arm and the detection result of the spline product appearance image by the machine vision model; finally, a control instruction is generated based on the information of the next working link to control the motion state of the robot arm. The present invention solves the problem that the prior art is difficult to determine the next working link based on the working data of the robot arm, resulting in inaccurate control of spline injection molding.

[0040] (2) By constructing a random forest model to quantify the relationship between work data and work links, the accuracy of judgment on the current work link can be improved, providing a theoretical basis for determining the next work link.

[0041] (3) The machine vision model detects whether the current spline product is qualified and, based on the test results, determines whether the next work step should be carried out according to the normal working order or the spline product should be recycled. The combination of the two models improves the ability to identify the working status of the robot arm, thereby improving the efficiency and accuracy of the robot arm injection molding work. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0043] Figure 1 This is a structural diagram of a robotic arm control system for spline injection molding in the present invention.

[0044] Figure 2 This is a flow chart of analyzing the working status of a robotic arm based on collected working data in one embodiment of the present invention.

[0045] Figure 3 A flowchart of a random forest model constructed in one embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0047] See also Figure 1-Figure 3 The present application provides a robotic arm control system for spline injection molding, including a hardware module and a software module; the hardware module includes a robotic arm body unit, a drive unit and a sensor unit,

[0048] The software module includes a motion control unit, a human-computer interaction unit, and a data management unit that are communicatively connected; the data management unit includes a data acquisition subunit, a data analysis subunit, and a data storage subunit; the data acquisition subunit is used to collect working data during the operation of the sensor unit robot arm, the working data including position information of each joint and end effector of the robot arm, the robot arm force, and the appearance image of the injection molded spline product; the robot arm force includes the gripping force of the end effector grasping the injection molded spline and the contact force between the robot arm and surrounding objects;

[0049] The data analysis subunit is used to analyze the working status of the robot arm based on the collected working data.

[0050] A control instruction is generated according to the working state and sent to the motion control unit to control the motion state of the robot arm.

[0051] Furthermore, analyzing the working status of the robot arm according to the collected working data specifically includes the following steps:

[0052] Step S1: batch-acquire the historical period robot arm working data from the data storage subunit;

[0053] The data storage subunit, a module that stores various long-term operational data related to the robotic arm, accumulates detailed records of the robotic arm's operations over different time periods and for various injection molding production tasks. This data, encompassing the positions of the robotic arm's joints and end effector, the forces applied by the robotic arm, and images of the molded spline products, forms the foundation for subsequent analysis.

[0054] Batch acquisition allows for the collection of sufficient and representative data samples at once. This helps to more comprehensively reflect the robot's performance under various real-world operating conditions, avoiding inaccurate subsequent analysis results due to insufficient or one-sided data. For example, in different batches of injection molding production, the specifications and materials of the splines may vary, and the robot's operation will also vary accordingly. Batch data acquisition can encompass these diverse situations, providing sufficient material for accurate analysis.

[0055] Data screening and organization: After acquiring data, data screening and organization are performed based on specific needs. For example, data points with obvious anomalies (such as those caused by sensor failure) can be removed, or data can be categorized by time sequence, task type, and other factors to facilitate more efficient model training.

[0056] Step S2: using the position information of each joint and end effector of the robotic arm and the force of the robotic arm as input variables and the corresponding working links as output labels, training a random forest model;

[0057] The positional information of the robot's joints and end effector directly reflects its posture and specific location in space, and is a key factor in determining its working phase. For example, during the pick-and-place phase, the end effector moves to a specific location to grab or place a spline; during the transport phase, its position trajectory exhibits a characteristic transition from one fixed point to another. Therefore, positional information can be very helpful in distinguishing different working phases.

[0058] Correlation of Robot Arm Forces: Robot arm forces include the gripping force of the end effector grasping the injection molded spline and the contact force with surrounding objects. During the pick-and-place phase, gripping force exhibits a specific pattern, gradually increasing until the spline is firmly grasped and decreasing to zero during placement. During the transport phase, the force is relatively stable. However, during the collaborative phase, when cooperating with other robots or equipment, the contact force varies. Therefore, force data can help more accurately define the work process.

[0059] Introduction to Random Forest Model:

[0060] Principle Overview: Random Forest is an ensemble learning algorithm based on multiple decision trees. It uses bootstrap sampling to extract multiple subsets of the original training dataset with replacement, then constructs a decision tree based on each subset. During the decision tree construction process, for each node, a subset of features (here, position information and force-related features) is randomly selected to find the optimal split point, thereby reducing the model's variance and improving generalization.

[0061] Training: Selected position information and force data are used as input variables, and corresponding work steps such as pick-and-place, handling, and collaboration are used as output labels. A large amount of historical data samples are fed into the random forest algorithm for training. Each decision tree learns a mapping between input variables and output labels. As the number of decision trees increases, the entire random forest model can integrate the judgment results of each tree to provide more accurate and stable work step judgments for newly input data.

[0062] Model evaluation and optimization:

[0063] During the training process, the model is typically evaluated using a set of metrics (such as accuracy, precision, recall, and F1 value) on a reserved validation dataset to check for overfitting or underfitting. If the model performance is found to be unsatisfactory, the model may be optimized by adjusting model parameters (such as the number of decision trees and the maximum depth of each tree), increasing the amount of data, or performing feature engineering (further processing, filtering, or combining input variables) to ensure that it can accurately determine the corresponding working steps based on the input real-time position and force information.

[0064] Furthermore, the evaluation index is calculated as follows:

[0065]

[0066] Where ACC is the accuracy rate, PPV is the precision rate, TRP is the recall rate, F1 is the F1 value, TP is the true positive rate, TN is the true negative rate, FP is the false positive rate, and FN is the false negative rate.

[0067] Step S3: Use the trained random forest model to determine the working link of the current robotic arm;

[0068] During the actual operation of the robotic arm, the sensor unit will collect the current position information of each joint and end effector of the robotic arm and the force information applied to the robotic arm in real time. These real-time data will be continuously transmitted to the data analysis sub-unit as input data for the model. The position information and force data collected in real time are sorted according to the format required by the model and input into the trained random forest model. Each decision tree in the model will perform calculations and judgments based on these input data, and then combine the output results of all decision trees (using voting) to finally determine whether the current working link of the robotic arm is the pick and place link, the handling link, or the collaborative link. For example, if most decision trees judge that the current situation meets the characteristics of the pick and place link, then the model outputs the result that the current robotic arm is in the pick and place link.

[0069] The model's assessment of the current work phase provides timely feedback to the subsequent processes of the data analysis subunit and other related system modules (such as the motion control unit). This result helps the entire system understand the robot's current working status so that appropriate decisions and adjustments can be made. For example, the motion control unit can control the robot's movements according to pre-set motion parameters for the current pick-and-place phase based on the current phase.

[0070] Step S4: Determine the next working link based on the current working link of the robotic arm and the detection result of the spline product appearance image by the machine vision model.

[0071] Detection of appearance images by machine vision models:

[0072] Image acquisition: Vision sensors capture real-time images of the injection-molded spline product. These images contain visual information such as the spline's shape, size, and surface quality. These images are then transmitted to a machine vision model for processing. The machine vision model analyzes these images using image processing algorithms (such as image recognition, feature extraction, and target detection). For example, the model detects defects in the spline (such as scratches and holes) and determines whether the spline's current position meets requirements (such as whether the angle is correct and whether it aligns with other components). These test results provide insights into the actual condition of the spline product and provide a visual basis for determining the next step in the process.

[0073] Furthermore, the sensor unit includes a position sensor, a force sensor and a vision sensor. The position sensor is used to provide real-time feedback on the position information of each joint and the end effector of the robotic arm. The force sensor is used to collect the gripping force of the end effector in grasping the spline and the contact force during the interaction with the surrounding environment. The vision sensor is used to obtain the appearance image of the spline product and the position of the injection mold.

[0074] Furthermore, the motion control unit includes a trajectory planning subunit, a speed control subunit and a motion coordination subunit. The trajectory planning subunit is used to plan the motion trajectory of the robot arm joint space or Cartesian space according to the injection molding process requirements; the speed control subunit is used to control the speed and acceleration of the robot arm during the movement process; the motion coordination subunit is used to coordinate the movement sequence and time rhythm between each robot arm or the robot arm and the injection molding machine when multiple robot arms work together or the robot arm and the injection molding machine cooperate with each other.

[0075] Furthermore, the robot body unit is used to realize the transformation of spatial position and posture to complete the actions of picking up, placing and transporting splines during the injection molding process, and the drive unit is used to provide power for the movement of the robot joints; the human-computer interaction unit includes an operation interface subunit and a programming interface subunit, and the operation interface subunit provides a visual operation interface for setting parameters, sending action instructions and displaying the movement status of the robot; the programming interface subunit is used by technicians to program and modify the action flow of the robot.

[0076] Furthermore, the training of the random forest model includes the following steps:

[0077] (1) Convert the position information of each joint and end effector of the robotic arm into three-dimensional coordinate data;

[0078] In the actual operation of a robotic arm, its joints and end effector have precise spatial positions. Representing these positions using three-dimensional coordinates is the most intuitive and convenient way to analyze and process them. Raw position information is often based on various sensor feedback data formats, which can be complex and not directly reflect spatial coordinates. To make this data more suitable for model training, conversion is required.

[0079] Specifically, the conversion work is carried out based on the structural parameters of the robot arm itself (such as the rotation angle range of each joint, the length of the connecting rod, etc.) and the pre-built robot arm kinematic model. Through a series of geometric calculations and coordinate transformations, those relative and scattered sensor readings are converted into unified coordinate values ​​in a specific three-dimensional coordinate system (such as the common Cartesian coordinate system), that is, x, y, and z coordinates. For example, for a robot arm with multiple rotational joints, key information such as the current rotation angle of each joint and the connection length between the connecting rods are known. Using the trigonometric function relationship and the rules of coordinate translation and rotation, the specific three-dimensional coordinate position of the end effector in the entire space is gradually calculated. In this way, the posture and location of the robot arm in space can be clearly and accurately presented, laying the foundation for subsequent models to use these position data to judge the working links.

[0080] (2) Convert the manipulator force data into two-dimensional data of magnitude and direction;

[0081] The force applied by the robotic arm is a vector whose magnitude and direction carry key information about its working state. To make this force data more effectively usable by the random forest model, it needs to be refined and converted into a two-dimensional data format consisting of magnitude and direction.

[0082] In actual operation, the original force data collected by the force sensor (which may be force components in multiple directions, or in the form of complex vector representations in a specific coordinate system) will be processed using corresponding mathematical methods. For example, the Pythagorean theorem is used to calculate the modulus of the force vector to determine its magnitude; then, through trigonometric functions (such as the inverse tangent function, etc.), combined with the direction setting of the coordinate system, the angle between the force direction and the positive direction of a reference coordinate axis is calculated, and then its direction angle is determined. Finally, the original force data is converted into a concise two-dimensional data form that contains key information. This form can not only fully retain information that is valuable for judging the work link, but also meet the requirements of the model input variables, which helps the subsequent decision tree to more accurately learn and judge the characteristic laws of the forces in different work links.

[0083] (3) Randomly extract N transformed input variables and construct a decision tree with the output labels;

[0084] The unique construction of the random forest model aims to improve the generalization and stability of the overall model by increasing the diversity among decision trees, thereby avoiding overfitting. Randomly sampling input variables plays a key role in this process.

[0085] After completing the conversion of position information and force data, a series of input variables are available for the model to use. At this point, N variables are randomly extracted from these converted input variable sets each time, and combined with the corresponding clearly labeled work links (such as pick-and-place links, handling links, and collaboration links as output labels), a decision tree is constructed. In the process of building a decision tree, based on the N extracted input variables and according to the intrinsic relationship between them and the output labels, the optimal splitting nodes and division rules are continuously searched (for example, different branches are divided according to the value range of a certain variable, so that the purity of different categories of output labels under the branches is maximized). The decision tree gradually learns how to accurately judge the corresponding work link based on the different values ​​of the input variables. Each decision tree is gradually grown and constructed based on the randomly extracted variables and corresponding output labels.

[0086] (4) Repeat the process of building decision trees until all input variables are traversed and a random forest is generated;

[0087] Constructing a random forest doesn't rely on a single decision tree to complete the judgment task, but rather requires the collaboration of many decision trees. Therefore, the above decision tree construction process must be repeated multiple times until all transformed input variables have the opportunity to be extracted and participate in the construction of the decision tree.

[0088] In other words, by continuously randomly extracting different combinations of input variables, decision trees with different feature learning capabilities are constructed one after another. As the number of decisions gradually increases, many decision trees are combined to form a random forest. When subsequently judging the current working link of the robot arm, the random forest can integrate the judgment results of each decision tree (for example, through voting to determine the final output category), giving full play to the advantages of each decision tree, thereby more accurately and stably judging the current working link of the robot arm. Compared with a single decision tree, this integrated approach can better cope with complex and changing actual working conditions and improve the overall judgment accuracy.

[0089] (5) Based on the predictive ability of each decision tree for the output label, determine the key input variables that have significant judgment ability for the work process;

[0090] After the random forest is constructed, although each decision tree participates in the learning process of judging the work link, in fact, different input variables play different roles in accurately judging the work link in different decision trees.

[0091] In order to further explore and utilize the most critical and influential information for judging work links, it is necessary to analyze and evaluate each decision tree. Specifically, the accuracy of each decision tree's prediction of the output label (i.e., work link) based on the input variables will be examined, which can be measured by a number of evaluation indicators (such as accuracy and recall). In this process, the contribution of different input variables to improving prediction accuracy in multiple decision trees is statistically analyzed. Those input variables that can frequently and significantly help decision trees more accurately judge work links will be identified as key input variables. These key input variables often contain deep-level characteristic laws that are closely related to work links, and are an important basis for subsequent optimization of the random forest model and improving its overall judgment ability.

[0092] (6) Optimize the goodness of fit of random forest based on key input variables.

[0093] After determining the key input variables, the random forest model can be optimized and adjusted based on these variables to improve its goodness of fit, that is, to enable the model to better judge the working links based on the input variables.

[0094] There are many specific optimization methods. For example, when constructing a decision tree or adjusting an existing decision tree, the probability of extracting key input variables can be increased, so that the decision tree can learn and divide more based on these important variables, and enhance the accuracy of its judgment of the work links. Or, according to the key input variables, some parameter settings of the decision tree (such as the conditions for node splitting, the depth of the tree, etc.) can be readjusted to make the decision tree structure more adaptable to the characteristic laws reflected by these key variables. Through such a series of optimization operations, the entire random forest model can be more accurate and stable when using input variables to judge the working links of the robotic arm, and better adapt to different actual work scenarios, thereby providing a reliable basis for the robotic arm control system to more reasonably plan subsequent work links.

[0095] Furthermore, the machine vision model is configured as a convolutional neural network model, including the following construction steps:

[0096] (1) Collect spline product appearance image data samples from historical periods;

[0097] During the injection molding process, a large number of images of spline product appearances produced in different batches and under different operating conditions accumulate over time. These historical image data samples form the foundation for building machine vision models. Collecting these images helps cover a wide range of possible spline appearance conditions, such as different shapes, colors, surface textures, and possible defect types. This allows the model to learn a rich and diverse range of spline product appearance characteristics to cope with the complex and changing conditions in actual production.

[0098] For example, the collected images may include complete and defect-free splines with a smooth appearance and standard dimensions; there may also be images of splines with various defects such as scratches, holes, and deformations; as well as images acquired under different lighting conditions and shooting angles, which fully reflect the appearance of spline products in actual production scenarios and provide sufficient data support for the accurate training of subsequent models.

[0099] (2) Label the images according to whether the splines are qualified;

[0100] After collecting images of the spline product's appearance, each image needs to be accurately annotated. This annotation is based on the key criteria of whether the spline is qualified. A qualified spline means its appearance meets production process requirements and has no defects that affect product quality and subsequent use. An unqualified spline, on the other hand, has issues such as surface flaws and shape deviations.

[0101] Through manual or semi-automatic annotation tools, the appearance of the splines in each image is carefully observed, and the images are clearly marked as "qualified" or "unqualified." For example, if the spline surface in the image is smooth, without obvious scratches or holes, and the dimensions meet the design requirements, it is marked as "qualified." Conversely, if the spline surface has visible scratches, uneven color, or a distorted shape, which does not meet the requirements, it is marked as "unqualified." This clear annotation provides a clear target output for subsequent model training, allowing the model to learn which image features correspond to qualified splines and which correspond to unqualified splines, thereby establishing a correlation between image features and the spline's qualification.

[0102] (3) Construct a convolutional network structure to extract image features from spline product appearance images;

[0103] Convolutional Neural Networks (CNNs) have powerful image feature extraction capabilities, centered around their unique convolutional network architecture. Constructing a convolutional network typically involves multiple convolutional layers, pooling layers, and fully connected layers.

[0104] Convolutional layer: A convolutional layer performs a convolution operation by sliding a convolution kernel (also called a filter) over the image, automatically extracting local features such as edges, textures, and lines. Different convolution kernels can capture different types of features, and operating multiple convolution kernels in parallel can extract a rich variety of local feature information. For example, some convolution kernels can detect straight lines on a spline surface, while others can highlight features such as edge contours.

[0105] Pooling layer: A pooling layer is often placed after a convolutional layer. Its main function is to reduce the dimensionality of the feature map obtained by convolution, reducing the amount of data while retaining key feature information and preventing overfitting. Common pooling methods include max pooling and average pooling. For example, max pooling selects the maximum value of a local area in the feature map as the representative value of that area. This can compress the data size without losing important features, improving computational efficiency.

[0106] Fully connected layer: After feature extraction and dimensionality reduction through multiple convolutional layers and pooling layers, all the previously extracted features are finally integrated and mapped through the fully connected layer, converting them into a feature representation suitable for subsequent classification tasks (determining whether the spline is qualified) so that they can be connected with the annotation information for model training.

[0107] Through such a convolutional network composed of a multi-layer structure, the collected spline product appearance images are processed in sequence, and key information that can effectively characterize the spline appearance characteristics is gradually extracted, laying the foundation for accurately judging whether the spline is qualified.

[0108] (4) Using image features as independent variables and corresponding annotation information as dependent variables, the convolutional neural network model is trained.

[0109] After using the convolutional network structure to extract the features of the spline product appearance image, these image features are used as independent variables, and the information previously labeled based on whether the spline is qualified is used as the dependent variable to train the convolutional neural network model.

[0110] During training, the model attempts to predict the corresponding annotation information (i.e., whether the spline is qualified) based on the input image features. It then compares this with the actual annotations and calculates a loss value (such as the commonly used cross-entropy loss). The loss value reflects the degree of difference between the model's prediction and the actual situation. Then, based on this loss value, an optimization algorithm (such as stochastic gradient descent) is used to backpropagate and adjust the model's parameters (such as the convolution kernel weights and biases) so that the model can reduce the loss value on the next prediction, that is, make the model's prediction closer to the actual annotation.

[0111] Through multiple iterative training and continuous adjustment of model parameters, the model gradually learns the inherent rules between image features and the pass or fail of the spline, and finally achieves the goal of accurately judging whether the input spline product appearance image is qualified, thereby providing a strong visual inspection basis for the robot control system to determine the next working link in combination with the current robot arm working link.

[0112] Furthermore, the detection result of the spline product appearance image by the combination of the machine vision model is used to determine the next working step, which is specifically:

[0113] (1) When the test result of the spline product is determined to be qualified, the next work link shall be carried out in the normal working order;

[0114] In the spline injection molding production process involving robotic arms, each work link is closely linked and has a predetermined sequence. For example, the pick-and-place link comes first, followed by the transportation link, and then it may be a link for subsequent processing in collaboration with other equipment. These links are carried out in an orderly manner to ensure that the entire injection molding production process is completed efficiently and with high quality.

[0115] After the machine vision model inspects the appearance image of the spline product and determines that the test result is qualified, it means that the appearance of the current spline product meets the production process requirements and there are no defects that will affect subsequent use or processing. At this point, the next work link of the robot arm can proceed in sequence according to the pre-set normal working order. For example, if the current robot arm is in the handling stage and the spline product passes the inspection, it will then enter the pick-and-place stage of placing the spline in the designated location according to the process, or cooperate with other robots and equipment according to the production schedule, and other subsequent links to ensure that the entire injection molding production task continues in an orderly manner, ensuring production efficiency and the final quality of the product.

[0116] (2) When the test result of the spline product is determined to be unqualified, the next step is to recycle the spline product.

[0117] However, in actual production, spline products inevitably fail to meet appearance requirements due to various reasons. In such cases, the machine vision model will determine that the product has failed inspection. These unqualified spline products may have surface defects such as scratches, holes, irregular shapes, and color deviations. If these products continue to be used in subsequent production steps, they may not only affect the quality of the final product, but may also damage other equipment and disrupt the entire production process.

[0118] Therefore, when the test results show that the spline product is unqualified, in order to avoid these adverse effects, the robot arm's next task is to recycle the spline product. According to the corresponding recycling path planning and operating instructions, the robot arm will move the unqualified spline products to a designated recycling area for subsequent centralized processing, cause analysis, and attempted repairs. This arrangement helps to isolate unqualified products in a timely manner and maintain the normal operation of other links on the production line. It also facilitates the company to collect statistics and trace quality issues that arise during the production process, so that appropriate improvement measures can be taken to optimize the injection molding production process, reduce the defective rate, and improve overall production efficiency.

[0119] Furthermore, the step of generating a control instruction according to the working state and sending it to a motion control unit to control the motion state of the robotic arm specifically includes the following steps:

[0120] (1) Obtain information about the next work step;

[0121] In the entire robotic arm control system used for spline injection molding, after a series of operations such as data collection, analysis, and judgment using relevant models, the relevant information for the robotic arm's next work step has been determined. This information clarifies the specific task the robotic arm will perform next, such as picking and placing the spline, transporting the spline to a specific location, or collaborating with other robotic arms or injection molding machines to complete a certain processing action. It is the key basis for subsequently generating precise control instructions. Different work steps correspond to different robotic arm movement requirements and operation details. Only by clearly knowing the specific details of the next work step can the subsequent movement state of the robotic arm be reasonably planned.

[0122] (2) generating control instructions based on the information, wherein the control instructions include the motion trajectory, speed, acceleration of the robotic arm, and the sequence and time rhythm of the movements of each robotic arm or between the robotic arm and the injection molding machine;

[0123] Based on the information obtained about the next work step, it begins to generate corresponding control instructions. This control instruction covers multiple key elements and aims to accurately control the motion state of the robot arm in all directions, so that it can successfully complete the task of the corresponding work step.

[0124] Motion trajectory planning: Depending on the work process, the robot arm's end effector needs to move along a specific path in three-dimensional space. For example, in the pick-and-place process, it is necessary to plan the optimal motion trajectory from the current position to the spline position (when grasping) or from the spline's current position to the target placement position (when placing). This may involve using mathematical methods such as spline curves to make the trajectory smoother, avoid abrupt changes in the robot arm's movement, improve the stability and efficiency of the movement, and ensure that the target point can be accurately reached for operation.

[0125] Speed ​​and acceleration control: Properly setting the robot's speed and acceleration parameters during movement is also extremely important. Different work processes have different requirements for speed and acceleration. For example, in the handling process, if the spline is fragile or requires extremely high placement accuracy, a lower speed and a more stable acceleration may be required to ensure that the spline does not shake, fall, or deviate from its position due to factors such as inertia during handling. In some scenarios where time efficiency is more important and the spline is relatively stable, appropriately increasing the speed and adjusting the acceleration can speed up the production process.

[0126] Coordination of action sequence and timing (for multiple robotic arms or coordination with injection molding machines): When multiple robotic arms work together or the robotic arms need to coordinate with the actions of the injection molding machine, the control instructions must also clearly define the action sequence and timing between the robotic arms or between the robotic arms and the injection molding machine. For example, after injection molding is completed, a robotic arm needs to wait for the injection molding machine to complete the mold opening before entering the mold area to pick up and place the sample. Here, it is necessary to accurately specify the time nodes for the robotic arms to start the action and the sequence of each related action to ensure that there is no interference or conflict between them, ensuring that the entire injection molding production process is seamless and efficient.

[0127] (3) Sending control instructions to the motion control unit;

[0128] The generated control instructions need to be accurately transmitted to the motion control unit. As the part of the software module specifically responsible for controlling the actual movement of the robot arm, the motion control unit drives the drive unit in the hardware module based on these received instructions, which in turn drives the robot body unit to perform the corresponding actions. This distribution process needs to ensure the integrity and accuracy of the instruction data. Through a stable communication link within the system (such as wired or wireless communication, depending on the system design), the control instructions containing detailed information such as motion trajectory, speed, acceleration, action sequence and timing are sent to the motion control unit to prepare for the actual motion control of the robot arm.

[0129] (4) The motion control unit controls the motion state of the robotic arm according to the command instructions.

[0130] After receiving the control instructions, the motion control unit begins to play a key role, precisely controlling the robot's motion. Based on the specified trajectory information, it uses a complex algorithm to decompose the trajectory into specific parameters such as the angle or displacement changes of each joint. It then directs the drive unit to provide the corresponding power to each joint of the robot, enabling the joints to move according to the predetermined trajectory, speed, acceleration, and other requirements, thereby driving the entire robot to achieve the desired spatial position and posture transformation.

[0131] When multiple robotic arms are working together or in conjunction with an injection molding machine, the motion control unit will strictly follow the sequence and timing specified in the instructions, coordinating the movements of the different robotic arms and between them and the injection molding machine to ensure they can work together in an orderly manner. For example, it can start the movements of different robotic arms in sequence at specified time intervals, or pause the movement of the robotic arms at a specific time point to wait for the injection molding machine to complete a certain process before continuing. Through such precise control, the entire spline injection molding production process can be carried out efficiently, stably, and accurately, improving production efficiency and product quality.

[0132] Furthermore, the software module also includes a safety control unit for monitoring in real time whether the robot arm body collides with the injection molding machine and the surrounding environment, and stopping or adjusting the action in time once a collision is detected.

[0133] In the production scenario of spline injection molding, the robotic arm usually needs to work in a relatively complex environment. It is surrounded not only by large equipment such as the injection molding machine, but also by various tooling fixtures, material storage areas, and other auxiliary production facilities. When the robotic arm is performing actions such as picking up and placing splines, transporting, and cooperating with other equipment, its range of motion is large and its movements are frequent. There is a high risk of collision with the injection molding machine or the surrounding environment. Once a collision occurs, it may not only cause structural damage to the robotic arm itself, affecting its subsequent normal use, but may also damage key production equipment such as the injection molding machine, leading to a series of problems such as production interruption and increased maintenance costs, and may even endanger the safety of the operator. Therefore, the safety control unit plays a vital role in the entire robotic arm control system. It is committed to real-time monitoring and prevention of possible collisions to ensure that the entire production process continues safely and stably.

[0134] The safety control unit monitors in real time whether the robot arm, the injection molding machine, and the surrounding environment have collided. This mainly relies on the coordinated work of multiple parts in the system, specifically:

[0135] Sensor data utilization: The sensor units in the hardware module play a key role in data collection. For example, the position sensor can provide real-time feedback on the precise position information of each joint and end effector of the robot arm. Through this position data, the safety control unit can grasp the specific posture and position of the robot arm in space, thereby determining whether it is approaching the injection molding machine or other objects in the surrounding environment, and whether there is a potential risk of collision. Force sensors are equally important. When the robot arm and surrounding objects approach to a certain extent, changes in contact force may occur. Force sensors can promptly capture abnormal changes in force, providing a powerful auxiliary judgment basis for collision monitoring.

[0136] Software algorithm coordination: The safety control unit itself is equipped with a special software algorithm for real-time analysis and processing of the data collected by the sensors. These algorithms will build a corresponding collision detection model based on the kinematic model of the robot arm, the range of the working space, and the pre-set safety areas of the injection molding machine and the surrounding environment. By continuously comparing the real-time position, force and other data of the robot arm with the safety threshold and spatial range set in the collision detection model, it can accurately determine whether the robot arm is about to collide with the injection molding machine or surrounding objects. For example, based on the current motion trajectory and speed of the robot arm, the algorithm can predict whether it will enter a spatial area defined as dangerous (close to the key parts of the injection molding machine or the collision restricted area of ​​surrounding objects, etc.) in the next very short time, thereby realizing the functions of early warning and real-time monitoring.

[0137] Once the safety control unit detects that there is a risk of collision or a collision has occurred between the robot body and the injection molding machine or the surrounding environment, it will quickly take appropriate measures to avoid more serious consequences:

[0138] Stopping the action immediately: The most direct and effective method is to immediately stop all current movements of the robot arm, cut off the power supply from the drive unit to the robot arm joints, and keep the robot arm in its current position without further movement. This prevents further collisions and reduces the damage to the robot arm and surrounding equipment. For example, if the robot arm's end effector is detected to be about to collide with the mold structure of an injection molding machine, the safety control unit will immediately issue a stop command, causing the robot arm to stop moving at the moment before the collision occurs, thus preventing damage to the mold and robot arm.

[0139] Adjusting the action: In some cases, based on the specific circumstances of the collision and the system's preset strategy, the safety control unit can also make appropriate adjustments to the robot arm's movements so that it can avoid the colliding object and return to a safe state of motion. This may involve replanning the robot arm's motion trajectory, changing its direction of motion, or adjusting its speed. For example, when it is detected that the robot arm is approaching the surrounding material storage area and has a tendency to collide during the process of transporting splines, the safety control unit can quickly calculate a new motion trajectory that avoids the area and instruct the motion control unit to control the robot arm to continue moving according to the new trajectory, ensuring that the production process can continue safely with minimal interruption.

[0140] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as above in terms of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can, without departing from the scope of the technical solution of the present invention, make some changes or modifications to equivalent embodiments using the technical contents disclosed above. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A robotic arm control system for spline injection molding, characterized in that: The system comprises a hardware module and a software module; the software module comprises a motion control unit, a human-computer interaction unit, and a data management unit in communication with each other; the data management unit comprises a data acquisition subunit, a data analysis subunit, and a data storage subunit; the data analysis subunit is used to analyze the working state of the robot arm based on the collected working data, and generate control instructions based on the working state and send them to the motion control unit to control the motion state of the robot arm; Analyzing the working status of the robot arm according to the collected working data specifically includes the following steps: S1. Batch obtain the historical period of robot arm working data from the data storage subunit; S2. Using the position information of each joint and end effector of the robotic arm and the force of the robotic arm as input variables and the corresponding working links as output labels, train a random forest model; the working links include pick-and-place links, handling links, and collaboration links; S3. Use the trained random forest model to determine the current working link of the robotic arm; S4. Determine the next working step based on the current working step of the robot arm and the detection result of the spline product appearance image by the machine vision model; The training of the random forest model includes the following steps: Convert the position information of each joint and end effector of the robotic arm into three-dimensional coordinate data; Convert the force data of the robotic arm into two-dimensional data of magnitude and direction; Randomly extract N transformed input variables and build a decision tree with the output labels; Repeat the process of building decision trees until all input variables are traversed and a random forest is generated; Based on the predictive power of each decision tree for the output label, determine the key input variables that have significant judgment power for the work process; Optimize the goodness of fit of the random forest based on key input variables; The detection result of the spline product appearance image combined with the machine vision model is used to determine the next work step, which is specifically: When the spline product test result is determined to be qualified, the next work link will be carried out according to the normal working order; When the test result of the sample product is determined to be unqualified, the next step is to recycle the sample product; The step of generating a control instruction according to the working state and sending it to a motion control unit to control the motion state of the robotic arm specifically includes the following steps: Get information about the next work step; Generate control instructions based on the information, including the motion trajectory, speed, acceleration of the robotic arm, and the sequence and timing of the movements of each robotic arm or between the robotic arm and the injection molding machine; Send control instructions to the motion control unit; The motion control unit controls the motion state of the robotic arm according to the command instructions; The hardware module includes a sensor unit, which includes a position sensor, a force sensor and a vision sensor. The position sensor is used to provide real-time feedback on the position information of each joint and the end effector of the robotic arm. The force sensor is used to collect the gripping force of the end effector in grasping the spline and the contact force during the interaction with the surrounding environment. The vision sensor is used to obtain the appearance image of the spline product and the position of the injection mold.

2. A robotic arm control system for spline injection molding according to claim 1, characterized in that: The machine vision model is configured as a convolutional neural network model and includes the following construction steps: Collect historical spline product appearance image data samples; The images are marked separately according to whether the spline is qualified; Construct a convolutional network structure to extract image features from spline product appearance images; The convolutional neural network model is trained with image features as independent variables and the corresponding annotation information as dependent variables.

3. The robotic arm control system for spline injection molding according to claim 1, characterized in that: The software module also includes a safety control unit for monitoring in real time whether the robot arm body collides with the injection molding machine and the surrounding environment, and stopping or adjusting the action in time once a collision is detected.

4. A robotic arm control system for spline injection molding according to claim 1, characterized in that: The motion control unit includes a trajectory planning subunit, a speed control subunit and a motion coordination subunit. The trajectory planning subunit is used to plan the motion trajectory of the robot arm joint space or Cartesian space according to the injection molding process requirements; the speed control subunit is used to control the speed and acceleration of the robot arm during the movement process; the motion coordination subunit is used to coordinate the movement sequence and time rhythm between each robot arm or the robot arm and the injection molding machine when multiple robot arms work together or the robot arm and the injection molding machine cooperate with each other.

5. The robotic arm control system for spline injection molding according to claim 1, characterized in that: The hardware module also includes a robotic arm body unit and a drive unit. The robotic arm body unit is used to achieve the transformation of spatial position and posture to complete the actions of picking up, placing and transporting splines during the injection molding process, and the drive unit is used to provide power for the movement of the robotic arm joints.

6. The robotic arm control system for spline injection molding according to claim 1, characterized in that: The force of the robotic arm includes the gripping force of the end effector grasping the injection molded spline and the contact force between the robotic arm and the surrounding objects; the data acquisition subunit is used to collect working data of the sensor unit robotic arm during operation, and the working data includes position information of each joint of the robotic arm and the end effector, the force of the robotic arm, and the appearance image of the injection molded spline product; The human-computer interaction unit includes an operation interface subunit and a programming interface subunit. The operation interface subunit provides a visual operation interface for setting parameters, sending action instructions and displaying the motion status of the robot arm; The programming interface subunit is used by technicians to program and modify the motion flow of the robotic arm.

Images