Picking robot arm action optimization method and system combined with machine learning

Abstract

The application provides a picking mechanical arm action optimization method and system combined with machine learning. First, action data sets of a picking mechanical arm in different picking scenes are obtained, which contain joint motion angles, speeds and action time of picking execution components. Correlation analysis is performed to generate action correlation rules. Then, a pre-trained machine learning model is called to establish a mapping relationship between the action correlation rules and a preset picking action optimization target, and a mapping relationship table is obtained. Then, an initial action optimization scheme containing joint adjustment parameters and picking execution component adjustment parameters is generated based on the mapping relationship table. Then, the initial scheme is subjected to action sequence adaptation processing to obtain an adapted action sequence. Finally, the adapted action sequence is converted into an action control instruction and sent to a picking mechanical arm control system to drive the mechanical arm to perform picking actions, thereby realizing optimization of the picking mechanical arm actions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a picking mechanical arm action optimization method and system combined with machine learning. BACKGROUND

[0002] In the field of agricultural automation, picking mechanical arms are increasingly widely used, which can significantly improve picking efficiency and reduce labor costs, and are of great significance to guarantee the stable supply of agricultural products. However, the existing picking mechanical arms have many deficiencies in action control.

[0003] On the one hand, the traditional picking mechanical arm action control method is often based on a preset fixed program, which lacks adaptability to different picking scenes. In actual picking process, picking scenes will vary greatly due to differences in crop types, growth states, environmental conditions and other factors. The fixed program is difficult to adjust the action of the mechanical arm in real time according to these changes, resulting in poor picking effect, and may cause incomplete picking, damage to crops and other problems.

[0004] On the other hand, the existing method usually does not fully consider the correlation between the movements of the joints of the mechanical arm and the coordination with the action of the picking execution component when optimizing the action of the picking mechanical arm. The mechanical arm is a complex motion system, and the movements of the joints influence each other. Only by comprehensively considering these factors can efficient and accurate picking action be achieved. Moreover, there is a lack of effective means to accurately convert the optimized action parameters into a mechanical arm executable action sequence, making it difficult to truly apply the optimization results to actual picking operations. SUMMARY

[0005] In view of the above-mentioned problems, in combination with the first aspect of the present application, the embodiments of the present application provide a picking mechanical arm action optimization method combined with machine learning, which comprises:

[0006] Obtaining a set of action data of a picking mechanical arm under different picking scenes, performing correlation analysis on the set of action data to generate action correlation rules, the set of action data comprising movement angle data, movement speed data of each joint of the mechanical arm and action duration data of the picking execution component;

[0007] Calling a pre-trained machine learning model, inputting the action correlation rules and a preset picking action optimization target into the machine learning model, establishing a mapping relationship between the action correlation rules and the picking action optimization target, and obtaining a mapping relationship table;

[0008] Generating an initial action optimization scheme of the picking mechanical arm based on the mapping relationship table, the initial action optimization scheme comprising adjustment angle parameters, adjustment speed parameters of each joint and adjustment duration parameters of the picking execution component;

[0009] perform action sequence adaptation processing on the initial action optimization scheme to convert parameters in the initial action optimization scheme into an action sequence executable by the robot arm, to obtain an adapted action sequence;

[0010] convert the adapted action sequence into picking robot arm action control instructions, and send the picking robot arm action control instructions to a picking robot arm control system, which drives the picking robot arm to perform picking actions after receiving the picking robot arm action control instructions.

[0011] In still another aspect, the embodiments of the present application also provide a picking robot arm action optimization method system combined with machine learning, which comprises a processor, a machine-readable storage medium, the machine-readable storage medium is connected with the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to realize the above-mentioned method.

[0012] Based on the above aspects, by obtaining the action data set of the picking robot arm in different picking scenarios and performing correlation analysis, the action correlation rule is generated, the pre-trained machine learning model is called, the mapping relationship between the action correlation rule and the preset picking action optimization target is established, the mapping relationship table is obtained, the corresponding action adjustment strategy can be quickly found according to different optimization targets, the initial action optimization scheme generated based on the mapping relationship table comprehensively considers the adjustment angle, speed of each joint and adjustment time of the picking execution component and other parameters, and the overall optimization of the picking robot arm action is realized. The action sequence adaptation processing is performed on the initial action optimization scheme to convert it into an action sequence executable by the robot arm, so that the optimization scheme can be accurately and correctly applied to the actual picking operation. Finally, the adapted action sequence is converted into action control instructions and sent to the picking robot arm control system, the robot arm is driven to perform the optimized picking action, the picking efficiency, accuracy and adaptability of the picking robot arm are effectively improved, and the crop damage rate in the picking process is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is an execution flow diagram of the picking robot arm action optimization method combined with machine learning provided by the embodiments of the present application.

[0014] Figure 2 is a schematic diagram of exemplary hardware and software components of the picking robot arm action optimization method system combined with machine learning provided by the embodiments of the present application. DETAILED DESCRIPTION

[0015] The present application will be specifically described below in conjunction with the drawings of the specification, Figure 1is a flowchart of a picking mechanical arm action optimization method combined with machine learning provided by an embodiment of the present application. The picking mechanical arm action optimization method combined with machine learning is described in detail below.

[0016] Step S110: Obtain a set of action data of the picking mechanical arm in different picking scenarios, perform correlation analysis on the set of action data, and generate action correlation rules. The set of action data includes motion angle data, motion speed data of each joint of the mechanical arm, and action duration data of a picking execution component.

[0017] In this embodiment, apple picking is taken as an application scenario, and the picking mechanical arm needs to perform picking actions in different apple growth states (if real size, fruit hanging position, and branch leaf density). First, a set of action data in this scenario needs to be obtained and action correlation rules are generated.

[0018] Step S111: Receive a set of action data transmitted by a joint sensor and an action timer of the picking mechanical arm. The joint sensor is used to collect motion angle data and motion speed data of each joint of the mechanical arm, and the action timer is used to collect action duration data of a picking execution component.

[0019] In the apple picking scenario, the picking mechanical arm is configured with six rotary joints, each of which is installed with a high-precision joint sensor. The joint sensor can collect motion angle data and motion speed data of each joint in the motion process in real time, and the sampling frequency is set to one hundred times per second. At the same time, an action timer is installed on a picking execution component (such as a pneumatic gripper) at the end of the mechanical arm, which is used to record the action duration data from the closure of the gripper to the grasping of the apple and then to the placing of the apple into the storage box. The joint sensor and the action timer transmit the collected data to a data processing terminal through wired Ethernet. The data processing terminal receives these data and stores them as a set of action data. Each piece of data in the set of action data includes information such as collection timestamp, joint number, motion angle data, motion speed data, and action duration data of the picking execution component.

[0020] Step S112: Divide the set of action data according to picking scenario types. Each picking scenario type corresponds to a set of scenario action data subsets. The picking scenario types are determined according to the morphological characteristics of the picking objects and the growth environment characteristics.

[0021] Based on the apple's morphological characteristics (fruit diameter, fruit weight, stem length) and growth environment characteristics (such as fruit height, distance from branches and leaves, and whether it is shaded by branches and leaves), harvesting scenarios are categorized into large fruit unshaded scenarios, medium fruit slightly shaded by branches and leaves scenarios, small fruit severely shaded by branches and leaves scenarios, and high-positioned large fruit scenarios. The data processing terminal performs scenario matching on each data point in the action data set according to preset scenario classification standards, grouping data belonging to the same harvesting scenario type together to form a subset of scenario action data for that scenario. For example, data with fruit diameter greater than a specific value, fruit height within a specific range, and no branch or leaf shading are categorized into the large fruit unshaded scenario action data subset.

[0022] Step S113: Perform data segmentation processing on each subset of scene action data, splitting each subset of scene action data into multiple action data blocks. Each action data block contains motion angle data, motion speed data, and action duration data of each joint of the robotic arm within a single complete picking action cycle.

[0023] For each subset of scene motion data, the data processing terminal segments the data based on the cyclical characteristics of the picking action. A single complete picking action cycle is defined as the entire process from the robotic arm's initial position, moving to locate the apple, the picking actuator's action, grasping the apple and moving it to the storage box, releasing the apple and returning to the initial position. By identifying the signals in the motion data indicating the robotic arm's start and return from the initial position, the start and end points of each complete picking action cycle are determined. Then, the subset of scene motion data is divided into multiple motion data blocks according to these start and end points. Each motion data block includes the motion angle data sequence, motion speed data sequence, and action duration data of all joints within that picking action cycle.

[0024] Step S114: Extract the associated features from each of the motion data blocks. The associated features include the correspondence between motion angle data and motion speed data of the same joint, the coordination relationship between motion angle data of different joints, and the correspondence between motion duration data of the picking and executing component and joint motion speed data.

[0025] For each action data block, the data processing terminal needs to extract the associated features, and the specific process is as follows.

[0026] Step S1141: Separate the motion angle data, motion speed data and action duration data of each joint of the robotic arm from each motion data block. Each data type corresponds to an independent data sequence.

[0027] The data processing terminal parses the motion data blocks and extracts the motion angle and velocity data of each joint based on the joint number and data type field, forming separate motion angle and velocity data sequences for each joint. Simultaneously, it extracts the motion duration data of the picking actuator, forming an independent motion duration data sequence. For example, the motion angle data sequence for joint 1 is a series of angle values ​​arranged in chronological order, the motion velocity data sequence for joint 1 is the velocity value at the corresponding time point, and the motion duration data sequence for the picking actuator is the duration value for each picking action cycle.

[0028] Step S1142: For the motion angle data and motion speed data of the same joint, establish a one-to-one correspondence according to the time node sequence. The motion angle data value and motion speed data value of each time node form a data pair. Multiple data pairs form the angle-speed correspondence sequence of the joint. The angle-speed correspondence sequence is the correspondence between the motion angle data and motion speed data of the same joint.

[0029] Taking joint 1 as an example, the data processing terminal matches the data in the motion angle data sequence and motion velocity data sequence of joint 1 according to the chronological order of the timestamps. Each motion angle data value and motion velocity data value at the same time point forms a data pair. All these data pairs are arranged in chronological order to form the angle-velocity correspondence sequence of joint 1. Other joints are processed in the same way, generating their own angle-velocity correspondence sequences.

[0030] Step S1143: Select a joint group in the robotic arm that has a motion coordination relationship. The joint group includes two or more joints that move simultaneously during the picking action. Extract the motion angle data sequence of each joint in the joint group.

[0031] Based on the kinematic characteristics of the apple-picking robotic arm, joint groups with coordinated motion relationships are identified. For example, joint 2 (shoulder rotation joint) and joint 3 (elbow flexion joint) need to move simultaneously during the extension and retraction of the robotic arm, therefore they are grouped into one joint group. The data processing terminal extracts the motion angle data sequences of joint 2 and joint 3 in this joint group from the motion data block.

[0032] Step S1144: Calculate the time synchronization of the joint motion angle data sequence in the joint group. Determine the synchronization level by comparing the time difference of the peak occurrence in different joint motion angle data sequences. The synchronization level is related to the time difference.

[0033] For the motion angle data sequences of joints 2 and 3, the data processing terminal identifies all peak points in the two sequences, i.e., points where the angle value reaches a local maximum. Then, it calculates the time difference between the occurrence of each corresponding peak point in the two sequences, and uses the average of all time differences as an indicator of time synchronization. The smaller the time difference, the higher the level of synchronization; the larger the time difference, the lower the level of synchronization, thus establishing a correspondence between the level of synchronization and the time difference.

[0034] Step S1145: Based on the synchronization level and the percentage change in the motion angle data of each joint, establish a description of the collaborative relationship between the motion angle data of different joints in the joint group. The description of the collaborative relationship includes the synchronization level value and the percentage change in the angle value.

[0035] Calculate the total amplitude of angle changes in the motion angle data sequences of joints 2 and 3, i.e., the difference between the maximum and minimum values ​​in each sequence. Then calculate the ratio of the total amplitude of angle changes between the two joints, and combine it with the previously obtained synchronization level value to form a description of the cooperative relationship of this joint group. For example, if the synchronization level value is 0.8 (out of 1), and the ratio of the angle change amplitudes of joints 2 and 3 is 1.2:1, this information together constitutes the description of the cooperative relationship.

[0036] Step S1146: Extract the action duration data of the picking execution component from each action data block, determine the start time and end time of the action duration, extract the motion speed data sequence of all joints from the start time to the end time, and calculate the average value and change range of the motion speed data of each joint within the corresponding time period.

[0037] The motion duration data of the picking execution component is obtained from the motion data block, and the start and end times of the motion duration are determined based on the timestamps. Then, the motion velocity data sequence of all joints between these two timestamps is extracted. For the velocity sequence of each joint, the average value of all data points is calculated, and the difference between the maximum and minimum values ​​is taken as the magnitude of change.

[0038] Step S1147: Establish the correspondence between the action duration data of the picking execution component and the average value and variation range of the movement speed data of each joint. Each action duration data value corresponds to a set of average value and variation range values ​​of joint movement speed, forming a duration-speed correspondence. Integrate the angle-speed correspondence of the same joint, the coordination relationship of movement angle data between different joints, and the duration-speed correspondence of the picking execution component to form the associated features of each action data block.

[0039] The motion duration data of the picking actuator is correlated with the average speed and variation range of each joint. One motion duration data value corresponds to the average speed and variation range of all joints, thus forming a duration-speed correspondence. Finally, the angle-speed correspondence of the same joint, the coordination relationship of the joint group, and the duration-speed correspondence are integrated to form the complete correlation feature of the motion data block.

[0040] Step S115: Calculate the similarity of associated features between different action data blocks in the same subset of scene action data. For each associated feature, calculate the similarity of its data sequence in different data blocks separately to obtain the independent similarity score of each feature. Based on the importance weight of each associated feature in the picking action, combine all independent similarity scores to obtain the overall similarity of associated features.

[0041] In a subset of motion data from a large, unobstructed fruit scene, two motion data blocks, A and B, are selected. For the angle-velocity correspondence feature, a dynamic time warping algorithm is used to calculate the similarity of the angle-velocity sequences at the same joint in both data blocks, yielding an independent similarity score for this feature. For the cooperative relationship feature, similarity is calculated by comparing the differences in synchronization level values ​​and the proportion of angle change amplitude, resulting in a corresponding independent similarity score. For the duration-velocity correspondence feature, differences in motion duration data values, as well as differences in the average speed and change amplitude of each joint, are calculated to obtain an independent similarity score. Based on the requirements of the picking action, importance weights are assigned to the three correlation features; for example, the angle-velocity correspondence weight is 0.4, the cooperative relationship weight is 0.3, and the duration-velocity correspondence weight is 0.3. The independent similarity score of each feature is multiplied by its corresponding weight, and the products are summed to obtain the overall correlation feature similarity between motion data blocks A and B.

[0042] Step S116: Select action data block groups with similar association features based on the association feature similarity. Each action data block group contains multiple action data blocks whose association feature similarity meets preset conditions.

[0043] The preset condition for the similarity of association features is set to be greater than or equal to 0.7. The data processing terminal calculates the overall similarity of association features among all action data blocks in the same scene action data subset, and groups action data blocks with a similarity of greater than or equal to 0.7 into multiple action data block groups. The action data blocks within each action data block group have similar association features, reflecting the robotic arm's action patterns under similar harvesting conditions.

[0044] Step S117: Analyze the commonalities of the associated features in each action data block group, and extract rule entries that can reflect the association patterns of the action data blocks in the action data block group. Each rule entry contains a specific correspondence description of the associated features.

[0045] For each motion data block group, the data processing terminal analyzes the correlation characteristics of all motion data blocks within the group to identify commonalities. For example, in a certain motion data block group, the angle-velocity correspondence sequence of joint 1 in all data blocks shows a pattern where the velocity first increases and then decreases as the angle increases. The synchronization level in the coordination relationship between joints 2 and 3 is above 0.75, the ratio of angle change amplitude is stable at around 1.2:1, and the motion duration data of the picking and executing component is positively correlated with the average velocity of joint 4. These commonalities are organized into rule entries, and each rule entry describes in detail the specific correspondence between the correlation characteristics.

[0046] Step S118: Classify and organize all the rule entries according to the picking scenario type. Each picking scenario type corresponds to a set of rule entries, forming a complete action association rule.

[0047] Step S1181: Establish a classification directory of picking scene types. The classification directory contains all the determined picking scene type names and corresponding scene feature descriptions. The scene feature descriptions include the specific content of the morphological characteristics of the picking object and the characteristics of the growing environment.

[0048] The data processing terminal establishes a classification directory for harvesting scene types. This directory lists all the categorized harvesting scene types, such as large fruit unobstructed scene and medium fruit slightly obstructed by branches and leaves scene. Each scene type has a detailed description of its characteristics. For example, the characteristics of the large fruit unobstructed scene are: fruit diameter greater than a specific value, fruit weight within a specific range, moderate stem length, fruit hanging height between 1.5 meters and 2.5 meters, and no branches or leaves obstructing the fruit, allowing for direct harvesting.

[0049] Step S1182: Traverse each rule entry and extract the scene information corresponding to the associated features involved in the rule entry. The scene information includes the picking scene type to which the action data block from which the rule entry originates belongs.

[0050] The data processing terminal iterates through all generated rule entries one by one, extracting the harvesting scenario type of the action data block from the metadata of each rule entry. This information is the scenario information corresponding to the rule entry.

[0051] Step S1183: Based on the extracted scene information, assign the rule entries to the corresponding picking scene type in the classification directory to form the initial rule group for each scene type.

[0052] Based on the extracted scene information, each rule entry is categorized into the corresponding picking scene type in the classification directory. For example, rule entries originating from large fruit unobstructed scene action data blocks are assigned to this scene type, and all rule entries assigned to the same scene type form the initial rule group for that scene type.

[0053] Step S1184: Perform rule conflict detection on the initial rule group for each scene type, compare the association feature descriptions and correspondences between different rule entries in the initial rule group, and mark rule entries with the same association feature descriptions but different correspondences as conflicting rule pairs.

[0054] For each scenario type's initial rule group, the data processing terminal performs pairwise comparisons of the rule entries within the group. It compares whether the descriptions of the associated features between the rule entries are consistent, and whether the correspondences between the associated features are the same. If two rule entries describe the same associated features but have different correspondences—for example, both describing the angle-velocity correspondence of joint 1, but one rule states that velocity increases as the angle increases, while the other states that velocity decreases as the angle increases—then these two rule entries are marked as a conflicting rule pair.

[0055] Step S1185: For each conflicting rule pair, query the correlation feature similarity statistics of its source action data block group, retain the rule entry with the largest source data block group size and the largest average correlation feature similarity, and delete the other conflicting rule entry.

[0056] For each conflicting rule pair, the data processing terminal queries the relevant statistical information of the action data block groups from which each rule entry originates, including the size of the data block group (i.e., the number of action data blocks within the group) and the average similarity of the associated features within the group. The size and average similarity of the two data block groups are compared, and the rule entry from the source data block group with the larger size and higher average similarity is retained. The other conflicting rule entry is removed from the initial rule group.

[0057] Step S1186: Supplement the rule entries corresponding to the missing associated features in the initial rule group for each scene type. If any associated feature has no corresponding rule entry under the scene type, select the rule entry with the highest correlation from the rule group of similar picking scene types and adapt and modify it to form a supplementary rule entry.

[0058] Check the initial rule group for each scene type to see if it covers all the rule entries corresponding to the expected associated features. If a certain associated feature is found to lack a corresponding rule entry in this scene type (e.g., the rule entries for the collaboration relationship of joints 5 and 6 are missing in the scene with slight occlusion of medium-sized fruit branches and leaves), then search for the rule group of a similar scene type, such as the rule group for the scene with no occlusion of large fruit branches and leaves. Calculate the correlation degree between each rule entry in that scene's rule group and the missing associated feature, and select the rule entry with the highest correlation degree. Based on the characteristics of the scene with slight occlusion of medium-sized fruit branches and leaves, adapt and modify the selected rule entries, such as adjusting the synchronicity level threshold and angle change amplitude ratio in the collaboration relationship, to form supplementary rule entries and add them to the initial rule group for this scene type.

[0059] Step S1187: Logically sort the rule groups for each scene type. According to the execution order of the actions involved in the associated features, divide the rule entries into joint motion association rules, execution component action association rules, and collaborative action association rules. After sorting according to the complexity of the associated features, add scene identifiers to the sorted rule groups. The scene identifiers include the picking scene type name and the rule group version number.

[0060] For each scenario type's rule group, rules are categorized based on the execution order of the actions involved in the associated features. Rule entries describing the relationship between the angle and velocity of a single joint are grouped into joint motion association rules; rule entries describing the relationship between the duration of the picking action and the joint velocity are grouped into action association rules for the picking component; and rule entries describing the coordinated motion of multiple joints are grouped into coordinated action association rules. Within each category, rules are sorted from low to high complexity of the associated features; for example, rules relating angle and velocity of a single joint are sorted first, followed by coordinated rules for multiple joints. After sorting, a scenario identifier is added to the rule group. The scenario identifier consists of the picking scenario type name and the rule group version number, such as "Large Fruit Unobstructed Scenario_V1.0".

[0061] Step S1188: Summarize all rule groups for all picking scenario types and generate a rule index table. The rule index table includes the scenario type name, rule group version number, and number of rule entries.

[0062] The data processing terminal aggregates rule groups for all scenario types and then generates a rule index table. This rule index table records the name of each scenario type, the corresponding rule group version number, and the number of rule entries contained in that rule group, facilitating quick querying and retrieval of the corresponding rule group later.

[0063] Step S1189: Integrate the summarized rule groups and rule index tables to form complete action association rules, and store them in the rule database.

[0064] All the summarized scenario type rule groups and rule index tables are integrated to form a complete action association rule system, which is then stored in a pre-established rule database. The rule database adopts a relational database structure, creating corresponding data tables for rule groups and rule entries to facilitate data management, updates, and queries.

[0065] Step S120: Call the pre-trained machine learning model, input the action association rules and the preset picking action optimization target into the machine learning model, establish the mapping relationship between the action association rules and the picking action optimization target, and obtain the mapping relationship table.

[0066] In the apple picking scenario, after setting the optimization target for picking actions, the corresponding action association rules are called from the rule database. Both are then input into a pre-trained machine learning model, and a mapping relationship is established and a mapping relationship table is generated through model calculation.

[0067] Step S121: Determine the preset picking action optimization targets. The picking action optimization targets include the picking action completion efficiency target, the picking action stability target, and the picking object damage rate control target. Each optimization target has a corresponding descriptive index.

[0068] The preset optimization targets for the picking action are specifically set as follows: The description index for the picking action completion efficiency target is the number of picking actions completed per unit time, requiring a specific picking frequency to be achieved while ensuring picking quality; the description index for the picking action stability target is the speed fluctuation amplitude during the movement of each joint of the robotic arm, requiring the fluctuation amplitude to be controlled within a specific range; the description index for the picking object damage rate control target is the proportion of damage such as indentations and scratches on the apple surface during the picking process, requiring the damage rate to be lower than a specific threshold.

[0069] Step S122: Decompose the action association rule into multiple rule units, each rule unit corresponding to a specific association feature description. At the same time, decompose the picking action optimization target into multiple target units, each target unit corresponding to a description index of the optimization target.

[0070] The data processing terminal decomposes the action association rules. For each rule group under each picking scenario type, each rule entry is broken down into independent rule units based on its association feature description. For example, the rule entry "When the angle of joint 1 increases, the speed first increases and then decreases" is broken down into an independent rule unit, which only corresponds to the specific association feature description of the angle-speed correspondence of joint 1. At the same time, the picking action completion efficiency target, the picking action stability target, and the picking object damage rate control target are each broken down into target units. Each target unit corresponds to a descriptive index. For example, the target unit "Number of pickings per unit time" corresponds to the descriptive index of the picking action completion efficiency target, the target unit "Joint speed fluctuation amplitude" corresponds to the descriptive index of the picking action stability target, and the target unit "Apple damage ratio" corresponds to the descriptive index of the picking object damage rate control target.

[0071] Step S123: Input the rule unit and the target unit into a pre-trained machine learning model, the machine learning model including a feature input layer, an association modeling layer and a mapping output layer.

[0072] After all the decomposed rule units and target units are organized according to a preset format, they are input into a pre-trained machine learning model. This machine learning model has a three-layer network structure, consisting of a feature input layer, an association modeling layer, and a mapping output layer. The layers are connected in a fully connected manner to realize the association modeling and mapping relationship output between rule units and target units.

[0073] Step S124: In the feature input layer, feature encoding processing is performed on the rule unit and the target unit to convert the text-based rule unit and target unit into vector-based feature data. Each rule unit corresponds to a rule feature vector, and each target unit corresponds to a target feature vector.

[0074] The feature input layer first preprocesses the text-based rule units and target units, including removing redundant expressions and standardizing terminology. Then, word embedding is used to encode the preprocessed text, converting each word into a fixed-dimensional vector. Average pooling is then applied to these word vectors to generate the rule feature vectors for the rule units and the target feature vectors for the target units. Each rule feature vector and target feature vector is a fixed-length numerical sequence that represents the core feature information of the corresponding unit.

[0075] Step S125: Input the regular feature vector and the target feature vector into the association modeling layer. The association modeling layer adopts a multilayer perceptron structure and calculates the correlation degree between the regular feature vector and the target feature vector through linear transformation and nonlinear activation function.

[0076] The association modeling layer consists of three hidden layers, employing a multilayer perceptron structure. The concatenated regular feature vector and target feature vector are input into the first hidden layer, where a linear transformation is performed using the weight matrix. After processing by a non-linear activation function, the output feature data is fed into the second hidden layer. The second and third hidden layers repeat this linear transformation and non-linear activation process, progressively uncovering the deep association between the regular and target feature vectors. Finally, the association degree is calculated through the output layer of the association modeling layer, representing the strength of the association between the regular and target units.

[0077] Step S126: Determine the matching relationship between the rule unit and the target unit based on the correlation degree value. The rule unit and the target unit that meet the preset conditions form a matching pair. Each matching pair includes a rule unit and a corresponding target unit.

[0078] The preset condition for the correlation degree value is set to be greater than or equal to 0.6. The data processing terminal compares the correlation degree value output by the correlation modeling layer with the preset condition. If the correlation degree value between a rule unit and a target unit is greater than or equal to 0.6, it is determined that there is a matching relationship between the two, forming a matching pair. For example, the correlation degree value between the rule unit "synchronicity level of joint 2 and joint 3 is above 0.75" and the target unit "joint velocity fluctuation amplitude" is 0.72, which meets the preset condition, and the two form a matching pair.

[0079] Step S127: Group all the matching pairs according to the picking scene type, and form a local mapping relationship for each picking scene type by the matching pairs under each picking scene type.

[0080] Based on the harvesting scenario type to which the rule units in each matching pair belong, all matching pairs are grouped. For example, matching pairs of rule units originating from a large, unobstructed fruit scene are grouped together to form a local mapping relationship for that scenario type. This local mapping relationship only reflects the matching situation between rule units and target units in a large, unobstructed fruit scene.

[0081] Step S128: Integrate the local mapping relationships of each picking scene type, supplement the association and connection relationships of matching pairs between different scenes, and form a global mapping relationship covering all picking scene types.

[0082] The local mapping relationships for all picking scenario types are summarized, and the commonalities and differences between matching pairs in different scenarios are analyzed. For matching pairs in different scenarios that are related, the description of their correlation and connection is supplemented. For example, the matching pair "the duration of the picking action of the component is positively correlated with the speed of joint 4" in the scenario of medium-sized fruit with slight shading by branches and leaves is related to a similar matching pair in the scenario of large fruit with no shading. The connection relationship between the two in terms of speed threshold and duration range is supplemented, and finally a global mapping relationship covering all picking scenario types is formed.

[0083] Step S129: Present the global mapping relationship in tabular form to obtain a mapping relationship table. The row dimension of the table is the rule unit, and the column dimension is the target unit. The corresponding correlation value and matching condition are recorded in the table cell.

[0084] The global mapping relationship is converted into a table. The table lists all rule units in rows, with each rule unit occupying a row; and all target units in columns, with each target unit occupying a column. Each cell in the table records the correlation score between the corresponding rule unit and the target unit, as well as the matching condition (i.e., whether the correlation score meets the preset conditions). For example, a cell might record "Correlation score: 0.68, Matching condition: Meets," clearly showing the mapping relationship between the rule unit and the target unit, thus obtaining the mapping relationship table.

[0085] Step S130: Generate an initial motion optimization scheme for the picking robotic arm based on the mapping relationship table. The initial motion optimization scheme includes adjustment angle parameters, adjustment speed parameters, and adjustment duration parameters of each joint and the picking execution component.

[0086] In the apple picking scenario, based on the matching relationship and correlation value between the rule unit and the target unit in the mapping relationship table, combined with the original motion parameters, an initial motion optimization scheme containing the adjustment parameters of each joint and the adjustment parameters of the picking execution component is calculated and generated.

[0087] Step S131: Extract the rule unit and the matching target unit corresponding to the current picking scenario type from the mapping relationship table, and determine the associated features that need to be optimized and the corresponding optimization target description indicators under the current picking scenario type.

[0088] Based on the current apple picking scenario (e.g., large, unobstructed fruit), all rule units corresponding to this scenario type and the target units matching these rule units are selected from the mapping table. By analyzing these rule units and target units, the relevant features that need to be optimized in the current scenario are determined, such as the angle-velocity correspondence of joint 1 and the collaborative relationship between joint 2 and joint 3. At the same time, the corresponding optimization target description indicators are clarified, such as the number of pickings per unit time and the fluctuation range of joint speed.

[0089] Step S132: Based on the association feature description in the rule unit, locate the original motion parameters corresponding to the association feature in the motion data set. The original motion parameters include the original motion angle data, original motion speed data, and original motion duration data of each joint of the robotic arm and the picking execution component.

[0090] Based on the specific description of the associated features in the rule unit, a search and matching process is performed in the motion dataset. For example, for the rule unit "when the angle of joint 1 increases, the speed first increases and then decreases", the original motion angle data sequence and the original motion speed data sequence of joint 1 are located in the motion dataset; for the rule unit "the synchronization level of joint 2 and joint 3 is above 0.75", the original motion angle data sequences of joint 2 and joint 3 are located; at the same time, the original motion duration data sequence of the picking execution component is located. These data together constitute the original motion parameters.

[0091] Step S133: Referring to the optimized target description index in the target unit, input the original action parameters into a predefined performance evaluation model. The performance evaluation model outputs the estimated performance index value based on the current action parameters, and calculates the difference between the estimated performance index value and the optimized target description index.

[0092] The predefined performance evaluation model is trained based on historical harvesting data and can predict harvesting performance based on the input raw motion parameters. After inputting the raw motion parameters into the model, the model outputs estimated performance indicators such as the number of harvests per unit time, the amplitude of joint speed fluctuation, and the proportion of apple damage. These estimated performance indicators are compared with their corresponding optimization target descriptive indicators, and the differences between the two are calculated to obtain the discrepancy between the estimated performance indicators and the optimization target descriptive indicators.

[0093] Step S134: Determine the adjustment direction of each original action parameter based on the difference and the correlation value in the mapping table. The adjustment direction is determined according to the positive or negative attribute of the difference and the magnitude of the correlation value.

[0094] Analyze the positive and negative attributes of the differences. If the estimated performance index is lower than the optimization target index (e.g., the estimated number of harvests per unit time is lower than the target number), the original motion parameters need to be adjusted to improve performance. If the estimated performance index is higher than the optimization target index (e.g., the estimated joint speed fluctuation is greater than the target fluctuation), the original motion parameters need to be adjusted to reduce fluctuation. Simultaneously, consider the correlation values ​​between the corresponding rule units and the target units in the mapping table. The higher the correlation value, the greater the influence of the rule unit on the target unit, and the more important it is to prioritize the original motion parameters corresponding to that rule unit when determining the adjustment direction. Based on these factors, determine the adjustment direction for each original motion parameter, such as increasing the movement speed of joint 1 and decreasing the proportion of angle changes in joints 2 and 3.

[0095] Step S135: Based on the adjustment direction and the preset adjustment step length rule, calculate the adjustment angle parameter, adjustment speed parameter and adjustment duration parameter of each joint and the picking execution component. The adjustment angle parameter is the result of superimposing the original motion angle data and the adjustment amount, the adjustment speed parameter is the result of superimposing the original motion speed data and the adjustment amount, and the adjustment duration parameter is the result of superimposing the original action duration data and the adjustment amount.

[0096] Step S1351: Obtain a preset adjustment step size rule, wherein the adjustment step size rule includes adjustment step size coefficients corresponding to different correlation values, and the correlation values ​​and adjustment step size coefficients are in a corresponding relationship.

[0097] The system configuration file retrieves preset adjustment step size rules, which specify the adjustment step size coefficients corresponding to different correlation degree value ranges. For example, when the correlation degree value is between 0.9 and 1.0, the adjustment step size coefficient is 0.1; when the correlation degree value is between 0.8 and 0.9, the adjustment step size coefficient is 0.08, and so on, establishing a correspondence between correlation degree values ​​and adjustment step size coefficients.

[0098] Step S1352: Based on the correlation values ​​between the rule unit and the target unit in the mapping relationship table, select the corresponding adjustment step size coefficient from the adjustment step size rules.

[0099] For each original motion parameter that needs adjustment, the correlation value between its corresponding rule unit and target unit in the mapping table is found. Based on this correlation value, the appropriate adjustment step size coefficient is selected from the adjustment step size rules. For example, if the correlation value of a certain original motion parameter is 0.85, the selected adjustment step size coefficient is 0.08.

[0100] Step S1353: For the original motion angle data of each joint of the robotic arm, determine the positive or negative attribute of the adjustment amount according to the adjustment direction. When the adjustment direction is to increase the angle, the adjustment amount is positive, and when the adjustment direction is to decrease the angle, the adjustment amount is negative.

[0101] For the original motion angle data of each joint of the robotic arm, the sign of the adjustment amount is determined based on the established adjustment direction. If the adjustment direction is to increase the joint motion angle, the adjustment amount is set to a positive value; if the adjustment direction is to decrease the joint motion angle, the adjustment amount is set to a negative value.

[0102] Step S1354: Calculate the adjustment amount of the joint adjustment angle. The adjustment amount is equal to the original motion angle data multiplied by the adjustment step length coefficient. The adjustment angle parameter is equal to the original motion angle data plus the adjustment amount.

[0103] Taking the original motion angle data of joint 1 as an example, multiply the original data by the selected adjustment step size coefficient to obtain the adjustment amount of the joint 1 adjustment angle. If the original motion angle data is A, the adjustment step size coefficient is k, the adjustment amount is A×k, and the adjustment angle parameter is A+(A×k). Other joints obtain their respective adjustment angle parameters in the same way.

[0104] Step S1355: For the original motion speed data of each joint of the robotic arm, determine the positive or negative attribute of the adjustment amount according to the adjustment direction. When the adjustment direction is to increase the speed, the adjustment amount is positive, and when the adjustment direction is to decrease the speed, the adjustment amount is negative.

[0105] For the raw motion speed data of each joint of the robotic arm, the sign of the adjustment amount is determined according to the adjustment direction. When the adjustment direction is to increase the joint motion speed, the adjustment amount is positive; when the adjustment direction is to decrease the joint motion speed, the adjustment amount is negative.

[0106] Step S1356: Calculate the adjustment amount of the joint adjustment speed. The adjustment amount is equal to the original motion speed data multiplied by the adjustment step length coefficient. The adjustment speed parameter is equal to the original motion speed data plus the adjustment amount.

[0107] Taking the original motion velocity data of joint 2 as an example, multiply it by the corresponding adjustment step length coefficient to obtain the adjustment amount of the joint 2's adjustment velocity. If the original motion velocity data is V, the adjustment step length coefficient is k, the adjustment amount is V×k, and the adjustment velocity parameter is V+(V×k). The adjustment velocity parameters of the other joints are calculated in the same way.

[0108] Step S1357: For the original action duration data of the picking execution component, determine the positive or negative attribute of the adjustment amount according to the adjustment direction. When the adjustment direction is to extend the duration, the adjustment amount is positive; when the adjustment direction is to shorten the duration, the adjustment amount is negative.

[0109] For the original action duration data of the picking actuator, the sign of the adjustment amount is determined based on the adjustment direction. When the adjustment direction is to extend the action duration, the adjustment amount is positive; when the adjustment direction is to shorten the action duration, the adjustment amount is negative.

[0110] Step S1358: Calculate the adjustment amount of the picking execution component adjustment time. The adjustment amount is equal to the original action time data multiplied by the adjustment step coefficient. The adjustment time parameter is equal to the original action time data plus the adjustment amount.

[0111] Multiply the original action duration data of the picking execution component by the corresponding adjustment step size coefficient to obtain the adjustment amount of the adjustment duration. If the original action duration data is T, the adjustment step size coefficient is k, the adjustment amount is T×k, and the adjustment duration parameter is T+(T×k).

[0112] Step S1359: Record the calculation process of the adjustment angle parameters, adjustment speed parameters, and adjustment duration parameters of each joint, including the original data values, adjustment step coefficients, adjustment amounts, and final parameter values.

[0113] The original motion angle data, adjustment step length coefficient, adjustment amount, and adjustment angle parameters of each joint, as well as the original motion speed data, adjustment step length coefficient, adjustment amount, and adjustment speed parameters, and the calculation process information such as the original action duration data, adjustment step length coefficient, adjustment amount, and adjustment duration parameters of the picking and grasping execution component, are stored in a log file for easy traceability and verification.

[0114] Step S136: Arrange the calculated adjustment angle parameters, adjustment speed parameters, and adjustment duration parameters of the picking execution component according to the action execution sequence, with each action execution stage corresponding to a set of parameter combinations.

[0115] Based on the sequence of actions performed during apple picking, the picking process is divided into several stages: initial positioning, robotic arm extension, picking execution, robotic arm retraction, and fruit release. The calculated adjustment angle parameters, speed parameters, and adjustment time parameters of the picking execution components are allocated according to the needs of each action stage. Each stage corresponds to a set of parameter combinations containing relevant joint and execution component parameters. For example, the robotic arm extension stage corresponds to the combination of adjustment angle and speed parameters for joints 2 and 3, while the picking execution stage corresponds to the combination of adjustment time parameters for the picking execution components and adjustment speed parameters for joint 4.

[0116] Step S137: Supplement the transition parameters between the various parameter combinations, wherein the transition parameters include the transition time and speed change gradient of the joint adjustment between adjacent action phases.

[0117] Analyze the parameter change requirements between adjacent action execution phases and supplement transition parameters. The transition time is set as the time interval between parameter adjustments between two adjacent phases to ensure a smooth transition of the robotic arm's movements; the speed change gradient is set as the rate of change of joint speed between adjacent phases to avoid sudden speed changes that could lead to motion instability. For example, the transition time from the robotic arm extension phase to the picking execution phase is set to a specific value, and the speed change gradient of joint 2 is set to a specific rate of change to ensure natural motion transitions.

[0118] Step S138: Integrate the parameter combination with the transition parameters to form an initial motion optimization scheme containing a complete motion optimization parameter system.

[0119] The parameters for each stage of the action execution, along with the transition parameters, are integrated in chronological order to form a complete parameter system covering the entire picking process. This complete parameter system includes all parameters such as the adjustment angle and speed of each joint at different stages, the adjustment time of the picking execution components, and the transition time and speed change gradient between stages, which together constitute the initial action optimization scheme.

[0120] Step S140: Perform motion sequence adaptation processing on the initial motion optimization scheme, converting the parameters in the initial motion optimization scheme into a motion sequence that the robotic arm can execute, to obtain the adapted motion sequence.

[0121] For each parameter in the initial motion optimization scheme, format conversion and timing arrangement are performed to convert it into a motion sequence that the robotic arm control system can recognize and execute, i.e., an adapted motion sequence.

[0122] Step S141: Extract the adjustment angle parameters, adjustment speed parameters, adjustment duration parameters of the picking execution component, and transition connection parameters of each joint in the initial motion optimization scheme.

[0123] The adjustment angle parameter sequence and adjustment speed parameter sequence of each joint are extracted one by one from the initial motion optimization scheme. The adjustment duration parameter of the execution component and the transition time and speed change gradient in the transition connection parameters are also extracted to prepare for the subsequent motion sequence conversion.

[0124] Step S142: Convert the adjustment angle parameters of each joint into an angle control sequence for the movement of the robotic arm joints. Each angle control sequence contains target joint angle values ​​corresponding to multiple time nodes, and the interval between time nodes is determined according to the adjustment speed parameters.

[0125] For each joint's adjustment angle parameter sequence, the time interval is determined based on the joint's adjustment speed parameter. The faster the adjustment speed, the smaller the time interval; the slower the adjustment speed, the larger the time interval. At each time node, a corresponding joint angle target value is determined. These time nodes and angle target values ​​are arranged sequentially to form the joint's angle control sequence. For example, if joint 1 has a higher adjustment speed parameter, the time interval is set to a smaller value, and each time node corresponds to a progressively increasing angle target value, forming the angle control sequence for joint 1.

[0126] Step S143: Convert the adjustment speed parameters of each joint into a speed control sequence for the movement of the robotic arm joints. Each speed control sequence contains target joint speed values ​​corresponding to multiple time nodes. The speed control sequence and the angle control sequence are synchronized at the time nodes.

[0127] Using the time nodes of the angle control sequence as a reference, the adjustment speed parameters of each joint are assigned to the corresponding time nodes, and the target joint speed value is determined for each time node, forming a speed control sequence. It is ensured that the time nodes of the speed control sequence are completely consistent with the time nodes of the angle control sequence, achieving synchronous control of angle and speed. For example, if the angle control sequence of joint 2 is set to T1, T2, T3... at a certain time node, then the speed control sequence of joint 2 is set with target speed values ​​at the corresponding time nodes T1, T2, T3..., so that at each time node, joint 2 can achieve the preset angle target while maintaining the corresponding speed target, avoiding motion stuttering or overshoot caused by mismatch between angle and speed.

[0128] Step S144: Convert the adjustment duration parameter of the picking execution component into an action control sequence of the picking execution component. The action control sequence includes the start time node, action maintenance time node and stop time node of the picking execution component. The action maintenance time node corresponds to the adjustment duration parameter.

[0129] Based on the adjustment duration parameter of the picking actuator and the timeline of the overall robotic arm movement, the start time node, action maintenance time node, and stop time node of the picking actuator are determined. The start time node is set as the moment when the robotic arm completes positioning and the picking actuator begins its movement; the action maintenance time node is determined according to the adjustment duration parameter, that is, the time node reached after a time period equal to the adjustment duration parameter from the start time node, representing the moment when the picking actuator completes the core actions such as grasping or cutting; the stop time node is set as the moment when the picking actuator returns to its initial state after completing its movement. These three time nodes are arranged sequentially, and the corresponding actuator movement state (start, maintenance, stop) of each node is labeled to form the motion control sequence of the picking actuator. For example, if the adjustment duration parameter of the picking actuator is a specific duration, the start time node is set to the time T5 when the robotic arm joint adjusts to the picking position, the action maintenance time node is T5 plus the time T6 corresponding to the adjustment duration parameter, and the stop time node is set to T7 after T6 when the actuator completes its reset, thus forming the motion control sequence of the actuator.

[0130] Step S145: Convert the transition connection parameters into a transition control sequence. The transition control sequence includes the joint angle transition target value, the speed transition target value, and the transition time node between adjacent action phases. The transition control sequence is used to connect adjacent angle control sequences and speed control sequences.

[0131] For the transition time and speed change gradient in the transition parameters, the transition time node, joint angle transition target value, and speed transition target value are determined by combining the angle control sequence and speed control sequence of adjacent action stages. The transition time node is set as the connection moment between the end of the previous action stage and the beginning of the next action stage; the joint angle transition target value is set as the transition value between the final angle target value of the previous stage and the initial angle target value of the next stage, ensuring smooth angle change; the speed transition target value is determined according to the speed change gradient, that is, it is gradually adjusted from the current speed target value to the intermediate value of the speed target value of the next stage according to the gradient. The transition time node, the angle transition target value of each joint, and the speed transition target value are integrated in chronological order to form a transition control sequence. For example, in the transition process between the end of the extension stage of the robotic arm and the beginning of the picking execution stage, the transition time node is set as the final time node T8 of the extension stage, the angle transition target value of joint 2 is set as the intermediate value between the final angle of the extension stage and the initial angle of the picking stage, and the speed transition target value is set as the transition value from the speed of the extension stage to the initial speed of the picking stage according to the preset gradient. These parameters together constitute the transition control sequence of this transition process.

[0132] Step S146: Integrate the angle control sequence, speed control sequence, action control sequence of the picking execution component, and transition control sequence according to the time node sequence, with each time node corresponding to a complete set of control parameters.

[0133] For example, step S1461: Extract all time nodes in the angle control sequence to form an angle time node set; extract all time nodes in the speed control sequence to form a speed time node set; extract all time nodes in the picking execution component action control sequence to form an execution time node set; extract all time nodes in the transition control sequence to form a transition time node set.

[0134] The angle control sequence of each joint is traversed, and all time nodes are collected. After removing duplicate nodes, an angle time node set is formed. Using the same method, time nodes are extracted from the speed control sequence, the picking execution component action control sequence, and the transition control sequence to form a speed time node set, an execution time node set, and a transition time node set. The time nodes in each set are initially arranged in chronological order to prepare for subsequent merging.

[0135] Step S1462: Merge the set of angle time nodes, the set of velocity time nodes, the set of execution time nodes, and the set of transition time nodes, remove duplicate time nodes, and sort them in chronological order to form a unified time node sequence.

[0136] All nodes from the four time point sets are placed into a temporary set. By comparing the time information of the nodes, duplicate nodes are removed, and only unique time points are retained. Then, these unique nodes are sorted in chronological order from earliest to latest to form a unified time point sequence that runs through the entire harvesting process, ensuring that the time base of all control sequences is consistent.

[0137] Step S1463: Traverse each time node in the unified time node sequence, find the joint angle target value corresponding to the time node from the angle control sequence. If the time node does not exist directly in the angle control sequence, perform linear interpolation based on the joint angle target values ​​of adjacent time nodes to obtain the supplementary angle target value of the time node.

[0138] Each node in the unified time node sequence is iterated through one by one. For each node, the angle control sequence of each joint is searched to see if there is a common time node. If it exists, the joint angle target value corresponding to that node is directly extracted; if it does not exist, the two adjacent time nodes (i.e., the preceding and following time nodes) are found, and the joint angle target values ​​of these two nodes in the angle control sequence are obtained. The supplementary angle target value of the current time node is calculated by linear interpolation to ensure the continuity of joint angles on the time axis. For example, if node T9 in the unified time node sequence is located between T8 and T10 in the angle control sequence, and the joint 3 angle target value corresponding to T8 is A8, and the angle target value corresponding to T10 is A10, then the supplementary angle target value corresponding to T9 is calculated by linear interpolation.

[0139] Step S1464: Find the joint speed target value corresponding to each time node in the speed control sequence. If the time node does not exist directly in the speed control sequence, perform linear interpolation based on the joint speed target values ​​of adjacent time nodes to obtain the supplementary speed target value for that time node.

[0140] Similar to the method for obtaining the angle target value, for each node in the unified time node sequence, a search is performed in the speed control sequence of each joint. If the corresponding node exists, the speed target value is directly extracted; if it does not exist, a supplementary speed target value is obtained by linear interpolation based on the speed target values ​​of the preceding and following nodes, ensuring a smooth transition of joint speed throughout the entire movement and avoiding sudden speed changes.

[0141] Step S1465: Find the action state of the execution component corresponding to each time node from the action control sequence of the picking execution component. The action state of the execution component includes start state, maintenance state, and stop state. If the time node is between the start time node and the maintenance time node, the state is maintenance state; if the time node is between the maintenance time node and the stop time node, the state is stop state.

[0142] For each time node in the unified time node sequence, it is compared with the start time node, action maintenance time node, and stop time node in the action control sequence of the picking execution component. If the time node is equal to the start time node, the action state is the start state; if the time node is between the start time node and the action maintenance time node, the action state is the maintenance state; if the time node is equal to the action maintenance time node, the action state is the critical state transitioning from the maintenance state to the stop state; if the time node is between the action maintenance time node and the stop time node, the action state is the stop state; if the time node is equal to or later than the stop time node, the action state is the stop completed state.

[0143] Step S1466: Find the transition parameters corresponding to each time node from the transition control sequence. The transition parameters include the joint angle transition target value and the speed transition target value. If the time node is within the transition time node range, extract the corresponding transition parameters; if it is outside the transition time node range, set the transition parameters to zero.

[0144] Each node in the unified time node sequence is matched with the transition time node range in the transition control sequence. The transition time node range is the interval from the transition start time node to the transition end time node. If the current time node is within this interval, the corresponding joint angle transition target value and velocity transition target value are extracted from the transition control sequence. If the current time node is before the transition start time node or after the transition end time node, the joint angle transition target value and velocity transition target value corresponding to that node are both set to zero, indicating that no transition control needs to be performed at this time.

[0145] Step S1467: Combine the target or supplementary target angle value, target or supplementary target speed value, action status of the actuator and transition parameters corresponding to each time node to form a complete control parameter group for that time node. Arrange all the complete control parameter groups of all time nodes in chronological order to form an integrated control parameter group sequence, with each time node corresponding to a complete set of control parameters.

[0146] For each node in the unified time node sequence, the previously acquired target angle values ​​(or supplementary target angle values), target velocity values ​​(or supplementary target velocity values), action status of the picking and grasping actuator, and transition parameters are summarized to form a complete control parameter set for that time node. For example, the control parameter set for time node T11 includes the target angle values ​​for joints 1 to 6, the target velocity values ​​for joints 1 to 6, the maintenance status of the picking and grasping actuator, and the transition parameters for each joint (if in the transition phase). All complete control parameter sets for all time nodes are arranged sequentially according to time to form an integrated control parameter set sequence, ensuring that each time node has unique and complete control parameter support.

[0147] Step S147: Delete duplicate time nodes and duplicate control parameters that occur during the integration process, retain the unique control parameter group corresponding to each time node, perform time axis calibration on the integrated control parameter group, so that all control parameter groups are arranged according to a uniform time interval to form a continuous motion control parameter sequence, and convert the motion control parameter sequence into an instruction format sequence that the robotic arm control system can recognize to obtain the adapted motion sequence.

[0148] In the integrated sequence of control parameter groups, duplicate time nodes are checked. If any are found, one set of control parameter groups is retained, and the parameter groups corresponding to the remaining duplicate nodes are deleted. Simultaneously, control parameter groups at different time nodes are compared. If duplicate parameter groups with identical parameters exist, redundant groups can be retained or deleted based on the requirements of motion continuity. Subsequently, the control parameter group sequence is calibrated along a time axis using a unified time interval as a benchmark. If the time intervals of adjacent parameter groups do not conform to the unified standard, intermediate parameter groups are supplemented through interpolation, or the node times are adjusted to ensure consistent intervals, forming a continuous and uninterrupted sequence of motion control parameters. Finally, according to the communication protocol and instruction format requirements of the robotic arm control system, each parameter group in the motion control parameter sequence is converted into a corresponding instruction code. For example, the target value of the joint angle is converted into a numerical code recognizable by the control system, and the motion state of the actuator is converted into a corresponding state instruction code. These instruction codes are arranged in chronological order to form an adapted motion sequence.

[0149] Step S150: Convert the adapted action sequence into action control instructions for the picking robotic arm, and send the action control instructions for the picking robotic arm to the picking robotic arm control system. After receiving the action control instructions for the picking robotic arm, the picking robotic arm control system drives the picking robotic arm to perform the picking action.

[0150] The instruction codes in the adapted motion sequence are encapsulated according to the instruction frame format of the robotic arm control system. Each instruction frame includes information such as timestamp, joint number, parameter type, parameter value, and checksum to ensure the accuracy and integrity of instruction transmission. After encapsulation, the robotic arm motion control instructions are sent to the robotic arm control system via industrial Ethernet or a dedicated communication bus. Upon receiving the instructions, the robotic arm control system first verifies the instruction frames. After successful verification, it parses the control parameters for each time node and drives the servo motors of each joint to move at preset angles and speeds based on the parameters. It also controls the picking execution components to start, maintain, and stop actions at designated time nodes, ultimately completing the entire apple picking process.

[0151] Step S210: Collect historical action association rules and historical action optimization targets of the picking robotic arm in historical picking scenarios, decompose the historical action association rules into historical rule units, and decompose the historical action optimization targets into historical target units to form a model training dataset. The model training dataset contains multiple sets of corresponding samples of historical rule units and historical target units.

[0152] Historical action association rules accumulated during past apple harvesting processes are extracted from a rule database. These rules cover different historical harvesting scenarios (such as apple harvesting scenarios in different seasons and orchards). Simultaneously, corresponding historical harvesting action optimization objectives are collected, including historical harvesting efficiency objectives, action stability objectives, and damage rate control objectives. The historical action association rules are decomposed into multiple historical rule units using the same decomposition method as the current rule units. Each historical rule unit corresponds to a specific historical association feature description. The historical harvesting action optimization objectives are decomposed into historical target units, each historical target unit corresponding to a historical optimization index. Each historical rule unit is matched with its corresponding historical target unit to form a set of samples. Multiple sets of samples together constitute the model training dataset.

[0153] Step S220: Divide the model training dataset into a training sample subset and a validation sample subset according to a preset ratio. The training sample subset is used for model parameter training, and the validation sample subset is used for model performance verification.

[0154] The preset ratio is set as 70% for the training sample subset and 30% for the validation sample subset. A random sampling method is used to select 70% of the samples from the model training dataset to form the training sample subset, and the remaining 30% to form the validation sample subset. During the sampling process, it is ensured that both the training and validation sample subsets cover samples from different historical harvesting scenarios to avoid uneven sample distribution that could lead to model training bias.

[0155] Step S230: Input the subset of training samples into the feature input layer of the machine learning model, perform feature encoding processing on the historical rule units and historical target units, and generate historical rule feature vectors and historical target feature vectors.

[0156] The historical rule units and historical target units from the training sample subset are input into the feature input layer of the machine learning model. The feature input layer first preprocesses the text content of the historical rule units and historical target units, including removing meaningless auxiliary words, standardizing the expression of technical terms, and splitting long texts into shorter sentences. After preprocessing, a bag-of-words model combined with the TF-IDF algorithm is used to extract features from the text. Each historical rule unit is converted into a fixed-dimensional historical rule feature vector, and each historical target unit is converted into a historical target feature vector of the same dimension. Each element in the vector corresponds to the weight value of a feature word, representing the importance of that feature word in the text.

[0157] Step S240: Input the historical rule feature vector and the historical target feature vector into the association modeling layer, initialize the multilayer perceptron structure parameters of the association modeling layer, including the weight matrix and bias vector of each layer, and set the number of training iterations and the learning rate.

[0158] The generated historical rule feature vector is concatenated with the historical target feature vector to form a combined feature vector, which is then input into the association modeling layer. The multilayer perceptron in the association modeling layer contains three hidden layers. The first layer has a specific number of neurons, and the number of neurons in the second and third layers decreases sequentially. The weight matrices and bias vectors of each hidden layer are initialized. The initial values ​​of the weight matrices are generated using the Xavier initialization method, and the initial values ​​of the bias vectors are set to zero. A specific number of training iterations and a specific learning rate are set. The learning rate controls the magnitude of each parameter update, and the number of iterations controls the total number of training epochs.

[0159] Step S250: In each iteration, the correlation prediction value between the historical rule feature vector and the historical target feature vector is calculated through the correlation modeling layer. The correlation prediction value is compared with the preset correlation true value in the sample, and the prediction error is calculated.

[0160] In each iteration, the combined feature vector passes through the first hidden layer of the association modeling layer, undergoes matrix multiplication with the weight matrix of that layer, and is then added with the bias vector to obtain a linear transformation result. This result is then subjected to a non-linear transformation using the ReLU activation function to generate the output features of the first hidden layer. The output features of the first hidden layer are input into the second hidden layer, and the linear transformation and ReLU activation process is repeated to obtain the output features of the second hidden layer. The output features of the second hidden layer are then input into the third hidden layer, undergoing a linear transformation and a Sigmoid activation function to obtain the association degree prediction value. This prediction value ranges from 0 to 1, representing the predicted association degree between historical rule units and historical target units. The association degree prediction value is compared with the manually labeled true association degree values ​​in the samples, and the mean squared error loss function is used to calculate the prediction error between the two. The mean squared error loss function is calculated as the average of the squares of the differences between the predicted and true values.

[0161] Step S260: Based on the prediction error, the weight matrix and bias vector of the association modeling layer are adjusted using the gradient descent algorithm, and the encoding parameters of the feature input layer are updated to reduce the overall prediction error of the training sample subset.

[0162] Based on the calculated prediction error, the stochastic gradient descent algorithm is used to backpropagate the error signal. First, the partial derivative of the loss function with respect to the output of the association modeling layer is calculated. Then, according to the chain rule, the partial derivatives of the loss function with respect to the weight matrices and bias vectors of the third, second, and first hidden layers are calculated sequentially. Based on the direction and magnitude of the partial derivatives, and in conjunction with the preset learning rate, the weight matrices and bias vectors of each layer are adjusted to reduce the prediction error. Simultaneously, the feature word weight calculation parameters of the TF-IDF algorithm in the feature input layer are adjusted based on the error signal to optimize the feature encoding effect and further reduce the overall prediction error of the training sample subset.

[0163] Step S270: After completing a preset number of iterations, input the validation sample subset into the trained machine learning model, calculate the correlation prediction error of the validation sample subset, and if the prediction error meets the preset requirements, determine that the model training is complete.

[0164] After the model completes a preset number of iterations, the historical rule units and historical target units from the validation sample subset are input into the trained machine learning model. Following the same processing flow as the training samples, historical rule feature vectors and historical target feature vectors are generated. These are then processed by the association modeling layer to calculate the association degree prediction value. The average prediction error of the validation sample subset is calculated. If this error is less than a preset error threshold (i.e., meets the preset requirements), it indicates that the model has good generalization ability and can accurately predict the association degree of different samples, thus confirming that the model training is complete.

[0165] Step S280: If the prediction error of the verification sample subset does not meet the preset requirements, adjust the number of training iterations and the learning rate, and retrain the model until the prediction error of the verification sample subset meets the preset requirements. Save the parameters of each layer of the trained machine learning model to form a callable pre-trained machine learning model.

[0166] If the average prediction error of the validation sample subset is greater than or equal to the preset error threshold, it indicates that the model may have underfitting or overfitting issues. In this case, adjust the number of training iterations, appropriately increasing the number of iterations to address the underfitting problem; simultaneously adjust the learning rate. If the learning rate was previously too high, causing parameter oscillations, appropriately decrease the learning rate; if the learning rate was too low, causing slow convergence, appropriately increase the learning rate. After adjustment, retrain the model using the adjusted parameters, repeating steps S230 to S270 until the prediction error of the validation sample subset meets the preset requirements. After model training is complete, save all model parameters, including the encoding parameters of the feature input layer, the weight matrices and bias vectors of each hidden layer in the association modeling layer, to the model file, forming a pre-trained machine learning model that can be called upon at any time.

[0167] Figure 2The diagram illustrates exemplary hardware and software components of a harvesting robotic arm motion optimization method system 100 incorporating machine learning, which can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the harvesting robotic arm motion optimization method system 100 incorporating machine learning and to perform the functions described in this application.

[0168] The system 100 for optimizing the actions of a harvesting robotic arm using machine learning can be a general-purpose server or a special-purpose server; both can be used to implement the machine learning-integrated robotic arm action optimization method of this application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the load.

[0169] For example, the machine learning-integrated harvesting robot motion optimization method system 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the machine learning-integrated harvesting robot motion optimization method system 100 may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The machine learning-integrated harvesting robot motion optimization method system 100 also includes an I / O interface 150 between the computer and other input / output devices.

[0170] For ease of explanation, only one processor is described in the machine learning-integrated robotic arm motion optimization method system 100. However, it should be noted that the machine learning-integrated robotic arm motion optimization method system 100 of this application may also include multiple processors. Therefore, the steps executed by one processor as described in this application may also be executed jointly or individually by multiple processors. For example, if the processor of the machine learning-integrated robotic arm motion optimization method system 100 executes steps A and B, it should be understood that steps A and B may also be executed jointly by two different processors or individually by one processor. For example, the first processor executes step A, the second processor executes step B, or the first processor and the second processor jointly execute steps A and B.

[0171] Furthermore, this embodiment of the invention also provides a readable storage medium, wherein computer-executable instructions are preset in the readable storage medium, and when the processor executes the computer-executable instructions, the above-mentioned method for optimizing the actions of the picking robotic arm combined with machine learning is implemented.

[0172] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for optimizing the movements of a harvesting robotic arm by incorporating machine learning, characterized in that, The method includes: Obtain motion data sets of the picking robotic arm under different picking scenarios, perform correlation analysis on the motion data sets, and generate motion correlation rules. The motion data sets include motion angle data, motion speed data, and motion duration data of each joint of the robotic arm and the picking execution component. Call a pre-trained machine learning model, input the action association rules and the preset picking action optimization target into the machine learning model, establish the mapping relationship between the action association rules and the picking action optimization target, and obtain a mapping relationship table; Based on the mapping table, an initial motion optimization scheme for the picking robot arm is generated. The initial motion optimization scheme includes adjustment angle parameters, adjustment speed parameters, and adjustment duration parameters for each joint and the picking execution component. The initial motion optimization scheme is subjected to motion sequence adaptation processing, and the parameters in the initial motion optimization scheme are converted into a motion sequence that can be executed by the robotic arm to obtain the adapted motion sequence. The adapted action sequence is converted into action control instructions for the picking robotic arm, and the action control instructions for the picking robotic arm are sent to the picking robotic arm control system. After receiving the action control instructions for the picking robotic arm, the picking robotic arm control system drives the picking robotic arm to perform the picking action. The step of performing motion sequence adaptation processing on the initial motion optimization scheme, converting the parameters in the initial motion optimization scheme into a motion sequence executable by the robotic arm, to obtain an adapted motion sequence includes: Extract the adjustment angle parameters, adjustment speed parameters, adjustment duration parameters of the picking execution component, and transition connection parameters of each joint in the initial motion optimization scheme; The adjustment angle parameters of each joint are converted into angle control sequences for the movement of the robotic arm joints. Each angle control sequence contains target joint angle values ​​corresponding to multiple time nodes, and the interval between time nodes is determined according to the adjustment speed parameters. The adjustment speed parameters of each joint are converted into speed control sequences for the movement of the robotic arm joints. Each speed control sequence contains target joint speed values ​​corresponding to multiple time nodes. The speed control sequence and the angle control sequence are kept synchronized at the time nodes. The adjustment duration parameter of the picking execution component is converted into an action control sequence of the picking execution component. The action control sequence includes the start time node, action maintenance time node and stop time node of the picking execution component. The action maintenance time node corresponds to the adjustment duration parameter. The transition parameters are converted into a transition control sequence, which includes joint angle transition target values, speed transition target values ​​and transition time nodes between adjacent motion phases. The transition control sequence is used to connect adjacent angle control sequences and speed control sequences. The angle control sequence, speed control sequence, action control sequence of the picking execution component, and transition control sequence are integrated according to the time node sequence, with each time node corresponding to a complete set of control parameters; Duplicate time nodes and duplicate control parameters that occur during the integration process are deleted, and the unique control parameter group corresponding to each time node is retained. The integrated control parameter group is time-axis calibrated so that all control parameter groups are arranged according to a uniform time interval to form a continuous motion control parameter sequence. The motion control parameter sequence is converted into an instruction format sequence that the robotic arm control system can recognize to obtain an adapted motion sequence. The process of acquiring motion data sets of the harvesting robotic arm under different harvesting scenarios, performing correlation analysis on the motion data sets, and generating motion correlation rules includes: The system receives a set of motion data transmitted by the picking robotic arm through joint sensors and motion timers. The joint sensors are used to collect motion angle data and motion speed data of each joint of the robotic arm, and the motion timers are used to collect motion duration data of the picking execution components. The action data set is divided according to the picking scene type. Each picking scene type corresponds to a set of scene action data subsets. The picking scene type is determined according to the morphological characteristics and growth environment characteristics of the picking object. Each subset of scene action data is segmented into multiple action data blocks. Each action data block contains motion angle data, motion speed data, and action duration data of each joint of the robotic arm within a single complete picking action cycle. Extract the associated features from each of the motion data blocks. The associated features include the correspondence between motion angle data and motion speed data of the same joint, the coordination relationship between motion angle data of different joints, and the correspondence between motion duration data of the picking and executing component and joint motion speed data. Calculate the similarity of associated features between different action data blocks in the same subset of action data for the same scene. For each associated feature, calculate the similarity of its data sequence in different data blocks separately to obtain the independent similarity score of each feature. Based on the importance weight of each associated feature in the picking action, combine all independent similarity scores to obtain the overall similarity of associated features. Based on the correlation feature similarity, action data block groups with similar correlation features are selected, and each action data block group contains multiple action data blocks whose correlation feature similarity meets preset conditions. Analyze the commonalities of the associated features in each action data block group, and extract rule entries that can reflect the association patterns of the action data blocks in the action data block group. Each rule entry contains a description of the specific correspondence between the associated features. All the rule entries are categorized and organized according to the picking scenario type. Each picking scenario type corresponds to a set of rule entries, forming a complete action association rule.

2. The method for optimizing the actions of a harvesting robotic arm by combining machine learning according to claim 1, characterized in that, The extraction of associated features from each action data block includes: From each of the motion data blocks, the motion angle data, motion speed data, and motion duration data of each joint of the robotic arm and the picking execution component are separated, and each data type corresponds to an independent data sequence; For the motion angle data and motion velocity data of the same joint, a one-to-one correspondence is established according to the time node sequence. The motion angle data value and motion velocity data value of each time node form a data pair. Multiple data pairs form the angle-velocity correspondence sequence of the joint. The angle-velocity correspondence sequence is the correspondence between the motion angle data and motion velocity data of the same joint. Select joint groups in the robotic arm that have a motion coordination relationship, wherein the joint group contains two or more joints that move simultaneously during the picking action, and extract the motion angle data sequence of each joint in the joint group; The temporal synchronicity of the joint motion angle data sequences in the joint group is calculated, and the synchronicity level is determined by comparing the time difference of the peak occurrence in different joint motion angle data sequences. The synchronicity level is related to the time difference. Based on the synchronization level and the proportion of changes in the motion angle data of each joint, a description of the collaborative relationship between the motion angle data of different joints in the joint group is established. The collaborative relationship description includes the synchronization level value and the proportion of the angle change. Extract the action duration data of the picking execution component from each action data block, determine the start and end time points of the action duration, extract the motion speed data sequence of all joints during the period from the start time point to the end time point, and calculate the average value and change range of the motion speed data of each joint in the corresponding time period. Establish a correspondence between the action duration data of the picking execution component and the average value and variation range of the joint movement speed data. Each action duration data value corresponds to a set of average joint movement speed and variation range values, forming a duration-speed correspondence. Integrate the angle-speed correspondence of the same joint, the coordination relationship of movement angle data between different joints, and the duration-speed correspondence of the picking execution component to form the associated features of each action data block.

3. The method for optimizing the actions of a harvesting robotic arm by combining machine learning according to claim 1, characterized in that, The rule entries are categorized and organized according to the picking scenario type, with each picking scenario type corresponding to a set of rule entries, forming a complete action association rule, including: Establish a classification directory of picking scene types. The classification directory contains all the identified picking scene type names and corresponding scene feature descriptions. The scene feature descriptions include specific content such as the morphological characteristics of the picking objects and the characteristics of the growing environment. Traverse each rule entry and extract the scene information corresponding to the associated features involved in the rule entry. The scene information includes the picking scene type to which the action data block from which the rule entry originates belongs. Based on the extracted scene information, the rule entries are assigned to the corresponding picking scene type in the category directory to form the initial rule group for each scene type; For each scenario type, an initial rule group is used to detect rule conflicts. The association feature descriptions and correspondences between different rule entries in the initial rule group are compared. If there are rule entries with the same association feature descriptions but different correspondences, they are marked as conflicting rule pairs. For each conflicting rule pair, query the correlation feature similarity statistics of the action data block group from which it originates, retain the rule entry with the largest size of the source data block group and the largest average correlation feature similarity, and delete the other conflicting rule entry. Supplement the rule entries corresponding to the missing related features in the initial rule group for each scene type. If any related feature has no corresponding rule entry under the scene type, select the rule entry with the highest correlation from the rule group of similar picking scene types and adapt and modify it to form a supplementary rule entry. Logically sort the rule groups for each scenario type. According to the execution order of the actions involved in the associated features, the rule entries are divided into joint motion association rules, execution component action association rules, and collaborative action association rules. After sorting according to the complexity of the associated features, a scenario identifier is added to the sorted rule groups. The scenario identifier includes the picking scenario type name and the rule group version number. Summarize the rule groups for all picking scenario types and generate a rule index table. The rule index table includes the scenario type name, rule group version number, and number of rule entries. The summarized rule groups and rule index tables are integrated to form complete action association rules, which are then stored in the rule database.

4. The method for optimizing the actions of a harvesting robotic arm by combining machine learning according to claim 1, characterized in that, The process involves calling a pre-trained machine learning model, inputting the action association rules and preset picking action optimization targets into the machine learning model, establishing a mapping relationship between the action association rules and the picking action optimization targets, and obtaining a mapping relationship table, including: Determine the preset optimization goals for the picking action. The optimization goals for the picking action include the efficiency goal of completing the picking action, the stability goal of the picking action, and the damage rate control goal of the picking object. Each optimization goal has a corresponding descriptive index. The action association rule is decomposed into multiple rule units, each rule unit corresponding to a specific association feature description. At the same time, the picking action optimization target is decomposed into multiple target units, each target unit corresponding to a description index of the optimization target. The rule unit and the target unit are input into a pre-trained machine learning model, which includes a feature input layer, an association modeling layer, and a mapping output layer. In the feature input layer, feature encoding processing is performed on the rule units and the target units to convert the text-based rule units and target units into vector-based feature data. Each rule unit corresponds to a rule feature vector, and each target unit corresponds to a target feature vector. The regular feature vector and the target feature vector are input into the association modeling layer. The association modeling layer adopts a multilayer perceptron structure and calculates the correlation degree between the regular feature vector and the target feature vector through linear transformation and nonlinear activation function. The matching relationship between the rule unit and the target unit is determined based on the correlation value. The rule unit and the target unit that meet the preset conditions form a matching pair. Each matching pair contains a rule unit and a corresponding target unit. All the matching pairs are grouped according to the picking scene type, and the matching pairs under each picking scene type form a local mapping relationship for that picking scene type; The local mapping relationships of each picking scene type are integrated, and the association and connection relationships of matching pairs between different scenes are supplemented to form a global mapping relationship covering all picking scene types. The global mapping relationship is presented in tabular form to obtain the mapping relationship table. The row dimension of the table is the rule unit, and the column dimension is the target unit. The corresponding correlation value and matching condition are recorded in the table cell.

5. The method for optimizing the actions of a harvesting robotic arm by combining machine learning according to claim 4, characterized in that, The step of inputting the rule unit and the target unit into a pre-trained machine learning model includes: Collect historical action association rules and historical action optimization targets of the harvesting robot arm in historical harvesting scenarios. Decompose the historical action association rules into historical rule units and the historical action optimization targets into historical target units to form a model training dataset. The model training dataset contains multiple sets of corresponding samples of historical rule units and historical target units. The model training dataset is divided into a training sample subset and a validation sample subset according to a preset ratio. The training sample subset is used for model parameter training, and the validation sample subset is used for model performance verification. The training sample subset is input into the feature input layer of the machine learning model to perform feature encoding on the historical rule unit and the historical target unit, thereby generating historical rule feature vector and historical target feature vector. Input the historical rule feature vector and the historical target feature vector into the association modeling layer, initialize the multilayer perceptron structure parameters of the association modeling layer, including the weight matrix and bias vector of each layer, and set the number of training iterations and the learning rate. In each iteration, the correlation prediction value between the historical rule feature vector and the historical target feature vector is calculated through the correlation modeling layer. The correlation prediction value is then compared with the preset true correlation value in the sample to calculate the prediction error. Based on the prediction error, the gradient descent algorithm is used to adjust the weight matrix and bias vector of the association modeling layer, and the encoding parameters of the feature input layer are updated to reduce the overall prediction error of the training sample subset. After completing a preset number of iterations, the validation sample subset is input into the trained machine learning model, and the correlation prediction error of the validation sample subset is calculated. If the prediction error meets the preset requirements, the model training is determined to be complete. If the prediction error of the validation sample subset does not meet the preset requirements, adjust the number of training iterations and the learning rate, and retrain the model until the prediction error of the validation sample subset meets the preset requirements. Then, save the parameters of each layer of the trained machine learning model to form a pre-trained machine learning model that can be called.

6. The method for optimizing the actions of a harvesting robotic arm by combining machine learning according to claim 1, characterized in that, The initial motion optimization scheme for the harvesting robotic arm, generated based on the mapping table, includes: Extract the rule units and matching target units corresponding to the current picking scenario type from the mapping relationship table, and determine the associated features that need to be optimized and the corresponding optimization target description indicators under the current picking scenario type; Based on the association feature description in the rule unit, locate the original motion parameters corresponding to the association feature in the motion data set. The original motion parameters include the original motion angle data, original motion speed data, and original motion duration data of each joint of the robotic arm and the picking execution component. Referring to the optimized target description index in the target unit, the original action parameters are input into a predefined performance evaluation model. The performance evaluation model outputs a predicted performance index value based on the current action parameters, and the difference between the predicted performance index value and the optimized target description index is calculated. Based on the differences and the correlation values ​​in the mapping table, the adjustment direction of each original action parameter is determined. The adjustment direction is determined according to the positive or negative attribute of the differences and the magnitude of the correlation values. Based on the adjustment direction and the preset adjustment step length rule, the adjustment angle parameter, adjustment speed parameter and adjustment duration parameter of each joint are calculated. The adjustment angle parameter is the result of superimposing the original motion angle data and the adjustment amount, the adjustment speed parameter is the result of superimposing the original motion speed data and the adjustment amount, and the adjustment duration parameter is the result of superimposing the original action duration data and the adjustment amount. The calculated adjustment angle parameters, adjustment speed parameters, and adjustment duration parameters of the picking execution components are arranged according to the action execution sequence, with each action execution stage corresponding to a set of parameter combinations; Supplement the transition parameters between the various parameter combinations, which include the transition time and speed change gradient of joint adjustment between adjacent action phases; By integrating parameter combinations with transition parameters, an initial motion optimization scheme containing a complete motion optimization parameter system is formed.

7. The method for optimizing the actions of a harvesting robotic arm by combining machine learning according to claim 6, characterized in that, Based on the adjustment direction and the preset adjustment step length rule, the adjustment angle parameters, adjustment speed parameters, and adjustment duration parameters of the picking execution component for each joint are calculated. The adjustment angle parameters are the superposition result of the original motion angle data and the adjustment amount, including: Obtain a preset adjustment step size rule, wherein the adjustment step size rule includes adjustment step size coefficients corresponding to different correlation values, and the correlation values ​​and adjustment step size coefficients are in a corresponding relationship; Based on the correlation values ​​between the rule units and the target units in the mapping table, the corresponding adjustment step size coefficient is selected from the adjustment step size rules; For the original motion angle data of each joint of the robotic arm, the positive or negative attribute of the adjustment amount is determined according to the adjustment direction. When the adjustment direction is to increase the angle, the adjustment amount is positive, and when the adjustment direction is to decrease the angle, the adjustment amount is negative. Calculate the adjustment amount of the joint adjustment angle. The adjustment amount is equal to the original motion angle data multiplied by the adjustment step length coefficient. The adjustment angle parameter is equal to the original motion angle data plus the adjustment amount. For the original motion speed data of each joint of the robotic arm, the positive and negative attributes of the adjustment amount are determined according to the adjustment direction. When the adjustment direction is to increase the speed, the adjustment amount is positive, and when the adjustment direction is to decrease the speed, the adjustment amount is negative. Calculate the adjustment amount of the joint adjustment speed. The adjustment amount is equal to the original motion speed data multiplied by the adjustment step length coefficient. The adjustment speed parameter is equal to the original motion speed data plus the adjustment amount. For the original action duration data of the picking execution component, the positive or negative attribute of the adjustment amount is determined according to the adjustment direction. When the adjustment direction is to extend the duration, the adjustment amount is positive, and when the adjustment direction is to shorten the duration, the adjustment amount is negative. Calculate the adjustment amount for the adjustment time of the picking execution component. The adjustment amount is equal to the original action time data multiplied by the adjustment step coefficient. The adjustment time parameter is equal to the original action time data plus the adjustment amount. Record the calculation process of adjustment angle parameters, adjustment speed parameters, and adjustment duration parameters of each joint, including original data values, adjustment step coefficients, adjustment amounts, and final parameter values.

8. A method system for optimizing the movements of a harvesting robotic arm by combining machine learning, characterized in that, The machine learning-integrated robotic arm motion optimization method system includes a processor and a memory, the memory and the processor being connected, the memory being used to store programs, instructions or code, and the processor being used to execute the programs, instructions or code in the memory to implement the machine learning-integrated robotic arm motion optimization method as described in any one of claims 1-7.

Images