Shaft hole data processing method and device, equipment and computer readable storage medium

Abstract

This application provides a method, apparatus, device, and computer-readable storage medium for shaft and hole data processing. The method includes: determining the initial contact state between the assembled shaft and the assembled hole, and performing state transition processing on the initial contact state based on a preset shaft and hole exploration trajectory to obtain N target contact states, where N is a positive integer; for each target contact state, determining the contact sensing data between the assembled shaft and the assembled hole in the target contact state, and performing trajectory prediction based on the N contact sensing data to obtain a predicted exploration trajectory and a trajectory compensation amount; performing control quantity prediction based on the predicted exploration trajectory and the trajectory compensation amount to obtain a target control quantity, and using the target control quantity to control a robotic arm to perform shaft and hole assembly. This application can improve the efficiency and accuracy of shaft and hole assembly.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and computer-readable storage medium for processing shaft hole data. Background Technology

[0002] Shaft-hole assembly is a fundamental process in mechanical manufacturing, its core objective being the precise fit between shafts and holes to form a stable mechanical connection. With the rapid development of industrial automation, precision manufacturing, and intelligent manufacturing, shaft-hole assembly technology faces increasingly higher demands. As an important research direction in the field of robotics applications, shaft-hole assembly not only has broad application prospects in manufacturing but also promotes the development of intelligent manufacturing technology. Researching robot shaft-hole assembly technology based on force control and teach-and-learn can facilitate the optimization and upgrading of intelligent manufacturing systems, thereby improving overall production efficiency and product quality.

[0003] In related technologies, the design process of shaft and hole assembly strategies is complex and difficult to generalize to various shaft and hole assembly tasks. The design and debugging processes rely on expert knowledge, resulting in low efficiency and accuracy of shaft and hole assembly operations. Summary of the Invention

[0004] This application provides a shaft hole data processing method, apparatus, device, and computer-readable storage medium, which can improve the efficiency and accuracy of shaft hole assembly.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a shaft hole data processing method, the method comprising:

[0007] The initial contact state between the assembly shaft and the assembly hole is determined, and based on the preset shaft-hole exploration trajectory, the initial contact state is processed to obtain N target contact states, where N is a positive integer. The state transition process is to transform the initial contact state into multiple contact states.

[0008] For each target contact state, the contact sensing data of the assembly shaft and the assembly hole under the target contact state are determined, and trajectory prediction is performed based on N contact sensing data to obtain the predicted exploration trajectory and trajectory compensation amount.

[0009] Based on the predicted exploration trajectory and the trajectory compensation amount, the control quantity is predicted to obtain the target control quantity, and the target control quantity is used to control the robotic arm to perform shaft hole assembly.

[0010] This application provides a shaft hole data processing device, including:

[0011] The state transition module is used to determine the initial contact state between the assembly shaft and the assembly hole, and based on the preset shaft-hole exploration trajectory, to perform state transition processing on the initial contact state to obtain N target contact states, where N is a positive integer. The state transition processing is to transform the initial contact state into multiple contact states.

[0012] The first prediction module is used to determine the contact sensing data of the assembly shaft and the assembly hole in the target contact state for each target contact state, and to perform trajectory prediction based on N contact sensing data to obtain the predicted exploration trajectory and trajectory compensation amount.

[0013] The second prediction module is used to predict the control quantity based on the predicted exploration trajectory and the trajectory compensation amount, obtain the target control quantity, and use the target control quantity to control the robotic arm to perform shaft hole assembly.

[0014] This application provides an electronic device, the electronic device comprising:

[0015] Memory is used to store executable instructions or computer programs.

[0016] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the shaft hole data processing method provided in the embodiments of this application.

[0017] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs for implementing the shaft hole data processing method provided in this application when executed by a processor.

[0018] This application provides a computer program product, including computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, they implement the shaft hole data processing method provided in this application.

[0019] The embodiments of this application have the following beneficial effects:

[0020] By applying the embodiments of this application, the initial contact state between the assembly shaft and the assembly hole is determined, and based on the preset shaft-hole exploration trajectory, the initial contact state is processed to obtain N target contact states, where N is a positive integer. Based on the preset shaft-hole exploration trajectory, multiple contact states between the assembly shaft and the assembly hole can be dynamically switched. Then, for each target contact state, the contact perception data of the assembly shaft and the assembly hole in the target contact state is determined, and trajectory prediction is performed based on the N contact perception data to obtain the predicted exploration trajectory and trajectory compensation amount. This realizes the comprehensive prediction of the shaft-hole exploration trajectory and trajectory compensation amount by combining the contact perception data of the assembly shaft and the assembly hole in multiple contact states, which can improve the accuracy of shaft-hole assembly. Then, based on the predicted exploration trajectory and trajectory compensation amount, the control quantity is predicted to obtain the target control quantity, and the target control quantity is used to control the robotic arm to perform shaft-hole assembly. In this way, by using contact perception data under different contact states, the shaft hole exploration trajectory and trajectory compensation amount are predicted, and the control quantity used for shaft hole assembly is determined comprehensively based on the predicted exploration trajectory and trajectory compensation amount. This enables compensation for the predicted exploration trajectory, realizing tactile-based fine shaft hole assembly operation, reducing the design and debugging costs of shaft hole assembly, and thus improving the efficiency and anti-interference capability of shaft hole assembly operation. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the application mode of the shaft hole data processing method provided in the embodiments of this application;

[0022] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0023] Figure 3A This is a schematic diagram of the first process of the shaft hole data processing method provided in the embodiments of this application;

[0024] Figure 3B This is a schematic diagram of the second process of the shaft hole data processing method provided in the embodiments of this application;

[0025] Figure 3C This is a schematic diagram of the application structure of the prediction model provided in the embodiments of this application;

[0026] Figure 4 This is a schematic diagram of six-dimensional force sensing provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the shaft hole exploration strategy and contact sensing change data provided in the embodiments of this application;

[0028] Figure 6 This is a schematic diagram of the structure of the prediction model provided in the embodiments of this application;

[0029] Figure 7This is a schematic diagram of the prediction model training process provided in the embodiments of this application.

[0030] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0033] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0034] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0035] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0036] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0037] 1) Shaft and hole assembly: This is a core process in mechanical assembly, which refers to the process of accurately inserting shaft-type parts (such as shafts, pins, and gear shafts) into hole-type parts (such as bearing holes, hub holes, and flange holes) to form a fit relationship (clearance, transition, or interference fit).

[0038] 2) Six-dimensional force sensor: a device capable of simultaneously measuring forces (F) in three orthogonal directions. x F y F z ) and torque in three directions (M) x M y M z The high-precision sensor provides complete time and space mechanical information.

[0039] This application provides a shaft hole data processing method, apparatus, device, and computer-readable storage medium, which can improve the efficiency and accuracy of shaft hole assembly.

[0040] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. The following will describe exemplary applications when the device is implemented as a server.

[0041] See Figure 1 , Figure 1 This is a schematic diagram illustrating the application mode of the shaft hole data processing method provided in the embodiments of this application, for example. Figure 1 The system involves server 200, network 300, and terminal 400. Terminal 400 is connected to server 200 through network 300, which can be a wide area network, a local area network, or a combination of both.

[0042] During the shaft-hole assembly process, the terminal 400 acquires the initial contact state between the assembly shaft and the assembly hole, and performs state transition processing on the initial contact state based on the preset shaft-hole exploration trajectory to obtain N target contact states, where N is a positive integer. For each target contact state, the terminal determines the contact perception data between the assembly shaft and the assembly hole in the target contact state, and then sends the contact perception data between the assembly shaft and the assembly hole in the target contact state to the server 200. The server 200 performs trajectory prediction based on the N contact perception data to obtain the predicted exploration trajectory and trajectory compensation amount. Based on the predicted exploration trajectory and trajectory compensation amount, the server predicts the control amount to obtain the target control amount, and sends the target control amount to the terminal 400. The terminal 400 uses the target control amount to generate control commands, and controls the robotic arm to perform shaft-hole assembly through the control commands. The terminal 400 can be an intelligent robot.

[0043] In some embodiments, the shaft hole data processing method provided in this application can also be implemented by a terminal 400. During the shaft hole assembly process, the terminal 400 acquires the initial contact state of the assembly shaft and the assembly hole, and performs state transition processing on the initial contact state based on a preset shaft hole exploration trajectory to obtain N target contact states, where N is a positive integer. For each target contact state, the terminal 400 determines the contact perception data of the assembly shaft and the assembly hole in the target contact state. Based on the N contact perception data, the terminal 400 performs trajectory prediction to obtain the predicted exploration trajectory and trajectory compensation amount. Based on the predicted exploration trajectory and trajectory compensation amount, the terminal 400 performs control quantity prediction to obtain the target control quantity, and uses the target control quantity to generate control commands. The control commands are then used to control the robotic arm to perform shaft hole assembly.

[0044] In some embodiments, the server (e.g., server 200) can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0045] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be a terminal or a server. Figure 2 The illustrated electronic device includes at least one processor 410, a memory 450, and at least one network interface 420. The various components of the electronic device are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0046] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0047] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0048] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory. ， The volatile memory can be random access memory (RAM). The memory 450 described in the embodiments of this application is intended to include any suitable type of memory.

[0049] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0050] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0051] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0052] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A shaft hole data processing device 455 stored in memory 450 is shown. It can be software in the form of programs and plug-ins, including the following software modules: state transition module 4551, first prediction module 4552, and second prediction module 4553. These modules are logical and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0053] In some embodiments, the terminal can implement the shaft hole data processing method provided in this application embodiment by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as communication APPs; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0054] The method for processing shaft hole data provided in this application will be described in conjunction with exemplary applications and implementations of the server equipment provided in the embodiments of this application.

[0055] The following describes the shaft hole data processing method provided in the embodiments of this application. For example, in order to facilitate understanding of the shaft hole data processing method provided in the embodiments of this application, the description is based on the application in a shaft hole assembly scenario.

[0056] As mentioned above, the electronic device implementing the shaft hole data processing method of the embodiments of this application can be a terminal, a server, or a combination of both. The following description uses an electronic device as an example of a terminal to illustrate the shaft hole data processing method provided in the embodiments of this application. See also... Figure 3A , Figure 3A This is a schematic diagram of the first process of the shaft hole data processing method provided in the embodiments of this application, which will be combined with Figure 3A The steps shown are explained.

[0057] In step 301, the initial contact state between the assembly shaft and the assembly hole is determined, and based on the preset shaft-hole exploration trajectory, the initial contact state is processed to obtain N target contact states.

[0058] Here, N is a positive integer. The state transition process involves transforming the initial contact state into multiple contact states, including the state where the assembly shaft and assembly hole begin to contact, the state where the assembly shaft and assembly hole are aligned, and the state where the assembly shaft is inserted into the assembly hole. The assembly shaft is a cylindrical or stepped mechanical part; for example, it can be a shaft, pin, gear shaft, etc. The assembly hole is a cylindrical inner cavity on a mechanical device that mates with the assembly shaft to accommodate it; for example, it can be a bearing hole, hub hole, flange hole, etc. The initial contact state can be a state where the assembly shaft and assembly hole are not in contact.

[0059] The shaft-hole exploration trajectory is the path by which the assembled shaft gradually approaches the assembly hole to achieve shaft-hole assembly. Preset shaft-hole exploration trajectories include linear approach, tilted spiral positioning, and linear insertion. Linear approach involves moving the assembled shaft horizontally along the axial direction of the assembly hole from its current position and orientation. Tilted spiral positioning involves tilting the assembled shaft in any direction along the axial direction of the assembly hole from its current position and orientation, exploring outwards in a spiral motion. Linear insertion involves vertically inserting the assembled shaft into the assembly hole from its current position and orientation.

[0060] A six-dimensional force sensor can be used to collect contact force information between the assembly shaft and the assembly hole. This contact force information includes three components of the contact force in the X, Y, and Z coordinate axes, as well as the torque of each component about the three coordinate axes. Based on the contact force information, the initial contact state between the assembly shaft and the assembly hole is determined. And based on a preset shaft-hole exploration trajectory, during the process of inserting the assembly shaft into the assembly hole, the initial contact state is dynamically switched to multiple contact states, which are also known as N target contact states.

[0061] In some embodiments, the shaft-hole exploration trajectory includes a first exploration trajectory, a second exploration trajectory, and a third exploration trajectory. Based on the preset shaft-hole exploration trajectory, the initial contact state is processed to obtain N target contact states. This can be achieved by performing the following steps: Based on the first exploration trajectory, control the assembly shaft to approach the assembly hole; during the process of bringing the assembly shaft to the assembly hole, when a first preset event is detected, convert the initial contact state to the first contact state, and control the assembly shaft to align with the assembly hole based on the second exploration trajectory; during the process of aligning the assembly shaft with the assembly hole, when a second preset event is detected, convert the first contact state to the second contact state, and control the assembly shaft to insert into the assembly hole based on the third exploration trajectory; during the process of inserting the assembly shaft into the assembly hole, when a third preset event is detected, convert the second contact state to the third contact state; and determine the first contact state, the second contact state, and the third contact state as N target contact states.

[0062] Here, the first exploration trajectory is a straight-line approach. Based on the first exploration trajectory, the assembly shaft is controlled to move horizontally along the axial direction of the assembly hole to approach the assembly hole. The preset event can be represented by a series of discrete events, including the assembly shaft contacting the assembly hole, the assembly shaft aligning with the assembly hole, and the assembly shaft being inserted into the assembly hole.

[0063] During the process of bringing the assembly shaft closer to the assembly hole, the next shaft hole exploration trajectory is determined based on the current contact state and the shaft hole exploration trajectory. The trajectory switching equation for the shaft hole exploration trajectory is q. k+1 =δ(q) k , σ k ), where δ represents the trajectory switching function, qk+1 q represents the shaft hole exploration trajectory at the next moment. k σ represents the current hole exploration trajectory. k This indicates the current contact state.

[0064] When a first preset event is detected, which could be the contact between the assembly shaft and the assembly hole, resulting in the first updated contact state (also known as the first contact state), the initial contact state is converted to the first contact state. Based on the trajectory switching equation and the first contact state, the first exploration trajectory is switched to the second exploration trajectory. The second exploration trajectory is a tilted helical positioning, and the assembly shaft is controlled to align with the assembly hole based on the second exploration trajectory.

[0065] During the alignment of the assembly shaft with the assembly hole, when a second preset event is detected (which could be the alignment of the assembly shaft with the assembly hole resulting in a second updated contact state, also known as the second contact state), the first contact state is converted to the second contact state. Based on the trajectory switching equation and the second contact state, the second exploration trajectory is switched to a third exploration trajectory. The third exploration trajectory is a straight-line insertion, and the assembly shaft is controlled to insert into the assembly hole based on the third exploration trajectory.

[0066] During the process of inserting the assembly shaft into the assembly hole, when a third preset event is detected, the third preset event can be that the assembly shaft is inserted into the assembly hole, resulting in a third updated contact state, which is also the third contact state. At this time, the second contact state is converted into the third contact state. The first contact state, the second contact state, and the third contact state involved in the shaft-hole assembly process constitute N target contact states.

[0067] In this embodiment, by using a preset shaft hole exploration trajectory, the initial contact state between the assembly shaft and the assembly hole is processed during the insertion of the assembly shaft into the assembly hole to obtain N target contact states. The contact state can be dynamically adjusted according to the preset shaft hole exploration trajectory, thereby improving the adaptability, accuracy and robustness of the shaft hole assembly process and significantly improving the success rate and efficiency of shaft hole assembly.

[0068] Continue to refer to Figure 3A In step 302, for each target contact state, the contact sensing data of the assembly shaft and the assembly hole in the target contact state are determined, and trajectory prediction is performed based on N contact sensing data to obtain the predicted exploration trajectory and trajectory compensation amount.

[0069] Here, the contact sensing data includes contact position, contact posture, contact force, and contact torque. The contact sensing data for each target contact state between the assembled shaft and the assembly hole is determined, resulting in N contact sensing data points. Using the trained prediction model, trajectory prediction is performed on the N contact sensing data points to obtain the predicted exploration trajectory and trajectory compensation amount used for shaft-hole assembly.

[0070] In some embodiments, see Figure 3B , Figure 3B This is a schematic diagram of the second process of the shaft hole data processing method provided in the embodiments of this application. Figure 3A In step 302 shown, "for each target contact state, determine the contact sensing data between the assembly shaft and the assembly hole under the target contact state," which can be achieved through... Figure 3B Steps 3021 to 3023 are implemented, and will be explained in detail below.

[0071] In step 3021, for each target contact state, the contact time interval between the assembly shaft and the assembly hole in the target contact state is determined.

[0072] Here, when the target contact state is the first contact state, the first contact state is the state of contact between the assembly shaft and the assembly hole. The first time when the assembly shaft and the assembly hole transition to the first contact state is collected by a six-dimensional force sensor, and the second time when the assembly shaft and the assembly hole transition from the first contact state to the second contact state is collected. Then, the first time is used as the start time and the second time is used as the end time to construct a half-open and half-closed interval, thereby obtaining the first contact time interval when the assembly shaft and the assembly hole are in the first contact state.

[0073] For example, if the first time is 8:00:00 on 2025 / 4 / 01 and the second time is 8:00:15 on 2025 / 4 / 01, then the first contact time interval is [2025 / 4 / 01 8:00:00, 2025 / 4 / 01 8:00:15].

[0074] When the target contact state is the second contact state, the second contact state is the state in which the assembly shaft and the assembly hole are aligned. The third time when the assembly shaft and the assembly hole transition from the second contact state to the third contact state is collected by a six-dimensional force sensor. Then, the second time is used as the start time and the third time is used as the end time to construct a half-open and half-closed interval, thereby obtaining the second contact time interval when the assembly shaft and the assembly hole are in the second contact state.

[0075] When the target contact state is the third contact state, which is the state in which the assembly shaft is inserted into the assembly hole, the fourth time when the assembly shaft and the assembly hole complete the assembly is collected by a six-dimensional force sensor. Then, the third time is used as the start time and the fourth time is used as the end time to construct a half-open and half-closed interval, thereby obtaining the third contact time interval when the assembly shaft and the assembly hole are in the third contact state.

[0076] In step 3022, the contact position, contact posture, contact force and contact torque of the assembled shaft are obtained at each sampling time point within the contact time interval.

[0077] Here, according to a preset time interval (e.g., 100ms), multiple sampling time points are set in the first contact time interval, the second contact time interval, and the third contact time interval. For each contact time interval, the contact position, contact posture, contact force, and contact torque of the assembled shaft are collected at each sampling time point in the contact time interval by a six-dimensional force sensor, so as to obtain the contact position, contact posture, contact force, and contact torque corresponding to the three contact time intervals.

[0078] In step 3023, each contact position, contact posture, contact force, and contact torque are determined as contact sensing data between the assembly shaft and the assembly hole in the target contact state.

[0079] Here, the contact position, contact posture, contact force, and contact torque corresponding to the first contact time interval are defined as the contact sensing data between the assembled shaft and the assembly hole in the first contact state. The contact position, contact posture, contact force, and contact torque corresponding to the second contact time interval are defined as the contact sensing data between the assembled shaft and the assembly hole in the second contact state. The contact position, contact posture, contact force, and contact torque corresponding to the third contact time interval are defined as the contact sensing data between the assembled shaft and the assembly hole in the third contact state.

[0080] In this embodiment, for each target contact state, the contact time interval between the assembly shaft and the assembly hole in the target contact state is determined. The contact position, contact posture, contact force, and contact torque corresponding to the contact time interval are determined as the contact perception data between the assembly shaft and the assembly hole in the target contact state. This can obtain multi-dimensional contact perception data based on the time and space dimensions, which is convenient for subsequent shaft hole exploration trajectory prediction based on multi-dimensional contact perception data. This enhances the tactile perception capability in the shaft hole assembly process, thereby improving the accuracy of shaft hole assembly.

[0081] In some embodiments, trajectory prediction based on N contact sensing data to obtain a predicted exploration trajectory and trajectory compensation amount can be achieved by performing the following steps: obtaining a trained prediction model, which includes at least a bidirectional long short-term memory neural network, a feature concatenation layer, a fully connected layer, and a normalization layer; for each contact sensing data, calling the bidirectional long short-term memory neural network to extract features from the contact sensing data to obtain a contact sensing vector; calling the feature concatenation layer to concatenate the N contact sensing vectors to obtain a contact state representation; calling the normalization layer to perform trajectory prediction based on the contact state representation to obtain a predicted exploration trajectory; and calling the fully connected layer to predict trajectory deviation based on the contact state representation to obtain a trajectory compensation amount.

[0082] Here, the prediction model is used to predict the shaft-hole exploration trajectory and trajectory compensation amount. The prediction model can be a neural network model. The Bidirectional Long Short-Term Memory (BiLSTM) neural network in the prediction model is called to extract features from the contact sensing data corresponding to N contact time intervals, obtaining N contact sensing vectors. The feature concatenation layer in the prediction model is called to concatenate the N contact sensing vectors, obtaining a concatenated vector, which is used as a low-dimensional contact state representation. The normalization layer in the prediction model is called to classify and predict the exploration trajectory based on the contact state representation, obtaining the predicted exploration trajectory. The fully connected layer in the prediction model is called to perform trajectory deviation prediction using a multi-layer fully connected network on the contact state representation, obtaining the trajectory compensation amount.

[0083] Example, reference Figure 3C , Figure 3C This is a schematic diagram of the application structure of the prediction model provided in this application embodiment. The prediction model 310 includes a bidirectional long short-term memory neural network 311, a feature concatenation layer 312, a fully connected layer 313, and a normalization layer 314. The bidirectional long short-term memory neural network 311 is invoked to extract features from N contact sensing data 315, obtaining N contact sensing vectors. The feature concatenation layer 312 is invoked to concatenate the N contact sensing vectors, obtaining a contact state representation. The normalization layer 314 is invoked to predict the trajectory based on the contact state representation, obtaining a predicted exploration trajectory. The fully connected layer 313 is invoked to predict the trajectory deviation based on the contact state representation, obtaining the trajectory compensation amount.

[0084] In this embodiment, the trained prediction model extracts and concatenates feature vectors from N contact sensing data to obtain a contact state representation. Based on the contact state representation, the shaft hole exploration trajectory and trajectory deviation are predicted to obtain the predicted exploration trajectory and trajectory compensation amount. By combining multi-dimensional contact sensing data and prediction model under different contact states, the shaft hole exploration trajectory and trajectory compensation amount can be accurately predicted, which significantly improves the intelligence level and operation accuracy of the shaft hole assembly process.

[0085] In some embodiments, training the prediction model can also be achieved by performing the following steps: acquiring sample contact perception data of the assembly shaft and assembly hole in the sample target contact state; performing prediction processing on the sample contact perception data based on the prediction model to obtain the sample predicted exploration trajectory and sample trajectory compensation amount; predicting control quantity based on the sample predicted exploration trajectory and sample trajectory compensation amount to obtain the sample control quantity, and controlling the movement of the robotic arm based on the sample control quantity to determine the sample contact force and the position after movement; determining the reward value and loss value based on the position after movement, the target position, the sample contact force, and the expected contact force; and training the prediction model based on the reward value and loss value to obtain the trained prediction model.

[0086] Here, sample data is acquired during the assembly process of the shaft and hole. This sample data includes N sample target contact states of the shaft and hole, and N sample contact perception data under these N sample target contact states. A bidirectional long short-term memory neural network in the prediction model is invoked to extract features from the N sample contact perception data, resulting in N sample contact perception vectors. A feature concatenation layer in the prediction model is then invoked to concatenate the N sample contact perception vectors, yielding a sample contact state representation. A normalization layer in the prediction model is invoked to predict the trajectory based on the sample contact state representation, resulting in a predicted exploration trajectory. A fully connected layer in the prediction model is invoked to predict the trajectory deviation based on the contact state representation, yielding a sample trajectory compensation amount. The sample trajectory control amount and sample trajectory compensation amount corresponding to the predicted exploration trajectory are summed to obtain a sample control amount. This sample control amount is then used to control the movement of the robotic arm for shaft and hole assembly. At this point, the sample contact force between the shaft and hole, and the position of the shaft after movement, are determined.

[0087] The target position of the assembly shaft, the expected contact force between the assembly shaft and the assembly hole, and the safety indication function (1 when there is no contact) are preset. The first squared Euclidean distance between the moved position and the target position, and the second squared Euclidean distance between the sample contact force and the expected contact force are determined. Based on the preset first weighting coefficient, second weighting coefficient, and third weighting coefficient, the first squared Euclidean distance, the second squared Euclidean distance, and the safety indication function are adjusted to obtain the density reward value. For example, the density reward value can be determined by the following formula (1):

[0088] R dense =-w1‖x t -x goal || 2 -w2‖F t -F desired || 2 +w3I safe (1)

[0089] Among them, R dense This represents the dense reward value, where w1 represents the first weighting coefficient, w2 represents the second weighting coefficient, w3 represents the third weighting coefficient, and x represents the third weighting coefficient. t Indicates the position after the movement, x goal Indicates the target location, F t F represents the contact force of the sample. desired I represents the desired contact force. safe Represents the safety indication function, ||x t -x goal || 2 Represents the squared value of the first Euclidean distance, ||F t -F desired || 2 This represents the squared value of the second Euclidean distance.

[0090] When the squared value of the first Euclidean distance is less than or equal to the first preset threshold, and the squared value of the second Euclidean distance is less than or equal to the second preset threshold, the sparse reward value is determined to be non-zero; otherwise, the sparse reward value is determined to be zero. For example, the sparse reward value can be determined using the following formula (2):

[0091]

[0092] Among them, R sparse Represents the sparse reward value, r success denoted by , where ∈ represents the preset reward value, δ represents the first preset threshold, and δ represents the second preset threshold.

[0093] The reward value is obtained by summing the dense reward value and the sparse reward value. For example, the reward value can be determined by the following formula (3):

[0094] R = R dense +R sparse (3)

[0095] Where R represents the reward value, R dense R represents the dense reward value. sparse This represents sparse reward values.

[0096] In the process of controlling a robotic arm to assemble shafts and holes using multiple sample control variables, the cumulative reward value is determined. A Soft Actor-Critic (SAC) reinforcement learning algorithm can be used to calculate the expected value of the cumulative reward value, as well as the mean square error between the moved position of the assembled shaft and the expected moved position. The expected value and the mean square error of the cumulative reward value are then weighted and summed to obtain the loss value. Based on the reward value and the loss value, the model parameters of the prediction model are updated to obtain the trained prediction model.

[0097] In this embodiment, the reward value and loss value are determined based on the sample contact force between the assembly shaft and the assembly hole, the position of the assembly shaft after movement, the target position of the assembly shaft, and the expected contact force between the assembly shaft and the assembly hole. Based on the reward value and loss value, the prediction model is trained, which can improve the quality of the prediction model's learning of the contact state representation and accurately predict the shaft hole exploration trajectory and trajectory compensation amount, thereby improving the accuracy of shaft hole assembly.

[0098] Continue to refer to Figure 3A In step 303, control quantity prediction is performed based on the predicted exploration trajectory and trajectory compensation amount to obtain the target control quantity, and the target control quantity is used to control the robotic arm to perform shaft hole assembly.

[0099] Here, the shaft hole exploration trajectory is predicted multiple times to obtain the final predicted exploration trajectory and its trajectory compensation amount. The control amount corresponding to the final predicted exploration trajectory and the trajectory compensation amount are summed to obtain the final trajectory control amount, which is also the target control amount. Based on the target control amount, control commands are generated and sent to the robot compliant control system. The robot includes at least one robotic arm. Based on the control commands, the robot compliant control system controls the robotic arm to perform the shaft hole assembly operation.

[0100] In some embodiments, the following steps may also be performed: when the number of trajectory predictions is less than the prediction number threshold, exploration compensation is performed on the predicted exploration trajectory based on the trajectory compensation amount to obtain an updated exploration trajectory; based on the updated exploration trajectory, updated contact sensing data is determined during the process of inserting the assembly shaft into the assembly hole, and trajectory prediction is performed based on the updated contact sensing data to obtain an updated predicted exploration trajectory and an updated trajectory compensation amount.

[0101] Here, the number of trajectory predictions refers to the number of predictions for the shaft-hole exploration trajectory. This number of predictions is preset, and the prediction threshold is the upper limit of the prediction count for the shaft-hole exploration trajectory. If the number of predictions is less than the threshold, the control variable and trajectory compensation variable corresponding to the current predicted exploration trajectory are summed to obtain the updated exploration trajectory. The initial contact state between the assembly shaft and the assembly hole is redefined. Based on the updated exploration trajectory, during the insertion of the assembly shaft into the assembly hole, a state transition process is performed on the initial contact state to obtain N target contact states. N contact time intervals are determined for the assembly shaft and assembly hole to be in the N target contact states, and the contact position, contact posture, contact force, and contact torque corresponding to the N contact time intervals are defined as updated contact sensing data. Using the trained prediction model, trajectory prediction is performed on the updated contact sensing data to obtain the updated predicted exploration trajectory and the updated trajectory compensation variable.

[0102] For example, if the prediction count threshold is 3 times, after the first trajectory prediction is performed on the contact sensing data, the prediction count is 1, which is less than the prediction count threshold. Then, based on the current predicted trajectory compensation amount, exploration compensation is performed on the predicted exploration trajectory to obtain an updated exploration trajectory. Based on the updated exploration trajectory, a second trajectory prediction is performed on the updated contact sensing data.

[0103] In this embodiment of the application, when the number of trajectory predictions is less than the prediction number threshold, exploration compensation is performed on the predicted exploration trajectory to obtain an updated exploration trajectory. Based on the updated exploration trajectory, the trajectory prediction process is repeated, which enables multiple shaft hole exploration trajectory predictions, improves the accuracy of shaft hole exploration trajectory prediction, and thus improves the accuracy of shaft hole assembly based on the final predicted shaft hole exploration trajectory.

[0104] In some embodiments, the target control quantity is obtained by predicting the control quantity based on the predicted exploration trajectory and the trajectory compensation amount. This can be achieved by performing the following steps: when the number of trajectory predictions reaches the prediction number threshold, the trajectory parameters of the updated predicted exploration trajectory are parsed to obtain the trajectory control quantity corresponding to the updated predicted exploration trajectory; the trajectory control quantity and the updated trajectory compensation amount are summed to obtain the target control quantity.

[0105] Here, when the number of trajectory predictions reaches a threshold, the currently updated predicted exploration trajectory is determined. This trajectory is then parsed into a sequence of trajectory-related state parameters, including position, velocity, and direction. This sequence is input into the dynamic model to obtain the trajectory control quantity corresponding to the updated predicted exploration trajectory. The trajectory control quantity and its corresponding updated trajectory compensation quantity are summed to obtain the target control quantity, which is then used to control the robotic arm for shaft and hole assembly.

[0106] In some embodiments, the shaft and hole data processing method provided in this application can be applied to the field of intelligent manufacturing. The initial contact state between the assembly shaft and the assembly hole is determined on an intelligent cloud platform. Based on a preset shaft and hole exploration trajectory, the initial contact state is processed to obtain N target contact states, where N is a positive integer. This allows for dynamic switching between multiple contact states of the assembly shaft and the assembly hole based on the preset shaft and hole exploration trajectory. Then, for each target contact state, contact perception data of the assembly shaft and the assembly hole in the target contact state is determined. Based on the N contact perception data, trajectory prediction is performed to obtain the predicted exploration trajectory and trajectory compensation amount. This achieves comprehensive prediction of the shaft and hole exploration trajectory and trajectory compensation amount by combining contact perception data of the assembly shaft and the assembly hole in multiple contact states, which can improve the accuracy of shaft and hole assembly. Furthermore, based on the predicted exploration trajectory and trajectory compensation amount, control quantity prediction is performed to obtain the target control quantity, and the target control quantity is used to control the robotic arm to perform shaft and hole assembly. In this way, by using contact perception data under different contact states, the shaft hole exploration trajectory and trajectory compensation amount are predicted, and the control quantity used for shaft hole assembly is determined comprehensively based on the predicted exploration trajectory and trajectory compensation amount. This enables compensation for the predicted exploration trajectory, realizing tactile-based fine shaft hole assembly operation, reducing the design and debugging costs of shaft hole assembly, and thus improving the efficiency and anti-interference capability of shaft hole assembly operation.

[0107] The following will describe an exemplary application of the shaft hole data processing method provided in the embodiments of this application in a shaft hole assembly scenario.

[0108] Shaft and hole assembly is a typical contact-rich task. Using robots for shaft and hole assembly requires sub-millimeter-level positioning accuracy to meet high-precision assembly requirements. Furthermore, shaft and hole assembly necessitates complex force and position control strategies to handle multi-stage contact requirements. Reinforcement learning can interactively learn optimal assembly strategies, reducing reliance on expert experience. However, tactile-based shaft and hole assembly methods have limited perception range and poor robustness to positional errors. While vision-based methods offer stronger perception capabilities, they are more sensitive to environmental interference. Therefore, it is necessary to address the limitations of traditional tactile and vision-based methods.

[0109] The design process of shaft and hole assembly strategies in related technologies is complex and difficult to generalize to various shaft and hole assembly tasks. For example, shaft and hole assembly methods based on hard coding rely on expert knowledge in the design and debugging process, model-based shaft and hole assembly methods rely on accurate models, and shaft and hole assembly methods based on deep learning rely on the collection of a large amount of contact data under different contact states, resulting in low efficiency and accuracy of shaft and hole assembly operations.

[0110] This application proposes a shaft hole data processing method to address the problems existing in related technologies, which includes the following improvements compared to related technologies:

[0111] Multidimensional contact sensing data (N contact sensing data points in the above embodiments) in space and time are collected using a six-dimensional force sensor. A hybrid motion model for multidimensional contact sensing is designed, considering static conditions at different contact angles and positions, as well as dynamic conditions of sliding direction and velocity. A reinforcement learning neural network strategy (the prediction model in the above embodiments) is used to learn a low-dimensional contact state representation from the high-dimensional multidimensional contact sensing data. This contact state representation is then used to predict the shaft-hole assembly operation strategy and the shaft-hole exploration compensation process (the predicted exploration trajectory and trajectory compensation amount in the above embodiments), thereby achieving efficient shaft-hole assembly operations and improving the robustness of the tactile assembly strategy to positional errors.

[0112] The shaft hole data processing method provided in this application utilizes a shaft hole exploration strategy (the preset shaft hole exploration trajectory in the above embodiments) and a long short-term memory network during the shaft hole assembly process to enhance tactile perception, guide the shaft hole exploration process, alleviate some observable problems, and improve the efficiency and error robustness of shaft hole assembly. In this application embodiment, a six-dimensional force sensor collects spatial and temporal contact perception data. The six-dimensional force sensor can sense contact forces at a point in space, in a certain contact direction, and in multiple instantaneous directions, reflecting the current contact state between the assembled shaft and the assembly hole, but it is insufficient to reflect the contact information required for the assembly operation. Using the contact perception data, the assembled shaft and assembly hole achieve safe contact through collision detection or compliance control. For example, see reference... Figure 4 , Figure 4 This is a schematic diagram of six-dimensional force sensing provided in an embodiment of this application. In the contact 401 perpendicular to the desktop, the vertical contact force F between the assembly shaft and the assembly hole can be decomposed into a force F along the x-axis direction. x Force F along the y-axis y and the force F along the z-axis z Force F x The corresponding contact torque is M x Force F y The corresponding contact torque is M y Force F z The corresponding contact torque is M z In the inclined tabletop contact 402, the inclined contact force F between the assembly shaft and the assembly hole can be decomposed into a force F along the x-axis. x Force F along the y-axis y and the force F along the z-axis z Force F x The corresponding contact torque is M x Force F yThe corresponding contact torque is M y Force F z The corresponding contact torque is M z .

[0113] The shaft-hole data processing method provided in this application provides a shaft-hole exploration strategy that slides the contact surfaces of the assembled shaft and the assembled hole relative to each other, generating richer contact information. This contact information includes contact forces under static conditions at different angles and positions, as well as contact forces under dynamic conditions of sliding direction and speed. This application uses a combination of straight lines and inclined spirals to design a trajectory-based shaft-hole exploration strategy, which can be divided into three stages: the first stage is straight-line approach, where the assembled shaft is moved horizontally along the axial direction of the assembled hole from its current position and orientation; the second stage is inclined spiral positioning, where the assembled shaft is tilted in any direction along the axial direction of the assembled hole from its current position and orientation, and explored outwards in a spiral manner; the third stage is straight-line insertion. During the shaft-hole assembly process, the next shaft-hole exploration strategy can be determined based on the current contact state and the shaft-hole exploration strategy.

[0114] A hybrid motion model is constructed by combining a finite state machine and shaft-hole exploration strategies. This hybrid motion model is defined as a seven-tuple: H = (Q, Σ, X, δ, φ, q0, x0), where Q represents the set of shaft-hole exploration strategies, Q = {q1, q2, ..., q0}. n}, where σ1 represents "straight line approach", σ2 represents "tilted spiral positioning", and σ3 represents "straight line insertion". Σ represents the set of discrete events, Σ={σ1, σ2, ... σ n σ1 represents "the assembly shaft contacts the assembly hole", σ2 represents "the assembly shaft aligns with the assembly hole", and σ3 represents "the assembly shaft is inserted into the assembly hole". δ represents the shaft-hole exploration strategy switching function, δ(q1, σ1) = q2, X represents the continuous state space, φ represents the continuous dynamic equation, q0 represents the initial shaft-hole exploration strategy, and x0 represents the initial continuous state. The shaft-hole exploration strategy switching equation for the hybrid motion model is q k+1 =δ(q) k , σ k ).

[0115] A hybrid motion model combines symbolic and numerical methods to achieve hybrid dynamics. Discrete events trigger continuous dynamic switching of the shaft-hole exploration strategy, employing different strategies under different contact states to enhance the robot's tactile perception capabilities during shaft-hole assembly. Furthermore, the temporal data for tactile perception is obtained from a six-dimensional force sensor mounted on the robot's wrist, sampled at a frequency of 100Hz. (See example reference.) Figure 5 , Figure 5This is a schematic diagram of the shaft hole exploration strategy and contact sensing change data provided in an embodiment of this application. The shaft hole exploration strategy 501 includes straight approach, tilted spiral positioning, and straight insertion. The contact sensing change data 502 includes contact force F. x Contact torque M x Sensing data changing over time, and contact force F. y Contact torque M y Perceptual data that changes over time.

[0116] The prediction process for the shaft-hole assembly operation strategy is explained below. Using neural networks such as Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTMs), and Transformers, feature vectors are extracted from multi-dimensional contact perception data (covering a series of data sampled at multiple locations and time points). Contact state representations are then learned to predict the shaft-hole exploration trajectory and trajectory compensation amount, improving tactile perception capabilities in both spatial and temporal dimensions. For example, refer to [reference needed]. Figure 6 , Figure 6 This is a schematic diagram of the predictive model provided in the embodiments of this application. The contact position, contact posture, contact force, and contact torque between the assembly shaft and the assembly hole constitute multi-dimensional contact sensing data. t and multi-dimensional contact sensing data O t The data is input into a Bidirectional Long Short-Term Memory (BiLSTM) neural network to obtain multiple contact sensing vectors. These vectors are then concatenated to obtain a concatenated vector, which serves as a low-dimensional contact state representation. This low-dimensional contact state representation is processed through multiple fully connected and normalization layers. The policy output 601 of the prediction model can output an event trigger label, which is used to trigger the predicted shaft-hole exploration strategy. The compensation output 602 of the prediction model is used to predict the trajectory compensation amount. Multiple shaft-hole exploration strategy predictions are performed. When predicting the shaft-hole exploration strategy for the next time, the assembled shaft is slid relative to the contact surface where the assembly hole is located according to the previously predicted shaft-hole exploration strategy, and multi-dimensional contact sensing data is re-acquired. By combining the prediction model and the hybrid motion model, the predicted exploration trajectory is compensated, achieving efficient tactile perception and precise shaft-hole assembly operations.

[0117] The training process of the prediction model is explained below. Various reinforcement learning algorithms, such as Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC), can be used to learn multi-dimensional contact perception and shaft-hole exploration strategies. These algorithms can learn contact state representations based on the success rate defined by the shaft-hole assembly operation effect, predict and compensate for shaft-hole exploration strategies, reduce reliance on expert experience, and achieve tactile-based shaft-hole assembly.

[0118] A training framework for the prediction model was designed based on residual reinforcement learning (the residual is reflected in the learning of the shaft hole exploration strategy; the trajectory-based multi-stage shaft hole exploration strategy is universal and can be applied to most shaft hole assembly tasks, while the prediction model is trained using a reinforcement learning algorithm for shaft hole assembly tasks with different shapes and materials). See the example below for reference. Figure 7 , Figure 7 This is a schematic diagram of the prediction model training process provided in this application embodiment. The interactive environment 701 observes the intelligent agent 702. In the interactive environment 701, task space compliant control controls the robot to perform dexterous shaft-hole assembly operations. The interactive environment, consisting of the robot and the task, includes a robot composed of a robotic arm and a dexterous hand, the shaft-hole assembly task, and the compliant control system. The intelligent agent 702 includes a trajectory-based shaft-hole exploration strategy, a neural network strategy, and a reinforcement learning SAC algorithm. The reinforcement learning SAC algorithm interacts with the interactive environment 701, learning multi-dimensional tactile perception data processing and shaft-hole exploration compensation during the interaction, and updating the neural network strategy using the reinforcement learning SAC algorithm. The trajectory-based shaft-hole exploration strategy is used as the basic strategy, which consists of a state machine, a shaft-hole exploration trajectory, and control parameters. Data concatenation and the neural network strategy are used as residual strategies. The residual strategies generate state signals and send these signals to the state machine of the basic strategy. The state machine is used to select the shaft-hole exploration trajectory and control parameters, while the residual strategies can generate trajectory compensation amounts. The outputs of the basic strategy and the residual strategy are superimposed and sent as control instructions to the task space compliant control in the interactive environment 701.

[0119] In some embodiments, during the process of updating the neural network policy using the reinforcement learning SAC algorithm, a state space, action space, and reward value are designed. The state space data is composed of multi-dimensional contact sensing data of the assembly shaft and assembly hole under different contact states. The action space data is composed of discrete events during the shaft-hole assembly process and the predicted trajectory compensation amounts. Dense and sparse reward values ​​are designed based on the multi-dimensional contact sensing data, and the dense and sparse reward values ​​are summed to obtain the reward value.

[0120] For example, the dense reward value can be determined by the following formula (1):

[0121] R dense =-w1‖x t -x goal || 2 -w2‖F t -F desired || 2 +w3I safe (1)

[0122] Among them, R dense This represents the dense reward value, where w1 represents the first weighting coefficient, w2 represents the second weighting coefficient, w3 represents the third weighting coefficient, and x represents the third weighting coefficient. t Indicates the position of the assembled shaft after movement, x goal Indicates the target position of the assembled shaft component, F t F represents the sample contact force between the assembled shaft and the assembly hole. desired I represents the desired contact force between the assembled shaft and the assembly hole. safe This represents the safety indication function (1 for no contact), ||x t -x goal || 2 Represents the squared value of the first Euclidean distance, ||F t -F desired || 2 This represents the squared value of the second Euclidean distance.

[0123] When the squared value of the first Euclidean distance is less than or equal to the first preset threshold, and the squared value of the second Euclidean distance is less than or equal to the second preset threshold, the sparse reward value is determined to be non-zero; otherwise, the sparse reward value is determined to be zero. For example, the sparse reward value can be determined using the following formula (2):

[0124]

[0125] Among them, R sparse Represents the sparse reward value, r success denoted by , where ∈ represents the preset reward value, δ represents the first preset threshold, and δ represents the second preset threshold.

[0126] The reward value is obtained by summing the dense reward value and the sparse reward value. For example, the reward value can be determined by the following formula (3):

[0127] R = R dense +R sparse (3)

[0128] Where R represents the reward value, R dense R represents the dense reward value. sparse This represents sparse reward values.

[0129] In the aforementioned application scenario of shaft and hole assembly, a segmented shaft and hole exploration strategy based on trajectory and a bidirectional long short-term memory network are used to enhance the tactile perception capability during the shaft and hole assembly process. The multi-dimensional contact perception data processing and shaft and hole exploration strategy are defined as Markov decision processes. A reinforcement learning algorithm is used to train a prediction model. The trained prediction model learns the contact state representation and compensates for the shaft and hole exploration strategy, thereby realizing tactile shaft and hole assembly. This reduces the design and debugging costs of shaft and hole assembly, improves the operational efficiency and anti-interference capability of shaft and hole assembly, and enhances the robustness to position errors, providing a solution for the large-scale, intelligent, and stable development of flexible manufacturing.

[0130] The following description continues to illustrate the exemplary structure of the shaft hole data processing device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules in the shaft-hole data processing device 455 stored in the memory 450 may include: a state transition module 4551, used to determine the initial contact state between the assembly shaft and the assembly hole, and perform state transition processing on the initial contact state based on a preset shaft-hole exploration trajectory to obtain N target contact states, where N is a positive integer, and the state transition processing is to transform the initial contact state into multiple contact states; a first prediction module 4552, used to determine the contact sensing data between the assembly shaft and the assembly hole in the target contact state for each target contact state, and perform trajectory prediction based on the N contact sensing data to obtain a predicted exploration trajectory and a trajectory compensation amount; and a second prediction module 4553, used to predict the control amount based on the predicted exploration trajectory and the trajectory compensation amount to obtain a target control amount, and use the target control amount to control the robotic arm to perform shaft-hole assembly.

[0131] In some embodiments, the shaft-hole exploration trajectory includes a first exploration trajectory, a second exploration trajectory, and a third exploration trajectory. The state transition module 4551 is further configured to control the assembly shaft to approach the assembly hole based on the first exploration trajectory; during the process of bringing the assembly shaft closer to the assembly hole, when a first preset event is detected, the initial contact state is converted to the first contact state, and the assembly shaft is controlled to align with the assembly hole based on the second exploration trajectory; during the process of aligning the assembly shaft with the assembly hole, when a second preset event is detected, the first contact state is converted to the second contact state, and the assembly shaft is controlled to insert into the assembly hole based on the third exploration trajectory; during the process of inserting the assembly shaft into the assembly hole, when a third preset event is detected, the second contact state is converted to the third contact state; and the first contact state, the second contact state, and the third contact state are determined as N target contact states.

[0132] In some embodiments, the first prediction module 4552 is further configured to determine the contact time interval between the assembly shaft and the assembly hole in the target contact state for each target contact state; acquire the contact position, contact posture, contact force and contact torque corresponding to each sampling time point of the assembly shaft in the contact time interval; and determine each contact position, contact posture, contact force and contact torque as contact sensing data between the assembly shaft and the assembly hole in the target contact state.

[0133] In some embodiments, the first prediction module 4552 is further configured to obtain a trained prediction model, the prediction model including at least a bidirectional long short-term memory neural network, a feature concatenation layer, a fully connected layer, and a normalization layer; for each contact sensing data, the bidirectional long short-term memory neural network is invoked to extract features from the contact sensing data to obtain a contact sensing vector; the feature concatenation layer is invoked to concatenate N contact sensing vectors to obtain a contact state representation; the normalization layer is invoked to perform trajectory prediction based on the contact state representation to obtain a predicted exploration trajectory; and the fully connected layer is invoked to perform trajectory deviation prediction based on the contact state representation to obtain a trajectory compensation amount.

[0134] In some embodiments, the first prediction module 4552 is further configured to acquire sample contact perception data of the assembly shaft and the assembly hole in the sample target contact state; perform prediction processing on the sample contact perception data based on the prediction model to obtain the sample predicted exploration trajectory and the sample trajectory compensation amount; perform control quantity prediction based on the sample predicted exploration trajectory and the sample trajectory compensation amount to obtain the sample control quantity, and control the movement of the robotic arm based on the sample control quantity to determine the sample contact force and the position after movement; determine the reward value and the loss value based on the position after movement, the target position, the sample contact force and the expected contact force; and train the prediction model based on the reward value and the loss value to obtain the trained prediction model.

[0135] In some embodiments, the second prediction module 4553 is further configured to, when the number of trajectory predictions is less than the prediction number threshold, perform exploration compensation on the predicted exploration trajectory based on the trajectory compensation amount to obtain an updated exploration trajectory; based on the updated exploration trajectory, determine updated contact sensing data during the process of inserting the assembly shaft into the assembly hole, and perform trajectory prediction based on the updated contact sensing data to obtain an updated predicted exploration trajectory and an updated trajectory compensation amount.

[0136] In some embodiments, the second prediction module 4553 is further configured to, when the number of trajectory predictions reaches the prediction number threshold, parse the trajectory parameters of the updated predicted exploration trajectory to obtain the trajectory control quantity corresponding to the updated predicted exploration trajectory; and sum the trajectory control quantity and the updated trajectory compensation quantity to obtain the target control quantity.

[0137] This application provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the shaft hole data processing method described above in this application.

[0138] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the shaft hole data processing method provided in this application. For example, ... Figure 3A The method for processing shaft hole data is shown.

[0139] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0140] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0141] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0142] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0143] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for processing shaft hole data, characterized in that, The method includes: The initial contact state between the assembly shaft and the assembly hole is determined, and based on the preset shaft-hole exploration trajectory, the initial contact state is processed to obtain N target contact states, where N is a positive integer. The state transition process is to transform the initial contact state into multiple contact states. For each target contact state, determine the contact sensing data between the assembly shaft and the assembly hole under the target contact state; Obtain a trained prediction model, which includes at least a bidirectional long short-term memory neural network, a feature concatenation layer, a fully connected layer, and a normalization layer; For each of the contact sensing data, the bidirectional long short-term memory neural network is invoked to extract features from the contact sensing data to obtain a contact sensing vector; the feature concatenation layer is invoked to concatenate the N contact sensing vectors to obtain a contact state representation; the normalization layer is invoked to predict the trajectory based on the contact state representation to obtain a predicted exploration trajectory; and the fully connected layer is invoked to predict the trajectory deviation based on the contact state representation to obtain the trajectory compensation amount. Based on the predicted exploration trajectory and the trajectory compensation amount, the control quantity is predicted to obtain the target control quantity, and the target control quantity is used to control the robotic arm to perform shaft hole assembly.

2. The method according to claim 1, characterized in that, The shaft hole exploration trajectory includes a first exploration trajectory, a second exploration trajectory, and a third exploration trajectory. Based on the preset shaft hole exploration trajectory, the initial contact state is subjected to state transition processing to obtain N target contact states, including: Based on the first exploration trajectory, the assembly shaft is controlled to approach the assembly hole; During the process of bringing the assembly shaft close to the assembly hole, when a first preset event is detected, the initial contact state is converted to a first contact state, and the assembly shaft is controlled to align with the assembly hole based on the second exploration trajectory. During the process of aligning the assembly shaft with the assembly hole, when a second preset event is detected, the first contact state is converted to the second contact state, and the assembly shaft is controlled to insert into the assembly hole based on the third exploration trajectory. During the process of inserting the assembly shaft into the assembly hole, when a third preset event is detected, the second contact state is converted to the third contact state; The first contact state, the second contact state, and the third contact state are determined as N target contact states.

3. The method according to claim 2, characterized in that, For each target contact state, determining the contact sensing data between the assembly shaft and the assembly hole under the target contact state includes: For each target contact state, determine the contact time interval between the assembly shaft and the assembly hole in the target contact state; Obtain the contact position, contact posture, contact force, and contact torque of the assembled shaft at each sampling time point within the contact time interval; Each contact position, contact posture, contact force, and contact torque is determined as the contact sensing data between the assembly shaft and the assembly hole under the target contact state.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Acquire sample contact sensing data of the assembly shaft and the assembly hole in the sample target contact state; Based on the prediction model, the sample contact perception data is processed to obtain the sample predicted exploration trajectory and the sample trajectory compensation amount. Based on the sample predicted exploration trajectory and sample trajectory compensation amount, control quantity prediction is performed to obtain sample control quantity, and the robotic arm is controlled to move based on the sample control quantity to determine the sample contact force and the position after movement. Based on the moved position, target position, sample contact force, and expected contact force, the reward value and loss value are determined. The prediction model is trained based on the reward value and the loss value to obtain the trained prediction model.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: When the number of trajectory predictions is less than the prediction number threshold, the predicted exploration trajectory is compensated based on the trajectory compensation amount to obtain an updated exploration trajectory. Based on the updated exploration trajectory, updated contact sensing data is determined during the process of bringing the assembly shaft close to the assembly hole, and trajectory prediction is performed based on the updated contact sensing data to obtain an updated predicted exploration trajectory and an updated trajectory compensation amount.

6. The method according to claim 5, characterized in that, The step of predicting the control quantity based on the predicted exploration trajectory and the trajectory compensation amount to obtain the target control quantity includes: When the number of trajectory predictions reaches the prediction number threshold, the trajectory parameters of the updated predicted exploration trajectory are parsed to obtain the trajectory control quantity corresponding to the updated predicted exploration trajectory. The target control quantity is obtained by summing the trajectory control quantity and the updated trajectory compensation quantity.

7. A shaft hole data processing device, characterized in that, The device includes: The state transition module is used to determine the initial contact state between the assembly shaft and the assembly hole, and based on the preset shaft-hole exploration trajectory, to perform state transition processing on the initial contact state to obtain N target contact states, where N is a positive integer. The state transition processing is to transform the initial contact state into multiple contact states. The first prediction module is used to determine the contact perception data of the assembly shaft and the assembly hole in each target contact state; acquire a trained prediction model, which includes at least a bidirectional long short-term memory neural network, a feature concatenation layer, a fully connected layer, and a normalization layer; for each contact perception data, call the bidirectional long short-term memory neural network to extract features from the contact perception data to obtain a contact perception vector; call the feature concatenation layer to concatenate N contact perception vectors to obtain a contact state representation; call the normalization layer to perform trajectory prediction based on the contact state representation to obtain a predicted exploration trajectory; and call the fully connected layer to perform trajectory deviation prediction based on the contact state representation to obtain a trajectory compensation amount. The second prediction module is used to predict the control quantity based on the predicted exploration trajectory and the trajectory compensation amount, obtain the target control quantity, and use the target control quantity to control the robotic arm to perform shaft hole assembly.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the shaft hole data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the shaft hole data processing method according to any one of claims 1 to 6 is implemented.

Images