Model training method, clock synchronization method, electronic device and program product

Abstract

The invention provides a model training method, a clock synchronization method, electronic equipment and a program product, and relates to the technical field of machine learning. The model training method comprises the following steps: determining a first proportional-integral parameter of a target proportional-integral adjustment model; determining a clock synchronization compensation amount based on the first proportional-integral parameter; determining reward data of the reinforcement learning algorithm based on the clock synchronization compensation amount; and adjusting the target proportional-integral adjustment model based on the reward data so as to determine a second proportional-integral parameter based on the target proportional-integral adjustment model. The target proportional-integral adjustment model can dynamically output the proportional-integral parameters according to the real-time clock synchronization state, so that the clock synchronization precision is optimized.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a model training method, a clock synchronization method, an electronic device, and a program product. Background Art

[0002] In the field of industrial automation equipment control, achieving precise synchronization across all nodes often requires high communication frequencies and strict synchronization mechanisms. The current mainstream proportional-integral control method typically uses fixed parameters, often based on empirical data. This makes it difficult to adapt to complex and changing operating conditions, unable to meet the high-precision clock synchronization requirements of automation equipment in diverse environments, and unable to guarantee the stability and reliability of motor control. Summary of the Invention

[0003] In view of this, embodiments of the present application provide a model training method, a clock synchronization method, an electronic device, and a program product.

[0004] In a first aspect, an embodiment of the present application provides a model training method, including: determining a first proportional integral parameter of a target proportional integral regulation model; determining a clock synchronization compensation amount based on the first proportional integral parameter; determining reward data of a reinforcement learning algorithm based on the clock synchronization compensation amount; and adjusting the target proportional integral regulation model based on the reward data so as to determine a second proportional integral parameter based on the target proportional integral regulation model.

[0005] In combination with the first aspect, in certain implementations of the first aspect, the clock synchronization compensation amount is determined based on the first proportional integral parameter, including: determining the clock error samples of the master station and the slave station at the current moment; determining the cumulative clock error samples of the master station and the slave station from the starting moment to the current moment, the starting moment indicating the moment when the clock synchronization adjustment is started; and calculating the clock synchronization compensation amount based on the first proportional integral parameter, the clock error samples at the current moment, and the cumulative clock error samples.

[0006] In combination with the first aspect, in certain implementations of the first aspect, the clock error samples and / or the accumulated clock error samples at the current moment are determined based on the target simulation environment.

[0007] In combination with the first aspect, in certain implementations of the first aspect, the target proportional integral regulation model includes at least two hidden layers and at least one output layer; the hidden layer uses a rectified linear unit activation function for nonlinear mapping; and the loss function of the output layer includes a mean square error loss function.

[0008] In the second aspect, an embodiment of the present application provides a clock synchronization method, including: determining the actual clock error of the master station and the slave station at the current moment; using a target proportional integral regulation model to process the actual clock error at the current moment to obtain a proportional integral parameter, wherein the target proportional integral regulation model is trained based on the method described in the first aspect; based on the proportional integral parameter, calculating the clock synchronization compensation amount between the master station and the slave station.

[0009] In combination with the second aspect, in certain implementations of the second aspect, the method further includes: correcting the clock of the master station using a clock synchronization compensation amount so that the clock of the master station is synchronized with the clock of the slave station.

[0010] In combination with the second aspect, in certain implementations of the second aspect, the clock synchronization method is applied to the robotic arm of the embodied robot; before using the target proportional-integral regulation model to process the actual clock error at the current moment, it also includes: setting the optional parameter range of the proportional-integral parameter output by the target proportional-integral regulation model based on the working condition of the robotic arm.

[0011] In combination with the second aspect, in certain implementations of the second aspect, the operating conditions include at least one of load, movement frequency, and clock synchronization delay threshold, the clock synchronization delay threshold represents the threshold corresponding to the actual clock error at the current moment; the boundary value of the optional parameter range of the proportional integral parameter increases with the increase of load; the boundary value of the optional parameter range of the proportional integral parameter decreases with the decrease of movement frequency; the boundary value of the optional parameter range of the proportional integral parameter increases with the decrease of the clock synchronization delay threshold.

[0012] In a third aspect, an embodiment of the present application provides a model training device, comprising: a first determination module for determining a first proportional integral parameter of a target proportional integral regulation model; a second determination module for determining a clock synchronization compensation amount based on the first proportional integral parameter; a third determination module for determining reward data of a reinforcement learning algorithm based on the clock synchronization compensation amount; and an adjustment module for adjusting the target proportional integral regulation model based on the reward data, so as to determine a second proportional integral parameter based on the target proportional integral regulation model.

[0013] In a fourth aspect, an embodiment of the present application provides a clock synchronization device, including: a fourth determination module, used to determine the actual clock error of the master station and the slave station at the current moment; a processing module, used to use a target proportional integral regulation model to process the actual clock error at the current moment to obtain a proportional integral parameter, wherein the target proportional integral regulation model is trained based on the method described in the first aspect; a calculation module, used to calculate the clock synchronization compensation amount between the master station and the slave station based on the proportional integral parameter.

[0014] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program for executing the methods described in the first and second aspects.

[0015] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; and the processor is configured to execute the methods described in the first and second aspects.

[0016] In a seventh aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on an electronic device, the electronic device implements the method described in the first and second aspects.

[0017] This application determines the first proportional integral parameter of the target proportional integral regulation model, determines the clock synchronization compensation amount based on this parameter, further uses the reinforcement learning algorithm to determine the reward data based on the clock synchronization compensation amount, and finally adjusts the target proportional integral regulation model based on the reward data to determine the second proportional integral parameter, forming a dynamic optimization closed loop, and finally obtaining a trained target proportional integral regulation model. This process enables the target proportional integral regulation model to dynamically output proportional integral parameters according to the real-time clock synchronization state, thereby optimizing the clock synchronization accuracy and effectively overcoming the limitations of relying on fixed parameters for proportional integral regulation. In addition, through the introduction of the reinforcement learning algorithm, the target proportional integral regulation model can continuously learn and optimize the output proportional integral parameters according to the actual synchronization effect, so as to adapt to different working conditions and environmental changes, and improve the adaptability and stability of the target proportional integral regulation model in clock synchronization.

[0018] In practical applications, such as robot control systems, this target proportional-integral regulation model can significantly improve its clock synchronization performance, thereby improving the collaborative working ability of the robot's joints, ensuring the stability and reliability of motor control, and meeting the robot's requirements for high-precision clock synchronization in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 The figure is a flow chart of the model training method provided in one embodiment of the present application.

[0021] Figure 2 The figure shows a training process diagram of a target proportional integral regulation model provided in one embodiment of the present application.

[0022] Figure 3 FIG2 is a flow chart of a clock synchronization method provided in an embodiment of the present application.

[0023] Figure 4 Shown is a structural diagram of a model training device provided in one embodiment of the present application.

[0024] Figure 5 FIG2 is a schematic diagram of the structure of a clock synchronization device provided in one embodiment of the present application.

[0025] Figure 6 Shown is a structural schematic diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] Figure 1 FIG. 1 is a flow chart of a model training method according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps.

[0028] Step S110 , determining a first proportional-integral parameter of a target proportional-integral regulation model.

[0029] Specifically, the first proportional-integral parameter includes a proportional gain coefficient and an integral gain coefficient.

[0030] The proportional gain coefficient indicates the strength of the proportional regulation effect and is the magnification of the proportional regulation item. In the clock synchronization scenario, when there is clock deviation, the proportional gain coefficient determines the immediate response strength to the clock error at the current moment. For example, if the proportional gain coefficient is K p , the clock deviation detected at the current moment is e(t), and the proportional adjustment term is K p ×e(t). It can be seen that K p The larger the value of , the stronger the response to the clock deviation at the current moment, and the greater the regulatory effect.

[0031] Integral gain coefficient K iIndicates the strength of the integral regulation effect, which regulates the accumulation of clock error over time, with the goal of eliminating steady-state error. It is understandable that in clock synchronization, after proportional regulation, there may still be a small residual clock error. The integral regulation will gradually increase the regulation effect according to the time integral value of the residual clock error until the deviation is completely eliminated. For example, the integral regulation term is It can be seen that K i The larger it is, the stronger the integration effect is and the higher the sensitivity to clock error accumulation is.

[0032] It is understood that the first proportional-integral parameter refers to the proportional-integral parameter output by the target proportional-integral regulation model before the current training round. For example, if the target proportional-integral regulation model is in the second training round, then the first proportional-integral parameter refers to the proportional-integral parameter output in the first training round. Optionally, the target proportional-integral regulation model is a DeepQ network.

[0033] Step S120: determining a clock synchronization compensation amount based on the first proportional integral parameter.

[0034] The clock synchronization compensation refers to the value that needs to be adjusted to correct the deviation caused by the asynchronous clocks of the joints of the automation equipment. Its purpose is to synchronize the clocks of each joint and reduce the impact of clock deviation on motor control.

[0035] Optionally, the clock synchronization compensation amount

[0036] Step S130: Determine reward data of the reinforcement learning algorithm based on the clock synchronization compensation amount.

[0037] In the reinforcement learning algorithm, reward data is an evaluation indicator for the target proportional-integral regulation model after taking specific actions (such as determining specific proportional-integral parameters) under specific states. It is used to guide the learning process of the target proportional-integral regulation model, that is, to enable the target proportional-integral regulation model to learn which actions are beneficial based on the level of reward data, thereby continuously optimizing its strategy to achieve the long-term optimal goal.

[0038] Optionally, since the desired clock synchronization compensation is as close to zero as possible, that is, the clock offset is zero, the reward data R = -abs(u(t)). Specifically, when u(t) is not equal to zero, abs(u(t)) is a positive number. The negative sign causes the reward data to become negative, indicating that the current clock synchronization state is poor and requires adjustment. If u(t) is equal to zero, that is, there is no clock offset, abs(u(t)) = 0, and R = 0. This indicates that the ideal clock synchronization state has been achieved, with no deviation, and the reward value is zero.

[0039] The reward data calculation method in this embodiment is to motivate the target proportional-integral regulation model to reduce the clock offset u(t) as much as possible during the reinforcement learning process. That is, during the learning process, the target proportional-integral regulation model tends to select those actions (i.e., proportional-integral parameters) that can make u(t) as small as possible, so as to obtain a higher reward value R (i.e., closer to zero or zero).

[0040] Step S140 : adjusting the target proportional-integral adjustment model based on the reward data, so as to determine a second proportional-integral parameter based on the target proportional-integral adjustment model.

[0041] The second proportional-integral parameter refers to the proportional-integral parameter obtained after optimization of the target proportional-integral adjustment model. This parameter will replace the first proportional-integral parameter to achieve a better clock synchronization effect.

[0042] As can be seen from the previous description, the target proportional-integral adjustment model is a reinforcement learning model. It evaluates the effect of the first proportional-integral parameter based on the reward data and adjusts the network parameters within the model accordingly. Then, the target proportional-integral adjustment model outputs the second proportional-integral parameter through continuous learning based on the adjusted network parameters.

[0043] For example, the target proportional integral adjustment model adopts the first proportional integral parameter (K p1 , K i1 ) calculates the clock synchronization compensation amount, and determines the corresponding reward data R1 based on the clock synchronization compensation amount. Then, the target proportional integral adjustment model adjusts the network parameters within the model according to the value of R1 and outputs different combinations of proportional integral parameters, such as trying the second proportional integral parameter (K p2 , K i2 ). For the new parameter combination, the parameter will also be used to calculate the clock synchronization compensation amount, and the new reward data R2 will be determined based on the actual effect. Assume that after multiple rounds of adjustments, the model determines the proportional integral parameter to be (K pn , K in ) The reward data Rn corresponding to it is the largest so far, then the target proportional integral regulation model will output (K pn , K in ) is used as the final network parameters to obtain the trained target proportional integral regulation model.

[0044] This embodiment determines the first proportional integral parameter of the target proportional integral regulation model, determines the clock synchronization compensation amount based on this parameter, further uses the reinforcement learning algorithm to determine the reward data based on the clock synchronization compensation amount, and finally adjusts the target proportional integral regulation model based on the reward data to determine the second proportional integral parameter, forming a dynamic optimization closed loop, and finally obtaining a trained target proportional integral regulation model. This process enables the target proportional integral regulation model to dynamically output proportional integral parameters according to the real-time clock synchronization state, thereby optimizing the clock synchronization accuracy and effectively overcoming the limitations of relying on fixed parameters for proportional integral regulation. In addition, through the introduction of the reinforcement learning algorithm, the target proportional integral regulation model can continuously learn and optimize the output proportional integral parameters according to the actual synchronization effect, so as to adapt to different working conditions and environmental changes, thereby improving the adaptability and stability of the target proportional integral regulation model in clock synchronization.

[0045] In practical applications, such as robot control systems, this target proportional-integral regulation model can significantly improve its clock synchronization performance, thereby improving the collaborative working ability of the robot's joints, ensuring the stability and reliability of motor control, and meeting the robot's requirements for high-precision clock synchronization in complex environments.

[0046] from Figure 1 As can be seen from the examples, determining the clock synchronization compensation is a key step in the model training process. To more accurately determine the clock synchronization compensation and adapt to complex and changing operating conditions, the step of "determining the clock synchronization compensation based on the first proportional-integral parameter" has been further refined. Specifically, the clock error samples of the master and slave stations at the current moment are determined; the accumulated clock error samples of the master and slave stations from the starting moment to the current moment are determined; and the clock synchronization compensation is calculated based on the first proportional-integral parameter, the clock error samples at the current moment, and the accumulated clock error samples.

[0047] The master station refers to the central node responsible for control and management in the communication network, and the slave station is each device node controlled in the communication network, such as the motor of a robotic arm of an automation device. Optionally, the communication network is EtherCAT communication.

[0048] The clock error sample refers to the specific numerical sample of the clock difference between the master station and the slave station in the current time step. It can directly reflect the degree of asynchrony between the master station and the slave station clocks at the current moment.

[0049] Cumulative clock error samples refer to the sample data formed by the cumulative sum of the clock differences between the master and slave stations from the start time to the current time, where the start time indicates the time when clock synchronization adjustment begins. It can be seen that the cumulative clock error samples not only consider the clock error in the current time step, but also integrate the errors in previous time steps, reflecting the overall accumulation of clock asynchrony over a period of time. It should be noted that the time step refers to a discrete time interval and can optionally be determined based on the communication frequency between the master and slave stations. For example, if the communication frequency between the master and slave stations is 1000Hz, then the time step is 1ms.

[0050] For example, the starting time when the master station and the slave station start to adjust the clock synchronization is t0, and the current time is t5. t0-t1, t1-t2, t2-t3, t3-t4, and t4-t5 are each a time step. Assume that at time t5, the clock error sample between the master station and the slave station is determined to be e(t4-t5). At the same time, the cumulative clock error sample between the master station and the slave station from time t0 to time t5 is calculated as It is known that the proportional gain coefficient in the first proportional integral parameter is K p , the integral gain coefficient is K i Then, the clock synchronization compensation can be calculated as: For example, K p is 2, K i is 0.5, e(t4-t5) is 0.1ms, If it is 0.3ms, then substituting it into the formula, we can get the clock synchronization compensation amount u(t) = 2×0.1+0.5×0.3 = 0.35.

[0051] In this embodiment, by separately determining the current clock error sample and the accumulated clock error sample, both real-time information and historical accumulation of clock error can be comprehensively obtained. This comprehensive consideration avoids the problem of insufficient or inaccurate compensation caused by focusing only on the current clock error.

[0052] Furthermore, in some embodiments, the clock error sample and / or the accumulated clock error sample at the current moment are determined based on the target simulation environment.

[0053] The target simulation environment is a virtual simulation environment built for model training, designed to simulate various operating conditions and scenarios in real-world applications. Within this environment, clock errors can be proactively set and added based on training needs and the simulation of actual operating conditions. These errors can be random or patterned, enabling the model to learn how to handle various complex clock synchronization issues during training and output the corresponding proportional-integral tuning parameters.

[0054] Specifically, when the model operates as an intelligent agent in the target simulation environment, it continuously interacts with the environment, acquiring information such as current and accumulated clock error samples. These error samples serve as processing data for the model, which is used to calculate clock synchronization compensation. By continuously introducing different clock errors into the environment, the model can be trained under a variety of error conditions, learning how to determine the optimal proportional-integral parameters under various operating conditions, thereby achieving high-precision clock synchronization.

[0055] In this embodiment, by introducing clock errors in the target simulation environment, the model can be exposed to a rich variety of error scenarios that fit actual working conditions during the training phase, and continuously learn in the complex simulated environment, gradually mastering the optimal proportional integral parameters under different clock error conditions, so that it can quickly and accurately deal with various clock deviation problems in actual applications, enhancing the practicality and robustness of the model in complex and changing scenarios.

[0056] Next, building on the aforementioned model training method, the structure of the target proportional-integral regulation model is further clarified to improve model performance and training results. Specifically, the target proportional-integral regulation model includes at least two hidden layers and at least one output layer; the hidden layers use a rectified linear unit activation function for nonlinear mapping; and the output layer's loss function includes a mean squared error loss function.

[0057] Specifically, the hidden layer, located between the input and output layers, extracts and transforms features from the input data to capture complex patterns and relationships within it, providing the model with deeper representational capabilities. The rectified linear unit activation function outputs the input value when it is greater than or equal to 0, and 0 otherwise. This effectively introduces nonlinearity, accelerates model convergence, and alleviates the vanishing gradient problem.

[0058] The output layer is the final layer of the model, responsible for mapping the features extracted by the hidden layers to the final output. The mean squared error loss function measures the difference between the predicted value output by the model and the ideal true value. It is calculated by squaring the difference between the predicted value and the true value and then averaging the squared difference across all samples to effectively reflect the accuracy of the model's predictions.

[0059] This embodiment constructs a target proportional-integral regulation model comprising at least two hidden layers and at least one output layer, providing the model with sufficient depth and complexity to enable multi-level feature extraction and transformation of input clock error data. Furthermore, the hidden layer uses a rectified linear unit activation function, which not only introduces nonlinearity and enhances the model's ability to fit complex relationships, but also effectively alleviates the vanishing gradient problem, accelerates model convergence, and makes the training process more efficient. The output layer uses a mean squared error loss function, providing a clear and easily calculated direction for optimizing model parameters, helping the model gradually reduce prediction errors during training.

[0060] Figure 2 The figure shows the training process of the target proportional integral regulation model provided by an embodiment of the present application. Figure 2 As shown in the figure, first, the agent and environment are initialized. Specifically, at the outset, the environment needs to be initialized to simulate the motion of the automated device. Simultaneously, the agent, or the target proportional-integral regulation model in reinforcement learning, is initialized, including setting weights, defining its structure, and how it interacts with the initialized environment. Next, training parameters are set, such as the learning rate, discount factor, and number of training rounds. Then, a training loop begins, based on the set number of training rounds.

[0061] Before completing all rounds, the environment needs to be reset before the next round of loops. This means resetting the environment to its initial state to ensure that the target proportional integral regulation model starts from a consistent starting point. Within each single round of loops, the agent needs to select an action. That is, the target proportional integral regulation model selects an action based on the current strategy and the observed environmental state. This action determines how the target proportional integral regulation model adjusts the output proportional integral parameters. Furthermore, the target proportional integral regulation model executes the selected action in the environment, then obtains feedback from the environment, including the new state and reward data. The new state, action, reward data, and other experiences are saved to the memory bank, and the total reward data obtained by the target proportional integral regulation model in the current single round of loops is calculated.

[0062] If enough experience has accumulated in the memory bank, the training process can begin. That is, a batch of experience data is randomly sampled from the memory bank to optimize the target proportional-integral regulation model's ability to select actions.

[0063] When the amount of data in the memory bank is insufficient, the accumulated experience is insufficient for effective learning and policy updates. In this case, the exploration library needs to be updated. This is to encourage the target proportional-integral control model to conduct more exploration attempts in the absence of sufficient valid experience. Optionally, the exploration library records actions that the target proportional-integral control model has not yet fully explored. These actions involve different proportional-integral parameter adjustment methods. By updating the exploration library, the target proportional-integral control model can try these uncommon actions in subsequent interactions, thereby increasing the opportunity to gather diverse experience. For example, when a robotic arm in an automated device is operating under certain conditions, the target proportional-integral control model has already tried some common clock synchronization adjustment actions, but these actions have not been satisfactory. In this case, updating the exploration library can encourage the target proportional-integral control model to try some uncommon parameter adjustment combinations. This helps the target proportional-integral control model gain a more comprehensive understanding of the impact of different actions on clock synchronization, laying the foundation for effective learning and policy optimization when the memory bank data is sufficient.

[0064] Finally, the single-round cycle is continued until all rounds are completed, and the target proportional-integral regulation model that meets the optimization goal is obtained.

[0065] Figure 3 FIG. 1 is a flow chart of a clock synchronization method according to an embodiment of the present application. Figure 3 As shown, the method includes the following steps.

[0066] Step S310: determining the actual clock error between the master station and the slave station at the current moment.

[0067] As mentioned above, the master station refers to the central node responsible for control and management in the communication network, and the slave station is the individual device nodes controlled in the communication network, such as the motor of the robotic arm of the automation equipment.

[0068] The actual clock error between the master and slave stations at the current moment reflects the degree of time deviation between the current slave station clock and the master station clock. Subsequent clock synchronization compensation operations all depend on this error value.

[0069] Step S320 : Using the target proportional-integral adjustment model, the actual clock error at the current moment is processed to obtain proportional-integral parameters.

[0070] The target proportional-integral regulation model is obtained based on the model training method described in the aforementioned embodiment. Optionally, during application, the target proportional-integral regulation model implicitly captures the cumulative effect of historical errors, i.e., the accumulated clock error from the starting moment to the current moment, through a time series state memory or sliding window. The model autonomously learns the integral characteristics of the error and outputs proportional-integral parameters based on the model's input data, namely, the actual clock error at the current moment.

[0071] Step S330 : Calculate the clock synchronization compensation between the master station and the slave station based on the proportional integral parameter.

[0072] Optionally, the solution mentioned in the embodiment of the reference model training method is used to calculate the clock synchronization compensation between the master station and the slave station based on the proportional integral parameter.

[0073] In this embodiment, the actual clock error of the master station and the slave station at the current moment is first determined, and then the target proportional integral regulation model is used to process the actual clock error to obtain the proportional integral parameter. This model can dynamically adjust the output parameters according to different error conditions. Compared with the traditional fixed parameter regulation method, it can adapt to various working conditions more flexibly and accurately, further improve the accuracy and stability of clock synchronization, and avoid problems such as poor synchronization effect or system oscillation caused by fixed parameters. Finally, the clock synchronization compensation amount is calculated based on the proportional integral parameter. This compensation amount can effectively reduce or even eliminate the clock error between the master station and the slave station, and achieve high-precision clock synchronization between the master station and the slave station, thereby improving the operating efficiency and reliability of the entire automation equipment. In particular, for equipment that requires multiple joints to work closely together, it can maintain stable and precise movement under high-frequency control, avoiding problems such as motion delays and error accumulation caused by clock asynchrony.

[0074] Then, in Figure 3 On the basis of the embodiment shown, the clock of the master station is further corrected by using the clock synchronization compensation amount, so that the clock of the master station is synchronized with the clock of the slave station.

[0075] Specifically, the clock synchronization compensation is used to correct the master station clock, that is, the calculated compensation is applied to the master station clock to adjust the master station clock so that the master station clock and the slave station clock gradually reach a synchronized state.

[0076] By aligning the master's clock with the slaves, this solution resolves slave synchronization issues caused by the master's own high jitter when used as a reference clock, reduces the risk of phase-locked loops (PLLs) in the slaves, and improves the clock synchronization accuracy and stability of the automation equipment. Furthermore, it enables the master to better adapt to the clock characteristics of the slaves, avoiding unstable synchronization time caused by master-slave asynchrony, thereby enhancing the reliability and anti-interference capabilities of the automation equipment. This solution ensures the stable operation of equipment that requires close coordination between multiple nodes, ensuring efficient collaboration between nodes based on precise time synchronization, and effectively improving the motion accuracy and task execution success rate of the automation equipment.

[0077] In some embodiments, the clock synchronization method is applied to a robotic arm of an embodied robot. Before processing the actual clock error at the current moment using a target proportional-integral regulation model, the method further includes: setting an optional parameter range for a proportional-integral parameter output by the target proportional-integral regulation model based on the working condition of the robotic arm.

[0078] The optional range of proportional-integral parameters refers to the reasonable range of proportional and integral coefficients that are pre-set in actual applications based on the specific working conditions of the robot arm. This range restricts the possible values ​​of the proportional-integral parameters output by the model, ensuring that the parameters are optimized and adjusted while meeting the robot arm's current task requirements.

[0079] Understandably, different operating conditions have different requirements for clock synchronization accuracy and adjustment speed. For example, when a robot arm performs high-precision assembly operations, higher clock synchronization accuracy and more delicate parameter adjustments are required. In this case, the selectable range of the proportional-integral parameter can be set to a smaller and more precise range. On the other hand, when performing rough handling tasks, the clock synchronization requirements are relatively low, and the selectable parameter range can be appropriately relaxed to increase the flexibility of model adjustment.

[0080] The pre-set optional parameter range provides a reasonable parameter boundary for the subsequent target proportional-integral adjustment model when processing the actual clock error, so that the model can output more optimized proportional-integral parameters within the range that meets the current working conditions of the robot arm, thereby more accurately achieving the synchronization of the master and slave clocks.

[0081] Specifically, in some embodiments, the working condition includes at least one of a load, a motion frequency, and a clock synchronization delay threshold, where the clock synchronization delay threshold represents a threshold corresponding to an actual clock error at a current moment.

[0082] Optionally, the boundary value of the optional parameter range of the proportional-integral parameter increases as the load increases. The reason is that during the operation of the robotic arm, the size of the load will directly affect the movement characteristics of the robotic arm and the stability of the clock synchronization. When the load is small, the movement of the robotic arm is relatively easy, and the pressure of clock synchronization is also small. At this time, a smaller proportional-integral parameter can meet the synchronization requirements. However, when the load increases, the robotic arm may have a larger clock error during the movement, and more flexible and larger parameter adjustments are required to cope with this error change. Therefore, it is necessary to expand the boundary value of the optional range of the proportional-integral parameter.

[0083] In this way, the clock synchronization method can better adapt to the working conditions under different loads, ensuring that the master and slave clocks can be effectively synchronized under various load conditions, thereby improving the stability and reliability of the robot arm control.

[0084] Optionally, the boundary value of the optional parameter range of the proportional integral parameter decreases as the motion frequency decreases. It is understandable that the motion frequency reflects the speed of the robot arm's movement. When the motion frequency is high, the robot arm requires faster and more accurate clock synchronization to ensure that its various components can operate in coordination. At this time, a larger proportional integral parameter range is required to more flexibly adjust the clock error and meet the synchronization requirements under high-frequency motion. When the motion frequency decreases, that is, the robot arm moves relatively slowly, the real-time and accuracy requirements for clock synchronization are reduced, and a smaller proportional integral parameter range can meet the synchronization requirements.

[0085] In this way, the clock synchronization method can better adapt to the working requirements of the robot arm at different motion frequencies, optimize parameter settings, improve the stability and adaptability of the robot arm control system, ensure effective synchronization of the master and slave clocks at various motion frequencies, and ensure the normal operation of the robot arm.

[0086] Optionally, the boundary value of the proportional-integral parameter's optional parameter range increases as the clock synchronization delay threshold decreases. Specifically, the clock synchronization delay threshold is used to measure the maximum acceptable value of the actual clock error at the current moment, which can reflect the robot's requirement for clock synchronization accuracy. When the robot requires more precise clock synchronization, the clock synchronization delay threshold will decrease, which requires the boundary value of the proportional-integral parameter's optional range to increase. This is because a smaller clock synchronization delay threshold has a lower tolerance for clock error. To achieve such high-precision synchronization requirements, a more flexible and wider parameter adjustment range is required.

[0087] By expanding the optional range of proportional-integral parameters, the clock synchronization process can be optimized more effectively to adapt to stricter clock synchronization requirements, ensuring that the clock synchronization method can better meet the needs of the robotic arm under high-precision tasks and improving the stability and reliability of the robotic arm's operation.

[0088] Combined with the above Figures 1 to 3 , describes the method embodiment of the present application in detail, and the following is combined with Figure 4 and Figure 5 , the device embodiment of the present application is described in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so for parts not described in detail, reference can be made to the previous method embodiment.

[0089] Figure 4 The figure shows a schematic diagram of the structure of the model training device provided in one embodiment of the present application. Figure 4 As shown, the model training device 40 provided in the embodiment of the present application includes:

[0090] A first determining module 410 is configured to determine a first proportional-integral parameter of a target proportional-integral regulation model;

[0091] A second determining module 420 is configured to determine a clock synchronization compensation amount based on a first proportional integral parameter;

[0092] A third determination module 430 is configured to determine reward data of a reinforcement learning algorithm based on the clock synchronization compensation amount;

[0093] The adjustment module 440 is configured to adjust the target proportional-integral adjustment model based on the reward data, so as to determine a second proportional-integral parameter based on the target proportional-integral adjustment model.

[0094] In one embodiment of the present application, the second determination module 420 is also used to determine the clock error samples of the master station and the slave station at the current moment; determine the cumulative clock error samples of the master station and the slave station from the starting moment to the current moment, where the starting moment indicates the moment when the clock synchronization adjustment is started; and calculate the clock synchronization compensation amount based on the first proportional integral parameter, the clock error samples at the current moment, and the cumulative clock error samples.

[0095] In one embodiment of the present application, the clock error samples and / or the accumulated clock error samples at the current moment are determined based on the target simulation environment.

[0096] In one embodiment of the present application, the target proportional integral regulation model includes at least two hidden layers and at least one output layer; the hidden layer uses a rectified linear unit activation function for nonlinear mapping; and the loss function of the output layer includes a mean square error loss function.

[0097] Figure 5 The figure shows a schematic diagram of the structure of a clock synchronization device provided by an embodiment of the present application. Figure 5 As shown, the clock synchronization device 50 provided in the embodiment of the present application includes:

[0098] A fourth determining module 510 is configured to determine an actual clock error between the master station and the slave station at a current moment;

[0099] A processing module 520 is configured to process the actual clock error at the current moment using a target proportional-integral adjustment model to obtain a proportional-integral parameter, wherein the target proportional-integral adjustment model is trained based on the method described in the first aspect;

[0100] The calculation module 530 is used to calculate the clock synchronization compensation between the master station and the slave station based on the proportional integral parameter.

[0101] In an embodiment of the present application, the calculation module 530 is further configured to correct the clock of the master station using the clock synchronization compensation amount so that the clock of the master station is synchronized with the clock of the slave station.

[0102] In one embodiment of the present application, the clock synchronization method is applied to the robotic arm of the embodied robot; the fourth determination module 510 is also used to set an optional parameter range of the proportional integral parameter output by the target proportional integral regulation model based on the working condition of the robotic arm.

[0103] In one embodiment of the present application, the operating conditions include at least one of load, movement frequency and clock synchronization delay threshold, the clock synchronization delay threshold represents the threshold corresponding to the actual clock error at the current moment; the boundary value of the optional parameter range of the proportional integral parameter increases with the increase of load; the boundary value of the optional parameter range of the proportional integral parameter decreases with the decrease of movement frequency; the boundary value of the optional parameter range of the proportional integral parameter increases with the decrease of the clock synchronization delay threshold.

[0104] Below, reference Figure 6 To describe the electronic device according to the embodiment of the present application. Figure 6 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.

[0105] like Figure 6 As shown, the electronic device 60 includes one or more processors 601 and a memory 602 .

[0106] The processor 601 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 60 to perform desired functions.

[0107] The memory 602 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 601 may execute the program instructions to implement the methods of the various embodiments of the present application described above and / or other desired functions. The computer-readable storage medium may also store various contents, such as a first proportional-integral parameter, a clock synchronization compensation amount, and reward data.

[0108] In one example, the electronic device 60 may further include an input device 603 and an output device 604 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0109] The input device 603 may include, for example, a keyboard, a mouse, and the like.

[0110] The output device 604 can output various information to the outside, including the first proportional integral parameter, clock synchronization compensation amount, reward data, etc. The output device 604 can include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0111] Of course, to simplify, Figure 6 Only some of the components related to the present application in the electronic device 60 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 60 may further include any other appropriate components according to specific application scenarios.

[0112] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described above in this specification.

[0113] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0114] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method according to various embodiments of the present application described above in this specification.

[0115] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0116] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0117] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0118] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0119] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0120] The above description has been provided for the purpose of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A model training method, characterized in that: include: determining a first proportional-integral parameter of a target proportional-integral regulation model; determining a clock synchronization compensation amount based on the first proportional-integral parameter; Determining reward data for a reinforcement learning algorithm based on the clock synchronization compensation amount; Based on the reward data, the target proportional-integral adjustment model is adjusted to determine a second proportional-integral parameter based on the target proportional-integral adjustment model.

2. The model training method according to claim 1, characterized in that The determining of the clock synchronization compensation amount based on the first proportional-integral parameter includes: Determine the clock error sample of the master station and the slave station at the current moment; Determine the accumulated clock error samples of the master station and the slave station from a starting time to the current time, where the starting time indicates the time when clock synchronization adjustment starts; The clock synchronization compensation amount is calculated based on the first proportional-integral parameter, the clock error sample at the current moment, and the accumulated clock error sample.

3. The model training method according to claim 2, characterized in that The current clock error sample and / or the accumulated clock error sample are determined based on a target simulation environment.

4. The model training method according to claim 1, characterized in that The target proportional integral regulation model includes at least two hidden layers and at least one output layer; The hidden layer uses a rectified linear unit activation function to perform nonlinear mapping; The loss function of the output layer includes a mean square error loss function.

5. A clock synchronization method, characterized in that: include: Determine the actual clock error between the master station and the slave station at the current moment; Using a target proportional-integral regulation model, processing the actual clock error at the current moment to obtain a proportional-integral parameter, wherein the target proportional-integral regulation model is trained based on the method according to any one of claims 1 to 4; Based on the proportional integral parameter, a clock synchronization compensation amount between the master station and the slave station is calculated.

6. The clock synchronization method according to claim 5, wherein: Also includes: The clock of the master station is corrected using the clock synchronization compensation amount so that the clock of the master station is synchronized with the clock of the slave station.

7. The clock synchronization method according to claim 5, characterized in that: The clock synchronization method is applied to the mechanical arm of an embodied robot; Before using the target proportional-integral regulation model to process the actual clock error at the current moment, the method further includes: Based on the working condition of the robotic arm, an optional parameter range of the proportional-integral parameter output by the target proportional-integral regulation model is set.

8. The clock synchronization method according to claim 7, wherein: The working condition includes at least one of a load, a motion frequency, and a clock synchronization delay threshold, wherein the clock synchronization delay threshold represents a threshold corresponding to the actual clock error at the current moment; The boundary value of the optional parameter range of the proportional integral parameter increases as the load increases; The boundary value of the optional parameter range of the proportional integral parameter decreases as the motion frequency decreases; The boundary value of the optional parameter range of the proportional-integral parameter increases as the clock synchronization delay threshold decreases.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which is used to execute the model training method described in any one of claims 1 to 4 and the clock synchronization method described in any one of claims 5 to 8.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the model training method described in any one of claims 1 to 4 and the clock synchronization method described in any one of claims 5 to 8.

11. A computer program product, characterized in that The computer program product includes instructions, which, when executed on an electronic device, enable the electronic device to implement the model training method described in any one of claims 1 to 4 and the clock synchronization method described in any one of claims 5 to 8.

Images