Method, device and equipment for monitoring mechanical arm system and storage medium

Abstract

The application relates to a monitoring method, device and equipment of a mechanical arm system and a storage medium, which are applied to a system monitoring center, the system monitoring center comprises a mechanical arm system function center and an optoelectronic tracking system, the method comprises the following steps: a heartbeat signal is generated, and the heartbeat signal is sent to the mechanical arm system function center and the optoelectronic tracking system; first position information transmitted by the optoelectronic tracking system is received, the first position information is position information of a mechanical arm in a first coordinate system, which is acquired by the optoelectronic tracking system in response to the heartbeat signal; feedback information generated by the mechanical arm system function center in response to the heartbeat signal and used for state detection of the mechanical arm is acquired; and feedback state information for the mechanical arm system is generated according to the first position information and the feedback information. The method provided by the application can monitor the position and joints of the mechanical arm, so that the positioning accuracy of the mechanical arm and the load condition of the control joints are ensured, and the control accuracy of the mechanical arm system is improved to a certain extent.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a monitoring method, apparatus, device, and storage medium for a robotic arm system. Background Technology

[0002] With the development of robot-assisted surgical equipment, the use of surgical robots in orthopedic surgery is becoming increasingly common. The surgical robot system includes a robotic arm system, which is the specific component that executes the surgical robot's operating commands. Therefore, to achieve precise control of the robotic arm, it is necessary to monitor it. Summary of the Invention

[0003] To address the aforementioned technical problems, this application provides a monitoring method, apparatus, device, and storage medium for a robotic arm system, which can monitor the position and joints of the robotic arm and improve the control accuracy of the robotic arm system.

[0004] In a first aspect, this application provides a monitoring method for a robotic arm system, applied in a system monitoring center, the system monitoring center including a robotic arm system functional center and a photoelectric tracking system, the method comprising:

[0005] A heartbeat signal is generated and sent to the functional center of the robotic arm system and the photoelectric tracking system;

[0006] The system receives first position information transmitted by the photoelectric tracking system, wherein the first position information is the position information of the robotic arm in the first coordinate system obtained by the photoelectric tracking system in response to the heartbeat signal.

[0007] The system obtains feedback information generated by the functional center of the robotic arm system in response to the heartbeat signal during state detection of the robotic arm.

[0008] Based on the first position information and the feedback information, feedback status information for the robotic arm system is generated.

[0009] Optionally, the system monitoring center may also include a robotic arm system monitoring center.

[0010] Optionally, generating feedback status information for the robotic arm system based on the first position information and the feedback information includes:

[0011] The first location information is sent to the monitoring center of the robotic arm system.

[0012] The robotic arm system monitoring center generates feedback status information for the robotic arm system based on the first location information and the feedback information.

[0013] Optionally, the feedback information includes the second position information and joint information of the robotic arm in the first coordinate system.

[0014] Optionally, generating feedback status information for the robotic arm system based on the first position information and the feedback information includes:

[0015] The displacement information of the robotic arm is calculated based on the first position information and the second position information;

[0016] The joint information is input into the prediction model, and the predicted joint information is output. The prediction model is trained based on the previously acquired joint information.

[0017] Based on the displacement information and the predicted joint information, feedback state information for the robotic arm system is generated.

[0018] Optionally, the heartbeat signal includes the current heartbeat count code.

[0019] Optionally, calculating the displacement information of the robotic arm based on the first position information and the second position information includes:

[0020] Establish the association between the first location information and the second location information and the current heartbeat count code;

[0021] Based on the aforementioned correlation, the difference between the first position information and the second position information is calculated to obtain the displacement information of the robotic arm.

[0022] Optionally, the robotic arm includes multiple joints, the joint information includes joint angle information and joint load information, and the prediction model includes a prediction angle model and a prediction load model.

[0023] Optionally, the step of inputting the joint information into the prediction model and outputting predicted joint information includes:

[0024] Based on the joint angle information, the predicted angle model is used to predict the angle information of the multiple joints at the next heartbeat count code, thus obtaining the predicted angle information.

[0025] Based on the joint load information, the predicted load model predicts the load information of the multiple joints at the next heartbeat count, thus obtaining the predicted load information.

[0026] The predicted joint information includes the predicted angle information and the predicted load information.

[0027] Optionally, generating feedback state information for the robotic arm system based on the displacement information and the predicted joint information includes:

[0028] The displacement information is compared with a first preset threshold to generate first state information;

[0029] The predicted joint information is compared with the second preset threshold to generate second state information;

[0030] Based on the first state information and the second state information, feedback state information for the robotic arm system is generated.

[0031] Optionally, when the robotic arm system is in detection mode, the method further includes:

[0032] Obtain the previous first position information and the previous second position information that are related to the previous heartbeat's alignment code;

[0033] Calculate the first displacement increment based on the first position information and the previous first position information;

[0034] Calculate the second displacement increment based on the second position information and the previous second position information;

[0035] If the difference between the first displacement increment and the second displacement increment is greater than a third preset threshold, the positioning accuracy of the photoelectric tracking system is adjusted.

[0036] Secondly, this application provides a monitoring device for a robotic arm system, applied in a system monitoring center, the system monitoring center including a robotic arm system functional center and a photoelectric tracking system, the device comprising:

[0037] The first generation unit is used to generate a heartbeat signal and send the heartbeat signal to the functional center of the robotic arm system and the photoelectric tracking system;

[0038] A receiving unit is configured to receive first position information transmitted by the photoelectric tracking system, wherein the first position information is the position information of the robotic arm in a first coordinate system obtained by the photoelectric tracking system in response to the heartbeat signal.

[0039] The acquisition unit is used to acquire feedback information generated by the robotic arm system functional center in response to the heartbeat signal when performing state detection on the robotic arm;

[0040] The second generation unit is used to generate feedback status information for the robotic arm system based on the first position information and the feedback information.

[0041] Thirdly, this application provides an electronic device, comprising:

[0042] Memory;

[0043] Processor; and

[0044] Computer programs;

[0045] The computer program is stored in the memory and configured to be executed by the processor to implement the monitoring method of the robotic arm system as described above.

[0046] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the monitoring method for the robotic arm system as described above.

[0047] This application provides a monitoring method for a robotic arm system, applied in a system monitoring center. The system monitoring center includes a robotic arm system functional center and a photoelectric tracking system. The method includes: generating a heartbeat signal and sending the heartbeat signal to the robotic arm system functional center and the photoelectric tracking system; receiving first position information transmitted by the photoelectric tracking system, the first position information being the position information of the robotic arm in a first coordinate system acquired by the photoelectric tracking system in response to the heartbeat signal; acquiring feedback information generated by the robotic arm system functional center in response to the heartbeat signal to perform state detection on the robotic arm; and generating feedback state information for the robotic arm system based on the first position information and the feedback information. The method provided in this application can monitor the position and joints of the robotic arm to ensure the positioning accuracy of the robotic arm and the load condition of the control joints, and to a certain extent, improves the control accuracy of the robotic arm system. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This application provides a structural diagram of the system monitoring center.

[0051] Figure 2 A flowchart illustrating a monitoring method for a robotic arm system provided in this application;

[0052] Figure 3 A schematic diagram of the second location information provided in this application;

[0053] Figure 4 A schematic diagram illustrating the joint angle information provided in this application;

[0054] Figure 5 A schematic diagram of the joint load information provided in this application;

[0055] Figure 6 A schematic diagram of the structure of a monitoring device for a robotic arm system provided in this application;

[0056] Figure 7 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0057] To better understand the above-mentioned objectives, features, and advantages of this application, the solution of this application will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0058] Many specific details are set forth in the following description in order to provide a full understanding of this application, but this application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of this application, and not all embodiments.

[0059] To address the aforementioned technical problems, this application provides a monitoring method for a robotic arm system. By using a top-down heartbeat signal, the robotic arm system monitoring center is controlled to automatically detect the robotic arm's positioning accuracy and joint prediction, generating feedback status information. This method enables real-time and accurate monitoring of the robotic arm's position and joints, ensuring the robotic arm's positioning accuracy and controlling the load on the joints, and also improving the control accuracy of the robotic arm system to a certain extent. Detailed descriptions are provided through one or more of the following embodiments.

[0060] Figure 1The diagram illustrates the structure of the system monitoring center provided in this application. The system monitoring center includes a robotic arm system monitoring center, a robotic arm system functional center, and a photoelectric tracking system. The system monitoring center sends heartbeat signals to the robotic arm system functional center, the robotic arm system monitoring center, and the photoelectric tracking system for heartbeat detection. The heartbeat signal includes a heartbeat alignment code, which is used to distinguish the feedback information of the current heartbeat signal, the previous heartbeat signal, and the next heartbeat signal. After receiving the heartbeat signal, the photoelectric tracking system transmits first position information to the system monitoring center, which then forwards the first position information to the robotic arm system monitoring center. The first position information is the position coordinates of the robotic arm's end effector in a first coordinate system, which can be the patient's coordinate system. Simultaneously, after receiving the heartbeat signal, the robotic arm system functional center transmits feedback information from the robotic arm's end effector to the robotic arm system monitoring center. This feedback information includes the position coordinates (second position information) in the first coordinate system, robotic arm joint angle information, and robotic arm joint load information. The robotic arm system monitoring center integrates the feedback information and the first position information to generate feedback status information for the robotic arm system, and then feeds the feedback status information back to the system monitoring center.

[0061] Based on the above embodiments, Figure 2 A flowchart illustrating a monitoring method for a robotic arm system provided in this application, applied to the aforementioned system monitoring center, specifically includes the following: Figure 2 Steps S210 to S240 are shown below:

[0062] S210. Generate a heartbeat signal and send the heartbeat signal to the robotic arm system functional center and the photoelectric tracking system.

[0063] Understandably, the system monitoring center includes a clock that can be set to trigger a heartbeat signal every t1. Subsequently, in response to the clock's triggering operation, the system monitoring center generates a heartbeat signal and sends it to the robotic arm system function center and the photoelectric tracking system.

[0064] S220, Receive the first position information transmitted by the photoelectric tracking system.

[0065] The first position information is the position information of the robotic arm in the first coordinate system obtained by the photoelectric tracking system in response to the heartbeat signal.

[0066] Optionally, the system monitoring center may further include a robotic arm system monitoring center, which, after receiving the first location information, sends the first location information to the robotic arm system monitoring center.

[0067] Understandably, based on the above S210, after receiving the heartbeat signal, the photoelectric tracking system will acquire the current position coordinates of the robotic arm in the patient coordinate system collected by its internal positioning device, and transmit this as the first position information to the system monitoring center. After receiving the first position information, the system monitoring center will forward it to the robotic arm system monitoring center.

[0068] S230. Obtain feedback information generated by the functional center of the robotic arm system in response to the heartbeat signal for state detection of the robotic arm.

[0069] Understandably, based on the above S220, the robotic arm system functional center responds to the heartbeat signal to automatically detect the status of the robotic arm, generates feedback information, and transmits the feedback results to the robotic arm system monitoring center.

[0070] S240. Generate feedback status information for the robotic arm system based on the first position information and the feedback information.

[0071] Optionally, in step S240 above, feedback status information for the robotic arm system is generated based on the first position information and the feedback information. This can be achieved through the following steps:

[0072] The robotic arm system monitoring center generates feedback status information for the robotic arm system based on the first location information and the feedback information.

[0073] Understandably, based on the above S230 and S220, the robotic arm system monitoring center integrates the first position information and feedback information to generate feedback status information for the robotic arm system, and feeds the feedback status information back to the system monitoring center. The feedback status information reflects the current status of the robotic arm.

[0074] Optionally, the feedback information includes the second position information and joint information of the robotic arm in the first coordinate system.

[0075] Understandably, each time a heartbeat signal is received, the robotic arm system monitoring center receives a set of data related to the heartbeat counter code of that heartbeat signal. Each set of data includes first position information and feedback information. The feedback information includes second position information and joint information. The joint information includes joint angle information and joint load information. Based on each set of data, displacement information, joint angle information, and joint load information can be obtained. Each set of data is associated with the heartbeat counter code. By arranging each set of data in order according to the value of the heartbeat counter code, the association between displacement information and heartbeat counter code, the association between joint angle information and heartbeat counter code, and the association between joint load information and heartbeat counter code can be obtained.

[0076] Optionally, in step S240 above, feedback status information for the robotic arm system is generated based on the first position information and the feedback information. This can be achieved through the following steps:

[0077] Based on the first position information and the second position information, the displacement information of the robotic arm is calculated; the joint information is input into the prediction model, and the predicted joint information is output. The prediction model is trained based on the previously acquired joint information; based on the displacement information and the predicted joint information, feedback state information for the robotic arm system is generated.

[0078] Understandably, the difference between the first and second position information is calculated to obtain the current displacement information of the robotic arm. Then, the joint information is input into a pre-built prediction model to predict the joint information to be associated with the next heartbeat alignment code, and the predicted joint information is output. The prediction model is trained based on the previously acquired joint information. Subsequently, based on the displacement information and the predicted joint information, feedback state information for the robotic arm system is generated.

[0079] Optionally, the displacement information of the robotic arm is calculated based on the first position information and the second position information, which can be achieved through the following steps:

[0080] Establish a correlation between the first position information and the second position information and the current heartbeat count code; based on the correlation, calculate the difference between the first position information and the second position information to obtain the displacement information of the robotic arm.

[0081] Understandably, after receiving a heartbeat signal, the photoelectric tracking system can establish a correlation between the first position information and the heartbeat count code. The heartbeat count code can be seen as the current number of heartbeat signals sent, which is counted by a counter. The heartbeat count code can also be understood as a count value for heartbeats, denoted as heartbeat count. For example, the first position information is (xn, yn, zn), where xn represents the correlation between the heartbeat count code and the position coordinates, and (xn, yn, zn) represents the spatial coordinates of the robotic arm's end effector at the nth heartbeat count from the photoelectric tracking system. The robotic arm system monitoring center can also establish the correlation between the first position information and the heartbeat count code after receiving the first position information. The correlation between the feedback information and the heartbeat count code can be established at the robotic arm system functional center or the robotic arm system monitoring center; the specific establishment of the correlation is not limited here. For example, the second position information is (Xn, Yn, Zn), representing the spatial coordinates of the robotic arm's end effector at the nth heartbeat count from the robotic arm system functional center. Subsequently, based on the correlation between the position information, the difference between the first position information and the second position information is calculated to obtain the displacement information of the robotic arm. Since the photoelectric tracking system has high absolute position detection accuracy for spatial positioning, and the spatial coordinates measured by the functional center of the robotic arm system are obtained through incremental calculation, the absolute positioning accuracy of the robotic arm can be obtained by calculating the difference between the two coordinates each time. The calculation formula for the displacement information based on the first position information and the second position information is shown in formula (1):

[0082]

[0083] In the formula, the first position information is (xn, yn, zn), the second position information is (Xn, Yn, Zn), the displacement information is (Δx, Δy, Δz), and n is the heartbeat count. The heartbeat count corresponds to the heartbeat count code and may be the same as the heartbeat count code. The displacement information calculated by formula (1) can be used to determine the positioning accuracy of the robotic arm under each heartbeat count. If the displacement value of each axis is less than the set threshold, the position control of the robotic arm is considered to be accurate.

[0084] For example, see Figure 3 , Figure 3The schematic diagram of the second position information provided in this application includes (X1,Y1,Z1), (X2,Y2,Z2), (X3,Y3,Z3)...(Xn,Yn,Zn), representing n second position information provided by the functional center of the robotic arm system. Each second position information includes coordinate values ​​of the x, y, and z axes, where (X1,Y1,Z1) represents the spatial coordinate value of the robotic arm during the first heartbeat count, and so on, with (Xn,Yn,Zn) representing the spatial coordinate value of the robotic arm during the nth heartbeat count. It is understood that a set of first position information (x1,y1,z1), (x2,y2,z2), (x3,y3,z3)...(xn,yn,zn) obtained by the photoelectric tracking system also includes coordinate values ​​of the x, y, and z axes; see the second position information for details, which will not be elaborated here.

[0085] Optionally, the robotic arm includes multiple joints, the joint information includes joint angle information and joint load information, and the prediction model includes a prediction angle model and a prediction load model.

[0086] Optionally, the joint information described above is input into the prediction model, and the predicted joint information is output. This can be achieved through the following steps:

[0087] Based on the joint angle information, the predicted angle model predicts the angle information of the multiple joints at the next heartbeat count code, thus obtaining predicted angle information; based on the joint load information, the predicted load model predicts the load information of the multiple joints at the next heartbeat count code, thus obtaining predicted load information; wherein, the predicted joint information includes the predicted angle information and the predicted load information.

[0088] Understandably, establishing the correlation between joint angle information and joint load information and the heartbeat signal involves determining the angle information and load information of each joint under the heartbeat count. Subsequently, the joint angle information is input into the prediction angle model to predict the angles of the multiple joints in the robotic arm at the next heartbeat count, thus obtaining the predicted angle information. The heartbeat signal includes the current heartbeat count; the heartbeat count of the next heartbeat signal is denoted as the next heartbeat count, and the heartbeat count of the previous heartbeat signal is denoted as the previous heartbeat count. The heartbeat count is obtained by encoding the heartbeat count. Each heartbeat signal has a corresponding set of data, including first position information, second position information, joint angle information, and joint load information. For example, obtaining the joint angle information from heartbeat count 1 to heartbeat count n... Based on the information of n joint angles, the motion joint information of each axis can be obtained. This means that each robotic arm will have enough motion data. Under fixed applicable scenarios and actions, the motion difference of each axis is not very large. Therefore, the motion of each axis can be predicted. Specifically, the motion data can be converted into a function of time to construct a prediction angle model. The prediction angle model can be obtained through motion simulation. Using multiple data of a single axis, the motion data of each axis at the (n+1)th heartbeat count can be obtained. The current heartbeat alignment code corresponds to the nth heartbeat count, the next heartbeat alignment code corresponds to the (n+1)th heartbeat count, and the previous heartbeat alignment code corresponds to the (n-1)th heartbeat count. The prediction angle model is shown in formula (2):

[0089]

[0090] In the formula, F represents the predicted angle model, (A1-m, A2-m, A3-m, A4-m, A5-m, A6-m, A7-m...An-m) are the joint angle information of the m-axis from heartbeat count 1 to n, m is the joint axis included in the robotic arm, n+1 represents the (n+1)th heartbeat count, and A... (n+1)-m This represents the joint angle information of axis m at the (n+1)th heartbeat count, which is the predicted angle information. Understandably, based on the predicted angle model, motion prediction can be performed for each axis, making the risk of failure predictable before a certain axis is about to enter a dead point.

[0091] For example, see Figure 4 , Figure 4This is a schematic diagram of the joint angle information provided in this application. (An-1, An-2, An-3, An-4, An-5, An-6, An-7) represent the angle data of each joint axis of the robotic arm at the nth heartbeat count. The robotic arm includes 7 joint axes, and each axis includes one angle data at one heartbeat count. For example, A1-1 represents the angle data of axis 1 at the 1st heartbeat count, and An-1 represents the angle data of axis 1 at the nth heartbeat count.

[0092] Understandably, by using the load prediction model based on joint load information, the load information of multiple joints at the next heartbeat count is predicted, and the predicted load information is obtained. The construction of the predicted load model is the same as the predicted angle model mentioned above, as shown in formula (3):

[0093]

[0094] In the formula, H represents the predicted angle model, (G1-m, G2-m, G3-m, G4-m, G5-m, G6-m, G7-m...Gn-m) are the joint load information of the m-axis from heartbeat count 1 to n, and G... (n+1)-m This represents the joint load information of the m-axis at the (n+1)th heartbeat count, which is the predicted load information.

[0095] For example, see Figure 5 , Figure 5 This is a schematic diagram of the joint load information provided in this application. (Gn-1, Gn-2, Gn-3, Gn-4, Gn-5, Gn-6, Gn-7) represent the load data of each joint axis of the robotic arm at the nth heartbeat count. For example, Gn-1 represents the load data of axis 1 at the nth heartbeat count.

[0096] Understandably, the increments between adjacent load and angle data on the same axis can also be calculated to adjust the prediction model and understand the incremental changes during the movement of each joint axis, for example... Figure 4 The calculation of the increments of A1-1 and A2-1 shown yields ΔA1-2, denoted as the angle increment, which is used to adjust the prediction angle model, for example. Figure 5 The increments of G1-1 and G2-1 shown are calculated to obtain ΔG1-2, denoted as the load increment, which is used to adjust the predicted load model.

[0097] Optionally, the above-mentioned generation of feedback state information for the robotic arm system based on the displacement information and the predicted joint information can be achieved through the following steps:

[0098] The displacement information is compared with a first preset threshold to generate first state information; the predicted joint information is compared with a second preset threshold to generate second state information; and feedback state information for the robotic arm system is generated based on the first state information and the second state information.

[0099] Understandably, after calculating the displacement information, it is compared with a first preset threshold to determine the accuracy of the robotic arm's positioning. If the displacement information is less than the first preset threshold, it indicates that the robotic arm's position control is accurate, and a first state information of accurate control is generated. This first state information can be in the form of data encoding. Otherwise, it indicates that the robotic arm's position control accuracy is low, and position calibration is required. After predicting the angle data of each axis during operation, the predicted angle information is compared with a second preset threshold to predict the risk of failure. This means determining whether each axis will enter a dead point in the next movement. If the predicted angle information is less than the second preset threshold, it means that the next movement will not enter a dead point. Otherwise, it means that the next movement is very likely to enter a dead point. Based on the predicted angle information, the stiffness of the joints that are about to enter a dead point is adjusted to prevent the robotic arm from crashing due to the user entering a dead point during free dragging.

[0100] Optionally, when the robotic arm system is in detection mode, the method further includes:

[0101] The system acquires the previous first position information and the previous second position information that are related to the previous heartbeat alignment code; calculates the first displacement increment based on the first position information and the previous first position information; calculates the second displacement increment based on the second position information and the previous second position information; if the difference between the first displacement increment and the second displacement increment is greater than a third preset threshold, the positioning accuracy of the photoelectric tracking system is adjusted.

[0102] Understandably, the robotic arm system includes a working mode and a detection mode. In working mode, only position information needs to be recorded; there is no need to calculate displacement increments based on the position information. In detection mode, it is necessary to detect the positioning accuracy of the photoelectric tracking system. Specifically, the previous first position information and the previous second position information, which are related to the previous heartbeat alignment code, are acquired. The first position information, which is related to the current heartbeat alignment code, is recorded as the current first position information, and the second position information is recorded as the current second position information. Subsequently, for the photoelectric tracking system, the difference between the current first position information and the previous first position information is calculated to obtain the first displacement increment. The first displacement increment represents the displacement of the robotic arm end effector calculated by the photoelectric tracking system between two heartbeat counts. For the functional center of the robotic arm system, the difference between the current second position information and the previous second position information is calculated to obtain the second displacement increment. The second displacement increment represents the displacement of the robotic arm end effector calculated by the robotic arm system between two heartbeat counts. At the same time, the accuracy of the second displacement increment can also be regarded as the repeatability accuracy of the robotic arm, and the accuracy of the first displacement increment can also be regarded as the accuracy of the photoelectric tracking system. Understandably, theoretically, the repeatability of the robotic arm system is less than that of the photoelectric tracking system. When the difference between the displacement increments calculated by the two systems is large, it indicates that the positioning function of the photoelectric tracking system has malfunctioned. That is, the difference between the first and second displacement increments is calculated and compared with the third preset threshold. If the difference is greater than the third preset threshold, it indicates that the positioning accuracy of the photoelectric tracking system has malfunctioned, and the positioning accuracy of the photoelectric tracking system needs to be adjusted in the detection mode. The specific formula for calculating the displacement increment is shown in the following formula (4):

[0103]

[0104] In the formula, ΔS n-1-n The second displacement increment can also be represented as ΔS. (n-1)-n Where n-1 represents the (n-1)th heartbeat count, and n represents the nth heartbeat count, the difference between the previous second position information associated with the (n-1)th heartbeat count and the current second position information associated with the nth heartbeat count is calculated to obtain the second displacement increment. It can be understood that the first displacement increment ΔS (n-1)-n The calculation formula is the same as formula (4), and will not be repeated here.

[0105] The monitoring method for the robotic arm system provided in this application sends heartbeat signals to the photoelectric tracking system, the robotic arm system functional center, and the robotic arm system monitoring center for heartbeat detection. Subsequently, by acquiring the first position information fed back by the photoelectric tracking system and the second position information and joint information fed back by the robotic arm system functional center, the current joint angle prediction, joint load, and positioning accuracy of the robotic arm are detected, generating feedback status information for the robotic arm system. This allows the system monitoring center to understand the positioning status of the robotic arm with each heartbeat count based on the feedback status information, determine the control accuracy of the robotic arm position, accurately predict the movement of the joint axes in the robotic arm, provide early warning for joint stiffness adjustment, reduce the risk of joint axes entering dead points, and prevent joint load from exceeding limits, effectively improving the control accuracy of the robotic arm in the surgical robot system.

[0106] Figure 6 This is a schematic diagram of the structure of a monitoring device for a robotic arm system provided in this application. The monitoring device can execute the processing flow provided in the embodiments of the robotic arm system monitoring method, and is applied to a system monitoring center. The system monitoring center includes a robotic arm system functional center and a photoelectric tracking system, such as... Figure 6 As shown, the monitoring device 600 of the robotic arm system includes a first generating unit 610, a receiving unit 620, an acquisition unit 630, and a second generating unit 640, wherein:

[0107] The first generation unit 610 is used to generate a heartbeat signal and send the heartbeat signal to the robotic arm system functional center and the photoelectric tracking system;

[0108] The receiving unit 620 is used to receive the first position information transmitted by the photoelectric tracking system, wherein the first position information is the position information of the robotic arm in the first coordinate system obtained by the photoelectric tracking system in response to the heartbeat signal;

[0109] Acquisition unit 630 is used to acquire feedback information generated by the robotic arm system functional center in response to the heartbeat signal when performing state detection on the robotic arm.

[0110] The second generation unit 640 is used to generate feedback status information for the robotic arm system based on the first position information and the feedback information.

[0111] Optionally, the system monitoring center in device 600 may also include a robotic arm system monitoring center.

[0112] Optionally, the second generation unit 600 is used for:

[0113] The first location information is sent to the monitoring center of the robotic arm system.

[0114] The robotic arm system monitoring center generates feedback status information for the robotic arm system based on the first location information and the feedback information.

[0115] Optionally, the feedback information in device 600 includes the second position information and joint information of the robotic arm in the first coordinate system.

[0116] Optionally, the second generation unit 600 is used for:

[0117] The displacement information of the robotic arm is calculated based on the first position information and the second position information;

[0118] The joint information is input into the prediction model, and the predicted joint information is output. The prediction model is trained based on the previously acquired joint information.

[0119] Based on the displacement information and the predicted joint information, feedback state information for the robotic arm system is generated.

[0120] Optionally, the heartbeat signal in device 600 includes a current heartbeat count code.

[0121] Optionally, the second generation unit 600 is used for:

[0122] Establish the association between the first location information and the second location information and the current heartbeat count code;

[0123] Based on the aforementioned correlation, the difference between the first position information and the second position information is calculated to obtain the displacement information of the robotic arm.

[0124] Optionally, the robotic arm in device 600 includes multiple joints, the joint information includes joint angle information and joint load information, and the prediction model includes a prediction angle model and a prediction load model.

[0125] Optionally, the second generation unit 600 is used for:

[0126] Based on the joint angle information, the predicted angle model is used to predict the angle information of the multiple joints at the next heartbeat count code, thus obtaining the predicted angle information.

[0127] Based on the joint load information, the predicted load model predicts the load information of the multiple joints at the next heartbeat count, thus obtaining the predicted load information.

[0128] The predicted joint information includes the predicted angle information and the predicted load information.

[0129] Optionally, the second generating unit 600 is used to include:

[0130] The displacement information is compared with a first preset threshold to generate first state information;

[0131] The predicted joint information is compared with the second preset threshold to generate second state information;

[0132] Based on the first state information and the second state information, feedback state information for the robotic arm system is generated.

[0133] Optionally, device 600 is also used for:

[0134] Obtain the previous first position information and the previous second position information that are related to the previous heartbeat's alignment code;

[0135] Calculate the first displacement increment based on the first position information and the previous first position information;

[0136] Calculate the second displacement increment based on the second position information and the previous second position information;

[0137] If the difference between the first displacement increment and the second displacement increment is greater than a third preset threshold, the positioning accuracy of the photoelectric tracking system is adjusted.

[0138] Figure 6 The monitoring device of the robotic arm system in the illustrated embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0139] Figure 7 This is a schematic diagram of an electronic device provided in this application. See below for details. Figure 7 The diagram illustrates a structural schematic suitable for implementing the electronic device 700 of this application. The electronic device 700 of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 7 The electronic device shown is merely an example and should not impose any limitations on the functionality and scope of this application.

[0140] like Figure 7As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703 to implement the monitoring method of the robotic arm system as described in the embodiments of this application. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0141] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0142] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the monitoring method for the robotic arm system as described above. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of this application.

[0143] It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0144] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0145] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0146] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0147] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0149] The units described in this application can be implemented in software or hardware. The names of the units are not, in some cases, limiting of the unit itself.

[0150] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0151] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0152] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or gateway that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or gateway. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or gateway that includes said element.

[0153] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A monitoring method for a robotic arm system, characterized in that, The method is applied to a monitoring center, which includes a system monitoring center, a robotic arm system functional center, and a photoelectric tracking system. A heartbeat signal is generated and sent to the functional center of the robotic arm system and the photoelectric tracking system, wherein the heartbeat signal includes the current heartbeat count code; The system receives first position information transmitted by the photoelectric tracking system, wherein the first position information is the position information of the robotic arm in the first coordinate system obtained by the photoelectric tracking system in response to the heartbeat signal. The system obtains feedback information generated by the functional center of the robotic arm system in response to the heartbeat signal to perform state detection on the robotic arm. The feedback information includes the second position information and joint information of the robotic arm in the first coordinate system. The robotic arm includes multiple joints, and the joint information includes joint angle information and joint load information. Establish the association between the first location information and the second location information and the current heartbeat count code; Based on the correlation, the difference between the first position information and the second position information is calculated to obtain the displacement information of the robotic arm; The joint information is input into the prediction model, and the predicted joint information is output. The prediction model is trained based on the previously acquired joint information. The predicted joint information includes predicted angle information and predicted load information. The displacement information is compared with a first preset threshold to generate first state information; The predicted joint information is compared with the second preset threshold to generate second state information; Based on the first state information and the second state information, feedback state information for the robotic arm system is generated.

2. The method according to claim 1, characterized in that, The monitoring center also includes a robotic arm system monitoring center, and the method further includes: The first location information is sent to the monitoring center of the robotic arm system. The robotic arm system monitoring center generates feedback status information for the robotic arm system based on the first location information and the feedback information.

3. The method according to claim 1, characterized in that, The prediction model includes a prediction angle model and a prediction load model. The step of inputting the joint information into the prediction model and outputting predicted joint information includes: Based on the joint angle information, the predicted angle model is used to predict the angle information of the multiple joints at the next heartbeat count code, thereby obtaining the predicted angle information. Based on the joint load information, the predicted load model predicts the load information of the multiple joints at the next heartbeat count, thus obtaining the predicted load information.

4. The method according to claim 1, characterized in that, When the robotic arm system is in detection mode, the method further includes: Obtain the previous first position information and the previous second position information that are related to the previous heartbeat's alignment code; Calculate the first displacement increment based on the first position information and the previous first position information; Calculate the second displacement increment based on the second position information and the previous second position information; If the difference between the first displacement increment and the second displacement increment is greater than a third preset threshold, the positioning accuracy of the photoelectric tracking system is adjusted.

5. A monitoring device for a robotic arm system, used to implement the monitoring method for the robotic arm system according to any one of claims 1-4, characterized in that, Applied to a monitoring center, the monitoring center includes the system monitoring center, the robotic arm system functional center, and the photoelectric tracking system, and the device includes: The first generation unit is used to generate a heartbeat signal and send the heartbeat signal to the functional center of the robotic arm system and the photoelectric tracking system; A receiving unit is configured to receive first position information transmitted by the photoelectric tracking system, wherein the first position information is the position information of the robotic arm in a first coordinate system obtained by the photoelectric tracking system in response to the heartbeat signal. The acquisition unit is used to acquire feedback information generated by the robotic arm system functional center in response to the heartbeat signal when performing state detection on the robotic arm; The second generation unit is used to generate feedback status information for the robotic arm system based on the first position information and the feedback information.

6. An electronic device, characterized in that, include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the monitoring method of the robotic arm system as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the monitoring method for the robotic arm system as described in any one of claims 1 to 4.

Images