A method, apparatus, and terminal device for indicating the position of a trolley array.

Abstract

This application relates to the field of surgical robot technology, and proposes a method, apparatus, and terminal device for indicating the position of a trolley array. In this method, the base of the robotic arm is first used as a position reference to detect whether the trolley array and the optical navigation device are located on the same side of the base. Since the robotic arm of the surgical trolley is mounted on the base, if the trolley array and the optical navigation device are on the same side of the base, the optical path from the optical navigation device to the trolley array will not be blocked by the robotic arm, and the installation position of the trolley array can be considered appropriate. Conversely, if the trolley array and the optical navigation device are on different sides of the base, the optical path from the optical navigation device to the trolley array may be blocked by the robotic arm, indicating that the installation position of the trolley array needs adjustment. The installation position of the trolley array can then be corrected manually or automatically, thereby improving the success rate and accuracy of the robotic arm's registration and calibration.

Description

Technical Field

[0001] This application relates to the field of surgical robot technology, and in particular to a method, apparatus and terminal device for indicating the position of a trolley array. Background Technology

[0002] To meet the demands for precision and stability in clinical surgical procedures, surgical robots are being used more and more widely. For ease of movement, surgical robots can be structured as surgical trolleys. Their robotic arms require pre-operative registration and calibration. This registration and calibration process involves using optical navigation equipment to capture optical markers on an array of markers on the surgical trolley, using the positional information of these markers to complete the calibration. However, if the trolley array is improperly positioned, the optical markers may be obstructed by objects such as the robotic arms of the surgical trolley, leading to a decrease in the success rate and accuracy of registration and calibration. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method, apparatus, and terminal device for indicating the position of a trolley array, which can provide a prompt when the trolley array is installed in an inappropriate position, thereby improving the success rate and accuracy of the robotic arm's registration and calibration.

[0004] The first aspect of this application provides a method for indicating the position of a trolley array, including:

[0005] The detection trolley array and optical navigation equipment are located on the same side of the robotic arm's base; both the robotic arm and the trolley array are mounted on the surgical trolley.

[0006] If the trolley array and the optical navigation device are located on different sides of the base, it indicates that the installation position of the trolley array needs to be adjusted.

[0007] In this embodiment, the base of the robotic arm is used as a position reference to detect whether the trolley array and the optical navigation device are located on the same side of the base. Since the robotic arm of the surgical trolley is mounted on the base, if the trolley array and the optical navigation device are located on the same side of the base, the optical path from the optical navigation device to the trolley array will not be blocked by the robotic arm, and the installation position of the trolley array can be considered appropriate. Conversely, if the trolley array and the optical navigation device are located on different sides of the base, the optical path from the optical navigation device to the trolley array may be blocked by the robotic arm. In this case, it will indicate that the installation position of the trolley array needs to be adjusted. The installation position of the trolley array can then be corrected manually or automatically, thereby improving the success rate and accuracy of the robotic arm's registration and calibration.

[0008] In one implementation of this application, the robotic arm is equipped with an end effector array. Detecting whether the trolley array and the optical navigation device are located on the same side of the robotic arm's base can include:

[0009] Obtain the first spatial coordinates of the trolley array in the coordinate system of the optical navigation device and the second spatial coordinates of the end array in the coordinate system of the optical navigation device;

[0010] Based on the first and second spatial coordinates, the third spatial coordinates of the trolley array in the base coordinate system and the fourth spatial coordinates of the optical navigation device in the base coordinate system are obtained.

[0011] Based on the third and fourth spatial coordinates, determine whether the trolley array and the optical navigation device are located on the same side of the base.

[0012] Furthermore, determining whether the trolley array and the optical navigation device are located on the same side of the base based on the third and fourth spatial coordinates can include:

[0013] Obtain the first y-coordinate in the third spatial coordinate system and the second y-coordinate in the fourth spatial coordinate system; wherein the y-axis direction corresponds to the width direction of the surgical cart.

[0014] If the product of the first y-coordinate and the second y-coordinate is greater than zero, then the trolley array and the optical navigation device are located on the same side of the base.

[0015] In one implementation of this application, after detecting whether the trolley array and the optical navigation device are located on the same side of the robot arm's base, the following may be further included:

[0016] If the trolley array and the optical navigation device are located on the same side of the base, then the position of the trolley array is detected to be outside the preset space range;

[0017] If the position of the trolley array exceeds the preset space range, a collision risk will be indicated.

[0018] Furthermore, detecting whether the position of the trolley array exceeds a preset spatial range may include:

[0019] If the trolley array is located on the first side of the base, it is detected whether the position of the trolley array exceeds the first spatial range; wherein, the first spatial range is a preset spatial range located on the first side of the base;

[0020] If the trolley array is located on the second side of the base, it is detected whether the position of the trolley array exceeds the second spatial range; wherein, the second spatial range is a preset spatial range located on the second side of the base.

[0021] Furthermore, detecting whether the position of the cart array exceeds the first spatial range can include:

[0022] Obtain the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system;

[0023] Determine the maximum x-coordinate, maximum y-coordinate, and maximum z-coordinate of each boundary marker point in the spatial coordinate system of the base coordinate system;

[0024] If the maximum x-coordinate, maximum y-coordinate, or maximum z-coordinate exceeds the first spatial range, then the position of the trolley array is determined to be outside the first spatial range.

[0025] Furthermore, detecting whether the position of the cart array exceeds the second spatial range can include:

[0026] Obtain the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system;

[0027] Determine the maximum x-coordinate, minimum y-coordinate, and maximum z-coordinate of each boundary marker point in the spatial coordinate system of the base coordinate system;

[0028] If the maximum x-coordinate, minimum y-coordinate, or maximum z-coordinate exceeds the second spatial range, then the position of the trolley array is determined to be outside the second spatial range.

[0029] Furthermore, after detecting whether the position of the trolley array exceeds the preset spatial range, it may also include:

[0030] If the position of the trolley array exceeds the preset space range, the x-axis, y-axis, and z-axis excess distances of the trolley array are detected and displayed; where the x-axis, y-axis, and z-axis excess distances are the distances by which the trolley array exceeds the preset space range in the x-axis, y-axis, and z-axis directions, respectively.

[0031] A second aspect of this application provides a position indication device for a trolley array, comprising:

[0032] The detection module is used to detect whether the trolley array and the optical navigation device are located on the same side of the robotic arm's base; wherein, both the robotic arm and the trolley array are mounted on the surgical trolley;

[0033] The prompting module indicates that the installation position of the trolley array needs to be adjusted if the trolley array and the optical navigation device are located on different sides of the base.

[0034] A third aspect of this application provides a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the position prompting method for a trolley array as provided in the first aspect of this application.

[0035] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the position indication method for a trolley array as provided in the first aspect of this application.

[0036] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the position prompting method for a trolley array as provided in the first aspect of this application.

[0037] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of a surgical robot provided in an embodiment of this application;

[0039] Figure 2 This is a flowchart of a position indication method for a trolley array provided in an embodiment of this application;

[0040] Figure 3 This is a schematic diagram showing the orientation of the trolley array and optical navigation device provided in this application embodiment, located on the same side of the robotic arm base;

[0041] Figure 4 This is a schematic diagram showing the orientation of the trolley array and optical navigation device provided in the embodiments of this application located on different sides of the robotic arm base;

[0042] Figure 5 This is a schematic diagram of the functional modules of a surgical robot provided in an embodiment of this application;

[0043] Figure 6 yes Figure 5 A schematic diagram illustrating the workflow of the directional detection module for the central trolley array;

[0044] Figure 7 yes Figure 5 A schematic diagram of the workflow of the collision detection module for the trolley array;

[0045] Figure 8 yes Figure 5 A schematic diagram of the interface output by the interface display module;

[0046] Figure 9 This is a schematic diagram of the structure of a position indication device for a trolley array provided in an embodiment of this application;

[0047] Figure 10 This is a schematic diagram of a terminal device provided in an embodiment of this application. Detailed Implementation

[0048] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail. Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0049] Surgical robots require pre-operative registration and calibration of their robotic arms. The positional information of the trolley array used in the calibration process significantly impacts the robotic arm registration and subsequent procedures. Registration and calibration utilize optical navigation equipment to capture optical markers on the trolley array, using the positional information of these markers to complete the calibration. If the trolley array is improperly positioned, these optical markers may be obstructed by objects such as the robotic arm, preventing the optical navigation equipment from accurately capturing their positions and thus affecting the success and accuracy of registration and calibration. To address these issues, this application provides a method, apparatus, and terminal device for indicating the position of the trolley array, which can provide prompts when the trolley array is improperly positioned, thereby improving the success and accuracy of robotic arm registration and calibration. For more specific technical details regarding the embodiments of this application, please refer to the method embodiments described below.

[0050] Please see Figure 1 This diagram illustrates a surgical robot according to an embodiment of this application. Since the surgical robot has a trolley-like structure, it can be called a surgical trolley. The surgical trolley can be equipped with components such as a robotic arm, a trolley array, and a display screen. The robotic arm can be mounted on a base, and an end effector array can be set at the end of the robotic arm. Both the trolley array and the end effector array are equipped with optical markers. The position information of the optical markers can be collected by an optical tracking system (OTS) to obtain the attitude of the trolley array and the end effector array. [Further details about the application are needed for a complete translation.] Figure 1 Before registering and calibrating the robotic arm, the method provided in this application embodiment can be used to check whether the installation position of the trolley array is appropriate, thereby improving the success rate and accuracy of registration and calibration.

[0051] It should be understood that the execution subject of the various method embodiments of this application can be various types of terminal devices or servers, such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, surgical carts, surgical robots, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), large-screen TVs, etc. The embodiments of this application do not impose any restrictions on the specific type of terminal device and server.

[0052] Please see Figure 2 This application illustrates a method for indicating the position of a trolley array according to an embodiment of the present application, including:

[0053] 201. Check whether the trolley array and optical navigation equipment are located on the same side of the robotic arm's base;

[0054] It should be understood that the implementing entity of this method embodiment can be either the surgical cart itself or a third-party terminal device other than the surgical cart and optical navigation device. Both the cart array and the robotic arm are mounted on the surgical cart, with the robotic arm mounted on a base. The surgical cart and the optical navigation device are typically placed in different locations within the same geographical space. For example, in an operating room setting, the optical navigation device is generally placed at the head of the operating bed, while the surgical cart is generally placed on the left or right side of the operating bed.

[0055] First, it is determined whether the trolley array and the optical navigation device are located on the same side of the robotic arm's base. That is, using the robotic arm's base as a reference, it is determined whether the trolley array and the optical navigation device are on the same side relative to the base.

[0056] like Figure 3 The diagram shows the orientation of the trolley array and the optical navigation equipment on the same side of the robotic arm base. Figure 3 The diagram on the left shows the surgical cart positioned to the left of the bed. Figure 3 The diagram on the right shows the surgical trolley positioned to the right of the bed. The trolley array and robotic arm are mounted on the surgical trolley, while the optical navigation system is located at the head of the operating bed. Figure 3 As can be seen, since the robotic arm base is located in the middle of the surgical trolley, and the trolley array is placed on the side close to the optical navigation device, that is, the trolley array and the optical navigation device are located on the same side of the robotic arm base, the optical path from the optical navigation device to the trolley array will not be blocked by the robotic arm. Therefore, it can be considered that the installation position of the trolley array is appropriate.

[0057] like Figure 4 The diagram shows the orientation of the trolley array and optical navigation equipment on different sides of the robotic arm base. Figure 4 The diagram on the left shows the surgical cart positioned to the left of the bed. Figure 4 The diagram on the right shows the surgical trolley positioned to the right of the bed. The trolley array and robotic arm are mounted on the surgical trolley, while the optical navigation system is located at the head of the operating bed. Figure 4 It can be seen that, since the robotic arm base is located in the middle of the surgical trolley, and the trolley array is placed on the side away from the optical navigation device, that is, the trolley array and the optical navigation device are located on different sides of the robotic arm base, the optical path from the optical navigation device to the trolley array may be blocked by the robotic arm. Therefore, it can be considered that the installation position of the trolley array is improper and needs to be adjusted.

[0058] In one implementation of this application, it can be determined whether the trolley array and the optical navigation device are located on the same side of the robotic arm base by distance detection. For example, a distance sensor can be set to detect a first distance between the optical navigation device and the trolley array, and a second distance between the optical navigation device and the robotic arm. If the first distance is less than the second distance, it indicates that the trolley array and the optical navigation device are located on the same side of the robotic arm base; otherwise, it indicates that the trolley array and the optical navigation device are located on different sides of the robotic arm base.

[0059] In one implementation of this application, the robotic arm is equipped with an end effector array. Detecting whether the trolley array and the optical navigation device are located on the same side of the robotic arm's base can include:

[0060] (1) Obtain the first spatial coordinates of the trolley array in the coordinate system of the optical navigation device and the second spatial coordinates of the end array in the coordinate system of the optical navigation device;

[0061] (2) Based on the first spatial coordinates and the second spatial coordinates, the third spatial coordinates of the trolley array in the base coordinate system and the fourth spatial coordinates of the optical navigation device in the base coordinate system are obtained;

[0062] (3) Determine whether the trolley array and the optical navigation device are located on the same side of the base based on the third spatial coordinates and the fourth spatial coordinates.

[0063] An optical marker array, called an end effector array, can be set at the end of the robotic arm. In this embodiment, the attitudes of the trolley array and the end effector array can be obtained using the optical markers of the trolley array and the end effector array, respectively. That is, the spatial coordinates of the trolley array in the optical navigation device coordinate system (represented by the first spatial coordinates, which includes the spatial coordinates of each optical marker on the trolley array) and the spatial coordinates of the end effector array in the optical navigation device coordinate system (represented by the second spatial coordinates, which also includes the spatial coordinates of each optical marker on the end effector array) can be obtained using an optical navigation device. After obtaining the first and second spatial coordinates, a transformation link between different spatial coordinate systems can be established using the transformation relationship of spatial coordinate systems, thereby obtaining the spatial coordinates of the trolley array in the base coordinate system (represented by the third spatial coordinates) and the spatial coordinates of the optical navigation device in the base coordinate system (represented by the fourth spatial coordinates). Here, the base coordinate system corresponds to the base of the robotic arm. Since the third and fourth spatial coordinates are both spatial coordinates in the base coordinate system, it is easy to determine whether the trolley array and the optical navigation device are located on the same side of the base based on the third and fourth spatial coordinates.

[0064] Specifically, the established spatial coordinate system may include an optical navigation device coordinate system (corresponding to the optical navigation device), an end effector array coordinate system (corresponding to the end effector array), a trolley array coordinate system (corresponding to the trolley array), a base coordinate system (corresponding to the base of the robotic arm), and a flange coordinate system (corresponding to a flange on the robotic arm). The third spatial coordinate of the trolley array in the base coordinate system and the fourth spatial coordinate of the optical navigation device in the base coordinate system, obtained based on the first and second spatial coordinates, may include:

[0065] (1) Based on the first spatial coordinates and the second spatial coordinates, construct the first coordinate transformation relationship between the optical navigation device coordinate system and the trolley array coordinate system, and construct the second coordinate transformation relationship between the optical navigation device coordinate system and the end array coordinate system;

[0066] (2) Calculate the third coordinate transformation relationship between the base coordinate system and the flange coordinate system using positive kinematics;

[0067] (3) Using the second coordinate transformation relationship, the third coordinate transformation relationship, and the fourth coordinate transformation relationship between the pre-calibrated flange coordinate system and the end array coordinate system, determine the fifth coordinate transformation relationship between the optical navigation device coordinate system and the base coordinate system;

[0068] (4) Based on the first coordinate transformation relationship and the fifth coordinate transformation relationship, the first spatial coordinates are transformed to obtain the third spatial coordinates;

[0069] (5) Determine the fourth spatial coordinates based on the fifth coordinate transformation relationship.

[0070] For example, using the first spatial coordinates and the second spatial coordinates, a coordinate transformation matrix can be constructed from the coordinate system of the optical navigation device to the coordinate system of the trolley array. And constructing the coordinate transformation matrix from the optical navigation device coordinate system to the end array coordinate system. Where OTS represents optical navigation equipment, Trolley Marker represents trolley array, and Robot Marker represents end effector array. Then, the coordinate transformation matrix from the base coordinate system to the flange coordinate system can be calculated using forward kinematics. By using the pre-defined coordinate transformation relationship between the flange coordinate system and the end array coordinate system, a coordinate transformation matrix from the flange coordinate system to the end array coordinate system can be constructed. It should be noted that the flange coordinate system used here can be a pre-calibrated theoretical flange coordinate system shared by a large number of surgical trolleys. Due to individual differences between each surgical trolley, the actual flange coordinate system of a particular trolley may deviate slightly from this theoretical flange coordinate system. However, since this deviation is small, it will not significantly affect the acquisition of the coordinate transformation relationship described above. Furthermore, this avoids the extensive work of calibrating the flange coordinate system for each surgical trolley. Next, using... and By performing calculations, the coordinate transformation matrix from the optical navigation device coordinate system to the base coordinate system can be obtained. After that, using This allows the spatial coordinates in the optical navigation device's coordinate system to be converted to spatial coordinates in the base coordinate system. For example, combining... and By performing coordinate transformation on the first spatial coordinates of the trolley array in the coordinate system of the optical navigation device, the third spatial coordinates of the trolley array in the coordinate system of the base can be obtained. Since the spatial coordinates of the optical navigation device in its own coordinate system are known, obtaining... Then, it can be used The spatial coordinates of the optical navigation device in its own coordinate system are transformed to obtain the fourth spatial coordinates of the optical navigation device in the base coordinate system.

[0071] Furthermore, determining whether the trolley array and the optical navigation device are located on the same side of the base based on the third and fourth spatial coordinates can include:

[0072] (1) Obtain the first y-coordinate in the third spatial coordinate system and the second y-coordinate in the fourth spatial coordinate system; wherein the y-axis direction corresponds to the width direction of the surgical cart;

[0073] (2) If the product of the first y coordinate and the second y coordinate is greater than zero, then the trolley array and the optical navigation device are located on the same side of the base.

[0074] Assuming the base coordinate system is set so that the x-axis corresponds to the length of the surgical cart, the y-axis corresponds to the width of the surgical cart, and the z-axis corresponds to the height of the surgical cart, determining whether the cart array and the optical navigation device are located on the same side of the base based on the third and fourth spatial coordinates only requires comparing the y-coordinates. Specifically, first, obtain the first y-coordinate in the third spatial coordinate system and the second y-coordinate in the fourth spatial coordinate system. If the product of the first and second y-coordinates is greater than zero, it indicates that the cart array and the optical navigation device are on the same side of the base; otherwise, it indicates that the cart array and the optical navigation device are on different sides of the base. For example, assuming the third spatial coordinate is (x... T y T , z T The fourth spatial coordinate is (x) in the base coordinate system. O y O , z O If y T *y O If y > 0, it means the trolley array and the optical navigation device are located on the same side of the base; if y T *y O <0 indicates that the trolley array and the optical navigation device are located on different sides of the base.

[0075] 202. If the trolley array and the optical navigation device are located on different sides of the base, it indicates that the installation position of the trolley array needs to be adjusted.

[0076] If the trolley array and the optical navigation device are detected to be on different sides of the base, the optical path from the optical navigation device to the trolley array may be blocked by the robotic arm. In this case, the installation position of the trolley array can be adjusted in various ways. For example, the information requiring adjustment can be displayed on the screen of the surgical trolley, or the information can be read aloud. The installation position of the trolley array can then be corrected manually or automatically, adjusting it to be on the same side of the base as the optical navigation device. This ensures that the optical path from the optical navigation device to the trolley array is not blocked by the robotic arm, thereby improving the success rate and accuracy of registration calibration.

[0077] In one implementation of this application, after detecting whether the trolley array and the optical navigation device are located on the same side of the robot arm's base, the following may be further included:

[0078] (1) If the trolley array and the optical navigation device are located on the same side of the base, then check whether the position of the trolley array exceeds the preset space range;

[0079] (2) If the position of the trolley array exceeds the preset space range, it will indicate that there is a risk of collision with the trolley array.

[0080] In addition to detecting whether the placement of the trolley array is appropriate, this embodiment can further detect whether there is a collision risk at the location of the trolley array. Generally, the location of the trolley array needs to be set within the range of a virtual collision module, that is, within a preset virtual space. This limits the movement space of the robotic arm and prevents collisions between the robotic arm and the trolley array. For example, a space range of length A (mm), width B (mm), and height C (mm) can be preset. The system detects whether the location of the trolley array exceeds this space range (specifically, it detects whether any boundary point of the trolley array exceeds this space range). If it does, it indicates that there is a collision risk and the installation position of the trolley array needs to be adjusted; otherwise, it indicates that there is no collision risk, meaning the installation position of the trolley array is appropriate.

[0081] Furthermore, detecting whether the position of the trolley array exceeds a preset spatial range may include:

[0082] (1) If the position of the trolley array is on the first side of the base, then detect whether the position of the trolley array exceeds the first spatial range; wherein, the first spatial range is a preset spatial range located on the first side of the base;

[0083] (2) If the position of the trolley array is on the second side of the base, then detect whether the position of the trolley array exceeds the second space range; wherein, the second space range is the preset space range located on the second side of the base.

[0084] The trolley array can be installed on different sides of the base, such as the left, right, front, and rear sides. Obviously, the spatial ranges on different sides are different. If the trolley array is located on the first side of the base, it is checked whether the position of the trolley array exceeds a first spatial range, which is a preset spatial range on the first side of the base. If the trolley array is located on the second side of the base, it is checked whether the position of the trolley array exceeds a second spatial range, which is a preset spatial range on the second side of the base. The first and second sides can be different sides of the left, right, front, or rear sides, respectively.

[0085] Furthermore, detecting whether the position of the cart array exceeds the first spatial range can include:

[0086] (1) Obtain the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system;

[0087] (2) Determine the maximum x-coordinate, maximum y-coordinate, and maximum z-coordinate of each boundary marker point in the spatial coordinates of the base coordinate system;

[0088] (3) If the maximum x-coordinate, maximum y-coordinate or maximum z-coordinate exceeds the first spatial range, then the position of the trolley array is determined to be outside the first spatial range.

[0089] Assume the x-axis of the base coordinate system corresponds to the length of the surgical trolley, the y-axis corresponds to the width of the surgical trolley and points positively to the left, and the z-axis corresponds to the height of the surgical trolley. Taking the trolley array on the left side of the base as an example, the first side is the left side, and the first spatial range is the preset spatial range located on the left side of the base. First, obtain the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system. For example, if the trolley array is a quadrilateral, obtain the spatial coordinates of its four corner points (i.e., boundary marker points) in the base coordinate system, denoted as (x1, y1, z1), (x2, y2, z2), (x3, y3, z3), and (x4, y4, z4), respectively. Then, determine the maximum x-coordinate, maximum y-coordinate, and maximum z-coordinate of each boundary marker point in the base coordinate system. For example, determine the maximum x-coordinate of (x1, x2, x3, x4). Max The maximum y-coordinate in (y1, y2, y3, y4) Max The maximum z-coordinate among (z1, z2, z3, z4) Max If x Max y Max Or z Max If any one of them exceeds the first spatial range, then it can be determined that the position of the trolley array exceeds the first spatial range.

[0090] Furthermore, detecting whether the position of the cart array exceeds the second spatial range can include:

[0091] (1) Obtain the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system;

[0092] (2) Determine the maximum x-coordinate, minimum y-coordinate, and maximum z-coordinate of each boundary marker point in the spatial coordinates of the base coordinate system;

[0093] (3) If the maximum x-coordinate, minimum y-coordinate or maximum z-coordinate exceeds the second spatial range, then the position of the trolley array is determined to be outside the second spatial range.

[0094] Taking the trolley array located on the right side of the base as an example, the second side is the right side, and the second spatial range is the preset spatial range located on the right side of the base. First, obtain the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system. For example, if the trolley array is a quadrilateral, obtain the spatial coordinates of its four corner points (i.e., boundary marker points) in the base coordinate system, denoted as (x1, y1, z1), (x2, y2, z2), (x3, y3, z3), and (x4, y4, z4), respectively. Then, determine the maximum x-coordinate, minimum y-coordinate, and maximum z-coordinate of each boundary marker point in the base coordinate system. For example, determine the maximum x-coordinate x1, minimum y-coordinate, and maximum z-coordinate of (x1, x2, x3, x4). Max The smallest y-coordinate in (y1, y2, y3, y4) Min The maximum z-coordinate among (z1, z2, z3, z4) Max If x Max y Min Or z Max If any one of the values ​​exceeds the second spatial range, it can be determined that the position of the trolley array exceeds the second spatial range. Since the origin of the base coordinate system is set at the base of the robotic arm, and the robotic arm is located in the middle of the surgical trolley, when the trolley array is on the left side of the base, the y-coordinate of its marker point is positive, so the maximum y-coordinate is detected at this time. When the trolley array is on the right side of the base, the y-coordinate of its marker point is negative, so the minimum y-coordinate is detected at this time.

[0095] Furthermore, after detecting whether the position of the trolley array exceeds the preset spatial range, it may also include:

[0096] If the position of the trolley array exceeds the preset space range, the x-axis, y-axis, and z-axis excess distances of the trolley array are detected and displayed; where the x-axis, y-axis, and z-axis excess distances are the distances by which the trolley array exceeds the preset space range in the x-axis, y-axis, and z-axis directions, respectively.

[0097] When the position of the trolley array is detected to exceed the preset spatial range, in addition to indicating a collision risk, the system can also detect and display the distance the trolley array exceeds the preset spatial range in each direction. If it does not exceed the preset spatial range in a certain direction, the corresponding excess distance can be zero or a null value. Assuming that the x-axis of the base coordinate system corresponds to the length direction of the surgical trolley, the y-axis corresponds to the width direction of the surgical trolley, and the z-axis corresponds to the height direction of the surgical trolley, the system can detect the x-axis excess distance of the trolley array beyond the preset spatial range in the x-axis direction, the y-axis excess distance of the trolley array beyond the preset spatial range in the y-axis direction, and the z-axis excess distance of the trolley array beyond the preset spatial range in the z-axis direction, respectively. For example, after obtaining the maximum x-coordinate, maximum y-coordinate (or minimum y-coordinate), and maximum z-coordinate of the boundary marker points of the trolley array using the method described above, the distance by which the maximum x-coordinate exceeds a preset spatial range can be calculated as the x-axis excess distance, the distance by which the maximum y-coordinate (or minimum y-coordinate) exceeds a preset spatial range can be calculated as the y-axis excess distance, and the distance by which the maximum z-coordinate exceeds a preset spatial range can be calculated as the z-axis excess distance. Finally, the x-axis excess distance, y-axis excess distance, and z-axis excess distance of the trolley array can be displayed on the screen of the surgical trolley, which can conveniently guide the user to make appropriate adjustments to the installation position of the trolley array.

[0098] In this embodiment, the base of the robotic arm is used as a position reference to detect whether the trolley array and the optical navigation device are located on the same side of the base. Since the robotic arm of the surgical trolley is mounted on the base, if the trolley array and the optical navigation device are located on the same side of the base, the optical path from the optical navigation device to the trolley array will not be blocked by the robotic arm, and the installation position of the trolley array can be considered appropriate. Conversely, if the trolley array and the optical navigation device are located on different sides of the base, the optical path from the optical navigation device to the trolley array may be blocked by the robotic arm. In this case, it will indicate that the installation position of the trolley array needs to be adjusted. The installation position of the trolley array can then be corrected manually or automatically, thereby improving the success rate and accuracy of the surgical robot's registration and calibration.

[0099] To facilitate understanding of the position indication method for the trolley array provided in the embodiments of this application, the following are some practical application scenarios.

[0100] like Figure 5 The diagram shown is a functional module schematic of a surgical robot provided in an embodiment of this application. Figure 5The diagram illustrates the main functional modules used by the surgical robot before registration and calibration, including a trolley array orientation detection module, a trolley array collision detection module, and an interface display module. The trolley array orientation detection module primarily detects whether the trolley array and the optical navigation device are located on the same side of the robotic arm base. The trolley array collision detection module primarily detects whether there is a collision risk in the trolley array's position. The interface display module primarily displays information such as trolley array position adjustment prompts, whether the trolley array is installed correctly, and the distance the trolley array exceeds the preset spatial range in each direction.

[0101] like Figure 6 As shown, Figure 5 A schematic diagram of the workflow of the trolley array orientation detection module. First, the attitude of the trolley array and the attitude of the end effector array of the robotic arm are captured using an optical navigation device, i.e., the spatial coordinates of the trolley array and the end effector array in the coordinate system of the optical navigation device are obtained. Then, using forward kinematics and the transformation relationships between various spatial coordinate systems, the spatial coordinates of the trolley array and the optical navigation device in the base coordinate system of the robotic arm are calculated, respectively (x...). T y T , z T ) and (x O y O , z O Next, determine y. T *y O The value > 0 indicates whether the condition is true. If it is false, it means the trolley array and the optical navigation device are located on different sides of the base, prompting the user to adjust the installation position of the trolley array. If it is true, it means the trolley array and the optical navigation device are located on the same side of the base, indicating that the installation position of the trolley array is appropriate. Next, the trolley array collision detection module can be loaded. This module can include a left-side collision detection module and a right-side collision detection module. The left-side collision detection module corresponds to a preset space area on the left side of the base, and the right-side collision detection module corresponds to a preset space area on the right side of the base. When y... T *y O If > 0 is true, then y can be further judged. T The value > 0 indicates whether the condition is met. If it is, it means the trolley array is on the left side of the base, and the left-side collision detection module is activated. If it is not, it means the trolley array is on the right side of the base, and the right-side collision detection module is activated. For the working principles of the left-side and right-side collision detection modules, please refer to the following... Figure 7 .

[0102] like Figure 7 As shown, Figure 5A schematic diagram of the collision detection module for the trolley array. First, the spatial coordinates of each boundary marker point of the trolley array in the base coordinate system are obtained, assumed to be (x1, y1, z1), (x2, y2, z2), (x3, y3, z3), and (x4, y4, z4). Then, the maximum x-coordinate x among (x1, x2, x3, x4) is determined. Max The maximum z-coordinate among (z1, z2, z3, z4) Max As for the y-coordinate, first detect the aforementioned y-coordinate. T If the value > 0 is true, then load the left-side collision detection module, which corresponds to obtaining the maximum y-coordinate among (y1, y2, y3, y4). Max If this condition is not met, then load the collision detection module on the right, which corresponds to obtaining the minimum y-coordinate among (y1, y2, y3, y4). Min Then, determine x respectively. Max y Max y Min and z Max Whether it exceeds the boundary of the preset space range; specifically, when the left collision detection module is loaded, it determines x. Max y Max and z Max Whether it exceeds the boundary of the left space range; when the right collision detection module is loaded, determine x. Max y Min and z Max Does it exceed the boundary of the space on the right? Finally, if x is found... Max y Max y Min and z Max If at least one boundary of the trolley array exceeds the preset spatial range, a warning is issued indicating a collision risk. Additionally, the distance the trolley array extends beyond the preset spatial range in each direction can be detected and recorded: x-axis excess distance, y-axis excess distance, and z-axis excess distance.

[0103] For some special surgical scenarios, the installation position of the trolley array may not be within the preset collision detection module's range. In such cases, the position of the collision detection module can be compensated by moving it according to the spatial coordinates of the captured boundary markers of the trolley array. For example, the distance of each spatial marker beyond the boundary of the collision detection module in each direction can be calculated, and these distance values ​​can then be compensated into the rotation and translation matrix of the collision detection module, thereby compensating for the position movement of the collision detection module. Afterward, the new collision detection module obtained after compensation can be loaded to perform collision risk detection on the trolley array.

[0104] like Figure 8 As shown, Figure 5The diagram shows the interface output by the interface display module. Figure 8 The top of the image is a top view 801 of the surgical trolley, and the upper right corner is a top view 802 of the envelope box of the loaded right-side collision detection module. In the top view 802, mark 8021 indicates that the position of the trolley array exceeds the minimum limit of the right-side collision detection module in the y-direction, mark 8022 indicates that the position of the trolley array exceeds the maximum limit of the right-side collision detection module in the x-direction, and mark 8023 indicates that the position of the trolley array exceeds the maximum limit of the right-side collision detection module in the z-direction. Figure 8 Below is a side view of the surgical trolley, where 803 represents the side view of the trolley array and 804 represents the side view of the envelope box of the loaded right-side collision detection module. Figure 8 The lower right corner displays the x-axis, y-axis, and z-axis distances of the trolley array, as well as a message 805 indicating whether the trolley array is properly installed. It can be seen that through... Figure 8 The interface shown allows users to clearly see whether the trolley array is installed correctly, whether there is a risk of collision at its installation location, and the distance that should be adjusted in each direction if there is a risk of collision.

[0105] In summary, the embodiments of this application provide a method for indicating the position of a trolley array, which can indicate whether the installation position of the trolley array is appropriate and whether there is a risk of collision. The installation position of the trolley array can then be corrected manually or automatically, thereby improving the success rate and accuracy of the robotic arm's registration and calibration.

[0106] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0107] The above mainly describes a method for indicating the position of a trolley array. The following will describe a device for indicating the position of a trolley array.

[0108] Please see Figure 9 One embodiment of a position indication device for a trolley array in this application includes:

[0109] The detection module 901 is used to detect whether the trolley array and the optical navigation device are located on the same side of the base of the robotic arm; wherein, both the robotic arm and the trolley array are mounted on the surgical trolley;

[0110] The prompt module 902 is used to prompt that the installation position of the trolley array needs to be adjusted if the trolley array and the optical navigation device are located on different sides of the base.

[0111] In one implementation of this application, the robotic arm is equipped with an end effector array, and the detection module may include:

[0112] The spatial coordinate acquisition unit is used to acquire the first spatial coordinates of the trolley array in the coordinate system of the optical navigation device and the second spatial coordinates of the end array in the coordinate system of the optical navigation device.

[0113] The spatial coordinate calculation unit is used to obtain the third spatial coordinate of the trolley array in the base coordinate system and the fourth spatial coordinate of the optical navigation device in the base coordinate system based on the first spatial coordinate and the second spatial coordinate;

[0114] The orientation determination unit is used to determine whether the trolley array and the optical navigation device are located on the same side of the base based on the third spatial coordinates and the fourth spatial coordinates.

[0115] Furthermore, the orientation determination unit may include:

[0116] The y-coordinate acquisition sub-unit is used to acquire the first y-coordinate in the third spatial coordinate system and the second y-coordinate in the fourth spatial coordinate system; wherein, the y-axis direction corresponds to the width direction of the surgical cart.

[0117] The orientation determination subunit is used to determine that the trolley array and the optical navigation device are located on the same side of the base if the product of the first y-coordinate and the second y-coordinate is greater than zero.

[0118] In one implementation of this application, the position indication device for the trolley array may further include:

[0119] The collision detection module is used to detect whether the position of the trolley array exceeds the preset space range if the trolley array and the optical navigation device are located on the same side of the base.

[0120] The collision warning module is used to warn that the trolley array is at risk of collision if its position exceeds a preset spatial range.

[0121] Furthermore, the collision detection module may include:

[0122] The first collision detection unit is used to detect whether the position of the trolley array exceeds the first spatial range if the position of the trolley array is on the first side of the base; wherein, the first spatial range is a preset spatial range located on the first side of the base;

[0123] The second collision detection unit is used to detect whether the position of the trolley array exceeds the second spatial range if the position of the trolley array is on the second side of the base; wherein the second spatial range is a preset spatial range located on the second side of the base.

[0124] Furthermore, the first collision detection unit may include:

[0125] The first spatial coordinate acquisition sub-unit is used to acquire the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system;

[0126] The first maximum coordinate determination sub-unit is used to determine the maximum x-coordinate, maximum y-coordinate, and maximum z-coordinate of each boundary marker point in the spatial coordinates of the base coordinate system.

[0127] The first collision determination sub-unit is used to determine that the position of the trolley array exceeds the first spatial range if the maximum x-coordinate, maximum y-coordinate, or maximum z-coordinate exceeds the first spatial range.

[0128] Furthermore, the second collision detection unit may include:

[0129] The second spatial coordinate acquisition sub-unit is used to acquire the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system;

[0130] The second maximum coordinate determination sub-unit is used to determine the maximum x-coordinate, minimum y-coordinate, and maximum z-coordinate of each boundary marker point in the spatial coordinates of the base coordinate system.

[0131] The second collision determination sub-unit is used to determine that the position of the trolley array exceeds the second spatial range if the maximum x-coordinate, minimum y-coordinate, or maximum z-coordinate exceeds the second spatial range.

[0132] Furthermore, the position indication device for the trolley array may also include:

[0133] The distance display module is used to detect and display the x-axis distance, y-axis distance, and z-axis distance of the trolley array if the position of the trolley array exceeds the preset spatial range; wherein the x-axis distance, y-axis distance, and z-axis distance are the distances by which the trolley array exceeds the preset spatial range in the x-axis direction, y-axis direction, and z-axis direction, respectively.

[0134] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a position indication method for a trolley array as described in any of the above embodiments.

[0135] This application also provides a computer program product that, when run on a terminal device, causes the terminal device to execute the position prompting method for the trolley array as described in any of the above embodiments.

[0136] Figure 10 This is a schematic diagram of a terminal device provided in an embodiment of this application. For example... Figure 10As shown, the terminal device 10 of this embodiment includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and executable on the processor 100. When the processor 100 executes the computer program 102, it implements the steps in the embodiments of the various trolley array position indication methods described above, for example... Figure 2 Steps 201 to 202 are shown. Alternatively, when the processor 100 executes the computer program 102, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 9 The functions of modules 901 to 902 are shown.

[0137] The computer program 102 can be divided into one or more modules / units, which are stored in the memory 101 and executed by the processor 100 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 102 in the terminal device 10.

[0138] The processor 100 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0139] The memory 101 can be an internal storage unit of the terminal device 10, such as a hard disk or memory of the terminal device 10. The memory 101 can also be an external storage device of the terminal device 10, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 10. Furthermore, the memory 101 can include both internal and external storage units of the terminal device 10. The memory 101 is used to store the computer program and other programs and data required by the terminal device. The memory 101 can also be used to temporarily store data that has been output or will be output.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0141] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0142] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0143] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0146] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0148] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for indicating the position of a trolley array, characterized in that, include: The detection trolley array and the optical navigation device are located on the same side of the base of the robotic arm; wherein the robotic arm and the trolley array are both mounted on a surgical trolley; If the trolley array and the optical navigation device are located on different sides of the base, it indicates that the installation position of the trolley array needs to be adjusted. The robotic arm is equipped with an end effector array; the detection of whether the trolley array and the optical navigation device are located on the same side of the robotic arm's base includes: Obtain the first spatial coordinates of the trolley array in the coordinate system of the optical navigation device and the second spatial coordinates of the end array in the coordinate system of the optical navigation device; Based on the first spatial coordinates and the second spatial coordinates, the third spatial coordinates of the trolley array in the base coordinate system and the fourth spatial coordinates of the optical navigation device in the base coordinate system are obtained; Based on the third spatial coordinates and the fourth spatial coordinates, determine whether the trolley array and the optical navigation device are located on the same side of the base.

2. The method as described in claim 1, characterized in that, Determining whether the trolley array and the optical navigation device are located on the same side of the base based on the third spatial coordinates and the fourth spatial coordinates includes: Obtain the first y-coordinate in the third spatial coordinate system and the second y-coordinate in the fourth spatial coordinate system; wherein the y-axis direction corresponds to the width direction of the surgical cart. If the product of the first y-coordinate and the second y-coordinate is greater than zero, then the trolley array and the optical navigation device are determined to be located on the same side of the base.

3. The method as described in claim 1 or 2, characterized in that, After determining whether the detection cart array and the optical navigation device are located on the same side of the robotic arm's base, the method further includes: If the trolley array and the optical navigation device are located on the same side of the base, then it is detected whether the position of the trolley array exceeds the preset spatial range; If the position of the trolley array exceeds the preset space range, a collision risk is indicated.

4. The method as described in claim 3, characterized in that, The detection of whether the position of the trolley array exceeds the preset spatial range includes: If the position of the trolley array is on the first side of the base, then it is detected whether the position of the trolley array exceeds the first spatial range; wherein, the first spatial range is a preset spatial range located on the first side of the base; If the position of the trolley array is on the second side of the base, then it is detected whether the position of the trolley array exceeds the second spatial range; wherein, the second spatial range is a preset spatial range located on the second side of the base.

5. The method as described in claim 4, characterized in that, The detection of whether the position of the trolley array exceeds the first spatial range includes: Obtain the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system; Determine the maximum x-coordinate, maximum y-coordinate, and maximum z-coordinate of each boundary marker point in the spatial coordinates of the base coordinate system; If the maximum x-coordinate, the maximum y-coordinate, or the maximum z-coordinate exceeds the first spatial range, then the position of the trolley array is determined to be outside the first spatial range.

6. The method as described in claim 4, characterized in that, The detection of whether the position of the trolley array exceeds the second spatial range includes: Obtain the spatial coordinates of each boundary marker point on the trolley array in the base coordinate system; Determine the maximum x-coordinate, minimum y-coordinate, and maximum z-coordinate of each boundary marker point in the spatial coordinates of the base coordinate system; If the maximum x-coordinate, the minimum y-coordinate, or the maximum z-coordinate exceeds the second spatial range, then the position of the trolley array is determined to be outside the second spatial range.

7. The method as described in claim 3, characterized in that, After detecting whether the position of the trolley array exceeds a preset spatial range, the method further includes: If the position of the trolley array exceeds the preset space range, the x-axis excess distance, y-axis excess distance, and z-axis excess distance of the trolley array are detected and displayed; wherein, the x-axis excess distance, y-axis excess distance, and z-axis excess distance are the distances by which the trolley array exceeds the preset space range in the x-axis direction, y-axis direction, and z-axis direction, respectively.

8. A position indication device for a trolley array, characterized in that, include: A detection module is used to detect whether the trolley array and the optical navigation device are located on the same side of the base of the robotic arm; wherein, both the robotic arm and the trolley array are mounted on a surgical trolley; The prompting module is used to prompt that the installation position of the trolley array needs to be adjusted if the trolley array and the optical navigation device are located on different sides of the base; The robotic arm is equipped with an end effector array, and the detection module includes: A spatial coordinate acquisition unit is used to acquire the first spatial coordinates of the trolley array in the coordinate system of the optical navigation device and the second spatial coordinates of the end array in the coordinate system of the optical navigation device. A spatial coordinate calculation unit is used to obtain the third spatial coordinates of the trolley array in the base coordinate system and the fourth spatial coordinates of the optical navigation device in the base coordinate system based on the first spatial coordinates and the second spatial coordinates; The orientation determination unit is used to determine whether the trolley array and the optical navigation device are located on the same side of the base based on the third spatial coordinates and the fourth spatial coordinates.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the position prompting method for the trolley array as described in any one of claims 1 to 7.

Images