An experimental device and method for visual measurement of various types of software actuator postures

Abstract

The present invention discloses an experimental device and method for visual measurement of the posture of various types of soft drives, belonging to the field of soft visual measurement. The present invention connects the base and the top plate through a first column and a second column, installs a plane clamping base on the upper surface of the base, installs a side clamping base and a side camera base on the side of the column, and installs a first magnetic clamping plate and a second magnetic clamping plate on the upper surface of the top plate. The visual measurement method is to extract the coordinates of the marker points and the overall contour of the soft drive, and use the coordinate system conversion to realize the screening of the bending contour of the soft drive, thereby realizing the accurate extraction of the bending data of the soft drive. The present invention provides a stable and reliable placement and fixing device for the measurement of the soft drive, and can flexibly adjust the placement position of the soft drive and the camera. The device has a simple structure and is easy to assemble; the visual measurement method is a simple and fast soft drive posture measurement strategy with a small amount of calculation and accurate posture measurement results.

Description

Technical Field

[0001] The present invention designs an experimental device for visual measurement of the posture of a soft actuator, which is suitable for research experiments on the posture measurement of a soft actuator using a monocular or binocular camera. By using the coordinate recognition and contour extraction methods of specific landmark points, the posture and curvature of the soft actuator can be accurately extracted. The present invention mainly belongs to the field of soft visual measurement. Background Art

[0002] With the continuous advancement of science and technology, the field of soft robotics has developed rapidly. As core components of soft robots, the performance and control accuracy of soft actuators play a crucial role in their overall operational effectiveness. Soft actuators are highly flexible, adaptable, and safe, enabling them to perform a variety of tasks in complex and changing environments, such as minimally invasive surgery in the medical field. Therefore, the research and development of soft actuators has important scientific significance and practical application value. Accurate posture measurement is fundamental to achieving precise control of soft actuators. Pose measurement data is also crucial for performance evaluation and optimization of soft actuators. By detecting and analyzing the posture changes of soft actuators under different operating conditions, we can gain a deeper understanding of their kinematic characteristics, mechanical properties, and reliability. However, due to the flexibility and deformability of their materials, traditional rigid object measurement methods are difficult to directly apply. Currently, two common measurement methods for soft actuators are contact and non-contact. Because contact measurement may physically interfere with the soft actuator and alter its performance, non-contact measurement is the most commonly used method. Visual measurement is an efficient, simple, and repeatable non-contact measurement method. The present invention proposes an experimental device and method for visual measurement of the posture of a soft drive. The device can be used for measuring soft drives of different structures. The soft drive is fixed at a specified position by a knob. The placement of the measuring camera and the ambient brightness are adjusted according to changes in the measurement environment to achieve accurate measurement of the posture of the soft drive. The measurement method can accurately screen the curved parts of the soft drive by extracting the coordinates of the landmark points and the overall contour of the soft drive, successfully meeting the research needs for soft drive measurement and having high value in solving the problem of soft drive posture measurement. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to design an experimental device and method for visual measurement of the posture of a soft drive, which is simple and reliable to operate and can be used to conveniently and accurately realize the posture extraction of the soft drive.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] An experimental device for visual measurement of the position and posture of a soft drive comprises a base, a top plate, a first column, a second column, a plane clamping base, a side bottom plate, a side clamping base, a side camera base, a first magnetic clamping plate, a second magnetic clamping plate, a plane clamping knob, a side clamping knob, a soft drive, and a camera.

[0006] The base is the supporting base of the entire device. The first column is repeatedly installed in the threaded holes in the lower left, upper left and upper right corners of the base through bolt connection, and the second column is installed in the threaded hole in the lower right corner of the base through bolt connection.

[0007] The top plate is installed at the top of the entire device and is bolted into the threaded holes on the upper surfaces of the first and second columns. The shorter side of the top plate is parallel to the shorter side of the base, and the longer side of the top plate is parallel to the longer side of the base. This assembly process forms the external support frame of the experimental device.

[0008] The plane clamping base is installed in the corresponding threaded hole on the left upper surface of the base through bolt connection. The plane clamping knob is symmetrically installed in the threaded holes on both sides of the plane clamping base through thread connection.

[0009] The side base plate is bolted to the corresponding threaded holes on the right side surfaces of the first column at the lower left corner and the first column at the upper left corner. The side clamping base is bolted to the corresponding threaded hole in the middle of the side base plate. The side clamping knob is threaded to the threaded hole above the side clamping base.

[0010] The side camera base is installed in the sliding guide rail in the middle of the second column through bolt connection.

[0011] The first magnetic clamp is adsorbed on the left and right sides of the upper surface of the top plate through a magnetic attraction function, and the second magnetic clamp is adsorbed on the upper part of the upper surface of the top plate through a magnetic attraction function.

[0012] The soft drive has two different placement methods. One is to place the soft drive plane on the upper surface of the base, and place the initial section of the soft drive inside the plane clamping base, and by adjusting the tightness of the plane clamping knob, the soft drive is fixed on the two-dimensional plane; the other is to place the soft drive inside the side clamping base, and by adjusting the tightness of the side clamping knob, the soft drive is fixed on the vertical plane.

[0013] The camera has two different placement methods, one is to place the camera in the groove in the middle of the top plate, place the camera to the corresponding position according to the shape of the groove, and determine the best shooting position by adjusting the position of the camera in the groove; the other is to place the camera in the groove on the upper surface of the left half of the side camera base, and determine the best shooting position by adjusting the position of the camera in the groove. Both of the above-mentioned camera placement methods can achieve flexible adjustment of the camera position, and fix the camera position by the camera's own gravity. The curtain can be fixed to the upper surface of the top plate by the first magnetic clamp and the second magnetic clamp. The color, size and position of the curtain can be adjusted according to environmental requirements. The curtain is an additional device of the experimental device, and whether to use it can be determined according to experimental conditions.

[0014] A visual measurement method for extracting the posture of a soft drive comprises the following steps: placing the soft drive, fixing the soft drive, pressurizing the soft drive, placing markers, placing a camera, adjusting the environment, acquiring images, processing images, and extracting data.

[0015] The placement of the soft drive is to place the soft drive at a suitable position in the plane clamping device or the side clamping device according to the measurement requirements; the fixing of the soft drive is to adjust the clamping knob to a suitable position to ensure the fixation of the soft drive and prevent displacement of the soft drive during the subsequent pressurization process.

[0016] The soft actuator pressurization refers to pressurizing the soft actuator to a specified air pressure through a gas pressurizing device, so that the soft actuator generates bending deformation under the action of the air pressure difference.

[0017] The placement of the marking points is to place two circular marking point stickers of the same color and size at the starting point and the ending point of the bending according to the location where the soft drive is bent.

[0018] The camera placement is to place the shooting camera in the appropriate position in the top camera groove or the side camera base groove according to the measurement requirements, ensuring that the camera can completely capture the software driver and the landmark points; the environmental adjustment is to adjust the color, size and placement of the screen according to the environmental conditions to ensure that the camera's shooting effect can meet the experimental requirements.

[0019] The image acquisition refers to the acquisition of images by controlling the camera shooting function through a computer; the image processing refers to the image processing of the specified image by a computer after the image acquisition is completed; the data extraction refers to the extraction of position information such as the coordinates of the landmark points and the bending contour of the software driver after image processing of the acquired image.

[0020] The image processing method of the present invention processes images captured by a monocular / binocular camera and extracts the pose data of a soft drive using Python and the relevant functions of the OpenCV library. Circular markers are attached to the starting and ending points of the soft drive to be tested. By capturing an image of the soft drive with the markers, the center coordinates of the two markers are extracted using RGB and HSV color conversion and setting an appropriate HSV threshold. After the markers are extracted, the overall outer contour of the soft drive is extracted using a flood fill algorithm. The outer contour of the soft drive is filtered based on the coordinates of the markers at the starting and ending points to extract the outer contour of the curved portion. The slope of the line connecting the center coordinates of the two markers is calculated, and the coordinates of the contour points of the curved portion of the soft drive are converted to the horizontal X-axis. The density-based clustering algorithm DBSCAN is used to remove outliers from the data, and a manual noise point extraction function is provided to ensure the accuracy of the extracted data. The curved contour is accurately extracted by projecting perpendicular to the connecting line.

[0021] The present invention has the following beneficial effects compared with the prior art:

[0022] (1) Two placement methods are set up for different states of the soft robot. The plane clamping base can fix the position of the soft driver in the plane direction to avoid vertical displacement, which is suitable for shooting through the camera on the top plate. The plane clamping base can fix the soft driver by adjusting the plane base clamping knobs on both sides, which is suitable for fixing soft drivers of different structures, shapes and sizes; the side clamping base can fix the initial end of the soft driver, which is suitable for shooting through the camera of the side camera base, and the side clamping base can fix the soft driver by adjusting the side clamping knob on the top, which is suitable for fixing soft drivers of different structures, shapes and sizes, and can adjust the insertion depth of the soft driver. The above two placement methods of the soft driver can achieve real-time fixation during the experiment. The plane clamping base and the side clamping base both reserve the position of the air inlet pipe, which can achieve shooting during the gas pressurization test.

[0023] (2) The second column adopts a sliding guide rail design similar to an I-beam structure. By adjusting the position of the fixing bolt on the left side of the side camera base, the side camera base can be infinitely adjusted in the vertical direction, making the camera position adjustment easier and faster. The side camera base groove and the top plate camera groove both adopt an enlarged and lengthened design, which can provide a suitable installation position for most single / binocular cameras on the market and can achieve infinite adjustment of the horizontal position, which can effectively reduce the difficulty of camera position adjustment and realize real-time adjustment of the camera position during shooting.

[0024] (3) The first magnetic clamp and the second magnetic clamp can be used to install a screen on the top surface of the top plate. The use of the screen can adjust factors such as the ambient light of the camera shooting environment. The present invention uses magnetic attraction to fix the clamps, which greatly improves the flexibility and versatility of the clamping function of the clamps. The positions of the first magnetic clamp and the second magnetic clamp can be adjusted arbitrarily according to experimental requirements, and the clamping process of the magnetic clamp will not cause physical damage to the screen. It has the characteristics of simple, convenient and flexible adjustment.

[0025] (4) The soft drive posture measurement method adopted by the present invention only requires pasting two marking points to realize the posture measurement of the soft drive in different states, and the operation is simple; it is applicable to soft drives of various structures and has a wide range of applications; in addition, the number of marking points pasted is small, which greatly reduces the algorithm processing amount of the image processing process, making the measurement process faster.

[0026] In summary, this software actuator position visual measurement test bench not only allows for the fixation and adjustment of the software actuator at different locations, ensuring reliable and stable positioning, but also enables the placement of cameras according to different requirements, completing shooting tasks at different angles. By posting two markers at designated locations, the position and data of the software actuator in different states can be measured and extracted, resulting in accurate measurement results and a simple and quick measurement process. This device has a rigorous structure and is easy to install. Using this device for visual position measurement of software actuators is reliable and stable, simple to operate, and highly practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Further objects, functions and advantages of the present invention will be clarified through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0028] Figure 1 The schematic diagram shows the structure of the experimental device for visual measurement of the soft actuator's posture;

[0029] Figure 2 The schematic diagram of the overall flow chart of the visual processing algorithm in the present invention is shown;

[0030] Figure 3 Schematic diagram showing the principle of obtaining curve contour using the two-point projection method DETAILED DESCRIPTION

[0031] The present invention will be further described in detail below with reference to the accompanying drawings.

[0032] The present invention proposes an experimental device and a measurement method for visual measurement of the posture of a soft drive. The structure of the experimental device is as follows: Figure 1As shown, the device includes a base 1, a top plate 2, a first column 3-1, a second column 3-2, a plane clamping base 4, a side bottom plate 5, a side clamping base 6, a side camera base 7, a first magnetic clamping plate 8-1, a second magnetic clamping plate 8-2, a plane clamping knob 9, a side clamping knob 10, a soft driver 11, and a camera 12;

[0033] Unless otherwise specified, the quantity of the above components is 1 piece;

[0034] Connections between the various parts that make up the device:

[0035] The base plate 1 is the bottom support of the entire experimental device. There are three first columns 3-1, which are respectively installed in the threaded holes corresponding to the lower left corner, upper left corner and upper right corner of the base plate 1. The second column 3-2 is installed in the threaded hole in the lower right corner of the base plate 1.

[0036] The top plate 2 has the same outer contour shape as the bottom plate 1 and is installed in the threaded holes on the upper surfaces of the first column 3-1 and the second column 3-2;

[0037] The plane clamping base 4 is installed in the corresponding threaded hole on the left side of the base 1;

[0038] The side bottom plate 5 is installed in the threaded holes on the right side surface of the first column 3-1 at the lower left corner and the upper left corner of the base 1;

[0039] The side clamping base 6 is installed in the corresponding threaded hole on the right side surface of the side bottom plate 5;

[0040] The side camera base 7 is installed on the slide rail surface of the front surface of the second column 3-2 and is connected by bolts and nuts. The position of the side camera base 7 can be infinitely adjusted within a limited range and can be fixed at a specified position.

[0041] There are two first magnetic clamps 8-1, which are placed on the left and right sides of the upper surface of the top plate 2; the second magnetic clamp 8-2 is placed on the upper side of the upper surface of the top plate 2; the first magnetic clamp 8-1 and the second magnetic clamp 8-2 are adsorbed on the upper surface of the top plate 2, and there is no fixed position. The positions of the first magnetic clamp 8-1 and the second magnetic clamp 8-2 can be flexibly adjusted according to needs;

[0042] There are two plane clamping knobs 9, which are installed in the corresponding threaded holes on the left and right sides of the plane clamping base 4; the side clamping knobs 10 are installed in the corresponding threaded holes on the upper surface of the side clamping base 6; the plane clamping knobs 9 and the side clamping knobs 10 are adjustable parts and can be flexibly adjusted according to the size and position of the soft driver 11 to ensure that the position of the soft driver 11 is fixed;

[0043] The soft driver 11 is a part of the overall experimental device to be measured. It has no fixed shape or size. Depending on the experimental requirements, the soft driver 11 can be placed in the flat clamping base 4 or the side clamping base 6.

[0044] The camera 12 is a tool component for collecting images and has no fixed shape and size. According to experimental needs, the camera 12 can be placed in a corresponding groove of the top plate 2 or the side camera base 7;

[0045] The software driver posture visual measurement algorithm proposed in the present invention includes the following steps:

[0046] Use the camera to capture images including the software driver and landmarks. If the camera type is a monocular camera, the camera image is recorded as P0. If the camera type is a binocular camera, the left and right camera images are recorded as P1 and P2 respectively.

[0047] The image processing algorithm in the present invention processes the above-mentioned P0, P1 and P2 separately;

[0048] First, obtain the image information to be processed. If the camera type is a monocular camera, directly process the camera image P0; if the camera type is a binocular camera, split the camera image into P1 and P2, and process them separately. In the following description, P0, P1, and P2 are collectively referred to as P.

[0049] The image P is filtered using Gaussian filtering and mean filtering algorithms to effectively reduce noise in the image P, eliminate excessively fine details in the image P, make the image P smoother, and improve the readability of the image and the accuracy of subsequent analysis;

[0050] Convert the image P from RGB color space to HSV color space using the cv2.cvtColor color space conversion algorithm; set the upper and lower thresholds of H, S, and V for image screening in HSV color space according to the color and brightness of the marker points;

[0051] The image part within the threshold range is extracted using the cv2.inRange threshold extraction algorithm to obtain the specific positions of the two landmarks.

[0052] Remove noise from image P through cv.morphologyEx morphological algorithm;

[0053] The outer contour edges of the two marker points in the image P are extracted using the cv2.Canny edge detection algorithm; the specific coordinate information of the outer contours of the two marker points is found using the cv2.findContours contour search algorithm, and the center coordinates of the two coordinate points are calculated. The pixel coordinates of the first coordinate point and the second coordinate point are recorded as (X1, Y1) and (X2, Y2) respectively;

[0054] The area occupied by the soft drive in the image P is found by using the cv2.floodFill flood fill algorithm; the outer contour of the soft drive as a whole is extracted by using the cv2.findCoutours contour search algorithm;

[0055] Calculate the relative positions of the marker points (X1, Y1) and (X2, Y2), and note that the angle between the line connecting the two points and the u axis of the horizontal pixel coordinate system is θ, then we have

[0056] The coordinate axes of the pixel coordinate system are ν and u respectively. The coordinate origin is the marker point (X1, Y1), the line connecting the marker points (X1, Y1) and (X2, Y2) is the X-axis, and the vertical direction of the line is the Y-axis. This coordinate system is called the soft body coordinate system.

[0057] The conversion relationship between the pixel coordinate system and the software coordinate system is

[0058]

[0059] The coordinates of the overall contour points of the curved part of the soft driver are transformed as described above. Starting from the starting point of the curved part, the coordinate information of each pixel is traversed with a step size of 1 pixel to obtain the contour projection of the soft driver in a certain direction. The extracted contour point coordinate information is then subjected to a reverse coordinate transformation to obtain the coordinate information in the pixel coordinate system of the original image P.

[0060] The ε neighborhood of the coordinate point and the core point ρ are calculated and the outliers in the extracted data are screened out by the density-based clustering algorithm DBSCAN algorithm;

[0061] where N ε (p)={q∈D|d(p,q)≤ε},N ε (p) is the ε neighborhood of point p, d(p,q) is the distance between point p and point q; where |N ε (p)|≥min_samples,|N ε (p)| is the number of points in the ε-domain of point p, min_samples is the user-defined minimum sample number threshold;

[0062] The cv2.EVENT_LBUTTONDOWN and cv2.ENENT_MOUSEMOVE functions are used to extract the mouse position. Based on the selected mouse position, the data values ​​of the selected area are removed to achieve the function of manually removing noise points. Finally, the coordinate point position information is exported to an external file through the pandas.DataFrame function for subsequent processing.

Claims

1. An experimental device for visual measurement of the position and posture of a soft actuator, characterized by: include: Bottom plate (1), top plate (2), first column (3-1), second column (3-2), plane clamping base (4), side bottom plate (5), side clamping base (6), side camera base (7), first magnetic clamping plate (8-1), second magnetic clamping plate (8-2), plane clamping knob (9), side clamping knob (10), soft driver (11), camera (12); The bottom plate (1) is a bottom support member, and the first columns (3-1) are three in total, which are respectively installed in threaded holes corresponding to the lower left corner, upper left corner and upper right corner of the bottom plate (1); the second column (3-2) is installed in the threaded hole at the lower right corner of the bottom plate (1); The top plate (2) has the same outer contour shape as the bottom plate (1) and is installed in the threaded holes on the upper surfaces of the first column (3-1) and the second column (3-2); The plane clamping base (4) is installed in the corresponding threaded hole on the left side of the base plate (1); The side bottom plate (5) is installed in the threaded holes on the right side surface of the first upright column (3-1) at the lower left corner and the upper left corner of the base (1); The side clamping base (6) is installed in the corresponding threaded hole on the right side surface of the side bottom plate (5); The side camera base (7) is mounted on the slide rail surface of the front side surface of the second column (3-2) and is connected by bolts and nuts to achieve infinite position adjustment of the side camera base (7) within a limited range and can be fixed at a designated position; There are two first magnetic clamps (8-1), which are placed on the left and right sides of the upper surface of the top plate (2); the second magnetic clamp (8-2) is placed on the upper side of the upper surface of the top plate (2); the first magnetic clamp (8-1) and the second magnetic clamp (8-2) are adsorbed on the upper surface of the top plate (2); There are two plane clamping knobs (9), which are installed in the corresponding threaded holes on the left and right sides of the plane clamping base (4); the side clamping knobs (10) are installed in the corresponding threaded holes on the upper surface of the side clamping base (6); the plane clamping knobs (9) and the side clamping knobs (10) are adjustable parts, which can be flexibly adjusted according to the size and position of the soft driver (11) to ensure that the position of the soft driver (11) is fixed; two marking points are provided on the soft driver (11); The soft driver (11) is a part to be measured in the overall experimental device and has no fixed shape or size. According to experimental requirements, the soft driver (11) is placed in the plane clamping base (4) or the side clamping base (6); The camera (12) is a tool component for collecting images and has no fixed shape and size. According to experimental requirements, the camera (12) is placed in a corresponding groove of the top plate (2) or the side camera base (7).

2. The experimental device for visual measurement of the position and posture of a soft actuator according to claim 1, characterized in that: The invention comprises two soft drive placement devices with different functions, namely a plane clamping base (4) and a side clamping base (6), which are suitable for placing and fixing soft drives in different postures; and comprises two camera placement devices, namely a top plate (2) groove and a side camera base (7), and the side camera base (7) can be steplessly adjusted in position on a second column (3-2) according to needs.

3. A visual measurement method for implementing the experimental device for visual measurement of the position and posture of a soft drive as claimed in claim 1, characterized in that: The camera (12) according to claim 1 is used to capture a posture state image P of the soft drive, and the image P is subjected to the following image processing steps to obtain the curvature and curvature profile of the soft drive; The image P is filtered using Gaussian filtering and mean filtering algorithms to effectively reduce noise in the image P, eliminate excessively fine details in the image P, make the image P smoother, and improve the readability of the image and the accuracy of subsequent analysis; The software driver and environment image in image P are converted from RGB color space to HSV color space through the cv2.cvtColor color space conversion algorithm; the upper and lower thresholds of H, S, and V for image screening in HSV color space are set according to the color and brightness of the marker points; The image portion within the threshold range is extracted by using the cv2.inRange threshold extraction algorithm to obtain the location information of the two marking points posted on the soft drive (11) in claim 1; The outer contour edges of the two marker points are extracted by the cv2.Canny edge detection algorithm; the specific coordinate information of the outer contours of the two marker points is obtained by the cv2.findContours contour search algorithm, and the center coordinates of the two marker points are calculated; The area occupied by the soft driver (11) in claim 1 in the image P is found by using the cv2.floodFill flood filling algorithm; the outer contour of the entire soft driver (11) is extracted by using the cv2.findCoutours contour finding algorithm; Calculate the relative positions of the marker points (x1, y1) and (x2, y2), and let the angle between the line connecting the two marker points and the u axis of the horizontal pixel coordinate system be θ, then we have Wherein, x1 and y1 are the X coordinate and Y coordinate of the first marker point in the pixel coordinate system, x2 and y2 are the X coordinate and Y coordinate of the second marker point in the pixel coordinate system, sinθ is the sine value of the angle between the line connecting the two marker points and the horizontal pixel coordinate axis, and cosθ is the cosine value of the angle between the line connecting the two marker points and the horizontal pixel coordinate axis; Let the horizontal and vertical coordinates of a point in the pixel coordinate system be u and ν respectively, and let the horizontal and vertical coordinates of the point after rotation be x2 and y2 respectively. Then the conversion relationship between the two coordinate systems before and after the conversion is The coordinates of the overall contour points of the curved part of the soft driver are subjected to the above-mentioned coordinate transformation. Starting from the first landmark point, the coordinate information of each pixel is traversed with a step size of 1 pixel to obtain the contour projection of the soft driver in a certain direction. The extracted contour point coordinate information is subjected to the reverse coordinate transformation again to obtain the coordinate information in the pixel coordinate system of the original image P. The ε neighborhood of the coordinate point and the core point ρ are calculated and the outliers in the extracted data are screened out by the density-based clustering algorithm DBSCAN algorithm; where N ε (p)={q∈D|d(p,q)≤ε} where N ε (p) is the ε neighborhood of point p, d(p,q) is the distance between point p and point q; where |N ε (p)|≥min_samples,|N ε (p)| is the number of points in the ε-domain of point p, min_samples is the user-defined minimum sample number threshold; The cv2.EVENT_LBUTTONDOWN and cv2.ENENT_MOUSEMOVE functions are used to extract the mouse position. According to the position selected by the mouse, the data value of the selected area is eliminated to realize the function of manually removing noise points. Finally, the coordinate point position information is exported to an external file through the pandas.DataFrame function.

4. The visual measurement method according to claim 2, wherein: The visual measurement method has a low requirement for the number of landmarks; at least two landmarks are needed to extract the curved contour of the soft drive. The image processing algorithm uses landmark extraction, coordinate system transformation, and unidirectional projection to screen the contour of the soft drive. The curved contour extraction method also includes the functions of outlier removal and manual noise removal. It is suitable for the pose measurement of soft actuators with most different structural shapes.

Images