Method and device for classifying liquid viscosity in a container based on multi-dimensional force information perception

Abstract

A method and device for classifying the viscosity of liquid in a container based on multi-dimensional force information perception. Among them, the hardware part includes a Universal Robot manipulator, and its end effector is equipped with a two-finger gripper, and a six-dimensional force sensor is installed at the end of the manipulator; the manipulator is controlled by the PC side to hold and shake the container filled with the liquid to be measured. During the shaking, the data of the six-dimensional force sensor is collected through ROS under Ubuntu, and then through operations such as digital filtering, feature extraction, and support vector machine classification on the collected time series signals, the viscosity of the liquid to be measured in the container is obtained. The present invention can achieve classifying liquids using a robot without contacting the liquid, providing good front-end perception for subsequent robot pouring operations and effectively classifying liquids that are difficult to distinguish visually.

Description

Technical Field

[0001] The present invention relates to the technical field of liquid viscosity detection, and particularly relates to a method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception. Background Art

[0002] As an important property determining the physical behavior of a liquid, for humans, the approximate range of the liquid viscosity can be actively perceived by using visual and wrist force sense information in cooperation with actions such as shaking and stirring, and the viscosity can be easily obtained. However, with the development of information technology and the automation industry, more and more service robots appear in life, and the same problem will also occur on robots. Therefore, the research on the perception of liquid viscosity is particularly important.

[0003] Using a viscometer to measure viscosity is a traditional detection method, such as: capillary method, rotation method, falling ball method, vibration method. Although the values measured by this method are the most accurate, because its equipment is expensive and it needs to directly contact the liquid to be measured, it belongs to invasive detection, which is not suitable for viscosity detection in daily life scenarios. In addition, due to the complexity of the detection equipment, it often requires specialized researchers to perform detection operations, and the learning cost is relatively high. Christof et al. based on Kinect

[0004] The visual sensor constructs three-dimensional motion information when the liquid shakes, and uses the three-dimensional point cloud data to train a neural network classifier, which has achieved good results on containers of various shapes, but a large amount of data sets are still needed to generalize to unknown liquids, and the viscosity detection based on the visual sensor often has high requirements for the environment and equipment. Summary of the Invention

[0005] To solve the deficiencies of the background art, the present invention proposes a non-invasive liquid viscosity detection method mounted on a robot, that is, a method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception. The specific scheme is as follows:

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception, comprising the following steps,

[0008] Based on a motion control module, characterized in that the motion control module includes a Universal Robot six-degree-of-freedom robotic arm, a Robotiq gripper, a six-axis force sensor, and a container filled with the liquid to be measured;

[0009] The six-axis force sensor is installed at the end of the robotic arm, and the Robotiq gripper is connected to the six-axis force sensor. When the viscosity classification of the liquid to be measured is required, the gripper is controlled to hold the container;

[0010] The six - dimensional force sensor is a sensor that can simultaneously measure the force and torque components in three directions in space, namely Fx, Fy, Fz and the torques Tx, Ty, Tz on these three force components;

[0011] The data acquisition stage includes the following steps:

[0012] S1. Use the ROS operating system built on Ubuntu to implement the motion control of the robotic arm, control the robotic arm to clamp the container filled with the liquid to be measured and make it in a horizontal state, and enter the motion control preparation stage;

[0013] S2. Control the robotic arm to shake the liquid horizontally along the horizontal direction parallel to the ground, make several horizontal back - and - forth movements, generate an oscillation excitation for the liquid in the container, and then keep the robotic arm stationary and enter the data acquisition preparation stage;

[0014] S3. After the robotic arm stops moving, use the rosbag data recording functional package provided by ROS, and obtain the oscillation information generated by the liquid in the container by collecting the data of the six - dimensional force sensor at the end of the robotic arm, thus completing the data acquisition stage;

[0015] S4. Replace the liquid with different viscosities in the container, and at the same time set three different filling levels for each liquid. Repeat steps S1 - S3, and control the robotic arm to give the same shaking excitation to obtain the oscillation information of different liquids.

[0016] On the other hand, the present invention also discloses a computer - readable device storing a computer program. When the computer program is executed by a processor, the processor is caused to execute the steps of the above - mentioned method.

[0017] In summary, a method for classifying the viscosity of liquid in a container based on multi - dimensional force information perception of the present invention, wherein the hardware part includes a Universal Robot robotic arm (the end - effector is equipped with a two - finger gripper) and a six - dimensional force sensor (installed at the end of the robotic arm); the robotic arm is controlled by a PC to clamp and shake the container filled with the liquid to be measured. During the shaking, the data of the six - dimensional force sensor is collected through ROS under Ubuntu, and then through operations such as digital filtering, feature extraction, and support vector machine classification on the collected time - series signals, the viscosity of the liquid to be measured in the container is obtained. The present invention can achieve classifying liquids using a robot without contacting the liquid, providing good front - end perception for subsequent robot pouring operations and effectively classifying liquids that are difficult to distinguish visually.

[0018] The method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception of the present invention controls the manipulator to generate disturbances to the liquid in the container, uses a six-dimensional force sensor to collect the damping signal of the liquid sloshing, extracts key information from the oscillation signal using catch22 after S-G filtering, and at the same time uses the NCA technology to effectively improve the classification efficiency of the model. It realizes the discrimination of the liquid viscosity using a non-invasive detection method, effectively avoids the pollution of the liquid by the traditional detection method, and is deployed on the robot in a more convenient way, which is beneficial to making up for the deficiency of the robot in visual perception, enabling the robot to better perceive liquids in different scenarios such as kitchens and food factories in the future, and thus serving people more intelligently. Description of the Drawings

[0019] Figure 1 is the flowchart of the viscosity classification method in the embodiment of the present invention;

[0020] Figure 2 is the side view of the viscosity classification system in the embodiment of the present invention;

[0021] Figure 3 is the front view of the viscosity classification system in the embodiment of the present invention. Detailed Embodiments

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0023] According to Figure 1 as shown in the flowchart, in the embodiment of the present invention, the manipulator is first controlled for horizontal movement through the Moveit motion control package software in the viscosity classification ROS, and it is ensured that the same motion excitation is used when collecting liquid sloshing data subsequently. For example, the manipulator is uniformly controlled to horizontally shake the container 2 times.

[0024] After the manipulator stops moving, quickly enter the data collection stage to ensure that the sloshing information of the liquid is collected to the maximum extent, and use ROSbag to save 2 s of six-dimensional force sensor data, and its data includes six-channel data of the sensor: Fx, Fy, Fz, Tx, Ty, Tz.

[0025] The following is a specific description.

[0026] The hardware structure of its viscosity classification ROS includes: a Universal Robot six-degree-of-freedom manipulator 1, a Robotiq gripper 2, a six-dimensional force sensor 3, and a container 4 filled with the liquid to be measured.

[0027] The six-axis force sensor 3 is installed at the end of the robotic arm 1, and the Robotiq gripper 2 is connected to the six-axis force sensor 3. When viscosity classification of the liquid to be measured is required, the gripper is controlled to hold the container.

[0028] The described six-axis force sensor is a sensor that can simultaneously measure force and torque components in three directions in space, namely Fx, Fy, Fz and the torques Tx, Ty, Tz on these three force components.

[0029] Specifically, for a method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception in an embodiment of the present invention, the data acquisition stage includes the following steps:

[0030] S1. Use the ROS operating system built under Ubuntu to achieve motion control of the robotic arm. Specifically, control the robotic arm to pick up the container filled with the liquid to be measured and make it in a horizontal state, entering the motion control preparation stage.

[0031] S2. Control the robotic arm to shake the liquid horizontally parallel to the ground, make several horizontal back-and-forth movements, generate sufficient oscillation excitation for the liquid in the container, and then keep the robotic arm stationary, entering the data acquisition preparation stage.

[0032] S3. After the robotic arm stops moving, use the rosbag data recording function package provided by ROS to obtain the oscillation information generated by the liquid in the container by collecting the data of the six-axis force sensor at the end of the robotic arm, completing the data acquisition stage.

[0033] S4. Replace the liquid with different viscosities in the container, and at the same time set three different filling levels for each liquid. Repeat steps S1 - S3, and control the robotic arm to give the same shaking excitation to obtain the oscillation information of different liquids.

[0034] Among them, the principle of the described liquid oscillation information is as follows: Liquid viscosity represents a kind of flow resistance inside the liquid. When the fluid is flowing, there is a frictional resistance between adjacent fluid layers due to relative motion, and this frictional resistance is called viscous force, and its magnitude is measured by viscosity. Viscosity is related to the type of substance, temperature, and concentration. When the container is shaken horizontally, the motion of the liquid in the container is a damped oscillation, and the oscillation frequency and decay rate of the damped oscillation are directly related to the height and viscosity of the liquid. Through machine learning methods, data filtering, feature extraction, and machine learning classification are performed on the oscillation data collected by the six-axis force sensor, and finally viscosity classification of common liquids in life is achieved.

[0035] The data filtering is achieved by designing a Savitzky-Golay (S-G) filter to smooth the acquired oscillatory time-series information. The most prominent feature of this filter is that it can ensure the trend and width of the signal remain unchanged while filtering out noise. Therefore, in this invention, this filter is applied to the acquired time-series data, which can smooth the data while filtering out the high-frequency noise of the sensor. Compared with traditional methods such as mean filtering and median filtering, S-G filtering can maximize the retention of the trend of the original force signal, which guarantees the effectiveness of the subsequent extracted features.

[0036] The described feature extraction method uses Catch22 proposed by Lubba et al., which are 22 high-performance time-series features. This is a filtered version based on the hctsa feature library, including linear and nonlinear autocorrelation, continuous differences, numerical distribution and outliers, and volatility scaling characteristics. Catch22 is suitable for the dynamic changes commonly encountered in time-series data mining tasks. Although the classification accuracy is reduced by 7% compared to using the complete hctsa feature library, it completes the dimensionality reduction task from 4791 features to 22 features, reducing the computational time required to extract features from the time series by approximately 1000 times, and the computational time scales approximately linearly with the change in the length of the time series, which greatly improves the efficiency of feature extraction. Using this method to extract features from the Fy, Tx, and Tz data on each force data, 22 high-performance features are obtained on all three channels, preparing for the subsequent classification task.

[0037] The described feature screening method uses Nearest Neighbor Component Analysis (NCA), which is a powerful feature selection technique capable of handling very high-dimensional data sets. Its essence is a simple and effective distance measure learning algorithm. The purpose of the NCA algorithm is to learn an adaptive learning method to learn a distance metric that maximizes the accuracy of random nearest neighbor classification, and complete the transformation from high-dimensional to low-dimensional by controlling the dimension of the metric matrix. In this experiment, the NCA algorithm is used to rank the weights of the 22*3 features extracted from the three force channels, and the most representative features are screened out, effectively avoiding the "curse of dimensionality" problem.

[0038] For the described machine learning classification, the traditional and efficient Support Vector Machine (SVM) machine learning method is used to classify the extracted features, and the five-fold cross-validation method is used to verify the accuracy of the model. SVM was proposed by mathematicians Corinna Cortes and Vapnik et al. as early as 1963. It is one of the most classic and popular classification methods in machine learning in recent decades. For pattern recognition problems with small samples, non-linearity, and high dimensions, SVM has strong advantages. The basic principle of this algorithm is: map the input vector to a high-dimensional feature space, and find the optimal separating hyperplane in this space to classify unknown samples. When a suitable mapping function is selected, in the feature space, most linearly inseparable problems can be converted into linearly separable problems by introducing a kernel function. Considering that the Gaussian kernel function has good generalization effect and good performance in various occasions with different dimensions and sample numbers, and it is also the most widely used kernel function at present, the SVM algorithm used in this experiment also uses the Gaussian kernel function as the method for feature space transformation.

[0039] This method is based on the principle of damped oscillation: after the container shakes horizontally, the motion of the liquid in the container is damped oscillation, and the oscillation frequency and decay rate of the damped oscillation are directly related to the height and viscosity of the liquid.

[0040] According to the above principle of damped oscillation, in order to ensure the generalization of the collected shaking data and eliminate the influence of the liquid height on the liquid shaking data, when collecting liquid data, the filling degree of each liquid in the container is divided into 1 / 3, 1 / 2, and 2 / 3, and 100 pieces of data are collected for each filling degree. Therefore, a total of 300 pieces of data are collected for each viscosity liquid.

[0041] Since the collected data contains noise, a second-order Savitzky-Golay (S-G) filter is designed in this method to filter the data.

[0042] In this method, considering that the essence of the liquid shaking data is time series data, the catch22 proposed by Lubba et al. is adopted. This is a set of 22 efficient time series features selected from a filtered version of the hctsa feature library. The special thing is that feature extraction is only performed on the data of the Fy, Tx, and Tz channels for each piece of collected data, and the other three data channels that do not generate obvious shaking information are ignored, totaling 3 * 22 features.

[0043] To eliminate unnecessary or redundant features and explore the use of fewer features to achieve good classification results, even if it means sacrificing some accuracy, the Nearest Neighbor Component Analysis (NCA) method is combined. This is a powerful feature selection technique that can handle very high-dimensional datasets. Finally, the features are sorted according to their contribution degrees, and only 1 / 3 of the features are retained. A total of 22 features are left on the Fy, Tx, and Tz channels.

[0044] Finally, for viscosity classification, the traditional and efficient Support Vector Machine (SVM) machine learning method is used to classify the extracted features, and the accuracy of the model is verified using five-fold cross-validation.

[0045] To explore the classification effect of force perception methods and quantify viscous liquids, the method of this embodiment uses dimethyl silicone oil with different viscosities as experimental objects, namely 20 cst, 50 cst, 100 cst, 350 cst, 500 cst, 1000 cst, and water with a viscosity of 1 cst, a total of 7 liquids. The correct classification rate of the viscosities of the 7 liquids obtained using this method is 93.3%, which is a 3.1% increase in accuracy compared to when the NCA technology was not used.

[0046] At the same time, this method is used to test the classification effect in actual life scenarios, and several common liquids in life, such as water, milk, yogurt, and detergent, are detected. (Among them, there are liquids with similar colors visually, such as milk and yogurt) Using the same settings as in the previous experiment, each liquid is experimented with at filling levels of 1 / 3, 1 / 2, and 2 / 3, and 100 data are collected each time, for a total of 1200 samples. The same feature extraction method is used, and the 22 important features screened in the above experiment are used for model training. Finally, the classification effect for the four common liquids reaches 99.6%, with only a 1.5% error when the viscosity is relatively high.

[0047] This effectively corroborates the effectiveness of classification based on force sensor perception and provides a new perception method to help robots better perceive liquids in different scenarios such as kitchens and food factories in the future.

[0048] In summary, the method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception controls the robotic arm to generate disturbances to the liquid in the container, uses a six-dimensional force sensor to collect the damping signal of the liquid sloshing, extracts key information from the oscillation signal using catch22 after S-G filtering, and effectively improves the classification efficiency of the model using the NCA technology. It realizes the discrimination of the liquid viscosity using a non-invasive detection method, effectively avoids the pollution of the liquid by traditional detection methods, and is deployed on the robot in a more convenient way, which is beneficial to make up for the deficiency of the robot in visual perception, enabling the robot to better perceive liquids in different scenarios such as kitchens and food factories in the future, and thus serving people more intelligently.

[0049] On the other hand, the present invention also discloses a computer-readable storage medium storing a computer program, which when executed by a processor, causes the processor to execute the steps of any of the above methods.

[0050] On yet another hand, the present invention also discloses a computer device including a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, it causes the processor to execute the steps of any of the above methods.

[0051] In another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when running on a computer, causes the computer to execute the steps of any of the above methods in the embodiments.

[0052] It can be understood that the system provided by the embodiments of the present invention corresponds to the method provided by the embodiments of the present invention. For the explanations, examples and beneficial effects of related content, reference can be made to the corresponding parts in the above methods.

[0053] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0054] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception, based on a motion control module, characterized in that, the motion control module includes a Universal Robot six-degree-of-freedom robotic arm, a Robotiq gripper, a six-axis force sensor, and a container filled with the liquid to be measured; the six-axis force sensor is installed at the end of the robotic arm, and the Robotiq gripper is connected to the six-axis force sensor. When the viscosity classification of the liquid to be measured is required, the gripper is controlled to hold the container. After the container is horizontally shaken, the motion of the liquid in the container is damped oscillation, and the oscillation frequency and attenuation rate of the damped oscillation are directly related to the height and viscosity of the liquid. Through machine learning methods, data filtering, feature extraction, feature screening and machine learning classification are performed on the oscillation data collected by the six-axis force sensor, and finally the viscosity classification of common liquids in life is realized; wherein feature extraction is to extract features from the Fy, Tx, and Tz data on each force data; the six-axis force sensor is a sensor that can simultaneously measure the force and torque components in three directions in space, namely Fx, Fy, Fz and the torques Tx, Ty, Tz on these three force components; the data acquisition stage includes the following steps: S1. Use the ROS operating system built under Ubuntu to realize the motion control of the robotic arm, control the robotic arm to pick up the container filled with the liquid to be measured and make it in a horizontal state, and enter the motion control preparation stage; S2. Control the robotic arm to shake the liquid horizontally along the horizontal direction parallel to the ground, make several horizontal round trips, generate oscillation excitation for the liquid in the container, and then keep the robotic arm stationary and enter the data acquisition preparation stage; S3. After the robotic arm stops moving, use the rosbag data recording function package provided by ROS to collect the six-axis force sensor data at the end of the robotic arm to obtain the oscillation information generated by the liquid in the container, and complete the data acquisition stage; S4. Replace the liquid with different viscosities in the container, and at the same time set three different filling levels for each liquid, and repeat steps S1-S3, control the robotic arm to give the same shaking excitation to obtain the oscillation information of different liquids.

2. The method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception according to claim 1, characterized in that, the data filtering uses a Savitzky-Golay filter to smooth the collected oscillation time series information and retain the waveform characteristics of the data to the greatest extent.

3. The method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception according to claim 1, characterized in that, the feature extraction method uses Catch22. Catch22 is a filtered version based on the hctsa feature library, including linear and non-linear autocorrelation, continuous difference, numerical distribution and outliers, and wave scaling characteristics; Catch22 is used to extract features from the Fy, Tx, and Tz data on each force data, and 22 high-performance features are obtained on all three channels to prepare for the subsequent classification task.

4. The method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception according to claim 1, It is characterized in that the feature screening method adopts the Nearest Neighbor Component Analysis (NCA) algorithm, and uses the NCA algorithm to rank the weights of 22 * 3 features extracted from three force channels, and screen out the most representative features.

5. The method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception according to claim 1, It is characterized in that for the machine learning classification, the Support Vector Machine machine learning method is used to classify the extracted features, and the five-fold cross-validation method is used to verify the accuracy of the model.

6. The method for classifying the viscosity of liquid in a container based on multi-dimensional force information perception according to claim 1, It is characterized in that when collecting liquid data, the filling degree of each liquid in the container is divided into 1 / 3, 1 / 2, and 2 / 3, and 100 pieces of data are collected for each filling degree, so a total of 300 pieces of data are collected for each viscosity liquid.

7. A computer-readable device stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 6.

Images