Ground feature data measurement method and device, electronic equipment and storage medium

Abstract

The embodiment of the application provides a kind of ground feature data measurement method, device, electronic equipment and storage medium, belong to robot perception technical field.The method is applied to the robot tactile perception system based on vision, including elastomer, the outer surface of elastomer is in contact with ground, and the inner surface of elastomer is provided with mark array;Method includes: obtaining the motion image of elastomer based on mark array, obtains target image data;Image feature extraction is carried out to target image data, and motion image feature data is obtained;Displacement field construction is carried out based on motion image feature data, and three-dimensional displacement data is obtained;Friction evaluation and elasticity evaluation are carried out to ground based on three-dimensional displacement data, and friction feature data and elasticity feature data are obtained;Characteristic information of ground is generated based on friction feature data and elasticity feature data, and target ground feature data is obtained.The embodiment of the application can obtain rich ground feature data, and improve the measurement accuracy of ground feature data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot perception, and particularly relates to a ground feature data measurement method and device, electronic equipment and a storage medium. BACKGROUND

[0002] A robot tactile perception system can obtain surface features and texture information of an object, and realize more fine motion control. However, the existing robot tactile perception system applied to a foot is limited by factors such as the size of the robot foot, the size of the sensor and the cost of the equipment, and it is difficult for the robot tactile perception system to obtain high-precision foot tactile data in real time, and thus it is difficult to ensure the accuracy of the ground feature data. Therefore, how to improve the accuracy of ground feature data measurement has become a technical problem to be solved. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a ground feature data measurement method, device, electronic equipment and storage medium, which aims to improve the accuracy of ground feature data measurement.

[0004] To achieve the above-mentioned purpose, a first aspect of the embodiments of the present application provides a ground feature data measurement method applied to a robot tactile perception system based on vision, the robot tactile perception system comprising an elastomer, an outer surface of the elastomer being in contact with the ground, and an inner surface of the elastomer being provided with a marker array; the method comprising:

[0005] obtaining a motion image of the elastomer based on the marker array to obtain target image data;

[0006] performing image feature extraction on the target image data to obtain motion image feature data;

[0007] performing displacement field construction based on the motion image feature data to obtain three-dimensional displacement data;

[0008] performing friction evaluation on the ground based on the three-dimensional displacement data to obtain friction feature data;

[0009] performing elasticity evaluation on the ground based on the three-dimensional displacement data to obtain elasticity feature data;

[0010] generating feature information of the ground based on the friction feature data and the elasticity feature data to obtain target ground feature data.

[0011] In some embodiments, the displacement field construction based on the motion image feature data to obtain three-dimensional displacement data comprises:

[0012] The motion image feature data is subjected to landmark contour feature extraction to obtain landmark contour feature data; wherein the landmark contour feature data comprises contour center position data and contour area data;

[0013] The contour position expression data is constructed based on the contour center position data and preset initial contour information data;

[0014] The contour position expression data is subjected to optimization processing to obtain center position optimization data;

[0015] Displacement calculation is performed based on the initial contour information data, the contour area data and the center position optimization data to obtain the three-dimensional displacement data.

[0016] In some embodiments, the contour position expression data is constructed based on the contour center position data and preset initial contour information data, comprising:

[0017] Distance calculation is performed on the initial contour information data and contour center position data to obtain displacement estimation data;

[0018] Matrix construction is performed based on the displacement estimation data to obtain the contour position expression data.

[0019] In some embodiments, the three-dimensional displacement data comprises first direction position data, second direction position data and third direction position data; the ground is subjected to friction evaluation based on the three-dimensional displacement data to obtain friction feature data, comprising:

[0020] Friction force calculation is performed on the first direction position data and the second direction position data based on preset first calibration coefficients to obtain friction force estimation data;

[0021] First pressure calculation is performed on the third direction position data based on preset second calibration coefficients to obtain first positive pressure estimation data;

[0022] Friction coefficient estimation is performed based on the friction force estimation data and the first positive pressure estimation data to obtain the friction feature data.

[0023] In some embodiments, the ground is subjected to elasticity evaluation based on the three-dimensional displacement data to obtain elasticity feature data, comprising:

[0024] Third direction interpolation calculation is performed based on the three-dimensional displacement data to obtain target interpolation data;

[0025] Second pressure calculation is performed on the target interpolation data based on preset third calibration coefficients to obtain second positive pressure estimation data;

[0026] perform contact area estimation based on the target interpolation data to obtain target area estimation data;

[0027] perform radius estimation on the target area estimation data to obtain target radius estimation data;

[0028] perform elasticity estimation based on preset parameters of the elastic body and the target radius estimation data to obtain the elasticity feature data.

[0029] In some embodiments, the target interpolation data is obtained by performing third direction interpolation calculation based on the three-dimensional displacement data, including:

[0030] perform region construction based on the first direction position data and the second direction position data to obtain target restriction region data;

[0031] perform interpolation processing on the third direction position data based on the target restriction region data to obtain the target interpolation data.

[0032] In some embodiments, the motion image feature data is obtained by performing image feature extraction on the target image data, including:

[0033] perform grayscale processing on the target image data to obtain grayscale image data;

[0034] obtain preset grayscale processing threshold data of the robot tactile perception system;

[0035] perform binary conversion on the grayscale image data based on the preset grayscale processing threshold data to obtain the motion image feature data.

[0036] To achieve the above object, a second aspect of the embodiment of the present application proposes a ground feature data estimation device, which comprises:

[0037] a motion image acquisition module configured to acquire a motion image of an elastic body based on a marker array to obtain target image data;

[0038] an image feature extraction module configured to perform image feature extraction on the target image data to obtain motion image feature data;

[0039] a displacement field construction module configured to perform displacement field construction based on the motion image feature data to obtain three-dimensional displacement data;

[0040] a friction evaluation module configured to perform friction evaluation on a ground surface based on the three-dimensional displacement data to obtain friction feature data;

[0041] an elasticity evaluation module configured to perform elasticity evaluation on the ground surface based on the three-dimensional displacement data to obtain elasticity feature data;

[0042] The ground feature data generation module is configured to generate feature information of the ground based on the friction feature data and the elasticity feature data, and obtain target ground feature data.

[0043] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0044] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0045] The ground feature data measurement method, device, electronic device and storage medium provided by the present application are applied to a visual-based robot tactile perception system, which obtains target image data by acquiring a motion image of an elastic body based on a marker array; performs image feature extraction on the target image data to obtain motion image feature data; constructs a displacement field based on the motion image feature data to obtain three-dimensional displacement data; performs friction evaluation on the ground based on the three-dimensional displacement data to obtain friction feature data; performs elasticity evaluation on the ground based on the three-dimensional displacement data to obtain elasticity feature data; and generates feature information of the ground based on the friction feature data and the elasticity feature data to obtain target ground feature data. The three-dimensional displacement data of the inner surface marker array of the elastic body is constructed, and then rich ground feature data is obtained by calculation, thereby improving the measurement accuracy of the ground feature data. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a structural schematic diagram of a visual-based robot tactile perception system provided by the embodiments of the present application;

[0047] Figure 2 is a flowchart of a ground feature data measurement method provided by the embodiments of the present application;

[0048] Figure 3 is a flowchart of step S102 in Figure 2

[0049] Figure 4 is a flowchart of step S103 in Figure 2

[0050] Figure 5 is a flowchart of step S302 in Figure 4

[0051] Figure 6 is a flowchart of step S302 in Figure 2 ​​​the flowchart of step S104 in

[0052] Figure 7 is Figure 2 the flowchart of step S105 in

[0053] Figure 8 is Figure 7 the flowchart of step S601 in

[0054] Figure 9 is a structural schematic diagram of an elastomer preparation device of a visual-based robot tactile perception system provided by an embodiment of the present application;

[0055] Figure 10 is a schematic diagram of three-dimensional position data variation of a marker point provided by an embodiment of the present application;

[0056] Figure 11 is a friction-pressure scatter plot provided by an embodiment of the present application;

[0057] Figure 12 is a vertical direction displacement distribution schematic diagram of a robot tactile perception system provided by an embodiment of the present application;

[0058] Figure 13 is a structural schematic diagram of a ground feature data measurement device provided by an embodiment of the present application;

[0059] Figure 14 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0060] The figure mark: elastomer 100, shell 200, camera 300, light source 400, marker array 110, marker point 120, upper mold pouring opening 500, upper mold 600, lower mold 700, non-contact area 800, contact area 900. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0062] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0064] The robot tactile perception system can obtain surface features and texture information of an object, and realize more fine motion control. The robot tactile perception system applied to the foot can help the robot to better adjust the motion posture in real time and perceive potential dangers that may exist on the ground.

[0065] The robot foot tactile has the characteristics of large sensing range, high real-time requirement and large contact force. The foot tactile sensor based on the electromagnetic properties such as triboelectricity, piezoelectricity, resistance and capacitance is limited by the device size, signal-to-noise ratio of output signal, complex mechanical structure or array density, and it is difficult to realize high-resolution positioning of large space contact force, and is limited by the cost of multi-channel analog signal acquisition device, and it is also difficult to realize real-time acquisition of large-scale array tactile signal, and it is difficult to ensure the accuracy of ground feature data. The tactile sensor based on vision, such as TacTip, can obtain higher-dimensional image data with high real-time performance, thanks to the rapid development of computer vision technology and image acquisition technology. However, TacTip is based on feature point detection and optical flow algorithm when tracking feature points, and the algorithm has high complexity, which limits the time efficiency of the system during operation. At the same time, the internal vision system of the classic TacTip uses a monocular camera, which cannot obtain motion information in the depth direction of the marker point.

[0066] At present, the related technology cannot obtain high-precision robot foot pressure distribution data in real time through monocular vision, and cannot accurately obtain ground feature data. Therefore, how to improve the accuracy of ground feature data measurement has become a technical problem to be solved.

[0067] Based on this, the embodiments of the present application provide a ground feature data measurement method and device, electronic equipment and storage medium, which aims to improve the accuracy of ground feature data measurement.

[0068] The ground feature data measurement method and device, electronic equipment and storage medium provided by the embodiments of the present application are specifically explained by the following embodiments. First, the ground feature data measurement method in the embodiments of the present application is described.

[0069] Figure 1 is a structural schematic diagram of a robot tactile perception system based on vision provided by the embodiments of the present application, Figure 1 (a) is a front view of the robot tactile perception system based on vision, Figure 1 (b) is a sectional view of the robot tactile perception system based on vision, Figure 1(c) is an exploded view of the visual-based robotic tactile perception system.

[0070] The robotic tactile perception system comprises an elastomer 100, a shell 200, a camera 300 and a light source 400.

[0071] The elastomer 100 is an integrally formed elastic shell-shaped object, the material of the elastomer 100 is light-tight material, the outer surface of the elastomer 100 is in contact with the ground and can be deformed under the action of force, the inner surface of the elastomer 100 is provided with a first area, and the first area is provided with a marker array 110.

[0072] The marker array 110 comprises a plurality of marker points 120, and the marker points 120 are embedded in the elastomer 100; the elastic modulus of the material of the marker points 120 is less than the elastic modulus of the material of the elastomer 100, the surface of the marker points 120 is flush with the inner surface of the elastomer 100, and the color of the marker points 120 is different from the color of the elastomer 100.

[0073] The shell 200 is made of opaque hard material, and the shell 200 and the elastic connection form an opaque cavity.

[0074] The camera 300 is installed at the bottom center of the shell 200, the body of the camera 300 is located in the opaque cavity, and the shooting area of the camera 300 is opposite to the first area, which is used to record the movement of the marker array 110.

[0075] The light source 400 is arranged in the opaque cavity to provide light for the first area.

[0076] It should be noted that the material of the elastomer 100 can include but is not limited to silicone, rubber, polydimethylsiloxane, which is not limited in this embodiment.

[0077] It should be noted that the material of the elastomer 100 can also be a material with good light transmission, which needs to be mixed with a dye to reduce the light transmission of the material.

[0078] The elasticity and fatigue strength of the elastomer 100 need to be determined according to the specific application scene, which is related to the stress of the elastomer 100 in the scene.

[0079] The elastic modulus of the material of the marker points 120 should be less than the elastic modulus of the material of the elastomer 100, so that when the elastomer 100 region where the marker points 120 are located is elongated, compressed or deformed, the marker points 120 can also be elongated, compressed or deformed, and the color of the marker points 120 will not change. In addition, the color of the marker points 120 is different from the color of the elastomer 100, thereby distinguishing the color of the marker points 120 and the color of the elastomer 100, and having good distinguishability.

[0080] The shell 200 provides an assembly position for the elastic body 100, the camera 300 and the light source 400, so that the relative position between the elastic body 100, the light source 400, the camera 300 and the shell 200 does not change when the robot tactile perception system is working.

[0081] It should be noted that the shell 200 and the elastic body 100 can be fixed by screws, and can also be fixed by buckles, which are not limited in the embodiment.

[0082] It should be noted that the camera 300 can be a normal color camera, a black and white camera, a depth camera, an infrared camera, an event camera, etc., which are not limited in the embodiment.

[0083] It should be noted that the focal length, field of view, shutter mode, physical size and photosensitive element size of the camera 300 need to be determined according to the specific scene.

[0084] Figure 2 It is an optional flowchart of the ground feature data calculation method provided by the embodiment of the application, and the method is applied to the robot tactile perception system based on vision, Figure 1 The method can include but is not limited to steps S101 to S106.

[0085] Step S101, acquiring a motion image of the elastic body 100 based on the marker array 110 to obtain target image data;

[0086] Step S102, performing image feature extraction on the target image data to obtain motion image feature data;

[0087] Step S103, constructing a displacement field based on the motion image feature data to obtain three-dimensional displacement data;

[0088] Step S104, performing friction evaluation on the ground based on the three-dimensional displacement data to obtain friction feature data;

[0089] Step S105, performing elasticity evaluation on the ground based on the three-dimensional displacement data to obtain elasticity feature data;

[0090] Step S106, generating feature information of the ground based on the friction feature data and the elasticity feature data to obtain target ground feature data.

[0091] The steps S101 to S106 shown in the embodiments of the present application obtain target image data by acquiring a motion image of the marker array 110 on the inner surface of the elastomer 100; perform image feature extraction on the target image data to obtain motion image feature data; construct a displacement field based on the motion image feature data to obtain three-dimensional displacement data; perform friction evaluation of the ground based on the three-dimensional displacement data to obtain friction feature data; perform elasticity evaluation of the ground based on the three-dimensional displacement data to obtain elasticity feature data; generate feature information of the ground based on the friction feature data and the elasticity feature data to obtain target ground feature data. By constructing the three-dimensional displacement data of the marker array 110 on the inner surface of the elastomer 100, and then calculating to obtain rich ground feature data, the calculation accuracy of the ground feature data is improved.

[0092] In step S101 of some embodiments, the camera 300 of the vision-based robot tactile perception system captures a motion image of a first region of the inner surface of the elastomer 100, wherein the motion image includes motion information of the marker points 120 in the marker array 110, which can be used to construct three-dimensional displacement data of the elastomer 100, and then calculate the ground feature data.

[0093] Specifically, the target image data is denoted as I.

[0094] Please refer to Figure 3 In some embodiments, step S102 can include but is not limited to steps S201 to S203:

[0095] Step S201, performing grayscale processing on the target image data to obtain grayscale image data;

[0096] Step S202, obtaining preset grayscale processing threshold data of the robot tactile perception system;

[0097] Step S203, performing binary conversion on the grayscale image data based on the preset grayscale processing threshold data to obtain motion image feature data.

[0098] The steps S201 to S203 shown in the embodiments of the present application perform grayscale processing on the target image data, and then perform binary conversion processing on the grayscale image data by using the preset grayscale processing threshold data, to convert the three-dimensional image data into two-dimensional data, and obtain the motion image feature data, which is convenient for subsequent calculation.

[0099] Performing grayscale processing on each pixel point of the target image data to obtain grayscale image data.

[0100] Specifically, after performing grayscale processing on the target image data I, the grayscale image data I is obtained gi wherein subscript i is the image of the i-th frame of the target image data.

[0101] It should be noted that the preset gray processing threshold data is a preset parameter of the robot tactile perception system, and the preset gray processing threshold data is related to the parameters of the camera 300 and the light source 400.

[0102] In step S203 of some embodiments, if the gray value of the pixel point in the gray image data is greater than or equal to the preset gray processing threshold data, the gray value of the pixel point is set to a first value; if the gray value of the pixel point in the gray image data is less than the preset gray processing threshold data, the gray value of the pixel point is set to a second value; and then the binarization of the gray data is completed.

[0103] Specifically, the motion image feature data is I gi The binarization conversion formula of the motion image feature data is shown in equation (1):

[0104]

[0105] Where N1 is the first value, N2 is the second value, X is the preset gray processing threshold data, and Y is the gray value of the pixel point in the gray data image.

[0106] Since the color of the elastomer 100 is different from the color of the marker point 120, after the gray image data is binarized, the marker point 120 and the elastomer 100 can be quickly distinguished, and a good distinction degree is achieved, which facilitates subsequent identification of the marker point 120 and construction of a three-dimensional displacement field, and realizes reaction of the displacement of the elastomer 100 by the displacement of the marker point 120.

[0107] Please refer to Figure 4 In some embodiments, step S103 can include but is not limited to steps S301 to S304:

[0108] Step S301, marker contour feature extraction is performed on the motion image feature data to obtain marker contour feature data; wherein the marker contour feature data includes contour center position data and contour area data;

[0109] Step S302, based on the contour center position data and the preset initial contour information data, contour position expression data is constructed;

[0110] Step S303, the contour position expression data is optimized to obtain center position optimization data;

[0111] Step S304, based on the initial contour information data, the contour area data and the center position optimization data, displacement calculation is performed to obtain three-dimensional displacement data.

[0112] The steps S301 to S304 shown in the embodiments of the present application are used to extract the marker contour feature data of the marker point 120 in the motion image feature data, and then construct the displacement field according to the marker contour feature data to obtain the three-dimensional displacement data, accurately acquire the motion information of the marker point 120 in the depth direction, and more accurately acquire the motion information, i.e., the displacement condition, of the elastic body 100, thereby improving the accuracy of the acquired ground feature data.

[0113] It should be noted that the marker contour feature extraction can be realized by a Canny edge detection algorithm, a boundary tracking algorithm or other algorithms, and is not limited in the embodiments.

[0114] It should be noted that the contour center position data is two-dimensional data.

[0115] In addition, the marker contour feature data can further include contour perimeter data.

[0116] Specifically, the marker contour feature data is denoted as C i , wherein C i ={c i0 ,c i1 ,…,c ik}(k=1,2,…,K), a total of K contours, the i-th contour in the j-th frame is denoted as c ij , and the center position data of each contour is denoted as (x ij ,y ij ), and the contour area data of each contour is denoted as A ij .

[0117] In addition, the contour perimeter data of each contour is denoted as L ij .

[0118] Please refer to Figure 5 In some embodiments, the step S302 can include but is not limited to steps S401 to S402:

[0119] Step S401: performing distance calculation on the initial contour information data and the contour center position data to obtain displacement estimation data;

[0120] Step S402: performing matrix construction based on the displacement estimation data to obtain contour position expression data.

[0121] The steps S401 to S402 shown in the embodiment of the present application are as follows: the preset initial contour information data is obtained; distance calculation is performed on the initial contour information data and the contour center position data to obtain displacement estimation data; and matrix construction is performed based on the displacement estimation data to obtain contour position expression data, so that the motion information of the marker points 120 in the depth direction is accurately obtained, the motion information, i.e. the displacement, of the elastic body 100 can be more accurately obtained, and the accuracy of the obtained ground feature data is improved.

[0122] It should be noted that the preset initial contour information data can be the contour information data of the marker array 110 on the inner surface of the elastic body 100 in the initial state, the contour information data corresponding to the marker array 110 in the first frame image of the target image data, or the contour information data corresponding to the frame image data at the starting time of a certain action, which is not limited in the embodiment.

[0123] Specifically, the initial contour information data is obtained after the initial image is subjected to grayscale processing, binary conversion and marker point contour feature extraction.

[0124] Specifically, the initial contour information data is denoted as C1, the contour center position data of each contour is (x 1j ,y 1j ), and the contour area data of each contour is A 1j .

[0125] For the marker contour feature data of i≠1, a score matrix G i is created, and the element in the kth row and the lth column of the matrix G i is denoted as g kl , which is the displacement estimation data, and represents the loss function value between the kth contour in the marker contour feature data set C i of the current frame and the lth contour in the initial contour information data C1.

[0126] It should be noted that the loss function value between the kth contour in the marker contour feature data set C i of the current frame and the lth contour in the initial contour information data C1 can be calculated by at least one of the following methods:

[0127] Method one: the distance between the contour center position data of the kth contour in the marker contour feature data set C i of the current frame and the contour center position data of the lth contour in the initial contour information data C1 is calculated.

[0128] In step S402 of some embodiments, the distance between the contour center position data of the kth contour in the marker contour feature data set C iEuclidean distance between the contour center position data of the kth contour in the current frame and the contour center position data of the lth contour in the initial contour information data C1, to obtain displacement estimation data g kl As shown in equation (2):

[0129]

[0130] It should be noted that the distance can also be Manhattan distance, Chebyshev distance, etc., which is not limited in the embodiment.

[0131] Method two: calculate the labeled contour feature data set C of the current frame i The difference between the contour area data of the kth contour in the current frame and the contour area data of the lth contour in the initial contour information data C1, to obtain displacement estimation data g kl As shown in equation (3):

[0132] g kl = |A ik -A 1l | (3)

[0133] Method three: calculate the labeled contour feature data set C of the current frame i The difference between the contour perimeter data of the kth contour in the current frame and the contour perimeter data of the lth contour in the initial contour information data C1, to obtain displacement estimation data g kl As shown in equation (4):

[0134] g kl = |L ik -L 1l | (4)

[0135] Finally, based on the displacement estimation data g kl Obtain contour position expression data G i .

[0136] In step S303 of some embodiments, the contour position expression data is optimized by optimizing the matching algorithm, to obtain the center position optimization data of the contour.

[0137] Specifically, based on the contour position expression data G i Construct the contour matching optimal sequence P, as shown in equation (5):

[0138]

[0139] Since there are K labeled points 120, the matched contours have K. P1, P2, …, P kThe sequence indicates that in the i-th image, the index of a contour detected is P1, the index of the contour in the reference image is 1, if the index is 2, the index in the reference image is P2, and so on.

[0140] The reason for performing the optimal matching is that the position of the marker point 120 changes due to the force, the position in the image changes, and the contour index of the marker point 120 changes.

[0141] Finally, the center position optimization data of each contour in each image is obtained by optimizing the optimal sequence P of the contour matching

[0142] In step S304 of some embodiments, since the elastic deformation of the marker point 120 is small, the area of the marker point 120 changes linearly in different frames of images, so the depth condition of the marker point 120 can be reflected by the area change rate of the marker point 120, and then the depth data of the marker, that is, the data in the Z-axis direction, is obtained.

[0143] Specifically, the three-dimensional displacement data includes first direction position data, second direction position data and third direction position data, wherein the first direction position data is in the X-axis direction, the second direction position data is in the Y-axis direction, and the third direction position data is in the Z-axis direction.

[0144] According to the initial contour information data C1, the contour area data A ij and the center position optimization data Perform displacement calculation to obtain three-dimensional displacement data s ik =[x ik ,y ik ,z ik ](k=1,2,…,K,i=1,2,…,T), as formula (6):

[0145]

[0146] Where t represents the image data of the t-th frame, and T represents the total number of image data frames.

[0147] Please refer to Figure 6 In some embodiments, step S104 can further include but is not limited to steps S501 to S503:

[0148] Step S501, based on the preset first calibration coefficient, the first direction position data and the second direction position data are calculated to obtain the friction force estimation data;

[0149] Step S502, based on the preset second calibration coefficient, the first pressure calculation is performed on the third direction position data to obtain the first positive pressure estimation data;

[0150] Step S503, based on the friction force estimation data and the first positive pressure estimation data, friction coefficient estimation is performed to obtain friction feature data.

[0151] The steps S501 to S503 shown in the embodiments of the present application are as follows: based on the preset first calibration coefficient, the first direction position data and the second direction position data are calculated to obtain the friction force estimation data; based on the preset second calibration coefficient, the third direction position data is calculated to obtain the first positive pressure estimation data; and based on the friction force estimation data and the first positive pressure estimation data, the friction coefficient estimation is performed, so that the friction feature data of the ground is accurately obtained, and the accuracy of obtaining the ground feature data is improved.

[0152] It should be noted that the preset first calibration coefficient is a friction calibration coefficient, which is calibrated by a standard friction force sensor.

[0153] It should be noted that the preset second calibration coefficient is a positive pressure calibration coefficient, which is calibrated by a standard pressure sensor.

[0154] Specifically, the preset first calibration coefficient is denoted as K f , based on the preset first calibration coefficient K f , the first direction position data x ik (k=1, 2, …, K, i=1, 2, …, T) and the second direction position data y ik (k=1, 2, …, K, i=1, 2, …, T) are calculated to obtain the friction force estimation data The specific calculation formula is shown in formula (7):

[0155]

[0156] The preset second calibration coefficient is denoted as K N1 , based on the preset second calibration coefficient K N1 , the third direction position data z ik (k=1, 2, …, K, i=1, 2, …, T) is calculated to obtain the first positive pressure estimation data The specific calculation formula is shown in formula (8):

[0157]

[0158] Based on the friction force estimation data and the first positive pressure estimation data , a scatter point set is constructed. The sliding friction line fitting is performed to obtain the inclination angle θ of the sliding friction line, and finally the friction characteristic data μ is calculated through a trigonometric function, and the specific calculation formula is shown in formula (9):

[0159] μ = arctan (θ) (9)

[0160] It should be noted that the sliding friction line fitting method can be least square method, Gauss Newton method, gradient descent method or other fitting methods, which is not limited in the embodiment.

[0161] Please refer to Figure 7 In some embodiments, step S105 includes but is not limited to steps S601 to S605:

[0162] Step S601, based on the third direction interpolation calculation of the three-dimensional displacement data, the target interpolation data is obtained;

[0163] Step S602, based on the preset third calibration coefficient, the second pressure calculation is performed on the target interpolation data, and the second positive pressure estimation data is obtained;

[0164] Step S603, based on the target interpolation data, the contact area estimation is performed, and the target area estimation data is obtained;

[0165] Step S604, the radius calculation is performed on the target area estimation data, and the target radius calculation data is obtained;

[0166] Step S605, based on the preset parameters of the elastic body 100 and the target radius calculation data, the elasticity calculation is performed, and the elastic characteristic data is obtained.

[0167] The steps S601 to S605 shown in the embodiment of the application are based on the third direction interpolation calculation of the three-dimensional displacement data, the target interpolation data is obtained; based on the preset third calibration coefficient, the second pressure calculation is performed on the target interpolation data, and the second positive pressure estimation data is obtained; based on the target interpolation data, the contact area estimation is performed, and the target area estimation data is obtained; the radius calculation is performed on the target area estimation data, and the target radius calculation data is obtained; based on the preset parameters of the elastic body 100 and the target radius calculation data, the elasticity calculation is performed, so that the elastic characteristic data of the ground is accurately obtained, and the accuracy of obtaining the ground characteristic data is improved.

[0168] Please refer to Figure 8 In some embodiments, step S601 can include but is not limited to steps S701 to S702:

[0169] Step S701, based on the first direction position data and the second direction position data, the region construction is performed, and the target limit region data is obtained;

[0170] Step S702, based on the target limit region data, the third direction position data is interpolated to obtain target interpolation data.

[0171] The steps S701 to S702 shown in the embodiments of the application are that the target limit region data is obtained by constructing a region based on the first direction position data and the second direction position data; and the target interpolation data is obtained by interpolating the third direction position data based on the target limit region data. The data can be smoothed, so as to reduce the data error caused by less data, and further improve the accuracy of obtaining the ground feature data.

[0172] In step S701 of some embodiments, step S701 can include but is not limited to the following steps:

[0173] The maximum value of the first direction position data x ik (k=1, 2, …, K, i=1, 2, …, T) is extracted to obtain the first maximum value data x imax The minimum value of the first direction position data x ik (k=1, 2, …, K, i=1, 2, …, T) is extracted to obtain the first minimum value data x imin ;

[0174] The maximum value of the second direction position data y ik (k=1, 2, …, K, i=1, 2, …, T) is extracted to obtain the second maximum value data y imax The minimum value of the second direction position data y ik (k=1, 2, …, K, i=1, 2, …, T) is extracted to obtain the second minimum value data y imin ;

[0175] The region is constructed based on the first maximum data x imax , the first minimum data x imin , the second maximum data y imax , and the second minimum data y imin , to obtain the target limit region data V i =(x∈[x imax ,x imin ],y∈[y imax ,y imin ]).

[0176] In step S702 of some embodiments, the third direction data z i =(x∈[x imax ,x imin ],y∈[y imax ,y imin ]) is interpolated based on the target limit region data V i .ik (k = 1, 2, …, K, i = 1, 2, …, T) are interpolated according to a preset interpolation quantity Q to obtain target interpolation data z ipq ′(p = 1, 2, …, Q, q = 1, 2, …, Q).

[0177] It should be noted that the method of interpolation processing can be linear interpolation, bilinear interpolation, nearest neighbor interpolation, bicubic interpolation, Lagrange interpolation, spline interpolation or other interpolation methods, and the specific implementation is not limited in the embodiment.

[0178] Specifically, in step S602 of some embodiments, the preset third calibration coefficient is denoted as another positive pressure calibration coefficient, denoted as K N2 , and based on the preset third calibration coefficient K N2 , the target interpolation data z ipq ′(p = 1, 2, …, Q, q = 1, 2, …, Q) is subjected to second pressure calculation to obtain second positive pressure estimation data The specific calculation formula is shown in formula (10):

[0179]

[0180] The maximum value in the target interpolation data z ipq ′(p = 1, 2, …, Q, q = 1, 2, …, Q) is extracted to obtain maximum interpolation data z imax ′, and the maximum interpolation data z imax ′ represents the maximum displacement data in the vertical direction.

[0181] Then, based on the maximum interpolation data z imax ′, the target interpolation data z ipq ′(p = 1, 2, …, Q, q = 1, 2, …, Q) is subjected to contact area estimation to obtain target area estimation data A con .

[0182] In step S603 of some embodiments, the contact area estimation is performed by binary conversion, and the specific calculation process is shown in formulas (11) and (12):

[0183]

[0184]

[0185] The target area estimation data A con is subjected to radius calculation to obtain target radius calculation data a;

[0186] Specifically, the target radius measurement data a is the nominal radius of the contact range of the elastomer 100 with the ground, and the calculation formula is shown in formula (13):

[0187]

[0188] In step S605 of some embodiments, the preset parameter of the elastomer 100 is the radius R of the elastomer 100.

[0189] Based on the preset parameter R of the elastomer 100 and the target radius measurement data a, the elasticity measurement is performed to obtain the elasticity characteristic data E, and the specific calculation formula is shown in formula (14):

[0190]

[0191] In step S106 of some embodiments, finally, the characteristic information of the ground is generated based on the friction characteristic data μ and the elasticity characteristic data E to obtain the target ground characteristic data, and then the functions of environment perception, obstacle detection, balance stability, gait control and object manipulation are realized, so that the robot can adapt to different terrains, avoid obstacles, maintain balance, adjust pace and interact with the environment.

[0192] In one embodiment, a visual-based robot tactile perception system is provided, which includes an elastomer 100, a shell 200, a camera 300 and a light source 400;

[0193] The elastomer 100 is made of one-piece silicone with a Shore hardness of 60 degrees and mixed with black dye, in the shape of a hemispherical shell with a radius of 45 mm and a thickness of 5 mm, manufactured by a preparation device as shown in Figure 9 The upper mold 600 and the lower mold 700 in Figure 9 are combined and installed together in the direction shown in the figure, and then the silicone material mixed with black dye is poured into the upper mold pouring opening 500, and after standing for 2-3 hours at 25°C, the elastomer 100 is obtained. The outer surface of the elastomer 100 contacts the ground and can be deformed under the action of force, and the inner surface of the elastomer 100 is provided with a first area, and the first area is provided with a marker array 110.

[0194] The marker array 110 includes a plurality of marker points 120, and the marker array 110 is a five-layer uniform k-cyclic array, and the marker points 120 are embedded in the elastomer 100, and the marker points 120 are in the form of hemispherical pits with a radius of 1 mm. Among them, the first layer is located at the center position of the first area, the number of marker points 120 is 1, the number of marker points 120 of the second layer is 5, the number of marker points 120 of the third layer is 12, the number of marker points 120 of the fourth layer is 16, and the number of marker points 120 of the fifth layer is 20.

[0195] In the hemispherical pit of the marker point 120, a brush is used to fill in white silicone material mixed with white fuel with a Shore hardness of 20 degrees. The surface of the marker point 120 is flush with the inner surface of the elastomer 100. After standing for 2-3 hours at 25°C, the white silicone is formed and solidified to obtain the white marker point 120. It should be noted that the white silicone of the marker point 120 has good adhesion with the elastomer 100, and will not crack or fall off when the elastomer 100 is stretched and deformed.

[0196] The shell 200 is made of opaque hard material, and the shell 200 is connected with the elastomer by threads to form an opaque cavity together, so as to avoid external light from shining on the sensor inside, ensure good light-proof property of the whole sensor, and ensure normal operation of the subsequent marker point 120 three-dimensional displacement field solving algorithm caused by changes in external ambient light.

[0197] It should be noted that the field of view of the camera 300 can ensure that the marker array 110 is still within the field of view of the camera 300 when the marker array 110 is deformed to the maximum amplitude.

[0198] The camera 300 is a monocular camera, which is installed at the bottom center of the shell 200. The body of the camera 300 is located in the opaque cavity, and the size of the camera 300 is 8mmx8mmx4mm. The camera 300 is connected with the printed circuit board through a wire to realize power supply and data transmission. The field of view angle of the camera 300 is 105°, the highest frame rate is 60FPS, the focal length is 1.78mm, the distance between the camera 300 and the marker point 120 is 20mm, and the shooting area of the camera 300 is directly opposite to the first area. When working, the camera 300 performs real-time image acquisition on the marker array 110 and records the movement of the marker array 110.

[0199] The light source 400 is arranged in the opaque cavity, and the light source 400 is a printed circuit board with a size of 30mmx20mm, which has three white LED lamp beads with a size of 0805 package, a working current of 20mA and a power of 0.1W. The light source 400 provides light for the surface of the elastomer 100 of the first area and the marker point 120 array.

[0200] In one embodiment, a specific ground feature data measurement scene is provided, wherein the number of marker points 120 of the marker array 110 is 55, and the ground feature data measurement method can include but is not limited to the following steps:

[0201] Step 1: The camera 300 of the robot tactile perception system based on vision takes a moving image of the marker array 110 of the first area of the inner surface of the elastomer 100, obtaining target image data I;

[0202] Step 2: The target image data is processed to obtain gray image data.

[0203] The pixel value of each pixel point in the target image data I is calculated, where the pixel value is denoted as (R, G, B), and the gray scale calculation is performed on the pixel value, and the specific calculation formula is shown in formula (15):

[0204]

[0205] The gray value g is obtained by performing gray scale calculation on each pixel value, and the gray value g is combined to obtain the gray image data I gi , where subscript i is the image of the i-th frame of the target image data.

[0206] Step 3: Obtain the preset gray processing threshold data of the robot tactile perception system; wherein the preset gray processing threshold data is 80.

[0207] Based on the preset gray processing threshold data 80, the binary conversion is performed on each pixel point in the gray image data I gi , and the specific calculation formula is shown in formula (16):

[0208]

[0209] The binary data b is obtained by performing binary conversion on each pixel value, and the binary data b is combined to obtain the moving image feature data I bi .

[0210] Step 4: The moving image feature data I bi is subjected to marker point contour feature extraction to obtain marker contour feature data C i , wherein C i ={c i0 ,c i1 ,…,c i55}, specifically, the i-th contour of the j-th frame is denoted as c ij , and the center position data of each contour is denoted as (x ij ,y ij ), and the contour area data of each contour is denoted as A ij .

[0211] Specifically, the findcontours function based on the OpenCV library is used for contour finding.

[0212] Step 5: Obtain initial contour information data C1, contour center position data of each contour is (x 1j ,y 1j ), and contour area data of each contour is A 1j ,

[0213] Step 6: For the labeled contour feature data i≠1, create a score matrix G i , G i ∈R 55 xR 55 , and the element in the kth row and the lth column of the matrix G i is g kl , which is displacement estimation data g kl ;

[0214] Calculate the Euclidean distance between the contour center position data of the kth contour in the current frame of the labeled contour feature data set C i and the contour center position data of the lth contour in the initial contour information data C1, to obtain displacement estimation data g kl , as shown in formula (17):

[0215]

[0216] Obtain contour position expression data G i based on the displacement estimation data g kl .

[0217] Step 7: Construct a contour matching optimal sequence P based on the contour position expression data G i , as shown in formula (18):

[0218]

[0219] Specifically, P1, P2, …, P 55 This order indicates that in the i-th frame image, the index of a contour detected is P1, and the index of this contour in the reference image is 1. If the index is 2, the index in the reference image is P2, and so on.

[0220] Finally, the Hungarian algorithm is used to optimize the contour matching optimal sequence P to obtain the center position optimization data

[0221] Specifically, the three-dimensional displacement data includes first direction position data, second direction position data and third direction position data, wherein the first direction position data is the X-axis direction, the second direction position data is the Y-axis direction, and the third direction position data is the Z-axis direction.

[0222] Step 8: According to the initial contour information data C1, the contour area data A 1j ,

[0213] and the contour matching optimal sequence P, the center position optimization data of each contour in each frame of image is obtained.ij and center position optimization data Displacement calculation is performed to obtain three-dimensional displacement data s ik = [x ik , y ik , z ik ](k = 1, 2, …, 55, i = 1, 2, …, T), as shown in equation (19):

[0223]

[0224] where t represents the image data of the tth frame, and T represents the total number of frames of image data.

[0225] Specifically, referring to Figure 10 , Figure 10 is a schematic diagram of the change of three-dimensional position data of the marker point 120 on the inner surface of the elastomer 100, wherein Figure 10 (a) is the initial position data of the marker point 120, Figure 10 (b) is the three-dimensional displacement data obtained after the marker point 120 moves and is calculated.

[0226] Step 9: The three-dimensional displacement field data s ik = [x ik , y ik , z ik ](k = 1, 2, …, 55, i = 1, 2, …, 500) detected in the 500 frames of target image data,

[0227] The preset first calibration coefficient is the friction calibration coefficient K f , and the preset second calibration coefficient is the first positive pressure calibration coefficient K N1 ,

[0228] Based on the preset first calibration coefficient K f , the first direction position data x ik (k = 1, 2, …, 55, i = 1, 2, …, 500) and the second direction position data y ik (k = 1, 2, …, 55, i = 1, 2, …, 500) are calculated to obtain the friction force estimation coefficient The specific calculation formula is shown in equation (20):

[0229]

[0230] Based on the preset second calibration coefficient K N1 , the third direction position data z ik (k = 1, 2, …, 55, i = 1, 2, …, 500) is calculated to obtain the first positive pressure estimation data The specific calculation formula is shown in formula (21):

[0231]

[0232] Referring to Figure 11 , Figure 11 The friction force estimation data and the first positive pressure estimation data are provided The scatter point set is constructed The scatter point distribution diagram is drawn, and the least square method is used for linear fitting based on the scatter point set

[0233] The inclination angle θ of the sliding friction line is calculated, and finally the friction characteristic data μ is calculated through the trigonometric function. The specific calculation formula is shown in formula (22):

[0234] μ=arctan(θ) (22)

[0235] Step 10: Maximum value extraction is performed on the first direction position data x ik (k=1,2,…,55,i=1,2,…,500) to obtain the first maximum value data x imax Minimum value extraction is performed on the first direction position data x ik (k=1,2,…,55,i=1,2,…,500) to obtain the first minimum value data x imin ;

[0236] Maximum value extraction is performed on the second direction position data y ik (k=1,2,…,55,i=1,2,…,500) to obtain the second maximum value data y imax Minimum value extraction is performed on the second direction position data y ik (k=1,2,…,55,i=1,2,…,500) to obtain the second minimum value data y imin ;

[0237] Based on the first maximum data x imax , the first minimum data x imin , the second maximum data y imax , and the second minimum data y imin , the region construction is performed to obtain the target limit region data V i =(x∈[x imax ,x imin ],y∈[y imax ,y imin ]).

[0238] Based on the target limit region data V i =(x∈[x imax,x imin ],y∈[y imax ,y imin ]) For third-party data z ik (k = 1, 2, ..., 55, i = 1, 2, ..., 500) Perform linear interpolation with 50 interpolation values ​​to obtain the target interpolated data z. ipq ′(p=1,2,…,50, q=1,2,…,50).

[0239] Step 11: The preset third calibration coefficient is denoted as the second positive pressure calibration coefficient, and is recorded as K. N2 Based on the preset third calibration coefficient K N2 For the target interpolation data z ipq The second pressure is calculated using the formula (p = 1, 2, ..., 50, q = 1, 2, ..., 50), yielding the estimated data for the second normal pressure. The specific calculation formula is shown in formula (23):

[0240]

[0241] For the target interpolation data z ipq The maximum value is extracted from (p=1,2,…,50,q=1,2,…,50) to obtain the maximum interpolation data z. imax ′, the maximum interpolation data z imax ′ represents the maximum displacement data in the vertical direction.

[0242] Then, based on the maximum interpolation data z imax For the target interpolation data z ipq The contact area is estimated using the formulas (p = 1, 2, ..., 50, q = 1, 2, ..., 50), resulting in target area estimation data A. con .

[0243] Contact area estimation is performed through binary transformation, and the specific calculation process is shown in formulas (24) and (25):

[0244] A con =∑ p＝1，2，...，50，q=1，2，...，50 f area (z ipq ') (twenty four)

[0245]

[0246] See Figure 12 , Figure 12Fig. 1 is a schematic diagram of vertical displacement distribution of a robot tactile perception system, wherein the non-contact area 800 is the area where the elastic body 100 is not in contact with the ground, and the contact area 900 is the area where the elastic body 100 is in contact with the ground, wherein the area of the contact area 900 is the target area estimation data A con .

[0247] The target area estimation data A con is calculated to obtain the target radius estimation data a; specifically, the target radius estimation data a is the nominal radius of the contact range of the elastic body 100 and the ground, and the calculation formula is shown in formula (26):

[0248]

[0249] The preset parameter of the elastic body 100 is the radius R of the elastic body 100, and the elasticity calculation is performed based on the preset parameter R of the elastic body 100 and the target radius estimation data a to obtain the elastic characteristic data E, and the specific calculation formula is shown in formula (27):

[0250]

[0251] In one embodiment, a calibration method of a visual-based robot tactile perception system is disclosed, and a standard force sensor is used to calibrate the visual-based robot tactile perception system of the present application, which includes horizontal calibration and vertical calibration.

[0252] Specifically, the calibration of the visual-based robot tactile perception system in the horizontal direction can include but is not limited to the following steps:

[0253] Step one: the standard rigid probe pushes the robot tactile perception system in the horizontal direction, and when the standard force sensor degree is stable at the target pressure f Newton meter, the current displacement field horizontal direction two components are recorded;

[0254] Step two: displacement calculation is performed based on the current displacement field horizontal direction two components, and the specific calculation process is shown in formula (28):

[0255]

[0256] Step three: calculate the horizontal calibration coefficient under the current target pressure, and the specific calculation process is shown in formula (29):

[0257]

[0258] Step four: the horizontal calibration coefficient k f(f=20, 30, 40, 50, 60) are averaged to obtain the target horizontal calibration coefficient, and the specific calculation process is shown in formula (30):

[0259]

[0260] It should be noted that the target horizontal calibration coefficient is the friction calibration coefficient, that is, the first preset calibration coefficient.

[0261] Calibrating the vision-based robot tactile perception system in the vertical direction can include, but is not limited to, the following steps:

[0262] Step 1: The standard rigid probe presses the robot tactile perception system in the vertical direction, and when the standard force sensor reading stabilizes at the target pressure N newton-meters, the current displacement field component in the vertical direction is recorded;

[0263] Step 2: Displacement calculation based on the current displacement field component in the vertical direction, and the specific calculation process is shown in formula (31):

[0264]

[0265] Step 3: Calculate the vertical calibration coefficient under the current target pressure, and the specific calculation process is shown in formula (32):

[0266]

[0267] Step 4: Average the vertical calibration coefficients k N (N=20, 30, 40, 50, 60) to obtain the target vertical calibration coefficient, and the specific calculation process is shown in formula (33):

[0268]

[0269] It should be noted that the target vertical calibration coefficient is the positive pressure calibration coefficient, which can be the first positive pressure calibration coefficient or the second positive pressure calibration coefficient, that is, the second preset calibration coefficient or the third calibration coefficient.

[0270] Referring to Figure 13 The embodiments of the present application also provide a ground feature data measurement device, which can implement the above ground feature data measurement method. The device comprises:

[0271] The motion image acquisition module 1301 is configured to acquire a motion image of the elastic body based on the marker array to obtain target image data.

[0272] The image feature extraction module 1302 is configured to perform image feature extraction on the target image data to obtain motion image feature data.

[0273] The displacement field construction module 1303 is configured to perform displacement field construction based on the motion image feature data, to obtain three-dimensional displacement data.

[0274] The friction evaluation module 1304 is configured to perform friction evaluation on the ground based on the three-dimensional displacement data, to obtain friction feature data.

[0275] The elasticity evaluation module 1305 is configured to perform elasticity evaluation on the ground based on the three-dimensional displacement data, to obtain elasticity feature data.

[0276] The ground feature data generation module is configured to generate feature information of the ground based on the friction feature data and the elasticity feature data, to obtain target ground feature data.

[0277] In the image feature extraction module 1302 of some embodiments, the image feature extraction module further includes:

[0278] The grayscale processing unit is configured to perform grayscale processing on the target image data, to obtain grayscale image data.

[0279] The preset grayscale processing threshold data acquisition unit is configured to acquire preset grayscale processing threshold data of the robot tactile perception system.

[0280] The binary conversion unit is configured to perform binary conversion on the grayscale image data based on the preset grayscale processing threshold data, to obtain motion image feature data.

[0281] In the displacement field construction module 1303 of some embodiments, the displacement field construction module further includes:

[0282] The contour feature extraction unit is configured to perform marker contour feature extraction on the motion image feature data, to obtain marker contour feature data; wherein the marker contour feature data includes contour center position data and contour area data.

[0283] The contour position expression data construction unit is configured to construct contour position expression data based on the contour center position data and preset initial contour information data.

[0284] The optimization processing unit is configured to perform optimization processing on the contour position expression data, to obtain center position optimization data.

[0285] The displacement calculation unit is configured to perform displacement calculation based on the initial contour information data, the contour area data, and the center position optimization data, to obtain three-dimensional displacement data.

[0286] In the contour position expression data construction unit of some embodiments, the contour position expression data construction unit further includes:

[0287] The distance calculation sub-unit is configured to perform distance calculation on the initial contour information data and the contour center position data to obtain displacement estimation data.

[0288] The matrix construction sub-unit is configured to perform matrix construction based on the displacement estimation data to obtain contour position expression data.

[0289] In the friction evaluation module 1304 of some embodiments, the friction evaluation module further includes:

[0290] The friction force calculation unit is configured to perform friction force calculation on the first direction position data and the second direction position data based on a preset first calibration coefficient to obtain friction force estimation data.

[0291] The first pressure calculation unit is configured to perform first pressure calculation on the third direction position data based on a preset second calibration coefficient to obtain first positive pressure estimation data.

[0292] The friction coefficient estimation unit is configured to perform friction coefficient estimation based on the friction force estimation data and the first positive pressure estimation data to obtain friction feature data.

[0293] In the elasticity evaluation module 1305 of some embodiments, the elasticity evaluation module further includes:

[0294] The interpolation calculation unit is configured to perform third direction interpolation calculation based on the three-dimensional displacement data to obtain target interpolation data.

[0295] The second pressure calculation unit is configured to perform second pressure calculation on the target interpolation data based on a preset third calibration coefficient to obtain second positive pressure estimation data.

[0296] The contact area estimation unit is configured to perform contact area estimation based on the target interpolation data to obtain target area estimation data.

[0297] The radius measurement unit is configured to perform radius measurement on the target area estimation data to obtain target radius measurement data.

[0298] The elasticity measurement unit is configured to perform elasticity measurement based on preset parameters of the elastic body and the target radius measurement data to obtain elasticity feature data.

[0299] In the interpolation calculation unit of some embodiments, the interpolation calculation unit further includes:

[0300] The region construction sub-unit is configured to perform region construction based on the first direction position data and the second direction position data to obtain target restriction region data.

[0301] The interpolation processing sub-unit is configured to perform interpolation processing on the third direction position data based on the target restriction region data to obtain the target interpolation data.

[0302] The specific implementation of the ground feature data measuring device is basically the same as the specific embodiment of the ground feature data measuring method described above, and will not be repeated here.

[0303] The embodiment of the present application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the ground feature data measuring method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0304] Please refer to Figure 14 , Figure 14 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0305] The processor 1401 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., and is used to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0306] The memory 1402 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 1402 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1402 and are called and executed by the processor 1401 to implement the ground feature data measuring method of the embodiments of the present application.

[0307] The input / output interface 1403 is used to realize information input and output.

[0308] The communication interface 1404 is used to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0309] The bus 1405 transmits information between various components (such as the processor 1401, the memory 1402, the input / output interface 1403, and the communication interface 1404) of the device.

[0310] The processor 1401, the memory 1402, the input / output interface 1403, and the communication interface 1404 are connected to each other through the bus 1405 to realize the communication connection between them inside the device.

[0311] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the ground feature data measurement method.

[0312] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device or other non-transitory solid-state storage device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0313] The ground feature data measurement method, device, electronic equipment and storage medium provided by the embodiment of the present application are applied to a robot tactile perception system based on vision, which obtains a motion image of an elastic body based on a marker array to obtain target image data; performs image feature extraction on the target image data to obtain motion image feature data; performs displacement field construction based on the motion image feature data to obtain three-dimensional displacement data; performs friction evaluation on the ground based on the three-dimensional displacement data to obtain friction feature data; performs elasticity evaluation on the ground based on the three-dimensional displacement data to obtain elasticity feature data; generates feature information of the ground based on the friction feature data and the elasticity feature data to obtain target ground feature data. The three-dimensional displacement data of the inner surface marker array of the elastic body is constructed, and then rich ground feature data is obtained by calculation, thereby improving the measurement accuracy of the ground feature data.

[0314] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0315] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0316] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0317] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the function modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0318] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is to be interpreted to only cover the embodiments of the application described herein and not a prior art. Moreover, the terms "comprising", "having", "including", and "containing" are to be construed open-ended terms (i.e., meaning "including, but not limited to,") unless otherwise noted to exclude such terms in context. The terms "a", "an", and "the" are used interchangeably with "one or more" or "at least one" unless otherwise indicated in context.

[0319] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be singular or plural.

[0320] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0321] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0322] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0323] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0324] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and the scope of the rights of the embodiments of the present application is not limited thereto. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A method of measuring ground feature data, characterized by, The application is applied to a visual-based robot tactile perception system, the robot tactile perception system comprises an elastomer, an outer surface of the elastomer is in contact with the ground, and an inner surface of the elastomer is provided with a marker array; the method comprises the following steps: Obtaining a motion image of the elastomer based on the marker array to obtain target image data; Performing image feature extraction on the target image data to obtain motion image feature data; Performing displacement field construction based on the motion image feature data to obtain three-dimensional displacement data; Performing friction evaluation on the ground based on the three-dimensional displacement data to obtain friction feature data; Performing elasticity evaluation on the ground based on the three-dimensional displacement data to obtain elasticity feature data; Generating feature information of the ground based on the friction feature data and the elasticity feature data to obtain target ground feature data.

2. The method of claim 1, wherein, The displacement field construction based on the motion image feature data to obtain three-dimensional displacement data comprises the following steps: Performing marker point contour feature extraction on the motion image feature data to obtain marker contour feature data; wherein the marker contour feature data comprises contour center position data and contour area data; Constructing contour position expression data based on the contour center position data and preset initial contour information data; Performing optimization processing on the contour position expression data to obtain center position optimization data; Performing displacement calculation based on the initial contour information data, the contour area data and the center position optimization data to obtain the three-dimensional displacement data.

3. The method of claim 2, wherein, The contour position expression data is constructed based on the contour center position data and preset initial contour information data, and comprises the following steps: Performing distance calculation on the initial contour information data and contour center position data to obtain displacement estimation data; Performing matrix construction based on the displacement estimation data to obtain the contour position expression data.

4. The method of claim 1, wherein, The three-dimensional displacement data comprises first direction position data, second direction position data and third direction position data; the friction evaluation on the ground based on the three-dimensional displacement data to obtain friction feature data comprises the following steps: Performing friction force calculation on the first direction position data and the second direction position data based on a preset first calibration coefficient to obtain friction force estimation data; Performing first pressure calculation on the third direction position data based on a preset second calibration coefficient to obtain first positive pressure estimation data; Performing friction coefficient estimation based on the friction force estimation data and the first positive pressure estimation data to obtain the friction feature data.

5. The method of claim 4, wherein, The elasticity evaluation on the ground based on the three-dimensional displacement data to obtain elasticity feature data comprises the following steps: Performing third direction interpolation calculation based on the three-dimensional displacement data to obtain target interpolation data; Performing second pressure calculation on the target interpolation data based on a preset third calibration coefficient to obtain second positive pressure estimation data; Performing contact area estimation based on the target interpolation data to obtain target area estimation data; Performing radius calculation on the target area estimation data to obtain target radius calculation data; Performing elasticity calculation based on preset parameters of the elastomer and the target radius calculation data to obtain the elasticity feature data.

6. The method of claim 5, wherein, The third direction interpolation calculation based on the three-dimensional displacement data obtains target interpolation data, including: Based on the first direction position data and the second direction position data, the region is constructed to obtain target limit region data; Based on the target limit region data, the third direction position data is interpolated to obtain the target interpolation data.

7. The method according to any one of claims 1 to 6, characterized in that, The image feature extraction of the target image data obtains motion image feature data, including: The target image data is subjected to grayscale processing to obtain grayscale image data; Obtain the preset gray processing threshold data of the robot tactile perception system; Based on the preset gray processing threshold data, the grayscale image data is subjected to binary conversion to obtain the motion image feature data.

8. A ground feature data measuring device, comprising: The device comprises: A motion image acquisition module for acquiring the motion image of the elastic body based on the marker array to obtain target image data; An image feature extraction module for image feature extraction of the target image data to obtain motion image feature data; A displacement field construction module for displacement field construction based on the motion image feature data to obtain three-dimensional displacement data; A friction evaluation module for friction evaluation of the ground based on the three-dimensional displacement data to obtain friction feature data; An elasticity evaluation module for elasticity evaluation of the ground based on the three-dimensional displacement data to obtain elasticity feature data; A ground feature data generation module for generating the characteristic information of the ground based on the friction feature data and the elasticity feature data to obtain target ground feature data.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the ground feature data calculation method of any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1-9. The computer program is executed by the processor to realize the ground feature data calculation method of any one of claims 1-7.

Images