A flexible multi-modal sensor object detection method based on CMYK color blocks

Abstract

The application discloses a flexible multi-mode sensor object detection method based on CMYK color blocks, which comprises the following steps: a flexible multi-mode sensor is composed of a flexible piezoresistive sensor and a flexible piezoelectric sensor, the flexible piezoresistive sensor and the flexible piezoelectric sensor in the flexible multi-mode sensor are respectively installed, the flexible piezoresistive sensor is worn on the back of the joint of each finger of a human hand or a mechanical hand, and the flexible piezoelectric sensor is worn on the inner side of the interdigital space of each finger of the human hand or the mechanical hand; when an object is gripped, the peak value of the resistance signal change collected by the flexible piezoresistive sensor on each finger is recorded, the object detection method provided by the application can directly represent the size and surface roughness information of the object detected by the sensor in the form of color blocks, the object detection difficulty is reduced, and the defect that only a single signal can be detected by the sensor in the object detection is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of flexible sensing, in particular to a flexible multi-modal sensor object detection method based on CMYK color blocks. BACKGROUND

[0002] With the development of artificial intelligence, embedded technology, intelligent robots and digital signal processing technology, the application field of flexible tactile sensor is also more and more extensive, which has penetrated into medical devices, industrial robots, intelligent vehicles and other aspects. The common detection method of flexible tactile sensor is to convert tactile signal into electrical signal for machine processing, which has the advantages of sensitive reaction, obvious signal change, light and thin volume, and strong environmental adaptability.

[0003] Object detection technology is widely used in production lines or workshops with increasing automation, and the popularity of object detection greatly reduces the labor cost in production and improves the unmanned and intelligent level of production. In actual production, visual sensors or photoelectric sensors are often used as common sensors for object detection, but visual sensors and photoelectric sensors are easily affected by different degrees of external environment such as light and dust, and the recognition accuracy will be greatly reduced in some special harsh environments. At the same time, if visual sensors are used, the detection algorithm design is difficult, and the software maintenance cost is high. Moreover, the lens of the visual sensor needs to be replaced after a certain period of use, and the hardware maintenance cost is also high. For photoelectric sensors, although the detection method is simpler than that of visual sensors, the detection accuracy is not as good as that of tactile sensors and visual sensors. Accordingly, the present application provides an object detection method with high detection accuracy and low maintenance cost. SUMMARY

[0004] In order to overcome the defects in the prior art, the present application provides a flexible multi-modal sensor object detection method based on CMYK color blocks. The resistance signal changes collected by the flexible piezoresistive sensor and the output voltage signal changes collected by the flexible piezoelectric sensor on each finger are recorded, the data collected by each finger sensor is processed, normalized, and then represented by the CMYK color representation method to represent the intensity of the two groups of signals. The two groups of intensity maps are added to obtain a multi-modal sensor signal intensity map. By comparing the multi-modal sensor signal intensity map with the reference standard map, the size and surface roughness characteristics of the detected object can be accurately obtained.

[0005] TECHNICAL SCHEME

[0006] A flexible multi-modal sensor object detection method based on CMYK color blocks includes the following steps:

[0007] Step one, the flexible multi-modal sensor is composed of a flexible piezoresistive sensor and a flexible piezoelectric sensor, the flexible piezoresistive sensor and the flexible piezoelectric sensor in the flexible multi-modal sensor are respectively installed, the flexible piezoresistive sensor is worn on the back of the joint of each finger of a human hand or a mechanical hand, and the flexible piezoelectric sensor is worn on the inner side of the interdigital part of each finger of the human hand or the mechanical hand;

[0008] Step two, when the object is gripped, the peak value of the resistance signal change collected by the flexible piezoresistive sensor on each finger is recorded, the corresponding peak value data on each finger is processed, and after normalization, the piezoresistive intensity color block diagram is represented by using the CMYK color representation method.

[0009] Step three, when the object is gripped, the peak value of the output voltage signal change collected by the flexible piezoelectric sensor on each finger is recorded, the corresponding peak value data on each finger is processed, and after normalization, the piezoelectric intensity color block diagram is represented by using the CMYK color representation method.

[0010] Step four, after the piezoresistive intensity color block diagram in step two and the piezoelectric intensity color block diagram in step three are obtained, the two are added to obtain a flexible multi-modal sensor object detection color block diagram, and the flexible multi-modal sensor object detection color block diagram obtained after the coupling of the two signals is more easily distinguished from the color block diagram of a single signal in terms of object size and topography.

[0011] Further, the flexible piezoresistive sensor is used to detect the size of the object, and as the diameter of the object decreases, the resistance change increases, and an Ave (average) variable is defined to represent the size of the object. For objects with consistent surface roughness, the deeper the color in the piezoresistive intensity color block diagram, the greater the bending angle of the flexible piezoresistive sensor, and the smaller the value of Ave.

[0012] The flexible piezoelectric sensor is used to detect the surface roughness of the object, and the curvature of the surface of the object is different, so the piezoelectric signals output for different objects are also different. The greater the surface curvature, the greater the surface roughness of the object, and as the surface curvature increases, the output voltage signal also increases. An Ave (average) variable is defined to represent the curvature of the overall topography of the object, and an SD (standard deviation) variable is used to represent the roughness of the surface topography of the object. For objects with the same diameter, the deeper the color in the piezoelectric intensity color block diagram, the rougher the object, the greater the value of Ave, and the greater the value of SD.

[0013] Further, the piezoresistive signal detected by the flexible piezoresistive sensor is denoted as R, the piezoelectric signal detected by the flexible piezoelectric sensor is denoted as E, and the two signals are represented by the CMYK color representation method and then added to obtain a flexible multi-modal sensor signal intensity diagram, denoted as B.

[0014] Further, a flexible multi-modal sensor signal reference standard map is established for comparison with the flexible multi-modal sensor signal intensity map to determine the size and surface roughness of the detected object.

[0015] Further, in the flexible multi-modal sensor signal reference standard map, the horizontal direction represents the size of the object, and the object with larger size is closer to the left side of the flexible multi-modal sensor signal reference standard map; the vertical direction represents the surface roughness of the object, and the object with larger surface curvature, i.e. the object with rougher surface, is closer to the upper side of the flexible multi-modal sensor signal reference standard map.

[0016] Advantages

[0017] Compared with the prior art, the present application has the following advantages:

[0018] 1. By using the CMYK color block representation method to process the signals collected by the sensor, the difficulty of object detection is reduced, and the size and surface roughness information of the object detected by each sensor are displayed in an intuitive color block manner;

[0019] 2. By combining the piezoresistive signal and the piezoelectric signal for processing, the defect that a single signal can only detect a single index in object detection is avoided;

[0020] 3. Using the flexible multi-modal sensor signal reference standard map as the comparison basis of the multi-modal sensor signal intensity map, the size and surface roughness information of the detected object can be obtained by comparison;

[0021] 4. Using the flexible tactile sensor as the sensor for object detection, the sensor can adapt to the harsh environment in actual industrial production and maintain the accuracy and sensitivity of detection. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a flexible piezoresistive sensor resistance change rate graph when detecting different objects;

[0023] Figure 2 is a flexible piezoresistive sensor piezoresistive signal intensity graph represented based on the CMYK method;

[0024] Figure 3 is a flexible piezoelectric sensor output voltage change graph when detecting different objects;

[0025] Figure 4 is a flexible piezoelectric sensor electric signal intensity graph represented based on the CMYK method;

[0026] Figure 5 is a flexible multi-modal sensor signal intensity graph represented based on the CMYK method when measuring different objects;

[0027] Figure 6 It is a reference standard diagram of flexible multimodal sensor signals based on the CMYK method;

[0028] Figure 7 It is the object recognition rate matrix of the flexible piezoresistive sensor;

[0029] Figure 8 It is the recognition rate matrix of objects detected by flexible piezoelectric sensors;

[0030] Figure 9 It is the recognition rate matrix of objects detected by the flexible multimodal sensor. Detailed Implementation

[0031] To better illustrate the content of this invention, the following description is provided in conjunction with the accompanying drawings and examples:

[0032] like Figures 1-9 As shown, this invention discloses a flexible multimodal sensor object detection method based on CMYK color blocks, comprising the following steps:

[0033] Step 1: The flexible multimodal sensor consists of a flexible piezoresistive sensor and a flexible piezoelectric sensor. The flexible piezoresistive sensor and the flexible piezoelectric sensor in the flexible multimodal sensor are installed separately. The flexible piezoresistive sensor is worn on the back of the joint of each finger of the human hand or robotic hand, and the flexible piezoelectric sensor is worn on the inside of the interphalangeal space of each finger of the human hand or robotic hand.

[0034] Step 2: When grasping an object, record the peak value of the resistance signal change collected by the flexible piezoresistive sensor on each finger. Process the corresponding peak data on each finger, normalize it, and then use CMYK color representation to represent the piezoresistive intensity color block diagram.

[0035] Step 3: When grasping an object, record the peak value of the output voltage signal change collected by the flexible piezoelectric sensor on each finger. Process the corresponding peak data on each finger, normalize it, and then use CMYK color representation to represent the piezoelectric intensity color block image.

[0036] Step four: After obtaining the piezoresistive intensity color block map from step two and the piezoelectric intensity color block map from step three, add the two together to obtain the flexible multimodal sensor object detection color block map. The flexible multimodal sensor object detection color block map obtained by coupling the two signals is more likely to distinguish the size and shape of the object than the color block map of a single signal.

[0037] Further, the flexible piezoresistive sensor is used to detect the size of the object, and the resistance change amount increases as the diameter of the object decreases, and an Ave (average) variable is defined to represent the size of the object. For objects with consistent surface roughness, the deeper the color in the piezoresistive intensity color block diagram, the greater the bending angle of the flexible piezoresistive sensor, and the smaller the value of Ave. The flexible piezoelectric sensor is used to detect the surface roughness of the object. The curvature of the object surface is different, so the piezoelectric signals output for different objects are also different. The greater the surface curvature, the greater the output voltage signal. An Ave (average) variable is defined to represent the curvature of the overall appearance of the object, and an SD (standard deviation) variable is used to represent the roughness of the surface appearance of the object. For objects with the same diameter, the deeper the color in the piezoelectric intensity color block diagram, the rougher the object, the greater the value of Ave, and the greater the value of SD.

[0038] Further, the piezoresistive signal detected by the flexible piezoresistive sensor is denoted as R, the piezoelectric signal detected by the flexible piezoelectric sensor is denoted as E, and the two signals are added after being represented by CMYK color representation method to obtain a flexible multi-modal sensor signal intensity diagram, denoted as B.

[0039] Further, a flexible multi-modal sensor signal reference standard diagram is developed for comparison with the flexible multi-modal sensor signal intensity diagram to determine the size and surface roughness characteristics of the detected object.

[0040] Further, in the flexible multi-modal sensor signal reference standard diagram, the horizontal direction represents the size of the object, and the larger the size of the object, the closer it is to the left side of the flexible multi-modal sensor signal reference standard diagram. The vertical direction represents the surface roughness of the object, and the greater the surface curvature, i.e., the rougher the surface, the closer it is to the top of the flexible multi-modal sensor signal reference standard diagram.

[0041] As shown in Figure 5 the piezoresistive signal detected by the flexible piezoresistive sensor is denoted as R, the piezoelectric signal detected by the flexible piezoelectric sensor is denoted as E, and the two signals are added after being represented by CMYK color representation method to obtain the flexible multi-modal sensor signal intensity diagram, denoted as B.

[0042] As shown in Figure 6 a flexible multi-modal sensor signal reference standard diagram is developed for comparison with the flexible multi-modal sensor signal intensity diagram to determine the size and surface roughness characteristics of the detected object.

[0043] As shown in Figure 7As shown, an object detection experiment was conducted using only the flexible piezoresistive sensor of the flexible multimodal sensor. The detection objects were six small balls of different sizes but the same surface roughness. The resistance signal was defined as being recognized successfully within ±5% error. Fifty grasping tests were performed on each of the six small balls, and the detection results were recorded. The detection results were converted into a recognition rate matrix. The experiment found that the flexible piezoresistive sensor can distinguish the size information of the object relatively well. However, when the size is the same, the recognition rate of small balls with different surface roughness is very low. Therefore, the piezoresistive signal detected by the flexible piezoresistive sensor is used as an indicator of the size of the detected object.

[0044] like Figure 8 As shown, an object detection experiment was conducted using only the flexible piezoelectric sensor of the flexible multimodal sensor. The detection objects were eight small balls of different sizes but with the same surface roughness. The resistance signal was defined as being recognized successfully if the error was within ±5%. Fifty grasping tests were performed on each of the eight small balls, and the detection results were recorded. The detection results were converted into a recognition rate matrix. The experiment found that the flexible piezoelectric sensor can distinguish the surface curvature information of the object, i.e., the surface roughness information of the object, relatively well. However, when the surface roughness is almost the same, since the surface curvature of the small balls with almost the same surface roughness is almost the same, the flexible piezoelectric sensor cannot effectively distinguish the size of the object. Therefore, the piezoelectric signal detected by the flexible piezoelectric sensor is used as an indicator for detecting the surface roughness of the object.

[0045] like Figure 9 As shown, an object detection experiment was conducted using a flexible multimodal sensor. Three identical small balls of different sizes and surface roughness were grasped 50 times, and the detection results were recorded. The detection results were converted into a recognition rate matrix. The experiment found that after combining the acquisition signals of the flexible piezoresistive sensor and the flexible piezoelectric sensor, the accuracy of the sensor in the object detection experiment was much higher than that of the single-signal sensor. It can not only effectively identify objects of different sizes, but also objects with different surface roughness.

[0046] Specifically, the electrical signals acquired by the flexible piezoresistive sensor and the flexible piezoelectric sensor are normalized and converted into CMYK color block images. The obtained piezoresistive intensity color block images are then coupled with the piezoelectric intensity color block images to obtain the signal intensity image of the flexible multimodal sensor. The signal intensity image of the flexible multimodal sensor is compared and analyzed with the signal reference standard image of the flexible multimodal sensor to determine the size and surface roughness characteristics of the detected object. Through repeated experiments, the object recognition rate matrix of the flexible multimodal sensor is obtained, demonstrating the convenience and accuracy of this object detection method for flexible multimodal sensors.

[0047] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the technical solutions of the present application are described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some of the technical features thereof can be replaced by equivalent features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A flexible multi-modal sensor object detection method based on CMYK color patches, characterized by, It comprises the following steps: Step one, the flexible multi-modal sensor is composed of a flexible piezoresistive sensor and a flexible piezoelectric sensor, the flexible piezoresistive sensor and the flexible piezoelectric sensor in the flexible multi-modal sensor are installed respectively, the flexible piezoresistive sensor is worn on the back of the joint of each finger of the human hand or the mechanical hand, and the flexible piezoelectric sensor is worn on the inner side of the interdigital of each finger of the human hand or the mechanical hand; Step two, when grasping an object, record the peak value of the resistance signal change collected by the flexible piezoresistive sensor on each finger, process the corresponding peak value data on each finger, normalize, and represent the piezoresistive intensity color block diagram by using the CMYK color representation method; Step three, when grasping an object, record the peak value of the output voltage signal change collected by the flexible piezoelectric sensor on each finger, process the corresponding peak value data on each finger, normalize, and represent the piezoelectric intensity color block diagram by using the CMYK color representation method; Step four, after obtaining the piezoresistive intensity color block diagram in step two and the piezoelectric intensity color block diagram in step three, add them to obtain a flexible multi-modal sensor object detection color block diagram, and the flexible multi-modal sensor object detection color block diagram obtained after coupling the two signals is easier to distinguish the size and topography of the object than the color block diagram of a single signal.

2. The method of claim 1, wherein the method is based on CMYK color patch based flexible multi-modal sensor object detection. The flexible piezoresistive sensor is used to detect the size of the object, and as the diameter of the object decreases, the resistance change increases, and an Ave variable is defined to represent the size of the object. For objects with consistent surface roughness, the deeper the color in the piezoresistive intensity color block diagram, the greater the bending angle of the flexible piezoresistive sensor, and the smaller the value of Ave. The flexible piezoelectric sensor is used to detect the surface roughness of the object. The curvature of the object surface is different, so the piezoelectric signals output for different objects are also different. The greater the surface curvature, the greater the surface roughness of the object, and as the surface curvature increases, the output voltage signal also increases. An Ave variable is defined to represent the curvature of the overall topography of the object, and an SD variable is used to represent the roughness of the surface topography of the object. For objects with the same diameter, the deeper the color in the piezoelectric intensity color block diagram, the rougher the object, the greater the value of Ave, and the greater the value of SD.

3. The method of claim 2, wherein the method is based on CMYK color patch based flexible multi-modal sensor object detection. The piezoresistive signal detected by the flexible piezoresistive sensor is denoted as R, the piezoelectric signal detected by the flexible piezoelectric sensor is denoted as E, and the two signals are added after being represented by the CMYK color representation method to obtain a flexible multi-modal sensor signal intensity diagram, denoted as B.

4. The method of claim 3, wherein the method is based on CMYK color patch based flexible multi-modal sensor object detection. A flexible multi-modal sensor signal reference standard diagram is established for comparison with the flexible multi-modal sensor signal intensity diagram to determine the size and surface roughness characteristics of the detected object.

5. The method of claim 4, wherein: In the flexible multi-modal sensor signal reference standard diagram, the horizontal direction represents the size of the object, and the larger the size of the object, the closer it is to the left side of the flexible multi-modal sensor signal reference standard diagram. The vertical direction represents the surface roughness of the object, and the greater the surface curvature, the rougher the object, and the closer it is to the upper part of the flexible multi-modal sensor signal reference standard diagram.

Images