A particulate matter blockage self-adaptive sensing module, robot and working method

Abstract

The application discloses a kind of particulate matter block adaptive sensing module, robot and working method, and the sensing module includes acoustic sensing component, it includes base and microphone, base is opened with air extraction hole;Optical sensing component, it includes skin and optical waveguide sensor, skin covers on base, cavity is reserved between skin and base, air extraction hole is communicated to cavity;Optical waveguide sensor is arranged on skin with each other staggered;Fill granule, which is filled in cavity;Air extraction equipment is connected with air extraction hole.When there is external force to skin, fill granule can move in compliance with the shape of object exerting external force, while optical waveguide sensor can detect the position of external force and the deformation amplitude of skin;Air in cavity can be extracted using air extraction equipment to make fill granule adhere to each other, improve bearing capacity while making external vibration better transmitted to microphone, high-frequency vibration is detected by acoustic detection mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a particle blocking self-adaptive sensing module, a robot and a working method. BACKGROUND

[0002] In the prior art, the implementation of the robot's tactile sensor mainly includes various types such as resistance type, capacitance type, piezoelectric type and optical type. These tactile sensors are used to detect the direction and position of external force, so as to perform closed-loop control on the driving mechanism of the robot. Among them, the flexible optical waveguide sensor has become one of the research hotspots in recent years due to its good flexibility, expandability and anti-electromagnetic interference capability. This kind of sensor usually uses the change of light intensity with structural deformation to realize pressure sensing, and can be combined with a flexible substrate to form a distributed tactile array. However, the output signal of this kind of sensor mainly reflects the low-frequency or quasi-static contact force change, and the response to high-frequency transient signals (such as vibration, collision and sliding) is weak, which is difficult to meet the demand for wideband tactile sensing in complex operation tasks.

[0003] At the same time, the research on human skin tactile system shows that the skin receptors have significant differences in response to different frequency bands: slow adapting (SA) receptors are sensitive to low-frequency or continuous pressure; fast adapting (FA) receptors are more sensitive to high-frequency vibration signals. It is the complementarity of these two types of receptors that enables human skin to perceive full-band tactile information from continuous pressure to transient vibration. In contrast, existing robot tactile modules often only cover one of the frequency bands, lacking the multi-modal fusion capability similar to biological skin. The particle blocking structure in the prior art is only used to provide shape adaptation or stiffness adjustment, and has not been used as a modulation medium for acoustic propagation. Its scattering, attenuation and frequency domain redistribution effects on vibration signals have not been utilized.

[0004] In summary, the existing system cannot realize wideband multi-modal tactile sensing of low-frequency pressure and high-frequency vibration in a single module, and lacks a physical coupling mechanism between optical signals and acoustic signals. Especially when the robot performs complex operations (such as fine assembly, flexible grasping and object state recognition), the sensing bandwidth and response speed of the existing module are insufficient, which seriously limits its application in high dynamic tasks. SUMMARY

[0005] The present application aims to at least solve one of the above technical problems in the prior art. To this end, the present application proposes a particle blocking self-adaptive sensing module, which aims to construct an optical-particle-acoustic multi-modal collaborative tactile sensing module with adjustable structure and shape adaptation, and capable of sensing low-frequency pressure and high-frequency vibration.

[0006] The application further provides a robot provided with the particle blocking self-adaptive sensing module and a working method of the particle blocking self-adaptive sensing module.

[0007] The particle blocking self-adaptive sensing module according to the first aspect of the application comprises:

[0008] An acoustic sensing assembly comprising a base and a microphone mounted on the base, the base being provided with an air extraction hole;

[0009] An optical sensing assembly comprising a skin and optical waveguide sensors, the skin being a flexible member and covering the base, a cavity being reserved between the skin and the base, the air extraction hole of the base being communicated with the cavity; the optical waveguide sensors are in the form of strips and in a plurality in number, each of the optical waveguide sensors being arranged on the skin in an interlaced manner;

[0010] Filler particles in a plurality in number and filled in the cavity;

[0011] An air extraction device connected with the air extraction hole;

[0012] The optical waveguide sensors can detect the bending deformation of the skin, the air extraction device can extract the air in the cavity to make the filler particles adhere to each other, and the microphone can detect the vibration through the filler particles.

[0013] The particle blocking self-adaptive sensing module according to the application has at least the following beneficial effects: when an external force acts on the skin, the filler particles can move in accordance with the shape of the object exerting the external force, and the optical waveguide sensors can detect the position of the external force and the deformation amplitude of the skin; the air extraction device can extract the air in the cavity to make the filler particles adhere to each other, thereby improving the bearing capacity and enabling the external vibration to be better transmitted to the microphone, the microphone can capture the high-frequency vibration signal modulated by the filler particles, thereby realizing high-sensitivity identification of slippage, friction and transient tactile events.

[0014] According to some embodiments of the application, the optical waveguide sensors are arranged in an array on the skin.

[0015] According to some embodiments of the application, the skin comprises a top portion and a side portion connected with each other, the top portion and the side portion of the skin and the base jointly define the cavity, and the optical waveguide sensors extend on the top portion and the side portion of the skin.

[0016] According to some embodiments of the application, the optical waveguide sensors are divided into first optical waveguide sensors and second optical waveguide sensors, the first optical waveguide sensors are parallel to a first plane, the second optical waveguide sensors are parallel to a second plane, and the first plane is perpendicular to the second plane.

[0017] According to some embodiments of the present application, the skin is a silica gel piece.

[0018] According to some embodiments of the present application, the particulate matter blockage adaptive sensing module further comprises a controller, which is electrically connected with the microphone, the optical waveguide sensor and the air extraction device.

[0019] The robot according to the second aspect of the present application comprises a robot arm and the above-mentioned particulate matter blockage adaptive sensing module, which is installed on the surface of the robot arm.

[0020] The robot according to the present application has at least the following beneficial effects: when the robot arm grips an object, the particulate matter blockage adaptive sensing module not only deforms according to the outer contour of the object, but also detects the gripping force and high-frequency vibration, and then feeds back to the driving mechanism of the robot to perform corresponding actions, thereby further improving the action accuracy of the robot.

[0021] According to some embodiments of the present application, the number of particulate matter blockage adaptive sensing modules is multiple and is installed in an array on the surface of the robot arm.

[0022] The working method according to the third aspect of the present application is based on the above-mentioned particulate matter blockage adaptive sensing module and comprises the following steps:

[0023] The particulate matter blockage adaptive sensing module approaches the object, and the skin deforms under pressure after contacting the object, and the filler particles displace under pressure to make the skin adapt to the outer contour of the object;

[0024] The optical waveguide sensor detects the deformation of the skin, thereby detecting the external force;

[0025] The air extraction device is turned on, the air in the cavity is extracted, and the filler particles are tightly attached to each other;

[0026] The vibration of the object is conducted to the microphone through the skin, the filler particles and the base in turn, and thereby the vibration is detected by the microphone;

[0027] The detection results of the microphone and the optical waveguide sensor are comprehensively analyzed, and the object is multi-modal sensed.

[0028] The working method according to the present application has at least the following beneficial effects: by comprehensively analyzing the detection results of the microphone and the optical waveguide sensor, the low-frequency force and the high-frequency vibration of the external force can be analyzed, thereby realizing collaborative sensing.

[0029] According to some embodiments of the present application, the analysis of the detection results of the microphone and the optical waveguide sensor includes the following steps:

[0030] Filtering the detection results of the microphone and the optical waveguide sensor;

[0031] For the microphone, calculating the short-time Fourier transform or power spectral density to obtain the spectral centroid, spectral bandwidth, spectral roll-off, spectral flatness, spectral entropy, peak frequency and peak power, and multi-segment band energy ratio;

[0032] For the optical waveguide sensor, extracting time-domain statistics, interval integration, and array-based spatial weight mapping;

[0033] After standardizing the frequency domain and time-frequency features of the microphone and the time domain and spatial features of the optical waveguide sensor, they are spliced to form a joint feature vector;

[0034] Using a machine learning model to classify and regress the joint feature vector;

[0035] Obtaining independent decisions for the acoustic and optical subsystem outputs, respectively, and then deciding the final output in a weighted voting or confidence fusion manner;

[0036] Using a time series model to regress or sequence classify the fused features;

[0037] Using the microphone high-frequency energy envelope or wavelet high-frequency coefficient, when exceeding the threshold, triggering the controller to increase the clamping force of the robot hand or stop the clamping action;

[0038] Combining the optical distribution energy and acoustic characteristics to determine whether the air extraction condition is met, and then selecting whether to trigger the air extraction device.

[0039] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings are used to provide a further understanding of the technical solutions disclosed in the present application, and constitute a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments disclosed in the present application, and do not constitute a limitation on the technical solutions of the present application.

[0041] Figure 1 A structural schematic diagram of a particulate matter blockage adaptive sensing module according to the first aspect of the present application;

[0042] Figure 2 A structural schematic diagram of an acoustic sensing assembly in the particulate matter blockage adaptive sensing module according to the first aspect of the present application;

[0043] Figure 3 FIG. 1 is a structural schematic diagram of an optical sensing assembly in a particulate matter blockage self-adaptive sensing module according to an embodiment of the first aspect of the present application;

[0044] Figure 4 FIG. 2 is a workflow diagram of the particulate matter blockage self-adaptive sensing module according to an embodiment of the first aspect of the present application.

[0045] FIG. 1 is a structural schematic diagram of an optical sensing assembly in a particulate matter blockage self-adaptive sensing module according to an embodiment of the first aspect of the present application; DETAILED DESCRIPTION

[0046] The embodiments of the present application are described below in detail with reference to the accompanying drawings. The same or similar components are denoted by the same or similar reference numerals throughout the drawings, and repeated description is omitted. The embodiments described below are merely examples for explaining the present application, and should not be construed as limiting the present application.

[0047] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, front, back, left, right, etc. is based on the orientation or position relationship shown in the drawings, and is only for the purpose of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element indicated must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0048] In the description of the present application, if the meaning of several is more than one, the meaning of multiple is more than two, greater than, less than, more than, etc. are understood as not including the number, above, below, etc. are understood as including the number. If it is described as first, second, etc. only for the purpose of distinguishing technical features, it cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the order of indicated technical features.

[0049] In the description of the present application, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be broadly understood, and the person skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0050] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "illustrative embodiment", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0051] In the prior art, the implementation of the tactile sensor of the robot mainly includes various types such as resistance type, capacitance type, piezoelectric type and optical type, and these tactile sensors are used to detect the action direction and action position of the external force, so as to perform closed-loop control on the driving mechanism of the robot. Among them, the flexible optical waveguide sensor has become one of the research hotspots in recent years due to its good flexibility, expandability and anti-electromagnetic interference capability. This kind of sensor usually uses the change of light intensity with structural deformation to realize pressure sensing, and can be combined with a flexible substrate to form a distributed tactile array. However, the output signal of this kind of sensor mainly reflects the low-frequency or quasi-static contact force change, and the response to high-frequency transient signals (such as vibration, collision, sliding) is weak, which is difficult to meet the demand for wideband tactile sensing in complex operation tasks.

[0052] At the same time, the research on human skin tactile system shows that the skin receptors have significant differences in response to different frequency bands: slow adapting (SA) receptors are sensitive to low-frequency or continuous pressure; fast adapting (FA) receptors are more sensitive to high-frequency vibration signals. It is the complementarity of these two types of receptors that enables human skin to perceive full-band tactile information from continuous pressure to transient vibration. In contrast, existing robot tactile modules often only cover one of the frequency bands, lacking the multi-modal fusion capability similar to biological skin. The particle blocking structure in the prior art is only used to provide shape adaptation or stiffness adjustment, and has not been used as a modulation medium for acoustic propagation, and its scattering, attenuation and frequency domain redistribution effects on vibration signals have not been utilized.

[0053] In summary, the existing system cannot realize wideband multi-modal tactile sensing of low-frequency pressure and high-frequency vibration in a single module, and also lacks a physical coupling mechanism between optical signals and acoustic signals. Especially when the robot performs complex operations (such as fine assembly, flexible grasping, object state recognition), the sensing bandwidth and response speed of the existing module are insufficient, which seriously limits its application in high dynamic tasks.

[0054] To this end, the application provides a particulate matter blockage adaptive sensing module, a robot and a working method. When an external force acts on the surface, the filling particles of the particulate matter blockage adaptive sensing module can move in accordance with the shape of the object exerting the external force, and the optical waveguide sensor can detect the position of the external force and the deformation amplitude of the surface; the air in the cavity is extracted by the air extraction device, so that the filling particles are closely attached to each other, the bearing capacity is improved, and the external vibration can be better transmitted to the microphone, and the high-frequency vibration is detected by acoustic detection.

[0055] The robot comprising the particulate matter blockage adaptive sensing module can not only deform in accordance with the outer contour of the object when the mechanical hand grips the object, but also detect the gripping force and high-frequency vibration, and then feed back to the driving mechanism of the robot to perform corresponding actions, so as to further improve the action accuracy of the robot.

[0056] According to the working method of the particulate matter blockage adaptive sensing module, the low-frequency force and high-frequency vibration of the external force can be analyzed by comprehensively analyzing the detection results of the microphone and the optical waveguide sensor, so as to realize collaborative sensing.

[0057] With reference to Figure 1 , the particulate matter blockage adaptive sensing module in the first aspect of the application comprises an acoustic sensing assembly 100, an optical sensing assembly 200, filling particles 300 and an air extraction device. The acoustic sensing assembly 100 is used to detect high-frequency vibration of the external environment, and the optical sensing assembly 200 is used to detect low-frequency force of the external environment. The filling particles 300 are used to transmit vibration to the acoustic sensing assembly 100, and on the other hand, can move in accordance with the external force, so that the particulate matter blockage adaptive sensing module can adaptively deform according to different object shapes. The air extraction device is used to extract air, so that the filling particles 300 are closely attached to each other, and then the vibration and force are better transmitted.

[0058] Specifically, with reference to Figure 2 , the acoustic sensing assembly 100 comprises a base 110 and a microphone 120 mounted on the base 110. The microphone 120 is used to record external sounds, and captures high-frequency vibration of the external environment by acoustic detection. It is worth noting that the base 110 is provided with an air extraction hole 130, and the air extraction device is connected with the air extraction hole 130.

[0059] With reference to Figure 3The optical sensing component 200 includes a skin 210 and optical waveguide sensors 220. The skin 210 is a flexible component that covers the base 110, and a cavity is reserved between the skin 210 and the base 110 for filling multiple filler particles 300. An air extraction port 130 of the base 110 communicates with the cavity, so that when the air extraction device is activated, the air in the cavity can be extracted, causing the skin 210 to shrink inward and making the individual filler particles 300 adhere tightly to each other, thus enabling better transmission of force and vibration through the filler particles 300. Multiple optical waveguide sensors 220 are strip-shaped and are arranged alternately on the skin 210. When the skin 210 deforms, the optical waveguide sensors 220 deform accordingly, and this deformation can be detected by the photosensitive element in the optical waveguide sensor 220, thereby sensing external forces.

[0060] Therefore, this particulate obstruction adaptive sensing module adopts an optical-particle-acoustic three-layer coupling structure. The filling particles 300 not only serve as shape adaptation and stiffness adjustment units but also as acoustic modulation media. By utilizing the different acoustic impedance, scattering paths, and energy attenuation characteristics of the filling particles 300 in loose and obstructed states, the response capability of the microphone 120 to high-frequency tactile events such as slippage, friction, and transient collisions is enhanced, thereby achieving wideband tactile perception and multimodal signal fusion, and improving the robot's robustness and recognition capabilities in grasping and manipulating tasks. Furthermore, by combining the detection results of the acoustic sensing component 100 and the optical sensing component 200, a wider tactile perception range from low to high frequencies can be covered.

[0061] Furthermore, the optical waveguide sensors 220 are arrayed on the skin 210 to form a uniformly distributed mesh detection structure, whose detection range can cover all positions on the skin 210.

[0062] Specifically, the skin 210 includes an interconnected top and side portions. The top, side portions of the skin 210 and the base 110 together enclose a cavity. The optical waveguide sensor 220 extends on the top and side portions of the skin 210, so that the force exerted on the top or side portions of the skin 210 can be detected by the optical waveguide sensor 220.

[0063] Specifically, the optical waveguide sensor 220 is divided into a first optical waveguide sensor and a second optical waveguide sensor. The first optical waveguide sensor is connected to the first plane (refer to...). Figure 3 The second optical waveguide sensor is parallel to the xoz plane in the second plane (refer to the xoz plane in the second plane). Figure 3 The first and second optical waveguide sensors are parallel to each other (the yoz plane in the image), and perpendicular to each other. Thus, the first and second optical waveguide sensors are perpendicularly intersecting.

[0064] Specifically, the skin 210 is a flexible piece with elasticity, so as to be deformed in compliance with external force. The skin 210 can be made of silicone, rubber or other elastic materials, and in the embodiment, the skin 210 is a silicone piece.

[0065] Further, the particulate matter blockage adaptive sensing module further comprises a controller, which is electrically connected with the microphone 120, the optical waveguide sensor 220 and the air extraction device, so that the controller can receive the detection data of the microphone 120 and the optical waveguide sensor 220, and also control the air extraction device to control the opening and closing timing and power output of the air extraction device.

[0066] In the second aspect of the present application, a robot comprises a mechanical hand and the above-mentioned particulate matter blockage adaptive sensing module. The particulate matter blockage adaptive sensing module is installed on the surface of the mechanical hand. When the mechanical hand grips an object, the object can generate pressure on the particulate matter blockage adaptive sensing module, thereby detecting the gripping force and possible vibration in the gripping process.

[0067] Further, the number of particulate matter blockage adaptive sensing modules is multiple and installed in an array on the surface of the mechanical hand, so that the gripping force is more comprehensively analyzed through multiple particulate matter blockage adaptive sensing modules.

[0068] Reference Figure 4 The optical waveguide sensor 220 specifically comprises a photodiode and a photosensitive diode. The light emitted by the photosensitive diode is received by the photodiode, and then the optical signal is converted into an electrical signal by an optical-to-electrical conversion module, which is collected by a 16-bit data acquisition card and transmitted to the PC end. At the same time, the acoustic signal of the microphone 120 is amplified by an amplifier, and the amplified signal also enters the PC end for comprehensive processing. After the two-way signal is processed and fused, the robot is guided to perform corresponding actions.

[0069] In the third aspect of the present application, a working method is provided for the above-mentioned particulate matter blockage adaptive sensing module, comprising the following steps:

[0070] S100. The particulate matter blockage adaptive sensing module approaches the object. After the skin 210 contacts the object, it is deformed under pressure, and the filling particles 300 are displaced under pressure to adapt the skin 210 to the contour of the object;

[0071] S200. The optical waveguide sensor 220 detects the deformation of the skin 210, and detects the position and size of the external force by detecting the deformation position and deformation amplitude;

[0072] S300. The air extraction device is turned on, the air in the cavity is extracted, the filling particles 300 are in close contact with each other, thereby improving the structural rigidity of the particulate matter blockage adaptive sensing module, and also preparing for subsequent vibration transmission.

[0073] S400. The vibration of the object is conducted to the microphone 120 through the skin 210, the filler particles 300 and the base 110 in turn, so that the vibration is detected by the microphone 120;

[0074] S500. The detection results of the microphone 120 and the optical waveguide sensor 220 are comprehensively analyzed to perform multi-modal perception on the object.

[0075] Further, the analysis of the detection results of the microphone 120 and the optical waveguide sensor 220 includes the following steps:

[0076] S510. The detection results of the microphone 120 and the optical waveguide sensor 220 are filtered; wherein the microphone 120 adopts a band-pass filter of 5 to 1000 Hz, and removes direct current components and normalizes energy of a short-time signal, and performs spectral subtraction or adaptive noise suppression when the noise environment is strong; the optical waveguide sensor 220 adopts a band-pass or high-pass filter of 0.1 to 50 Hz, and uses baseline correction and sliding average or adaptive filtering for latency and nonlinearity, and applies different gains and correction coefficients to compensate for amplitude deviation caused by stiffness change when the gas extraction equipment switches between gas extraction and non-extraction;

[0077] S520. For the microphone 120, a short-time Fourier transform or a power spectral density is calculated to obtain spectral centroid, spectral bandwidth, spectral roll-off, spectral flatness, spectral entropy, peak frequency and peak power, and multi-segment band energy ratio;

[0078] S530. For the optical waveguide sensor 220, time domain statistics, interval integration and array-based spatial weight mapping are extracted; wherein the time domain statistics include mean, peak, minimum, pulse rise and fall time, slope, variance and median, and the interval integration refers to contact energy, and these features are used to estimate contact position and static or quasi-static force;

[0079] S540. The frequency domain and time-frequency features of the microphone, and the time domain and spatial features of the optical waveguide sensor are standardized and spliced to form a joint feature vector;

[0080] S550. The joint feature vector is classified and regressed using a machine learning model; wherein the machine learning model can be random forest RF, KNN, LightGBM or neural network;

[0081] S560. Independent decisions are obtained by respectively outputting the acoustic and optical subsystems, and the final output is determined by weighted voting or confidence fusion;

[0082] S570. Regress or sequence classify the fused features using a time series model; wherein the time series model can use LSTM / Temporal Convolution model;

[0083] S580. Use the microphone high-frequency energy envelope or wavelet high-frequency coefficients to trigger the controller to increase the gripping force of the robot hand or stop the gripping action when the threshold is exceeded, which is used to prevent the grabbed object from slipping and adjust the grip force when sliding is detected;

[0084] S590. Combine optical distribution energy and acoustic features to determine whether the air extraction condition is met, i.e. whether the contact area and pressure threshold are met; then select whether to trigger the air extraction device.

[0085] The embodiments of the present application are described in detail above with reference to the drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the present application. In addition, the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

Claims

1. A particulate matter occlusion self-adapting sensing module, characterized in that, include: An acoustic sensing assembly includes a base and a microphone mounted on the base, the base having an air extraction port; An optical sensing assembly includes a skin and optical waveguide sensors. The skin is a flexible component that covers a base. A cavity is reserved between the skin and the base. An air extraction hole in the base is connected to the cavity. The optical waveguide sensors are strip-shaped and there are multiple of them. The optical waveguide sensors are arranged in an array on the skin, with each optical waveguide sensor interlaced with the others. Multiple filling particles are used to fill the cavity. An air extraction device connected to the air extraction port; The optical waveguide sensor can detect the bending deformation of the skin, the air extraction device can draw air from the cavity to make the filling particles stick together, and the microphone can detect vibration through the filling particles. The skin includes an interconnected top and side portions, the top and side portions of the skin and the base together enclosing the cavity, and the optical waveguide sensor extends from the top and side portions of the skin; The optical waveguide sensor is divided into a first optical waveguide sensor and a second optical waveguide sensor. The first optical waveguide sensor is parallel to a first plane, and the second optical waveguide sensor is parallel to a second plane. The first plane is perpendicular to the second plane. The outer skin is made of silicone.

2. The particulate matter occlusion self-aware module of claim 1, wherein: The particulate matter blockage adaptive sensing module also includes a controller, which is electrically connected to the microphone, the optical waveguide sensor and the air extraction device.

3. A robot, characterized in that, It includes a robotic arm and a particulate matter obstruction adaptive sensing module as described in any one of claims 1 to 2, wherein the particulate matter obstruction adaptive sensing module is mounted on the surface of the robotic arm.

4. The robot of claim 3, wherein: The particulate matter obstruction adaptive sensing modules are multiple and are mounted in an array on the surface of the robotic arm.

5. A method of operating a particulate matter occlusion adaptive awareness module according to any one of claims 1 to 2, characterized by, include: When the particulate matter obstruction adaptive sensing module approaches the object, the skin is deformed under pressure after contact with the object, and the filling particles are displaced under pressure so that the skin adapts to the shape and contour of the object. The optical waveguide sensor detects the deformation of the skin, thereby detecting external force; When the air extraction device is turned on, the air in the cavity is drawn away, and the filling particles adhere tightly to each other. The vibration of the object is transmitted sequentially through the skin, the filling particles, and the base to the microphone, thereby detecting the vibration through the microphone; By comprehensively analyzing the detection results of the microphone and the optical waveguide sensor, the object is subjected to multimodal perception.

6. The method of working according to claim 5, characterized in that: The analysis of the detection results from the microphone and the optical waveguide sensor includes the following steps: The detection results from the microphone and the optical waveguide sensor are filtered. For the microphone, calculate the short-time Fourier transform or power spectral density to obtain the spectral centroid, spectral bandwidth, spectral roll-off, spectral flatness, spectral entropy, peak frequency and peak power, and multi-band energy ratio; For the optical waveguide sensor, time-domain statistics, interval integrals, and array-based spatial weighting mappings are extracted. The frequency domain and time-frequency characteristics of the microphone, and the time domain and spatial characteristics of the optical waveguide sensor are standardized and then spliced ​​together to form a joint feature vector; Classifying and regressing the joint feature vector using a machine learning model; Obtaining independent decisions on acoustic and optical subsystem outputs, and then deciding the final output by weighted voting or confidence fusion; Regressing or sequence classifying the fusion features using a time series model; Using the high-frequency energy envelope or wavelet high-frequency coefficients of the microphone to trigger the controller to increase the clamping force of the robot hand or stop the clamping action when the threshold is exceeded; Combining optical distribution energy and acoustic features to determine whether the air extraction condition is met, and then selecting whether to trigger the air extraction device.

Images