A control method, device and system for a robotic arm

Abstract

The present application relates to the field of artificial intelligence technology, and in particular to a control method, device and system for a robotic arm; the control method comprises: acquiring a first data pair within a target area; the first data pair comprises first tactile data and first olfactory data; performing object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result; when the first object recognition result indicates that a target object exists within the target area, determining the object burial state of the target object; determining the working mode of the robotic arm based on the object burial state of the target object; in the present application, by adopting the first data pair for object recognition, the accuracy of the robotic arm recognition intelligent system is improved, so that the robotic arm can select a working mode based on the accurate recognition result, thereby improving the rescue intelligence and rescue accuracy of the robotic arm.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a control method, device and system for a robotic arm. Background Art

[0002] With the development of science and technology, rescue has gradually become scientific and intelligent; rescue agencies widely used in the rescue process include robotic arms, among which the recognition intelligent systems of robotic arms for identifying objects are mostly single sensory recognition, usually based on single visual or single tactile perception recognition; but single object recognition systems have certain shortcomings and deficiencies. For example, single visual perception recognition uses image recognition, and its recognition process has a large amount of information, high power consumption, and lengthy recognition processing time; for another example, due to the limitations of the perception of the surrounding environment during the recognition process, single visual perception recognition is difficult to identify objects in dim light or in an obscured environment, and single tactile perception recognition is difficult to accurately identify objects when multiple objects are superimposed on each other; based on the above-mentioned recognition defects, the robotic arm is unable to make the next work selection or makes an error in the work selection, resulting in low rescue intelligence and rescue accuracy of the robotic arm. Summary of the Invention

[0003] In response to the above-mentioned problems in the prior art, the purpose of this application is to improve the accuracy of the robotic arm recognition intelligent system, so that the robotic arm can select the working mode based on accurate recognition results, thereby improving the rescue intelligence and accuracy of the robotic arm.

[0004] In order to solve the above problems, the present application provides a control method for a robotic arm, comprising:

[0005] Acquire a first data pair within the target area; the first data pair includes first tactile data and first olfactory data;

[0006] performing object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result;

[0007] In a case where the first object recognition result indicates that a target object exists in the target area, determining an object burial state of the target object;

[0008] An operating mode of the robotic arm is determined based on the object burial state of the target object.

[0009] On the other hand, the present application also provides a control device for a robotic arm, comprising:

[0010] An acquisition module, configured to acquire a first data pair within a target area; the first data pair includes first tactile data and first olfactory data;

[0011] an object recognition module, configured to perform object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result;

[0012] a buried state determining module, configured to determine an object buried state of the target object when the first object recognition result indicates that a target object exists in the target area;

[0013] The working mode determination module is used to determine the working mode of the robotic arm based on the object burial state of the target object.

[0014] On the other hand, the present application also provides a robotic arm control system, comprising a robotic arm and a control device, wherein the robotic arm is communicatively connected to the control device;

[0015] The control device is configured to obtain a first data pair within a target area; the first data pair includes first tactile data and first olfactory data; perform object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result; if the first object recognition result indicates that a target object exists within the target area, determine an object burial state of the target object; and determine an operating mode of the robotic arm based on the object burial state of the target object.

[0016] The robotic arm includes a sensor array, the sensors are communicatively connected to the control device, and the sensor array is used to collect a first data pair within a target area and transmit the first data pair to the control device.

[0017] On the other hand, the present application also provides an intelligent recognition device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the control method of the robotic arm as mentioned above.

[0018] On the other hand, the present application also provides a computer storage medium, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the control method of the robotic arm as described above.

[0019] Due to the above technical solution, the control method of a robotic arm described in this application has the following beneficial effects:

[0020] The control method of the robotic arm in the present application obtains data pairs of the target area, where the data pairs include tactile data and olfactory data, and performs object recognition based on the tactile data and olfactory data. It can not only adapt to object recognition in various environments, but also improve the accuracy of the recognition results; when the target object is accurately identified, the burial status of the target object is determined, and the working mode of the robotic arm is determined based on the burial status, thereby improving the intelligence and accuracy of the robotic arm rescue. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] To more clearly illustrate the technical solution of this application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0022] Figure 1 This is a schematic diagram of the structure of a control system for a robotic arm provided in an embodiment of the present application;

[0023] Figure 2 This is a schematic diagram of the output voltage of a pressure sensor in a control system of a robotic arm provided in an embodiment of the present application;

[0024] Figure 3 This is a flow chart of a method for controlling a robotic arm provided in an embodiment of the present application;

[0025] Figure 4 This is a flow chart of an object recognition model in a method for controlling a robotic arm provided in an embodiment of the present application;

[0026] Figure 5 This is a flow chart of object recognition in a method for controlling a robotic arm provided in an embodiment of the present application;

[0027] Figure 6 This is a schematic diagram of a process for determining a buried state in a method for controlling a robotic arm provided in an embodiment of the present application;

[0028] Figure 7 This is a flow chart of a method for controlling a robotic arm according to an embodiment of the present application after selecting a working mode;

[0029] Figure 8 This is a flow chart of a method for controlling a robotic arm according to an embodiment of the present application after selecting a working mode;

[0030] Figure 9 This is a flow chart of safety measures in a method for controlling a robotic arm provided in an embodiment of the present application;

[0031] Figure 10 This is a schematic structural diagram of a control device for a robotic arm provided in an embodiment of the present application;

[0032] Figure 11 This is a hardware structure block diagram of a control method for a robotic arm provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] References to "one embodiment" or "embodiment" herein refer to specific features, structures, or characteristics that may be included in at least one implementation of the present application. Throughout the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "top," and "bottom," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely for ease of description and simplification. They do not indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and are therefore not to be construed as limiting the present application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly specifying the number of the technical features referred to. Thus, a feature designated "first" or "second" may explicitly or implicitly include one or more of the features. Furthermore, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential sequence. It should be understood that such terms are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0035] The following is combined with Figure 1 , introducing a control system of a robotic arm provided in an embodiment of the present application, the system includes a robotic arm 01 and a control device 02, and the robotic arm 01 is communicatively connected to the control device 02; specifically, the control device 02 can be a controller inside the robotic arm 01, or a controller inside a robot body connected to the robotic arm 01, or a cloud control server; the robotic arm 01 and the control device 02 can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0036] In a specific embodiment of the present application, the control device 02 is used to obtain a first data pair within the target area; the first data pair includes first tactile data and first olfactory data; object recognition is performed based on the first tactile data and the first olfactory data to obtain a first object recognition result; when the first object recognition result indicates that there is a target object within the target area, the object burial state of the target object is determined; based on the object burial state of the target object, the working mode of the robotic arm 01 is determined; the robotic arm 01 includes a sensor array, the sensor is communicatively connected to the control device 02, and the sensor array is used to collect the first data pair within the target area and transmit the first data pair to the control device 02.

[0037] In an embodiment of the present application, the robotic arm 01 transmits the data collected during operation to the control device 02, for example, the data pairs including tactile data and olfactory data collected are transmitted to the control device 02; and the control device 02 controls the specific working mode of the robotic arm 01 based on the data collected by the robotic arm 01, so that the robotic arm 01 can perform work applications based on actual application scenarios, thereby improving the intelligence of the robotic arm 01; in addition, the robotic arm 01 in the present application can be used in the rescue process, thereby improving the intelligence and accuracy of mechanical rescue, thereby improving the search and rescue success rate and survival rate.

[0038] It should be noted that the rescue process of the robotic arm 01 includes but is not limited to human rescue and animal rescue; the rescue scenarios include but are not limited to rescue in buried scenarios such as earthquakes, landslides and collapses, rescue in high-concentration harmful scenarios, and rescue in fire scenarios.

[0039] In the embodiment of the present application, after the robotic arm 01 touches the object, information such as the object's local micro-morphology, material hardness, and overall contour can be obtained, and then the object can be accurately identified, thereby improving the intelligence and accuracy of the rescue.

[0040] In a specific embodiment of the present application, the sensor array includes a pressure sensor provided with a hexagonal film, a gas sensor provided with a gas-sensitive material on interdigital electrodes, and a tactile sensor.

[0041] In the embodiment of the present application, the pressure sensor and the gas sensor can both be MESM silicon-based sensors; by adopting MESM silicon-based sensors, the adhesion of the sensor array to the robotic arm 01 is improved, and the sensitivity of the sensor is improved, thereby improving the accuracy of data acquisition.

[0042] In the embodiment of the present application, the tactile sensor can have the same structure as the pressure sensor, that is, a pressure sensor; the tactile sensor can also have a different structure from the pressure sensor, that is, a slip sensor; the tactile sensor can be independent of the pressure sensor, and the tactile sensor can also have the same structure as the pressure sensor; in the preparation process of the flexible sensor, it is necessary to perform silver paste reinforcement treatment and black glue protection treatment on the lead part after wire bonding, and perform silicone protection treatment on the sensitive area; by performing silver paste reinforcement treatment, the stability of the contact is improved, and by performing black glue protection treatment and silicone protection treatment, the damage rate of the contact is reduced.

[0043] In the embodiment of the present application, the curing temperatures of silver paste, black glue and silicone glue are 160° C., 105° C. and 110° C. respectively, and the curing time is 30-40 minutes.

[0044] In an embodiment of the present application, the pressure sensor includes a force-sensitive unit, wherein the force-sensitive unit is composed of a beam-membrane-island structure, wherein the structural thickness of the beam is approximately 4 to 6 μm, and a Wheatstone bridge circuit is integrated on the beam; the membrane includes a polycrystalline silicon membrane, and the thickness of the polycrystalline silicon membrane is approximately 2 μm; that is, the thickness of the beam is 2 to 3 times the thickness of the membrane, which ensures that the pressure sensor has good linearity and improves the sensitivity of the pressure sensor; specifically, when the pressure sensor is subjected to external force, the silicon beam will deform and then bend, thereby causing a change in the resistance value on the beam, resulting in a change in the output voltage value of the Wheatstone bridge; that is, the change in voltage value corresponds to the change in force on the pressure sensor.

[0045] In the embodiment of the present application, the measurement accuracy is improved by providing a hexagonal film and measuring the strain degree of the film.

[0046] In an embodiment of the present application, the Wheatstone bridge circuit can be a Wheatstone full-bridge circuit or a Wheatstone half-bridge circuit; by adopting the Wheatstone full-bridge circuit, the influence of external factors such as temperature and humidity on the resistance value can be eliminated, thereby improving the measurement stability and measurement sensitivity of the force sensor.

[0047] In the embodiment of the present application, the preparation process of the pressure sensor is as follows:

[0048] The front side of the silicon wafer is thermally oxidized, and the first photolithography is performed to determine the position of the varistor; the varistor is prepared by a boron ion implantation process; it is prepared by low-pressure chemical vapor deposition (LPCVD) to form a 0.3μm thick low-stress silicon nitride layer and a 0.8μm thick tetraethoxysilane layer; a second photolithography is performed, and micropore patterning is performed by silicon deep reactive ion etching (RIE); low-pressure chemical vapor deposition (LPCVD) is sequentially deposited to form a 0.2μm thick low-stress silicon nitride layer and a 0.8μm thick tetraethoxysilane layer; deep silicon etching is used to selectively remove the low-stress silicon oxide and tetraethoxysilane deposited at the bottom of the trench. The silane composite layer is etched to expose the bare silicon at the bottom surface of the hole; micropore patterning is performed by silicon deep reactive ion etching (RIE) to deepen the hole to form a pressure-sensing vacuum cavity; 40% potassium hydroxide solution is used to perform lateral bottom etching to complete the release of the cavity between the holes; low-pressure chemical vapor deposition is performed to form a layer of polycrystalline silicon with a thickness of 4μm for sealing the sensor; deeper trench etching is performed by deep silicon etching technology to determine the shape of the cantilever structure; the cantilever structure is released to a suspended state by wet etching with 25% potassium hydroxide aqueous solution; and a layer of 0.1μm thick aluminum film is sputtered to achieve interconnection of the piezoresistive Wheatstone bridge.

[0049] In the embodiment of the present application, the pressure sensor prepared by the above method has high sensitivity and can improve the accuracy of object perception; see the attached Figure 2 The curve represents the test process of the pressure sensor, in which, when the pressure sensor gradually approaches the object to be tested, the figure shows a relatively stable starting signal when the pressure sensor is not in contact with the object (approaching stage); the first data point represents the minimum value of the data, that is, the instantaneous point when the pressure sensor just contacts the object; during the contact process between the pressure sensor and the object (load stage), the output voltage of the pressure sensor gradually increases; the second data point represents the maximum slope point in the load stage; when the pressure sensor maintains contact with the object (holding stage), the output voltage of the pressure sensor remains in a stable range; the third data point represents the maximum value of the data; when the pressure sensor gradually separates from the object (release stage), the output voltage of the pressure sensor decreases.

[0050] In the embodiment of the present application, the tactile sensor and the pressure sensor have the same structure. By setting the tactile sensor and the pressure sensor as the same type of sensors, the manufacturing cost of the above-mentioned system can be effectively reduced. In the case where the tactile sensor is a pressure sensor prepared by the above-mentioned preparation method, the following feature value extraction corresponds to the first data point, the second data point and the third data point.

[0051] In the embodiment of the present application, by providing a gas sensitive material on the interdigital electrodes of the gas sensor, the sensitivity to different gases is improved, and the ability to distinguish different gases is improved.

[0052] In an embodiment of the present application, a plurality of different gas-sensitive materials can be set to modify the cross-value electrodes of the gas sensor. Specifically, different gas-sensitive sensors can be used to modify one gas sensor, or different gas-sensitive sensors can be used to modify multiple gas sensors. This application does not limit this. By setting a plurality of different gas-sensitive materials to modify the cross-value electrodes of the sensor, while constructing a six-channel gas sensor array, the recognition rate of the robot arm 01 for the target object in a variety of gas environments is improved.

[0053] In the embodiment of the present application, the number of tactile sensors is 70 and the number of olfactory sensors is 6; by using a small number of sensors, accurate target identification can be achieved, while reducing the sensor scale and sample size, so that the robot arm 01 can adapt to more complex environments, and improve the application scenario range of the robot arm 01, as well as improve the application response rate of the robot arm 01 under limited resource conditions.

[0054] In an embodiment of the present application, accurate target identification can be achieved by adopting the above-mentioned small number of tactile sensors and olfactory sensors, which reduces the amount of data required for the identification process, further reduces the power consumption of the robotic arm control system, and improves the recognition rate in the robotic arm control system.

[0055] In an embodiment of the present application, the robotic arm 01 includes a robotic palm, and the sensor array is closely attached to the fingertips and palm of the robotic palm; by closely attaching the sensor array to the fingertips and palm of the robotic palm, the robotic arm facilitates data collection while improving the protection capability of the sensors and reducing the possibility of damage to the sensors.

[0056] In the embodiment of the present application, the steps for integrating the sensor into the robotic palm are as follows:

[0057] Design and prepare highly sensitive silicon-based pressure sensors and gas sensors; integrate highly sensitive pressure sensors on flexible circuit boards to prepare flexible tactile sensor arrays; integrate highly sensitive gas sensors on flexible circuit boards to prepare flexible olfactory sensor arrays; integrate tactile sensor arrays and olfactory sensor arrays at the fingertips and palm of a robotic hand.

[0058] The method for integrating a pressure sensor into a flexible circuit board to prepare a flexible tactile sensor specifically includes:

[0059] A single pressure sensor is fixed to a flexible printed circuit board using vinyl. After wire bonding, silver paste is applied to the connection between the pressure sensor and the wire on the flexible printed circuit board to strengthen the connection, and the circuit board is placed on a 160°C hot plate for 30 minutes for curing. Vinyl is applied to the lead bonding area, and the circuit board is placed on a 105°C hot plate for 45 minutes. Silicone is applied to the surface of the pressure sensor and cured at room temperature for 24 hours.

[0060] The method of integrating a gas sensor on a flexible circuit board to prepare a flexible tactile sensor is the same as the method of integrating a pressure sensor on a flexible circuit board to prepare a flexible tactile sensor.

[0061] Preferably, in more complex burial scenarios, for example, a person is buried in rubble, the system in the present application can identify the human body based on olfactory assistance, thereby improving its recognition accuracy in complex environments; specifically, the human leg is buried in a pile of rubble, and the robotic arm 01 performs detection within a preset range to determine the position of the human body and its burial status.

[0062] Combine Figure 2 , introduces a control method for a robotic arm provided in an embodiment of the present application, the method comprising:

[0063] S1. Obtain a first data pair within the target area; the first data pair includes first tactile data and first olfactory data; the target area is the area that the robotic arm needs to detect, and the robotic arm can perform multiple layer-by-layer explorations within the target area. During the exploration process, the tactile sensor collects the first tactile data, and the gas sensor collects the first olfactory data.

[0064] In the embodiment of the present application, the target area is the area where rescue is to be carried out.

[0065] In an embodiment of the present application, the robotic arm can perform touch exploration and data collection within the target area based on a preset trajectory; specifically, a Cartesian space coordinate system can be established, a preset operating trajectory can be set in the Cartesian space coordinate system, and the degree of freedom trajectory can be solved using inverse kinematics, thereby controlling the robotic arm to perform multiple explorations layer by layer.

[0066] S2. Perform object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result. Object recognition refers to performing target object recognition on all objects in the target area to determine whether a target object exists in the target area. Specifically, the first tactile data and the first olfactory data may be used as inputs of an object recognition model to obtain an object recognition result. The object recognition result may be that the target object exists in the target area or that the target object does not exist in the target area. Figure 4The object recognition model is a model obtained by intelligently training the preset neural network based on sample detection data pairs and corresponding object recognition result labels; by adopting the object recognition model to identify the target object from the two dimensions of touch and smell, the recognition accuracy and efficiency are improved.

[0067] In the embodiment of the present application, the target object can be a person or an animal.

[0068] S3. When the first object recognition result indicates that a target object exists in the target area, determine the object burial state of the target object; the presence of the target object in the target area indicates that an object requiring rescue exists in the target area; the object burial state can be that the target object is buried by a non-target object, or that the target object is not buried by the non-target object.

[0069] S4. Determine the working mode of the robotic arm based on the object burial state of the target object.

[0070] In an embodiment of the present application, by acquiring data pairs of the target area, where the data pairs include tactile data and olfactory data, object recognition is performed based on the tactile data and olfactory data, which can not only adapt to object recognition in various environments, but also improve the accuracy of the recognition results; when the target object is accurately identified, the burial state of the target object is determined, and the working mode of the robotic arm is determined based on the burial state, thereby improving the intelligence of the robotic arm rescue.

[0071] Reference Attachment Figure 5 In the embodiment of the present application, S2 includes:

[0072] S201. A feature extraction layer based on an object recognition model performs feature extraction on the first tactile data and the first olfactory data respectively to obtain first tactile feature information and first olfactory feature information; feature extraction refers to extracting feature information from the first tactile data and the first olfactory data.

[0073] In an embodiment of the present application, the feature extraction layer includes a first feature extraction layer and a second feature extraction layer. The first feature extraction layer is used to extract feature values of the first tactile data to obtain first tactile feature information; the second feature extraction layer is used to extract feature values of the first olfactory data to obtain first olfactory feature information.

[0074] In an embodiment of the present application, the first feature extraction layer is a convolutional neural network (CNN), and the second feature extraction layer is a fully connected neural network; the feature extraction layer is similar to the preprocessing area of the star-nosed mole; specifically, by using a convolutional neural network to image the tactile data, based on the small amount of collected tactile data, a two-layer convolutional neural network is used for data preprocessing and feature value extraction, thereby improving the robotic arm's perception of the hardness of the material and the surface micromorphology of the object to be identified.

[0075] In the embodiment of the present application, the tactile sensor on the robotic arm is responsible for collecting the first tactile data, and the gas sensor is responsible for collecting the first olfactory data. The number of channels of the tactile sensor and the gas sensor corresponds to the number of the first tactile data and the first olfactory data, respectively. S201 includes:

[0076] A) performing filtering and scaling processing based on a set of first tactile data and a set of first olfactory data to obtain corresponding multiple tactile scaled data and multiple olfactory scaled data; a set of first tactile data includes multiple first tactile data, wherein each first tactile data corresponds to two data labels, namely, a corresponding collection tactile sensor label, and a corresponding collection time point label; a set of olfactory data includes multiple first olfactory data, wherein each first olfactory data corresponds to two data labels, namely, a corresponding collection olfactory sensor label, and a corresponding collection time point label; the filtering and scaling processing includes filtering processing and scaling processing, wherein the filtering processing refers to filtering out the first tactile data and the first olfactory data outside the preset collection data range; the scaling processing refers to scaling the first tactile data and the first olfactory data after the filtering processing, specifically, adjusting all the filtered data to within the interval [0,1], thereby obtaining multiple tactile scaled data and multiple olfactory scaled data.

[0077] In an embodiment of the present application, the collected tactile data and olfactory data are screened and processed to exclude tactile data and olfactory data that are not within the preset collected data range, thereby reducing the data collection error caused by various factors of the front-end sensor and improving the data validity; by scaling the tactile data and olfactory data, the tactile data and olfactory data are adjusted to within the interval [0,1], thereby simplifying the tactile data and olfactory data, facilitating subsequent algorithm recognition operations, thereby reducing the power consumption of the recognition operation and improving the recognition operation rate.

[0078] B) normalizing the multiple tactile scaling data and the multiple olfactory scaling data respectively to obtain tactile normalized data and olfactory normalized data; normalization processing refers to normalizing the multiple tactile scaling data and the multiple olfactory scaling data in this application respectively to obtain representative tactile normalized data and olfactory normalized data respectively.

[0079] In the embodiment of the present application, the tactile normalization process may adopt the following normalization process method:

[0080] Perform weighted average calculation based on multiple tactile scaling data to obtain maximum baseline data;

[0081] Obtaining minimum baseline data; the minimum baseline data refers to a weighted average of multiple initial tactile data collected by multiple tactile sensors when the robotic arm is not in contact with any object; wherein the multiple initial tactile data include multiple sets of tactile data collected by different sensors, and multiple sets of tactile data collected by the same sensor at different time points;

[0082] Obtain force smoothing data; force smoothing data refers to the data collected by the pressure sensor after entering a smooth state;

[0083] Normalization calculation is performed based on the maximum baseline data, the minimum baseline data and the force smoothing data to obtain the tactile normalized data.

[0084] Specifically, the following normalized calculation formula (1) can be used for calculation:

[0085]

[0086] Among them, U normalized Refers to the tactile normalized data; U′ refers to the force smoothing data; U min Refers to the minimum baseline data; U max Refers to the maximum baseline data.

[0087] In the embodiment of the present application, the normalization processing of the sense of smell can adopt the following normalization processing method:

[0088] Performing weighted average calculation based on multiple olfactory scaling data to obtain olfactory average data;

[0089] Obtain olfactory smoothing data;

[0090] Normalization calculation was performed based on the average olfactory data and the smoothed olfactory data to obtain the olfactory normalized data.

[0091] Specifically, the following normalized calculation formula (2) can be used for calculation:

[0092]

[0093] Among them, R normalized refers to the olfactory normalized data; R′ refers to the olfactory smoothed data; and R0 refers to the olfactory average data.

[0094] In the embodiment of the present application, by respectively adopting the above-mentioned tactile normalization processing and olfactory normalization processing, the accuracy of the normalized data is improved, thereby improving the accuracy of the object recognition result.

[0095] In other embodiments of the present application, the tactile normalization processing method is the same as the olfactory normalization processing method; it can be any one of the above-mentioned normalization processing methods, or it can be other normalization processing methods.

[0096] C) performing feature extraction on the tactile normalized data and the olfactory normalized data respectively to obtain first tactile feature information and first olfactory feature information.

[0097] S203. Determine a first weight corresponding to the first tactile feature information and a second weight corresponding to the first olfactory feature information based on the environmental information in the target area; determine a weight ratio of the tactile and olfactory data based on the environmental information in the target area, which is similar to the signal processing in the connection area of the mole.

[0098] In an embodiment of the present application, the first weight and the second weight can be adjusted in size based on the environmental information in the target area; for example, when the gas concentration in the air in the target area is greater than the preset concentration, the first weight is increased and the second weight is decreased; when the coverage rate of non-target objects in the target area is greater than the preset coverage rate, the first weight is decreased and the second weight is increased.

[0099] In the embodiment of the present application, the environmental information within the target area may be environmental information automatically determined by the control device, or may be environmental information input after human perception.

[0100] S205. Weighting the first tactile feature information based on the first weight to obtain first weighted feature information, and weighting the first olfactory feature information based on the second weight to obtain second weighted feature information;

[0101] S207 : Based on the feature fusion layer of the object recognition model, perform feature fusion on the first weighted feature information and the second weighted feature information to obtain a first object recognition result.

[0102] In an embodiment of the present application, by weighting the first tactile feature information and the first olfactory feature information based on environmental information, and performing feature fusion processing based on the weighted data, the impact of different environments on the recognition results is reduced, thereby improving the accuracy of the recognition results.

[0103] In an embodiment of the present application, the object recognition model includes a model obtained by performing intelligent recognition training on a preset neural network based on sample detection data pairs and corresponding object recognition result labels.

[0104] In the embodiment of the present application, S207 includes:

[0105] 1) Obtain a preset dimension; preferably, the preset dimension is 512.

[0106] 2) Based on the environmental information in the target area, a third weight is determined; the third weight refers to the ratio of the tactile data dimension to the preset fusion length; the preset fusion length is equal to twice the preset dimension.

[0107] 3) weighting the preset dimensions based on the third weight and the preset fusion length to obtain a tactile mapping dimension and an olfactory mapping dimension;

[0108] 4) Perform multimodal fusion calculation based on the first weighted feature information, the second weighted feature information, the tactile mapping dimension, the olfactory mapping dimension and the preset fusion length to obtain a first object recognition result; multimodal fusion calculation refers to MCB calculation, that is, the feature vectors of touch and smell are respectively obtained by mapping functions to obtain feature functions, and then the fused features are obtained based on Fourier transform and inverse Fourier transform; in the embodiment of the present application, by adopting multimodal fusion calculation, the interference of environmental factors in the actual rescue process can be reduced, and the error caused by device damage can be reduced, thereby improving the recognition accuracy of the target object by the robotic arm.

[0109] In the embodiment of the present application, the following formula (3) can be used for weighted calculation:

[0110]

[0111] Among them, T net Refers to the first tactile feature information; net Refers to the first olfactory characteristic information; k T refers to the first weight; k O refers to the second weight; k D refers to the third weight; N refers to the preset dimension; T′ net Refers to the first weighted feature information; O′ net Refers to the second weighted feature information; D T Refers to the tactile mapping dimension; D O Refers to the olfactory mapping dimension.

[0112] In the embodiment of the present application, the multimodal fusion calculation can be specifically referred to the following formula (4):

[0113] I fusion =MCB(T net , O net , n t , n o , d) (4)

[0114] Among them, T netRefers to the first weighted feature information; net Refers to the second weighted feature information; n t refers to the tactile mapping dimension; n o refers to the olfactory mapping dimension; d refers to the preset fusion length.

[0115] Specifically, the multimodal fusion calculation formula can be found in the following formula (5):

[0116] I fusion =FFT -1 (FFT(Ψ(Resize(T net， n t )))⊙FFT(Ψ(Resize(O net , n o )))) (5)

[0117] Among them, FFT refers to Fourier transform; FFT -1 Refers to the inverse Fourier transform; Resize(T net , n t ) refers to the first weighted feature information Tn et The dimension is adjusted to the tactile mapping dimension n t Function; Resize(O net , n o ) refers to the second weighted feature information O net The dimension is adjusted to the olfactory mapping dimension n o ⊙ refers to the vector product operation; Ψ refers to the function that maps the Resize function to the dimension d.

[0118] In the above formula (5), the conditions in formula (6) can always be met:

[0119] length(I fusion )=length(Ψ)=d (6)

[0120] Here, d refers to the preset fusion length.

[0121] In an embodiment of the present application, d is equal to the sum of the first mapping dimension and the second mapping dimension. Preferably, when the first mapping dimension and the second mapping dimension are both 512, the preset fusion length d is 1024.

[0122] In the embodiment of the present application, by adopting the above-mentioned multimodal fusion method, the mutual influence between tactile data and olfactory data is reduced, and the anti-interference ability of the recognition system is improved.

[0123] Reference Attachment Figure 6 In the embodiment of the present application, the first tactile data includes tactile data of multiple channels; S3 includes:

[0124] S301. Perform tactile recognition based on the tactile data of each channel to obtain a tactile recognition result corresponding to each channel. Tactile recognition refers to directly performing recognition based on the tactile data collected by the tactile sensors of each channel to obtain a tactile recognition result. Since the tactile sensations of the target object and the non-target object are not consistent, when it has been determined that the target object exists in the target area, the tactile recognition result can be a tactile recognition result of the target object or a tactile recognition result of the non-target object. The tactile recognition result of the target object indicates that the target object can be directly identified through the tactile data of the channel. The tactile recognition result of the non-target object indicates that the target object cannot be directly identified through the tactile data of the channel.

[0125] In the embodiment of the present application, non-target objects include but are not limited to steel bars, gravel, wood, etc.

[0126] S303. Based on the tactile recognition results corresponding to the respective channels, determine the number of first channels corresponding to the tactile recognition results of the target object and the number of second channels corresponding to the tactile recognition results of the non-target object; the number of first channels is the number of tactile sensors in contact with the target object; and the number of second channels is the number of tactile sensors in contact with the non-target object.

[0127] S305: Determine the object burial state of the target object based on the first channel number and the second channel number.

[0128] In an embodiment of the present application, the burial state can be characterized by a burial degree ratio obtained by performing a ratio operation based on the number of first channels and the number of second channels; based on the burial state, it is only necessary to determine whether the target object is buried, so only the tactile data of each channel can be used to directly perform tactile recognition judgment; by using tactile recognition, the burial state of the target object can be quickly judged, thereby quickly selecting the working mode, thereby improving the working efficiency of the robotic arm.

[0129] In the embodiment of the present application, S4 includes:

[0130] S401. When the target object is in a buried state, determine that the working mode of the robotic arm is a first working mode; the first working mode is used to convert the target object from a buried state to an unburied state; that is, remove non-target objects from the target object's body or surroundings, thereby converting the target object from a buried state to an unburied state.

[0131] Reference Attachment Figure 7 ,Specifically, after S4, the control method of the robotic arm includes:

[0132] S501. When the working mode of the robotic arm is the first working mode, obtain a second data pair; the second data pair includes second tactile data and second olfactory data; the second data pair is obtained when the robotic arm touches the object to be identified; when the robotic arm is in the first working mode, the robotic arm will perform touch identification on non-target objects around the target object, and grasp and transfer the non-target objects, thereby removing the non-target objects from the target object.

[0133] S601. Perform object recognition based on the second tactile data and the second olfactory data to obtain a second object recognition result; object recognition refers to the recognition of the object to be recognized touched by the robotic arm, and determines whether the object to be recognized is a non-target object; specifically, the second tactile data and the second olfactory data can be used as inputs of the object recognition model to obtain an object recognition result; wherein, the object recognition result can be that the robotic arm touches a non-target object, or that the robotic arm touches a target object; the touch recognition model is a model obtained by performing intelligent recognition training on a preset neural network based on sample detection data pairs and corresponding object recognition result labels.

[0134] S701. When the second object recognition result indicates that the robotic arm touches the non-target object, remove the non-target object from the target object; when the robotic arm touches the non-target object, control the robotic arm to grasp the non-target object and transfer the non-target object, thereby removing the non-target object from the target object.

[0135] In this embodiment of the present application, by acquiring a second data pair and performing object recognition based on the second tactile data and second olfactory data in the second data pair, the accuracy of object recognition is improved, thereby enhancing the reliability of the robotic arm. Furthermore, if the second object recognition result indicates that the robotic arm has touched a non-target object, the non-target object can be grasped and removed, thereby improving the practicality of the robotic arm.

[0136] In the embodiment of the present application, S4 further includes:

[0137] S403: If the target object is not buried, the robot arm is set to a second operating mode. The second operating mode is used to move the target object to a predetermined safe area. This includes but is not limited to dragging or lifting the target object by the robot arm to achieve a rescue.

[0138] Reference Attachment Figure 8 Specifically, after S4, the control method of the robotic arm further includes:

[0139] S503. When the working mode of the robotic arm is the second working mode, a third data pair is obtained within the area where the target object is located based on the robotic arm; the third data pair includes third tactile data and third olfactory data; the area where the target object is located is smaller than the target area that the robotic arm needs to detect; when the robotic arm is in the second working mode, the robotic arm will perform touch recognition on the target object, thereby determining the specific part of the target object that the robotic arm contacts.

[0140] S603. Perform part recognition based on the third tactile data and the third olfactory data to obtain the recognized part of the target object; part recognition refers to identifying a specific part of the target object to determine whether the contacted part is the target part; specifically, the third tactile data and the third olfactory data can be used as inputs of a part recognition model to obtain the recognized part; wherein, the recognized part can be characterized as the target part currently contacted by the robotic arm, or as the non-target part currently contacted by the robotic arm; the part recognition model is a model obtained by performing intelligent recognition training on a preset neural network based on sample detection data pairs and corresponding recognition part labels.

[0141] S703. When the identified part is the target part, the position of the target part is calibrated to obtain first calibration information; the first calibration information is used to instruct the target object to be transferred to a preset safe area; the target part refers to a part that cannot be directly dragged, pulled, or other rescue operations, and the first calibration information represents the position of the target part.

[0142] In an embodiment of the present application, the target part refers to a part of the human body or animal, such as the head, that cannot be dragged for rescue. The target part is calibrated to instruct rescue personnel to guide the work of the robotic arm based on the first calibration information, or the rescue personnel perform rescue based on the first calibration information, so that the target object is transferred to a preset safe area; wherein, the work guidance includes but is not limited to predicting the predicted position information of the draggable part, and controlling the robotic arm to explore and identify the predicted position information.

[0143] S803. When the identified part is a non-target part, control the robotic arm to transfer the target object to a preset safe area; the non-target part refers to a part that can be dragged, such as an arm or a leg; preferably, control the robotic arm to drag the non-target part, so that the target object is transferred to the preset safe area.

[0144] Reference Attachment Figure 9 In the embodiment of the present application, after S4, the control method of the robotic arm further includes:

[0145] S505 , obtaining force feedback information of the robotic arm; the force feedback information refers to force data collected and transmitted by the pressure sensors in the sensor array.

[0146] S605: When the force feedback information exceeds a preset threshold, the robotic arm is controlled to stop working, and the position of the robotic arm is calibrated to obtain second calibration information.

[0147] In an embodiment of the present application, force feedback information exceeding a preset threshold indicates that the current force intensity of the robotic arm is too large; wherein, the scenario in which the force intensity is too large may be when carrying a non-target object or when transporting a target object; at this time, the robotic arm is controlled to stop working to avoid further damage to the robotic arm; by calibrating the position of the robotic arm, rescue personnel can perform rescue based on the second calibration information, or they can control other robots or robotic arms based on the second calibration information to assist in the rescue work.

[0148] Combined with attachment Figure 10 , introducing a control device for a robotic arm provided in an embodiment of the present application, the device comprising:

[0149] A first data pair acquisition module 101 is configured to acquire a first data pair within a target area; the first data pair includes first tactile data and first olfactory data;

[0150] An object recognition module 201 is configured to perform object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result;

[0151] A buried state determination module 301 is configured to determine an object buried state of a target object when the first object recognition result indicates that the target object exists in the target area;

[0152] The working mode determination module 401 is configured to determine the working mode of the robot arm based on the object burial state of the target object.

[0153] Among them, the object recognition module includes:

[0154] a feature extraction unit, configured to perform feature extraction on the first tactile data and the first olfactory data based on a feature extraction layer of an object recognition model, to obtain first tactile feature information and first olfactory feature information;

[0155] a weight determination unit, configured to determine, based on environmental information within the target area, a first weight corresponding to the first tactile feature information and a second weight corresponding to the first olfactory feature information;

[0156] a weighted calculation unit, configured to weight the first tactile feature information based on a first weight to obtain first weighted feature information, and to weight the first olfactory feature information based on a second weight to obtain second weighted feature information;

[0157] The data fusion unit is used to perform feature fusion on the first weighted feature information and the second weighted feature information based on the feature fusion layer of the object recognition model to obtain a first object recognition result.

[0158] The burial state determination module includes:

[0159] a tactile recognition unit, configured to perform tactile recognition based on the tactile data of each channel and obtain a tactile recognition result corresponding to each channel;

[0160] a channel number determination unit, configured to determine, based on the tactile recognition results corresponding to the channels, a first channel number corresponding to the target object tactile recognition result and a second channel number corresponding to the non-target object tactile recognition result;

[0161] The burying state determining unit is configured to determine the object burying state of the target object based on the first channel quantity and the second channel quantity.

[0162] The working mode determination module includes:

[0163] The first working mode determining unit is used to determine the working mode of the robot arm to be the first working mode when the target object is in the buried state; the first working mode is used to convert the target object from the buried state to the unburied state.

[0164] The second working mode determining unit is used to determine that the working mode of the robotic arm is the second working mode when the target object is in an unburied state; the second working mode is used to transfer the target object to a preset safe area.

[0165] The robotic arm control device also includes:

[0166] a second data pair acquisition module, configured to acquire a second data pair when the operating mode of the robotic arm is the first operating mode; the second data pair includes second tactile data and second olfactory data; and the second data pair is acquired when the robotic arm touches the object to be identified;

[0167] a second recognition module, configured to perform object recognition based on the second tactile data and the second olfactory data to obtain a second object recognition result;

[0168] The removal working module is used to remove the non-target object from the target object when the second object recognition result indicates that the robotic arm touches the non-target object.

[0169] a third data pair acquisition module, configured to acquire, based on the robotic arm, a third data pair within the area where the target object is located when the robotic arm operates in the second operating mode; the third data pair including third tactile data and third olfactory data;

[0170] a part recognition module, configured to perform part recognition based on the third tactile data and the third olfactory data to obtain a recognized part of the target object;

[0171] A first calibration module is configured to calibrate the position of the target part when the identified part is the target part, and obtain first calibration information; the first calibration information is used to instruct the target object to be transferred to a preset safe area;

[0172] The transfer working module is used to control the robotic arm to transfer the target object to a preset safe area when the identified part is a non-target part.

[0173] A force acquisition module is used to obtain force feedback information of the robotic arm;

[0174] The second calibration module is used to control the robotic arm to stop working when the force feedback information exceeds a preset threshold, and to calibrate the position of the robotic arm to obtain second calibration information.

[0175] An embodiment of the present application also provides an intelligent recognition device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the control method of the robotic arm as described above.

[0176] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. In addition, the memory can include high-speed random access memory and non-volatile memory, such as at least one hard disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0177] The method embodiments provided in the embodiments of the present application can be executed in electronic devices such as mobile terminals, computer terminals, servers or similar computing devices. Figure 11 This is a hardware structure block diagram of an electronic device for a method of controlling a robotic arm provided in an embodiment of the present application. Figure 11As shown, the electronic device 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 910 (the processor 910 may include but is not limited to a microprocessor MCU or a programmable logic device FPG, etc.), a memory 930 for storing data, and one or more storage media 920 (such as one or more mass storage devices) for storing application programs 923 or data 922. Among them, the memory 930 and the storage medium 920 can be temporary storage or permanent storage. The program stored in the storage medium 920 may include one or more modules, each module may include a series of instruction operations in the electronic device. Furthermore, the central processing unit 910 may be configured to communicate with the storage medium 920 to execute a series of instruction operations in the storage medium 920 on the electronic device 900. The electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input and output interfaces 940, and / or one or more operating systems 921, such as Windows Server™, M OS X™, Unix™, Linux™, FreeBSD™, etc.

[0178] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of the electronic device 900. In one embodiment, the input / output interface 940 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices via a base station so as to communicate with the Internet. In one embodiment, the input / output interface 940 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0179] It can be understood by those skilled in the art that Figure 11 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 11 More or fewer components than shown, or with Figure 11 Different configurations shown.

[0180] An embodiment of the present application also provides a storage medium, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the control method of the robotic arm as described above.

[0181] The above description has fully disclosed the specific embodiments of this application. It should be noted that any changes made by those skilled in the art to the specific embodiments of this application do not depart from the scope of the claims of this application. Accordingly, the scope of the claims of this application is not limited to the above specific embodiments.

Claims

1. A method for controlling a robotic arm, characterized in that: include: Acquire a first data pair within the target area; the first data pair includes first tactile data and first olfactory data; performing object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result; In a case where the first object recognition result indicates that a target object exists in the target area, determining an object burial state of the target object; determining a working mode of the robotic arm based on a buried state of the target object; The performing object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result includes: Performing feature extraction on the first tactile data and the first olfactory data based on a feature extraction layer of an object recognition model to obtain first tactile feature information and first olfactory feature information; determining, based on environmental information within the target area, a first weight corresponding to the first tactile feature information and a second weight corresponding to the first olfactory feature information; weighting the first tactile feature information based on the first weight to obtain first weighted feature information, and weighting the first olfactory feature information based on the second weight to obtain second weighted feature information; Based on the feature fusion layer of the object recognition model, feature fusion is performed on the first weighted feature information and the second weighted feature information to obtain the first object recognition result.

2. A method for controlling a robotic arm according to claim 1, characterized in that: The first tactile data includes tactile data of multiple channels; The determining of the object burial state of the target object when the first object recognition result indicates that the target object exists in the target area includes: Performing tactile recognition based on the tactile data of each channel to obtain a tactile recognition result corresponding to each channel; Based on the tactile recognition results corresponding to the channels, determining the number of first channels corresponding to the tactile recognition results of the target objects and the number of second channels corresponding to the tactile recognition results of the non-target objects; An object burial state of the target object is determined based on the first channel number and the second channel number.

3. The control method of a robotic arm according to claim 1, characterized in that: The determining of the working mode of the robotic arm based on the buried state of the target object includes: When the target object is in a buried state, the operating mode of the robot arm is determined to be a first operating mode; the first operating mode is used to convert the target object from the buried state to an unburied state.

4. A method for controlling a robotic arm according to claim 3, characterized in that: After determining the working mode of the robotic arm based on the buried state of the target object, the method further includes: When the working mode of the robotic arm is the first working mode, obtaining a second data pair; the second data pair includes second tactile data and second olfactory data; the second data pair is obtained when the robotic arm touches the object to be identified; performing object recognition based on the second tactile data and the second olfactory data to obtain a second object recognition result; In a case where the second object recognition result indicates that the robotic arm has touched a non-target object, the non-target object is removed from the target object.

5. The control method of a robotic arm according to claim 1, characterized in that: The determining of the working mode of the robotic arm based on the buried state of the target object includes: When the target object is in an unburied state, the operating mode of the robotic arm is determined to be a second operating mode; the second operating mode is used to transfer the target object to a preset safe area.

6. A method for controlling a robotic arm according to claim 5, characterized in that: After determining the working mode of the robotic arm based on the buried state of the target object, the method further includes: When the working mode of the robotic arm is the second working mode, obtaining a third data pair within the area where the target object is located based on the robotic arm; the third data pair includes third tactile data and third olfactory data; performing part recognition based on the third tactile data and the third olfactory data to obtain an identified part of the target object; In the case where the identified part is a target part, calibrating the position of the target part to obtain first calibration information; the first calibration information is used to instruct the target object to be transferred to the preset safe area; In a case where the identified part is a non-target part, the robotic arm is controlled to transfer the target object to the preset safe area.

7. A method for controlling a robotic arm according to claim 3 or 5, characterized in that: The method further comprises: Obtaining force feedback information of the robotic arm; When the force feedback information exceeds a preset threshold, the robotic arm is controlled to stop working, and the position of the robotic arm is calibrated to obtain second calibration information.

8. A control device for a robotic arm, characterized in that: include: An acquisition module, configured to acquire a first data pair within a target area; the first data pair includes first tactile data and first olfactory data; an object recognition module, configured to perform object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result; a buried state determining module, configured to determine an object buried state of the target object when the first object recognition result indicates that a target object exists in the target area; a working mode determination module, configured to determine a working mode of the robotic arm based on a buried state of the target object; The performing object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result includes: Performing feature extraction on the first tactile data and the first olfactory data based on a feature extraction layer of an object recognition model to obtain first tactile feature information and first olfactory feature information; determining, based on environmental information within the target area, a first weight corresponding to the first tactile feature information and a second weight corresponding to the first olfactory feature information; weighting the first tactile feature information based on the first weight to obtain first weighted feature information, and weighting the first olfactory feature information based on the second weight to obtain second weighted feature information; Based on the feature fusion layer of the object recognition model, feature fusion is performed on the first weighted feature information and the second weighted feature information to obtain the first object recognition result.

9. A robotic arm control system, characterized in that: It includes a robotic arm and a control device, wherein the robotic arm is in communication with the control device; The control device is configured to obtain a first data pair within a target area, the first data pair including first tactile data and first olfactory data; perform object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result; and determine an object burial state of the target object when the first object recognition result indicates that a target object exists within the target area; determining a working mode of the robotic arm based on the buried state of the target object; performing object recognition based on the first tactile data and the first olfactory data to obtain a first object recognition result, including: performing feature extraction on the first tactile data and the first olfactory data based on a feature extraction layer of an object recognition model to obtain first tactile feature information and first olfactory feature information; determining a first weight corresponding to the first tactile feature information and a second weight corresponding to the first olfactory feature information based on environmental information in the target area; weighting the first tactile feature information based on the first weight to obtain first weighted feature information, and weighting the first olfactory feature information based on the second weight to obtain second weighted feature information; Based on the feature fusion layer of the object recognition model, perform feature fusion on the first weighted feature information and the second weighted feature information to obtain the first object recognition result; The robotic arm includes a sensor array, which is communicatively connected to the control device. The sensor array is used to collect a first data pair within a target area and transmit the first data pair to the control device.

10. A robotic arm control system according to claim 9, characterized in that: The sensor array includes a pressure sensor provided with a hexagonal film, a gas sensor provided with a gas sensitive material on interdigital electrodes, and a tactile sensor.

11. A robotic arm control system according to claim 9, characterized in that: The robotic arm includes a robotic palm, and the sensor array is closely attached to the fingertips and palm of the robotic palm.

Images