Dexterous manipulator, target detection system

Abstract

The application provides a dexterous manipulator and a target detection system. The dexterous manipulator comprises a mechanical body and a spectrum measurement device arranged on the mechanical body. The spectrum measurement device comprises an optoelectronic interaction module, a backend module and a flexible composite transmission link. The optoelectronic interaction module is arranged on a plurality of fingers. The backend module is arranged on a palm, a wrist or an arm. The flexible composite transmission link is arranged along a finger joint. The backend module comprises a light source and a control and data processing module. The light source is used for emitting light of a specific wave band. The optoelectronic interaction module is used for guiding the light emitted by the light source to a target object and performing optoelectronic conversion on the reflected and / or transmitted light of the target object to output an electric signal. The control and data processing module is used for driving the light source to emit light and receiving the electric signal, and processing and analyzing the signal. The application can effectively improve the multi-mode spectrum sensing capability of the dexterous manipulator.

Description

Technical Field

[0001] This invention relates to the field of intelligent control, and in particular to a dexterous robotic arm and a target detection system. Background Technology

[0002] In the fields of intelligent manufacturing, logistics sorting, and service robots, dexterous manipulators, as the end effectors of robots, directly determine the precision and intelligence level of their operations through their perception capabilities. As a key component for performing precise operations, the development of dexterous manipulators has evolved from early simple grasping to complex systems requiring multimodal perception and intelligent decision-making capabilities. Currently, mainstream perception solutions in the industry are mainly built around two dimensions: "vision" and "tactile feedback." On the one hand, force or torque sensors and tactile arrays integrated into the fingertips or palm are used to perceive contact forces, slippage, and texture; on the other hand, external 2D or 3D vision systems (such as cameras and LiDAR) are used to acquire the geometric shape, category, and pose information of objects in the work environment. The combination of vision and touch greatly improves the adaptability and precision of robot operations.

[0003] However, as applications expand to more complex and refined scenarios (such as precision sorting, personalized services, and precision assembly), the aforementioned sensing system reveals a fundamental limitation: a lack of direct detection capability of the inherent physical and chemical properties of objects themselves. For example, in recycling and sorting, distinguishing between similar-looking plastic and resin products; in textile operations, differentiating between silk and synthetic fabrics; or in agricultural product processing, non-destructively determining the sugar content and ripeness of fruits, existing dexterous hands are essentially in a state of "material blindness." Summary of the Invention

[0004] This invention provides a dexterous manipulator and a target detection system to enhance the multi-mode spectral sensing capability of the dexterous manipulator without sacrificing the manipulator's degrees of freedom, compactness, and reliability.

[0005] On one hand, embodiments of the present invention provide a dexterous robotic hand, including a mechanical body and a spectral measurement device disposed on the mechanical body; the mechanical body includes fingers, an arm, and a palm; the spectral measurement device includes: a photoelectric interaction module, a back-end module, and a flexible composite transmission link connecting the photoelectric interaction module and the back-end module; the photoelectric interaction module is disposed on multiple fingers; the back-end module is disposed on the palm, wrist, or arm; the flexible composite transmission link is arranged along the finger joints; the back-end module includes: a light source and a control and data processing module;

[0006] The light source is used to emit light in a specific wavelength band;

[0007] The photoelectric interaction module is used to guide the light emitted by the light source to the target object, and to perform photoelectric conversion on the light reflected and / or transmitted by the target object, and output an electrical signal.

[0008] The control and data processing module is used to drive the light source to emit light, and to receive the electrical signal through the flexible composite transmission link, and to process and analyze the electrical signal.

[0009] Optionally, the flexible composite transmission link includes an optical fiber link and an electrical signal link;

[0010] The optical fiber link is used to transmit the light emitted by the light source to the optoelectronic interaction module on each finger;

[0011] The electrical signal link is used to transmit the electrical signals generated by the optoelectronic interaction module to the control and data processing module.

[0012] Optionally, the light in the specific wavelength band includes any one of the following: visible light and near-infrared light;

[0013] The light source includes one or more light-emitting devices, which include any one or more of the following: superluminescent light-emitting diodes, LED arrays, and laser devices.

[0014] Optionally, the optoelectronic interaction module includes a photoelectric detection module; or the optoelectronic interaction module includes a photoelectric detection module and an optical path module.

[0015] An optical path module is used to provide an optical path window and channel to collimate the light emitted by the light source and illuminate the target object.

[0016] The photoelectric detection module is used to receive light reflected or transmitted from the target object, and to perform photoelectric conversion on the received light to output an electrical signal.

[0017] Optionally, the photoelectric interaction module on at least one finger includes the optical path module and the photoelectric detection module.

[0018] Optionally, the control and data processing module includes:

[0019] The control unit is used to provide drive signals to the light source;

[0020] A data processing unit is used to sample and analyze the electrical signal.

[0021] Optionally, the data processing unit is further configured to determine the validity of the measurement state based on the electrical signal and auxiliary information, and if the measurement state is determined to be invalid, mark the current frame data and / or provide an error message.

[0022] Optionally, the back-end module further includes a power module for providing operating power.

[0023] Optionally, a light-transmitting protective layer is provided on the outer side of the area where the optoelectronic interaction module is located on the finger.

[0024] Optionally, the control and data processing module is also used to adaptively switch measurement modes, including: reflection measurement mode and transmission measurement mode.

[0025] Optionally, the control and data processing module is further configured to determine the light intensity of the current transmitted light based on the electrical signal in the transmission measurement mode, and adaptively adjust the sampling parameters based on the light intensity.

[0026] On the other hand, embodiments of the present invention also provide a target detection system based on the dexterous manipulator, characterized in that the system comprises:

[0027] A mode selection module is used to select a measurement mode, which includes: a reflection measurement mode and a transmission measurement mode;

[0028] The control module is used to control a single finger or two or more adjacent fingers of the dexterous manipulator to align with a target object in the reflection measurement mode; and to control different fingers of the dexterous manipulator to clamp the target object in the transmission measurement mode.

[0029] Optionally, in the reflection measurement mode, the control and data processing module in the dexterous manipulator determines the spectral characteristics and / or material characteristics of the target object based on the electrical signal; in the transmission measurement mode, the control and data processing module in the dexterous manipulator determines the target object itself or its internal components based on the electrical signal.

[0030] Optionally, in the transmission measurement mode, the target object is any one of the following: a thin film, a semi-transparent fluid, or a non-uniform object.

[0031] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0032] The dexterous manipulator provided in this invention avoids physical limitations through spatial decoupling deployment, optimizes the signal transmission chain through photoelectric conversion, and supports multi-mode measurement by utilizing the inherent characteristics of the body's mechanical configuration. Thus, it achieves an effective unification of high-performance spectral sensing and dexterous hand movement capabilities at the system level. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of a dexterous robotic arm provided in an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of the spectral measurement device in a dexterous robotic arm according to an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of a target detection system based on a dexterous manipulator provided in an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of signal transmission under the reflection measurement mode in an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of signal transmission in the transmission measurement mode of an embodiment of the present invention. Detailed Implementation

[0038] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] Spectroscopic techniques, by analyzing the interaction between matter and light (such as absorption, reflection, transmission, and fluorescence), can obtain unique information about the chemical composition and microstructure of substances, and have been widely used in fields such as food safety, materials science, and biomedicine. Integrating spectrometers into dexterous hands to enable them to identify substances is considered a key direction for the evolution of the next generation of dexterous hands.

[0040] However, integrating spectral sensing functionality into a smart hand for practical application faces significant technical challenges, mainly in the following aspects:

[0041] (1) System integration contradictions are prominent: High-performance spectrometers (such as grating type and Fourier transform type) are usually bulky and heavy, and are extremely sensitive to thermal stability and mechanical vibration, making them impossible to directly embed inside the fingers of a dexterous hand with extremely limited space and accompanied by motor heating and high-frequency movement. Meanwhile, the highly integrated and chip-based spectral sensors currently cannot meet the identification requirements of complex substances in industrial applications in terms of key indicators such as measurement bandwidth, resolution and signal-to-noise ratio.

[0042] (2) Poor motion stability and reliability: To avoid the size problem, existing attempts often adopt a compromise solution, which is to place the main body of the desktop spectrometer on the robot base or external cabinet, and guide the optical path to the fingertip probe through one or more exposed thin optical fiber links. In this way, the optical fiber links will inevitably be bent, twisted and stretched repeatedly during the multi-joint, wide-range grasping movements of the dexterous hand, resulting in drastic fluctuations in light flux, signal baseline drift and poor repeatability of measurement results. At the same time, the exposed optical fiber links are easily hooked, worn or even broken by objects in the environment, which seriously impairs the mechanical robustness and long-term operational reliability of the system.

[0043] (3) Limited sensing mode and insufficient adaptability: Existing integrated solutions typically only place a reflective probe on a single fingertip, with fixed measurement angles and modes. This cannot meet the needs of comprehensive detection of complex objects. For example, for translucent bodies, thin films, or fluids, transmission measurement modes often provide richer internal information; for non-uniform samples, multi-angle measurements are required to eliminate surface characteristic interference. The single mode greatly limits the application potential of this technology in diverse scenarios.

[0044] To address this, embodiments of the present invention provide a dexterous robotic hand and a target detection system. Employing a separate optoelectronic and distributed front-end / back-end architecture, the system stably integrates high-performance spectral material sensing capabilities into the dexterous robotic hand, enhancing its multi-mode spectral sensing ability. This allows for multi-finger coordinated spectral measurements using the dexterous robotic hand.

[0045] Simultaneously refer to Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of a dexterous robotic arm provided in an embodiment of the present invention. Figure 2 A schematic diagram of the spectral measurement device in a dexterous robotic arm according to an embodiment of the present invention.

[0046] The dexterous robotic hand 100 includes a mechanical body 10 and a spectral measurement device mounted on the mechanical body 10. The mechanical body 10 includes fingers 11, an arm 12, and a palm 13.

[0047] The spectral measurement device includes: a photoelectric interaction module 21, a back-end module 22, and a flexible composite transmission link 23 connecting the photoelectric interaction module 21 and the back-end module 22; the photoelectric interaction module 21 is disposed on multiple fingers, such as at the fingertips; the back-end module 22 is disposed on the palm 13, wrist, or arm 12, such as on the back of the palm 13 or inside the arm 12. The flexible composite transmission link 23 is arranged along the finger joints. To avoid interference between the flexible composite transmission link 23 and other objects or devices in the working environment, the flexible composite transmission link 23 can be disposed inside or superficially on the fingers 11. The back-end module 22 may include: a light source 221 and a control and data processing module 222.

[0048] In this embodiment, the flexible composite transmission link 23 includes an optical fiber link 231 and an electrical signal link 232. The optical fiber link 231 serves as a light guiding channel and can be implemented using, but is not limited to, low-loss flexible silica optical fiber. The electrical signal link 232 serves as an electrical signal transmission channel and can specifically be implemented using thin cable or flexible circuitry.

[0049] Light source 221 is used to emit light in a specific wavelength band. The light in the specific wavelength band may include, for example, any of the following: visible light, near-infrared light, etc. Light source 221 may include one or more light-emitting devices, such as any one or more of the following: superluminescent diodes, LED arrays, laser devices, etc.

[0050] Accordingly, the fiber optic link 231 transmits the light emitted by the light source 221 to the optoelectronic interaction module 21.

[0051] The photoelectric interaction module 21 guides the light emitted by the light source 221 to the target object, and performs photoelectric conversion on the light reflected and / or transmitted by the target object, and outputs an electrical signal.

[0052] The control and data processing module 222 drives the light source 221 to emit light and receives the electrical signal through the electrical signal link 232 in the flexible composite transmission link 23, and processes and analyzes the electrical signal.

[0053] Depending on the actual detection requirements, some photoelectric interaction modules 21 on fingers 11 may only include a photoelectric detection module 211, while others may include both a photoelectric detection module 211 and an optical path module 212. Wherein:

[0054] The optical path module 212 provides an optical path window and channel to collimate the light emitted by the light source 221 and illuminate the target object. Specifically, the optical path module 212 needs to construct an outgoing optical path and / or a receiving optical path, and guide, shape, collimate, or focus the light emitted by the light source to illuminate the target object; and / or guide the reflected light and transmitted light from the target object to the photoelectric detection module 211.

[0055] The photoelectric detection module 211 is used to receive light reflected or transmitted from the target object, and to perform photoelectric conversion on the received light to output an electrical signal. This electrical signal is transmitted to the control and data processing module 222 via the electrical signal link 232.

[0056] In this embodiment of the invention, the end of the optical fiber link 231 is connected to the optical path module 212. The optical path module 212 can work with optical devices such as microlenses to collimate the light transmitted from the optical fiber link 231 and illuminate the target object. The photoelectric detection module 211 can use a germanium-silicon detector to directly receive the light reflected or transmitted from the target object and immediately perform photoelectric conversion on it, converting the optical signal into a corresponding current or voltage signal.

[0057] In practical implementation, an optical path module 212 and a photoelectric detection module 211 can be set simultaneously in the photoelectric interaction module 21 on at least one finger in order to better adapt to different application environments and target objects and meet different measurement needs.

[0058] For example, when it is necessary to measure the reflected light of a target object, a single finger equipped with both a photoelectric detection module 211 and an optical path module 212 can be used to project light onto the target object and collect the reflected light. When it is necessary to measure the transmitted light of a target object, the photoelectric interaction module 21 on two fingers can be used to project light onto the target object and collect the transmitted light. Therefore, the specific configuration of the photoelectric interaction module 21 on each finger can be determined as needed, and this embodiment of the invention does not limit this.

[0059] In some embodiments, the control and data processing module 222 may include: a control unit and a data processing unit. Wherein:

[0060] The control unit is used to provide a driving signal to the light source 221;

[0061] The data processing unit is used to sample and analyze the electrical signals transmitted via the electrical signal link 232. For example, it can perform acquisition, amplification, analog-to-digital conversion, calibration compensation, feature extraction, spectral and / or material information analysis, and adaptive adjustment of integration time and / or analog gain on the electrical signals. It can also use the signals as reference quantities for subsequent light intensity normalization processing. This unit can be implemented using analog front-end circuitry in conjunction with a main control chip, and may further include an FPGA (Field-Programmable Gate Array), DSP (Digital Signal Processing), edge computing chip, or dedicated spectral calculation chip. The specific choice depends on the actual application requirements, and this embodiment of the invention does not limit this. For example, in a non-limiting application, the data processing unit can determine the spectral and / or material characteristics of a target object through sampling analysis.

[0062] In some embodiments, the data processing unit may further determine the validity of the measurement state based on the electrical signal and some auxiliary information. If the measurement state is determined to be invalid, the current frame data may be marked and / or an error message may be displayed. For example, a low-quality identifier may be added to the current frame data, which may be a binary label, a quality score, or a confidence parameter. Sampled frames marked as low-quality may be directly discarded, reduced in weight for fusion, or trigger a resampling process in subsequent processing.

[0063] The data processing unit can determine the validity of the measurement state based on the electrical signal output by the photoelectric detection module 211 and optional pose, force, or visual auxiliary information. For example, if the electrical signal determines that the returned light intensity is lower than a preset minimum effective threshold, reaches a saturation threshold, or fluctuates by more than a preset proportion within multiple consecutive sampling periods, it is determined to be an abnormal light intensity; if the measurement geometry estimated based on the finger joint position, end pose, or differences in response from multiple receiving channels deviates from the calibration range, it is determined to be a measurement angle deviation; if there is a sudden drop in returned light, local channel imbalance, or an abnormal response inconsistent with the contact state, it is determined that there is obstruction.

[0064] In some embodiments, the back-end module 22 may further include a miniature spectrometer, which can be used for system calibration, reference measurement, or auxiliary analysis.

[0065] The power required for each of the above modules can be provided by an external power source or by an internal power module located in the back-end module. This embodiment of the invention does not limit the specific power supply required for these modules.

[0066] In some embodiments, a light-transmitting protective layer may be provided on the outside of the area where the photoelectric interaction module 21 is located on the finger (such as the fingertip). This protective layer may be made of a highly wear-resistant and scratch-resistant material (such as sapphire glass) and cover the photosensitive area of ​​the photoelectric interaction module 21 to provide physical isolation and protection for the internal components.

[0067] In some embodiments, Figure 2 The control and data processing module 222 can also adaptively switch measurement modes, including a reflection measurement mode and a transmission measurement mode. Further, in the transmission measurement mode, the current transmitted light intensity can be determined based on the electrical signal, and the sampling parameters can be adaptively adjusted based on the light intensity. For example, the data processing unit calculates the current transmitted light intensity based on the received electrical signal, and the control unit or analog front-end configuration unit adjusts the exposure time and / or the analog gain of the preamplifier in the photoelectric detection module 211. For instance, when the transmitted light intensity is below a lower threshold, the exposure time and / or the analog gain of the preamplifier in the photoelectric detection module 211 are increased; when the transmitted light intensity is above an upper threshold, the exposure time and / or the analog gain are decreased.

[0068] The dexterous manipulator provided in this invention physically decouples the spectral detection function and, through a distributed layout structure of separate optoelectronic components, arranges a small optoelectronic interaction module 21 on the front finger, while the rear module is arranged in relatively spacious areas with small range of motion, such as the back of the hand, wrist, or arm, to accommodate larger, higher-power, or more stable functional units in the system. The modules in the rear module can be integrated into a single physical module, or they can be multiple separate but functionally coordinated modules, and these different functional modules can be distributed in different areas. This structural design resolves the contradiction between high-performance spectral detection and the extremely small space and high dynamic motion of the dexterous hand, while maintaining the original degrees of freedom of the dexterous manipulator. Furthermore, flexible fiber optic links and data lines can be arranged inside or on the surface of the finger to transmit light from the rear light source to the fingertip and transmit the photoelectric signal captured by the fingertip back to the rear processing unit, ensuring stable signal transmission under multi-degree-of-freedom bending motion of the finger and overcoming the defects of traditional external fiber optic links that are prone to loss and snagging.

[0069] Compared with existing dexterous robotic arms, the dexterous robotic arm provided in this embodiment of the invention has the following advantages:

[0070] (1) Spatial decoupling and distributed deployment of functional modules: The spectral sensing device is physically decomposed into three functionally defined sub-modules—including a back-end module for the light source and control and data processing modules, a flexible composite transmission link, and a front-end optoelectronic interaction module. By deploying the high-power, large-volume light source and control and data processing modules in the relatively smooth back of the hand or arm area, and integrating miniaturized optoelectronic detection and light projection components only at the fingertips, the system complexity and fingertip spatial constraints are decoupled. This deployment method, in principle, allows the use of high-performance, non-miniaturized spectral components, ensuring a baseline for sensing performance.

[0071] (2) Optimization of Optical-to-Electrical Conversion and Signal Transmission: In the optical path design, the illumination light is emitted from the rear light source, transmitted to the fingertip via a low-loss optical fiber link, and illuminates the target object; the optical signal carrying the target object information is immediately converted into an electrical signal by the photodetector integrated in the fingertip. This design transforms the long-distance transmission of optical signals that are easily affected by bending and vibration into the transmission of relatively stable electrical signals, significantly reducing the optical signal drift and loss caused by the deformation of the optical fiber link during the movement of the dexterous manipulator, and improving the dynamic measurement stability and robustness of the system.

[0072] (3) Based on the multi-finger configuration, the system natively supports multimodal sensing: Utilizing the independent movement of multiple fingers of the dexterous robotic hand, light source output terminals and photodetectors can be arranged at the fingertips of different fingers. From the perspective of optical measurement principles, this layout naturally constitutes a variable optical measurement path. By controlling the finger pose, multiple spectral acquisition modes such as reflection measurement (using the optical path of a single finger or adjacent finger) and transmission measurement (relative to the optical path of two fingers) can be flexibly realized, breaking through the limitations of single-point fixed probes in measurement modes and enhancing the adaptability to complex objects being measured.

[0073] Accordingly, embodiments of the present invention also provide a target detection system based on the above-described dexterous manipulator, such as... Figure 3 The diagram shown is a structural schematic of the target detection system.

[0074] The target detection system 300 includes: a mode selection module 301 and a control module 302. Wherein:

[0075] The mode selection module 301 is used to select a measurement mode, which includes a reflection measurement mode and a transmission measurement mode.

[0076] The control module 302 is used to control a single finger or two or more adjacent fingers of the dexterous manipulator 100 to align with a target object in the reflection measurement mode; and to control different fingers of the dexterous manipulator 100 to clamp the target object in the transmission measurement mode.

[0077] It should be noted that the aforementioned mode selection module 301 can respond to user commands to select a measurement mode, which can be implemented by software and / or hardware, and this embodiment of the invention does not limit this. For example, it can be a voice recognition module, a function selection button, or a switch, etc.

[0078] The control module 302 can also be integrated into the back-end module 22 of the dexterous manipulator 100, either as an independent physical module or integrated into a physical module with the control and data processing module 222.

[0079] The target detection system provided in this invention utilizes the multi-finger coordination of a dexterous robotic arm to easily achieve two different spectral measurement modes. The following describes the process in conjunction with... Figure 4 and Figure 5 The working principles of these two different spectral measurement modes will be explained in detail.

[0080] like Figure 4 The diagram shown is a schematic diagram of signal transmission in the reflection measurement mode of an embodiment of the present invention.

[0081] This example uses a photoelectric interaction module 21 mounted on a single finger 11 for illustration. The photoelectric interaction module 21 includes both a collimating optical port and a photodetector. The finger 11 performs illumination and received reflected light. A rear light source 221 is transmitted via flexible optical fiber to the light-emitting end of the finger 11 and illuminates the surface of the target object 20. The reflected light from the surface of the target object 20 is received by the photodetector on the finger 11.

[0082] Figure 4 The example shown uses a single finger to complete both illumination and reflected light reception. It should be noted that in reflectance measurement mode, the above measurement process can also be completed by two or more adjacent fingers. For example, the tip of the first finger can be used as the light emitter, and the tip of the second finger can be used as the photodetector. The first finger emits illumination light towards the surface of the target object, and the second finger receives the light reflected from the surface. For ease of description and distinction, the former can be called same-finger reflectance measurement, and the latter can be called adjacent-finger reflectance measurement.

[0083] In the reflection measurement mode, the control and data processing module in the dexterous manipulator can determine the spectral characteristics and / or material characteristics of the target object 20 based on the electrical signal output from the front end. The specific methods for spectral analysis and material determination are not limited in this embodiment of the invention, and any related technologies can be used.

[0084] like Figure 5 The diagram shown is a schematic diagram of signal transmission in the transmission measurement mode of an embodiment of the present invention.

[0085] In this example, the first finger 1 and the second finger 2 (or multiple fingers working together) clamp the target object 20, so that the target object 20 is located between the two fingers.

[0086] The light emitted by the light source 221 in the back-end module 22 is transmitted through optical fiber to the light output port of the first optoelectronic interaction module 21-1 on the first finger 1 and then illuminates the surface of the target object 20. After the illumination light penetrates the target object 20, it is received by the photodetector in the second optoelectronic interaction module 21-2 on the second finger 2. The electrical signal output by the photodetector is transmitted to the control and data processing module 222 in the back-end module 22, and the data processing module 222 processes and analyzes the electrical signal.

[0087] Transmission measurement mode is particularly suitable for scenarios involving thin films, semi-transparent fluids, non-uniform objects, or where the internal composition needs to be detected. In other words, in such applications, the target object 20 can be, but is not limited to, any of the following: thin films, semi-transparent fluids, non-uniform objects, etc.

[0088] Once the light intensity falls within the preset dynamic range, formal sampling begins. The control and data processing module 222 can normalize and correct the transmission measurement results by combining the dark current baseline and the reference channel signal. The reference channel signal can originate from a photodetector installed at the transmitter for monitoring, an idle reference measurement, or pre-calibrated data.

[0089] In some embodiments, the control and data processing module 222 can also determine the degree of optical path clamping and stability during the transmission measurement process. The degree of clamping can be determined based on the distance between the fingers, joint displacement, contact pressure, clamping force, or estimated driving current; the stability can be determined based on the fluctuation amplitude of transmitted light intensity, contact force fluctuation, change in finger distance, and change in end-effector posture over multiple consecutive sampling periods. The system only outputs the current transmission measurement result as valid when the above indicators meet preset threshold conditions.

[0090] In an optional implementation, the control and data processing module 222 can also control the switching between reflection and transmission modes. For example, the system can default to performing reflection measurement first to obtain the initial response information of the target object; subsequently, the control and data processing module 222 determines whether the target object is suitable for transmission measurement based on the transmission light presampling results, the estimated distance or displacement of the pinched fingers, contact force information, and one or more of the optional visual recognition results or prior features of the reflection spectrum. The determination can be implemented using threshold rules, weighted scoring, or classification models, without requiring all conditions to be met simultaneously. When the determination result indicates that the target object meets the transmission measurement conditions, the control and data processing module 222 switches the corresponding light source emission channel, receiving channel, and sampling parameters, and outputs a transmission measurement request to the control module 302; the control module 302 then controls the corresponding finger to move to the transmission measurement posture. If the continuous presampling results during the transmission measurement process are lower than the minimum effective threshold, or the optical path stability does not meet the requirements, the control and data processing module 222 outputs a mode rollback request, and the control module 302 controls the robot to roll back to the reflection measurement posture.

[0091] Therefore, the above configuration can make full use of the fingertip minimization integration design to ensure the rapid acquisition of high signal-to-noise ratio reflection or transmission spectral data in the grasping or light-touching posture. It can realize the acquisition and analysis of reflected or transmitted light without significantly increasing the volume of a single fingertip, and thus realize surface material detection.

[0092] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0093] In this application's embodiments, "multiple" refers to two or more. The descriptions of "first," "second," etc., appearing in this application's embodiments are merely illustrative and for distinguishing the described objects; they have no order and do not indicate a specific limitation on the number of devices in this application's embodiments, nor do they constitute any limitation on the embodiments of this application.

[0094] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A dexterous robotic hand, characterized in that, The device includes a mechanical body and a spectral measurement device mounted on the mechanical body. The mechanical body includes fingers, an arm, and a palm. The spectral measurement device includes a photoelectric interaction module, a back-end module, and a flexible composite transmission link connecting the photoelectric interaction module and the back-end module. The photoelectric interaction module is mounted on multiple fingers. The back-end module is mounted on the palm, wrist, or arm. The flexible composite transmission link is arranged along the finger joints. The back-end module includes a light source and a control and data processing module. The light source is used to emit light in a specific wavelength band; The photoelectric interaction module is used to guide the light emitted by the light source to the target object, and to perform photoelectric conversion on the light reflected and / or transmitted by the target object, and output an electrical signal. The control and data processing module is used to drive the light source to emit light, and to receive the electrical signal through the flexible composite transmission link, and to process and analyze the electrical signal.

2. The dexterous robotic hand according to claim 1, characterized in that, The flexible composite transmission link includes an optical fiber link and an electrical signal link; The optical fiber link is used to transmit the light emitted by the light source to the optoelectronic interaction module on each finger; The electrical signal link is used to transmit the electrical signals generated by the optoelectronic interaction module to the control and data processing module.

3. The dexterous robotic hand according to claim 1, characterized in that, The specific wavelength band of light includes any one of the following: visible light and near-infrared light; The light source includes one or more light-emitting devices, which include any one or more of the following: superluminescent light-emitting diodes, LED arrays, and laser devices.

4. The dexterous robotic hand according to claim 1, characterized in that, The optoelectronic interaction module includes an optoelectronic detection module; or the optoelectronic interaction module includes an optoelectronic detection module and an optical path module. An optical path module is used to provide an optical path window and channel to collimate the light emitted by the light source and illuminate the target object. The photoelectric detection module is used to receive light reflected or transmitted from the target object, and to perform photoelectric conversion on the received light to output an electrical signal.

5. The dexterous robotic hand according to claim 4, characterized in that, The photoelectric interaction module on at least one finger includes the optical path module and the photoelectric detection module.

6. The dexterous robotic hand according to claim 1, characterized in that, The control and data processing module includes: The control unit is used to provide drive signals to the light source; A data processing unit is used to sample and analyze the electrical signal.

7. The dexterous robotic hand according to claim 6, characterized in that, The data processing unit is further configured to determine the validity of the measurement state based on the electrical signal and auxiliary information, and to mark and / or provide an error message for the current frame data if the measurement state is determined to be invalid.

8. The dexterous robotic hand according to claim 1, characterized in that, The back-end module also includes a power module for providing operating power.

9. The dexterous robotic hand according to claim 1, characterized in that, A light-transmitting protective layer is set on the outside of the area where the optoelectronic interaction module is located on the finger.

10. The dexterous manipulator according to any one of claims 1 to 7, characterized in that, The control and data processing module is also used to adaptively switch measurement modes, including: reflection measurement mode and transmission measurement mode.

11. The dexterous robotic hand according to claim 10, characterized in that, The control and data processing module is also used to determine the light intensity of the current transmitted light based on the electrical signal in the transmission measurement mode, and to adaptively adjust the sampling parameters based on the light intensity.

12. A target detection system based on the dexterous manipulator according to any one of claims 1 to 9, characterized in that, The system includes: A mode selection module is used to select a measurement mode, which includes: a reflection measurement mode and a transmission measurement mode; The control module is used to control a single finger or two or more adjacent fingers of the dexterous manipulator to align with a target object in the reflection measurement mode; and to control different fingers of the dexterous manipulator to clamp the target object in the transmission measurement mode.

13. The target detection system according to claim 12, characterized in that, In the reflection measurement mode, the control and data processing module in the dexterous manipulator determines the spectral characteristics and / or material characteristics of the target object based on the electrical signal; In the transmission measurement mode, the control and data processing module in the dexterous manipulator determines the target object itself or its internal components based on the electrical signal.

14. The target detection system according to claim 12, characterized in that, In the transmission measurement mode, the target object is any one of the following: a thin film, a semi-transparent fluid, or a non-uniform object.

Images