Chemical information driven intelligent detection method and system for body

Abstract

The invention provides a physical intelligence detection method and system driven by chemical information, and relates to the technical field of physical intelligence and chemical sensing. Comprising a physical sensing module, a chemical sensing module, an intelligent acquisition module, a multi-modal fusion analysis module, a tool execution module and a global optimization module, the physical sensing module is used for positioning a target, the micro chemical sensing module is used for acquiring multi-modal spectral information, the intelligent acquisition module is used for optimizing parameters and preprocessing data, the multi-modal fusion analysis module is used for realizing cross-modal feature coupling and large model aided decision making, and the body execution module is used for completing sampling and response. And accurate positioning, molecular-level identification and intelligent decision making of the target substance in a complex environment are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of embodied intelligence and chemical sensing technology, in particular, to a chemical information driven embodied intelligence detection method and system. BACKGROUND

[0002] Embodied intelligence, as a frontier cross-direction of artificial intelligence and robotics, emphasizes that the intelligent agent forms a closed-loop optimization in autonomous perception, cognitive decision-making and behavior execution through real-time interaction with the physical entity and the environment. Existing embodied intelligence systems have shown strong ability in visual navigation, object manipulation and other tasks, but their perception system relies heavily on physical sensors such as optics and mechanics, and can only achieve macroscopic form recognition in essence, and cannot obtain material molecular composition information. This fundamental limitation leads to a high risk of misjudgment in scenarios involving material nature judgment: visual sensors cannot distinguish between water and acetone with similar appearances, tactile feedback cannot distinguish between syrup and dangerous chemicals with similar viscosity, and auditory detection of gas leakage cannot determine its chemical hazard level. Physical sensing can only provide surface information of "what it looks like", but cannot answer the essential question of "what it is". This perception blind spot may cause disastrous consequences in industrial safety, emergency rescue and other scenarios.

[0003] Chemical sensing technology (such as Raman spectroscopy, infrared spectroscopy) can provide molecular fingerprint information of materials, with molecular-level recognition specificity. However, traditional chemical sensing systems are mostly off-body fixed platforms, lacking active exploration ability, and sampling points rely on manual preset, unable to dynamically adjust strategies according to environmental feedback. More importantly, there is a huge gap between chemical sensing and physical sensing in data modalities, spatiotemporal resolution, and decision logic, and existing technologies have failed to achieve the organic integration of the two. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a chemical information driven embodied intelligence detection method and system, which can realize accurate qualitative analysis of target substances by deeply coupling multi-modal chemical sensing modules with embodied intelligence framework.

[0005] In a first aspect, the embodiments of the present application provide a chemical information driven embodied intelligence detection method, which is applied to a chemical information driven embodied intelligence detection system. The system adopts a hierarchical and progressive heterogeneous multi-agent architecture, including a physical perception module, a miniature chemical sensing module, an intelligent collection module, a multi-modal fusion analysis module, and an embodied execution module. The method comprises the following steps: Based on the physical perception module, the macroscopic physical characteristics of the measured object are captured through visual, auditory and tactile sensors, a three-dimensional topological structure model is constructed, and a suspected target area positioning result is output. The suspected target area positioning result is the macroscopic candidate area range in three-dimensional space; The optimal contact point for spectral acquisition is selected from the macroscopic candidate region range, and the mechanical finger of the integrated spectral probe is driven to move to the optimal contact point to simultaneously acquire multimodal spectral information and generate raw spectral data. Based on the intelligent acquisition module receiving the raw spectral data and the feedback contact state information from the micro chemical sensing module, the acquisition parameters are adjusted in real time, and the raw spectral data is optimized to obtain standardized spectral data. Based on the multimodal fusion analysis module, the physical properties and standardized spectral data of the tested object are fused heterogeneously through a cross-modal attention mechanism to construct chemical-physical coupled features. Combined with a large language model in the chemical field, the output is structured decision data including substance identification results, hazard level and disposal instructions. Based on the embodied execution module, corresponding actions are performed according to the structured decision data, including risk warnings and controlling the robotic fingers to perform secondary sampling.

[0006] In some embodiments, the chemical information-driven embodied intelligent detection system further includes a global optimization module, and the method further includes the following steps: The global optimization module collects feedback data from other modules, and the collection strategy, fusion weights, and decision thresholds are dynamically adjusted through reinforcement learning algorithms.

[0007] In some embodiments, the physical perception module captures the macroscopic physical characteristics of the object under test through visual, auditory, and tactile sensors, constructs a three-dimensional topological structure model, and outputs the location result of the suspected target area, including the following steps: Based on the physical perception module, visual, auditory and tactile sensors are activated simultaneously to collect multi-source data of the object under test and extract the macroscopic physical features of the object under test, including geometric morphological features, spatial pose features and basic surface physical features. Based on the extracted macroscopic physical features, a three-dimensional reconstruction algorithm is used to construct a three-dimensional topological model of the object under test. The object features in the three-dimensional topological structure model are compared with the preset test object feature template to determine and output the suspected target area location result; the suspected target area location result is the macroscopic candidate area range in three-dimensional space.

[0008] In some embodiments, selecting the optimal contact point for spectral acquisition from the macroscopic candidate region range includes the following steps: Based on the embodied execution module, the physical perception module moves to the macroscopic candidate region range, and the basic physical features of the surface at different points are gradually captured by the tactile sensor. The optimal contact point for spectral acquisition is selected from the macroscopic candidate region based on preset surface physical characteristics criteria.

[0009] In some embodiments, the multimodal spectral information includes Raman scattering, infrared absorption, fluorescence emission, and ultraviolet-visible light spectral information; the acquisition parameters of the micro chemical sensing module include the sliding speed and pressing depth of the mechanical finger, as well as the focusing depth, integration time, and laser power of the spectrometer.

[0010] In some embodiments, optimizing the original spectral data to obtain standardized spectral data includes the following steps: The raw spectral data are preprocessed sequentially, including ghost peak removal, self-supervised noise reduction, and background interference correction. A comprehensive quality score is calculated based on spectral similarity of the preprocessed original spectral data; wherein, when the comprehensive quality score is greater than or equal to a preset threshold, the preprocessed original spectral data is standardized into a uniform format to obtain standardized spectral data; if the comprehensive quality score is less than the preset threshold, the data is re-acquired.

[0011] In some embodiments, the multimodal fusion analysis module performs heterogeneous feature fusion on the obtained physical properties and standardized spectral data of the measured object through a cross-modal attention mechanism to construct chemical-physical coupled features, including the following steps: The multimodal fusion analysis module employs a dual-stream coding architecture to perform split coding on the obtained macroscopic physical properties and standardized spectral data of the measured object, extracting chemical and microscopic physical features. Based on timestamps and spatial coordinates, chemical features and microphysical features are temporally aligned and spatially registered; and the temporally aligned chemical features and microphysical features are fused through a cross-modal attention mechanism to construct chemical-physical coupled features.

[0012] Secondly, embodiments of this application provide a chemical information-driven embodied intelligent detection system. This system adopts a hierarchical and progressive heterogeneous multi-agent architecture, including a physical sensing module, a micro chemical sensing module, an intelligent acquisition module, a multimodal fusion and analysis module, and an embodied execution module, for each module to collaboratively implement the steps of a chemical information-driven embodied intelligent detection method as described in any one of the first aspects.

[0013] Thirdly, this application also provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the chemical information-driven embodied intelligent detection method described in any of the second aspects above are performed.

[0014] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the chemical information-driven embodied intelligence detection method described in any one of the first aspects.

[0015] The chemical information-driven embodied intelligent detection method and system described in this application achieves molecular-level identification, precise positioning, and intelligent decision-making of target substances in complex environments through a physical sensing module to locate the target, a micro chemical sensing module to acquire multimodal spectral information, an intelligent acquisition module to optimize parameters and preprocess data, a multimodal fusion and analysis module to realize cross-modal feature coupling and large model-assisted decision-making, and an embodied execution module to complete sampling and response. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of the chemical information-driven embodied intelligent detection method described in an embodiment of this application is shown; Figure 2 A schematic diagram of the structure of the chemical information-driven embodied intelligent detection system described in an embodiment of this application is shown; Figure 3 A schematic diagram of the architecture of the embodied intelligent finger system according to an embodiment of this application is shown; Figure 4 A schematic diagram of the multimodal deep learning model architecture according to an embodiment of this application is shown; Figure 5 A schematic diagram of the structure of the electronic device described in an embodiment of this application is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0019] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0021] In view of the technical problems raised in the background art, this application provides a chemical information-driven embodied intelligent detection method and system, which can achieve accurate qualitative analysis of target substances by deeply coupling a multimodal chemical sensing module with an embodied intelligent framework.

[0022] See the instruction manual appendix Figure 1 Included with instruction manual Figure 2 This application provides a chemical information-driven embodied intelligent detection method, applied to a chemical information-driven embodied intelligent detection system. The system employs a hierarchical, heterogeneous multi-agent architecture, including a physical sensing module, a micro-chemical sensing module, an intelligent acquisition module, a multimodal fusion and analysis module, and an embodied execution module. The method includes the following steps: S1. Based on the physical perception module, the macroscopic physical characteristics of the object under test are captured by visual, auditory, and tactile sensors, a three-dimensional topological structure model is constructed, and the location result of the suspected target area is output; the location result of the suspected target area is the range of macroscopic candidate areas in three-dimensional space; S2. Select the optimal contact point for spectral acquisition from the macroscopic candidate region range, and drive the mechanical finger of the micro spectral sensing unit to move to the optimal contact point to simultaneously acquire multimodal spectral information and generate raw spectral data. S3. Based on the intelligent acquisition module receiving the raw spectral data and the feedback contact state information acquired by the chemical sensing module, the acquisition parameters are adjusted in real time, and the raw spectral data is optimized to obtain standardized spectral data. S4. Based on the multimodal fusion analysis module, the physical properties and standardized spectral data of the tested object are fused heterogeneously through a cross-modal attention mechanism to construct chemical-physical coupling features. Combined with a large language model in the chemical field, the output is structured decision data including substance identification results, hazard level and disposal instructions. S5. Based on the embodied execution module, perform corresponding disposal actions according to the structured decision data, including risk warning and controlling the mechanical finger to perform secondary sampling.

[0023] In other embodiments, the chemical information-driven embodied intelligent detection system further includes a global optimization module, and the chemical information-driven embodied intelligent detection method further includes the following steps: S6. Based on the global optimization module, feedback data from other modules is collected, and the collection strategy, fusion weights, and decision thresholds are dynamically adjusted through reinforcement learning algorithms.

[0024] Specifically, in step S1, the physical perception module is used to quickly locate the macroscopic suspected target area from the complex environment, eliminating background areas without the target object, thus narrowing down the range for subsequent precise positioning. In one embodiment, firstly, the visual sensor, auditory sensor, and tactile sensor are simultaneously activated based on the physical perception module to perform a full-domain scan of the object under test and its surrounding environment, capturing the geometric shape image, spatial orientation sound, and basic surface tactile data of the object under test; then, the collected multi-source data is preliminarily analyzed to extract the macroscopic physical features of the object under test, including geometric shape features, spatial pose features, and basic surface physical properties; then, based on the extracted macroscopic physical features, a three-dimensional reconstruction algorithm is used to construct a three-dimensional topological structure model of the object under test and its environment; finally, the object features in the three-dimensional topological structure model are compared with the preset target object feature template to eliminate environmental background areas without the target object, determine and output the range of macroscopic candidate areas in three-dimensional space, i.e., the suspected target area positioning result.

[0025] In step S2, precise sampling points are first selected within the suspected target area to ensure that the spectral detector can stably adhere to the surface of the object under test, providing physical positional assurance for high-quality spectral acquisition. In one embodiment, based on the suspected target area positioning results output in step S1, the embodied execution module drives the tactile sensor of the physical perception module to enter the macroscopic candidate area to perform fixed-point fine scanning; the tactile sensor captures the microscopic physical characteristics of different points within the area point by point, specifically including surface texture details, local flatness, and hardness uniformity data of small areas; then, according to the set spectral acquisition adaptation standards such as no scratches or stains on the surface, local flatness error, and hardness uniformity deviation, the microscopic physical characteristics of each collected point are compared and screened one by one, and invalid points that do not meet the adaptation standards are eliminated; from the valid points that meet the standards, the point with the best signal acquisition conditions is selected, and its precise three-dimensional spatial coordinates are determined by a three-dimensional coordinate calibration algorithm, and the coordinates are output as the optimal contact point.

[0026] This drives the robotic finger to the target location, completing the raw data acquisition of multiple types of molecular spectra and obtaining the basic chemical characteristic data of the analyte. This is done in conjunction with the instructions attached. Figure 3 The embodied execution module plans the optimal motion trajectory based on the three-dimensional coordinates of the best contact point, combined with the joint movement range and obstacle avoidance requirements of the robotic finger. This drives the robotic finger, which integrates a micro-spectral acquisition unit, to move along the planned trajectory, precisely reaching the optimal contact point. Furthermore, through tactile feedback closed-loop control, the pressure of the robotic finger is adjusted to a stable range, ensuring that the acquisition port of the micro-spectral acquisition unit is in close and undamaged contact with the surface of the object under test. Through acquisition channels corresponding to Raman scattering, infrared absorption, fluorescence emission, and ultraviolet-visible spectra, four types of molecular spectral signals from the target area are simultaneously acquired under stable pressure, recording the correspondence between signal intensity and wavelength. The acquired four types of spectral signals are then converted from analog to digital and standardized in format, integrated to generate raw molecular-level spectral data, and transmitted to the intelligent acquisition module.

[0027] The miniature spectral acquisition unit can be either a fiber-optic guided miniature spectral acquisition unit consisting of a fiber optic probe and a body-mounted spectrometer, or a chip-based direct-acquisition miniature spectral acquisition unit consisting of a miniature spectrometer chip and an optical window. When using a fiber-optic guided miniature spectral acquisition unit, after ensuring close contact between the fiber optic probe and the surface of the analyte, the body-mounted spectrometer is triggered to simultaneously activate the acquisition channels of the four independent fiber optic probes, acquiring four types of spectral signals respectively. When using a chip-based direct-acquisition miniature spectral acquisition unit, after ensuring close contact between the optical window and the surface of the analyte, the four-channel miniature spectrometer chip is activated, simultaneously acquiring four types of spectral signals.

[0028] In one embodiment, replaceable miniature spectral sensing units are embedded along the phalanx axis in the distal phalanx areas of the index, middle, ring, and little fingers, which house the execution module. These units can be single-mode fiber optic probes or miniature spectrometer chips directly encapsulated within the fingertips; both are functionally equivalent and selectable in parallel. The index finger spectral detector is specifically designed for Raman scattering spectroscopy, with an excitation wavelength of 785 nm and a spectral range of 150-2500 cm⁻¹. - ¹; The middle finger spectrometer is specifically designed for infrared absorption spectroscopy, operating in the 2.5–5 μm wavelength band, with a spectral range of 400–4000 cm⁻¹. - ¹; The ring finger spectrometer is dedicated to fluorescence emission spectroscopy, with an excitation wavelength of 375 nm and an emission range of 400–800 nm; the little finger spectrometer is dedicated to ultraviolet-visible extinction spectroscopy, covering the 200–800 nm band. When using fiber optic probes, all four spectrometers have a diameter of 1.0 mm, with the ends polished at a 45° angle to form a total internal reflection sampling surface. The end-face curvature radius is no greater than 15 mm to adapt to the physiological curvature of the fingertip. The miniature spectrometer adopts a four-channel parallel architecture, integrated into the robot's forearm cavity. Each channel is connected to the fingertip probe through an independent polarization-maintaining fiber bundle, with an optical path length of 0.5 m, completely eliminating optical crosstalk between channels through physical isolation. When using the chip direct sampling form, the miniature spectrometer chip encapsulated in the fingertip directly collects signals through a conformal optical window, eliminating the need for fiber optic transmission. The optical signal is directly coupled to the four-channel spectrometer in the same forearm cavity, thereby simplifying the optical path and reducing losses. A thin-film pressure sensor is integrated into the distal phalanx of the thumb to provide a reverse support force to stabilize the contact between the four fingers and the target object. The support force threshold is set to 2 N. The spectral resolution of each channel is no less than 2 nm, and the response time is less than 10 ms. The acquired optical signal is converted into an electrical signal by a spectrometer, and then the signal is enhanced and digitized by a four-channel synchronous analog-to-digital converter.

[0029] In step S3, the acquisition parameters are dynamically adjusted based on the contact state and the original spectral quality to improve data quality, output standardized spectral data, and eliminate the influence of interference signals on subsequent inference. In one embodiment, the intelligent spectral acquisition module achieves automated operation through a two-level architecture of adaptive parameter control unit and intelligent spectral preprocessing unit: at the hardware control automation level, the contact state between the finger and the target object is sensed in real time through a tactile sensor integrated in the fingertip. When the contact pressure reaches 1 N, the acquisition interruption is triggered to start the contact sensing. The index and middle fingers integrate spectral detectors, and the thumb provides reverse support force to stabilize the contact. The finger joints adopt a rope-driven flexible structure, and the contact force control adopts an impedance mode. The finger sliding speed and pressing depth are adjusted in real time according to the chemical signal gradient to achieve optimal sampling. Furthermore, by combining dynamically adjusted integration time (10 ms-10 s), excitation power (1-100 mW), and focusing control algorithms, adaptive parameter optimization is achieved to obtain the optimal signal-to-noise ratio sampling. At the algorithm preprocessing level, a peak identification algorithm is used to remove abnormally high numerical spectral peaks with fewer than 3 points, eliminating ghost peak interference. An embedded self-supervised noise reduction algorithm is used to adaptively suppress noise without clean labeled data, achieving online intelligent noise reduction. Iterative reweighted polynomial fitting is applied to dynamic background baseline estimation to achieve accurate correction of complex fluorescence backgrounds. Closed-loop quality assessment feedback is achieved by calculating spectral similarity index. When the comprehensive quality score is higher than or equal to a preset threshold, the processed original spectral data is standardized into a unified format to obtain standardized spectral data. When the comprehensive quality score is lower than a preset threshold, a re-acquisition command is automatically triggered, forming a complete closed-loop control of acquisition, processing, and evaluation.

[0030] In step S4, the main focus is on fusing physical and chemical characteristics to generate structured decision data that includes substance identification results, hazard levels, and disposal recommendations. In one embodiment, this is illustrated in the appendix to the specification. Figure 4The multimodal fusion parsing module first synchronously receives the macroscopic physical characteristics and constructed 3D topological model output from the physical perception module, as well as the standardized spectral data output from the intelligent acquisition module. It then establishes a multi-source data association index and processes the data using a dual-stream coding architecture. For example, for the chemical stream, a 1D-CNN encoder encodes the standardized spectral data, extracting molecular features such as peak position, peak intensity, and peak width. For the physical stream, a VIT encoder parses the spatial geometry and texture features of the 3D topological model, while a 1D-transformer encodes the dynamic temporal features of the surface physical properties. Next, a cross-modal attention mechanism is activated, achieving temporal alignment and spatial registration of chemical and physical features based on timestamps and spatial coordinates, eliminating dimensional differences and spatiotemporal biases in heterogeneous data. The aligned chemical and physical features are then fused, and the feature interaction attention module mines the intrinsic correlation between the two types of features, constructing a dimensionally unified chemical-physical coupled feature tensor. Finally, the coupled feature tensor is input into the multimodal Transformer decoder, and an end-to-end inference model outputs substance identification, classification, and concentration prediction results. The substance identification relies on a scenario-adaptive database, which is categorized by application field and includes a standard spectral library covering ≥10,000 chemical substances. It dynamically loads corresponding data subsets and optimizes judgment thresholds based on the detection scenario (e.g., hazardous chemical monitoring, home health screening), enabling seamless switching across scenarios. After identification, the autonomous decision-making module dynamically invokes a dedicated large-scale language model in the chemical field, linked to queries of molecular structure and materials safety databases, outputting the substance's physicochemical properties and generating structured decision data including confidence level, hazard level, and disposal recommendations to achieve intelligent response in multiple scenarios.

[0031] The chemical-specific large language model can be obtained by fine-tuning and training an existing general-purpose large language model in combination with a chemical knowledge base. The specific training process should be a technical means well known to those skilled in the art, and will not be elaborated here.

[0032] In step S5, the main task is to parse the decision data, classify and execute corresponding handling actions such as sampling, evacuation, and alarm, and provide feedback on the execution status to complete the implementation of the detection task. In one embodiment, the embodied execution module receives the structured decision data output by the multimodal fusion parsing module, extracts the core information of the handling instructions, and clarifies the instruction type. Based on the instruction type, it matches the corresponding execution strategy and parses the handling instructions into executable mechanical parameters. For example, if the instruction is supplementary sampling, the mechanical finger is driven to move along the planned trajectory to a new point, and triggers secondary sampling after stable contact; if the instruction is safe evacuation, the mechanical finger is controlled to quickly evacuate along the planned trajectory and stops moving after reaching a safe position; if the instruction is to trigger an alarm, the audible and visual alarm module is activated simultaneously, and the evacuation action is executed. Furthermore, the system collects the mechanical finger's movement position, pressure stability, and alarm module activation status in real time, generates action execution feedback data, and transmits it to the global optimization module.

[0033] Step S6 primarily involves collecting full-process feedback data through the global optimization module to achieve adaptive evolution of the detection strategy and improve the versatility and robustness of the method. In one embodiment, the global optimization module synchronously receives feedback data from each stage, establishing a full-process feedback data pool; it standardizes the collected multi-source feedback data, removes abnormal feedback values, and classifies and labels the data according to the process dimension; it constructs a multi-dimensional reward function with the core optimization objectives of improving detection success rate, reducing average detection time, improving signal-to-noise ratio, and enhancing decision confidence; it inputs the reward function parameters into a reinforcement learning algorithm, and evaluates and iteratively trains the current full-process parameters through a policy network; based on the training results, it dynamically adjusts core parameters such as contact strategy, multimodal fusion weights, and decision thresholds; after every 100 complete detection tasks, it verifies the optimized parameters, and if the preset optimization objectives are met, it solidifies the current strategy parameters; it synchronously distributes the updated strategy parameters to each module to guide the execution of subsequent detection tasks, forming a closed loop of full-process adaptive evolution of feedback, optimization, and application.

[0034] The following section describes a chemical information-driven embodied intelligent detection method based on two specific application scenarios.

[0035] I. Non-invasive detection applications of health indicators based on infrared absorption and fluorescence spectroscopy: Optical positioning of the detection area. The physical perception module scans the inside of the user's wrist using a visual sensor to identify suitable detection sites with smooth skin, uniform skin color, and abundant subcutaneous blood vessels. The module combines short-wave infrared imaging (wavelength 940 nm) to assist in judging the distribution of superficial blood vessels, and finally selects a flat area with an area of ​​not less than 1.5 square centimeters, outputting the three-dimensional spatial coordinates P1 of its center point.

[0036] Dual-spectral probe contact and synchronous acquisition. The embodied actuator module controls the movement of its end-effector, a mechanical finger integrating a dual-spectral probe, to coordinate P1. The probe's front end is an optical window, internally encapsulating an independent fiber optic channel. The probe makes perpendicular contact with the skin surface with a stable pressure of approximately 0.5 N. After stable contact, the system synchronously triggers data acquisition from two spectral channels: first, near-infrared absorption spectroscopy acquisition. A dual-wavelength LED light source (center wavelengths of 760 nm and 850 nm, respectively) is activated, alternately irradiating the skin and collecting diffuse reflected light. The spectrometer continuously acquires data at a frequency of 100 times per second for 10 seconds, obtaining two time-synchronized dynamic reflected light intensity data. Second, skin autofluorescence spectroscopy acquisition. A 375 nm ultraviolet LED light source is synchronously triggered to excite the skin tissue, and fluorescence emission signals from 400-650 nm are collected through another fiber optic channel. The spectrometer integration time is set to 2 s, continuously acquiring and averaging 5 times to obtain steady-state fluorescence spectral data.

[0037] Spectral processing and physiological feature extraction. The intelligent acquisition module processes the raw spectral data. First, blood oxygen and heart rate are extracted. Dual-wavelength dynamic reflected light intensity data is analyzed, extracting the light intensity change component caused by arterial pulsation. The real-time blood oxygen saturation value is calculated based on Lambert-Beer's law, and the heart rate is calculated from its periodic fluctuations. Second, metabolic state assessment is performed. Steady-state fluorescence spectra and changes in relative fluorescence intensity are analyzed as indicators of local tissue energy metabolism and oxidative stress. Finally, quality assessment is conducted. The signal-to-noise ratio of the pulse waveform and the intensity of characteristic peaks in the fluorescence spectrum are calculated. If the signal quality is unsatisfactory, the system prompts the user to remain still and automatically re-acquires the data.

[0038] Multimodal fusion and health status decision-making. The multimodal fusion parsing module receives processed features such as blood oxygen saturation, heart rate, metabolic ratio, and estimated skin surface temperature. The module first aligns the features, then uses an attention mechanism to focus on analyzing the coordination between blood oxygen levels (reflecting oxygen delivery) and tissue metabolic ratios (reflecting oxygen utilization). It then combines this with a physiological knowledge base for comprehensive reasoning, outputting a structured health report. For example, when blood oxygen is normal but the metabolic ratio is abnormally high, it may indicate tissue microcirculation disorders, and the system will output corresponding warnings and suggestions.

[0039] Embodied Interaction and Response Execution. The embodied execution module performs operations based on decision data: it clearly displays real-time pulse waveforms, blood oxygen values, metabolic status charts and interpretations on the screen; if an alert is triggered, it broadcasts health advice via voice and can send an encrypted test summary to a designated terminal.

[0040] Global optimization. The global optimization module collects signal quality, user feedback, and individual difference data (such as the impact of skin color on light absorption) for each detection. Through reinforcement learning algorithms, it dynamically optimizes the optimal contact pressure, light source intensity, and personalized warning threshold for specific users, enabling continuous adaptive improvement in system performance.

[0041] II. Application of SERS detection for pesticide residues on fruit surfaces: Macroscopic suspected area localization. The physical perception module simultaneously activates the visual sensor and structured light projector to perform a 3D scan of the apple surface to be tested, acquiring its surface point cloud data. The processor runs point cloud processing and surface reconstruction algorithms to calculate the surface curvature and identify locally flat areas with a surface curvature radius greater than or equal to 25 mm and an area greater than or equal to 1 square centimeter. This area is marked as the target area for suspected pesticide residue, and the 3D spatial coordinates P0 of the center point of this area are output.

[0042] Flexible SERS Substrate Replacement and Scraping Sampling. A replaceable flexible surface-enhanced Raman scattering (SERS) substrate is integrated into the fingertip of the sampling finger (e.g., the index finger). This substrate is surface-modified with gold / silver nanoparticles to enhance Raman detection sensitivity. First, a stepper motor inside the sampling finger advances the substrate a predetermined length (e.g., 0.5 mm), exposing a clean, unused substrate area to the fingertip. Then, the embodied execution module, starting from coordinate P0, controls the sampling finger to scrape along a pre-defined "U"-shaped trajectory (5 mm on each side) on the apple surface at a speed of 5 mm per second and a normal pressure of 0.3 N. The thumb provides counter-support for stability. The scraping process is repeated three times to ensure sufficient transfer and enrichment of any pesticide residues present on the apple surface onto the flexible SERS substrate surface.

[0043] In-situ spectral acquisition of adjacent fingers. After scraping and sampling, the sampling finger is slightly lifted. The adjacent detection finger (such as the middle finger) then moves so that the optical window of its integrated miniature Raman spectroscopy probe is aligned with the enriched area of ​​the sampling fingertip. The working distance between the probe and the substrate surface is stabilized at 4.5 mm using a precision micro-displacement stage. Subsequently, a 785 nm wavelength laser is triggered, and three Raman spectral signals are continuously acquired at an integration time of 3 s and a laser power of 30 mW. These signals are then averaged locally to obtain the raw spectral data S0.

[0044] Spectral optimization and quality assessment. The intelligent acquisition module performs standardized preprocessing on the raw spectral data S0. This preprocessing includes: fluorescence background correction using an adaptive iterative reweighted penalized least squares method; adaptive noise reduction using a wavelet transform algorithm; and removal of abnormal spectral peaks (ghost peaks) according to preset rules. Subsequently, the similarity score between the preprocessed spectrum and the system's built-in "Fruit and Vegetable Pesticide Residue SERS Standard Sub-Spectrum Library" is calculated. If the similarity score is greater than or equal to the preset threshold of 0.85, the spectral quality is deemed acceptable, and standardized spectral data S1 is output; if it is lower than the threshold, an automatic feedback instruction is issued to extend the integration time and re-acquire the data, repeating the process up to two times.

[0045] Cross-modal feature fusion and intelligent decision-making. The multimodal fusion parsing module simultaneously receives physical features of the apple surface (such as peel color, texture, and estimated wax layer thickness) from the physical perception module and standardized spectral data S1 from the intelligent acquisition module. The module employs a dual-stream coding architecture, extracting physical and chemical feature vectors through visual coding and spectral coding networks, respectively. A cross-modal attention mechanism is used to temporally align and deeply fuse the two feature streams, generating a unified chemical-physical coupled feature. This coupled feature is input into the subsequent decision model and used in conjunction with a chemical domain knowledge base for reasoning, ultimately outputting structured decision data. This data includes the substance identification result (e.g., chlorpyrifos), identification confidence level (e.g., 95%), hazard level (e.g., level 2), and specific handling instructions (e.g., "perform secondary verification sampling and activate alarm").

[0046] Secondary sampling and response execution. The embodied execution module performs corresponding operations according to the handling instructions. First, it controls the sampling finger to perform a secondary scraping sampling in the same area, and then collects a verification spectrum S2 through the detection finger again. If the verification spectrum confirms the initial test result, it is judged as a positive test. Subsequently, the system immediately activates the audible and visual alarm to issue an on-site alarm, and uploads the complete test data (including spectrum, identification result, time, and location) to the cloud server for storage and filing through the communication interface.

[0047] Global optimization. The global optimization module collects and records feedback data from the entire detection task chain, such as the number of times the baseband was used, the signal-to-noise ratio of the acquired signal, and the decision accuracy. This data is used to drive the reinforcement learning model to learn offline or online, dynamically optimizing the strategy parameters of each module, such as updating the baseband lifespan prediction model and adjusting the optimal scraping pressure under different surface materials, thereby achieving adaptive improvement and continuous optimization of the overall system performance.

[0048] This application provides a chemical information-driven embodied intelligent detection method. It employs a progressive perception strategy of macroscopic discriminative feature localization and microscopic adaptive feature selection, significantly improving the accuracy of suspected target area localization and the precision of optimal contact point selection, effectively reducing the probability of invalid sampling. Through a dual-dimensional adaptive parameter optimization mechanism that coordinates contact state and raw spectral data, combined with multi-step spectral preprocessing processes such as noise reduction and correction, the quality of spectral data is ensured. A dual-stream coding architecture achieves precise alignment and fusion of chemical and physical features, supplemented by a large chemistry language model to enhance physicochemical information, significantly improving the accuracy of substance identification and the reliability of hazard level assessment. A full-process feedback-driven reinforcement learning optimization loop is constructed to achieve dynamic evolution of detection parameters, adapting to complex and ever-changing detection scenarios.

[0049] Based on the same inventive concept, this application also provides a chemical information-driven embodied intelligent detection system, including a physical sensing module, a micro-chemical sensing module, an intelligent acquisition module, a multimodal fusion analysis module, and an embodied execution module, for each module to collaboratively implement the steps of the aforementioned chemical information-driven embodied intelligent detection method. Since the principle of the system in this application's solution is similar to the aforementioned chemical information-driven embodied intelligent detection method, the implementation of the system can refer to the implementation of the method; repeated details will not be elaborated further.

[0050] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0051] Based on the same concept of the present invention, as shown in the appendix to the specification. Figure 5 As shown in the embodiment of this application, an electronic device 500 is provided. The electronic device 500 includes: at least one processor 501, at least one network interface 504 or other user interface 503, a memory 505, and at least one communication bus 502. The communication bus 502 is used to enable communication between these components. The electronic device 500 may optionally include a user interface 503, including a display (e.g., touchscreen, LCD, CRT, holographic imaging, or projector), a keyboard, or a clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0052] Memory 505 may include read-only memory and random access memory, and provides instructions and data to processor 501. A portion of memory 505 may also include non-volatile random access memory (NVRAM).

[0053] In some implementations, memory 505 stores executable modules or data structures, or subsets thereof, or extended sets thereof: The 5051 operating system contains various system programs used to implement various basic business functions and handle hardware-based tasks. Application module 5052 contains various applications, such as launchers, media players, and browsers, to implement various application functions.

[0054] In this embodiment of the application, the processor 501 executes steps such as those of a chemical information-driven embodied intelligence detection method by calling programs or instructions stored in the memory 505.

[0055] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs steps such as those in a chemical information-driven embodied intelligence detection method.

[0056] Specifically, the storage medium can be a general-purpose storage medium, such as a portable disk or hard drive. When the computer program on the storage medium is run, it can achieve accurate qualitative analysis of the target substance by deeply coupling the multimodal chemical sensing module with the embodied intelligent framework.

[0057] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, and the indirect coupling or communication connection of the apparatus or units may be electrical, mechanical, or other forms.

[0058] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0059] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0060] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0061] Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A chemical information-driven embodied intelligent detection method, characterized in that, An embodied intelligent detection system driven by chemical information is provided. This system employs a hierarchical, heterogeneous multi-agent architecture, including a physical sensing module, a micro-chemical sensing module, an intelligent acquisition module, a multimodal fusion and analysis module, and an embodied execution module. The method includes the following steps: The physical perception module captures the macroscopic physical characteristics of the object under test through visual, auditory, and tactile sensors, constructs a three-dimensional topological structure model, and outputs the location result of the suspected target area; the location result of the suspected target area is the range of macroscopic candidate areas in three-dimensional space; The optimal contact point for spectral acquisition is selected from the macroscopic candidate region range, and the mechanical finger of the micro chemical sensing module is driven to move to the optimal contact point to simultaneously acquire multimodal spectral information and generate raw spectral data. Based on the intelligent acquisition module receiving the raw spectral data and the feedback contact state information from the micro chemical sensing module, the acquisition parameters are adjusted in real time, and the raw spectral data is optimized to obtain standardized spectral data. Based on the multimodal fusion analysis module, the physical properties and standardized spectral data of the tested object are fused heterogeneously through a cross-modal attention mechanism to construct chemical-physical coupled features. Combined with a large language model in the chemical field, the output is structured decision data including substance identification results and disposal instructions. Based on the embodied execution module, corresponding actions are performed according to the structured decision data, including risk warnings and controlling the robotic fingers to perform secondary sampling.

2. The chemical information-driven embodied intelligent detection method according to claim 1, characterized in that, The chemical information-driven embodied intelligent detection system further includes a global optimization module, and the method further includes the following steps: The global optimization module collects feedback data from other modules, and the collection strategy, fusion weights, and decision thresholds are dynamically adjusted through reinforcement learning algorithms.

3. The chemical information-driven embodied intelligent detection method according to claim 1, characterized in that, The physical perception module captures the macroscopic physical characteristics of the object under test through visual, auditory, and tactile sensors, constructs a three-dimensional topological structure model, and outputs the location result of the suspected target area, including the following steps: Based on the physical perception module, visual, auditory and tactile sensors are activated simultaneously to collect multi-source data of the object under test and extract the macroscopic physical features of the object under test, including geometric morphological features, spatial pose features and basic surface physical features. Based on the extracted macroscopic physical features, a three-dimensional reconstruction algorithm is used to construct a three-dimensional topological model of the object under test. The object features in the three-dimensional topological structure model are compared with the preset test object feature template to determine and output the suspected target area location result; the suspected target area location result is the macroscopic candidate area range in three-dimensional space.

4. The chemical information-driven embodied intelligent detection method according to claim 3, characterized in that, The process of selecting the optimal contact point for spectral acquisition from the macroscopic candidate region includes the following steps: Based on the embodied execution module, the physical perception module moves to the macroscopic candidate region range, and the basic physical features of the surface at different points are gradually captured by the tactile sensor. The optimal contact point for spectral acquisition is selected from the macroscopic candidate region based on preset surface physical characteristics criteria.

5. The chemical information-driven embodied intelligent detection method according to claim 4, characterized in that, in, Multimodal spectral information includes Raman scattering, infrared absorption, fluorescence emission, and ultraviolet-visible light spectral information; the acquisition parameters of the miniature chemical sensing module include the sliding speed and pressing depth of the mechanical finger, as well as the focusing depth, integration time, and laser power of the spectrometer.

6. The chemical information-driven embodied intelligent detection method according to claim 5, characterized in that, The optimization of the original spectral data to obtain standardized spectral data includes the following steps: The raw spectral data are preprocessed sequentially, including ghost peak removal, self-supervised noise reduction, and background interference correction. A comprehensive quality score is calculated based on spectral similarity of the preprocessed original spectral data; wherein, when the comprehensive quality score is greater than or equal to a preset threshold, the preprocessed original spectral data is standardized into a uniform format to obtain standardized spectral data; if the comprehensive quality score is less than the preset threshold, the data is re-acquired.

7. The chemical information-driven embodied intelligent detection method according to claim 6, characterized in that, The multimodal fusion analysis module performs heterogeneous feature fusion on the obtained physical properties and standardized spectral data of the tested object through a cross-modal attention mechanism to construct chemical-physical coupled features, including the following steps: The multimodal fusion analysis module employs a dual-stream coding architecture to perform split coding on the obtained macroscopic physical properties and standardized spectral data of the measured object, extracting chemical and microscopic physical features. Based on timestamps and spatial coordinates, chemical features and microphysical features are temporally aligned and spatially registered; and the temporally aligned chemical features and microphysical features are fused through a cross-modal attention mechanism to construct chemical-physical coupled features.

8. A chemical information-driven embodied intelligent detection system, characterized in that, The system adopts a hierarchical and progressive heterogeneous multi-agent architecture, including a physical sensing module, a micro chemical sensing module, an intelligent acquisition module, a multimodal fusion and analysis module, and an embodied execution module, which are used to collaboratively implement the steps of the chemical information-driven embodied intelligent detection method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of a chemical information-driven embodied intelligent detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a chemical information-driven embodied intelligent detection method as described in any one of claims 1 to 7.

Images