An intelligent palpation positioning system and method based on multi-modal perception and anatomical model matching
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-05
- Publication Date
- 2026-06-02
Smart Images
Figure CN122123642A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of medical auxiliary equipment and artificial intelligence technology, in particular to an intelligent positioning system and method that fuses high-precision tactile sensing, spatial positioning, digital human anatomy model and intelligent recognition algorithm, and can automatically recognize human bony landmarks, joints and muscles and other soft tissues through palpation. BACKGROUND
[0002] In the fields of rehabilitation medicine, physical therapy, sports injury assessment, and traditional Chinese medicine massage, palpation is a crucial basic skill. The operator locates bony landmarks, assesses joint alignment, identifies muscles, tendons, ligaments, and other soft tissue states through finger touch, sliding, and perception, thereby determining the injury site, assessing dysfunction, and developing a treatment plan. However, this core clinical skill has long relied on the personal experience, touch, and subjective judgment of the operator (such as therapists, physicians), with many inherent limitations: First, strong experience dependence and long learning curve; the "touch" required for precise palpation, which is the subtle discrimination of tissue hardness, elasticity, texture, temperature, abnormal nodules, or friction, cannot be fully transmitted through textbooks or image materials and must be acquired through long-term and extensive clinical practice. This makes it costly and time-consuming to train a skilled palpation expert.
[0003] Second, strong subjectivity, poor consistency and repeatability; different operators, or even the same operator at different times, may have significant differences in palpation perception and conclusions for the same patient and the same part. This subjectivity leads to a lack of objective standards for assessment results, affecting the consistency of diagnosis and optimization of treatment plans, especially in multi-disciplinary consultations or efficacy comparison studies.
[0004] Third, the process is difficult to quantify and record; traditional palpation is a "black box" process, and the information perceived by the operator cannot be objectively recorded and quantified. Key operating parameters such as palpation pressure, movement path, and time spent at a particular location, as well as conclusions such as tightness, relaxation, and tenderness, cannot form structured data that can be traced and analyzed. This is not conducive to accurate review of the treatment process, objective assessment of efficacy, and effective communication between doctors and patients or among peers.
[0005] Fourth, difficult to locate complex anatomical regions; in areas with rich muscle layers and overlapping structures, such as the rotator cuff muscle group and deep spinal muscle groups, it is difficult to clearly distinguish between superficial and deep muscles based on tactile sensation, leading to positioning errors. Finding deep or small bony landmarks, such as transverse processes of the lumbar spine and small bones of the wrist, is also challenging.
[0006] To overcome these limitations, the industry has made some technical explorations, which can be mainly divided into the following categories, but none of them can fundamentally solve the problem of intelligent and objective palpation: (1) Intelligent sensing and force feedback gloves: There are technologies in the prior art for measuring the pressure distribution when the hand contacts an object, focusing on simulating tactile sensations in virtual reality. The core function of such technologies is limited to "measurement of physical signals" or "feedback of force sensation". They can collect data such as pressure and posture, or output simulated tactile sensations to the user, but they have no ability to understand the clinical implications of these data. The system cannot answer key clinical questions such as "which muscle is being pressed now?" or "which tendon does the sliding path correspond to?" The data output is disconnected from the human anatomy.
[0007] (2) Medical imaging and three-dimensional anatomical model systems: The prior art provides high-precision human three-dimensional anatomical model display methods for teaching and preoperative planning. Such systems provide excellent visual references, but they are static and passive knowledge bases. When performing physical palpation, the operator cannot automatically associate and match the real spatial position and touch pressure of the fingers with specific structures in the model in real time. The operator still needs to perform "tactile-visual" conversion and comparison in his mind, and the cognitive load and skill threshold of palpation have not been reduced.
[0008] (3) Motion capture and surface electromyography (sEMG) systems: Such systems are often used for motion analysis in rehabilitation training and can capture limb motion trajectories or monitor muscle electrical activity. However, they are mainly used to observe the macro results of "muscle activation" or "joint movement", rather than to identify the "touched anatomical structure itself". The installation position and target of their sensors (such as optical markers and sEMG electrodes) do not match the fine positioning requirements of palpation operations.
[0009] In summary, the existing technologies present a state of disconnection between "perception-model-recognition": the sensing device only collects "signals" without knowing "what". The anatomical model only shows "morphology" without knowing "touch".
[0010] There is a lack of an "intelligent bridge" between the two that can perform real-time semantic analysis and spatial matching.
[0011] Therefore, there is an urgent and unmet need in the field of clinical practice and medical education: an intelligent palpation assistance technology that can deeply integrate objective tactile sensing data, accurate spatial positioning information, and deep anatomical knowledge, achieving real-time, automatic, and accurate anatomical structure recognition and positioning. This technology needs to be able to convert the operator's subjective "hand feeling" into objective, quantifiable, and directly related to standard anatomical semantics data, thereby reducing the technical threshold of palpation, improving the consistency and recordability of diagnosis, and ultimately revolutionizing the traditional palpation mode. Summary of the Invention
[0012] This invention designs an intelligent palpation positioning system and method based on multimodal perception and anatomical model matching. The technical problems it solves are the lack of objective data in the traditional palpation process and the lack of intelligent association between perceptual data and clinical anatomical semantics.
[0013] To solve the aforementioned technical problems, the present invention adopts the following solution: An intelligent palpation positioning system based on multimodal perception and anatomical model matching includes: a multimodal perception glove for collecting multi-source physical signals during the operator's palpation process; an anatomical digital model database for storing three-dimensional human anatomical models with semantic labels and spatial coordinate information; an intelligent recognition and positioning engine for real-time matching and reasoning between the multi-source physical signals and the models in the anatomical digital model database to identify and locate specific anatomical structures being palpated; and an interactive display terminal for real-time presentation of the recognition and positioning results output by the intelligent recognition and positioning engine.
[0014] Preferably, the multimodal sensing glove includes: a distributed pressure sensor array integrated into the touch contact area of the glove for measuring the pressure distribution and dynamic changes of the touch contact surface; and an inertial measurement unit integrated into the glove for measuring the spatial posture and motion trajectory of the hand.
[0015] Preferably, the multimodal sensing glove further includes an electromyography (EMG) sensor disposed on the extension portion of the glove, used to monitor the EMG activity signals of the operator's hand or forearm to assist in determining the intention to touch and the state of force applied.
[0016] Preferably, the intelligent recognition and positioning engine includes: a reference point recognition module, used to identify bony landmarks or joints by matching sensor data from the initial palpation stage with an anatomical model, and use these as reference points for spatial positioning; and a muscle extension recognition module, used to infer and identify muscle or tendon tissue associated with the reference point based on the reference point recognition, the trajectory data during the palpation movement process, and the anatomical model information.
[0017] Preferably, the intelligent recognition and positioning engine further includes: a precise positioning confirmation module, used to analyze the dynamic signal characteristics generated by repeated or deep pressing on a specific area, distinguish layered muscles or different tissues, and perform confidence calibration and final confirmation on the recognition results.
[0018] Preferably, the interactive and display terminal is an augmented reality device used to overlay the names, boundaries, or hierarchical relationships of the identified anatomical structures onto the operator's real field of vision in the form of virtual images.
[0019] Preferably, the system further includes a data storage and report generation module for encrypting and storing the complete palpation path, the identified anatomical structure sequence, key positioning point data and operation annotations, and generating a structured palpation report.
[0020] A smart palpation positioning method, characterized by comprising the following steps: Step S1: Simultaneously collect multi-source sensor data streams during the operator's palpation operation using a multimodal sensing glove; Step S2: Match the multi-source sensor data with the pre-stored anatomical digital model to identify the initial anatomical reference point to be palpated and establish a spatial mapping relationship; Step S3: Using the initial anatomical reference point as a reference, based on the continuous palpation movement trajectory data and combined with the topological relationship in the anatomical model, extend to identify and locate the surrounding muscles or soft tissues associated with the reference point. Step S4: Output and display the identification and positioning results in real time.
[0021] Preferably, in step S3, the target muscle or tendon is inferred by analyzing the changes in the direction, length, speed and pressure distribution of the palpation movement path and performing probability matching calculations with the muscle direction, origin and insertion points and surface projection information in the anatomical model.
[0022] Preferably, the method further includes step S5: by analyzing the dynamic mechanical characteristics generated by repeated or deep pressing on a specific area, distinguishing between superficial and deep muscles or different tissue types, and verifying and precisely locating the preliminary identification results of step S3.
[0023] Preferably, the procedure further includes step S6: encrypting and storing the palpation movement trajectory, the identified anatomical structure sequence, key positioning point information, and operation process data, and automatically generating a structured palpation record report.
[0024] The intelligent palpation localization system and method based on multimodal perception and anatomical model matching has the following beneficial effects: (1) This invention, through a multimodal sensing glove, for the first time synchronously and with high precision digitally acquires key physical parameters during palpation—including precise pressure distribution, spatial movement trajectory, and the operator's force application intention reflected by electromyographic signals. This transforms the traditionally subjective and ineffable palpation experience, which relies on personal "feel," into a measurable, analyzable, and reproducible objective data stream, providing a solid data foundation for the quantitative assessment of palpation skills, the establishment of standardized operating procedures, and the objective comparison of therapeutic effects.
[0025] (2) The core breakthrough of this invention lies in its intelligent recognition and positioning engine. It does not simply display sensor readings, but rather deeply integrates and matches real-time collected multimodal data with a digital model rich in anatomical knowledge (including structural names, spatial coordinates, and topological relationships). It can proactively determine what structure is being touched. This endows the device with clinical diagnostic auxiliary value, directly serving the core clinical needs of locating damage and assessing function.
[0026] (3) This invention features a unique extended positioning logic from a reference point to soft tissue. After identifying a reliable anatomical reference point (such as a joint), it can intelligently track the operator's subsequent palpation path using this as the center, and infer and highlight the most likely soft tissue to be explored and its surface projection range in real time based on knowledge such as muscle origin and insertion points and direction of travel stored in the anatomical model database. This capability is particularly suitable for areas with overlapping muscle layers and complex structures (such as the rotator cuff and paravertebral region), effectively assisting the operator in distinguishing between superficial and deep tissues, reducing positioning errors, and shortening palpation time.
[0027] (4) This invention integrates a precise positioning confirmation module, which can analyze pressure change patterns and electromyographic signal characteristics under dynamic operations such as repeated pressing and deep pressing. By utilizing the differences in mechanical response of different tissues (such as tendons and muscle bellies, and different layers of muscles) under dynamic pressure, the system can perform secondary verification and fine calibration on the preliminary identification results, thereby outputting a high-confidence final positioning result. This enhances the reliability of the system output, making it closer to the identification and judgment process of a high-level operator.
[0028] (5) The system of this invention not only provides real-time guidance, but also records the data and results of the entire palpation process, generating a structured electronic report. This provides detailed evidence for the formulation and adjustment of rehabilitation treatment plans, provides repeatable and traceable vivid cases for medical teaching, and provides a standardized interactive medium for remote consultation and cross-institutional collaboration. Therefore, this invention is not only an auxiliary device, but also a platform tool that can improve the overall medical quality control and medical education level. Attached Figure Description
[0029] Figure 1 : Overall architecture block diagram of the intelligent palpation positioning system described in this invention.
[0030] Figure 2 : Schematic diagram of sensor distribution on the palm side of the multimodal sensing glove in this invention.
[0031] Figure 3 : A schematic diagram of the sensor distribution on the back of the multimodal sensing glove in this invention.
[0032] Figure 4 : A schematic diagram of the workflow of the intelligent recognition and positioning engine in this invention.
[0033] Figure 5 : A schematic diagram of the complete process from reference point identification to muscle extension localization in the method of the present invention.
[0034] Explanation of reference numerals in the attached figures: 1—Electromyography sensor; 2—Pressure sensor array; 3—Inertial processing microprocessor unit; 4—Flexible circuit wiring; 5—Inertial measurement unit; 6—Data interface and microprocessor unit; 7—Pressure sensor array. Detailed Implementation
[0035] The following is combined Figures 1 to 5 The present invention will be further described as follows: like Figure 1 As shown, Figure 1 As shown, the intelligent palpation positioning system of the present invention mainly comprises four parts in terms of both physical and logical aspects: a multimodal sensing glove, an anatomical digital model database, an intelligent recognition and positioning engine, and an interactive and display terminal. During system operation, the operator wears the multimodal sensing glove to perform palpation on the subject; the multi-source sensor data collected by the glove is transmitted via wired (e.g., USB) or wireless (e.g., Bluetooth 5.2, Wi-Fi) methods to the computing unit running the intelligent recognition and positioning engine, such as a high-performance tablet computer, a dedicated processing box, or a cloud server; the engine processes, matches, and infers the sensor data in real time by calling the local or remote anatomical digital model database; the final result is fed back to the operator in real time through the interactive and display terminal, such as augmented reality glasses, a tablet computer screen, or a projection device.
[0036] like Figure 2 and Figure 3 As shown, the multimodal sensing glove is the physical interface for system data acquisition, and its core is a variety of sensors integrated on the palm and back of the hand of the glove substrate.
[0037] The distributed pressure sensor arrays 2 and 7 are the core components for achieving tactile perception. For example... Figure 2 As shown, flexible piezoresistive or capacitive pressure sensors are embedded in the main palpable areas of the fingertips, finger pads, and palm of the glove in a high-density grid, such as 4-16 sensing nodes per square centimeter. These sensor arrays can measure the pressure distribution, the movement trajectory of the pressure center point, and dynamic pressure changes on the palpable contact surface in real time and at high resolution.
[0038] like Figure 3 As shown, a 9-axis inertial measurement unit 5 is integrated into the back of the hand or wrist, including a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. The IMU is used to accurately measure the absolute attitude (pitch, roll, yaw angle), angular velocity, and acceleration of the hand in three-dimensional space, thereby tracking the movement trajectory and spatial position of the hand in real time.
[0039] like Figure 3 As shown, a surface electromyography (SEMG) sensor 1 with at least one channel is integrated into the sleeve portion of the glove extending to the forearm. This SEMG sensor 1 is used to monitor weak electrical signals of the relevant muscle groups in the operator's hand and forearm. These signals are directly related to the operator's palpation intentions, such as preparing to apply force, maintaining a light touch, or performing deep pressure, providing auxiliary information for the system to determine the operator's force application state and palpation pattern.
[0040] The glove integrates a microcontroller unit: an inertial processing microprocessor unit 3 and a data interface and microprocessor unit 6, responsible for synchronously acquiring the raw signals from the aforementioned multi-source sensors, such as performing 100Hz sampling rate, preliminary filtering, noise reduction, and analog-to-digital conversion. The processed data is packaged through the data interface and microprocessor unit 6, aggregated via flexible circuit trace 4, and then transmitted to the main computing unit via a wireless module.
[0041] Workflow of the intelligent recognition and positioning engine of this invention like Figure 4 As shown, the intelligent recognition and positioning engine is the brain of the system, and its software workflow includes the following core modules in sequence: The data input and alignment module receives the raw data stream from the glove and first performs spatiotemporal alignment of the multi-source (pressure, IMU, sEMG) data to ensure that all data have a unified timestamp and spatial reference system.
[0042] The reference point identification module is activated at the initial stage of palpation (e.g., the first 3-5 seconds of stable contact). It extracts features from the current pressure distribution (such as contour shape, central pressure value, and hardness characteristics) and, combined with the absolute spatial pose information provided by the IMU, matches them against pre-stored bony landmarks and joint feature databases for different body positions in the anatomical digital model database. The matching algorithm can employ a pre-trained deep learning classification model (such as a fusion model of convolutional neural network (CNN) and long short-term memory network (LSTM)). Once a match is successful (e.g., identifying the acromion), that point is established as the origin of the anatomical coordinate system for the current palpation.
[0043] Once the reference point is identified, the spatial mapping and coordinate system construction module immediately establishes a local three-dimensional Cartesian coordinate system with that point as the origin, based on the standard anatomical orientation of the human body. Simultaneously, through a coordinate transformation algorithm, the sensor coordinate system on the glove is dynamically mapped to this fixed local anatomical coordinate system in real time, ensuring that all subsequent hand movements can be accurately tracked in anatomical space.
[0044] Muscle Extension Recognition Module: This module is activated when the operator's finger begins to slide from the identified reference point. It continuously tracks the movement path, direction, and speed (trajectory data) of the finger's pressure center. Simultaneously, it queries the anatomical model database for the course, surface projection range, and spatial relationships of all muscles and tendons originating from or passing through this reference point. Using trajectory-model path matching algorithms (such as Dynamic Time Warping (DTW) or probability-based Bayesian inference), the system calculates the matching degree between the current movement trajectory and each candidate muscle path in real time. The muscle with the highest matching degree (e.g., the long head tendon of the biceps brachii) is used as the initial recognition result, and its most likely surface projection area is calculated.
[0045] Precise Positioning Confirmation Module: This module plays a crucial role in further improving recognition accuracy, especially when dealing with overlapping muscle layers. It is triggered when the system detects repeated pressure, circumferential pressure, or deep pressure applied by the operator in the initially identified area. It analyzes the changing patterns of pressure distribution (such as peak pressure and pressure diffusion shape), sEMG signal intensity changes, and subtle adjustments to the trajectory under these dynamic operations. These dynamic mechanical characteristics are used to distinguish between superficial and deep tissues, or between tendons and muscle bellies. For example, cord-like tendons exhibit concentrated and minimally variable pressure distribution under repeated pressure, while muscle bellies show more diffuse pressure. By calibrating and validating the initial results, a final high-confidence positioning result is output.
[0046] like Figure 5 As shown, taking palpation of the shoulder joint area as an example, the complete working process of the system is demonstrated: Initial contact and reference point identification, corresponding Figure 5 The upper part: The operator's gloved index finger makes stable contact with the patient's outer shoulder. The system collects initial pressure and posture data, matches it with the model through the reference point recognition module, and successfully provides feedback of "recognition result: acromion" at the acromion position, and establishes an anatomical coordinate system (X, Y, Z axes) centered on this point.
[0047] Muscle extension recognition and tracking, corresponding Figure 5 The lower part: The operator slides their fingers forward and downward from the acromion, intending to locate the long head tendon of the biceps brachii. The system tracks this sliding trajectory in real time and immediately highlights the surface projection path of the long head tendon of the biceps brachii in the anatomical model below the trajectory, while displaying "Trajectory analysis in progress... Matching degree: 85%" on the UI interface. This provides the operator with intuitive real-time guidance.
[0048] Precise location confirmation: The operator repeatedly presses the highlighted area several times to confirm the tendon's location. The system analyzes the dynamic pressure characteristics of the multiple presses, confirming that the target is a cord-like structure, thus refining the identification result and finally confirming it as "biceps brachii long head tendon (surface projection point)", while increasing the confidence level to 96%, and notifying the operator that the location is complete through an interactive terminal (such as a short vibration).
[0049] All sensor data, identified anatomical structure sequences, palpation paths, key location coordinates, timestamps, and operator-added voice annotations throughout the palpation process are encrypted and stored by the system. Based on this structured data, the intelligent recognition and positioning engine can automatically generate a standardized palpation assessment report for use in medical record archiving, treatment planning, or teaching review.
[0050] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the implementation of the present invention is not limited to the above-described manner. Any improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.
Claims
1. An intelligent palpation positioning system based on multimodal perception and anatomical model matching, characterized in that, include: Multimodal sensing gloves are used to collect multi-source physical signals during the operator's palpation process; Anatomical digital model database, storing three-dimensional human anatomical models with semantic labels and spatial coordinate information; The intelligent recognition and positioning engine is used to perform real-time matching and reasoning between the multi-source physical signals and the models in the anatomical digital model database in order to identify and locate the specific anatomical structures being palpated. An interactive and display terminal is used to present the recognition and positioning results output by the intelligent recognition and positioning engine in real time.
2. The intelligent palpation positioning system based on multimodal perception and anatomical model matching according to claim 1, characterized in that: The multimodal sensing glove includes: A distributed pressure sensor array, integrated into the touch contact area of the glove, is used to measure the pressure distribution and dynamic changes on the touch contact surface; An inertial measurement unit, integrated into the glove, is used to measure the spatial posture and motion trajectory of the hand.
3. The intelligent palpation positioning system based on multimodal perception and anatomical model matching according to claim 2, characterized in that: The multimodal sensing gloves also include: An electromyography (EMG) sensor, located in the extension of the glove, is used to monitor the EMG activity signals of the operator's hand or forearm to help determine the intention to palpate and the state of force applied.
4. The intelligent palpation positioning system based on multimodal perception and anatomical model matching according to claim 1, characterized in that: The intelligent recognition and positioning engine includes: The reference point identification module is used to identify bony landmarks or joints by matching the sensor data from the initial palpation stage with the anatomical model, and use these as reference points for spatial positioning. The muscle extension recognition module is used to infer and identify the muscle or tendon tissue associated with the reference point based on the reference point recognition, the trajectory data during the palpation movement process, and the anatomical model information.
5. The intelligent palpation positioning system based on multimodal perception and anatomical model matching according to claim 4, characterized in that: The intelligent recognition and positioning engine also includes: The precise positioning and confirmation module is used to analyze the dynamic signal characteristics generated by repeated or deep pressure on a specific area, distinguish layered muscles or different tissues, and perform confidence calibration and final confirmation on the recognition results.
6. The intelligent palpation positioning system based on multimodal perception and anatomical model matching according to claim 1, characterized in that: The interactive and display terminal is an augmented reality device used to overlay the names, boundaries, or hierarchical relationships of the identified anatomical structures onto the operator's real field of vision as virtual images.
7. The intelligent palpation positioning system based on multimodal perception and anatomical model matching according to claim 1, characterized in that: The system also includes a data storage and report generation module, which is used to encrypt and store the complete palpation path, the identified anatomical structure sequence, key positioning point data and operation annotations, and generate a structured palpation report.
8. A smart palpation positioning method based on the system according to any one of claims 1-7, characterized in that, Includes the following steps: Step S1: Simultaneously collect multi-source sensor data streams during the operator's palpation operation using a multimodal sensing glove; Step S2: Match the multi-source sensor data with the pre-stored anatomical digital model to identify the initial anatomical reference point to be palpated and establish a spatial mapping relationship; Step S3: Using the initial anatomical reference point as a reference, based on the continuous palpation movement trajectory data and combined with the topological relationship in the anatomical model, extend to identify and locate the surrounding muscles or soft tissues associated with the reference point. Step S4: Output and display the identification and positioning results in real time.
9. The intelligent palpation positioning method according to claim 8, characterized in that: In step S3, the direction, length, speed and pressure distribution changes of the palpation movement path are analyzed, and probability matching calculations are performed with the muscle direction, origin and insertion points and surface projection information in the anatomical model to infer the target muscle or tendon.
10. The intelligent palpation positioning method according to claim 8, characterized in that: It also includes step S5: by analyzing the dynamic mechanical characteristics generated by repeated or deep pressure on a specific area, distinguishing between superficial and deep muscles or different tissue types, and verifying and precisely locating the preliminary identification results of step S3; It also includes step S6: encrypting and storing the palpation movement trajectory, the identified anatomical structure sequence, key positioning point information and operation process data, and automatically generating a structured palpation record report.