Acupuncture point positioning system of acupuncture robot based on cooperation of visual identification and mechanical arm

Abstract

The invention discloses an acupuncture robot acupoint positioning system based on visual identification and mechanical arm cooperation. The system comprises an image acquisition module, an image preprocessing module, an acupoint identification module, a mechanical arm control module, a data storage and optimization module and a doctor end interaction module. Image features are extracted through high-speed image acquisition and an efficient image preprocessing algorithm; a YOLO algorithm carried in an embedded system realizes the recognition of key nodes of a human body, further determines the position of an acupuncture point based on cun measurement calculation, and improves the recognition accuracy through the comparison of multiple frames of images; the mechanical arm moves to the position above the acupuncture point to execute operations such as disinfection and needle-knife operation; a doctor can perform real-time monitoring and intervention through the control panel, and the system also stores diagnosis and treatment data to a cloud end and optimizes an algorithm regularly. The problems that traditional acupuncture point positioning is low in precision, poor in efficiency, insufficient in adaptability and the like are effectively solved, the precision, efficiency and stability of acupuncture diagnosis and treatment are remarkably improved, and the method has wide application prospects in traditional Chinese medicine clinical treatment, first aid and other scenes.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and in particular to an acupuncture robot acupoint positioning system based on visual recognition and robotic arm collaboration. Background Technology

[0002] In traditional acupuncture treatment, acupoint location mainly relies on the doctor's experience and manual operation. This method is not only inefficient, but its accuracy is also easily affected by the doctor's subjective factors and fatigue. Especially when dealing with a large number of patients, it is difficult to ensure the consistency and stability of treatment. At the same time, for some inexperienced doctors, accurately locating acupoints and performing precise acupuncture is quite challenging.

[0003] The current medical industry places higher demands on the precision, safety, and efficiency of acupuncture treatment. Precise acupoint location is crucial to ensuring the efficacy of acupuncture, a requirement that traditional methods struggle to meet. Furthermore, with the aging population and increasing demand for Traditional Chinese Medicine (TCM) treatments, improving the efficiency and quality of acupuncture treatment is urgently needed. Against this backdrop, an acupoint location system based on visual recognition and robotic arm collaboration has emerged, aiming to overcome the limitations of traditional acupuncture and provide patients with more reliable and efficient treatment services. Summary of the Invention

[0004] The technical problem solved by this invention is the low efficiency of traditional manual acupoint location. It changes the reliance on doctors' experience to determine acupoints one by one, significantly improving the speed of diagnosis and treatment. It also overcomes the problem of poor accuracy by using innovative algorithms and technologies to reduce location errors caused by doctors' subjective factors and individual differences.

[0005] The technical solution of this invention is: an acupuncture robot acupoint positioning system based on visual recognition and robotic arm collaboration, comprising an image acquisition module, an image preprocessing module, an acupoint recognition module, a robotic arm control module, a data storage and optimization module, and a doctor-side interaction module, as detailed below:

[0006] Preferably, an acupuncture robot acupoint positioning system based on visual recognition and robotic arm collaboration is characterized in that,

[0007] The image acquisition module consists of a depth camera, which is used to acquire images of the patient's skin surface in real time and transmit the acquired video information to the doctor's interactive module, while simultaneously inputting the images into the embedded system inside the robot.

[0008] The image preprocessing module performs preprocessing operations such as denoising, edge detection, and image enhancement on the acquired images to output high-quality image data.

[0009] The acupoint recognition module uses a lightweight YOLO algorithm embedded in the system. By identifying key nodes of the human body and the positional information between the key nodes and acupoints, it determines the precise location of each acupoint based on measurement and calculation, and improves the recognition accuracy by comparing multiple frames of images.

[0010] After the acupoint is initially located, the robotic arm control module controls the robotic arm to move above the acupoint and disinfects the skin of the acupoint. At the same time, the video of the acupoint location result is transmitted to the doctor's interactive module via the network.

[0011] The data storage and optimization module stores the acupoint location information in a cloud storage server after each acupoint search operation. The system optimizes the image recognition algorithm by analyzing historical data to improve the accuracy and efficiency of acupoint search.

[0012] The doctor-side interaction module allows doctors to view the positioning results in real time via the control panel, and send confirmation signals or correction commands. The operation SDK converts the correction commands into real-time adjustment commands for the robot.

[0013] Preferably, an acupuncture robot acupoint localization system based on visual recognition and robotic arm collaboration is characterized by comprising the following steps:

[0014] (1) After the system starts up, the depth camera in the image acquisition module starts to work and acquires images of the patient's skin surface in real time. These images are transmitted to the doctor's interactive module to facilitate the doctor to view the patient's condition in real time for diagnosis. On the other hand, the images are input into the embedded system inside the robot to prepare for subsequent analysis and processing.

[0015] (2) The image entering the embedded system is then transmitted to the image preprocessing module. In this module, the image undergoes noise reduction, edge detection and image enhancement in sequence to output high-quality image data, providing a clear and accurate image basis for acupoint recognition.

[0016] (3) High-quality image data enters the acupoint recognition module. This module uses a lightweight YOLO algorithm to determine the precise location of the acupoint. At the same time, in order to reduce the recognition error of a single frame image, this module compares multiple frames of images to further improve the recognition accuracy.

[0017] (4) After the acupoint recognition is completed, the robotic arm control module receives the positioning information and controls the robotic arm to move above the acupoint to disinfect the acupoint skin. At the same time, the acupoint positioning result video is transmitted to the doctor's terminal interaction module through the network, so that the doctor can keep track of the positioning in real time.

[0018] (5) The doctor can check the positioning results through the control panel. If the positioning is considered accurate, the doctor can send a confirmation signal to allow the robot to continue the subsequent needle knife operation. If the positioning is inaccurate, the doctor can issue a correction command. The operation SDK will convert the command into the robot's real-time adjustment command to ensure that the robot can accurately adjust the acupoint position according to the doctor's feedback.

[0019] (6) After each acupoint search operation is completed, the data storage and optimization module stores the acupoint location information in the cloud storage server. The system will periodically analyze this historical data, optimize the image recognition algorithm, improve the accuracy and efficiency of acupoint search, and achieve continuous improvement of robot needle knife operation.

[0020] Preferably, the image acquisition module employs a depth camera, which converts a two-dimensional depth image into point cloud data in three-dimensional space by measuring the distance to each pixel in the scene. Assuming the intrinsic parameter matrix of the depth camera is... The three-dimensional coordinates (X, Y, Z) of this pixel in the camera coordinate system can be calculated using the following formula: Where f x f y Let c be the focal length of the camera in the x and y directions, respectively. x c y The x and y coordinates of the principal point of the image are given, and d(x, y) is the depth value of that pixel.

[0021] Preferably, the depth information acquired by the depth camera needs to be registered with the color visual image. This allows the depth information to be combined with human features in the image. Through the camera calibration process, the conversion relationship between the depth camera and the color camera is obtained, which can be represented by a rotation matrix R and a translation vector T. Where x d y d z d This represents the coordinates of the point in the 3D space of the depth camera coordinate system. The trailing "1" indicates a homogeneous term used for matrix operations during homogeneous transformations. c y c z c It is the coordinate value of the point in the three-dimensional space under the color camera coordinate system. Similarly, the "1" at the end is a homogeneous term.

[0022] Preferably, the acupoint recognition module includes preprocessing operations such as noise reduction, edge detection, and image enhancement.

[0023] The denoising preprocessing operation employs bilateral filtering, which can smooth noise while preserving the geometric details of the skin surface. First, key parameters are determined based on the characteristics of the point cloud data and the noise level, including the spatial standard deviation (σ). s ), range standard deviation (σ) rAnd the size of the neighborhood N(p), then for each point p in the point cloud, traverse the points q in its neighborhood N(p), according to the formula and Calculate the spatial weights w respectively s (p, q) and range weight w r (p, q), and finally according to the bilateral filtering formula (where the depth value of point q is used as I(q)) to update the point cloud value. This process suppresses point cloud noise, preserves geometric details, and provides a good data foundation for subsequent acupoint localization based on point cloud features.

[0024] The edge detection preprocessing operation identifies the edge information of objects in the image using a specific edge detection algorithm. Edges are areas of dramatic grayscale changes in an image, representing important information such as the object's contour. Edge detection highlights the boundary between the acupoint location and surrounding tissues, providing clearer features for subsequent acupoint identification. Using Canny edge detection, a three-dimensional Gaussian filter is first applied, with the Gaussian filter kernel function being... Then, the gradient magnitude and direction are calculated. The Sobel operator is used to calculate the gradient magnitude G and gradient direction θ. For each pixel, its gradient magnitude is compared with the gradient magnitudes of its neighboring pixels based on its gradient direction. If it is not a local maximum, its gradient magnitude is set to 0. Finally, two thresholds T are set. h (High threshold) and T l (Low threshold) Gradient magnitude greater than T h The pixels marked as strong edges have an amplitude less than T. l Pixels with magnitudes between two values ​​are marked as non-edges; if they are adjacent to strong edges, they are marked as edges; otherwise, they are marked as non-edges.

[0025] The image enhancement preprocessing operation enhances the image, improving its clarity, contrast, and other visual effects. It utilizes three-dimensional histogram equalization, where the histogram counts the number of pixels at different spatial locations and depth values, making the details in the image more obvious and facilitating more accurate analysis and location of acupoints by the acupoint recognition module.

[0026] Preferably, the acupoint recognition module includes the following steps:

[0027] (1) Using a lightweight YOLO algorithm optimized for human acupoint location, we can identify key nodes of the human body, such as fingertips, hairline edges, joint nodes, etc. These key nodes are important landmarks of human structure. Their positions are relatively fixed and easy to identify, providing a basic reference for subsequent acupoint location.

[0028] (2) Based on the location information between the key nodes and acupoints, the precise location of each acupoint is determined by using a method similar to the anatomical measurement commonly used by doctors, based on measurement and calculation. There are relatively fixed distances or positional relationships between different acupoints and specific key nodes. By quantifying these relationships, the location of the acupoints can be accurately found.

[0029] (3) In order to reduce the possible errors in single-frame image recognition, the system will further determine the acupoint location by comparing multiple frames of images. Since single-frame images may be affected by factors such as changes in light and slight movements of the human body, the identification of key nodes or the calculation of acupoints may be deviated. By comparing multiple frames of images and comprehensively analyzing the information of key nodes and acupoints in each frame, these errors can be effectively reduced and the accuracy of acupoint positioning can be improved.

[0030] Preferably, the key nodes are obtained by collecting and organizing a large amount of human anatomy research data, medical imaging data and clinical practice experience to obtain the precise positional relationship information between each key node and each related acupoint in three-dimensional space. This information includes the straight-line distance between the key node and the acupoint, the direction vector, and the positional relationship change pattern under different human body shapes and postures. Then, the collected three-dimensional positional relationship data is stored in a database. The database is indexed by key nodes, and each key node corresponds to a list containing three-dimensional positional relationship information of multiple acupoints.

[0031] Preferably, the measurement first involves storing the positional relationship information of each acupoint, including the acupoint name, distance d, and direction vector (θ). x θ y θ z ) and other related correction parameters, given the 3D coordinates (X0, Y0, Z0) of the human key nodes in the camera coordinate system, and the distance d between a key node and an acupoint found in the database, with the direction vector (θ) x θ y θ z First, convert "cun" to the actual length unit in the camera coordinate system (assuming 1 cun corresponds to an actual length of k). Then, the distance in the camera coordinate system is D = d × k. According to the principle of trigonometric functions, calculate the three-dimensional coordinates (X, Y, Z) of the acupoint in the camera coordinate system. The calculation formula is:

[0032] Preferably, the robotic arm control module includes the following steps:

[0033] (1) The acupoint recognition module transmits this positioning information to the robotic arm control module. The module receives the coordinate information of the acupoint in the coordinate system constructed based on the images captured by the camera.

[0034] (2) The robotic arm control module converts the image coordinates of the acupoints into the actual coordinates in the robotic arm workspace, and clarifies the transformation relationship between the image coordinate system (based on the camera imaging plane), the camera coordinate system (based on the camera optical center as the origin), and the robotic arm workspace coordinate system.

[0035] (3) The robotic arm control module plans a reasonable movement path based on the current position of the robotic arm and the position of the target acupoint;

[0036] (4) The robotic arm control module sends control signals to the robotic arm according to the planned path to precisely control the movement angle and speed of each joint of the robotic arm;

[0037] (5) When the robotic arm approaches the target acupoint, the robotic arm control module will make fine adjustments based on the feedback information in order to achieve high-precision positioning.

[0038] Preferably, the coordinate transformation from the image coordinate system to the camera coordinate system takes into account pixel size and image distortion correction. The transformation from the camera coordinate system to the robotic arm workspace coordinate system is based on the installation position and posture of the robotic arm. Assuming the position of the robotic arm base in the world coordinate system is given by the rotation matrix R, and the posture is represented by the rotation matrix R, the acupoint coordinates in the camera coordinate system can be converted to the coordinates in the robotic arm workspace coordinate system through a series of matrix operations, in combination with the extrinsic parameter matrix of the camera.

[0039] Preferably, the path planning uses a grid map method to model the working environment, and uses the A* algorithm to determine an optimal path. Its evaluation function is f(n) = g(n) + h(n), where g(n) is the actual cost from the starting point to node n, and h(n) is the estimated cost from node n to the target point. The Euclidean distance is used for calculation. Starting from the grid node corresponding to the current position of the robotic arm, the search for adjacent nodes is continuously expanded. The optimal expansion direction is selected according to the evaluation function value until the grid where the target acupoint is located is found, thereby determining a potential path from the starting point to the target point. Furthermore, a path smoothing algorithm is used to generate a new smooth curve as the optimized path by removing unnecessary turns and redundant nodes in the path.

[0040] Preferably, the cloud storage server in the data storage and optimization module adopts a distributed storage architecture to ensure data security and scalability;

[0041] The advantages of this invention compared to existing technologies are as follows: Traditional acupuncture positioning mainly relies on the doctor's experience and feel, which is subject to a certain degree of subjectivity and individual differences. However, this system utilizes advanced visual recognition technology to accurately identify the location and shape of acupoints on the human body. This system can assist doctors in completing repetitive tasks such as acupoint positioning and acupuncture, improving the quality and efficiency of medical services and providing new ideas and methods for the modernization and intelligent development of acupuncture treatment. Attached Figure Description

[0042] Figure 1 This is a system block diagram of the present invention;

[0043] Figure 2 This is a block diagram illustrating the implementation of the present invention; Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] This invention provides a technical solution: an acupuncture robot acupoint positioning system based on visual recognition and robotic arm collaboration, including an image acquisition module, an image preprocessing module, an acupoint recognition module, a robotic arm control module, a data storage and optimization module, and a doctor-side interaction module.

[0046] A process method for an acupuncture robot acupoint localization system based on visual recognition and robotic arm collaboration, such as... Figure 2 As shown, it includes the following steps:

[0047] Step 1: Image Acquisition and Preprocessing. The camera acquires real-time images of the patient's skin surface. This camera not only transmits video information to the doctor's interactive module for diagnosis, but also inputs these images into the robot's embedded system for analysis and processing.

[0048] Step Two: Simultaneously with image acquisition, the embedded system performs preprocessing, including denoising, edge detection, and image enhancement, to ensure image clarity and effectiveness, providing high-quality image data for subsequent acupoint recognition. During noise preprocessing, bilateral filtering is used to smooth noise while preserving the geometric details of the skin surface. First, key parameters are determined based on the characteristics of the point cloud data and the noise level, including the spatial standard deviation (σ). s ), range standard deviation (σ) r And the size of the neighborhood N(p); then for each point p in the point cloud, traverse the points q in its neighborhood N(p), according to the formula and Calculate the spatial weights w respectively s (p, q) and range weight w r (p, q), and finally according to the bilateral filtering formula (where the depth value of point q is used as I(q)) to update the point cloud value. This process suppresses point cloud noise, preserves geometric details, and provides a good data foundation for subsequent acupoint localization based on point cloud features.

[0049] In the edge detection preprocessing operation, a specific edge detection algorithm is used to identify the edge information of objects in the image. Edges are the parts of the image with drastic grayscale changes, representing important information such as the object's outline. Edge detection can highlight the boundary between the acupoint area and the surrounding tissue, providing clearer features for subsequent acupoint identification. Using Canny edge detection, a three-dimensional Gaussian filter is first performed, with the Gaussian filter kernel function being... Then, the gradient magnitude and direction are calculated. The Sobel operator is used to calculate the gradient magnitude G and gradient direction θ. For each pixel, its gradient magnitude is compared with the gradient magnitudes of its neighboring pixels based on its gradient direction. If it is not a local maximum, its gradient magnitude is set to 0. Finally, two thresholds T are set. h (High threshold) and T l (Low threshold) Gradient magnitude greater than T h The pixels marked as strong edges have an amplitude less than T. l Pixels with amplitudes between two values ​​are marked as non-edges. Pixels that are adjacent to strong edges are marked as edges, otherwise they are marked as non-edges.

[0050] In image enhancement preprocessing, image enhancement operations are performed to improve visual effects such as image clarity and contrast. Three-dimensional histogram equalization is used; the histogram counts the number of pixels at different spatial locations and depth values, making image details more apparent and facilitating more accurate analysis and location of acupoints by the acupoint recognition module. This is achieved by calculating the three-dimensional cumulative distribution function CDF(d, X, Y, Z) and applying the formula d... new =CDF(d, X, Y, Z) × D max Image enhancement is achieved by adjusting histogram equalization parameters, such as dividing the pixel value range into multiple intervals and stretching pixels in different intervals to varying degrees, thereby improving image clarity and contrast and making human structural features in the image more prominent.

[0051] Step 3: After image preprocessing, the robot uses the lightweight YOLO algorithm, which can be mounted on an embedded system, to automatically detect and locate the patient's acupoints. The YOLO algorithm first identifies key nodes of the human body (e.g., fingertips, hairline edges, joint nodes, etc.). Through the collection and organization of a large amount of human anatomy research data, medical imaging data, and clinical practice experience, it obtains the precise positional relationship information between each key node and each related acupoint in three-dimensional space. This information includes the straight-line distance between the key node and the acupoint, the direction vector, and the changes in positional relationship under different body shapes and postures. Then, the collected three-dimensional positional relationship data is stored in a database. The database is indexed by key nodes, and each key node corresponds to a list containing three-dimensional positional relationship information of multiple acupoints.

[0052] Then, based on the positional information between these key nodes and acupoints, the location of the acupoints is determined. This process is similar to the anatomical measurement method (cun) commonly used by doctors. Based on cun measurement and calculation, the acupoints are located to determine the precise location of each acupoint. The positional relationship information of each acupoint includes the acupoint name, distance d, and direction vector (θ). x θ y θ z ) and other related correction parameters, given the 3D coordinates (X0, Y0, Z0) of the human key nodes in the camera coordinate system, and the distance d between a key node and an acupoint found in the database, with the direction vector (θ) x θ y θ z First, convert "cun" to the actual length unit in the camera coordinate system (assuming 1 cun corresponds to an actual length of k). Then, the distance in the camera coordinate system is D = d × k. According to the principle of trigonometric functions, calculate the three-dimensional coordinates (X, Y, Z) of the acupoint in the camera coordinate system. The calculation formula is:

[0053] Step 4: Using the intrinsic and extrinsic parameter matrices of the depth camera obtained through pre-calibration, combined with the installation position and attitude parameters of the robotic arm, the coordinates of the acupoint in the image coordinate system are converted to the coordinates in the robotic arm's workspace coordinate system through homogeneous coordinate transformation. The A* algorithm is then used for path planning, dividing the robotic arm's workspace into a 10×10×10 grid map. When planning the path, the robotic arm's motion limitations and the position information of surrounding obstacles are considered to ensure that the robotic arm can move safely and quickly above the acupoint, with the path planning time controlled within 0.5 seconds. The arm control module controls the motor movement of each joint of the robotic arm by sending PWM pulse signals according to the planned path, enabling the robotic arm to accurately move above the acupoint. After reaching the designated position, the disinfection device at the front end of the robotic arm uses a spray disinfection method, using 75% alcohol to disinfect the acupoint epidermis for 5 seconds.

[0054] Step 5: The control panel of the doctor's interactive module uses a touch screen to display the acupoint location results in an intuitive graphical user interface. The interface displays real-time images of the patient captured by the robot and the identified acupoint location markings, while also providing operation buttons for doctors to confirm or correct their actions. When the doctor clicks the confirmation button, the control panel sends a confirmation signal to the robot via the network. When the doctor issues a correction command, they manually adjust the acupoint location markings on the touch screen. The control panel then transmits the corrected coordinate information to the robot's operation SDK via the network. The operation SDK converts the command into the robot's real-time adjustment action within 0.1 seconds.

[0055] Step Six: Store the acupoint location information for each acupoint search operation, including the patient's basic information, image data, the coordinates of the identified acupoints, and the operation time, in JSON format on a distributed storage server in the cloud. The storage server employs redundant storage technology to ensure data security and reliability. Simultaneously, the system regularly analyzes the historical data stored in the cloud weekly. Optimization algorithms from deep learning frameworks, such as the Adam optimization algorithm, are used to adjust and optimize the parameters of the image recognition algorithm. Through continuous algorithm optimization and analysis of a large amount of treatment data, the system can continuously optimize the image recognition algorithm, improve the accuracy and efficiency of acupoint search, and ultimately achieve continuous learning and improvement of the robotic needle knife operation.

[0056] The camera is a depth camera with high frame rate and high resolution to ensure that it can clearly capture the subtle features of the human skin surface. During installation, the camera angle and position are adjusted according to the robot's workspace and the patient's common acupuncture positions to ensure that the acupuncture site is fully covered and to avoid obstruction and reflection interference.

[0057] The embedded system configuration employs a powerful embedded system motherboard equipped with a high-performance processor, large-capacity memory, and high-speed storage devices. An image acquisition driver is integrated into the system to ensure a stable connection with the depth camera, enabling rapid transmission and reception of image data. Simultaneously, an image preprocessing algorithm library and acupoint recognition algorithm model are deployed to ensure efficient processing of image data.

[0058] The robotic arm control module has multi-degree-of-freedom control capabilities, which can precisely control the movement of each joint of the robotic arm. It is connected to the embedded system through a high-speed communication interface, receives acupoint coordinates and path planning information from the embedded system, and quickly responds to control the robotic arm to move above the designated acupoint.

[0059] The acupoint recognition algorithm first collects a large amount of image data with different human characteristics (including different ages, body types, skin colors, etc.), and accurately labels key human nodes and acupoint locations in each image. Using transfer learning, it fine-tunes the training for acupoint recognition based on a pre-trained general object detection model. Appropriate training parameters are set, such as 400-600 iterations and a learning rate of 0.001-0.01, to improve the model's accuracy and generalization ability for acupoint recognition. Then, the pre-processed images are input into a trained lightweight YOLO algorithm model. The model quickly identifies key human nodes and, based on a pre-established database of key node-acupoint location relationships, uses measurement to determine the precise location of acupoints. Simultaneously, it compares multiple consecutive frames of images and comprehensively analyzes changes in acupoint locations to reduce recognition errors in single frames.

[0060] For situations where skin diseases or tattoos may interfere with visual features, depth information is incorporated into the denoising or image enhancement process for filtering. Depth difference is added as a weighting factor. For each pixel in the depth image, its depth difference with neighboring pixels is calculated, and the filtering weights are adjusted based on this difference. The lightweight YOLO algorithm is improved by adding annotation and learning of deep structural features during training. In addition to identifying the surface positions of key nodes, the relative positional relationships between key nodes and bone endpoints are learned, such as subtle changes in bone contours in the depth image and muscle tension changes under different movements. This reduces reliance on surface texture features susceptible to skin diseases and tattoos. Simultaneously, a fusion... The hybrid model combines an anatomically based rule engine with a machine learning model. The rule engine stores basic anatomical rules, such as the relative positions of acupoints with structures like bones, muscles, and nerves. During the recognition process, the machine learning model inputs extracted image features into the rule engine. The rule engine then uses anatomical knowledge and input features to reason, helping to correct acupoint location errors that may be caused by skin lesions or tattoos. Once the machine learning model identifies key node locations, the rule engine infers the approximate location range of the acupoint based on known anatomical relationships and then further refines the acupoint location by combining image features, thereby reducing the influence of skin surface interference factors.

[0061] The robotic arm path planning algorithm first divides the robotic arm's workspace into small grids, such as 10×10×10 or 20×20×20 grid maps. Based on the actual size of the robotic arm and the location of potential obstacles, it determines whether each grid is passable. If an obstacle exists in a grid area, such as medical equipment placed in the workspace, the grid is marked as impassable. The A* algorithm uses an evaluation function f(n) = g(n) + h(n) to select the optimal path, where g(n) represents the actual cost of moving from the robotic arm's starting position to the current grid node. This cost can be calculated from factors such as the rotation angle and movement distance of each joint of the robotic arm. The rotation of each joint of the robotic arm requires a certain amount of energy, and g(n) is calculated based on the relationship between the rotation amount of the joint and the energy consumption. h(n) is the distance from the current grid node to the current grid node. The estimated cost of the grid where the target acupoint is located is generally estimated using Euclidean distance or Manhattan distance. Starting from the grid node corresponding to the current position of the robotic arm, the node is added to a priority queue (such as a min-heap). The priority queue is sorted in ascending order according to the value of the evaluation function f(n). Each time, the node with the smallest f(n) value is taken from the priority queue for expansion. That is, the adjacent grid nodes of the node are checked (adjacent grids in the directions of up, down, left, right, front, back, etc., the movement restrictions of the robotic arm need to be considered, such as certain directions that cannot be directly moved). For each passable adjacent node, its f(n) value is calculated and added to the priority queue. If the expanded node is the grid node where the target acupoint is located, a path from the starting position to the target acupoint has been found. Finally, by backtracking the path from the target node to the starting node, the complete movement path of the robotic arm is obtained.

[0062] In summary, through the above implementation methods, this acupuncture robot control system utilizes visual recognition and robotic arm collaboration to perform remote acupuncture treatment, greatly improving the accuracy of acupoint location, significantly enhancing treatment efficiency, and raising the level of intelligence, thus providing a new and efficient mode for acupuncture treatment.

Claims

1. An acupuncture robot acupoint positioning system based on visual recognition and cooperation with a mechanical arm, characterized in that, The application relates to an image-based acupoint positioning system, which comprises an image acquisition module, an image preprocessing module, an acupoint recognition module, a mechanical arm control module, a data storage and optimization module and a doctor terminal interaction module. The image acquisition module is composed of a depth camera and is used for collecting patient skin surface images in real time and transmitting the collected video information to the doctor terminal interaction module, and simultaneously inputting the images into an embedded system in the robot; The image preprocessing module performs denoising, edge detection and image enhancement and other preprocessing operations on the collected images to output high-quality image data; The acupoint recognition module adopts a lightweight YOLO algorithm loaded in the embedded system, determines key nodes of a human body, determines the accurate positions of each acupoint based on the position information between the key nodes and the acupoints and the inch measurement and calculation, and improves the recognition accuracy by comparing multiple images; The mechanical arm control module controls the mechanical arm to move above the acupoint and disinfect the acupoint skin after the acupoint is preliminarily positioned, and simultaneously transmits the acupoint positioning result video to the doctor terminal interaction module through a network; The data storage and optimization module stores the acupoint positioning information in a storage server in the cloud after each acupoint operation is completed, and the system optimizes the image recognition algorithm by analyzing historical data to improve the acupoint searching accuracy and efficiency; The doctor terminal interaction module enables the doctor to view the positioning result in real time through a control panel, send a confirmation signal or a correction instruction, and convert the correction instruction into a real-time adjustment instruction of the robot through an SDK.

2. The acupuncture robot acupoint positioning system based on visual recognition and mechanical arm cooperation according to claim 1, characterized in that, The application further relates to an image-based acupoint positioning method, which comprises the following steps: Step one: after the system is started, the depth camera in the image acquisition module starts to work and collects patient skin surface images in real time, and the images are transmitted to the doctor terminal interaction module to facilitate the doctor to view the patient condition in real time for diagnosis, and are input into the embedded system in the robot for subsequent analysis and processing; Step two: the images in the embedded system are transmitted to the image preprocessing module, the images are sequentially subjected to denoising, edge detection and image enhancement in the module, and high-quality image data is output to provide a clear and accurate image basis for acupoint recognition; Step three: the high-quality image data enters the acupoint recognition module, the lightweight YOLO algorithm is adopted to determine the accurate position of the acupoint, and multiple images are compared to further improve the recognition accuracy; Step four: after the acupoint recognition is completed, the mechanical arm control module receives the positioning information, controls the mechanical arm to move above the acupoint, disinfects the acupoint skin, and simultaneously transmits the acupoint positioning result video to the doctor terminal interaction module to enable the doctor to master the positioning condition in real time; Step five: the doctor views the positioning result through the control panel, sends a confirmation signal if the positioning is considered to be accurate, allows the robot to continue subsequent needle knife operation, or sends a correction instruction if the positioning is considered to be inaccurate, and an SDK is used to convert the instruction into a real-time adjustment instruction of the robot to ensure that the robot accurately adjusts the acupoint position according to the doctor feedback. Step six: After each operation is completed, the data storage and optimization module stores the acupoint positioning information in the cloud storage server. The system will regularly analyze these historical data to optimize the image recognition algorithm, improve the accuracy and efficiency of acupoint location, and achieve continuous improvement of the robot needle knife operation.

3. The acupoint positioning system of an acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 1, characterized in that, The image acquisition module adopts a depth camera, which can convert a two-dimensional depth image into point cloud data in a three-dimensional space by measuring the distance of each pixel point in a scene, assuming that the intrinsic matrix of the depth camera is Then the three-dimensional coordinates (X, Y, Z) of the pixel point in the camera coordinate system can be calculated by the following formula: Where f x , f y are the focal lengths of the camera in the x and y directions, c x , c y are the x and y coordinates of the image principal point, and d(x, y) is the depth value of the pixel point.

4. The acupoint positioning system of an acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 3, characterized in that, The depth camera, the depth information obtained needs to be registered with the color vision image, so as to combine the depth information with the human body features in the image, through the camera calibration process, the conversion relationship between the depth camera and the color camera can be obtained, which can be expressed by a rotation matrix R and a translation vector T as Where x d , y d , z d are the coordinate values of the point in the three-dimensional space under the depth camera coordinate system, and the last "1" is the homogeneous term, which is used for matrix operation in homogeneous transformation, x c , y c , z c are the coordinate values of the point in the three-dimensional space under the color camera coordinate system, and the last "1" is also the homogeneous term.

5. The acupoint positioning system of an acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 1, characterized in that, The image preprocessing module includes denoising, edge detection, and image enhancement preprocessing operations: The denoising preprocessing operation adopts bilateral filtering, which can smooth the noise while preserving the geometric details of the skin surface. First, key parameters are determined according to the point cloud data characteristics and noise level, including spatial standard deviation (σ s ), value range standard deviation (σ r ) and neighborhood N(p) size; then for each point p in the point cloud, the points q in its neighborhood N(p) are traversed, and the spatial weight w s (p, q) and the value range weight w r (p, q) are calculated according to the formulas and respectively, and finally the point cloud value is updated according to the bilateral filtering formula (wherein the depth value of the q point is taken as I(q)), which suppresses the point cloud noise and preserves the geometric details, providing a good data basis for subsequent point cloud feature-based acupoint positioning. The edge detection preprocessing operation uses Canny edge detection to first perform three-dimensional Gaussian filtering, with a Gaussian filter kernel function of where σ is a standard deviation, a larger σ value makes the filter kernel wider, resulting in a stronger smoothing effect, capable of removing more noise, but can cause the edge information to be blurred, then the gradient amplitude and direction are calculated, the Sobel operator is used to calculate the gradient amplitude G and the gradient direction θ, for each pixel point, according to its gradient direction, the gradient amplitude of the pixel point and the gradient amplitudes of the neighboring pixels are compared, if it is not a local maximum value, the gradient amplitude of the pixel point is set to 0, finally two threshold values T h (high threshold) and T l (low threshold) are set, pixels with a gradient amplitude greater than T h are marked as strong edges, pixels with a gradient amplitude less than T l are marked as non-edges, and pixels with a gradient amplitude between the two are marked as edges if adjacent to a strong edge, otherwise they are marked as non-edges; The image enhancement preprocessing operation enhances the image to improve the clarity and contrast of the image. The three-dimensional histogram equalization is used to count the number of pixels at different spatial positions and depth values. The histogram equalization and contrast stretching methods can be used to make the details in the image more obvious, which facilitates the acupoint recognition module to analyze and locate the acupoints more accurately.

6. The acupoint positioning system of an acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 1, characterized in that, The acupoint recognition module includes the following steps: Step one: Use the lightweight YOLO algorithm optimized for human acupoint positioning to identify the relatively fixed and easily identifiable key nodes of the human body. Step two: Based on the position information between the key nodes and acupoints, determine the accurate position of each acupoint based on the inch measurement and calculation. Step three: To improve the accuracy of acupoint recognition, the system continuously collects multiple frames of images, compares and optimizes the multiple frames of images, and fuses and analyzes the acupoint recognition results of each frame of image.

7. The acupoint positioning system of an acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 6, characterized in that, The key nodes are obtained by collecting and organizing a large amount of human anatomy research data, medical image data, and clinical practice experience to obtain the accurate position relationship information between each key node and each related acupoint in three-dimensional space. These information includes the straight-line distance between the key nodes and acupoints, the direction vector, and the position relationship change law under different human body types and postures. Then the collected three-dimensional position relationship data is stored in the database. The database is indexed by key nodes, and each key node corresponds to a list containing multiple acupoint three-dimensional position relationship information.

8. The acupoint positioning system of an acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 6, characterized in that, The measurement quantity, the position relationship information of each acupoint includes acupoint name, distance d, direction vector (θ x , θ y , θ z ) and other related correction parameters, the three-dimensional coordinates (X0, Y0, Z0) of the key nodes of the human body in the camera coordinate system are known, the distance d between a key node and an acupoint and the direction vector (θ x , θ y , θ z ) are queried from the database, the "cun" is converted into the actual length unit in the camera coordinate system (assuming that 1 cun corresponds to the actual length k), the distance D in the camera coordinate system is d x k, according to the trigonometric function principle, the three-dimensional coordinates (X, Y, Z) of the acupoint in the camera coordinate system are calculated, and the calculation formula is as follows:

9. The acupoint positioning system of an acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 1, characterized in that, The mechanical arm control module includes the following steps: Step one: the acupoint recognition module transmits these positioning information to the mechanical arm control module, which receives the coordinate information of the acupoint in the coordinate system constructed based on the camera-captured image; Step two: coordinate change, the mechanical arm control module converts the image coordinates of the acupoint into actual coordinates in the mechanical arm workspace; Step three: the mechanical arm control module plans a reasonable moving path according to the current position of the mechanical arm and the target acupoint position; Step four: the mechanical arm control module performs path planning, sends control signals to the mechanical arm, and accurately controls the movement angle and speed of each joint of the mechanical arm; Step five: when the mechanical arm approaches the target acupoint, in order to achieve high-precision positioning, the mechanical arm control module will make fine adjustments according to the feedback information.

10. The acupoint positioning system of the acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 9, characterized in that, The coordinate transformation, from the image coordinate system to the camera coordinate system, considers pixel size and image distortion correction, and from the camera coordinate system to the mechanical arm workspace coordinate system, performs coordinate transformation according to the installation position and attitude of the mechanical arm, assuming that the position of the mechanical arm base in the world coordinate system is (x0, Y0, Z0) and the attitude is represented by a rotation matrix R, combined with the extrinsic matrix of the camera, through a series of matrix operations, the acupoint coordinates in the camera coordinate system can be converted to the coordinates in the mechanical arm workspace coordinate system.

11. The acupoint positioning system of the acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 9, characterized in that, The path planning adopts the grid map method to model the working environment, uses the A* algorithm to determine an optimal path, and uses the path smoothing algorithm to remove unnecessary turns and redundant nodes in the path, generating a new smooth curve as the optimized path, while using the PID control algorithm to adjust the position and speed of the mechanical arm in real time, ensuring that the mechanical arm can accurately move to the target position according to the predetermined trajectory and speed.

12. The acupoint positioning system of the acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 1, characterized in that, The doctor interaction module is a touch screen control panel integrated with display function, through which the doctor can view the high-definition image or video stream transmitted from the patient end in real time, complete diagnosis and real-time supervision of the needle-knife robot, and easily select the needle-knife acupoint and send operation instructions.

13. The acupoint positioning system of the acupuncture robot based on visual recognition and mechanical arm cooperation according to claim 1, characterized in that, The cloud storage server in the data storage and optimization module adopts a distributed storage architecture to ensure the security and scalability of the data.

Images