Mechanical arm control system and method based on 3D human body model projection and image recognition

Abstract

The application relates to the technical field of mechanical arm control, and discloses a mechanical arm control system and method based on 3D human body model projection and image recognition, which comprises the following modules: a 3D human body model projection module, which acquires medical image data of a patient, constructs a 3D human body model of the patient, and projects the 3D human body model on the surface of the patient's body; an image recognition module, which collects a projection image on the surface of the patient's body in real time, and determines a position to be treated of the patient based on the projection image; a treatment scheme generation module, which acquires case information and the position to be treated of the patient, determines a customized treatment scheme of the patient according to the case information and the position to be treated based on a disease management database; and a mechanical arm control module, which selects a treatment tool and a treatment mode according to the customized treatment scheme, and controls the mechanical arm to be positioned at the position to be treated of the patient to treat the patient. The application can accurately determine the position to be treated of the patient, thereby improving the accuracy and effectiveness of treatment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mechanical arm control, in particular to a mechanical arm control system and method based on 3D human body model projection and image recognition. BACKGROUND

[0002] In the medical field, traditional treatment methods have certain limitations in positioning accuracy and personalized treatment. For example, for acupuncture, massage and other treatment methods, doctors mainly rely on experience and naked eye observation to determine the position of acupoints, which may cause positioning deviation and affect treatment effect. At the same time, due to different physical conditions, disease types and stages, different patients need highly personalized treatment plans, but existing treatment methods often cannot quickly and accurately develop plans that meet the specific conditions of each patient.

[0003] In addition, manual operation is affected by factors such as doctor fatigue and skill level difference, making it difficult to ensure the consistency and stability of treatment. With the development of medical technology, people have higher and higher requirements for the accuracy, personalization and stability of treatment of traditional treatment modes. Therefore, there is an urgent need for a technical solution that can improve the accuracy of treatment positioning, implement personalized treatment plan development, and ensure the stability and consistency of treatment operation. SUMMARY

[0004] The purpose of the present application is to provide a mechanical arm control system and method based on 3D human body model projection and image recognition to solve the above problems.

[0005] The present application provides a mechanical arm control system based on 3D human body model projection and image recognition, comprising:

[0006] A 3D human body model projection module configured to obtain medical image data of a patient, pre-process the medical image data, construct a 3D human body model of the patient, and project the 3D human body model on the surface of the patient's body, the 3D human body model including bones, muscles and acupoints;

[0007] An image recognition module configured to collect a projection image of the surface of the patient's body in real time, and determine a position to be treated of the patient based on the projection image;

[0008] A treatment plan generation module configured to obtain case information and the position to be treated of the patient, the treatment plan generation module being provided with a disease management database, and determining a customized treatment plan for the patient based on the disease management database and the case information and the position to be treated;

[0009] A mechanical arm control module configured to select a treatment tool and a treatment method according to the customized treatment plan, and control the mechanical arm to be positioned at the position to be treated of the patient for treatment.

[0010] Preferably, the 3D human body model projection module stores a standard 3D human body model, and the standard 3D human body model includes bones, muscles and acupoints.

[0011] According to the treatment requirement, the standard 3D human body model is projected on the surface of the patient's body.

[0012] Preferably, the 3D human body model projection module is configured to acquire medical image data of the patient, pre-process the medical image data, and construct a 3D human body model of the patient, including:

[0013] The medical image data includes CT scan data and MRI data.

[0014] The pre-processing of the medical image data includes denoising processing and segmentation processing.

[0015] A plurality of human template models are pre-set, a corresponding human template model is selected according to the basic information of the patient, and the pre-processed medical image data is filled into the human template model to obtain a 3D human body model.

[0016] Preferably, the 3D human body model projection module projects the 3D human body model on the surface of the patient's body, including:

[0017] The spatial position and attitude information of the patient are acquired by an optical tracking sensor;

[0018] The spatial position and attitude information are associated with the 3D human body model through rotation, translation and scaling operations;

[0019] When the association is completed, the 3D human body model is projected on the surface of the patient's body;

[0020] When the spatial position and attitude information are associated with the 3D human body model, specifically:

[0021] A plurality of reference points of the patient are determined according to the attitude information of the patient;

[0022] The 3D human body model is adjusted according to the reference points, and the attitude of the 3D human body model is adjusted to the attitude of the patient;

[0023] The adjusted 3D human body model is aligned with the patient according to the spatial position of the patient, and the coincidence degree of the 3D human body model and the patient is determined, and if the coincidence degree is higher than a coincidence degree threshold, the complete association is determined.

[0024] Preferably, the image recognition module is configured to acquire a projection image of the surface of the patient's body in real time, and determine a to-be-treated position of the patient based on the projection image, including:

[0025] acquiring a projection image of a patient's body surface in real time through a high-resolution camera;

[0026] extracting features of the projection image based on a deep learning algorithm to obtain key features of the projection image;

[0027] pre-establishing a treatment judgment model, inputting the key features into the treatment judgment model, outputting a position to be treated of the patient, and converting the position to be treated into a spatial coordinate.

[0028] Preferably, the method for establishing the treatment judgment model is specifically:

[0029] obtaining historical treatment data of a plurality of patients, the historical treatment data including treatment positions and types, corresponding meridians and disease images of each treatment position;

[0030] dividing the historical treatment data into a training set and a test set, training a neural network model based on the training set, testing the trained neural network model based on the test set, and setting the neural network model that passes the test as the treatment judgment model.

[0031] Preferably, the treatment scheme generation module is provided with a disease management database, and based on the disease management database, the customized treatment scheme of the patient is determined according to the case information and the position to be treated, including:

[0032] determining the disease type, symptoms and physical health level of the patient according to the case information;

[0033] The disease management database includes a plurality of disease types, each disease type is provided with corresponding symptoms and treatment schemes, and the treatment scheme includes a treatment method and a basic treatment parameter;

[0034] The disease management database is screened according to the disease type and symptoms of the patient, and the treatment scheme corresponding to the disease type and symptoms of the patient in the disease management database is determined when the disease type and symptoms of the patient are the same as the disease type and symptoms in the disease management database;

[0035] adjusting the basic treatment parameter according to the physical health level of the patient to obtain an adjusted treatment parameter;

[0036] determining the screened treatment method and the adjusted treatment parameter as the customized treatment scheme of the patient.

[0037] Preferably, the physical health level of the patient is determined according to the case information, specifically:

[0038] based on the acute physiology and chronic health evaluation system II, the health score of the patient is obtained according to the case information.

[0039] The first health score value, the second health score value and the third health score value are preset, and the first health score value, the second health score value and the third health score value increase in turn;

[0040] The health degree of the patient is determined according to the relationship between the health score value and the first health score value, the second health score value and the third health score value;

[0041] If the health score value is less than the first health score value, the health degree of the patient is set as the first health degree L1;

[0042] If the health score value is greater than or equal to the first health score value and less than the second health score value, the health degree of the patient is set as the second health degree L2;

[0043] If the health score value is greater than or equal to the second health score value and less than the third health score value, the health degree of the patient is set as the third health degree L3;

[0044] If the health score value is greater than or equal to the third health score value, the health degree of the patient is set as the fourth health degree L4; wherein L1>L2>L3>L4.

[0045] Preferably, the basic treatment parameter is adjusted according to the health degree of the patient to obtain an adjusted treatment parameter, including:

[0046] A corresponding adjustment coefficient is determined according to the health degree of the patient, and the basic treatment parameter is adjusted based on the adjustment coefficient to obtain an adjusted treatment parameter;

[0047] The adjustment coefficient is inversely proportional to the health degree of the patient.

[0048] Preferably, the mechanical arm control module is configured to select a treatment tool and a treatment method according to the customized treatment scheme, and control the mechanical arm to be positioned at a to-be-treated position of the patient to treat the patient, including:

[0049] If the treatment method in the customized treatment scheme of the patient is acupuncture treatment, the treatment tool is selected as an acupuncture needle;

[0050] If the treatment method in the customized treatment scheme of the patient is massage treatment, the treatment tool is selected as a massage head;

[0051] After the treatment tool is selected, a motion path of the mechanical arm is set according to the adjusted treatment parameter and the spatial coordinates of the to-be-treated position to treat the patient.

[0052] The application also discloses a mechanical arm control method based on 3D human body model projection and image recognition, which is applied to the mechanical arm control system based on 3D human body model projection and image recognition and comprises the following steps of:

[0053] Medical image data of a patient is acquired, the medical image data is preprocessed, a 3D human body model of the patient is constructed, and the 3D human body model is projected on the body surface of the patient, wherein the 3D human body model comprises bones, muscles and acupoints;

[0054] A projected image of the body surface of the patient is collected in real time, and a position to be treated of the patient is determined based on the projected image;

[0055] Case information and the position to be treated of the patient are acquired;

[0056] A disease management database is preset, and a treatment scheme of the patient is determined based on the disease management database, the case information and the position to be treated;

[0057] A treatment tool and a treatment mode are selected according to the treatment scheme, and a mechanical arm is controlled to be positioned at the position to be treated of the patient to treat the patient.

[0058] Compared with the prior art, the beneficial effects of the application are that the position to be treated of the patient can be accurately determined through the 3D human body model projection and image recognition technology, thereby improving the accuracy and effectiveness of treatment. By acquiring the case information and the position to be treated of the patient and combining the disease management database, a personalized treatment scheme can be made for each patient, thereby improving the treatment effect. Through the mechanical arm control module, automatic treatment of the patient can be realized, the operation burden of doctors is reduced, and the treatment efficiency is improved. Through the use of the 3D human body model projection and image recognition technology, the dependence on expensive medical equipment can be reduced, thereby reducing the medical cost. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only the embodiments of the application, and other drawings can also be obtained by those skilled in the art without any creative effort on the basis of the provided drawings.

[0060] Figure 1 is a functional block diagram of the mechanical arm control system based on 3D human body model projection and image recognition of the application.

[0061] Figure 2 is a flowchart of the mechanical arm control method based on 3D human body model projection and image recognition of the application. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0063] Embodiment one

[0064] As shown in the drawings, the present application provides a mechanical arm control system based on 3D human model projection and image recognition, comprising: Figure 1 A 3D human model projection module is configured to acquire medical image data of a patient, pre-process the medical image data, construct a 3D human model of the patient, and project the 3D human model on the surface of the patient's body, wherein the 3D human model includes bones, muscles, and acupoints.

[0065] An image recognition module is configured to collect a projection image of the surface of the patient's body in real time, and determine a position to be treated of the patient based on the projection image.

[0066] A treatment scheme generation module is configured to acquire case information and the position to be treated of the patient, and is provided with a disease management database therein. Based on the disease management database, the treatment scheme generation module determines a customized treatment scheme of the patient according to the case information and the position to be treated.

[0067] A mechanical arm control module is configured to select a treatment tool and a treatment method according to the customized treatment scheme, and control the mechanical arm to be positioned at the position to be treated of the patient to treat the patient.

[0068] The present application can significantly improve the accuracy and efficiency of treatment. Through 3D human model projection, doctors can intuitively see the internal structure of the patient, such as bones, muscles, and acupoints, so as to more accurately determine the treatment position. The real-time collection function of the image recognition module ensures dynamic tracking of the treatment position, so that the position of the mechanical arm can be adjusted in time even if the patient's body position changes during treatment. The treatment scheme generation module utilizes the disease management database to formulate a personalized treatment scheme in combination with the specific situation of the patient, so that the treatment is more targeted. The mechanical arm control module realizes the automation of the treatment process, reduces the operation burden of doctors, and improves the stability and safety of treatment.

[0069]

[0070] ​In some embodiments of the present application, the 3D human body model projection module is configured to obtain medical image data of a patient, pre-process the medical image data, and construct a 3D human body model of the patient. Specifically, the medical image data includes CT scan data and MRI data. Before constructing the 3D human body model, the module first performs a pre-processing step on the medical image data, which includes denoising processing and segmentation processing. Denoising processing aims to remove random noise in the image to improve the accuracy of subsequent processing and the quality of the model. Segmentation processing is to distinguish different tissues and organs in the image to facilitate subsequent 3D modeling. In order to further improve the efficiency and accuracy of modeling, a number of human body template models are preset. These template models are created based on the average anatomical structure of a large number of people and can represent people of different body types and ages. When constructing the 3D human body model of a specific patient, the system will select a human body template model closest to the patient's body type characteristics as a starting point according to the patient's basic information such as gender, age, height, weight, etc. After selecting the appropriate human body template model, the pre-processed medical image data will be filled into the template model. This filling process may involve adjusting and adapting the template model to ensure accurate alignment of the medical image data with the template model. In this way, a highly personalized and accurate 3D human body model can be generated, which not only reflects the specific anatomical structure of the patient, but also can be used for further medical analysis, surgical planning or educational demonstration, etc.

[0071] The 3D human body model projection module is configured to obtain medical image data of a patient, pre-process the medical image data, and construct a 3D human body model of the patient. Specifically, the medical image data includes CT scan data and MRI data. Before constructing the 3D human body model, the module first performs a pre-processing step on the medical image data, which includes denoising processing and segmentation processing. Denoising processing aims to remove random noise in the image to improve the accuracy of subsequent processing and the quality of the model. Segmentation processing is to distinguish different tissues and organs in the image to facilitate subsequent 3D modeling. In order to further improve the efficiency and accuracy of modeling, a number of human body template models are preset. These template models are created based on the average anatomical structure of a large number of people and can represent people of different body types and ages. When constructing the 3D human body model of a specific patient, the system will select a human body template model closest to the patient's body type characteristics as a starting point according to the patient's basic information such as gender, age, height, weight, etc. After selecting the appropriate human body template model, the pre-processed medical image data will be filled into the template model. This filling process may involve adjusting and adapting the template model to ensure accurate alignment of the medical image data with the template model. In this way, a highly personalized and accurate 3D human body model can be generated, which not only reflects the specific anatomical structure of the patient, but also can be used for further medical analysis, surgical planning or educational demonstration, etc.

[0072] It can be understood that the present application can accurately construct a 3D human body model of a patient, providing doctors with more intuitive and detailed information about the patient's internal structure. By obtaining CT scan data and MRI data of the patient, the system can capture detailed information about the patient's body, such as bone structure, muscle distribution, and internal organ location. Denoising processing and segmentation processing further improve the accuracy and reliability of the data, ensuring that the 3D human body model constructed is highly consistent with the actual situation of the patient. Selecting the corresponding human body template model and filling it with pre-processed medical image data not only simplifies the construction process, but also improves the accuracy and applicability of the model.

[0073] In some embodiments of the present application, the 3D human body model projection module projects the 3D human body model on the patient's body surface, comprising: acquiring the spatial position and posture information of the patient through an optical tracking sensor; associating the spatial position and posture information with the 3D human body model through rotation, translation and scaling operations; and projecting the 3D human body model on the patient's body surface after completing the association.

[0074] In the process of associating the spatial position and posture information with the 3D human body model, the specific process is as follows: determining a plurality of reference points of the patient according to the posture information of the patient; adjusting the 3D human body model according to the reference points, and adjusting the posture of the 3D human body model to the posture of the patient; aligning the adjusted 3D human body model with the patient according to the spatial position of the patient, and determining the coincidence degree of the 3D human body model and the patient, and if the coincidence degree is higher than the coincidence degree threshold, it is determined that the association is complete.

[0075] It can be understood that the present application can accurately project the 3D human body model on the patient's body surface, and realize the real-time alignment of the patient's body structure and the virtual model. The spatial position and posture information of the patient is acquired through the optical tracking sensor, which ensures the accuracy and real-time of the projection. The rotation, translation and scaling operations make the 3D human body model adapt to the actual posture of the patient flexibly, which improves the applicability and accuracy of the projection. In the process of determining the complete association of the 3D human body model and the patient, the adjustment of the reference points and the judgment of the coincidence degree further ensure the accuracy and reliability of the projection.

[0076] In some embodiments of the present application, the image recognition module is configured to collect the projection image of the patient's body surface in real time, and determine the position to be treated of the patient based on the projection image, comprising: collecting the projection image of the patient's body surface in real time through a high-resolution camera; extracting features of the projection image based on a deep learning algorithm to obtain key features of the projection image; pre-establishing a treatment judgment model, inputting the key features into the treatment judgment model, outputting the position to be treated of the patient, and converting the position to be treated into a spatial coordinate.

[0077] The image recognition module is configured to collect a projection image of the patient's body surface in real time, determine the position to be treated of the patient based on the projection image, and the specific operation steps include: first, a high-resolution camera is used to collect a projection image of the patient's body surface in real time, ensuring that the image is clear and contains sufficient detailed information. Then, a deep learning algorithm is used to analyze the collected projection image in depth, extracting key features in the image that can represent specific areas or markers on the patient's body surface. Next, a treatment judgment model is established in advance, which is trained by a large amount of data and can identify and analyze the relationship between image features and treatment positions. The extracted key features are input into the treatment judgment model, which will output the position to be treated of the patient according to its internal algorithm and training results. Finally, the identified position to be treated is converted into precise spatial coordinates to facilitate the subsequent treatment equipment to accurately locate the position and perform precise treatment operations. The whole process is automated and can quickly respond to the patient's treatment needs, improving the efficiency and accuracy of treatment.

[0078] It can be understood that the high-resolution camera is used to collect the projection image of the patient's body surface in real time, ensuring the high definition and detail capturing ability of the image. Based on the deep learning algorithm, the key features related to the position to be treated can be automatically identified and extracted, improving the accuracy and efficiency of identification. The pre-established treatment judgment model can quickly and accurately determine the position to be treated of the patient according to the key features and convert it into spatial coordinates, providing a reliable basis for the precise operation of the mechanical arm.

[0079] In some embodiments of the present application, the establishment method of the treatment judgment model is specifically: obtaining historical treatment data of a plurality of patients, the historical treatment data including treatment positions and types, corresponding meridians and disease images of each treatment position; dividing the historical treatment data into a training set and a test set, training a neural network model based on the training set, and testing the trained neural network model based on the test set, and setting the neural network model that passes the test as the treatment judgment model.

[0080] It can be understood that by obtaining the historical treatment data of the patient, individual differences and disease characteristics of different patients can be fully considered, and rich sample data can be provided for the establishment of the treatment judgment model. Dividing the historical treatment data into a training set and a test set can ensure the training effect and generalization ability of the model. Training the neural network model based on the training set can enable the model to learn the complex relationship between the treatment position and the corresponding type, the meridian to which it belongs, and the diseased image. Testing the trained neural network model based on the test set can verify the accuracy and reliability of the model and ensure its performance in actual application. Setting the neural network model that passes the test as the treatment judgment model provides a scientific basis for subsequent image recognition and treatment operation, further improving the accuracy and safety of surgical treatment.

[0081] In some embodiments of the present application, the treatment scheme generation module is provided with a disease management database. Based on the disease management database, a customized treatment scheme for the patient is determined according to the case information and the position to be treated, including: determining the disease type, symptoms and physical health of the patient according to the case information; the disease management database includes a plurality of disease types, each disease type is provided with corresponding symptoms and treatment schemes, and the treatment scheme includes a treatment method and a basic treatment parameter; the disease management database is screened according to the disease type and symptoms of the patient, and the treatment scheme corresponding to the disease type and symptoms of the patient in the disease management database is determined when the disease type and symptoms of the patient are the same as the disease type and symptoms in the disease management database; the basic treatment parameter is adjusted according to the physical health of the patient to obtain an adjusted treatment parameter; and the treatment method and the adjusted treatment parameter screened are determined as the customized treatment scheme for the patient.

[0082] It can be understood that by setting the disease management database, rich disease information and treatment scheme selection are provided for the generation of the customized treatment scheme. The disease type, symptoms and physical health of the patient are determined according to the case information, which can comprehensively consider the individual differences and disease characteristics of the patient and provide accurate basis for the development of the treatment scheme. The disease types contained in the disease management database are comprehensive, and the symptoms and treatment schemes corresponding to each disease type are detailed, ensuring the accuracy and scientificity of the selection of the treatment scheme. Through the screening of the disease management database, the treatment scheme matching the patient's condition can be quickly determined, improving the pertinence and efficiency of the treatment scheme. At the same time, the basic treatment parameter is adjusted according to the physical health of the patient, which can obtain a treatment parameter more suitable for the actual situation of the patient, further improving the accuracy and safety of the treatment. The treatment method and the adjusted treatment parameter screened finally constitute the customized treatment scheme for the patient, providing personalized treatment service for the patient and helping to improve the treatment effect and patient satisfaction.

[0083] In some embodiments of the present application, the health level of the patient is determined according to the case information, specifically: based on the acute physiology and chronic health evaluation system II, a health score of the patient is obtained according to the case information; a first health score value, a second health score value and a third health score value are pre-set, and the first health score value, the second health score value and the third health score value increase in turn; the health level of the patient is determined according to the relationship between the health score value and the first health score value, the second health score value and the third health score value; if the health score value is less than the first health score value, the health level of the patient is set as a first health level L1; if the health score value is greater than or equal to the first health score value and less than the second health score value, the health level of the patient is set as a second health level L2; if the health score value is greater than or equal to the second health score value and less than the third health score value, the health level of the patient is set as a third health level L3; if the health score value is greater than or equal to the third health score value, the health level of the patient is set as a fourth health level L4; wherein L1>L2>L3>L4.

[0084] It can be understood that by introducing the acute physiology and chronic health evaluation system II to score the health of the patient, the health level of the patient can be more objectively and scientifically evaluated. The scoring system comprehensively considers multiple aspects such as physiological indicators and chronic disease conditions of the patient, and can fully reflect the health status of the patient. According to the scoring result, the health level of the patient is divided into different grades, which helps doctors more accurately consider the physical tolerance of the patient in the development of treatment plans, so as to select a more appropriate treatment method and adjust the treatment parameters.

[0085] In some embodiments of the present application, the basic treatment parameters are adjusted according to the health level of the patient to obtain adjusted treatment parameters, including: determining a corresponding adjustment coefficient according to the health level of the patient, and adjusting the basic treatment parameters based on the adjustment coefficient to obtain the adjusted treatment parameters; wherein the adjustment coefficient is inversely proportional to the health level of the patient.

[0086] When the treatment method is acupuncture treatment, the treatment parameters are acupuncture depth, angle and force; acupuncture treatment also includes acupuncture, moxibustion, etc.; when the treatment method is massage treatment, the treatment parameters are massage force, frequency, direction and duration.

[0087] It can be understood that the individualized customization of the treatment scheme is realized by dynamically adjusting the treatment parameters according to the physical health of the patient. Specifically, for a patient with poor physical health, the corresponding adjustment coefficient is small, and the adjustment range of the basic treatment parameter is also small, which helps to avoid unnecessary burden on the patient's body due to excessive treatment intensity. On the contrary, for a patient with good physical health, the corresponding adjustment coefficient is large, and the adjustment range of the basic treatment parameter is also large, which can improve the treatment effect and accelerate the rehabilitation process of the patient on the premise of safety. This flexible adjustment of treatment parameters according to the actual situation of the patient not only enhances the flexibility and adaptability of the treatment, but also further improves the accuracy and satisfaction of the treatment.

[0088] In some embodiments of the present application, the mechanical arm control module is configured to select a treatment tool and a treatment method according to the customized treatment scheme, and control the mechanical arm to position at a to-be-treated position of the patient to treat the patient, comprising: if the treatment method in the customized treatment scheme of the patient is acupuncture treatment, selecting the treatment tool as an acupuncture needle; if the treatment method in the customized treatment scheme of the patient is massage treatment, selecting the treatment tool as a massage head; after selecting the treatment tool, setting the motion path of the mechanical arm according to the adjusted treatment parameters and the spatial coordinates of the to-be-treated position to treat the patient.

[0089] It can be understood that the intelligent selection of treatment tools and treatment methods by the mechanical arm control module according to the customized treatment scheme and the accurate positioning of the patient for treatment greatly improve the automation and intelligent level of the treatment. During acupuncture treatment, the accurate insertion of the acupuncture needle can accurately stimulate the acupoint to achieve the treatment effect of regulating blood and dredging meridians; during massage treatment, the accurate massage of the massage head can effectively relieve muscle tension and promote blood circulation. At the same time, setting the motion path of the mechanical arm according to the adjusted treatment parameters and the spatial coordinates of the to-be-treated position ensures the accuracy and safety of the treatment, and avoids injury to the patient due to improper operation.

[0090] Embodiment two

[0091] The present application provides a mechanical arm control system based on 3D human body model projection and image recognition, comprising:

[0092] A 3D human body model projection module, the 3D human body model projection module stores a standard 3D human body model, the standard 3D human body model includes bones, muscles and acupoints; configured to project the standard 3D human body model on the surface of the patient's body according to the treatment demand;

[0093] An image recognition module, configured to collect the projection image of the surface of the patient's body in real time, and determine the to-be-treated position of the patient based on the projection image;

[0094] A treatment scheme generation module is configured to acquire case information and a treatment position of a patient, and a disease management database is preset in the treatment scheme generation module, and a customized treatment scheme of the patient is determined according to the case information and the treatment position based on the disease management database.

[0095] A mechanical arm control module is configured to select a treatment tool and a treatment mode according to the customized treatment scheme, and control the mechanical arm to position at the treatment position of the patient to treat the patient.

[0096] It can be understood that the functions of the image recognition module, the treatment scheme generation module and the mechanical arm control module in the embodiment are the same as those in the first embodiment.

[0097] In the application, the standard 3D human model is projected on the surface of the patient's body and is adapted to the patient, so as to facilitate the acquisition of the projection image and the determination of the corresponding muscle and acupoint.

[0098] The application provides two different embodiments, which illustrate that the scheme can control the mechanical arm for treatment by direct projection, and can also control the mechanical arm for treatment according to the medical image data of the patient. Even when the medical image data of the patient cannot be obtained, the treatment can still be performed.

[0099] As shown in Figure 2 The application further discloses a mechanical arm control method based on 3D human model projection and image recognition, which is applied to the mechanical arm control system based on 3D human model projection and image recognition and comprises the following steps of:

[0100] Acquiring medical image data of a patient, pre-processing the medical image data, constructing a 3D human model of the patient, and projecting the 3D human model on the surface of the patient's body, wherein the 3D human model comprises bones, muscles and acupoints;

[0101] Real-time acquisition of a projection image on the surface of the patient's body, and determination of a treatment position of the patient based on the projection image;

[0102] Acquiring case information and a treatment position of a patient;

[0103] A disease management database is preset, and a treatment scheme of the patient is determined according to the case information and the treatment position based on the disease management database;

[0104] A treatment tool and a treatment mode are selected according to the treatment scheme, and the mechanical arm is controlled to position at the treatment position of the patient to treat the patient.

[0105] The application realizes intuitive display of the patient's body structure through 3D human model projection technology, making the treatment process more accurate and efficient. At the same time, combined with image recognition technology, it can track the patient's body changes in real time, ensuring the accuracy of the treatment position. In addition, through the preset disease management database, the patient's treatment plan can be quickly determined, improving the treatment efficiency and accuracy. This method is not only suitable for surgical treatment, but also can be widely used in rehabilitation treatment, physical therapy and other fields, providing strong support for the development of the medical industry.

[0106] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application and not to limit them, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that: it can still modify or equivalently replace the technical solutions of the present application, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

[0107] The system provided by the above examples is only illustrated by the division of the above functional modules. In actual application, the above functions can be completed by different functional modules according to needs, that is, the modules or steps in the embodiments of the present application are further decomposed or combined, for example, the modules of the above examples can be combined into one module, or further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present application are only for distinguishing each module or step, and should not be considered as an improper limitation of the present application.

[0108] Those skilled in the art should be aware that the modules, method steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or combination of the two. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been described in the above description. Whether the functions are realized by electronic hardware or software depends on the specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

Claims

1. A robotic arm control system based on 3D human body model projection and image recognition, characterized in that, include: A 3D human body model projection module is configured to acquire medical image data of a patient, preprocess the medical image data, construct a 3D human body model of the patient, and project the 3D human body model onto the patient's body surface. This includes: acquiring the patient's spatial position and posture information through an optical tracking sensor; associating the spatial position and posture information with the 3D human body model through rotation, translation, and scaling operations; and projecting the 3D human body model onto the patient's body surface after association. Specifically, associating the spatial position and posture information with the 3D human body model involves: determining several reference points for the patient based on their posture information; adjusting the 3D human body model according to the reference points to match the patient's posture; aligning the adjusted 3D human body model with the patient based on their spatial position, and determining the overlap between the 3D human body model and the patient. If the overlap is higher than an overlap threshold, a complete association is determined. The 3D human body model includes bones, muscles, and acupoints. An image recognition module is configured to acquire projected images of a patient's body surface in real time and determine the patient's treatment location based on the projected images. This includes: acquiring projected images of the patient's body surface in real time using a high-resolution camera; extracting features from the projected images using a deep learning algorithm to obtain key features; pre-establishing a treatment judgment model, inputting the key features into the treatment judgment model, outputting the patient's treatment location, and converting the treatment location into spatial coordinates. Specifically, the method for establishing the treatment judgment model involves: acquiring historical treatment data from several patients, including treatment locations and the type, meridian, and disease image corresponding to each treatment location; dividing the historical treatment data into a training set and a test set; training a neural network model based on the training set; testing the trained neural network model based on the test set; and setting the neural network model that passes the test as the treatment judgment model. A treatment plan generation module is configured to acquire patient case information and the location to be treated. The module includes a disease management database. Based on this database, a customized treatment plan is determined for the patient according to the case information and the location to be treated. This includes: determining the patient's disease type, symptoms, and overall health status based on the case information; the disease management database includes several disease types, each with corresponding symptoms and treatment plans, and each treatment plan includes treatment methods and basic treatment parameters; filtering the disease management database based on the patient's disease type and symptoms to determine the corresponding treatment plan when the patient's disease type and symptoms match those in the database; adjusting the basic treatment parameters based on the patient's overall health status to obtain adjusted treatment parameters; and finally, determining the selected treatment method and adjusted treatment parameters as the patient's customized treatment plan. Specifically, determining the patient's physical health status based on the case information involves: using the Acute Physiology and Chronic Health Status Scoring System II, scoring the patient's health based on the case information to obtain a health score value; pre-setting a first health score value, a second health score value, and a third health score value, with the first health score value, the second health score value, and the third health score value increasing sequentially; and determining the patient's physical health status based on the relationship between the health score value and the first health score value, the second health score value, and the third health score value. Adjusting the basic treatment parameters according to the patient's health status to obtain adjusted treatment parameters includes: determining a corresponding adjustment coefficient based on the patient's health status, and adjusting the basic treatment parameters based on the adjustment coefficient to obtain adjusted treatment parameters; wherein, the adjustment coefficient is inversely proportional to the patient's health status; The robotic arm control module is configured to select treatment tools and treatment methods according to the customized treatment plan, and control the robotic arm to position itself at the patient's treatment location to treat the patient. This includes: if the treatment method in the patient's customized treatment plan is acupuncture, then the treatment tool is selected as an acupuncture needle; if the treatment method in the patient's customized treatment plan is massage, then the treatment tool is selected as a massage head; after selecting a treatment tool, the movement path of the robotic arm is set according to the adjusted treatment parameters and the spatial coordinates of the treatment location to treat the patient.

2. The robotic arm control system based on 3D human body model projection and image recognition according to claim 1, characterized in that, The 3D human body model projection module stores a standard 3D human body model, which includes bones, muscles and acupoints. According to treatment needs, the standard 3D human body model is projected onto the patient's body surface.

3. The robotic arm control system based on 3D human body model projection and image recognition according to claim 2, characterized in that, The 3D human body model projection module is configured to acquire the patient's medical image data, preprocess the medical image data, and construct a 3D human body model of the patient, including: The medical imaging data includes CT scan data and MRI data; The medical image data is preprocessed, including denoising and segmentation. Several human body template models are preset. The corresponding human body template model is selected according to the patient's basic information, and the preprocessed medical image data is filled into the human body template model to obtain a 3D human body model.

4. The robotic arm control system based on 3D human body model projection and image recognition according to claim 1, characterized in that, The patient's physical health status was determined based on the aforementioned case information, specifically as follows: If the health score is less than the first health score, then the patient's physical health level is set as the first health level L1; If the health score is greater than or equal to the first health score and less than the second health score, then the patient's physical health level is set as the second health level L2. If the health score is greater than or equal to the second health score and less than the third health score, then the patient's physical health level is set to the third health level L3. If the health score is greater than or equal to the third health score, then the patient's physical health level is set as the fourth health level L4; where L1 > L2 > L3 > L4.

Images