Skin medical cosmetology system and main control device and storage medium thereof

Abstract

The invention relates to a skin medical beauty treatment system and a main control device and a storage medium thereof, and the main control device comprises an image processing module which is used for receiving two-dimensional image data of a target skin area of a target object obtained through a plurality of different collection modes, carrying out integration processing on the plurality of two-dimensional image data to obtain a two-dimensional image value set; the problem prediction module is used for obtaining problem prediction data and a to-be-treated lesion area according to the two-dimensional image value set and a skin problem prediction model; the scheme generation module is used for obtaining a treatment planning path according to the image data of the to-be-treated lesion area and the path planning model, and obtaining recovery effect prediction data according to the image data of the to-be-treated lesion area, the problem prediction data and the skin recovery prediction model; and generating a treatment scheme according to the treatment planning path and the recovery effect prediction data. By adopting the equipment, the efficiency and the effect of skin medical beauty personalized treatment can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of lasers, and particularly to a skin medical beauty system, its main control device, and a storage medium. Background Art

[0002] With the development of laser technology, skin medical beauty systems for skin diagnosis and treatment have emerged, such as laser systems and ultrasonic systems for skin diagnosis and treatment.

[0003] However, traditional skin medical beauty systems have different degrees of adaptation to image data collected by different types of skin detection instruments or different treatment devices, and cannot provide accurate lesion information and cannot assist doctors in selecting treatment plans. Moreover, traditional skin medical beauty systems rely on doctors to plan treatment plans. Since there are often deviations in the data reading of image information from different skin detectors, and there are differences in the experience and skill levels of doctors in clinical practice, the subjective evaluations of damages caused by operating parameters selected for different treatment devices and different situations are not unified. Therefore, the efficiency of personalized skin treatment is low and the effect is poor. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a skin medical beauty system, its main control device, and a storage medium that can improve the efficiency and effect of personalized skin medical beauty treatment.

[0005] In a first aspect, a main control device of a skin medical beauty system is provided, and the device includes:

[0006] An image processing module, configured to receive two-dimensional image data of a target skin area of a target object obtained through a variety of different acquisition methods, and perform integration processing on the multiple two-dimensional image data to obtain a set of two-dimensional image values;

[0007] A problem prediction module, configured to obtain problem prediction data and an area of a lesion to be treated according to the set of two-dimensional image values and a skin problem prediction model;

[0008] A solution generation module, configured to obtain a treatment planning path according to the image data of the area of the lesion to be treated and a path planning model, obtain recovery effect prediction data according to the image data of the area of the lesion to be treated, the problem prediction data, and a skin recovery prediction model, and generate a treatment plan according to the treatment planning path and the recovery effect prediction data.

[0009] In some embodiments, the image processing module is further configured to:

[0010] Screen multiple pixel points from each two-dimensional image as numerical sampling points;

[0011] Integrate the values and precision values of the numerical sampling points at the corresponding sampling positions of each two-dimensional image to obtain the integrated numerical values and integrated precision values of the numerical sampling points at each sampling position;

[0012] Construct a two-dimensional image numerical set according to each integrated numerical value and integrated precision value.

[0013] In some embodiments, the master control device further includes a human-computer interaction module, and the human-computer interaction module is used for:

[0014] Construct a three-dimensional image according to the two-dimensional image numerical set;

[0015] Display the to-be-treated lesion area and / or problem prediction data according to the problem prediction data and the to-be-treated lesion area in the three-dimensional image;

[0016] In response to the user's change operation on the to-be-treated lesion area and / or problem prediction data, feedback the changed data to the skin problem prediction model for re-prediction.

[0017] In some embodiments, the master control device further includes a coordinate calibration module, and the coordinate calibration module is used for:

[0018] Construct a first space coordinate system according to the two-dimensional image numerical set;

[0019] In response to a scheme execution request, construct a second space coordinate system according to the two-dimensional image data of the target skin area of the target object re-collected during the movement of the robotic arm;

[0020] Calibrate the first space coordinate system according to the second space coordinate system.

[0021] In some embodiments, the skin problem prediction model is configured to be trained in the following manner:

[0022] Collect normal skin image samples and lesion skin image samples;

[0023] Mark the lesion areas in the lesion skin image samples, and configure marking descriptions according to the problem areas, problem types, and / or problem severity grades;

[0024] Train a convolutional neural network according to the normal skin image samples and the lesion skin image samples, and construct a skin problem prediction model according to the trained convolutional neural network.

[0025] In some embodiments, the path planning model is configured to be trained in the following manner:

[0026] Use foreign objects with different granularities, different hardnesses, and / or different color depths as features to simulate lesion skin of different problem types, and collect lesion skin image data of each simulated lesion skin;

[0027] During the process of simulating laser treatment on each simulated diseased skin, the motion trajectory data of the path execution device is collected, and the path planning model is trained according to each diseased skin image data and the motion trajectory data corresponding to each diseased image data.

[0028] In some embodiments, the effect prediction model is configured to be trained in the following manner:

[0029] Obtain the damage level data and the remaining sequela type data caused after treating diseased skins of different problem types under different therapeutic instrument types and different operating parameters;

[0030] Periodically obtain the recovery effect data of the diseased skin at different recovery time stages;

[0031] Train the recovery effect prediction model according to different diseased skin image data and their corresponding therapeutic instrument types, operating parameters, damage level data, remaining sequela type data, and recovery effect data.

[0032] In some embodiments, the solution generation module is further configured to:

[0033] Determine the types and operating parameters of the therapeutic instruments corresponding to the recovery effects at different recovery time stages according to the recovery effect prediction data, and generate a treatment plan for the current time stage according to the types and operating parameters of the therapeutic instruments at the current time stage.

[0034] In a second aspect, a skin medical beauty system is provided, and the system includes:

[0035] A skin detection unit, which includes skin detectors with multiple different image acquisition methods, and is used to collect two-dimensional image data of the target skin area of the target object;

[0036] A main control device according to any one of the first aspect;

[0037] A path execution device, which is used to move according to the treatment planning path generated by the main control device under the control of the main control device;

[0038] A treatment unit, which is used to connect the target therapeutic instrument according to the treatment plan generated by the main control device under the control of the main control device, and determine the operating parameters of the target therapeutic instrument.

[0039] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it realizes receiving two-dimensional image data of a target skin area of a target object obtained through multiple different acquisition methods, and performing integration processing on the multiple two-dimensional image data to obtain a set of two-dimensional image numerical values; obtaining problem prediction data and an area of a lesion to be treated according to the set of two-dimensional image numerical values and a skin problem prediction model; obtaining a treatment planning path according to the image data of the area of the lesion to be treated and a path planning model, obtaining recovery effect prediction data according to the image data of the area of the lesion to be treated, the problem prediction data, and a skin recovery prediction model, and generating a treatment plan according to the recovery effect prediction data.

[0040] The main control device, the skin medical beauty system, and the storage medium of the above skin medical beauty system can integrate two-dimensional image data of a target skin area of a target object obtained through multiple different acquisition methods, and determine problem prediction data and an area of a lesion to be treated based on the integrated set of two-dimensional image numerical values for treatment path planning. Moreover, when generating a treatment plan, it combines the recovery effect prediction data predicted with the problem prediction data and the area of the lesion to be treated as input variables, can quickly and automatically generate a personalized treatment plan. Further, it can assist doctors in judging skin problems, selecting treatment instruments and corresponding operating parameters, etc. Therefore, it can improve the efficiency and effect of personalized treatment in skin medical beauty. Description of the Drawings

[0041] Figure 1 It is a schematic structural diagram of a skin medical beauty system in some embodiments;

[0042] Figure 2 It is a schematic structural diagram of the main control device in some embodiments;

[0043] Figure 3 It is a schematic flow diagram for training a skin problem prediction model in some embodiments;

[0044] Figure 4 It is a schematic flow diagram for training a path planning model in some embodiments;

[0045] Figure 5 It is a schematic flow diagram for training an effect prediction model in some embodiments;

[0046] Figure 6 It is a schematic structural diagram of the main control device in some other embodiments;

[0047] Figure 7 It is a schematic structural diagram of the main control device in some application examples. Detailed Embodiments

[0048] To make the objectives, technical solutions, and advantages of this application more clear and understandable, the following further details this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0049] In some embodiments, a skin medical beauty system is provided. This skin medical beauty system can be applied in the field of medical beauty. For example, it can be applied in the fields of laser or ultrasound treatment of the skin. Reference can be made to Figure 1 as shown Figure 1 which shows a schematic structural diagram of the skin medical beauty system in some embodiments. Specifically, this skin medical beauty system can include a skin detection unit 110, a main control device 120, a treatment execution device 130, and a laser light output device 140. Among them,

[0050] The skin detection unit 110 can include image acquisition devices with multiple different acquisition methods, and is used to acquire two-dimensional image data of the target skin area of the target object.

[0051] Among them, the skin detection unit 110 can realize quantitative analysis of the physiological and pathological characteristics of the skin, and can include a variety of skin detection instruments such as a dermoscope, a skin ultrasound device, and a skin CT (Computed Tomography) scanner, etc., which are used to acquire two-dimensional image data of the target skin area of the target object through different channels.

[0052] Exemplarily, a dermoscope can be composed of a variety of light sources (such as standard light, diffuse reflection, cross-polarized light, parallel-polarized light, ultraviolet light (UV), narrow-band blue, etc. multiple light tubes), a CMOS camera, and a 3D camera with a binocular grating structure. It can not only detect problems existing on the skin surface, but also intuitively display the hidden problems in the skin basal layer through quantitative analysis, and can truly reflect information such as skin wrinkles, textures, pores, invisible spots, blood vessels, pigmentation, erythema, skin color, surface spots, etc.

[0053] Exemplarily, a skin ultrasound device is a non-invasive diagnostic instrument that uses high-frequency ultrasonic waves to examine the skin and its appendages. It can clearly display each layer of the skin and fine structures such as skin appendages, and can display information on deep structures such as the skin epidermis, dermis layer, and subcutaneous tissue, especially the longitudinal depth information of lesions, such as the depth of lesion infiltration and the relationship with surrounding tissues.

[0054] Exemplarily, a skin CT scanner is an optical focusing principle and computer three-dimensional tomography imaging instrument, which can intuitively, real-time, and dynamically observe data such as the occurrence, development, curative effect, and skin lesion conditions of skin diseases.

[0055] Among them, the path execution device 130 may include a robotic arm.

[0056] Exemplarily, the end of the robotic arm may be designed to have a universal interface capable of fixing various accessories, and can be adapted to different common laser output devices.

[0057] Exemplarily, further, the end of the robotic arm may be equipped with a force sensor, a Micro-Electro-Mechanical System Gyroscope (MEMS Gyroscope), and a 3D (Three Dimensional) imaging component. The 3D imaging component can perform 3D real-time imaging on the target skin area of the target object; the micro mechanical gyroscope can monitor the angle between the laser emitted by the accessory and the treatment surface in real time; the force sensor can be used to detect the interaction force when the end of the robotic arm contacts the external environment, and detect any external force intervention during the treatment process.

[0058] The treatment unit 140 may include various types of treatment instruments. The laser emitted by the laser type treatment instrument can be absorbed by different specific target tissues in the skin according to its different wavelengths. Therefore, different wavelengths can be used for different skin diseases. The ultrasonic type treatment instrument utilizes the mechanical and thermal effects of ultrasonic waves in the skin tissue to promote skin tightening and lifting; the radio frequency type treatment instrument acts on the skin and subcutaneous tissue of the target object with radio frequency energy, causing pathological / physiological changes in the tissues and cells of the target object; the microcurrent stimulation type treatment instrument promotes blood circulation and cell metabolism by stimulating the skin of the target object with a weak current; the cryotherapy type treatment instrument reduces inflammation and pain by acting on the skin at a low temperature. Various types of treatment instruments can be connected through the end accessory interface of the robotic arm and act on the treatment surface.

[0059] Among them, the main control device 120 may be a computer device, including but not limited to the host of various intelligent medical devices, personal computers, laptop computers, smartphones, tablet computers, and portable wearable devices, etc.

[0060] Hereinafter, the main control device 120 will be further described in detail. Refer to Figure 2 as shown, Figure 2 shows a schematic structural diagram of the main control device in some embodiments. Among them, the main control device 120 may at least include the following modules:

[0061] The image processing module 210 is used to receive the two-dimensional image data of the target skin area of the target object obtained through various different acquisition methods, and perform integration processing on the multiple two-dimensional image data to obtain a set of two-dimensional image values.

[0062] Among them, the image processing module 210 can integrate and process the two-dimensional image data collected from various skin detection instruments to generate a set of two-dimensional image values. Further, a unified three-dimensional image can be generated based on the set of two-dimensional image values after integration processing and displayed to the user.

[0063] The problem prediction module 220 obtains problem prediction data and the lesion area to be treated based on the set of two-dimensional image values and the skin problem prediction model.

[0064] Among them, the skin problem prediction model can be pre-trained with normal skin image samples and lesion skin image samples, so that it can predict the lesion area to be treated in the target skin area through the input set of two-dimensional image values, and predict the possible problem areas, problem types, and / or problem severity grades of the lesion area to be treated to obtain problem prediction data.

[0065] More specifically, the skin problem prediction model can be constructed based on a machine learning model or an AI (Artificial Intelligence) model of a neural network. For example, a multimodal large language model (such as SkinGPT-4) can be used, or other machine learning models can be used as the basic model for training, which is not particularly limited here.

[0066] The solution generation module 230 is used to obtain a treatment planning path based on the image data of the lesion area to be treated and the path planning model, obtain recovery effect prediction data based on the image data of the lesion area to be treated, the problem prediction data, and the skin recovery prediction model, and generate a treatment plan based on the treatment planning path and the recovery effect prediction data.

[0067] Among them, the path planning model is a model that can reflect the correlation between lesion skins of different problem types with different granularities, different hardnesses, and / or different color depths and the motion trajectory data of the robotic arm. The path planning model can be stored in the memory of the main control device 120 or embedded in the intelligent processing unit of the path execution device (such as an intelligent robotic arm), and can be trained and constructed based on a machine learning model or an AI model of a neural network.

[0068] In this embodiment, it is possible to support calling the path planning model in the intelligent processing unit of a traditional intelligent robotic arm to plan the treatment path, or to use the path planning model trained in a creative way in the following embodiments of the present application to plan the treatment path.

[0069] Among them, the effect prediction model is a model that can reflect the correlation between the damage level data and the types of sequelae left after treating diseased skin of different problem types with different types of treatment instruments and different operating parameters, and the recovery effect data of the diseased skin at different recovery time stages. The effect prediction model can be stored in the memory of the main control device 120 and can be trained and constructed based on a machine learning model or an AI model of a neural network.

[0070] The main control device, the skin medical beauty system, and the storage medium of the above skin medical beauty system can integrate the two-dimensional image data of the target skin area of the target object obtained by a variety of different acquisition methods, determine the problem prediction data and the area of the diseased area to be treated based on the integrated two-dimensional image numerical set for treatment path planning, and combine the recovery effect prediction data predicted with the problem prediction data and the area of the diseased area to be treated as input variables when generating a treatment plan, which can quickly and automatically generate a personalized treatment plan. Further, it can assist doctors in judging skin problems, selecting treatment instruments and corresponding operating parameters, etc. Therefore, it can improve the efficiency and effect of personalized treatment of skin medical beauty.

[0071] In some embodiments, when the image processing module 210 is used to integrate and process multiple two-dimensional image data to obtain a two-dimensional image numerical set, specifically, it can also be used to: respectively screen multiple pixel points from each two-dimensional image as numerical sampling points; integrate the numerical values and precision values of the numerical sampling points at the corresponding sampling positions of each two-dimensional image to obtain the integrated numerical value and integrated precision value of the numerical sampling points at each sampling position; construct a two-dimensional image numerical set according to each integrated numerical value and integrated precision value.

[0072] In this embodiment, integrating the two-dimensional image data from different acquisition methods in units of numerical sampling points can realize the fusion of two-dimensional image data of different acquisition methods, thereby realizing the fusion of different information. Since the two-dimensional image data of different acquisition methods carry information of different skin layers, different skin depths or different imaging ranges, a numerical set containing more comprehensive and integrated skin information can be obtained through the integration of two-dimensional image data. Therefore, it can not only support and adapt to different skin detection instruments, unify and integrate the image data collected by different methods, improve the accuracy of subsequent three-dimensional image generation and problem prediction, etc., and provide data support for subsequent processing.

[0073] In some embodiments, reference may be made to Figure 6 as shown Figure 6 shows a schematic structural diagram of the main control device in some other embodiments, where Figure 2Based on this, the master control device 120 may further include a human-computer interaction module 240, and the human-computer interaction module 240 may be used to: construct a three-dimensional image according to the two-dimensional image numerical set; display the to-be-treated lesion area and / or problem prediction data in the three-dimensional image according to the problem prediction data and the to-be-treated lesion area; in response to the user's change operation on the to-be-treated lesion area and / or problem prediction data, feedback the changed data to the skin problem prediction model for re-prediction.

[0074] In this embodiment, the master control device 120 may also be provided with a data interface for human-computer interaction, which is used to access the human-computer interaction module 240. The human-computer interaction module 240 may include, but is not limited to, a display device and an input device. The display device may be used to display the constructed three-dimensional image and display the to-be-treated lesion area and / or problem prediction data in the three-dimensional image in a specific manner indicated by the user. The input device may support the user to input and / or change data in any feasible custom manner. For example, if the user has objections to the to-be-treated lesion area and / or problem prediction data predicted by the skin problem prediction model, the user can, through the "manual marking" function, customarily change or re-mark the lesion data, and can freely choose whether to let the skin problem prediction model calculate again, or also support directly using the data after manual change for subsequent processing.

[0075] Furthermore, regardless of the selection, the master control device 120 will transmit the data to the background database, that is, the background will automatically record the manually marked data, and during the non-working hours of the master control device 120, use the updated data as training samples to retrain the convolutional neural network in the skin problem prediction model to achieve the update and optimization of the model.

[0076] In this embodiment, an interface allowing the user to supplement or update the problem prediction data and the data of the to-be-treated lesion area through real-time data marking is added, which can timely avoid the hallucination limitation of AI technology and improve the expandability and adaptability of the entire device.

[0077] In some embodiments, with continued reference to Figure 6 , the master control device 120 further includes a coordinate calibration module 250, and the coordinate calibration module 250 may be used to: construct a first spatial coordinate system A1 according to the two-dimensional image numerical set; in response to a scheme execution request, construct a second spatial coordinate system A2 according to the two-dimensional image data of the target skin area of the target object re-acquired during the movement of the robotic arm; calibrate the first spatial coordinate system A1 according to the second spatial coordinate system A2.

[0078] In this embodiment, in response to a program execution request, that is, after the treatment process is started, before the treatment unit connected to the end of the path execution device (for example, the laser therapy instrument at the end of the intelligent robotic arm) emits light, the two-dimensional image data of the target skin area of the target object is collected again through the imaging component of the path execution device itself, and a spatial coordinate system is reconstructed as the second spatial coordinate system A2. The reconstructed second spatial coordinate system A2 is compared with the previously constructed first spatial coordinate system A1, so as to realize the calibration of the treatment planning path, avoid the movement of the treatment target point of the path execution device caused by the movement of the target object, and thus improve the safety of treatment.

[0079] In some embodiments, referring to Figure 3 as shown, Figure 3 shows a schematic flow chart of training a skin problem prediction model in some embodiments. The skin problem prediction model can be configured to be trained through the following steps:

[0080] Step S310: Collect normal skin image samples and diseased skin image samples;

[0081] Step S320: Mark the diseased areas in the diseased skin image samples, and configure marking instructions according to the problem areas, problem types, and / or problem severity grades;

[0082] Step S330: Train a convolutional neural network according to the normal skin image samples and the diseased skin image samples, and construct the skin problem prediction model according to the trained convolutional neural network.

[0083] In this embodiment, the main control device 120 may include a skin problem prediction model for skin problem diagnosis. The skin problem prediction model can be constructed based on a multimodal large language model (such as SkinGPT-4). The skin problem prediction model can be trained according to the marked normal skin image samples and the set of diseased skin image samples. The skin problem prediction model trained by the method of this embodiment can more accurately identify and process the two-dimensional image numerical set of the integrated skin image, determine problem prediction data such as the key problem areas, problem types, and problem severity grades of the target skin area, and can process the return value of the convolutional neural network in the skin problem prediction model through the software program of the scheduling foreground, so as to generate a comprehensive skin problem diagnosis report containing problem prediction data and information on the diseased areas to be treated. Further, the comprehensive skin problem diagnosis report can be integrated into the aforementioned visual three-dimensional image.

[0084] In some embodiments, referring to Figure 4 as shown, Figure 4The flowchart of training the path planning model in some embodiments is shown. Among them, the path planning model is configured to be trained through the following steps:

[0085] Step S410: Use foreign objects with different granularities, different hardnesses, and / or different color depths as features to simulate diseased skin of different problem types, and collect lesion skin image data of each simulated diseased skin;

[0086] Step S420: During the process of simulating laser treatment on each simulated diseased skin, collect the motion trajectory data of the path execution device, and train the path planning model according to each lesion skin image data and the motion trajectory data corresponding to each said lesion image data.

[0087] In this embodiment, the main control device 120 can call the built-in path planning model or the path planning model stored in the intelligent unit of the path execution device (for example, an intelligent robotic arm) to perform treatment path planning. The path planning model can be pre-trained. Specifically, pork with skin can be used to simulate the human body surface skin, and foreign objects with different granularities, different hardnesses, and / or different color depths are used as features to simulate diseased skin of different problem types. First, manually control the path execution device to emit laser and / or ultrasound to the simulated foreign object area, so as to simulate the progress of the skin treatment process, and collect image data from multiple angles through the imaging component built-in or external to the path execution device, providing training data for the path execution device to represent the correlation between the motion trajectory data and the imaging data of the simulated diseased area. Then, techniques such as Reinforcement Learning (RL) and Boosting can be used to train the path planning model to form the ability to solve the simulated skin treatment task, and enable the path execution device to synchronously form the imaging ability of visualizing the treatment planning path.

[0088] By using the method of this embodiment, the pre-trained path planning model can improve the accuracy of generating the treatment planning path and can adapt to different types of lesion data.

[0089] In some embodiments, refer to Figure 5 as shown, Figure 5 The flowchart of training the effect prediction model in some embodiments is shown. Among them, the effect prediction model is configured to be trained in the following manner:

[0090] Step S510: Obtain the damage level data and the remaining sequela type data caused by treating diseased skin of different problem types under different treatment instrument types and different operating parameters;

[0091] Step S520: Periodically obtain the recovery effect data of the diseased skin at different recovery time stages;

[0092] Step S530: Train the recovery effect prediction model according to different diseased skin image data and their corresponding therapeutic instrument types, operating parameters, injury level data, remaining sequela type data, and recovery effect data.

[0093] In this embodiment, based on animal experiments, the overall conclusions of the injury level data and the remaining sequela type data caused by treating different types of diseased skin with different therapeutic instrument types and different operating parameters can be obtained. The natural recovery cycle of animals can be used to simulate the natural recovery cycle of humans, and the recovery effect data of animal experiments can be periodically tracked and collected, so as to obtain the skin recovery effect data marked with the recovery time stage. Then, a treatment recovery database is established by combining this conclusion with the collected skin recovery effect data marked with the recovery time stage. Using this database, a recovery effect prediction model for postoperative skin recovery is formed. Further, the prediction result of the recovery effect prediction model can be visualized to assist users in making clinical decisions.

[0094] In this embodiment, since the recovery effect data of the skin at different recovery time stages, the correlation between different therapeutic instrument types and different operating parameters used for different types of diseased skin, and the evaluation of characteristics such as injury level data and remaining sequela type are introduced in the training of the model, the treatment plan generated by combining the recovery effect prediction data of the skin recovery prediction model can more accurately assist users in determining how to select the therapeutic instrument and the corresponding operating parameters at different time stages.

[0095] In some embodiments, the solution generation module 230 is further configured to: determine the types of therapeutic instruments and operating parameters corresponding to the recovery effects at different recovery time stages according to the recovery effect prediction data, and generate a treatment plan for the current time stage according to the types of therapeutic instruments and operating parameters at the current time stage.

[0096] In this embodiment, since the recovery effect prediction model trained according to the above embodiment can output the data reflecting the correlation between different therapeutic instrument types, different operating parameters, and the data of different types of diseased skin they act on and the recovery effect of the diseased skin at different recovery time stages, that is, the recovery effect data, the types of therapeutic instruments to be selected and the operating parameters of the therapeutic instrument at a certain or certain time stages can be determined according to the predicted recovery effect data.

[0097] Exemplarily, when the treatment instrument is a laser-type treatment instrument, the operating parameters may include, but are not limited to, parameters such as the light emission wavelength type, the light emission time, and the light emission energy.

[0098] It should be understood that although Figures 3 to 5 the steps in the flowchart of Figures 3 to 5 are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover,

[0099] In some embodiments, referring to Figure 7 shown, Figure 7 shows a schematic structural diagram of the main control device in some application examples. Among them, each module in the above-mentioned main control device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the main control device in hardware form or independent of it, or can be stored in the memory of the main control device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0100] In some application scenarios, the main control device can be an intelligent terminal device, and its internal structure diagram can be as Figure 7 shown. The main control device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the main control device is used to provide computing and control capabilities. The memory of the main control device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the main control device is used to communicate with external devices through a network connection. When the main control device is executed by the processor, it realizes the functions of the above embodiments. The display screen of the main control device can be a liquid crystal display screen or an electronic ink display screen. The input device of the main control device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the main control device, or an external keyboard, touchpad, or mouse, etc.

[0101] Those skilled in the art can understand that Figures 1 to 2 , Figures 6 to 7The structure shown is only a block diagram of some of the structures related to the solution of this application, and does not constitute a limitation on the main control device and the skin medical beauty system to which the solution of this application is applied. Specifically, the main control device and the skin medical beauty system may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0102] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: receiving two-dimensional image data of a target skin area of a target object obtained by a variety of different acquisition methods, and performing integration processing on the multiple two-dimensional image data to obtain a set of two-dimensional image values; obtaining problem prediction data and an area of a lesion to be treated according to the set of two-dimensional image values and a skin problem prediction model; obtaining a treatment planning path according to the image data of the area of the lesion to be treated and a path planning model, obtaining recovery effect prediction data according to the image data of the area of the lesion to be treated, the problem prediction data, and a skin recovery prediction model, and generating a treatment plan according to the treatment planning path and the recovery effect prediction data.

[0103] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: screening multiple pixel points from each two-dimensional image as numerical sampling points; integrating the numerical values and precision values of the numerical sampling points at the corresponding sampling positions of each two-dimensional image to obtain the integrated numerical value and integrated precision value of the numerical sampling points at each sampling position; constructing a set of two-dimensional image values according to the integrated numerical values and integrated precision values.

[0104] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: constructing a three-dimensional image according to the set of two-dimensional image values; displaying the area of the lesion to be treated and / or the problem prediction data in the three-dimensional image according to the problem prediction data and the area of the lesion to be treated; in response to a user's change operation on the area of the lesion to be treated and / or the problem prediction data, feeding the changed data back to the skin problem prediction model for re-prediction.

[0105] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: constructing a first spatial coordinate system according to the set of two-dimensional image values; in response to a scheme execution request, constructing a second spatial coordinate system according to the two-dimensional image data of the target skin area of the target object re-acquired during the movement of the robotic arm; calibrating the first spatial coordinate system according to the second spatial coordinate system.

[0106] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: collecting normal skin image samples and diseased skin image samples; marking the diseased areas in the diseased skin image samples, and configuring marking descriptions according to problem areas, problem types, and / or problem severity levels; training a convolutional neural network based on the normal skin image samples and the diseased skin image samples, and constructing a skin problem prediction model according to the trained convolutional neural network.

[0107] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: using foreign objects with different granularities, different hardnesses, and / or different color depths as features to simulate diseased skin of different problem types, and collecting diseased skin image data of each simulated diseased skin; during the process of performing simulated laser treatment on each simulated diseased skin, collecting the motion trajectory data of the path execution device, and training the path planning model according to each diseased skin image data and the motion trajectory data corresponding to each diseased image data.

[0108] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: obtaining damage level data and remaining sequela type data caused by treating diseased skin of different problem types under different treatment instrument types and different operating parameters; periodically obtaining recovery effect data of diseased skin at different recovery time stages; training a recovery effect prediction model according to different diseased skin image data and their corresponding treatment instrument types, operating parameters, damage level data, remaining sequela type data, and recovery effect data.

[0109] In some embodiments, when the computer program is executed by a processor, the following steps are further implemented: determining the types of treatment instruments and operating parameters corresponding to the recovery effects at different recovery time stages according to the recovery effect prediction data, and generating a treatment plan for the current time stage according to the types of treatment instruments and operating parameters at the current time stage.

[0110] Those of ordinary skill in the art can understand that implementing all or part of the processes in the above embodiments can be accomplished by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0111] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0112] In addition, the term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the characters in this article generally represent that the associated objects before and after are in an "or" relationship.

[0113] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

[0114] It should be noted that in the embodiments of the present application, for relevant data such as user information or user data (e.g., data of the target skin area of the target object, etc.), it is necessary to obtain the authorization and consent of the user before acquisition and processing. When the embodiments of the present application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the user, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions.

Claims

1. A main control device of a skin medical beauty system, the main control device comprising: An image processing module, used for receiving two-dimensional image data of a target skin area of ​​a target object obtained by a plurality of different acquisition methods, and integrating and processing a plurality of the two-dimensional image data to obtain a two-dimensional image value set; A problem prediction module, used to obtain problem prediction data and a lesion area to be treated according to the two-dimensional image value set and the skin problem prediction model; A plan generation module is used to obtain a treatment planning path based on the image data of the lesion area to be treated and the path planning model, obtain recovery effect prediction data based on the image data of the lesion area to be treated, the problem prediction data and the skin recovery prediction model, and generate a treatment plan based on the treatment planning path and the recovery effect prediction data.

2. The main control device according to claim 1, characterized in that: The image processing module is also used for: Selecting a plurality of pixel points from each of the two-dimensional images as numerical sampling points; Integrate the values ​​and precision values ​​of the numerical sampling points at the mutually corresponding sampling positions of the two-dimensional images to obtain the integrated values ​​and integrated precision values ​​of the numerical sampling points at each sampling position; The two-dimensional image value set is constructed according to each of the integrated values ​​and the integrated precision values.

3. The main control device according to claim 1, characterized in that: The main control device also includes a human-computer interaction module, which is used to: constructing a three-dimensional image according to the two-dimensional image value set; Displaying the lesion area to be treated and / or the problem prediction data in the three-dimensional image according to the problem prediction data and the lesion area to be treated; In response to the user's modification operation on the lesion area to be treated and / or the problem prediction data, the modified data is fed back to the skin problem prediction model for re-prediction.

4. The main control device according to claim 1, characterized in that: The main control device also includes a coordinate calibration module, which is used to: Constructing a first spatial coordinate system according to the two-dimensional image value set; In response to a request for executing a scheme, constructing a second spatial coordinate system according to the two-dimensional image data of the target skin area of ​​the target object re-collected by the robot arm during the movement; The first space coordinate system is calibrated according to the second space coordinate system.

5. The main control device according to claim 1, characterized in that: The skin problem prediction model is configured to be trained in the following manner: Collecting normal skin image samples and diseased skin image samples; Marking the lesion area in the lesion skin image sample, and configuring the marking instructions according to the problem area, problem type and / or problem severity level; A convolutional neural network is trained according to the normal skin image samples and the diseased skin image samples, and the skin problem prediction model is constructed according to the trained convolutional neural network.

6. The main control device according to claim 1, characterized in that: The path planning model is configured to be trained in the following manner: Using foreign bodies of different particle sizes, different hardnesses and / or different color depths as features to simulate lesion skins of different problem types, and collecting lesion skin image data of each simulated lesion skin; During the process of simulating laser treatment of each simulated diseased skin, the motion trajectory data of the path execution device is collected, and the path planning model is trained according to each diseased skin image data and the motion trajectory data corresponding to each diseased image data.

7. The main control device according to claim 1, characterized in that: The effect prediction model is configured to be trained in the following manner: Obtain data on the level of damage and the type of residual sequelae caused by treating skin lesions of different problem types under different treatment device types and different operating parameters; Periodically acquiring recovery effect data of the diseased skin at different recovery time stages; The recovery effect prediction model is trained according to different diseased skin image data and their corresponding treatment device types, operating parameters, injury level data, residual sequelae type data and recovery effect data.

8. The main control device according to claim 7, characterized in that: The solution generation module is also used for: The types of therapeutic devices and operating parameters corresponding to the recovery effects at different recovery time stages are determined based on the recovery effect prediction data, and a treatment plan for the current time stage is generated based on the types of therapeutic devices and operating parameters for the current time stage.

9. A skin medical beauty system, characterized in that: The system comprises: A skin detection unit, which includes a plurality of skin detectors with different image acquisition modes, for acquiring two-dimensional image data of a target skin area of ​​a target object; The main control device according to any one of claims 1 to 8; a path execution device, configured to move according to a treatment planning path generated by the main control device under the control of the main control device; The treatment unit includes a plurality of treatment apparatuses with different treatment modes, and is used to connect to a target treatment apparatus according to a treatment plan generated by the main control device under the control of the main control device, and to determine operating parameters of the target treatment apparatus.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it realizes: Receiving two-dimensional image data of a target skin area of ​​a target object obtained by a plurality of different acquisition methods, and integrating a plurality of the two-dimensional image data to obtain a two-dimensional image value set; Obtain problem prediction data and a lesion area to be treated according to the two-dimensional image value set and the skin problem prediction model; A treatment planning path is obtained based on the image data of the lesion area to be treated and the path planning model, recovery effect prediction data is obtained based on the image data of the lesion area to be treated, the problem prediction data and the skin recovery prediction model, and a treatment plan is generated based on the recovery effect prediction data.

Images