Image processing method of image data and intelligent laser device

Abstract

The invention relates to the technical field of laser, in particular to an image processing method of image data and an intelligent laser device, and the method comprises the steps: obtaining multi-modal image data of a first position of a user; reconstructing the multi-modal image data of the first position into a 3D image, and automatically identifying and acquiring basic information of a specified area according to the 3D image; inputting the basic information of the designated area into a scoring model constructed in advance, obtaining the state score of the designated area of the user, and obtaining related parameters according to the state score of the designated area of the user and a processing parameter setting rule set in advance; and according to the 3D image and the position of the designated area, an accurate path of the mechanical arm is calculated and generated, the mechanical arm reaches the designated area along the accurate path, and a laser processing head connected with the mechanical arm is operated according to related parameters. According to the image processing method utilizing the image data and the intelligent laser device, the problem that the accuracy and consistency of operation of the laser device are insufficient is solved, and accuracy, consistency and minimally invasive performance are achieved.

Description

Technical Field

[0001] The present invention relates to the field of laser technology, and in particular to an image processing method for image data and an intelligent laser device. Background Art

[0002] Nowadays, laser technology has been widely used in the medical field. Soon after the first ruby ​​laser came out, it was quickly used in ophthalmology to treat fundus diseases. Soon after, the birth of carbon dioxide laser was also quickly introduced into surgical operations for precise surgical cutting. With the continuous advancement of optical transmission technology and the advent of ND:YAG laser, it was quickly used in the endoscopic treatment of gastrointestinal diseases. With the continuous improvement of solid laser technology based on the yttrium aluminum garnet series, such as thulium Tm, holmium Ho, erbium Er, neodymium Nd, etc., and the emergence of high-power semiconductor laser technology, laser medical equipment has gradually achieved miniaturization and intelligence. These advances have brought a new climax to the clinical application of lasers, and laser devices have been widely used in clinical practice.

[0003] However, existing oral and skin laser devices rely entirely on manual operation, which is affected by the operating habits of different users and cannot achieve accuracy and consistency in operation. They lack effectiveness and safety. Existing oral and skin laser devices lack digital standard operating specifications, and there are large differences between the plans of different operators, making it difficult to ensure accuracy and consistency. At the same time, existing laser devices cannot achieve digital path planning, and manual operation is difficult to achieve accuracy.

[0004] Therefore, there is an urgent need for an image processing method for image data and an intelligent laser device. Summary of the invention

[0005] 1. Technical issues to be resolved In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an image processing method and an intelligent laser device for image data, which solve the technical problems in the prior art that the accuracy and consistency of laser operation are low and it is difficult to achieve precise operation.

[0006] (II) Technical solution In order to achieve the above object, the main technical solutions adopted by the present invention include: In a first aspect, the present invention provides an image processing method for image data, comprising: S100, obtaining multimodal image data of a user at a first position; the first position is inside the oral cavity or the skin to be measured; S200, reconstructing the multimodal image data of the user's first position into a 3D image, and automatically identifying and acquiring basic information of a designated area according to the 3D image; The basic information of the designated area includes: the location of the designated area, the size of the designated area, the depth of the designated area, and the irregularity of the designated area; S300, inputting the size of the designated area, the depth of the designated area, and the irregularity of the designated area in the basic information of the designated area into a scoring model constructed in advance, obtaining the designated area status score of the user, and obtaining relevant parameters according to the designated area status score of the user and a processing parameter setting rule set in advance; S400, calculating and generating a precise path of the robot arm according to the 3D image and the position of the designated area, the robot arm reaches the designated area along the precise path, and operating the laser processing head connected to the robot arm according to relevant parameters.

[0007] Optionally, in S200, automatically identifying and acquiring basic information of a designated area according to a 3D image includes: S210, using a training data set to train a designated area basic information acquisition model to obtain a trained designated area basic information acquisition model; S220, inputting the 3D image into a trained model for acquiring basic information of a designated area to acquire basic information of the designated area; The basic information acquisition model of the designated area is a convolutional neural network model; The training data set is past 3D image data and basic information of the corresponding designated area.

[0008] Optionally, the S210 specifically includes: The past 3D image data and the basic information of the corresponding designated area are input into the designated area basic information acquisition model. The cross entropy function and Adam optimizer are used with a learning rate of 0.001 until the cross entropy regression loss function converges to obtain the trained designated area basic information acquisition model.

[0009] Optionally, the S300 includes: S310, inputting the size of the designated area, the depth of the designated area, and the irregularity of the designated area in the basic information of the designated area into a scoring model constructed in advance, and obtaining a first score of the designated area state of the user-designated area; S320, normalizing the first score of the designated area status to obtain a designated area status score; S330: Obtain relevant parameters according to the designated area status score and the processing parameter setting rules set in advance.

[0010] Optionally, the S310 includes: The size of the specified area, the depth of the specified area, and the irregularity of the specified area are input into the following formula to obtain the first score of the user-specified area status: ; Among them, ω s is the influence weight of the specified area size, ω D is the influence weight of the specified area depth, ω F represents the influence weight of the specified area morphology, S is the size of the specified area, S max The maximum reference value for the specified region size, k s is the nonlinear intensity coefficient, D is the depth of the specified area, and D max is the maximum reference depth, β D is the steepness of the saturation curve, F is the irregularity of the specified area, and Score is the first score of the specified area status.

[0011] Optionally, the S320 includes: The first score of the designated area status is input into the following formula for normalization processing to obtain the designated area status score: ; Among them, Final Score is the status score of the specified area, ω s is the influence weight of the specified area size, ω D is the influence weight of the specified area depth, ω F Indicates the influence weight of the specified area morphology, k s is the nonlinear intensity coefficient, D max is the maximum reference depth, F max is the maximum irregularity set.

[0012] Optionally, the S330 includes: The specified area status score is input into the following formula to obtain relevant parameters: ; Among them, T is the relevant parameter, T min is the minimum value of the relevant parameter, T max is the maximum value of the relevant parameters, and FinalScore is the status score of the specified area; The relevant parameters include laser power, laser operation time and laser irradiation area.

[0013] Optionally, in S330, The minimum value of the laser power is 0.5W and the maximum value is 36W; The minimum laser operation time is 1 second and the maximum is 30 minutes. The minimum laser irradiation area is 0.5mm 2 , the maximum value is 49cm 2 .

[0014] Optionally, the S400 specifically includes: S410, based on the designated area position and the 3D image, using a cubic spline interpolation algorithm to generate a continuous path of the robot arm in the three-dimensional space; S420, obtaining the precise path of the robot arm's operation according to the shortest path algorithm and the continuous path of the robot arm in three-dimensional space, the robot arm moves to a designated area along the precise path, and operates the laser processing head connected to the robot arm according to relevant parameters.

[0015] In a second aspect, an embodiment of the present invention provides an intelligent laser device, the intelligent laser device is used to execute an image processing method of image data as described in any one of the first aspects, the intelligent laser device comprising: Robotic arm, laser processing head, sensor assembly, laser generator, central processing assembly and display screen; The mechanical arm, the laser processing head, the sensor assembly, the laser generator, the central processing assembly and the display screen are electrically connected to the central processing assembly respectively; The robotic arm is connected to the laser processing head via a rotary joint with a micro motor.

[0016] (III) Beneficial effects The beneficial effects of the present invention are as follows: an image processing method and an intelligent laser device of the present invention use 3D images to obtain basic information of a designated area, and score the user's situation through a scoring model, and obtain relevant parameters according to the score. Compared with the prior art, it can achieve the accuracy and consistency of laser operation, realize the personalization of operation plans, increase the suitability of operation plans, and improve the safety and effectiveness of laser operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a flow chart of an image processing method for image data according to Embodiment 1 of the present invention; Figure 2 is a schematic flow chart of an image processing method for image data in Embodiment 2 of the present invention; Figure 3 This is a simplified structural diagram of an intelligent laser device according to Example 5 of the present invention. DETAILED DESCRIPTION

[0018] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0019] The image processing method and intelligent laser device of the embodiment of the present invention realize the accuracy and consistency of the operation of the laser device, and solve the problem of insufficient accuracy, precision and consistency of the existing device. The present application scores the user's situation through a scoring model, obtains relevant parameters according to the score, realizes the accuracy and consistency of the laser operation, realizes the personalization of the operation plan, increases the suitability of the operation plan, and improves the effectiveness and safety of the laser operation.

[0020] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0021] Example 1

[0022] See also Figure 1 , an image processing method of image data of this embodiment includes: Step S100, obtaining multimodal image data of a user at a first position; the first position is inside the oral cavity or the skin to be measured; Step S200: reconstructing the multimodal image data of the user's first position into a 3D image, and automatically identifying and acquiring basic information of a designated area according to the 3D image; The basic information of the designated area includes: the location of the designated area, the size of the designated area, the depth of the designated area, and the irregularity of the designated area; Step S300: input the designated area size, designated area depth and designated area irregularity in the basic information of the designated area into a scoring model constructed in advance, obtain the designated area status score of the user, and obtain relevant parameters according to the designated area status score of the user and the processing parameter setting rules set in advance; Step S400: Calculate and generate a precise path for the robot arm based on the 3D image and the position of the designated area. The robot arm reaches the designated area along the precise path and operates a laser processing head connected to the robot arm according to relevant parameters.

[0023] The image processing method of image data of the present embodiment realizes the accuracy and consistency of image processing of multimodal image data of the oral cavity and skin, realizes the personalization of operation plans, increases the suitability of operation plans, and improves the effectiveness and safety of laser operation.

[0024] Example 2

[0025] See also Figure 2, an image processing method of image data of this embodiment includes: Step A100, obtaining multimodal image data of a user at a first position; the first position is inside the oral cavity or the skin to be measured; Step A200: reconstructing the multimodal image data of the user's first position into a 3D image, and automatically identifying and acquiring basic information of a designated area based on the 3D image; The basic information of the designated area includes: the location of the designated area, the size of the designated area, the depth of the designated area, and the irregularity of the designated area; Step A300: input the designated area size, designated area depth and designated area irregularity in the basic information of the designated area into a scoring model constructed in advance, obtain the designated area status score of the user, and obtain relevant parameters according to the designated area status score of the user and the processing parameter setting rules set in advance; Step A400: Calculate and generate a precise path for the robot arm based on the 3D image and the position of the designated area. The robot arm reaches the designated area along the precise path and operates a laser processing head connected to the robot arm according to relevant parameters.

[0026] Specifically, in A200, automatically identifying and obtaining basic information of a specified area based on a 3D image includes: A210, training a basic information acquisition model for a designated area using a training data set to obtain a trained basic information acquisition model for a designated area; A220, inputting the 3D image data into a trained model for obtaining basic information of a designated area to obtain basic information of the designated area; The model for acquiring basic information of the designated area is a convolutional neural network model; The training data set is the past 3D image data and the basic information of the corresponding designated area.

[0027] A210 specifically includes: The past 3D image data and the basic information of the corresponding designated area are input into the designated area basic information acquisition model. The cross entropy function and Adam optimizer are used with a learning rate of 0.001 until the cross entropy regression loss function converges to obtain the trained designated area basic information acquisition model.

[0028] At the same time, GS preprocessing is introduced during the training process to improve the model's ability to recognize spatial details of irregular boundaries; Furthermore, the steps of constructing and training a model for obtaining basic information of a specified area are as follows: Data preprocessing and enhancement: First, we collected past 3D images of the oral cavity and skin from multiple sources to ensure diversity. Then, we preprocessed the data, including denoising, standardization, data enhancement (such as rotation, scaling, flipping, etc.), and GS preprocessing to enhance the generalization ability of the model. Among them, GS preprocessing is to convert the 3D image into a sparse Gaussian point cloud representation to reduce noise interference and enhance edge details; The basic information acquisition model for the designated area in this embodiment is combined with GS preprocessing on the basis of the convolutional neural network, which can make up for the shortcomings of the convolutional neural network in not being able to grasp the global shape, and can effectively enhance the recognition of designated areas, improve boundary characterization and efficient use of data.

[0029] Label refinement and quality control: Each 3D image is equipped with detailed labels of basic information of designated areas, which are manually annotated by experienced doctors and undergo multiple levels of review to ensure their accuracy; Model architecture: A 3D convolutional neural network (3D CNN) is used as the basic information acquisition model for the specified area. At the same time, an attention mechanism is further introduced to enable the model to focus on the key areas in the image, i.e. the specified areas, thereby improving the detection accuracy.

[0030] Training optimization: Use the cross entropy function and Adam optimizer with a learning rate of 0.001 until the cross entropy regression loss function converges, and obtain the trained basic information of the specified area to obtain the model. At the same time, introduce the early stopping mechanism to avoid overfitting, and use data enhancement technology to increase the robustness of the model.

[0031] The learning rate is also dynamically adjusted during the training process. The learning rate is: ; Among them, η t represents the learning rate at t iterations, η0 is the initial learning rate, set to 0.001, is a term that decays with iteration t, where α is the decay coefficient. As the training progresses, this term will gradually decrease, causing the learning rate to gradually decrease; is a term that dynamically adjusts the learning rate according to the rate of change of the loss function and the gradient norm. represents the change in the loss function at the tth iteration, is the L2 norm of the gradient of the tth iteration, and β is the adjustment coefficient.

[0032] Furthermore, in order to enhance the interpretability of the model, the basic information acquisition model of the designated area in this embodiment also includes a set of visualization tools that can superimpose the basic information of the designated area on the original 3D image, so that the location and characteristics of the designated area can be intuitively seen.

[0033] In this embodiment, A300 specifically includes: A310, inputting the size of the designated area, the depth of the designated area, and the irregularity of the designated area in the basic information of the designated area into a scoring model constructed in advance, and obtaining a first score of the designated area state of the user-specified area; Specifically: Input the specified area size, specified area depth and specified area irregularity into the following formula to obtain the first score of the user-specified area status: ; Among them, ω s is the influence weight of the specified area size, ω D is the influence weight of the specified area depth, ω F represents the influence weight of the specified area morphology, S is the size of the specified area, S max The maximum reference value for the specified region size, k s is the nonlinear intensity coefficient, D is the depth of the specified area, and D max is the maximum reference depth, β D is the steepness of the saturation curve, F is the irregularity of the specified area, and Score is the first score of the specified area status.

[0034] The range of the irregularity of the specified area is 0-1, where 0 indicates a completely regular shape and 1 indicates an extremely irregular shape.

[0035] A320, normalizing the first score of the designated area status to obtain a designated area status score; The first score of the designated area status is input into the following formula for normalization processing to obtain the designated area status score: ; Among them, Final Score is the status score of the specified area, ω s is the influence weight of the specified area size, ω D is the influence weight of the specified area depth, ω F Indicates the influence weight of the specified area morphology, k s is the nonlinear intensity coefficient, D max is the maximum reference depth, F max is the maximum irregularity set.

[0036] A330. Obtain relevant parameters according to the designated area status score and the processing parameter setting rules set in advance.

[0037] The specified area status score is input into the following formula to obtain relevant parameters: ; Among them, T is the relevant parameter, T min is the minimum value of the relevant parameter, T max is the maximum value of the relevant parameters, and FinalScore is the status score of the specified area. The relevant parameters include laser power, laser operation time, and laser irradiation area; The minimum value of the laser power is 0.5W and the maximum value is 36W; The minimum laser operation time is 1 second and the maximum is 30 minutes. The minimum laser irradiation area is 0.5mm 2 , the maximum value is 49cm 2 .

[0038] In this embodiment, step S400 specifically includes: A410, based on the specified area position and 3D image, a cubic spline interpolation algorithm is used to generate a continuous path of the robot in three-dimensional space; A420. According to the shortest path algorithm and the continuous path of the robot arm in three-dimensional space, the precise path of the robot arm is obtained, the robot arm moves to the specified area along the precise path, and the laser processing head connected to the robot arm is operated according to relevant parameters.

[0039] In the specific implementation process, in order to ensure that the robot arm can safely and efficiently reach the designated area and operate the laser processing head on the robot arm, it is first necessary to plan the robot arm's operation path based on the specific location and 3D image of the designated area. In this process, the cubic spline interpolation algorithm can create a smooth and continuous path. The specific steps are as follows: Convert the position of the specified area into the reference coordinate system of the robot operation; Preprocess the original 3D image, including denoising, segmentation, feature extraction, etc., to clearly identify the designated area and its surrounding important structures; In the 3D image, select a series of key points according to clinical requirements, including the starting point, the end point, and the intermediate control points set to avoid important organs or structures, and use these points as nodes for cubic spline interpolation; at the same time, ensure that all selected key points are in the same coordinate system, usually the base coordinate system of the robot arm. If necessary, apply the necessary coordinate transformation matrix to adjust the position of the point; For each pair of adjacent key points, a cubic polynomial function is defined. The function is in the form of: ; Among them, t is the parameterized time variable, i represents the i-th curve; Set appropriate boundary conditions, such as zero velocity or acceleration at the start and end points (natural splines), or specify first or second derivatives at certain points (clamping conditions). This helps ensure smoothness and continuity of the path; Using the known key point positions and boundary conditions, a linear system of equations is established and solved to obtain the coefficient a of each polynomial. i , b i 、c i ,d i , that is, the continuous path of the robot arm in three-dimensional space is obtained.

[0040] Furthermore, after obtaining the continuous path of the robot arm in the three-dimensional space, the path needs to be further optimized, that is, according to the shortest path algorithm and the continuous path of the robot arm in the three-dimensional space, the precise path of the robot arm's operation is obtained. Specifically, the Dijkstra algorithm or other shortest path algorithms suitable for the three-dimensional environment are used to find the optimal path between the starting point and the target point.

[0041] In addition, in order to make the robot arm run smoothly, Bezier curves are used to smooth the path when obtaining the precise path of the robot arm, that is, to connect discrete path points into a continuous trajectory.

[0042] Among them, the Bezier curve is a parametric curve that defines the shape through control points. Before using the Bezier curve to smooth the path, it is necessary to first obtain the path nodes of the robot arm. This path is usually composed of a series of discrete nodes, namely: ; In this embodiment, a cubic Bezier curve is used to connect adjacent path nodes. Assume that the connecting node P i and P i+1 , follow these steps to determine: The tangent direction at each node can be calculated by approximating the tangent direction through the vectors of the adjacent nodes. For example, for node P i , its tangent direction vector It can be approximated as: , when i=0, ; For the connection node P i and P i+1 The cubic Bezier curve, if the four control points are P i , C i，1 , C i，2 , P i+1 , where C i，1 , C i，2 Determined according to the above tangent direction and proportional coefficient, assuming the proportional coefficient is k (usually between 0.2-0.5), then: ; ; For each segment of P i , C i，1 , C i，2 , P i+1 A cubic Bezier curve can be generated by uniformly sampling the parameter t∈[0,1]. For example, divide t from 0 to 1 into N equally spaced points t j =j / N, j=0,1,2,…,N, then the points on the curve can be calculated by the cubic Bezier curve formula: ; Q j is a point on the curve, t j To divide t from 0 to 1 into N equally spaced points, P i , C i，1 , C i，2 , P i+1 There are four control points.

[0043] By connecting the points generated by the Bezier curves of all segments, we get a smooth path.

[0044] At this stage, a PID controller (proportional-integral-differential controller) is also used to accurately monitor and dynamically adjust the operating state of the robot arm. The PID controller continuously receives data from sensors, such as the position, speed, and acceleration of the robot arm, and adjusts the output signal accordingly to correct the behavior that deviates from the set path. At the same time, the PID controller can also be used to adjust the relevant parameters of the laser processing head during processing to adapt to the individual differences of different users and real-time changes.

[0045] In summary, steps S410 and S420 can ensure that the robotic arm reaches the designated area position accurately according to the predetermined plan, while ensuring the safety and reliability of the operation.

[0046] Through the above method, a smooth, safe and efficient path from the starting position to the target designated area position can be obtained for the robotic arm to perform the laser operation task.

[0047] The image processing method of the image data of the present embodiment ensures the accuracy and consistency of the image processing of the multimodal image data of the oral cavity and the skin, realizes the personalization and suitability of the operation plan, and improves the effectiveness and safety of the laser operation.

[0048] Example 3

[0049] An image processing method of image data in this embodiment includes: Step C100, obtaining multimodal image data of a user at a first position; the first position is the inside of the oral cavity or the skin to be measured; Step C200: reconstructing the multimodal image data of the user's first position into a 3D image, and automatically identifying and acquiring basic information of a designated area based on the 3D image; The basic information of the designated area includes: the location of the designated area, the size of the designated area, the depth of the designated area, and the irregularity of the designated area; Step C300: input the designated area size, designated area depth and designated area irregularity in the basic information of the designated area into a scoring model constructed in advance, obtain the designated area status score of the user, and obtain relevant parameters according to the designated area status score of the user and the processing parameter setting rules set in advance; Step C400: Calculate and generate a precise path for the robot arm based on the 3D image and the position of the designated area. The robot arm reaches the designated area along the precise path and operates the laser processing head connected to the robot arm according to relevant parameters.

[0050] Specifically, in step C200, automatically identifying and acquiring basic information of a designated area according to a 3D image includes: Step C210: training the designated area basic information acquisition model using the training data set to obtain the trained designated area basic information acquisition model; Step C220: input the 3D image into the trained basic information acquisition model of the designated area to acquire the basic information of the designated area; The model for acquiring basic information of the designated area is a convolutional neural network model; The training data set is the past 3D image data and the basic information of the corresponding designated area.

[0051] In the specific implementation process, training the basic information acquisition model of the specified area specifically includes the following steps: Data collection: To ensure that the model has good generalization ability, we collect 3D images of past users and relevant information about the corresponding designated areas. These images come from a wide range of sources, covering users of different ages, genders, and races. They also include images of various skin conditions or oral conditions, such as skin ulcers, caries, periodontitis, malocclusion, etc. The data comes from hospitals and clinics.

[0052] Data denoising processing: The Non-Local Means Filtering method is used, which effectively removes noise by finding similar local areas in the image to average pixel values.

[0053] Standardization: The denoised image is standardized so that the image data has zero mean and unit variance.

[0054] Data enhancement, in order to increase the diversity of training data and improve the generalization ability of the model, this embodiment adopts a variety of data enhancement techniques, including rotation, scaling, flipping, etc. At the same time, an elastic deformation enhancement method is introduced. Elastic deformation is to apply a random elastic force field to the image to cause local deformation of the image. In specific implementation, a Gaussian kernel is used to generate a random displacement field, and then the image is interpolated according to the displacement field to obtain enhanced image data.

[0055] GS preprocessing: Convert the 3D image into a sparse Gaussian point cloud representation. First, voxelize the image and divide it into small voxels. Then, for each voxel, calculate the coordinates of its center point and the corresponding intensity value. Next, use the Gaussian function to fit these voxel center points to obtain a sparse Gaussian point cloud. In this way, noise interference can be reduced and edge details can be enhanced.

[0056] Label annotation: Each 3D image is equipped with a detailed label of basic information of the designated area, which is manually annotated by experienced doctors. The annotation content includes the location, size, irregularity and other characteristics of the designated area. For example, for the designated area of ​​the tooth, the doctor will mark the crown of the tooth, the boundary of the root, and the structure of the pulp cavity inside the tooth. To ensure the accuracy of the label, the annotated label will also be reviewed at multiple levels to ensure the accuracy and consistency of the label.

[0057] Model architecture: 3D convolutional neural network is used as the basic architecture of the model for obtaining basic information of the specified area. 3DCNN can directly process 3D image data and automatically extract features in the image through components such as convolution layer, pooling layer and fully connected layer. In the convolution layer, 3D convolution kernel is used for convolution operation, and its calculation formula is: ; Among them, x is the input 3D image, ω is the 3D convolution kernel, b is the bias term, y is the output after convolution, i, j, k are the coordinates of the 3D image data, m, n, l are the coordinates of the sliding window of the convolution kernel in three dimensions, and M, N, L are the sizes of the convolution kernel in three dimensions (i.e., the size of the convolution kernel).

[0058] Attention mechanism: In order to enable the model to focus on the key areas in the image, that is, the specified areas, the attention mechanism is introduced. Specifically, in the feature extraction process of 3D CNN, an attention module is added, which generates an attention weight map based on the input feature map, and then multiplies the attention weight map with the feature map to enhance the feature representation of the key area. The generation of the attention weight map can be calculated by the following formula: ; Among them, F is the input feature map, W1 and W2 are learnable weight matrices, σ is the sigmoid function, A is the attention weight map, and tanh is the hyperbolic tangent activation function.

[0059] Training Optimization: The cross entropy function is used as the loss function, and the Adam optimizer updates the model parameters. The calculation formula of the cross entropy loss function is: ; Among them, y i is the true label, is the predicted output of the model, that is, the basic information of the specified area, and N is the number of samples. The learning rate of the Adam optimizer is set to 0.001, and the cross entropy loss function converges by continuously iteratively updating the model parameters. i The sample weight coefficient is used to assign a weight to each sample according to its rarity or importance. For key area samples with a small number (such as some rare feature areas), a larger weight can be given to improve the model's learning effect on these samples.

[0060] At the same time, in the above formula, τ is the equilibrium parameter, is the KL divergence term, τ is used to balance the importance of the cross entropy term and the KL divergence term, and can be adjusted accordingly according to the specific task and data characteristics. The KL divergence term is introduced to measure the predicted distribution Q i With the prior just distribution P i For example, in the oral field, there is prior anatomical knowledge about structures such as teeth and gums. For example, the shape and position distribution of normal teeth have certain regularities. i Based on this prior knowledge, Q i This is the predicted distribution of the sample obtained by the basic information acquisition model of the specified area. By minimizing the KL divergence between the two, the prediction results of the model can be guided to be more consistent with the prior knowledge, thus enhancing the rationality and accuracy of the model prediction.

[0061] In the specific implementation process, the learning rate of the Adam optimizer can also be dynamically adjusted during the training process. The specific acquisition method is as follows: ; Among them, η t represents the learning rate at t iterations, η0 is the initial learning rate, set to 0.001, is a term that decays with iteration t, where α is the decay coefficient. As training progresses, this term will gradually decrease, causing the learning rate to gradually decrease. This ensures that the learning rate is large in the early stages of training and converges quickly; while the learning rate is small in the later stages of training to avoid missing the optimal solution.

[0062] is a term that dynamically adjusts the learning rate according to the rate of change of the loss function and the gradient norm. represents the change in the loss function at the tth iteration, is the L2 norm of the gradient of the tth iteration, and β is the adjustment coefficient, which is used to control the influence of the dynamic adjustment term.

[0063] Furthermore, to avoid overfitting of the model, an early stopping mechanism is introduced. During the training process, the data set is divided into a training set and a validation set. After each training cycle, the loss value of the model on the validation set is calculated. If the loss value on the validation set does not decrease for several consecutive cycles, the training is stopped and the model at this time is selected as the final trained model.

[0064] The preprocessed 3D image is input into the trained model for obtaining basic information of the designated area. The model will automatically identify and output the basic information of the designated area according to the input image, including the location of the designated area, the size of the designated area, the depth of the designated area, and the irregularity of the designated area. In order to enhance the interpretability of the model, the basic information acquisition model of the designated area of ​​the present embodiment also includes a set of visualization tools. The tool can superimpose the basic information of the designated area on the original 3D image, and intuitively display the location and characteristics of the designated area through different colors, transparencies, etc. For example, for the designated area of ​​the tooth, the caries area can be represented by red, and the healthy area can be represented by green. The doctor can intuitively observe the pathological condition of the tooth through the visualization tool, providing strong support for the treatment.

[0065] An image processing method for image data of the present embodiment trains a basic information acquisition model for a designated area to make the basic information acquisition model for the designated area more accurate and complete, thereby acquiring more accurate basic information of the designated area, thereby improving the accuracy and effectiveness of subsequent laser operations.

[0066] Example 4

[0067] The image processing method of image data in this embodiment is any one of the image processing methods of embodiment 1, embodiment 2 and embodiment 3, and will not be described in detail here.

[0068] The user in this embodiment uses an image processing method of the image data in this embodiment to process the image data. The specific steps are as follows: Use dermatoscope, ultrasound, MRI (magnetic resonance imaging), OCT (optical coherence tomography) and other imaging devices to obtain multimodal imaging data of the user's skin and related tissues to be tested. Use ultrasound, dental film machine, curved surface tomography X-ray machine, computer tomography, OCT (optical coherence tomography), oral scanner, oral endoscope and other imaging devices to obtain multimodal imaging data of the user's inside oral cavity.

[0069] This embodiment takes the image processing method of the intraoral image data as an example. After acquiring the intraoral multimodal image data of the user, the multimodal image data of the intraoral or skin of the user are aligned, and 3D Slicer (medical image layer fusion) is used to perform 3D reconstruction to obtain the user's 3D image data. Input the user's 3D image data into the trained model for obtaining basic information of the specified area to obtain the following basic information of the specified area: Designated area location: Gingiva in the left mandibular first molar area; Designated area size: Approximately 4mm 3 ; Depth of designated area: 2.5 mm from the gum surface to the deepest point; Designated area morphology: irregularity score is 0.7; Input the specified area size, specified area depth and specified area irregularity into the following formula to obtain the first score of the user-specified area status: ; Among them, ω s is the influence weight of the specified area size, set to 0.5, ω D is the influence weight of the specified area depth, set to 0.3, ω F represents the influence weight of the specified area morphology, which is set to 0.2, S is the size of the specified area, and S max The maximum reference value set for the specified area size is 10mm 3 , k s is the nonlinear intensity coefficient, set to 2, D is the depth of the specified area, D max is the maximum reference depth set, which is 5 mm, β D is the steepness of the saturation curve, F is the irregularity of the specified area, and Score is the first score of the specified area status.

[0070] According to the above formula, it can be concluded that the first score of the user's designated area status is 1.267; The first score of the specified area status is input into the following formula for normalization to obtain the specified area status score: ; Among them, Final Score is the status score of the specified area, ω s is the influence weight of the specified area size, ω D is the influence weight of the specified area depth, ω F Indicates the influence weight of the specified area morphology, k s is the nonlinear intensity coefficient, D max is the maximum reference depth, F max is the maximum irregularity set.

[0071] It can be concluded that the designated area status score is 26.22 points.

[0072] According to the status score of the specified area, substitute it into the following formula: ; Among them, T is the relevant parameter, T min is the minimum value of the relevant parameter, T max is the maximum value of the relevant parameters, and FinalScore is the status score of the specified area.

[0073] The relevant parameters can be obtained as follows: laser power: 1.42W; laser operation time: 11.56min; laser irradiation area: 3mm 2 .

[0074] The above-mentioned related parameters are used to guide the subsequent image processing process to ensure that the processing is both safe and effective.

[0075] Example 5

[0076] See also Figure 3 , an intelligent laser device of this embodiment, performs an image processing method of image data in any one of Embodiment 1, Embodiment 2, Embodiment 3 and Embodiment 4, and an intelligent laser device includes: Robotic arm, laser processing head, sensor assembly, laser generator, central processing assembly and display screen; The mechanical arm, the laser processing head, the sensor assembly, the laser generator, the central processing assembly and the display screen are electrically connected to the central processing assembly respectively; The robotic arm is connected to the laser processing head via a rotary joint with a micro motor.

[0077] In this embodiment, the robotic arm includes a control component and several electric gears. The electric gears are provided with a power source, and the laser processing heads are respectively connected to an electric gear. The control component is arranged on the electric gears, and the control component can accurately control the running direction and displacement of the laser processing head while controlling the rotation of the corresponding electric gears.

[0078] Furthermore, the robotic arm in this embodiment adopts a multi-degree-of-freedom design, combined with a precise sensor component, and can achieve micron-level control of the laser processing head.

[0079] At the same time, a replaceable optical fiber is integrated at the end of the laser processing head. The optical fiber has a micro-magnet attached near the tip. The magnetic field is driven by an external coil to achieve high-speed micro-swing of the optical fiber, thereby completing the two-dimensional scanning control of the laser beam. Based on this design architecture, the laser processing head only needs to be positioned to the working position through macroscopic movement, while the two-dimensional scanning and fine trajectory description are achieved by the magnetically controlled optical fiber mechanism. This electromechanical separation drive method not only ensures the system's large-range motion capability, but also ensures the dynamic response performance during precision scanning.

[0080] In the specific implementation process, the intelligent laser device also includes an alarm device. When the laser processing head exceeds the safe distance from the top of the designated area, the alarm device sounds an alarm. The safe distance is 5mm. An optical fiber transmission system is integrated between the laser processing head and the laser generator of this embodiment. The laser generator sends the laser to the laser processing head through the optical fiber transmission system to ensure efficient transmission and precise focusing of the laser energy. The laser processing head can achieve 360-degree rotation without dead angles through a micro motor connected by a rotating joint to meet the laser processing needs of different positions.

[0081] In addition, the laser processing head can perform reciprocating motion or circular motion around a designated area according to a set path.

[0082] The sensor components of this embodiment include: temperature sensors, optical sensors and other types of sensors; among them, the temperature sensor is installed on the laser processing head, and can monitor the temperature changes during the processing process in real time. Once the temperature exceeds the safe range, it will immediately feed back the signal to the central processing component, and the central processing component will adjust the power of the laser transmitter or suspend processing accordingly; the optical sensor is responsible for monitoring the propagation and reflection of the laser, and by analyzing the intensity, wavelength and other parameters of the reflected light, it determines whether the laser acts accurately on the target processing area and evaluates the processing effect.

[0083] The sensor component is connected to the central processing component through a special signal acquisition circuit, which can amplify, filter and process the weak signals collected by the sensor, and then transmit the processed signals accurately to the central processing component.

[0084] During the specific implementation process, the display screen uses an LCD screen and supports touch operation. Doctors can manually optimize and adjust relevant parameters through the touch screen.

[0085] In this embodiment, the intelligent laser device further includes: The storage device is used to store past user 3D images, basic information of the designated area, status scores of the designated area and related parameters, and the storage device is stored in an encrypted storage manner.

[0086] The storage device of this embodiment adopts advanced encryption technology to ensure the safe storage of user 3D images, basic information of designated areas, status scores of designated areas and related parameters. At the same time, it supports cloud backup function to prevent data loss and facilitate remote consultation and data sharing.

[0087] The intelligent laser device of this embodiment reduces the doctor's labor intensity, while increasing the accuracy of the operation, ensuring the security of user data, and improving the user's comfort.

[0088] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0089] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0090] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0091] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0092] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An image processing method for image data, characterized in that: include: S100, obtaining multimodal image data of a user at a first position; the first position is inside the oral cavity or the skin to be measured; S200, reconstructing the multimodal image data of the user's first position into a 3D image, and automatically identifying and acquiring basic information of a designated area according to the 3D image; The basic information of the designated area includes: the location of the designated area, the size of the designated area, the depth of the designated area, and the irregularity of the designated area; S300, inputting the designated area size, designated area depth and designated area irregularity in the basic information of the designated area into a scoring model constructed in advance, obtaining the designated area status score of the user, and obtaining relevant parameters according to the designated area status score of the user and a processing parameter setting rule set in advance; S400, calculating and generating a precise path of the robot arm according to the 3D image and the position of the designated area, the robot arm reaches the designated area along the precise path, and operating the laser processing head connected to the robot arm according to relevant parameters.

2. The image processing method of image data according to claim 1, characterized in that: In S200, automatically identifying and acquiring basic information of a designated area according to a 3D image includes: S210, using a training data set to train a designated area basic information acquisition model to obtain a trained designated area basic information acquisition model; S220, inputting the 3D image into a trained model for acquiring basic information of a designated area to acquire basic information of the designated area; The basic information acquisition model of the designated area is a convolutional neural network model; The training data set is past 3D image data and basic information of the corresponding designated area.

3. The image processing method of image data according to claim 2, characterized in that: The S210 specifically includes: The past 3D image data and the basic information of the corresponding designated area are input into the designated area basic information acquisition model. The cross entropy function and Adam optimizer are used with a learning rate of 0.001 until the cross entropy regression loss function converges to obtain the trained designated area basic information acquisition model.

4. The image processing method of image data according to claim 1, characterized in that: The S300 includes: S310, inputting the size of the designated area, the depth of the designated area, and the irregularity of the designated area in the basic information of the designated area into a scoring model constructed in advance, and obtaining a first score of the designated area state of the user-designated area; S320, normalizing the first score of the designated area status to obtain a designated area status score; S330: Obtain relevant parameters according to the designated area status score and parameter setting rules.

5. The image processing method of image data according to claim 4, characterized in that: The S310 includes: The size of the specified area, the depth of the specified area, and the irregularity of the specified area are input into the following formula to obtain the first score of the user-specified area status: ； Among them, ω s is the influence weight of the specified area size, ω D is the influence weight of the specified area depth, ω F represents the influence weight of the specified area morphology, S is the size of the specified area, S max The maximum reference value for the specified region size, k s is the nonlinear intensity coefficient, D is the depth of the specified area, and D max is the maximum reference depth, β D is the steepness of the saturation curve, F is the irregularity of the specified area, and Score is the first score of the specified area status.

6. The image processing method of image data according to claim 4, characterized in that: The S320 includes: The first score of the designated area status is input into the following formula for normalization processing to obtain the designated area status score: ； Among them, Final Score is the status score of the specified area, ω s is the influence weight of the specified area size, ω D is the influence weight of the specified area depth, ω F Indicates the influence weight of the specified area morphology, k s is the nonlinear intensity coefficient, D max is the maximum reference depth, F max is the maximum irregularity set.

7. The image processing method of image data according to claim 4, characterized in that: The S330 includes: The specified area status score is input into the following formula to obtain relevant parameters: ； Among them, T is the relevant parameter, T min is the minimum value of the relevant parameter, T max is the maximum value of the relevant parameters, and Final Score is the status score of the specified area; The relevant parameters include laser power, laser operation time and laser irradiation area.

8. The image processing method of image data according to claim 7, characterized in that: In the S330, The minimum value of the laser power is 0.5W and the maximum value is 36W; The minimum laser operation time is 1 second and the maximum is 30 minutes. The minimum laser irradiation area is 0.5mm 2 , the maximum value is 49cm 2 .

9. The image processing method of image data according to claim 1, characterized in that: The S400 specifically includes: S410, based on the designated area position and the 3D image, using a cubic spline interpolation algorithm to generate a continuous path of the robot arm in the three-dimensional space; S420, obtaining the precise path of the robot arm's operation according to the shortest path algorithm and the continuous path of the robot arm in three-dimensional space, the robot arm moves to a designated area along the precise path, and operates the laser processing head connected to the robot arm according to relevant parameters.

10. An intelligent laser device, characterized in that: The intelligent laser device is used to execute the image processing method of image data according to any one of claims 1 to 9, and the intelligent laser device comprises: Robotic arm, laser processing head, sensor assembly, laser generator, central processing assembly and display screen; The mechanical arm, the laser processing head, the sensor assembly, the laser generator, the central processing assembly and the display screen are electrically connected to the central processing assembly respectively; The robotic arm is connected to the laser processing head via a rotary joint with a micro motor.

Images