Accurate wound medicine spraying robot control method and system based on neural network

Abstract

The invention provides a precise wound medicine spraying robot control method and system based on a neural network. The method comprises the following steps: receiving operating room environment parameters and wound image information; determining the injection pressure of a robot nozzle according to the operating room environment parameters, and analyzing the wound image information by using a pre-trained neural network to obtain an image analysis result; a pre-trained neural network introduces a multi-scale analysis strategy for the wound area; according to an image analysis result, generating a three-dimensional motion trail of the robot nozzle and moving speeds and spraying angles of the robot nozzle at different positions in the three-dimensional motion trail; and according to the injection pressure of the robot nozzle, the three-dimensional motion track of the robot nozzle and the moving speed of the robot nozzle, a corresponding control instruction is generated, so that the robot nozzle performs corresponding operation according to the control instruction. According to the technical scheme, the accuracy of robot control is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of robot control technology, and in particular to a neural network-based precision wound spray robot control method and system. Background Art

[0002] With the continuous advancement of medical technology, spraying robots have been applied to the preparation stage of surgical operations, especially in wound cleaning and treatment. Traditional manual spraying methods have problems such as uneven spraying and difficulty in controlling drug dosage, which can easily lead to poor treatment effects. Therefore, it is urgent to develop a robot system that can automatically and accurately spray wounds.

[0003] At present, there are some robot systems for wound spraying that realize trajectory control of wound spraying robots during the spraying process based on image processing and path planning. However, spraying robots often rely on fixed parameter settings during the spraying process. Therefore, the robot nozzle moves forward at a constant speed and can spray evenly on flat wounds.

[0004] However, for complex wound areas, especially when there are bends, depth changes, etc. in the wound area, the accuracy of robot control cannot be achieved, which leads to uneven drug spraying. Summary of the invention

[0005] The embodiments of the present application provide a neural network-based precision wound spray robot control method and system to solve the problem of robot control accuracy in the prior art, thereby improving the uniformity of drug spraying.

[0006] In a first aspect, the present application provides a method for controlling a precision wound spraying robot based on a neural network, comprising:

[0007] Receive operating room environmental parameters and wound image information;

[0008] Determine the spray pressure of the robot nozzle according to the operating room environmental parameters, and use a pre-trained neural network to analyze the wound image information to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area;

[0009] Generating a three-dimensional motion trajectory of the robot nozzle, and a moving speed and a spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the image analysis result;

[0010] According to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle and the moving speed of the robot nozzle, a corresponding control instruction is generated so that the robot nozzle performs a corresponding operation according to the control instruction.

[0011] Optionally, the wound image information includes: visible light image information, infrared image information, ultrasound image information and depth image information;

[0012] The method of analyzing the wound image information using a pre-trained neural network to obtain an image analysis result includes:

[0013] Preprocessing the visible light image information, the infrared image information, the ultrasonic image information, and the depth image information respectively to obtain preprocessed visible light image information, the infrared image information, the ultrasonic image information, and the depth image information;

[0014] Segmenting the preprocessed visible light image information to determine the wound boundary in the visible light image, wherein the area within the wound boundary is the wound area, and the area outside the wound boundary is the background area;

[0015] Based on the pre-processed infrared image information, ultrasound image information and depth image information, the wound area is registered and fused to generate multi-modal fusion image information;

[0016] Based on the multimodal fusion image information, a multiscale analysis strategy was used to perform multiscale analysis on the wound area to obtain the image analysis results.

[0017] Optionally, the image analysis result is used to represent geometric features of a wound area in the wound image, and the geometric features include a degree of curvature;

[0018] The multi-scale analysis strategy is used to perform multi-scale analysis on the wound area based on the multi-modal fusion image information to obtain image analysis results, including:

[0019] Based on the multimodal fusion image information, multi-resolution pyramid technology is used to generate multimodal fusion images of different scales;

[0020] A variety of feature extraction algorithms are used to extract multiple features from the multimodal fusion images at each scale;

[0021] Fuse the same type of features at different scales to generate multi-scale feature vectors;

[0022] Optimizing the multi-scale feature vector to obtain an optimized feature vector;

[0023] Based on all optimized eigenvectors, the curvature of the wound area is determined.

[0024] Optionally, determining the curvature of the wound area according to all optimized eigenvectors includes:

[0025] According to all optimized feature vectors, a 3D reconstruction algorithm based on deep learning is used to perform 3D reconstruction to generate a 3D model of the wound area;

[0026] According to the three-dimensional model, a discrete geometry method is used to calculate the curvature value of each point in the wound area; the curvature value is used to reflect the degree of curvature of the corresponding point;

[0027] The degree of curvature of the wound area was determined based on the curvature values ​​of all points.

[0028] Optionally, the optimizing the multi-scale feature vector to obtain an optimized feature vector includes:

[0029] Use a preset dimensionality reduction method to perform dimensionality reduction processing on the multi-scale feature vector to obtain a reduced dimensionality feature vector;

[0030] Performing feature selection on the dimension-reduced feature vector by using a target feature selection algorithm to obtain a feature vector after feature selection;

[0031] The feature vector after feature selection is standardized to obtain a standardized feature vector, wherein the standardized feature vector is an optimized feature vector.

[0032] Optionally, generating a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, and a moving speed and a spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory, comprises:

[0033] According to the geometric characteristics of the wound area, a multi-gradient control strategy is used to divide the wound area into a plurality of grids with unequal spacing, wherein the number of target points in each grid is greater than a preset number, and the target point number is the number of points whose curvature value is greater than a preset curvature value;

[0034] Perform path planning based on all grids to obtain a path planning result; the path planning result is a three-dimensional motion trajectory of the robot nozzle;

[0035] According to the geometric characteristics of the wound area, the spray pressure of the robot nozzle and the nozzle diameter of the robot nozzle, the moving speed and spray angle of the robot nozzle at different positions in the three-dimensional motion trajectory are determined.

[0036] Optionally, after generating corresponding control instructions according to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, the method further includes:

[0037] Monitoring real-time feedback information during the process in which the robot nozzle performs corresponding operations according to the control instructions, wherein the real-time feedback information includes wound image change information;

[0038] The three-dimensional motion trajectory of the robot nozzle and / or the moving speed of the robot nozzle at different positions in the three-dimensional motion trajectory are adjusted through a preset feedback mechanism.

[0039] In a second aspect, the embodiment of the present application provides a neural network-based precision wound spray robot control system, comprising:

[0040] A receiving module, used for receiving operating room environment parameters and wound image information;

[0041] A determination analysis module is used to determine the spray pressure of the robot nozzle according to the operating room environment parameters, and analyze the wound image information using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area;

[0042] A first generating module is used to generate a three-dimensional motion trajectory of the robot nozzle, and a moving speed and a spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the image analysis result;

[0043] The second generating module is used to generate corresponding control instructions according to the injection pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions.

[0044] In a third aspect, an embodiment of the present application provides a computing device, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a neural network-based precision wound spray robot control method as described in any one of the first aspects.

[0045] In a fourth aspect, an embodiment of the present application provides a computer storage medium, characterized in that a computer program is stored therein, and when the computer program is executed by a computer, a neural network-based precision wound spray robot control method as described in any one of the first aspects is implemented.

[0046] In an embodiment of the present application, a method for controlling a precise wound spraying robot based on a neural network is provided, the method comprising: receiving operating room environmental parameters and wound image information; determining the spray pressure of the robot nozzle according to the operating room environmental parameters, and analyzing the wound image information using a pre-trained neural network to obtain an image analysis result; introducing a multi-scale analysis strategy for the wound area into the pre-trained neural network; generating a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, and the moving speed and spray angle of the robot nozzle at different positions in the three-dimensional motion trajectory; generating corresponding control instructions according to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions. This embodiment improves the analysis accuracy of the image analysis results and the refinement of the path planning by introducing a pre-trained neural network, especially a multi-scale analysis strategy for the wound area, thereby achieving a more precise and efficient wound spraying operation. Specifically, the multi-scale analysis strategy can better capture the multi-scale features of the wound area, ensuring that the robot nozzle can perform more refined spraying in complex areas. In addition, through automated path planning and parameter adjustment, the dependence on human intervention is reduced, and the reliability and efficiency of the robot system are improved. Among them, by providing adaptive scaling factors of different scales, the multi-resolution pyramid technology can better adapt to different types of image data and improve the accuracy and robustness of feature extraction and path planning. By calculating the curvature value of each point in the three-dimensional model of the wound to evaluate the curvature of the wound, it can help medical staff accurately identify complex areas of the wound and guide the robot to dynamically adjust the operating parameters to ensure that the drug evenly covers the wound surface.

[0047] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A flowchart of a neural network-based precision wound spraying robot control method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of the structure of a neural network-based precision wound spray robot control system provided in an embodiment of the present application;

[0051] Figure 3A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0053] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The sequence numbers of the operations, such as 11, 12, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0055] Figure 1 A flowchart of a neural network-based precision wound spraying robot control method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0056] S11, receiving operating room environment parameters and wound image information.

[0057] Among them, the precision wound spraying robot can be referred to as a robot, and the operating room environmental parameters can refer to various environmental factors that affect the robot's working performance during the surgical preparation stage, such as temperature, humidity, light intensity, etc. These parameters are crucial to ensure that the robot can perform tasks accurately. Wound image information can refer to image data of the patient's wound site obtained by a high-precision camera or other imaging device. This information is used for subsequent wound analysis.

[0058] S12. Determine the spray pressure of the robot nozzle according to the operating room environmental parameters, and use the pre-trained neural network to analyze the wound image information to obtain the image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area.

[0059] It should be understood that the robotic nozzle is part of the robot and can be referred to as the nozzle for short. The injection pressure refers to the pressure when the robotic nozzle injects the drug. Different wound conditions may require different injection pressures to ensure that the drug can effectively cover the wound surface without causing secondary damage. The multi-scale analysis strategy can refer to the method of using multiple scales (i.e., resolutions) for analysis when processing image data, which helps to capture wound characteristics at different levels and improve the diagnostic accuracy of the wound area.

[0060] This embodiment can ensure that the surface of the wound area is covered by determining the injection pressure of the robot nozzle. In addition, this embodiment can also determine the working state of the heating or cooling device of the robot nozzle according to the operating room environment parameters to ensure that the drug remains in a stable state during the injection process. This embodiment can form a working mode that adapts to the operating room environment through the injection pressure of the robot nozzle and the working state of the heating or cooling device of the robot nozzle.

[0061] S13. Generate a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, as well as a moving speed and a spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory.

[0062] Among them, the three-dimensional motion trajectory may refer to the movement path of the robot nozzle in space. In order to ensure that the drug can be sprayed evenly and accurately on the wound surface, it is necessary to calculate a suitable motion trajectory. The moving speed and the spraying angle may refer to the speed and direction of the robot nozzle moving along its predetermined trajectory. Reasonable speed and angle settings can ensure the safety and effectiveness of the drug spraying process. Optionally, this embodiment can also determine the amount of drug spraying at each position based on the image analysis results to ensure that the drug can cover the wound site while avoiding drug waste.

[0063] S14, generating corresponding control instructions according to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions.

[0064] The control instructions may be a series of specific operation commands calculated based on all the above parameters, and are used to guide the robot to perform the actual spraying action.

[0065] For example, in a modern surgical operating room, medical staff are preparing to use a precision wound spraying robot to assist in treating a patient's traumatic wound. First, the robot will collect the environmental parameters (such as temperature and humidity) in the current operating room through the sensors installed on it, and use a high-resolution camera to take pictures of the wound site. Next, the neural network in the robot system will conduct an in-depth analysis of these photos, not only identifying the specific location and shape of the wound, but also assessing the severity of the wound based on a multi-scale analysis strategy. Subsequently, based on the image analysis results provided by the neural network, the system will plan an optimized three-dimensional motion trajectory to ensure that the robot nozzle can approach the wound surface at a suitable angle and speed. In addition, the robot system can also adjust the spray pressure of the nozzle according to the environmental conditions in the operating room to adapt to different environmental changes. Finally, all this information will be integrated into a series of precise control instructions and sent to the robot for execution. Throughout the process, the robot can automatically adjust its operating parameters to ensure that the drug can cover the entire wound area safely, evenly and accurately.

[0066] In S11 to S14, this embodiment improves the analysis accuracy of image analysis results and the refinement of path planning by introducing a pre-trained neural network, especially a multi-scale analysis strategy for the wound area, thereby achieving a more accurate and efficient wound spraying operation. Specifically, the multi-scale analysis strategy can better capture the multi-scale characteristics of the wound area, ensuring that the robot nozzle can perform more precise spraying in complex areas. In addition, through automated path planning and parameter adjustment, it reduces dependence on human intervention and improves the reliability and efficiency of the robot system.

[0067] In some optional embodiments, the wound image information includes: visible light image information, infrared image information, ultrasound image information and depth image information. Accordingly, in S12, the wound image information is analyzed using a pre-trained neural network to obtain an image analysis result, including:

[0068] Step 121, preprocess the visible light image information, infrared image information, ultrasonic image information and depth image information respectively to obtain preprocessed visible light image information, infrared image information, ultrasonic image information and depth image information. Among them, the visible light image information can be a wound image taken under normal light conditions or under fill light conditions, which can intuitively display the appearance characteristics of the wound such as color and shape. The infrared image information can be a wound image obtained by infrared technology, which can reflect the temperature distribution of the wound and its surrounding tissues. Ultrasonic image information is an image generated by ultrasonic technology, which is mainly used to observe the damage of deep tissues of the wound, such as the state of internal structures such as muscles and bones. Depth image information is a wound image obtained by using a depth camera, which can provide spatial geometric information of the wound, such as the depth and volume of the wound. Preprocessing refers to a series of processing operations performed on various original images, the purpose of which is to improve the image quality and make it more suitable for subsequent analysis and processing. The preprocessing steps include denoising, contrast enhancement, etc.

[0069] Step 122, segment the preprocessed visible light image information to determine the wound boundary in the visible light image, the area within the wound boundary is the wound area, and the area outside the wound boundary is the background area. In this embodiment, the purpose of the segmentation process is to distinguish the wound area from the non-wound area in the visible light image.

[0070] Step 123: Based on the pre-processed infrared image information, ultrasound image information and depth image information, the wound area is registered and fused to generate multimodal fusion image information. Through image fusion technology, all the above image information is integrated into a multimodal fusion image, which not only contains the appearance characteristics of the wound, but also reflects its temperature distribution, deep tissue damage and spatial geometry information, which greatly enriches the medical staff's understanding of the wound condition.

[0071] Step 124: Based on the multimodal fusion image information, a multi-scale analysis strategy is used to perform a multi-scale analysis on the wound area to obtain an image analysis result.

[0072] Based on multimodal fusion images, multiscale analysis can capture the characteristics of wounds from different spatial scales. For example, the macroscale can help determine the overall boundary and area of ​​the wound, while the microscale can carefully observe the tiny structural changes inside the wound. This can provide medical staff with a more comprehensive basis for decision-making.

[0073] In this embodiment, the multimodal fusion image provides detailed information about the wound, and the multiscale analysis strategy further refines this information. The robot can generate a more accurate spraying trajectory based on this high-precision data to ensure that the drug only acts on the wound area and avoids covering healthy tissue.

[0074] It should be noted that the robot can dynamically adjust the injection pressure and movement speed according to the temperature distribution and tissue state in the multimodal fusion image. For example, for areas with higher temperatures, the injection pressure can be appropriately reduced to prevent tissue damage caused by overcooling; for deep wounds, the movement speed can be increased to ensure adequate coverage of the drug.

[0075] In summary, through multimodal fusion images and multi-scale analysis, medical staff can obtain comprehensive information about the wound and formulate personalized treatment plans. The robot can perform more precise operations based on these plans to improve the effect of uniform spraying.

[0076] In some optional embodiments, the image analysis result is used to represent the geometric features of the wound area in the wound image, and the geometric features include the degree of curvature; the geometric features may also include the shape of the wound boundary, the depth of each position in the wound area, etc. As a possible implementation, step 124, based on the multimodal fusion image information, adopts a multi-scale analysis strategy to perform multi-scale analysis on the wound area to obtain the image analysis result, including:

[0077] Step a1: Based on the multimodal fusion image information, use the multi-resolution pyramid technology to generate multimodal fusion images of different scales, for example, to generate images of multiple scales such as original resolution, 1 / 2 resolution, and 1 / 4 resolution.

[0078] In the multi-resolution pyramid technology, this embodiment can provide adaptive scaling factors of different scales to ensure that the multi-resolution pyramid technology can better adapt to different types of image data and improve the accuracy and robustness of feature extraction and path planning. Therefore, the multi-resolution pyramid technology that can provide adaptive scaling factors of different scales can adopt the following formula:

[0079]

[0080] in, represents the multimodal fusion image of the i-th scale, I represents the multimodal fusion image information, scale_factor i (C) represents the adaptive scaling factor of the i-th scale. i (C) can be obtained using the following formula:

[0081] scale_factor i (C) = exp(-α·(w 1 V i +w 2 K i +w 3 T i ));

[0082] Where α is a sensitivity parameter used to control the rate of change of the adaptive scaling factor. 1 、w 2 and w 3 Both are weight parameters, used to balance the influence of corresponding factors. i represents the image content complexity of the i-th scale, K i represents the average curvature value of the i-th scale, T i Represents the local texture complexity of the i-th scale.

[0083] Specifically, in, represents the image gradient amplitude at point p, Represents the total number of pixels of the i-th scale image. Where K(p) is the curvature value at point p. N is the total number of pixels in the i-th scale image. Where GLCM(p) represents the gray level co-occurrence matrix feature at point p, Represents the total number of pixels of the i-th scale image.

[0084] Step a2: Use multiple feature extraction algorithms to extract multiple features from the multimodal fusion image at each scale. For example, use an edge detection algorithm to extract edge features, use a texture analysis algorithm to extract texture features, and use a deep learning algorithm to extract high-level semantic features.

[0085] Exemplarily, step a2 may adopt the following formula:

[0086] Among them, F i It is a feature vector composed of multiple features extracted from the multimodal fusion image at the i-th scale. f represents the feature extraction algorithm, such as edge detection algorithm, texture analysis algorithm, deep learning algorithm, etc.

[0087] Step a3: fuse the same type of features at different scales to generate a multi-scale feature vector. For example, fuse edge features at different scales into a multi-scale edge feature vector.

[0088] Exemplarily, step a3 may adopt the following formula:

[0089] F multi-scale =concatenate(F 1 , F 2 , ..., F n );

[0090] Among them, F multi-scale is the multi-scale feature vector, F iIt is a feature vector composed of multiple features extracted from the multimodal fusion image of the i-th scale, and n is the number of scales.

[0091] Step a4: Optimize the multi-scale feature vector to obtain an optimized feature vector. For example, a dimensionality reduction technique such as principal component analysis is used to reduce the dimension of the feature vector and improve computational efficiency.

[0092] Exemplarily, step a4 may adopt the following formula:

[0093] F opt = PCA(F multi-scale , k);

[0094] Among them, F opt represents the optimized eigenvector, FCA represents the principal component analysis algorithm, and k is the number of principal components retained.

[0095] Step a5: Determine the curvature of the wound area based on all optimized feature vectors. For example, by analyzing multi-scale edge feature vectors, determine the curvature change of the wound area, and thus evaluate the curvature of the wound.

[0096] This embodiment captures the characteristics of the wound at different scales and can more carefully evaluate the geometric characteristics of the wound, such as the degree of curvature, boundary shape, and depth distribution. According to the geometric characteristics of the wound, the robot can dynamically adjust the spray pressure and speed to ensure that the drug can evenly cover the entire wound surface. Based on the multi-scale analysis results, the robot can generate a more accurate spray trajectory to ensure that the drug only acts on the wound area. In addition, the combination of multimodal fusion images and multi-scale analysis strategies reduces the uncertainty of human judgment, improves the standardization of operations, and reduces the risks caused by human judgment errors.

[0097] It should be noted that during the spraying process, the robot can monitor the changes in the wound in real time through multimodal sensors, dynamically adjust the spraying strategy, and ensure the timeliness and effectiveness of the spraying. The robot monitors the reaction of the wound in real time and dynamically adjusts the operating parameters to ensure the safety of the treatment process.

[0098] In the above embodiment, step a4, optimizing the multi-scale feature vector to obtain the optimized feature vector, includes:

[0099] Step a41, use a preset dimensionality reduction method to reduce the dimensionality of the multi-scale feature vector to obtain a reduced dimensionality feature vector. For example, the multi-scale feature vector is input into a principal component analysis model, the main components are calculated, the first N main components are retained, and a new reduced dimensionality feature vector is constructed, which can reduce the dimension of the feature vector, remove redundant information, and improve the speed and efficiency of subsequent processing. Wherein, N is an integer greater than 1. This embodiment does not specifically limit the specific value of N.

[0100] Step a42: perform feature selection on the reduced dimension feature vector by using a target feature selection algorithm to obtain a feature vector after feature selection. For example, recursive feature elimination is used as a feature selection algorithm to recursively remove the most irrelevant features according to the objective function (e.g., the prediction accuracy of the degree of wound curvature), and finally retain the most relevant set of features to further screen out the most valuable features for wound analysis, thereby improving the generalization ability and interpretability of the model.

[0101] Step a43: Standardize the feature vector after feature selection to obtain a standardized feature vector, which is an optimized feature vector. Standardization can ensure that each feature is on the same scale, avoid some features dominating the model training process due to excessive numerical range, and improve the stability and accuracy of the model.

[0102] Through three steps of dimensionality reduction, feature selection and standardization, the multi-scale feature vectors were optimized, significantly improving the accuracy of wound analysis and the operation accuracy of the spraying robot. This optimization process not only improves computational efficiency, but also enhances the performance and interpretability of the model, providing strong technical support for uniform spraying.

[0103] In the above embodiment, step a5, determining the curvature of the wound area according to all optimized feature vectors, includes:

[0104] Step a51, based on all optimized feature vectors, a three-dimensional reconstruction is performed using a three-dimensional reconstruction algorithm based on deep learning to generate a three-dimensional model of the wound area. A three-dimensional reconstruction algorithm may refer to the use of deep learning technology to restore the three-dimensional structure of an object from a two-dimensional image. This algorithm usually includes convolutional neural networks and generative adversarial networks, etc., which can extract features from multi-view or multi-modal images and generate high-precision three-dimensional models. The three-dimensional geometric structure generated by the three-dimensional reconstruction algorithm can intuitively display the spatial morphology of the wound. Specifically, the three-dimensional model corresponding to the generated wound area not only shows the surface shape of the wound, but also contains depth information, helping medical staff to fully understand the geometric characteristics of the wound.

[0105] Optionally, step a51 may use the following formula to generate a three-dimensional model of the wound area:

[0106] C=3D_Reconstruction(F opt );

[0107] Among them, C is the 3D model of the wound area, 3D_Reconstruction is the 3D reconstruction algorithm based on deep learning, and F opt represents the optimized feature vector.

[0108] Step a52: According to the three-dimensional model, the curvature value of each point in the wound area is calculated using a discrete geometry method; the curvature value is used to reflect the curvature degree of the corresponding point. In a three-dimensional model, a discrete geometry method can be used to calculate the geometric properties of each vertex or face, such as curvature, normal vector, etc. In this embodiment, the curvature value of each point in the wound area can be calculated using a discrete geometry method, and these curvature values ​​reflect the curvature degree of the wound surface.

[0109] Alternatively, the discrete geometry method can be formulated as follows:

[0110]

[0111] Where K(p) is the curvature value at point p, is the gradient at point p, is the Laplace operator at point p.

[0112] Step a53: determine the curvature of the wound area based on the curvature values ​​of all points.

[0113] Optionally, step a53 may adopt the following formula:

[0114]

[0115] Where B represents the curvature of the wound area, N is the number of points in the wound area, and K(p i ) is point p i γ is the smoothing coefficient, which is used to prevent numerical instability caused by excessive curvature values.

[0116] In the three-dimensional model of the wound area, the curvature value of each point can be used to evaluate the curvature of the point, helping medical staff to identify complex areas of the wound. By calculating the curvature values ​​of all points in the wound area, an overall curvature index can be obtained, which can be used to evaluate the complexity of the wound and formulate spraying strategies.

[0117] This embodiment can accurately evaluate the curvature of the wound by calculating the curvature value of each point, thereby helping medical staff to identify complex areas of the wound. In addition, according to the curvature of the wound, the robot can dynamically adjust the injection pressure, movement speed, injection angle, etc. to ensure that the drug can evenly cover the entire wound surface.

[0118] In some optional embodiments, S13, generating a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, and the moving speed and spray angle of the robot nozzle at different positions in the three-dimensional motion trajectory, includes:

[0119] Step 131, according to the geometric features of the wound area, the wound area is divided into a plurality of unequally spaced grids using a multi-gradient control strategy, wherein the number of target points in each grid is greater than a preset number, and the target point number is the number of points whose curvature value is greater than a preset curvature value. Different grids can be in different or different planes. A complex area is divided into a plurality of smaller sub-areas, each of which is called a grid. In this embodiment, gridding can help to more finely manage the features of each sub-area.

[0120] Exemplarily, the multi-gradient control strategy in step 131 may adopt the following formula:

[0121] G i =Grid_Division(C,threshold i , δ i );

[0122] Among them, G i is the i-th grid, C is the three-dimensional model of the wound area, threshold is the grid division threshold, such as the curvature threshold, δ i is the minimum size limit of the i-th grid.

[0123] Step 132: Perform path planning based on all grids to obtain a path planning result; the path planning result is a three-dimensional motion trajectory of the robot nozzle.

[0124] Step 133: Determine the moving speed and spray angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the geometric features of the wound area, the spray pressure of the robot nozzle, and the nozzle diameter of the robot nozzle.

[0125] Optionally, the geometric characteristics of the wound area can be characterized by the curvature K(p i ) and distance D(p i ) is used to embody the above. The formula for determining the moving speed of the robot nozzle at different positions in step 133 is as follows:

[0126]

[0127] Among them, v i Indicates the moving speed at the i-th position in the three-dimensional motion trajectory. max Indicates the maximum moving speed. The maximum moving speed is a preset constant used to limit the maximum speed of the nozzle. α indicates the curvature adjustment coefficient, which is used to adjust the effect of curvature on speed. β is the distance adjustment coefficient, which is used to adjust the effect of distance on speed. D(p i) is the distance from the i-th position to the wound edge. The closer the distance, the faster the nozzle moves to reduce unnecessary dwell time. γ′ represents the pressure adjustment coefficient, which is used to adjust the effect of injection pressure on speed. P is the injection pressure of the robot nozzle. The greater the pressure, the slower the nozzle moves to ensure that the drug can be sprayed more evenly.

[0128] Step 133 determines the formula for the spray angle of the robot nozzle at different positions as follows:

[0129]

[0130] Among them, θ i represents the injection angle at the i-th position, represents the partial derivative of the i-th position in the x direction, It represents the partial derivative of the i-th position point in the y direction, and ω is the weight coefficient, which is used to balance the influence of different directions. Represents the additional partial derivative of the i-th position in the x direction, for example, considering environmental effects. represents the additional partial derivative of the i-th position point in the y direction. δ represents the distance adjustment coefficient, which is used to adjust the effect of distance on angle. D(p i ) represents the distance from the i-th position to the wound edge. d is the nozzle diameter of the nozzle. The larger the diameter, the spray angle may be adjusted to ensure that the drug can cover a larger area.

[0131] The moving speed may refer to the moving speed of the robot nozzle at different positions in the three-dimensional motion trajectory. In this embodiment, the moving speed of the nozzle can be adjusted according to the geometric characteristics and injection requirements of the wound area. In areas with higher curvature values, the movement speed of the nozzle is slower to ensure that the medicine can be fully covered; in flat areas, the movement speed of the nozzle is faster to improve work efficiency. The injection angle may refer to the injection direction of the robot nozzle at different positions. In this embodiment, the injection angle of the nozzle can be adjusted according to the geometric characteristics and injection requirements of the wound area. In deep concave areas, the injection angle of the nozzle is tilted downward to ensure that the medicine can reach the bottom of the wound; in flat areas, the injection angle of the nozzle is perpendicular to the wound surface to ensure uniform coverage of the medicine.

[0132] This embodiment divides the wound area into multiple grids of unequal spacing through a multi-gradient control strategy, performs path planning based on all grids, generates a three-dimensional motion trajectory of the robot nozzle, and determines the movement speed and injection angle of the nozzle based on the geometric characteristics of the wound area, the injection pressure and the nozzle diameter. This process significantly improves the accuracy of spraying and the operating efficiency.

[0133] In some optional embodiments, after S14, generating corresponding control instructions according to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, the method further includes:

[0134] Step 15: monitor the real-time feedback information of the robot nozzle during the corresponding operation according to the control instruction, and the real-time feedback information includes wound image change information.

[0135] Step 16: adjusting the three-dimensional motion trajectory of the robot nozzle and / or the moving speed of the robot nozzle at different positions in the three-dimensional motion trajectory through a preset feedback mechanism.

[0136] According to the changes in the wound image in the real-time feedback information, the preset feedback mechanism can adjust the three-dimensional motion trajectory, movement speed or injection pressure of the robot nozzle to optimize the uniformity of the spraying effect. If the real-time feedback information shows that the drug coverage of a certain area is insufficient, the three-dimensional motion trajectory can be adjusted to make the nozzle cover the area again. If the real-time feedback information shows that the drug in a certain area is sprayed too quickly, the movement speed of the nozzle in that area can be slowed down to ensure uniform coverage of the drug.

[0137] This embodiment monitors the real-time feedback information during the operation of the robot nozzle in real time, and dynamically adjusts the three-dimensional motion trajectory and movement speed of the nozzle through a preset feedback mechanism. This process significantly improves the spraying accuracy and operation efficiency.

[0138] In some optional embodiments, the present embodiment may also set a stop mechanism corresponding to the robot, and stop the spraying operation when the stop mechanism is satisfied. The present application embodiment provides comprehensiveness of robot control and operational stability.

[0139] Figure 2 A schematic diagram of a neural network-based precise wound spraying robot control system provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the system includes:

[0140] The receiving module 21 is used to receive operating room environment parameters and wound image information.

[0141] The determination analysis module 22 is used to determine the spray pressure of the robot nozzle according to the operating room environmental parameters, and use the pre-trained neural network to analyze the wound image information to obtain the image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area.

[0142] The first generating module 23 is used to generate the three-dimensional motion trajectory of the robot nozzle, and the moving speed and spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the image analysis result.

[0143] The second generating module 24 is used to generate corresponding control instructions according to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions.

[0144] Figure 2 The neural network-based precise wound spray robot control system can execute Figure 1 The implementation principle and technical effect of the neural network-based precision wound spraying robot control method described in the illustrated embodiment will not be repeated. The specific way in which each module and unit performs operations in the neural network-based precision wound spraying robot control system in the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated here.

[0145] In one possible design, Figure 2 The neural network-based precise wound spray robot control system of the illustrated embodiment can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32 .

[0146] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0147] The processing component 32 is used to: receive operating room environmental parameters and wound image information; determine the injection pressure of the robot nozzle according to the operating room environmental parameters, and use a pre-trained neural network to analyze the wound image information to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area; generate a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, as well as the movement speed and injection angle of the robot nozzle at different positions in the three-dimensional motion trajectory; generate corresponding control instructions according to the injection pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle and the movement speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions.

[0148] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0149] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as random access memory (RAM), static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0150] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0151] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above-mentioned peripheral interface module can be an output device, an input device, etc.

[0152] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0153] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0154] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a neural network-based precise wound spray robot control method.

[0155] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0156] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A neural network-based precise wound spraying robot control method, characterized in that: include: Receive operating room environmental parameters and wound image information; Determine the spray pressure of the robot nozzle according to the operating room environmental parameters, and use a pre-trained neural network to analyze the wound image information to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area; Generating a three-dimensional motion trajectory of the robot nozzle, and a moving speed and a spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the image analysis result; According to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle and the moving speed of the robot nozzle, a corresponding control instruction is generated so that the robot nozzle performs a corresponding operation according to the control instruction.

2. The method according to claim 1, characterized in that The wound image information includes: visible light image information, infrared image information, ultrasonic image information and depth image information; The method of analyzing the wound image information using a pre-trained neural network to obtain an image analysis result includes: Preprocessing the visible light image information, the infrared image information, the ultrasonic image information, and the depth image information respectively to obtain preprocessed visible light image information, the infrared image information, the ultrasonic image information, and the depth image information; Segmenting the preprocessed visible light image information to determine the wound boundary in the visible light image, wherein the area within the wound boundary is the wound area, and the area outside the wound boundary is the background area; Based on the pre-processed infrared image information, ultrasound image information and depth image information, the wound area is registered and fused to generate multi-modal fusion image information; Based on the multimodal fusion image information, a multiscale analysis strategy was used to perform multiscale analysis on the wound area to obtain the image analysis results.

3. The method according to claim 2, characterized in that The image analysis result is used to represent the geometric features of the wound area in the wound image, and the geometric features include the degree of curvature; The multi-scale analysis strategy is used to perform multi-scale analysis on the wound area based on the multi-modal fusion image information to obtain image analysis results, including: Based on the multimodal fusion image information, multi-resolution pyramid technology is used to generate multimodal fusion images of different scales; A variety of feature extraction algorithms are used to extract multiple features from the multimodal fusion images at each scale; Fuse the same type of features at different scales to generate multi-scale feature vectors; Optimizing the multi-scale feature vector to obtain an optimized feature vector; Based on all optimized eigenvectors, the curvature of the wound area is determined.

4. The method according to claim 3, characterized in that Determining the curvature of the wound area according to all optimized eigenvectors includes: According to all optimized feature vectors, a 3D reconstruction algorithm based on deep learning is used to perform 3D reconstruction to generate a 3D model of the wound area; According to the three-dimensional model, a discrete geometry method is used to calculate the curvature value of each point in the wound area; the curvature value is used to reflect the degree of curvature of the corresponding point; The degree of curvature of the wound area was determined based on the curvature values ​​of all points.

5. The method according to claim 3, characterized in that: The step of optimizing the multi-scale feature vector to obtain an optimized feature vector includes: Use a preset dimensionality reduction method to perform dimensionality reduction processing on the multi-scale feature vector to obtain a reduced dimensionality feature vector; Performing feature selection on the dimension-reduced feature vector by using a target feature selection algorithm to obtain a feature vector after feature selection; The feature vector after feature selection is standardized to obtain a standardized feature vector, wherein the standardized feature vector is an optimized feature vector.

6. The method according to claim 3, characterized in that The generating of the three-dimensional motion trajectory of the robot nozzle according to the image analysis result, and the moving speed and spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory, comprises: According to the geometric characteristics of the wound area, a multi-gradient control strategy is used to divide the wound area into a plurality of grids with unequal spacing, wherein the number of target points in each grid is greater than a preset number, and the target point number is the number of points whose curvature value is greater than a preset curvature value; Perform path planning based on all grids to obtain a path planning result; the path planning result is a three-dimensional motion trajectory of the robot nozzle; According to the geometric characteristics of the wound area, the spray pressure of the robot nozzle and the nozzle diameter of the robot nozzle, the moving speed and spray angle of the robot nozzle at different positions in the three-dimensional motion trajectory are determined.

7. The method according to claim 1, characterized in that After generating corresponding control instructions according to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, the method further includes: Monitoring real-time feedback information during the process in which the robot nozzle performs corresponding operations according to the control instructions, wherein the real-time feedback information includes wound image change information; The three-dimensional motion trajectory of the robot nozzle and / or the moving speed of the robot nozzle at different positions in the three-dimensional motion trajectory are adjusted through a preset feedback mechanism.

8. A neural network-based precise wound spraying robot control system, characterized in that: include: A receiving module, used for receiving operating room environment parameters and wound image information; A determination analysis module is used to determine the spray pressure of the robot nozzle according to the operating room environment parameters, and analyze the wound image information using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area; A first generating module is used to generate a three-dimensional motion trajectory of the robot nozzle, and a moving speed and a spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the image analysis result; The second generating module is used to generate corresponding control instructions according to the injection pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a neural network-based precision wound spray robot control method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a neural network-based precision wound spray robot control method as described in any one of claims 1 to 7 is implemented.

Images