Precision Wound Spraying Robot Control Method and System Based on Neural Network

Abstract

The present application provides a control method and system for a precise wound spraying robot based on a neural network. Among them, the method includes: receiving operating room environment parameters and wound image information; determining the spraying pressure of the robot nozzle according to the operating room environment parameters, and analyzing the wound image information by using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area; generating a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, as well as the moving speed and spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory; generating corresponding control instructions according to the spraying pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions. The technical solution provided by the present application improves the accuracy of robot control.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of robot control, and in particular, to a precise wound spraying robot control method and system based on a neural network. Background Art

[0002] With the continuous progress of medical technology, spraying robots have been applied to the preparation stage of surgical operations, especially in wound cleaning. Traditional manual spraying methods have problems such as uneven spraying and difficulty in controlling the drug dosage, which easily lead to poor treatment effects. Therefore, it has become an urgent need to develop a robot system that can automatically and precisely spray wounds.

[0003] Currently, some robot systems for wound spraying achieve trajectory control of the wound spraying robot during the spraying process based on image processing and path planning. However, the spraying robot often relies on fixed parameter settings during the spraying process. Therefore, the robot nozzle moves forward at a constant speed and can spray evenly for planar wounds.

[0004] However, for complex wound areas, especially when there are curvatures and depth changes in the wound area, the accuracy of robot control cannot be achieved, resulting in uneven drug spraying. Summary of the Invention

[0005] The embodiments of the present application provide a precise wound spraying robot control method and system based on a neural network to solve the problem of the accuracy of robot control in the prior art, thereby improving the uniformity of drug spraying.

[0006] In a first aspect, the embodiments of the present application provide a precise wound spraying robot control method based on a neural network, including:

[0007] Receiving operating room environment parameters and wound image information;

[0008] Determining the spraying pressure of the robot nozzle according to the operating room environment parameters, and analyzing the wound image information by using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area;

[0009] Generating a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, and the moving speed and spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory;

[0010] Generating corresponding control instructions according to the spraying pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions.

[0011] Optionally, the wound image information includes: visible light image information, infrared image information, ultrasonic image information, and depth image information;

[0012] Analyzing the wound image information by using a pre-trained neural network to obtain an image analysis result, including:

[0013] Performing preprocessing on the visible light image information, infrared image information, ultrasonic image information, and depth image information respectively to obtain preprocessed visible light image information, infrared image information, ultrasonic image information, and depth image information;

[0014] Performing segmentation processing on the preprocessed visible light image information to determine the wound boundary in the visible light image, the area within the wound boundary is the wound area, and the area outside the wound boundary is the background area;

[0015] Based on the preprocessed infrared image information, ultrasonic image information, and depth image information, registering and fusing the wound area to generate multi-modal fusion image information;

[0016] Based on the multi-modal fusion image information, performing multi-scale analysis on the wound area by using a multi-scale analysis strategy to obtain an image analysis result.

[0017] Optionally, the image analysis result is used to represent the geometric features of the wound area in the wound image, and the geometric features include the degree of curvature;

[0018] The performing multi-scale analysis on the wound area by using a multi-scale analysis strategy based on the multi-modal fusion image information to obtain an image analysis result, including:

[0019] Based on the multi-modal fusion image information, using a multi-resolution pyramid technique to generate multi-modal fusion images of different scales;

[0020] Using a variety of feature extraction algorithms to extract a variety of features from the multi-modal fusion images of each scale;

[0021] Fusing the same type of features at different scales to generate a multi-scale feature vector;

[0022] Performing optimization processing on the multi-scale feature vector to obtain an optimized feature vector;

[0023] Determining the degree of curvature of the wound area according to all the optimized feature vectors.

[0024] Optionally, the determining the degree of curvature of the wound area according to all the optimized feature vectors includes:

[0025] According to all the optimized feature vectors, performing three-dimensional reconstruction by using a three-dimensional reconstruction algorithm based on deep learning to generate a three-dimensional model of the wound area;

[0026] According to the three-dimensional model, a discrete geometry method is used to calculate the curvature value of each point in the wound area; the curvature value is used to reflect the degree of curvature of the corresponding point;

[0027] The degree of curvature of the wound area was determined based on the curvature values of all points.

[0028] Optionally, the optimizing the multi-scale feature vector to obtain an optimized feature vector includes:

[0029] Use a preset dimensionality reduction method to perform dimensionality reduction processing on the multi-scale feature vector to obtain a reduced dimensionality feature vector;

[0030] Performing feature selection on the dimension-reduced feature vector using a target feature selection algorithm to obtain a feature vector after feature selection;

[0031] The feature vector after feature selection is standardized to obtain a standardized feature vector, which is an optimized feature vector.

[0032] Optionally, generating a three-dimensional motion trajectory of the robot nozzle, and movement speeds and spray angles of the robot nozzle at different positions in the three-dimensional motion trajectory according to the image analysis result, includes:

[0033] According to the geometric characteristics of the wound area, a multi-gradient control strategy is used to divide the wound area into multiple grids with unequal spacing, wherein the number of target points in each grid is greater than a preset number, and the target point number is the number of points with a curvature value greater than the preset curvature value;

[0034] Perform path planning based on all grids to obtain a path planning result; the path planning result is a three-dimensional motion trajectory of the robot nozzle;

[0035] According to the geometric characteristics of the wound area, the spray pressure of the robot nozzle and the nozzle diameter of the robot nozzle, the moving speed and spray angle of the robot nozzle at different positions in the three-dimensional motion trajectory are determined.

[0036] Optionally, after generating corresponding control instructions according to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, the method further includes:

[0037] monitoring real-time feedback information during the process of the robot nozzle performing corresponding operations according to the control instructions, wherein the real-time feedback information includes wound image change information;

[0038] The three-dimensional motion trajectory of the robot nozzle and / or the moving speed of the robot nozzle at different positions in the three-dimensional motion trajectory are adjusted through a preset feedback mechanism.

[0039] In a second aspect, the present application provides a neural network-based precision wound spraying robot control system, comprising:

[0040] A receiving module, used for receiving operating room environmental parameters and wound image information;

[0041] a determination and analysis module, configured to determine the spray pressure of the robot nozzle according to the operating room environmental parameters, and analyze the wound image information using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area;

[0042] A first generating module is used to generate a three-dimensional motion trajectory of the robot nozzle, as well as a moving speed and a spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the image analysis result;

[0043] The second generating module is used to generate corresponding control instructions according to the injection pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions.

[0044] In a third aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a neural network-based precision wound spray robot control method as described in any one of the first aspects.

[0045] In a fourth aspect, an embodiment of the present application provides a computer storage medium, characterized in that a computer program is stored therein, and when the computer program is executed by a computer, a neural network-based precision wound spray robot control method as described in any one of the first aspects is implemented.

[0046] In the embodiments of the present application, a control method for a precise wound spraying robot based on a neural network is provided. The method includes: receiving operating room environment parameters and wound image information; determining the spraying pressure of the robot nozzle according to the operating room environment parameters, and analyzing the wound image information using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area; generating a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, as well as the moving speed and spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory; generating corresponding control instructions according to the spraying pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions. In this embodiment, by introducing a pre-trained neural network, especially a multi-scale analysis strategy for the wound area, the analysis accuracy of the image analysis result and the refinement degree of path planning are improved, thereby realizing a more precise and efficient wound spraying operation. Specifically, the multi-scale analysis strategy can better capture the multi-scale features of the wound area, ensuring that the robot nozzle can perform more precise spraying in complex areas. In addition, through automated path planning and parameter adjustment, the dependence on manual intervention is reduced, and the reliability and efficiency of the robot system are improved. Among them, through the provision of adaptive scaling factors of different scales, the multi-resolution pyramid technology can better adapt to different types of image data, improving the accuracy and robustness of feature extraction and path planning. By calculating the curvature value of each point in the three-dimensional wound model to evaluate the curvature of the wound, it can help medical staff accurately identify the complex areas of the wound and guide the robot to dynamically adjust the operation parameters to ensure that the drug evenly covers the wound surface.

[0047] These aspects or other aspects of the present application will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is a flowchart of a control method for a precise wound spraying robot based on a neural network provided by an embodiment of the present application;

[0050] Figure 2 It is a schematic structural diagram of a control system for a precise wound spraying robot based on a neural network provided by an embodiment of the present application;

[0051] Figure 3A structural schematic diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

[0053] In some processes described in the specification, claims and above-mentioned drawings of the present application, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear herein or may be executed in parallel. The serial numbers of the operations, such as 11, 12, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are of different types.

[0054] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0055] Figure 1 A flowchart of a precise wound spraying robot control method based on a neural network provided by an embodiment of the present application, as Figure 1 shown, the method includes:

[0056] S11. Receive the operating room environment parameters and wound image information.

[0057] Among them, the precise wound spraying robot can be simply referred to as the robot. The operating room environment parameters can refer to various environmental factors that affect the working performance of the robot during the surgical preparation stage, such as temperature, humidity, light intensity, etc. These parameters are crucial for ensuring that the robot can accurately execute tasks. The wound image information can refer to the image data of the patient's wound area obtained by a high-precision camera or other imaging devices. These information are used for subsequent wound analysis.

[0058] S12. Determine the spraying pressure of the robot nozzle according to the operating room environment parameters, and analyze the wound image information by using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area.

[0059] It should be understood that the robotic nozzle is part of the robot and can be simply referred to as the nozzle. Injection pressure refers to the pressure applied by the robotic nozzle when spraying medication. Different wound conditions may require different injection pressures to ensure that the medication effectively covers the wound surface without causing secondary damage. Multi-scale analysis strategies can refer to methods that use multiple scales (i.e., resolutions) for analysis when processing image data. This helps capture wound characteristics at different levels and improves the diagnostic accuracy of the wound area.

[0060] This embodiment ensures coverage of the wound area by determining the spray pressure of the robotic nozzle. Furthermore, this embodiment can determine the operating state of the robotic nozzle's heating or cooling device based on operating room environmental parameters to ensure that the drug remains stable during spraying. This embodiment can establish an operating mode tailored to the operating room environment by adjusting the spray pressure of the robotic nozzle and the operating state of the robotic nozzle's heating or cooling device.

[0061] S13. Generate a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, as well as the movement speed and spray angle of the robot nozzle at different positions in the three-dimensional motion trajectory.

[0062] The three-dimensional motion trajectory may refer to the movement path of the robotic nozzle in space. In order to ensure that the drug can be sprayed evenly and accurately on the wound surface, a suitable motion trajectory needs to be calculated. The movement speed and spray angle may refer to the speed and direction of the robotic nozzle moving along its predetermined trajectory. Reasonable speed and angle settings can ensure the safety and effectiveness of the drug spraying process. Optionally, this embodiment can also determine the amount of drug sprayed at each position based on the image analysis results to ensure that the drug can cover the wound area while avoiding drug waste.

[0063] S14 , generating corresponding control instructions according to the spray pressure of the robot nozzle, the three-dimensional motion trajectory of the robot nozzle, and the moving speed of the robot nozzle, so that the robot nozzle performs corresponding operations according to the control instructions.

[0064] The control instructions may be a series of specific operation commands calculated based on all the above parameters, and used to guide the robot to perform the actual spraying action.

[0065] Exemplarily, in a modern operating room, medical staff are preparing to use a precise wound spraying robot to assist in treating a patient's traumatic wound. First, the robot collects the environmental parameters (such as temperature and humidity) in the current operating room through sensors installed on it, and at the same time takes photos of the wound area using a high-resolution camera. Next, the neural network in the robot system will perform in-depth analysis on these photos, not only identifying the specific location and shape of the wound, but also evaluating the severity of the wound according to the multi-scale analysis strategy. Subsequently, based on the image analysis results provided by the neural network, the system will plan an optimal three-dimensional motion trajectory to ensure that the robot nozzle can approach the wound surface at an appropriate angle and speed. In addition, the robot system can also adjust the spraying pressure of the nozzle according to the environmental conditions in the operating room to adapt to different environmental changes. Finally, all this information will be integrated into a series of precise control instructions and sent to the robot for execution. Throughout the process, the robot can automatically adjust its operating parameters to ensure that the drug can be safely, evenly, and accurately covered over the entire wound area.

[0066] In S11 - S14, in this embodiment, by introducing a pre-trained neural network, especially the multi-scale analysis strategy for the wound area, the analysis accuracy of the image analysis results and the refinement degree of path planning are improved, thus realizing a more precise and efficient wound spraying operation. Specifically, the multi-scale analysis strategy can better capture the multi-scale features of the wound area, ensuring that the robot nozzle can perform more precise spraying in complex areas. In addition, through automated path planning and parameter adjustment, the dependence on human intervention is reduced, and the reliability and efficiency of the robot system are improved.

[0067] In some alternative embodiments, the wound image information includes: visible light image information, infrared image information, ultrasonic image information, and depth image information. Correspondingly, in S12, the pre-trained neural network is used to analyze the wound image information to obtain image analysis results, including:

[0068] Step 121: Preprocess the visible light image information, infrared image information, ultrasonic image information, and depth image information respectively to obtain the preprocessed visible light image information, infrared image information, ultrasonic image information, and depth image information. Among them, the visible light image information can be a wound image taken under normal light conditions or under supplementary light conditions, which can intuitively display the appearance characteristics such as the color and shape of the wound. The infrared image information can be a wound image obtained by using infrared technology, which can reflect the temperature distribution of the wound and its surrounding tissues. The ultrasonic image information is an image generated by ultrasonic technology, mainly used to observe the damage of deep tissues of the wound, such as the state of internal structures such as muscles and bones. The depth image information is a wound image obtained by using a depth camera, which can provide the spatial geometric information of the wound, such as the depth and volume of the wound. Preprocessing refers to a series of processing operations performed on various original images, aiming to improve the image quality and make it more suitable for subsequent analysis and processing. The preprocessing steps include denoising, contrast enhancement, etc.

[0069] Step 122: Perform segmentation processing on the preprocessed visible light image information to determine the wound boundary in the visible light image. The area within the wound boundary is the wound area, and the area outside the wound boundary is the background area. In this embodiment, the purpose of the segmentation processing is to distinguish the wound area from the non-wound area in the visible light image.

[0070] Step 123: Based on the preprocessed infrared image information, ultrasonic image information, and depth image information, perform registration and fusion on the wound area to generate multi-modal fusion image information. Through image fusion technology, all the above image information is integrated into a multi-modal fusion image. This multi-modal fusion image not only contains the appearance characteristics of the wound, but also reflects its temperature distribution, deep tissue damage, and spatial geometric information, greatly enriching the understanding of the wound condition by medical staff.

[0071] Step 124: Based on the multi-modal fusion image information, adopt a multi-scale analysis strategy to perform multi-scale analysis on the wound area to obtain an image analysis result.

[0072] Based on the multi-modal fusion image, multi-scale analysis can capture the characteristics of the wound at different spatial scales. For example, the macroscopic scale can help determine the overall boundary and area of the wound, while the microscopic scale can carefully observe the microscopic structural changes inside the wound. This can provide more comprehensive decision-making basis for medical staff.

[0073] In this embodiment, the multi-modal fusion image provides detailed information about the wound, and the multi-scale analysis strategy further refines this information. The robot can generate a more accurate spraying trajectory based on these high-precision data, ensuring that the drug only acts on the wound area and avoiding covering healthy tissues.

[0074] It should be noted that, based on the temperature distribution and tissue state in the multimodal fusion image, the robot can dynamically adjust the injection pressure and moving speed. For example, for areas with higher temperatures, the injection pressure can be appropriately reduced to prevent tissue damage caused by overcooling; for wounds with greater depths, the moving speed can be increased to ensure that the drug fully covers the wound.

[0075] In summary, through multimodal fusion images and multiscale analysis, medical staff can obtain comprehensive information about the wound, thereby formulating personalized treatment plans. The robot can perform more precise operations according to these plans, improving the effect of uniform injection.

[0076] In some alternative embodiments, the image analysis result is used to represent the geometric features of the wound area in the wound image, and the geometric features include the degree of curvature; the geometric features may also include the shape of the wound boundary, the depth of each position within the wound area, etc. As a possible implementation, in step 124, based on the multimodal fusion image information, a multiscale analysis strategy is adopted to perform multiscale analysis on the wound area, and the image analysis result is obtained, including:

[0077] Step a1: Based on the multimodal fusion image information, use the multiresolution pyramid technique to generate multimodal fusion images of different scales. For example, images of multiple scales such as the original resolution, 1 / 2 resolution, 1 / 4 resolution, etc. are generated.

[0078] In the multiresolution pyramid technique, in this embodiment, different scales of adaptive scaling factors can be provided to ensure that the multiresolution pyramid technique can better adapt to different types of image data, improving the accuracy and robustness of feature extraction and path planning. Therefore, the multiresolution pyramid technique that can provide different scales of adaptive scaling factors can adopt the following formula:

[0079]

[0080] where represents the multimodal fusion image of the i-th scale, I represents the multimodal fusion image information, and scale_factor i (C) represents the adaptive scaling factor of the i-th scale. Among them, scale_factor i (C) can be obtained using the following formula:

[0081] scale_factor i (C) = exp(-α·(w1V i +w2K i +w3T i ));

[0082] Where α represents the sensitivity parameter, which is used to control the rate of change of the adaptive scaling factor. w1, w2, and w3 are weight parameters, which are used to balance the influence of the corresponding factors. i represents the image content complexity of the i-th scale, K i represents the average curvature value of the i-th scale, T i represents the local texture complexity of the i-th scale.

[0083] Specifically, in, represents the image gradient amplitude at point p, Represents the total number of pixels in the i-th scale image. Where K(p) is the curvature value at point p. N is the total number of pixels in the i-th scale image. Where GLCM(p) represents the gray level co-occurrence matrix feature at point p, Represents the total number of pixels in the i-th scale image.

[0084] Step a2: Use multiple feature extraction algorithms to extract multiple features from the multimodal fusion image at each scale. For example, edge detection algorithms can be used to extract edge features, texture analysis algorithms can be used to extract texture features, and deep learning algorithms can be used to extract high-level semantic features.

[0085] Exemplarily, step a2 may adopt the following formula:

[0086] Among them, F i is a feature vector composed of multiple features extracted from the multimodal fusion image at scale i. f represents the feature extraction algorithm, such as edge detection algorithm, texture analysis algorithm, deep learning algorithm, etc.

[0087] Step a3: Fuse the same type of features at different scales to generate a multi-scale feature vector. For example, edge features at different scales are fused into a multi-scale edge feature vector.

[0088] Exemplarily, step a3 may use the following formula:

[0089] F multi-scale =concatenate(F1,F2,...,F n );

[0090] Among them, F multi-scale is the multi-scale feature vector, F i It is a feature vector composed of multiple features extracted from the multimodal fusion image of the i-th scale, and n is the number of scales.

[0091] Step a4: Optimize the multi-scale feature vector to obtain an optimized feature vector. For example, a dimensionality reduction technique such as principal component analysis is used to reduce the dimension of the feature vector and improve computational efficiency.

[0092] Exemplarily, step a4 may adopt the following formula:

[0093] F opt =PCA(F multi-scale , k);

[0094] Among them, F opt represents the optimized eigenvector, FCA represents the principal component analysis algorithm, and k is the number of principal components retained.

[0095] Step a5: Determine the curvature of the wound area based on all optimized eigenvectors. For example, by analyzing multi-scale edge eigenvectors, determine the curvature change of the wound area, and thus evaluate the curvature of the wound.

[0096] This embodiment captures wound features at different scales, enabling a more detailed assessment of the wound's geometric characteristics, such as curvature, boundary shape, and depth distribution. Based on the wound's geometric characteristics, the robot can dynamically adjust the spray pressure and speed to ensure that the drug evenly covers the entire wound surface. Based on the multi-scale analysis results, the robot can generate a more precise spray trajectory to ensure that the drug only acts on the wound area. Furthermore, the combination of multimodal fusion images and multi-scale analysis strategies reduces the uncertainty of human judgment, improves the standardization of operations, and reduces the risks caused by human misjudgment.

[0097] It should be noted that during the spraying process, the robot uses multimodal sensors to monitor wound changes in real time and dynamically adjust the spraying strategy to ensure timely and effective spraying. By monitoring the wound's response in real time and dynamically adjusting operating parameters, the robot ensures the safety of the treatment process.

[0098] In the above embodiment, step a4, optimizing the multi-scale feature vector to obtain the optimized feature vector, includes:

[0099] Step a41: Reduce the dimensionality of the multi-scale feature vector using a preset dimensionality reduction method to obtain a reduced-dimensionality feature vector. For example, the multi-scale feature vector is input into a principal component analysis model, the principal components are calculated, the first N principal components are retained, and a new reduced-dimensionality feature vector is constructed. This can reduce the dimensionality of the feature vector, remove redundant information, and improve the speed and efficiency of subsequent processing. Wherein, N is an integer greater than 1. The specific value of N is not specifically limited in this embodiment.

[0100] Step a42: Perform feature selection on the dimensionality-reduced feature vectors through a target feature selection algorithm to obtain the feature vectors after feature selection. For example, use recursive feature elimination as the feature selection algorithm, and recursively remove the least relevant features according to the objective function (e.g., the prediction accuracy of wound curvature), and finally retain the most relevant set of features to further screen out the features most valuable for wound analysis, improving the generalization ability and interpretability of the model.

[0101] Step a43: Standardize the feature vectors after feature selection to obtain the standardized feature vectors, and the standardized feature vectors are the optimized feature vectors. Standardization can ensure that each feature is on the same scale, avoid some features dominating the model training process due to too large a numerical range, and improve the stability and accuracy of the model.

[0102] Through the three steps of dimensionality reduction, feature selection, and standardization, the multi-scale feature vectors are optimized, significantly improving the accuracy of wound analysis and the operation accuracy of the spraying robot. This optimization process not only improves the computational efficiency, but also enhances the performance and interpretability of the model, providing strong technical support for uniform spraying.

[0103] In the above embodiment, step a5: Determine the curvature of the wound area according to all the optimized feature vectors, including:

[0104] Step a51: According to all the optimized feature vectors, perform 3D reconstruction using a deep learning-based 3D reconstruction algorithm to generate a 3D model of the wound area. The 3D reconstruction algorithm can refer to using deep learning technology to recover the 3D structure of an object from 2D images. Such algorithms usually include convolutional neural networks and generative adversarial networks, etc., which can extract features from multi-view or multi-modal images and generate high-precision 3D models. The 3D geometric structure generated by the 3D reconstruction algorithm can intuitively display the spatial shape of the wound. Specifically, the generated 3D model corresponding to the wound area not only shows the surface shape of the wound, but also contains depth information, helping medical staff to comprehensively understand the geometric features of the wound.

[0105] Optionally, step a51 can use the following formula to generate the 3D model of the wound area:

[0106] C=3D_Reconstruction(F opt );

[0107] where C is the 3D model of the wound area, 3D_Reconstruction is the deep learning-based 3D reconstruction algorithm, and F opt represents the optimized feature vectors.

[0108] Step a52: According to the three-dimensional model, use the discrete geometry method to calculate the curvature value of each point within the wound area; the curvature value is used to reflect the degree of bending of the corresponding point. In the three-dimensional model, the discrete geometry method can be used to calculate the geometric properties of each vertex or patch, such as curvature, normal vector, etc. In this embodiment, through the discrete geometry method, the curvature value of each point within the wound area can be calculated, and these curvature values reflect the degree of bending of the wound surface.

[0109] Optionally, the discrete geometry method can adopt the following formula:

[0110]

[0111] where K(p) is the curvature value at point p, is the gradient at point p, is the Laplace operator at point p.

[0112] Step a53: Determine the degree of bending of the wound area based on the curvature values of all points.

[0113] Optionally, step a53 can adopt the following formula:

[0114]

[0115] where B represents the degree of bending of the wound area, N is the number of points within the wound area, K(p i ) is the curvature value at point p i . γ is a smoothing coefficient used to prevent numerical instability caused by excessive curvature values.

[0116] In the three-dimensional model of the wound area, the curvature value of each point can be used to evaluate the degree of bending of that point, helping medical staff identify complex areas of the wound. By calculating the curvature values of all points within the wound area, an overall bending degree index can be obtained, and this index can be used to evaluate the complexity of the wound and formulate a spraying strategy.

[0117] In this embodiment, by calculating the curvature value of each point, the degree of bending of the wound can be accurately evaluated, thereby helping medical staff identify complex areas of the wound. In addition, according to the degree of bending of the wound, the robot can dynamically adjust the spraying pressure, moving speed, spraying angle, etc., to ensure that the drug can evenly cover the entire wound surface.

[0118] In some alternative embodiments, S13: Generate a three-dimensional motion trajectory of the robot nozzle according to the image analysis result, and the moving speed and spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory, including:

[0119] Step 131: Based on the geometric characteristics of the wound area, a multi-gradient control strategy is used to divide the wound area into multiple unequally spaced grids. The number of target points in each grid is greater than a preset number, where the target number is the number of points with a curvature value greater than the preset curvature value. Different grids can be located in different or identical planes. A complex area is divided into multiple smaller sub-areas, each of which is referred to as a grid. In this embodiment, gridding can help more precisely manage the characteristics of each sub-area.

[0120] Exemplarily, the multi-gradient control strategy in step 131 may adopt the following formula:

[0121] G i =Grid_Division(C,threshold i , δ i );

[0122] Among them, G i is the i-th grid, C is the three-dimensional model of the wound area, threshold is the grid division threshold, such as the curvature threshold, δ i is the minimum size limit of the i-th grid.

[0123] Step 132: Perform path planning based on all grids to obtain a path planning result; the path planning result is a three-dimensional motion trajectory of the robot nozzle.

[0124] Step 133 : Determine the movement speed and spray angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the geometric characteristics of the wound area, the spray pressure of the robot nozzle, and the nozzle diameter of the robot nozzle.

[0125] Alternatively, the geometrical characteristics of the wound area can be characterized by the curvature K(p i ) and distance D(p i ) is used to reflect the above. The formula for determining the moving speed of the robot nozzle at different positions in step 133 is as follows:

[0126]

[0127] Among them, v i Indicates the moving speed at the i-th position in the three-dimensional motion trajectory. max Indicates the maximum moving speed. The maximum moving speed is a preset constant used to limit the maximum speed of the nozzle. α is the curvature adjustment coefficient, which is used to adjust the effect of curvature on speed. β is the distance adjustment coefficient, which is used to adjust the effect of distance on speed. D(p i) is the distance from the i-th position point to the wound edge. The closer the distance, the faster the moving speed of the nozzle, so as to reduce unnecessary residence time. γ′ represents the pressure adjustment coefficient, which is used to adjust the influence of the injection pressure on the speed. P is the injection pressure of the robot nozzle. The greater the pressure, the slower the moving speed of the nozzle, so as to ensure that the drug can be sprayed more evenly.

[0128] Step 133 The formula for determining the injection angle of the robot nozzle at different positions is as follows:

[0129]

[0130] Among them, θ i represents the injection angle at the i-th position, represents the partial derivative of the i-th position point in the x direction, represents the partial derivative of the i-th position point in the y direction, and ω is the weight coefficient, which is used to balance the influence of different directions. represents the additional partial derivative of the i-th position point in the x direction, such as considering the environmental influence. represents the additional partial derivative of the i-th position point in the y direction. δ represents the distance adjustment coefficient, which is used to adjust the influence of the distance on the angle. D(p i ) represents the distance from the i-th position point to the wound edge. d is the nozzle diameter of the nozzle. The larger the diameter, the injection angle may be adjusted to ensure that the drug can cover a larger area.

[0131] The moving speed can refer to the moving speed of the robot nozzle at different positions in the three-dimensional motion trajectory. In this embodiment, the moving speed of the nozzle can be adjusted according to the geometric characteristics and injection requirements of the wound area. In the area with a higher curvature value, the moving speed of the nozzle is slower to ensure that the drug can be fully covered; in the flat area, the moving speed of the nozzle is faster to improve work efficiency. The injection angle can refer to the injection direction of the robot nozzle at different positions. In this embodiment, the injection angle of the nozzle can be adjusted according to the geometric characteristics and injection requirements of the wound area. In the deep concave area, the injection angle of the nozzle is inclined downward to ensure that the drug can reach the bottom of the wound; in the flat area, the injection angle of the nozzle is perpendicular to the wound surface to ensure uniform coverage of the drug.

[0132] In this embodiment, the wound area is divided into multiple grids with unequal intervals through a multi-gradient control strategy. Path planning is carried out according to all the grids to generate the three-dimensional motion trajectory of the robot nozzle, and the moving speed and injection angle of the nozzle are determined according to the geometric characteristics, injection pressure and nozzle diameter of the wound area. This process significantly improves the spraying accuracy and operation efficiency.

[0133] In some alternative embodiments, after step S14 of generating corresponding control instructions according to the spraying pressure of the robotic nozzle, the three-dimensional motion trajectory of the robotic nozzle, and the moving speed of the robotic nozzle, the method further includes:

[0134] Step 15: Monitor the real-time feedback information during the process of the robotic nozzle performing corresponding operations according to the control instructions. The real-time feedback information includes wound image change information.

[0135] Step 16: Adjust the three-dimensional motion trajectory of the robotic nozzle and / or the moving speed of the robotic nozzle at different positions in the three-dimensional motion trajectory through a preset feedback mechanism.

[0136] According to the wound image change situation in the real-time feedback information, the preset feedback mechanism can adjust the three-dimensional motion trajectory, moving speed, or spraying pressure of the robotic nozzle to optimize the spraying uniformity effect. If the real-time feedback information shows that the drug coverage in a certain area is insufficient, the three-dimensional motion trajectory can be adjusted to make the nozzle re-cover that area. If the real-time feedback information shows that the drug is sprayed too fast in a certain area, the moving speed of the nozzle in that area can be slowed down to ensure uniform drug coverage.

[0137] In this embodiment, by real-time monitoring the real-time feedback information during the operation of the robotic nozzle and dynamically adjusting the three-dimensional motion trajectory and moving speed of the nozzle through a preset feedback mechanism, this process significantly improves the spraying accuracy and operation efficiency.

[0138] In some alternative embodiments, this embodiment can also set a stop mechanism corresponding to the robot to stop the spraying operation when the stop mechanism is satisfied. This application embodiment provides the comprehensiveness and operation stability of robot control.

[0139] Figure 2 It is a schematic structural diagram of a precise wound spraying robot control system based on a neural network provided by an embodiment of this application. As Figure 2 shown, the system includes:

[0140] A receiving module 21, configured to receive the operating room environment parameters and wound image information.

[0141] A determination and analysis module 22, configured to determine the spraying pressure of the robotic nozzle according to the operating room environment parameters, and analyze the wound image information by using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area.

[0142] A first generation module 23, configured to generate the three-dimensional motion trajectory of the robotic nozzle, as well as the moving speed and spraying angle of the robotic nozzle at different positions in the three-dimensional motion trajectory, according to the image analysis result.

[0143] The second generation module 24 is configured to generate corresponding control instructions according to the spraying pressure of the robotic nozzle, the three-dimensional motion trajectory of the robotic nozzle, and the moving speed of the robotic nozzle, so that the robotic nozzle performs corresponding operations according to the control instructions.

[0144] Figure 2 The above-mentioned precise wound spraying robot control system based on neural network can execute Figure 1 The precise wound spraying robot control method based on neural network described in the illustrated embodiment, its implementation principle and technical effects will not be elaborated. For the precise wound spraying robot control system based on neural network in the above-mentioned embodiment, the specific ways for each module and unit to execute operations have been described in detail in the embodiment related to this method, and will not be elaborated here.

[0145] In a possible design, Figure 2 The precise wound spraying robot control system based on neural network in the illustrated embodiment can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32.

[0146] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32.

[0147] The processing component 32 is configured to: receive the operating room environment parameters and the wound image information; determine the spraying pressure of the robotic nozzle according to the operating room environment parameters, and analyze the wound image information by using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area; generate the three-dimensional motion trajectory of the robotic nozzle according to the image analysis result, as well as the moving speed and spraying angle of the robotic nozzle at different positions in the three-dimensional motion trajectory; generate corresponding control instructions according to the spraying pressure of the robotic nozzle, the three-dimensional motion trajectory of the robotic nozzle, and the moving speed of the robotic nozzle, so that the robotic nozzle performs corresponding operations according to the control instructions.

[0148] Among them, the processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above methods. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for executing the above methods.

[0149] The storage component 31 is configured to store various types of data to support the operation of the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as random access memory (RAM), static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0150] Of course, the computing device may also necessarily include other components, such as input / output interfaces, display components, communication components, etc.

[0151] The input / output interface provides an interface between the processing component and the peripheral interface module, and the above peripheral interface module may be an output device, an input device, etc.

[0152] The communication component is configured to facilitate communication between the computing device and other devices in a wired or wireless manner, etc.

[0153] Among them, the computing device may be a physical device or an elastic computing host provided by a cloud computing platform, etc. At this time, the computing device may refer to a cloud server, and the above processing component, storage component, etc. may be basic server resources leased or purchased from a cloud computing platform.

[0154] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment shows a neural network-based control method for a precision wound spraying robot.

[0155] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0156] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0157] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A precise wound spraying robot control system based on a neural network, characterized in that, Including: A receiving module, configured to receive operating room environment parameters and wound image information; A determining and analyzing module, configured to determine the spraying pressure of a robotic nozzle according to the operating room environment parameters, and analyze the wound image information by using a pre-trained neural network to obtain an image analysis result; the pre-trained neural network introduces a multi-scale analysis strategy for the wound area; A first generating module, configured to generate a three-dimensional motion trajectory of the robotic nozzle according to the image analysis result, as well as the moving speed and spraying angle of the robotic nozzle at different positions in the three-dimensional motion trajectory; A second generating module, configured to generate corresponding control instructions according to the spraying pressure of the robotic nozzle, the three-dimensional motion trajectory of the robotic nozzle, and the moving speed of the robotic nozzle, so that the robotic nozzle performs corresponding operations according to the control instructions; The wound image information includes: visible light image information, infrared image information, ultrasonic image information, and depth image information; The analyzing the wound image information by using a pre-trained neural network to obtain an image analysis result includes: Performing preprocessing on the visible light image information, infrared image information, ultrasonic image information, and depth image information respectively to obtain preprocessed visible light image information, infrared image information, ultrasonic image information, and depth image information; Performing segmentation processing on the preprocessed visible light image information to determine the wound boundary in the visible light image, the area within the wound boundary is the wound area, and the area outside the wound boundary is the background area; Based on the preprocessed infrared image information, ultrasonic image information, and depth image information, registering and fusing the wound area to generate multi-modal fusion image information; Based on the multi-modal fusion image information, performing multi-scale analysis on the wound area by using a multi-scale analysis strategy to obtain an image analysis result; The image analysis result is used to represent the geometric features of the wound area in the wound image, and the geometric features include the degree of curvature; The performing multi-scale analysis on the wound area by using a multi-scale analysis strategy based on the multi-modal fusion image information to obtain an image analysis result includes: Based on the multi-modal fusion image information, using a multi-resolution pyramid technique to generate multi-modal fusion images of different scales; Using a variety of feature extraction algorithms to extract a variety of features from the multi-modal fusion images of each scale; Fusing the same type of features at different scales to generate a multi-scale feature vector; Performing optimization processing on the multi-scale feature vector to obtain an optimized feature vector; Determining the degree of curvature of the wound area according to all the optimized feature vectors; The generating a three-dimensional motion trajectory of the robotic nozzle according to the image analysis result, as well as the moving speed and spraying angle of the robotic nozzle at different positions in the three-dimensional motion trajectory includes: According to the geometric features of the wound area, adopting a multi-gradient control strategy to divide the wound area into multiple grids with unequal intervals, where the number of target points existing in each grid is greater than a preset number, and the target points are the number of points with a curvature value greater than a preset curvature value; Perform path planning based on all grids to obtain the path planning result; the path planning result is the three-dimensional motion trajectory of the robot nozzle. Determine the moving speed and spraying angle of the robot nozzle at different positions in the three-dimensional motion trajectory according to the geometric characteristics of the wound area, the spraying pressure of the robot nozzle, and the nozzle diameter of the robot nozzle.

Images