Industrial robot 3D visual identification method based on deep learning

Abstract

The invention discloses an industrial robot 3D visual identification method based on deep learning. The method comprises the following steps: step 1, carrying out data acquisition and preprocessing; step 2, extracting data features; 3, selecting and constructing a deep learning model; 4, training and optimizing the model; 5, evaluating and selecting the model; 6, deploying and applying the model; 7, continuously improving and maintaining the model; the industrial robot 3D visual identification method based on deep learning is an advanced industrial automation technology, the advantages of a deep learning algorithm and the advantages of a 3D visual technology are combined, the complete process of the industrial robot 3D visual identification method based on deep learning is formed through the steps, and the industrial robot 3D visual identification method based on deep learning is realized from data acquisition to model deployment and application. And through continuous improvement and maintenance, higher and faster identification precision and adaptability are provided for the industrial robot, and the technology plays a more and more important role in development of industrial automation and intelligence in the future.

Description

Technical Field

[0001] The present invention relates to the field of industrial robot recognition technology, and in particular to an industrial robot 3D vision recognition method based on deep learning. Background Art

[0002] Industrial robot vision is equivalent to equipping industrial robots with "eyes", allowing them to see objects clearly and tirelessly, and give full play to the function of human eye inspection and detection. This is very important in highly automated large-scale production. The machine vision system can convert the captured target into an image signal through a machine vision product, namely an image capture device, and transmit it to a dedicated image processing system to obtain the morphological information of the captured target. According to the pixel distribution, brightness, color and other information, it is converted into a digital signal. Then the image system performs various operations on these signals to extract the characteristics of the target, and then controls the equipment action on site according to the judgment results; The 3D vision technology in the currently commonly used industrial robot 3D vision recognition method obtains the three-dimensional information of the object, including shape, size, position, etc., through cameras, sensors and other equipment, and then extracts useful feature information from these data through a computer processing system, and classifies and recognizes them. However, due to the large amount of data and the similarity of data features, the efficiency and accuracy of data processing are limited when the data is processed by a conventional computer processing system. For this reason, this application proposes a 3D vision recognition method for industrial robots based on deep learning. Summary of the invention

[0003] The purpose of the present invention is to solve the shortcomings of the prior art mentioned in the background technology and to propose a 3D visual recognition method for industrial robots based on deep learning.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: The 3D visual recognition method of industrial robots based on deep learning includes the following steps: Step 1: Data collection and preprocessing; Step 2: Extract data features; Step 3: Selection and construction of deep learning model; Step 4: Train and optimize the model; Step 5: Evaluate and select the model; Step 6: Deploy and apply the model; Step 7: Continuously improve and maintain the model.

[0005] As a further step in the method of the present invention, step one includes data collection, data preprocessing and data enhancement; the specific steps are to first collect the data, then preprocess the data, and then enhance the data.

[0006] As a further step in the method of the present invention, step two includes feature design and deep learning feature extraction; the specific steps are to perform 3D feature design before deep learning, and then use the deep learning model to extract the deep learning features.

[0007] As a further step in the method of the present invention, step three includes model selection and model construction; the specific steps are to first select a suitable deep learning model architecture according to task requirements and data characteristics, and then build a specific deep learning model based on the selected model architecture.

[0008] As a further step in the method of the present invention, step four includes loss function design, data division, training process, verification and tuning; the specific steps are to first design a suitable loss function to measure the difference between the model prediction results and the actual labels, then divide the preprocessed data set into a training set, a verification set and a test set, and then use the preprocessed 3D data set to train the model, and then evaluate the performance of the model on the verification set, and tune the model according to the evaluation results.

[0009] As a further step in the method of the present invention, step five includes evaluation indicators, testing process and model selection; the specific steps are to first use a validation set or a test set to evaluate the performance of the model, then use the test set data to test the model, evaluate the generalization ability and actual recognition effect of the model, and then select the model with the best performance as the final model based on the evaluation results.

[0010] As a further step in the method of the present invention, step six includes model deployment, system integration and on-site testing and tuning. The specific steps are to first export the trained model into a format suitable for use by the industrial robot and deploy it to the visual recognition system of the industrial robot to form a 3D visual recognition module, and then integrate the 3D visual recognition module with other systems of the industrial robot to work together, and then test the system in an actual working environment, and perform necessary tuning and adjustments based on the test results.

[0011] As a further step in the method of the present invention, step seven: data collection and feedback, model optimization and system maintenance, the specific steps are to first continuously collect new 3D data during the actual operation process, and then further optimize and adjust the model according to the data feedback and performance evaluation results in actual applications to improve the recognition accuracy and generalization ability. Finally, the system needs to be maintained and serviced regularly to ensure the stable operation and long-term reliability of the system.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The deep learning-based 3D visual recognition method for industrial robots is an advanced industrial automation technology. It combines deep learning algorithms and 3D vision technology to form a closed-loop system from data collection to model deployment and application, and then to continuous improvement and maintenance. It can capture three-dimensional information of the surrounding environment and generate depth maps, point clouds and other data. The deep learning algorithm can automatically extract useful features from these data quickly and accurately, and classify and identify them. Through continuous optimization and improvement, the visual recognition ability of industrial robots can be improved, so that industrial robots can perform various tasks more accurately and flexibly to meet the needs of practical applications. DETAILED DESCRIPTION

[0013] The content of the present invention can be more easily understood by selecting the following detailed description of the preferred implementation method of the present invention and the embodiments included. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. When there is a conflict, the definition in this specification shall prevail.

[0014] As used herein, the term "prepared from" is synonymous with "comprising." As used herein, the terms "comprising," "including," "having," "containing," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a composition, process, method, article, or apparatus that comprises the listed elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such composition, process, method, article, or apparatus.

[0015] The conjunction "consisting of excludes any unspecified element, step, or component. If used in a claim, this phrase renders the claim closed-ended so that it does not include materials other than those described, except for conventional impurities associated therewith. When the phrase "consisting of" appears in a clause of the body of a claim rather than immediately following the subject matter, it limits only the elements described in that clause; other elements are not excluded from the claim as a whole.

[0016] When amount, concentration or other value or parameter is expressed as range, preferred range or a series of upper preferred value and lower preferred value limit range, this should be understood as specifically disclosing all ranges formed by any pairing of any range upper limit or preferred value and any range lower limit or preferred value, regardless of whether the range is disclosed separately. For example, when disclosing range "1 to 5", described range should be interpreted as including range "1 to 4", "1 to 3", "1 to 2", "1 to 2 and 4 to 5", "1 to 3 and 5" etc. When numerical range is described in this article, unless otherwise stated, the range is intended to include its end value and all integers and fractions within the range.

[0017] Singular forms include plural references unless the context clearly indicates otherwise. "Optional" or "either" means that the subsequently described event or incident may or may not occur, and that the description includes instances where the event occurs and instances where it does not.

[0018] Approximate terms in the specification and claims are used to modify quantities, indicating that the present invention is not limited to the specific quantity, but also includes acceptable and modified parts close to the quantity without causing changes in the relevant basic functions. Accordingly, the use of "about", "approximately", etc. to modify a numerical value means that the present invention is not limited to the exact numerical value. In some examples, the approximate terms may correspond to the accuracy of the instrument for measuring the numerical value. In the specification and claims of this application, range limitations can be combined and / or interchanged, and if not otherwise stated, these ranges include all subranges contained therein.

[0019] In addition, the indefinite articles "a" and "an" before the elements or components of the present invention have no restrictions on the quantity requirements (i.e. the number of occurrences) of the elements or components. Therefore, "a" or "an" should be interpreted as including one or at least one, and the elements or components in the singular form also include the plural form, unless the number is obviously intended to be in the singular form.

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0021] Deep learning algorithm is a branch of machine learning. It simulates the learning process of the human brain by building a deep neural network. In the 3D visual recognition of industrial robots, deep learning algorithm can automatically extract features in images and learn how to classify, identify and locate based on these features. 3D vision technology uses cameras, sensors and other devices to obtain three-dimensional information about objects, including shape, size, and position. Compared with two-dimensional vision, 3D vision technology can provide richer spatial information, enabling industrial robots to understand the surrounding environment more accurately. The 3D vision recognition method for industrial robots based on deep learning proposed in this application combines deep learning with 3D vision technology, which can significantly improve the recognition accuracy and adaptability of industrial robots. This combination can bring the following advantages: Feature extraction: Deep learning algorithms can automatically extract key features from 3D images, such as edges, corners, surfaces, etc. These features are crucial for subsequent recognition and positioning; Model training: By training the deep learning model with a large amount of 3D image data, the model can learn the shape, color, texture and other features of the object and establish the mapping relationship between these features and the object category; Real-time recognition: In industrial production, industrial robots need to recognize and locate objects in real time; Deep learning algorithms can quickly process 3D image data and achieve real-time recognition and positioning, thus meeting the real-time requirements of industrial production; Adaptability: Deep learning algorithms have strong adaptability and can cope with objects of different shapes, sizes and materials, which means that industrial robots can flexibly adapt to different production tasks without a lot of programming and debugging.

[0022] The 3D visual recognition method for industrial robots based on deep learning includes the following steps: S1. Data collection and preprocessing S1.1. Perform data acquisition: Use 3D sensors (such as structured light cameras, stereo vision cameras, LiDAR, etc.) to capture 3D data of the target object, which may include depth images, point cloud data, or voxel grids; S1.2, Data preprocessing: Preprocess the collected 3D data, such as denoising, filtering, registration (if data from multiple perspectives are involved), segmentation (separating the target object from the background), etc., to improve data quality and reduce the complexity of subsequent processing; S1.3. Data enhancement: In order to increase data diversity and improve the generalization ability of the model, the preprocessed data can be enhanced, such as rotation, scaling, translation and other transformations.

[0023] S2. Feature extraction S2.1. Feature design: Before deep learning, you may need to manually design some 3D features (although modern deep learning models can often automatically learn features), such as geometric features (points, lines, surfaces), statistical features (histograms, distributions), etc. However, this step is gradually replaced by automatic feature learning in modern deep learning methods; S2.2. Deep learning feature extraction: Use deep learning models (such as 3D convolutional neural networks, point cloud neural networks, etc.) to extract features from preprocessed 3D data. These models can automatically learn and extract useful feature representations from the data. These features may include shape, texture, edge, etc., which are used to describe the three-dimensional surface characteristics of the object; S2.3, Classification and Recognition: Select a suitable classifier, such as support vector machine (SVM) or neural network, to classify and identify the extracted features.

[0024] S3. Deep learning model selection and construction S3.1. Model selection: Select the appropriate deep learning model architecture based on task requirements (such as classification, detection, segmentation, etc.) and data characteristics (such as point cloud, depth image, etc.). For example, for point cloud data, you can choose models such as PointNet and PointNet++; for depth images, you can choose 3D convolutional neural network (3D CNN), etc. S3.2. Model construction: Based on the selected model architecture, build a specific deep learning model, including defining network layers, setting hyperparameters (such as learning rate, batch size, etc.), etc.

[0025] S4. Model training and optimization S4.1. Loss function design: Design a suitable loss function to measure the difference between the model prediction result and the actual label, such as cross entropy loss in classification tasks, mean square error in regression tasks, etc. S4.2, Data division: Divide the preprocessed data set into training set, validation set and test set for model training, validation and testing; S4.3, training process: use the preprocessed 3D data set to train the model, optimize the model parameters through the back propagation algorithm, reduce the loss function value, and improve the model performance; S4.4. Validation and tuning: Evaluate the performance of the model on the validation set and tune the model based on the evaluation results, such as adjusting hyperparameters, adding regularization terms, etc.

[0026] S5. Model evaluation and selection S5.1. Evaluation indicators: Use the validation set or test set to evaluate the performance of the model. Common evaluation indicators include accuracy, recall, F1 score, AUC, etc. S5.2. Testing process: Use the test set data to test the model and evaluate the generalization ability and actual recognition effect of the model; S5.3. Model selection: Based on the evaluation results, the model with the best performance is selected as the final model.

[0027] S6. Deployment and Application S6.1. Model deployment: To deploy the trained model on an industrial robot, it is usually necessary to convert the deep learning model into a format suitable for running on an industrial robot, such as TensorFlow Lite, ONNX, etc. S6.2, System Integration: Integrate the exported model into the visual recognition system of the industrial robot to form a 3D visual recognition module. The 3D visual recognition module is integrated with other systems of the industrial robot (such as motion control, path planning, etc.) to work together, and conduct overall testing to ensure that the system can operate stably and meet the expected waist requirements; S6.3. On-site testing and tuning: Test the system in the actual working environment and make necessary tuning and adjustments based on the test results.

[0028] S7.Continuous improvement and maintenance S7.1. Data collection and feedback: Continuously collect new 3D data during actual operation for continuous improvement and updating of the model; S7.2, Model optimization: Based on the data feedback and performance evaluation results in actual applications, the model is further optimized and adjusted to improve recognition accuracy and generalization ability; S7.3. System maintenance: Regularly maintain and service the system to ensure stable operation and long-term reliability of the system.

[0029] The above steps together constitute the complete process of the deep learning-based 3D visual recognition method for industrial robots. From data collection to model deployment and application, and then to continuous improvement and maintenance, a closed-loop system is formed that can capture three-dimensional information of the surrounding environment and generate depth maps, point clouds and other data. The deep learning algorithm can automatically extract useful features from these data, and classify and identify them. Through continuous optimization and improvement, the visual recognition capability of industrial robots can be improved to meet the needs of practical applications.

[0030] The examples referred to herein are merely illustrative and are used to explain some features of the method of the present invention. The appended claims are intended to claim the widest possible range that can be imagined, and the embodiments presented herein are only illustrations of selected implementations according to the combination of all possible embodiments. Therefore, it is the applicant's intention that the appended claims are not limited by the selection of examples that illustrate the features of the present invention. Some numerical ranges used in the claims also include sub-ranges within them, and changes in these ranges should also be interpreted as being covered by the appended claims where possible.

Claims

1. A 3D visual recognition method for industrial robots based on deep learning, characterized in that: The following steps are involved: Step 1: Data collection and preprocessing; Step 2: Extract data features; Step 3: Selection and construction of deep learning model; Step 4: Train and optimize the model; Step 5: Evaluate and select the model; Step 6: Deploy and apply the model; Step 7: Continuously improve and maintain the model.

2. The industrial robot 3D visual recognition method based on deep learning according to claim 1, characterized in that: Step one includes data collection, data preprocessing and data enhancement; the specific steps are to first collect the data, then preprocess the data, and then enhance the data.

3. The industrial robot 3D visual recognition method based on deep learning according to claim 1, characterized in that: Step 2 includes feature design and deep learning feature extraction; The specific steps are to design 3D features before deep learning, and then use the deep learning model to extract deep learning features.

4. The industrial robot 3D visual recognition method based on deep learning according to claim 1, characterized in that: Step three includes model selection and model construction; the specific steps are to first select a suitable deep learning model architecture based on task requirements and data characteristics, and then build a specific deep learning model based on the selected model architecture.

5. The industrial robot 3D visual recognition method based on deep learning according to claim 1, characterized in that: Step four includes loss function design, data division, training process, verification and tuning; the specific steps are to first design a suitable loss function to measure the difference between the model prediction results and the actual labels, then divide the preprocessed data set into training set, verification set and test set, and then use the preprocessed 3D data set to train the model, and then evaluate the performance of the model on the verification set, and tune the model according to the evaluation results.

6. The industrial robot 3D visual recognition method based on deep learning according to claim 1, characterized in that: Step five includes evaluation indicators, testing process and model selection; the specific steps are to first use the validation set or test set to evaluate the performance of the model, then use the test set data to test the model, evaluate the generalization ability and actual recognition effect of the model, and then select the model with the best performance as the final model based on the evaluation results.

7. The industrial robot 3D visual recognition method based on deep learning according to claim 1, characterized in that: Step six includes model deployment, system integration, and on-site testing and tuning. The specific steps are to first export the trained model into a format suitable for industrial robots and deploy it to the industrial robot's visual recognition system to form a 3D visual recognition module. The 3D visual recognition module is then integrated with other systems of the industrial robot to work together. The system is then tested in the actual working environment, and necessary tuning and adjustments are performed based on the test results.

8. The industrial robot 3D visual recognition method based on deep learning according to claim 1, characterized in that: Step 7: Data collection and feedback, model optimization and system maintenance. The specific steps are to continuously collect new 3D data during the actual operation process, and then further optimize and adjust the model based on the data feedback and performance evaluation results in actual applications to improve the recognition accuracy and generalization ability. Finally, the system needs to be maintained and serviced regularly to ensure its stable operation and long-term reliability.