Intelligent control system and method of industrial robots based on cloud platform

Through the intelligent control system of industrial robots based on cloud platform, computer vision and image processing technology are used to extract and analyze the feature of workpieces and generate target locations, which solves the problem that industrial robots are difficult to achieve intelligent sorting, and improves sorting accuracy and Productivity.

CN118163119BActive Publication Date: 2024-11-22BEIJING SHANGCHENG ZHIYIN ROBOT TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410608704.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-16
Publication Date
2024-11-22
Estimated Expiration
2044-05-16

AI Technical Summary

Technical Problem

The prior art is difficult to realize intelligent sorting of workpieces by industrial robots, resulting in an increase in manual sorting error rate and affecting production quality.

Method used

The intelligent control system of industrial robots based on cloud platforms is adopted to collect image data and five-part position coordinates of workpieces at different angles, and feature extraction and analysis are used using computer vision and image processing technology to generate the target position of workpieces to achieve accurate sorting.

Benefits of technology

It improves the accuracy and production efficiency of workpiece sorting, reduces the manual error rate, and improves production quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118163119B_ABST
    Figure CN118163119B_ABST
Patent Text Reader

Abstract

The present application relates to the field of intelligent control of industrial robots, and specifically discloses an intelligent control system and method for industrial robots based on a cloud platform, which first collects image data of workpieces to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces as input data, and then uses computer vision and image processing technology to extract and analyze features of these input data to generate the target position of the workpiece to be analyzed, and then the industrial robot performs precise sorting operations on the workpiece to be analyzed based on the target position. In this way, it is helpful to realize intelligent sorting of workpieces.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent control of industrial robots, and more specifically, to an intelligent control system and method for industrial robots based on a cloud platform. Background Art

[0002] With the rapid advancement of industrial technology, industrial robots play a key role in today's intelligent industrial production and have become an important part of the comprehensive automation of the manufacturing industry. As the tasks on the production line become increasingly arduous, the use of manual labor to perform some tedious workpiece sorting can no longer improve the production capacity of enterprises. With the increase in workload, the error rate of manual sorting has gradually increased, affecting production quality.

[0003] Therefore, we look forward to an intelligent control system and method for industrial robots based on a cloud platform, which can control industrial robots based on computer vision and image processing technology to achieve intelligent sorting of workpieces. Summary of the invention

[0004] In order to solve the above technical problems, this application is proposed. The embodiment of the present application provides an intelligent control system and method for industrial robots based on a cloud platform, which first collects image data of the workpiece to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces as input data, and then uses computer vision and image processing technology to extract and analyze these input data to generate the target position of the workpiece to be analyzed, and then the industrial robot performs precise sorting operations on the workpiece to be analyzed based on the target position. In this way, it helps to realize intelligent sorting of workpieces.

[0005] According to a first aspect of the present application, there is provided an industrial robot intelligent control system based on a cloud platform, comprising:

[0006] The workpiece data acquisition module to be analyzed is used to collect image data of the workpiece to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces;

[0007] The feature acquisition module of the workpiece to be analyzed is used to extract and analyze the image data of the workpiece to be analyzed at different shooting angles, the image data of the different types of reference workpieces, and the five-element position coordinates of the different types of reference workpieces to obtain the type feature matrix of the workpiece to be analyzed and the position feature matrix of the reference workpiece;

[0008] A feature fusion module for workpieces to be analyzed, used for performing feature fusion on the feature matrix of the workpiece type to be analyzed and the feature matrix of the reference workpiece position to obtain a feature matrix of the workpiece position to be analyzed;

[0009] The workpiece to be analyzed result generating module is used to generate the target position of the workpiece to be analyzed based on the position feature matrix of the workpiece to be analyzed.

[0010] In combination with the first aspect of the present application, in a cloud platform-based industrial robot intelligent control system of the first aspect of the present application, the workpiece feature acquisition module to be analyzed includes: a workpiece feature extraction unit to be analyzed, used to feature encode the image data of the workpiece to be analyzed at different shooting angles to obtain a global feature vector of the workpiece to be analyzed; a reference workpiece feature extraction unit, used to feature encode the image data of the different types of reference workpieces to obtain a comprehensive feature vector of the reference workpiece; a workpiece type feature acquisition unit, used to use a joint encoder to feature fuse the global feature vector of the workpiece to be analyzed and the comprehensive feature vector of the reference workpiece to obtain the type feature matrix of the workpiece to be analyzed; a reference workpiece position feature extraction unit, used to feature encode the five-element position coordinates of the different types of reference workpieces to obtain the reference workpiece position feature matrix.

[0011] According to a second aspect of the present application, a cloud platform-based industrial robot intelligent control method is provided, which includes:

[0012] Collect image data of the workpiece to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces;

[0013] Extracting and analyzing the image data of the workpiece to be analyzed at different shooting angles, the image data of the different types of reference workpieces, and the five-element position coordinates of the different types of reference workpieces to obtain a feature matrix of the workpiece type to be analyzed and a feature matrix of the reference workpiece position;

[0014] Performing feature fusion on the workpiece type feature matrix to be analyzed and the reference workpiece position feature matrix to obtain the workpiece position feature matrix to be analyzed;

[0015] A target position of the workpiece to be analyzed is generated based on the position feature matrix of the workpiece to be analyzed.

[0016] In combination with the second aspect of the present application, in a cloud platform-based industrial robot intelligent control method of the second aspect of the present application, the target position of the workpiece to be analyzed is generated based on the workpiece position feature matrix to be analyzed, including: a workpiece position feature extraction unit to be analyzed, used to pass the workpiece position feature matrix to be analyzed through a workpiece position feature extraction module to be analyzed based on a convolutional neural network model to obtain a workpiece position generation feature vector to be analyzed; a workpiece position feature optimization unit to be analyzed, used to adjust the workpiece position generation feature vector to be analyzed based on the probability responsiveness of the distribution direction to obtain an optimized workpiece position generation feature vector to be analyzed; a workpiece position generation unit to be analyzed, used to pass the optimized workpiece position generation feature vector to be analyzed through a generator to generate the target position of the workpiece to be analyzed.

[0017] Compared with the prior art, the present application provides an intelligent control system and method for industrial robots based on a cloud platform, which first collects image data of the workpiece to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces as input data, and then uses computer vision and image processing technology to extract and analyze the input data to generate the target position of the workpiece to be analyzed, and then the industrial robot performs accurate sorting operations on the workpiece to be analyzed based on the target position. In this way, it is helpful to realize intelligent sorting of workpieces. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 The figure illustrates a schematic block diagram of an industrial robot intelligent control system based on a cloud platform according to an embodiment of the present application.

[0020] Figure 2 The figure illustrates a schematic block diagram of a feature acquisition module for a workpiece to be analyzed in a cloud platform-based industrial robot intelligent control system according to an embodiment of the present application.

[0021] Figure 3 The figure illustrates a schematic block diagram of a workpiece feature extraction unit to be analyzed in a workpiece feature acquisition module to be analyzed in a cloud platform-based industrial robot intelligent control system according to an embodiment of the present application.

[0022] Figure 4The figure illustrates a schematic block diagram of a reference workpiece feature extraction unit in a workpiece feature acquisition module to be analyzed in a cloud platform-based industrial robot intelligent control system according to an embodiment of the present application.

[0023] Figure 5 The figure illustrates a schematic block diagram of a reference workpiece position feature extraction unit in a workpiece feature acquisition module to be analyzed in a cloud platform-based industrial robot intelligent control system according to an embodiment of the present application.

[0024] Figure 6 The figure illustrates a schematic block diagram of a module for generating results of workpieces to be analyzed in a cloud platform-based industrial robot intelligent control system according to an embodiment of the present application.

[0025] Figure 7 The figure illustrates a flow chart of an intelligent control method for an industrial robot based on a cloud platform according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0027] Exemplary Systems

[0028] Figure 1 The figure shows a schematic block diagram of an industrial robot intelligent control system based on a cloud platform according to an embodiment of the present application. Figure 1 As shown, according to an embodiment of the present application, a cloud platform-based industrial robot intelligent control system 100 includes: a workpiece data acquisition module 110 to be analyzed, which is used to collect image data of the workpiece to be analyzed at different shooting angles, different types of reference workpiece image data, and five-element position coordinates of different types of reference workpieces; a workpiece feature acquisition module 120 to be analyzed, which is used to extract and analyze the image data of the workpiece to be analyzed at different shooting angles, the different types of reference workpiece image data, and the five-element position coordinates of the different types of reference workpieces to obtain a workpiece type feature matrix to be analyzed and a reference workpiece position feature matrix; a workpiece feature fusion module 130 to be analyzed, which is used to perform feature fusion on the workpiece type feature matrix to be analyzed and the reference workpiece position feature matrix to obtain a workpiece position feature matrix to be analyzed; and a workpiece result generation module 140 to be analyzed, which is used to generate a target position of the workpiece to be analyzed based on the workpiece position feature matrix to be analyzed.

[0029] Industrial robots are automated equipment used to perform various tasks in manufacturing and production processes. They are usually designed to replace manual labor to increase production efficiency, improve product quality and reduce production costs.

[0030] With the rapid advancement of industrial technology, industrial robots play a key role in today's industrial intelligent production and have become an important part of the comprehensive automation of the manufacturing industry. With the increasing heavy tasks of the production line, it is no longer possible to improve the production capacity of enterprises by still using manual labor to perform some tedious workpiece sorting. With the increase in workload, the error rate of manual sorting has gradually increased, affecting production quality. Therefore, an industrial robot intelligent control system and method based on a cloud platform is expected to control the industrial robot based on computer vision and image processing technology to achieve intelligent sorting of workpieces.

[0031] In the embodiment of the present application, the workpiece data acquisition module 110 to be analyzed is used to collect image data of the workpiece to be analyzed at different shooting angles, different types of reference workpiece image data, and five-element position coordinates of different types of reference workpieces. It should be understood that the image data at different shooting angles can provide more comprehensive and diverse workpiece information, which is helpful to obtain the features and details of the workpiece from different perspectives, and improve the recognition and sorting accuracy of the robot to the workpiece. Therefore, in order to replace the comprehensive analysis of the workpiece features, the image data of the workpiece to be analyzed at different shooting angles is first collected. Next, considering that there are a large number of different types of workpiece image data in the cloud platform, these different types of workpiece image data can help the system learn and identify different types of workpieces, and improve the generalization ability and adaptability of the system. Therefore, in order to more accurately identify the type of workpiece to be analyzed, it is necessary to obtain different types of reference workpiece image data from the cloud platform. Further, it should be understood that the five-element position coordinates of the workpiece include the spatial position coordinates, pitch angle, and azimuth of the workpiece, which provide detailed information about the position and posture of the workpiece, which can help the robot accurately locate and grasp the workpiece to ensure accurate operation. Therefore, in order to better locate the target position of the workpiece to be analyzed, it is also necessary to obtain the five-element position coordinates (spatial position coordinates, pitch angle and azimuth angle of the workpiece) corresponding to different types of reference workpieces from the cloud platform.

[0032] In the embodiment of the present application, the workpiece feature acquisition module 120 to be analyzed is used to extract and analyze the image data of the workpiece to be analyzed at different shooting angles, the image data of the reference workpieces of different types, and the five-element position coordinates of the reference workpieces of different types to obtain the workpiece type feature matrix to be analyzed and the reference workpiece position feature matrix. It should be understood that after collecting these input data, the input data are further subjected to feature extraction and analysis tasks.

[0033] Specifically, Figure 2 The figure shows a schematic block diagram of a workpiece feature acquisition module to be analyzed in a cloud platform-based industrial robot intelligent control system according to an embodiment of the present application. Figure 2 As shown, the workpiece feature acquisition module 120 to be analyzed includes: a workpiece feature extraction unit 121 to be analyzed, used for performing feature encoding on the image data of the workpiece to be analyzed at different shooting angles to obtain a global feature vector of the workpiece to be analyzed; a reference workpiece feature extraction unit 122, used for performing feature encoding on the image data of the reference workpieces of different types to obtain a comprehensive feature vector of the reference workpiece; a workpiece type feature acquisition unit 123, used for performing feature fusion on the global feature vector of the workpiece to be analyzed and the comprehensive feature vector of the reference workpiece using a joint encoder to obtain a feature matrix of the workpiece type to be analyzed; and a reference workpiece position feature extraction unit 124, used for performing feature encoding on the five-element position coordinates of the reference workpieces of different types to obtain the reference workpiece position feature matrix.

[0034] First, feature extraction and analysis are performed on the image data of the workpiece to be analyzed at different shooting angles. Specifically, Figure 3 The figure shows a schematic block diagram of a workpiece feature extraction unit in a workpiece feature acquisition module in a cloud platform-based industrial robot intelligent control system according to an embodiment of the present application. Figure 3 As shown, the workpiece feature extraction unit 121 to be analyzed includes: an image enhancement subunit 121-1, which is used to pass the image data of the workpiece to be analyzed at different shooting angles through an image enhancement module based on a generative adversarial network to obtain a plurality of pixel enhanced images of the workpiece to be analyzed; a workpiece local feature extraction subunit 121-2, which is used to pass the plurality of pixel enhanced images of the workpiece to be analyzed through a workpiece local feature extractor to be analyzed based on a depth feature fusion module to obtain a plurality of local feature matrices of the workpiece to be analyzed; and a workpiece global feature extraction subunit 121-3, which is used to arrange the plurality of local feature matrices of the workpiece to be analyzed into a global three-dimensional tensor of the workpiece to be analyzed and then pass the result through a workpiece global feature extraction module to be analyzed based on a three-dimensional convolutional neural network model to obtain a global feature vector of the workpiece to be analyzed.

[0035] It should be understood that the image may be affected by factors such as illumination and noise, and the quality of the image can be improved by enhancement, making it clearer and easier to analyze. Therefore, an image enhancement module based on a generative adversarial network is used to perform image enhancement processing on the image data of the workpiece to be analyzed at different shooting angles to obtain the plurality of pixel-enhanced images of the workpiece to be analyzed. Among them, the generative adversarial network (GAN) is a generative model that can learn and generate images similar to real data. Through training, GAN can generate high-quality image data, which helps to enhance the image of the workpiece to be analyzed.

[0036] In a specific embodiment of the present application, the image enhancement subunit 121-1 is used to: input the image data of the workpiece to be analyzed at different shooting angles into the generator of the pixel enhancement module based on the adversarial generation network to obtain the multiple pixel enhanced images of the workpiece to be analyzed.

[0037] Next, considering that the image contains not only shallow features such as details, textures, edges, etc., but also more advanced and abstract deep features. Therefore, in order to make full use of the complementarity between deep features and shallow features and extract more representative and rich features, a local feature extractor of the workpiece to be analyzed based on a deep and shallow feature fusion module is used to perform feature encoding on the multiple pixel-enhanced images of the workpiece to be analyzed to obtain the multiple local feature matrices of the workpiece to be analyzed. Among them, the deep and shallow feature fusion module can combine the advantages of deep features and shallow features to provide a more comprehensive and richer feature representation, which helps to capture different levels of information in the image.

[0038] In a specific embodiment of the present application, the local feature extraction subunit 121-2 of the workpiece to be analyzed is used to: extract a shallow feature matrix from the i-th layer of the local feature extractor of the workpiece to be analyzed, the i-th layer being the first to sixth layers of the local feature extractor of the workpiece to be analyzed; extract a deep feature matrix from the j-th layer of the local feature extractor of the workpiece to be analyzed, the ratio between the j-th layer and the i-th layer being greater than or equal to 5; and use the shallow-deep feature fusion module to fuse the shallow feature matrix and the deep feature matrix to obtain each local feature matrix of the workpiece to be analyzed in the multiple local feature matrices of the workpiece to be analyzed.

[0039] Next, considering that the arrangement can integrate the correlation between different local features into a unified data structure, it is helpful for the model to better understand and utilize this information. Therefore, the multiple local feature matrices of the workpiece to be analyzed are first arranged into a global three-dimensional tensor of the workpiece to be analyzed. Then, a global feature extraction module for the workpiece to be analyzed based on a three-dimensional convolutional neural network model is used to feature encode the global three-dimensional tensor of the workpiece to be analyzed to obtain the global feature vector of the workpiece to be analyzed. In this way, by using a three-dimensional convolutional neural network for global feature extraction, the system can effectively capture the feature information of the workpiece as a whole, achieve more accurate and comprehensive workpiece identification and sorting, and improve the intelligence level and performance of the industrial robot system.

[0040] In a specific embodiment of the present application, the global feature extraction subunit 121-3 of the workpiece to be analyzed is used to: use each layer of the three-dimensional convolutional neural network model to perform the following on the input data in the forward transfer of the layer: convolution processing on the input data to obtain a convolution feature map; pooling processing based on a feature matrix on the convolution feature map to obtain a pooled feature map; nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the input of the first layer of the three-dimensional convolutional neural network model is the global three-dimensional tensor of the workpiece to be analyzed, and the output of the last layer of the three-dimensional convolutional neural network model is the global feature vector of the workpiece to be analyzed.

[0041] Then, the features of the different types of reference workpiece image data are extracted and analyzed. Specifically, Figure 4 The figure shows a schematic block diagram of a reference workpiece feature extraction unit in a workpiece feature acquisition module to be analyzed in an industrial robot intelligent control system based on a cloud platform according to an embodiment of the present application. Figure 4 As shown, the reference workpiece feature extraction unit 122 includes: a reference workpiece feature acquisition subunit 122-1, used for respectively passing the different types of reference workpiece image data through a reference workpiece feature extraction module including a two-dimensional convolution kernel to obtain a plurality of reference workpiece feature matrices; a reference workpiece comprehensive feature acquisition subunit 122-2, used for arranging the plurality of reference workpiece feature matrices into a reference workpiece comprehensive three-dimensional tensor and then passing the tensor through a reference workpiece comprehensive feature extraction module based on a spatial attention mechanism to obtain the reference workpiece comprehensive feature vector.

[0042] It should be understood that the convolutional neural network model has excellent performance in image processing. Based on this, a reference workpiece feature extraction module including a two-dimensional convolution kernel is used to extract features of the different types of reference workpiece image data to obtain the multiple reference workpiece feature matrices. Among them, the use of a two-dimensional convolution kernel for feature extraction can effectively extract spatial features in image data, which is helpful to realize the intelligent sorting of workpieces by industrial robots.

[0043] In a specific embodiment of the present application, the reference workpiece feature acquisition subunit 122-1 is used to: use each layer of the reference workpiece feature extraction module including the two-dimensional convolution kernel to perform the following on the input data in the forward transfer of the layer: use the convolution units of each layer of the reference workpiece feature extraction module including the two-dimensional convolution kernel to perform convolution processing based on the two-dimensional convolution kernel on the input data to obtain a convolution feature map; use the pooling units of each layer of the reference workpiece feature extraction module including the two-dimensional convolution kernel to perform pooling processing on the convolution feature map along the channel dimension to obtain a pooling feature map; and use the activation units of each layer of the reference workpiece feature extraction module including the two-dimensional convolution kernel to perform nonlinear activation on the feature values ​​of each position in the pooling feature map to obtain an activation feature map; wherein the output of the last layer of the reference workpiece feature extraction module including the two-dimensional convolution kernel is the multiple reference workpiece feature matrices.

[0044] Next, considering that arranging multiple reference workpiece feature matrices into a three-dimensional tensor can integrate the feature information of different reference workpieces into a comprehensive data structure, it is conducive to comprehensive analysis and processing. Therefore, in order to improve the accuracy of workpiece classification, the multiple reference workpiece feature matrices are arranged as a reference workpiece comprehensive three-dimensional tensor. In this way, it can provide strong support for the analysis of workpiece types. Then, a reference workpiece comprehensive feature extraction module based on a spatial attention mechanism is used to perform deep feature extraction on the reference workpiece comprehensive three-dimensional tensor to obtain the reference workpiece comprehensive feature vector. Among them, the spatial attention mechanism can help the model focus on the important feature areas in the input data, thereby improving the model's perception of key information and helping to improve the effect of feature extraction.

[0045] In a specific embodiment of the present application, the reference workpiece comprehensive feature acquisition subunit 122-2 is used to: use the convolution coding part of the reference workpiece comprehensive feature extraction module to perform deep convolution coding on the reference workpiece comprehensive three-dimensional tensor to obtain an initial convolution feature map; input the initial convolution feature map into the spatial attention part of the reference workpiece comprehensive feature extraction module to obtain a spatial attention map; pass the spatial attention map through a Softmax activation function to obtain a spatial attention feature map; calculate the point-by-point multiplication of the spatial attention feature map and the initial convolution feature map to obtain a reference workpiece comprehensive feature map; perform a feature matrix-based mean pooling operation on the reference workpiece comprehensive feature map to obtain the reference workpiece comprehensive feature vector.

[0046] Then, the global feature vector of the workpiece to be analyzed and the comprehensive feature vector of the reference workpiece are subjected to feature fusion operation. It should be understood that the joint encoding of the global feature vector of the analyzed workpiece and the comprehensive feature vector of the reference workpiece can promote the interaction and integration between information, and help to more accurately distinguish the types of the workpieces to be divided. Therefore, a joint encoder is used to perform feature fusion on the global feature vector of the workpiece to be analyzed and the comprehensive feature vector of the reference workpiece to obtain the feature matrix of the type of the workpiece to be analyzed. Among them, the use of the joint encoder can reduce the transmission loss of information between different modules, avoid unnecessary loss and distortion of information, and help maintain the integrity and consistency of features.

[0047] In a specific embodiment of the present application, the workpiece type feature acquisition unit 123 is used to: use the joint encoder to jointly encode the global feature vector of the analysis workpiece and the comprehensive feature vector of the reference workpiece using the following formula to generate the workpiece type feature matrix to be analyzed; wherein the formula is: ,in represents vector multiplication, represents the feature matrix of the workpiece type to be analyzed, represents the global feature vector of the analysis workpiece, represents the comprehensive feature vector of the reference workpiece, represents the transpose of the comprehensive feature vector of the reference workpiece.

[0048] Finally, the five-element position coordinates of the different types of reference workpieces are feature extracted and analyzed. Specifically, Figure 5 The figure shows a schematic block diagram of a reference workpiece position feature extraction unit in a workpiece feature acquisition module to be analyzed in an industrial robot intelligent control system based on a cloud platform according to an embodiment of the present application. Figure 5 As shown, the reference workpiece position feature extraction unit 124 includes: a reference workpiece coordinate feature acquisition subunit 124-1, which is used to pass the five-element position coordinates of the different types of reference workpieces through a sorting position feature extraction module based on a context encoder to obtain five coordinate feature vectors of each reference workpiece, and cascade them to obtain multiple reference workpiece position feature vectors; a reference workpiece position feature acquisition subunit 124-2, which is used to arrange the multiple reference workpiece position feature vectors into a reference workpiece position input matrix and then pass them through a reference workpiece position feature extraction module based on a two-dimensional convolutional neural network model to obtain the reference workpiece position feature matrix.

[0049] First, the sorting position feature extraction module of the context encoder is used to extract coordinate features of the five-element position coordinates of the different types of reference workpieces to obtain the five coordinate feature vectors of each reference workpiece. Among them, the context encoder performs well in processing sequence data and can effectively model and extract features of the five-element position coordinate sequence of the reference workpiece, thereby more accurately describing the workpiece position features. Then, the five coordinate feature vectors of each reference workpiece are cascaded to obtain the position feature vectors of each reference workpiece. In this way, it helps the system to better understand the position features of different types of workpieces and improve the sorting accuracy.

[0050] In a specific embodiment of the present application, the reference workpiece coordinate feature acquisition subunit 124-1 is used to: use the embedding layer of the context encoder to convert each coordinate element of the five-element position coordinate into a coordinate vector to obtain a sequence of coordinate vectors; use the converter of the context encoder to process the sequence of coordinate vectors to obtain five coordinate feature vectors; and cascade the five coordinate feature vectors to obtain the reference workpiece position feature vector.

[0051] Next, considering that arranging multiple reference workpiece position feature vectors into a matrix form can better integrate the position information of different types of workpieces, based on this, the multiple reference workpiece position feature vectors are first arranged into a reference workpiece position input matrix. Then, a context encoder-based sorting position feature extraction module is used to feature encode the reference workpiece position input matrix to obtain the reference workpiece position feature matrix. Among them, the convolutional neural network can gradually extract abstract features through multi-layer convolution and pooling operations, and can automatically learn the complex relationship between features. This helps the system better understand the complexity and diversity of workpiece position features.

[0052] In a specific embodiment of the present application, the reference workpiece position feature acquisition subunit 124-2 is used to: use each layer of the two-dimensional convolutional neural network model to perform the following operations on the input data in the forward transfer of the layer: use the convolution units of each layer of the two-dimensional convolutional neural network model to perform convolution processing based on a two-dimensional convolution kernel on the input data to obtain a convolution feature map; use the pooling units of each layer of the two-dimensional convolutional neural network model to perform pooling processing on the convolution feature map along the channel dimension to obtain a pooling feature map; and use the activation units of each layer of the two-dimensional convolutional neural network model to perform nonlinear activation on the feature values ​​of each position in the pooling feature map to obtain an activation feature map; wherein the output of the last layer of the two-dimensional convolutional neural network model is the reference workpiece position feature matrix.

[0053] In an embodiment of the present application, the workpiece feature fusion module 130 to be analyzed is used to perform feature fusion on the workpiece type feature matrix to be analyzed and the reference workpiece position feature matrix to obtain the workpiece position feature matrix to be analyzed. It should be understood that the type feature and position feature of the workpiece to be analyzed are closely related. By fusing these two features, the system can better understand the position distribution of different types of workpieces in space, so as to sort and process them more accurately. Therefore, in order to more comprehensively understand the features and position information of the workpieces and improve the accuracy and efficiency of sorting, the workpiece type feature matrix to be analyzed and the reference workpiece position feature matrix are feature fused to obtain the workpiece position feature matrix to be analyzed.

[0054] In a specific embodiment of the present application, the workpiece feature fusion module 130 to be analyzed is used to fuse the workpiece type feature matrix to be analyzed and the reference workpiece position feature matrix using the following cascade formula to obtain the workpiece position feature matrix to be analyzed; wherein the cascade formula is: ,in, represents the feature matrix of the workpiece type to be analyzed, represents the reference workpiece position feature matrix, represents the position feature matrix of the workpiece to be analyzed, Represents a cascade function.

[0055] In the embodiment of the present application, the workpiece to be analyzed result generating module 140 is used to generate the target position of the workpiece to be analyzed based on the workpiece position feature matrix to be analyzed. Specifically, Figure 6 The figure shows a schematic block diagram of a workpiece result generation module to be analyzed in a cloud platform-based industrial robot intelligent control system according to an embodiment of the present application. Figure 6 As shown, the workpiece result generation module 140 to be analyzed includes: a workpiece position feature extraction unit 141 to be analyzed, which is used to pass the workpiece position feature matrix to be analyzed through the workpiece position feature extraction module to be analyzed based on the convolutional neural network model to obtain a workpiece position generation feature vector to be analyzed; a workpiece position feature optimization unit 142 to be analyzed, which is used to adjust the workpiece position generation feature vector to be analyzed based on the probability responsiveness of the distribution direction to obtain an optimized workpiece position generation feature vector to be analyzed; a workpiece position generation unit 143 to be analyzed, which is used to pass the optimized workpiece position generation feature vector to be analyzed through a generator to generate the target position of the workpiece to be analyzed.

[0056] It should be understood that only feature extraction may lead to information loss or oversimplification. When processing workpiece position features, if only simple feature extraction is performed without considering context information, the position features of the workpiece may not be fully expressed. Therefore, in order to obtain higher-level features, the workpiece position feature extraction module to be analyzed based on the convolutional neural network model is used to perform deep convolution encoding on the workpiece position feature matrix to be analyzed to obtain the workpiece position generation feature vector to be analyzed.

[0057] In particular, in the technical solution of the present application, it is considered that if the feature vector generated by the position of the workpiece to be analyzed has a high consistency in the distribution direction, the model will be more accurate in classification or recognition, because similar workpieces will produce similar feature vectors, thereby reducing the risk of misclassification. Consistent feature expression can help the model better generalize to new and unseen data. This means that the model can maintain high performance even when faced with new workpiece images or different shooting conditions. In machine learning, overfitting is a common problem, especially when using complex models. By improving the consistency of the feature vector generated by the position of the workpiece to be analyzed, the overfitting of the model to the training data can be reduced, thereby improving the reliability of the model in practical applications. In practical applications, the image of the workpiece may change due to various factors (such as light changes, shooting angles, slight deformations of the workpiece, etc.). Improving the consistency of the feature vector generated by the position of the workpiece to be analyzed can make the model more robust to these changes, thereby improving the stability of the model. Based on this, in order to improve the expression consistency of the feature vector generated by the position of the workpiece to be analyzed in its distribution direction, in the technical solution of the present application, the feature vector generated by the position of the workpiece to be analyzed is adjusted based on the probability responsiveness of the distribution direction.

[0058] Specifically, in a specific embodiment of the present application, the workpiece position feature optimization unit 142 includes: performing probability responsiveness adjustment on the workpiece position generation feature vector to be analyzed based on the distribution direction to obtain a modulation feature vector; and calculating the position point multiplication between the modulation feature vector and the workpiece position generation feature vector to be analyzed to obtain the optimized workpiece position generation feature vector to be analyzed.

[0059] Among them, the probability responsiveness adjustment based on the distribution direction is performed on the feature vector generated by the position of the workpiece to be analyzed to obtain the modulated feature vector, including: calculating the mean and variance of the feature set composed of all eigenvalues ​​of the feature vector generated by the position of the workpiece to be analyzed; using the inverse of the variance of the feature set as the exponent of the natural constant to calculate the exponential function value with the natural constant as the base to obtain a first exponential function value; subtracting one from the first exponential function value to obtain a first difference; using the mean of the feature set as the exponent of the natural constant to calculate the exponential function value with the natural constant as the base to obtain a second exponential function value; dividing the eigenvalue of the predetermined position of the feature vector generated by the position of the workpiece to be analyzed by the first difference and then subtracting the second exponential function value, and then passing through the ReLU function to obtain a first activation value; multiplying the first activation value by the eigenvalue of the predetermined position of the feature vector generated by the position of the workpiece to be analyzed, taking the absolute value, and then passing through the softmax function to obtain the eigenvalue of the predetermined position of the modulated feature vector.

[0060] In particular, the probability responsiveness adjustment based on the distribution direction of the feature vector generated by the position of the workpiece to be analyzed to obtain the modulation feature vector also includes: adjusting the probability responsiveness based on the distribution direction of the feature vector generated by the position of the workpiece to be analyzed to obtain the modulation feature vector according to the following formula, wherein the formula is: ,in, The first character vector representing the position of the workpiece to be analyzed is generated The eigenvalues ​​at the positions, and is the mean and variance of the feature set consisting of all eigenvalues ​​of the feature vector generated by the position of the workpiece to be analyzed, represents the linear rectification function, represents the normalized exponential function, The modulation feature vector The feature value at each position.

[0061] That is, in order to improve the expression consistency of the feature vector generated by the position of the workpiece to be analyzed in its distribution direction, in the technical solution of the present application, the feature vector generated by the position of the workpiece to be analyzed is adjusted in a probabilistic responsiveness based on the distribution direction. By adjusting the feature vector generated by the position of the workpiece to be analyzed in a probabilistic responsiveness, the directional recursion is activated by using the statistical characteristics of the feature distribution, so as to accurately infer the directional distribution of the feature at each sampling position. - A directional squeezing-incentive mechanism supported by Bayesian inference composed of functions is used to obtain the confidence value of the sampling position with attention enhancement, so as to improve the expression consistency of the feature vector generated by each position of the workpiece to be analyzed in its distribution direction.

[0062] Next, consider that the generator can learn complex nonlinear mapping relationships. In the industrial robot sorting task, there may be a complex nonlinear relationship between the position generation feature vector of the workpiece to be analyzed and the target position of the workpiece, and the generator can learn and map this complex relationship. Therefore, in order to more accurately analyze the workpiece sorting position, the generator is used to perform feature analysis on the optimized position generation feature vector of the workpiece to be analyzed to generate the target position of the workpiece to be analyzed. In this way, the industrial robot can perform accurate sorting operations on the workpiece to be analyzed based on the target position.

[0063] In summary, the cloud-based industrial robot intelligent control system according to the embodiment of the present application is explained, which first collects image data of the workpiece to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces as input data, and then uses computer vision and image processing technology to extract and analyze the input data to generate the target position of the workpiece to be analyzed, and then the industrial robot performs accurate sorting operations on the workpiece to be analyzed based on the target position. In this way, it is helpful to realize intelligent sorting of workpieces.

[0064] As described above, the cloud-based industrial robot intelligent control system 100 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server for intelligent control of an industrial robot based on a cloud platform. In one example, the cloud-based industrial robot intelligent control system 100 according to the embodiment of the present application can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the cloud-based industrial robot intelligent control system 100 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the cloud-based industrial robot intelligent control system 100 can also be one of the many hardware modules of the wireless terminal.

[0065] Alternatively, in another example, the cloud platform-based industrial robot intelligent control system 100 and the wireless terminal may also be separate devices, and the cloud platform-based industrial robot intelligent control system 100 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0066] Exemplary Methods

[0067] Figure 7 The figure shows a flow chart of an industrial robot intelligent control method based on a cloud platform according to an embodiment of the present application. Figure 7As shown, according to the cloud platform-based industrial robot intelligent control method of the embodiment of the present application, the method includes: S1, collecting image data of the workpiece to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces; S2, respectively performing feature extraction and analysis on the image data of the workpiece to be analyzed at different shooting angles, the image data of the different types of reference workpieces, and the five-element position coordinates of the different types of reference workpieces to obtain a feature matrix of the workpiece type to be analyzed and a feature matrix of the reference workpiece position; S3, performing feature fusion on the feature matrix of the workpiece type to be analyzed and the feature matrix of the reference workpiece position to obtain a feature matrix of the workpiece position to be analyzed; S4, generating a target position of the workpiece to be analyzed based on the feature matrix of the workpiece position to be analyzed.

[0068] Here, those skilled in the art can understand that the specific functions and operations of each step in the above-mentioned cloud platform-based industrial robot intelligent control method have been referred to above. Figure 1 The description of the cloud platform-based industrial robot intelligent control system has been introduced in detail, and therefore, its repeated description will be omitted.

[0069] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, apparatuses and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0070] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0071] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0072] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0073] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. Secondary words are used to indicate names, but not to indicate any particular order.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit of the technical solution of the present invention.

Claims

1. An industrial robot intelligent control system based on a cloud platform, characterized in that: include: The workpiece data acquisition module to be analyzed is used to collect image data of the workpiece to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces; The feature acquisition module of the workpiece to be analyzed is used to extract and analyze the image data of the workpiece to be analyzed at different shooting angles, the image data of the different types of reference workpieces, and the five-element position coordinates of the different types of reference workpieces to obtain the type feature matrix of the workpiece to be analyzed and the position feature matrix of the reference workpiece; A feature fusion module for workpieces to be analyzed, used for performing feature fusion on the feature matrix of the workpiece type to be analyzed and the feature matrix of the reference workpiece position to obtain a feature matrix of the workpiece position to be analyzed; A workpiece result generation module to be analyzed, used to generate a target position of the workpiece to be analyzed based on the position feature matrix of the workpiece to be analyzed; The module for generating the result of the workpiece to be analyzed includes: A workpiece position feature extraction unit to be analyzed, used for obtaining a workpiece position generation feature vector by passing the workpiece position feature matrix to be analyzed through a workpiece position feature extraction module to be analyzed based on a convolutional neural network model; The unit for optimizing the position characteristics of the workpiece to be analyzed is used for adjusting the probability responsiveness of the feature vector generated by the position of the workpiece to be analyzed based on the distribution direction to obtain an optimized feature vector generated by the position of the workpiece to be analyzed; A unit for generating the position of the workpiece to be analyzed, used for generating the feature vector of the optimized position of the workpiece to be analyzed through a generator to generate the target position of the workpiece to be analyzed; Wherein, the position feature optimization unit of the workpiece to be analyzed comprises: Performing probability responsiveness adjustment based on the distribution direction on the feature vector generated by the position of the workpiece to be analyzed to obtain a modulation feature vector; and Calculate the position point multiplication between the modulation feature vector and the feature vector generated by the position of the workpiece to be analyzed to obtain the optimized feature vector generated by the position of the workpiece to be analyzed; The process of generating a feature vector of the position of the workpiece to be analyzed and adjusting the probability responsiveness based on the distribution direction to obtain a modulation feature vector includes: Calculating the mean and variance of a feature set composed of all eigenvalues ​​of the feature vector generated by the position of the workpiece to be analyzed; Using the inverse of the variance of the feature set as the exponent of the natural constant, calculating an exponential function value with the natural constant as the base to obtain a first exponential function value; Subtracting one from the first exponential function value to obtain a first difference value; Using the mean value of the feature set as the exponent of the natural constant, calculating an exponential function value with the natural constant as the base to obtain a second exponential function value; The feature value of the predetermined position of the feature vector generated by the position of the workpiece to be analyzed is divided by the first difference value and then subtracted from the second exponential function value and then passed through the ReLU function to obtain a first activation value; The first activation value is multiplied by the feature value of the predetermined position of the feature vector generated by the position of the workpiece to be analyzed, and then the absolute value is taken and passed through the softmax function to obtain the feature value of the predetermined position of the modulation feature vector.

2. The cloud platform-based industrial robot intelligent control system according to claim 1 is characterized in that: The feature acquisition module for the workpiece to be analyzed includes: A feature extraction unit for the workpiece to be analyzed, used for performing feature encoding on image data of the workpiece to be analyzed at different shooting angles to obtain a global feature vector of the workpiece to be analyzed; A reference workpiece feature extraction unit, used for performing feature encoding on the different types of reference workpiece image data to obtain a reference workpiece comprehensive feature vector; A workpiece type feature acquisition unit, configured to use a joint encoder to perform feature fusion on the global feature vector of the workpiece to be analyzed and the comprehensive feature vector of the reference workpiece to obtain a feature matrix of the workpiece type to be analyzed; The reference workpiece position feature extraction unit is used to perform feature encoding on the five-element position coordinates of the different types of reference workpieces to obtain the reference workpiece position feature matrix.

3. The cloud platform-based industrial robot intelligent control system according to claim 2 is characterized in that: The workpiece feature extraction unit to be analyzed includes: An image enhancement subunit, used for respectively passing the image data of the workpiece to be analyzed at different shooting angles through an image enhancement module based on a generative adversarial network to obtain a plurality of pixel enhanced images of the workpiece to be analyzed; A subunit for extracting local features of workpieces to be analyzed, used for respectively passing the plurality of pixel-enhanced images of workpieces to be analyzed through a local feature extractor of the workpiece to be analyzed based on a deep and shallow feature fusion module to obtain a plurality of local feature matrices of the workpieces to be analyzed; The global feature extraction subunit of the workpiece to be analyzed is used to arrange the multiple local feature matrices of the workpiece to be analyzed into a global three-dimensional tensor of the workpiece to be analyzed, and then obtain the global feature vector of the workpiece to be analyzed through a global feature extraction module of the workpiece to be analyzed based on a three-dimensional convolutional neural network model.

4. The cloud platform-based industrial robot intelligent control system according to claim 3 is characterized in that: The reference workpiece feature extraction unit comprises: A reference workpiece feature acquisition subunit, used for respectively passing the different types of reference workpiece image data through a reference workpiece feature extraction module including a two-dimensional convolution kernel to obtain a plurality of reference workpiece feature matrices; The reference workpiece comprehensive feature acquisition subunit is used to arrange the multiple reference workpiece feature matrices into a reference workpiece comprehensive three-dimensional tensor and then obtain the reference workpiece comprehensive feature vector through a reference workpiece comprehensive feature extraction module based on a spatial attention mechanism.

5. The cloud platform-based industrial robot intelligent control system according to claim 4 is characterized in that: The reference workpiece position feature extraction unit comprises: A reference workpiece coordinate feature acquisition subunit, used for passing the five-element position coordinates of the different types of reference workpieces through a sorting position feature extraction module based on a context encoder to obtain five coordinate feature vectors of each reference workpiece, and cascading them to obtain multiple reference workpiece position feature vectors; The reference workpiece position feature acquisition subunit is used to arrange the multiple reference workpiece position feature vectors into a reference workpiece position input matrix and then obtain the reference workpiece position feature matrix through a reference workpiece position feature extraction module based on a two-dimensional convolutional neural network model.

6. A cloud platform-based industrial robot intelligent control method, using the cloud platform-based industrial robot intelligent control system according to claim 1, characterized in that ,include: Collect image data of the workpiece to be analyzed at different shooting angles, image data of different types of reference workpieces, and five-element position coordinates of different types of reference workpieces; Extracting and analyzing the image data of the workpiece to be analyzed at different shooting angles, the image data of the different types of reference workpieces, and the five-element position coordinates of the different types of reference workpieces to obtain a feature matrix of the workpiece type to be analyzed and a feature matrix of the reference workpiece position; Performing feature fusion on the workpiece type feature matrix to be analyzed and the reference workpiece position feature matrix to obtain the workpiece position feature matrix to be analyzed; A target position of the workpiece to be analyzed is generated based on the position feature matrix of the workpiece to be analyzed.

7. The cloud platform-based industrial robot intelligent control method according to claim 6, characterized in that: The image data of the workpiece to be analyzed at different shooting angles, the image data of the different types of reference workpieces, and the five-element position coordinates of the different types of reference workpieces are respectively extracted and analyzed to obtain a feature matrix of the workpiece type to be analyzed and a feature matrix of the reference workpiece position, including: A feature extraction unit for the workpiece to be analyzed, used for performing feature encoding on image data of the workpiece to be analyzed at different shooting angles to obtain a global feature vector of the workpiece to be analyzed; A reference workpiece feature extraction unit, used for performing feature encoding on the different types of reference workpiece image data to obtain a reference workpiece comprehensive feature vector; A workpiece type feature acquisition unit, configured to use a joint encoder to perform feature fusion on the global feature vector of the workpiece to be analyzed and the comprehensive feature vector of the reference workpiece to obtain a feature matrix of the workpiece type to be analyzed; The reference workpiece position feature extraction unit is used to perform feature encoding on the five-element position coordinates of the different types of reference workpieces to obtain the reference workpiece position feature matrix.