Target recognition and positioning method for intelligent collaborative robots based on deep learning

Abstract

The present invention discloses a target recognition and positioning method for an intelligent collaborative robot based on deep learning, and relates to the field of data processing technology. The method comprises the following steps: acquiring data of an image to be recognized to obtain target recognition parameter data; analyzing the target recognition parameter data to obtain the existence of a mechanical workpiece to be recognized and positioning it; analyzing the edge similarity of the mechanical workpiece to be recognized; analyzing the angle similarity of the mechanical workpiece to be recognized through the target recognition parameter data; recognizing the mechanical workpiece to be recognized through deep learning to obtain a recognition result; and combining the edge similarity, angle similarity and recognition result to realize the recognition of the mechanical workpiece to be recognized. The present invention realizes the recognition of the mechanical workpiece to be recognized by combining the edge similarity, angle similarity and recognition result, thereby achieving the effect of accurately identifying the mechanical workpiece to be recognized, and solves the problem in the prior art that it is difficult to accurately and efficiently locate and recognize the mechanical workpiece.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a target recognition and positioning method of an intelligent collaborative robot based on deep learning. Background Art

[0002] Mechanical workpieces are everywhere in modern society and are indispensable. The mechanical manufacturing industry has been expanded on a large scale with the progress of the times. In the mechanical manufacturing process, the identification and positioning of mechanical workpieces is one of the key technologies in modern manufacturing, which involves automated processing, quality control, production efficiency and other aspects. In the traditional mechanical processing field, workers complete the positioning and processing of workpieces through visual inspection and manual adjustment. This method is limited by people's reaction speed and accuracy, and is prone to errors due to fatigue. With the development of science and technology, especially the advancement of computer vision, sensor technology and artificial intelligence algorithms, intelligent robots have emerged, which can be used to identify and locate mechanical workpieces.

[0003] The existing mechanical workpiece identification and positioning system attaches a QR code or RFID tag to the mechanical workpiece, reads the information on the tag through a scanner, obtains the specific information and location of the mechanical workpiece, and realizes the identification and positioning function of the mechanical workpiece.

[0004] For example, the invention patent with announcement number: CN113775939B announces an online identification and positioning method for new leakage in a water supply network, which includes two parts: an offline module and an online module. In the offline module, the response threshold and level of the monitoring system to new leakage events are determined by statistical analysis of the historical data of the monitoring points, and the hydraulic state of the new leakage is previewed by the hydraulic model of the network, so as to construct a database of alarm response patterns for new leakage in the network. In the online module, the real-time monitoring data of the pressure monitoring points is subjected to over-threshold discrimination and pattern matching to realize online identification and rapid positioning of new leakage events.

[0005] For example, the invention patent with announcement number: CN116663785B announces a low-voltage fault location method based on topology recognition, including: step S1: establish a fault topology map, obtain fault data in real time, and determine the corresponding initial problem set based on the fault data and the fault topology map; step S2: mark the initial area according to the initial problem set and the fault data; step S3: identify the initial area characteristics, modify the initial problem set based on the initial area characteristics, and obtain the pending problem set; step S4: reduce the initial area according to the pending problem set and the fault data to obtain the positioning area; step S5: dispatch corresponding maintenance personnel to perform fault positioning and maintenance according to the positioning area.

[0006] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0007] In the prior art, a high-resolution camera is used to capture an image of a workpiece, and a machine learning algorithm is used to identify the type of workpiece. However, in the process of acquiring the image, different lighting conditions, ambient light, dust, vibration and other factors may cause the content of the workpiece image to change, thereby affecting the accuracy of recognition. When problems such as stains and scratches appear on the surface of the workpiece, the effect of recognition and positioning may also be affected. In a workpiece with a multi-layer structure or a combination of multiple components, some components may block other components, affecting the accuracy of recognition and positioning, resulting in inaccurate positioning and recognition results of the workpiece, and obtaining erroneous positioning and recognition results. As a result, it is difficult to accurately and efficiently position and identify mechanical workpieces. Summary of the invention

[0008] The embodiments of the present application solve the problem in the prior art that it is difficult to accurately and efficiently locate and identify mechanical workpieces by providing a target recognition and positioning method for an intelligent collaborative robot based on deep learning, thereby achieving accurate and efficient positioning and identification of mechanical workpieces.

[0009] An embodiment of the present application provides a target recognition and positioning method for an intelligent collaborative robot based on deep learning, comprising the following steps: acquiring an image to be recognized by an intelligent collaborative robot; acquiring data of the image to be recognized to obtain target recognition parameter data; analyzing the target recognition parameter data to obtain the existence of a mechanical workpiece to be recognized and positioning it; after locating the mechanical workpiece to be recognized, analyzing the edge similarity of the mechanical workpiece to be recognized through the target recognition parameter data; analyzing the angle similarity of the mechanical workpiece to be recognized through the target recognition parameter data; recognizing the mechanical workpiece to be recognized through deep learning to obtain a recognition result; and recognizing the mechanical workpiece to be recognized by combining the edge similarity, angle similarity and recognition result.

[0010] Furthermore, the specific analysis process of analyzing the target recognition parameter data to obtain the existence of the mechanical workpiece to be identified and locate it is: obtaining regional abnormality parameter data based on the target recognition parameter data; preprocessing the regional abnormality parameter data; obtaining a regional abnormality degree evaluation index based on the analysis of the preprocessed regional abnormality parameter data, and the regional abnormality degree evaluation index is used to reflect the possibility of the existence of the mechanical workpiece to be identified; analyzing the existence of the mechanical workpiece to be identified based on the regional abnormality degree evaluation index and locating it.

[0011] Furthermore, the specific method for obtaining the regional abnormality degree evaluation index is as follows: obtaining the number of images to be identified and numbering the images to be identified; dividing the images to be identified into regions to be identified and numbering the regions to be identified; performing multiple data collection on the regions to be identified and numbering the data collection times; obtaining regional abnormality parameter data; constructing a calculation formula for the regional abnormality degree evaluation index according to the regional abnormality parameter data; the specific calculation formula for the regional abnormality degree evaluation index is:

[0012]

[0013] In the formula, Expressed as s 0 The qth image to be recognized 0 The evaluation index of the degree of regional abnormality in the area to be identified, s 0 =1,2,...,s,s 0 represents the number of the image to be identified, s represents the total number of images to be identified, and q 0 =1,2,...,q,q 0 represents the number of the area to be identified, q represents the total number of areas to be identified, Expressed as s 0 The qth image to be recognized 0 The area to be identified is in the c 0 The number of pixels at the first brightness level during data collection, c 0 =1,2,...,c,c 0 represents the number of data collection times, c represents the total number of data collection times, Expressed as s 0 The qth image to be recognized 0 The area to be identified is in the c 0 The number of pixels at the second brightness level during the data collection, Expressed as s 0 The qth image to be recognized 0 The area to be identified is in the c 0 The number of pixels at the third brightness level during the data collection, Expressed as s 0 The qth standard image corresponding to the image to be recognized 0 The standard value of the number of pixels of the first brightness level in the area to be identified, Expressed as s 0 The qth standard image corresponding to the image to be recognized 0 The standard value of the number of pixels of the second brightness level in the area to be identified, Expressed as s 0 The qth standard image corresponding to the image to be recognized 0The standard value of the number of pixels at the third brightness level in the area to be identified, B 1 It is expressed as the weight ratio of the number of pixels at the first brightness level in the evaluation index of the degree of regional abnormality, B 2 It is expressed as the weight ratio of the number of pixels at the second brightness level in the evaluation index of the degree of regional abnormality, B 3 It is expressed as the weight ratio of the number of pixels at the third brightness level in the evaluation index of the degree of regional abnormality.

[0014] Furthermore, the specific analysis process of analyzing the existence of the mechanical workpiece to be identified and locating it according to the regional abnormality degree evaluation index is as follows: obtaining the regional abnormality degree evaluation index, obtaining the first regional abnormality threshold and the second regional abnormality threshold, comparing the regional abnormality degree evaluation index with the first regional abnormality threshold and the second regional abnormality threshold; when the regional abnormality degree evaluation index is less than the first regional abnormality threshold or the regional abnormality degree evaluation index is greater than the second regional abnormality threshold, it means that the mechanical workpiece to be identified exists in the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index, and the mechanical workpiece to be identified is located, and the position of the mechanical workpiece to be identified is the actual position corresponding to the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index; when the regional abnormality degree evaluation index is not less than the first regional abnormality threshold and the regional abnormality degree evaluation index is not greater than the second regional abnormality threshold, it means that the mechanical workpiece to be identified does not exist in the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index, and subsequent analysis is stopped.

[0015] Furthermore, the specific analysis process of analyzing the edge similarity of the mechanical workpiece to be identified through the target recognition parameter data after locating the mechanical workpiece to be identified is: obtaining the workpiece edge similarity parameter data through the target recognition parameter data; obtaining the workpiece edge similarity evaluation index according to the workpiece edge similarity parameter data, and the workpiece edge similarity evaluation index is used to reflect the degree of similarity between the mechanical workpiece to be identified and the type of mechanical workpiece in terms of the edge; analyzing the edge similarity of the mechanical workpiece to be identified according to the workpiece edge similarity evaluation index.

[0016] Furthermore, the specific method for obtaining the workpiece edge similarity evaluation index is as follows: obtaining the number of mechanical workpieces to be identified in the to-be-identified area of ​​the to-be-identified image, and numbering the mechanical workpieces to be identified in the to-be-identified area of ​​the to-be-identified image; performing multiple data collection on the to-be-identified area, and numbering the number of data collection times; obtaining all mechanical workpiece types, and numbering the mechanical workpiece types; obtaining workpiece edge similarity parameter data; constructing a workpiece edge similarity evaluation index calculation formula according to the workpiece edge similarity parameter data; the specific workpiece edge similarity evaluation index calculation formula is:

[0017]

[0018] In the formula, Denote the jth 0 The mechanical workpiece to be identified and the zth 0 The workpiece edge similarity evaluation index of the mechanical workpiece type, j 0 =1,2,...,j,j 0 represents the number of the mechanical workpiece to be identified, j represents the total number of mechanical workpieces to be identified, and z 0 =1,2,...,z,z 0 It represents the number of the mechanical workpiece type, z represents the total number of mechanical workpiece types, Denote the jth 0 The maximum edge gradient strength of the mechanical workpiece to be identified, Denote the jth 0 The minimum edge gradient strength of the mechanical workpiece to be identified, Denote the jth 0 The mechanical workpiece to be identified is in the c 0 The total edge length during data collection, c 0 =1,2,...,c,c 0 represents the number of data collection times, c represents the total number of data collection times, Represented as z 0 The maximum edge gradient strength reference value of the mechanical workpiece type, Represented as z 0 The minimum edge gradient strength reference value for the type of mechanical workpiece, Represented as z 0 Reference value of total edge length of mechanical workpiece type, D 1 It is expressed as the weight ratio of the maximum edge gradient intensity in the workpiece edge similarity evaluation index, D 2 It is expressed as the weight ratio of the total edge length in the workpiece edge similarity evaluation index.

[0019] Furthermore, the specific analysis process of analyzing the edge similarity of the mechanical workpiece to be identified based on the workpiece edge similarity evaluation index is: obtaining the workpiece edge similarity evaluation index, obtaining the first edge evaluation threshold and the second edge evaluation threshold; when the workpiece edge similarity evaluation index is less than the first edge evaluation threshold and the workpiece edge similarity evaluation index is greater than the second edge evaluation threshold, it indicates that the edge of the mechanical workpiece to be identified is similar to that of the mechanical workpiece type; when in other situations, it indicates that the edge of the mechanical workpiece to be identified is not similar to that of the mechanical workpiece type.

[0020] Further, the specific analysis process of analyzing the angle similarity of the mechanical workpiece to be identified through the target recognition parameter data is: obtaining the inner angle similarity parameter data according to the target recognition parameter data; normalizing the inner angle similarity parameter data; analyzing the normalized inner angle similarity parameter data to obtain the workpiece inner angle similarity evaluation index, wherein the workpiece inner angle similarity evaluation index is used to reflect the similarity between the mechanical workpiece to be identified and the type of mechanical workpiece in terms of angle; analyzing the angle similarity of the mechanical workpiece to be identified according to the workpiece inner angle similarity evaluation index; the specific analysis process of analyzing the angle similarity of the mechanical workpiece to be identified according to the workpiece inner angle similarity evaluation index is: obtaining the workpiece inner angle similarity evaluation index, obtaining the first inner angle similarity evaluation threshold and the second inner angle similarity evaluation threshold, comparing the workpiece inner angle similarity evaluation index with the first inner angle similarity evaluation threshold and the second inner angle similarity evaluation threshold, when the workpiece inner angle similarity evaluation index is greater than the first inner angle similarity evaluation threshold and less than the second inner angle similarity evaluation threshold, it indicates that the angle of the mechanical workpiece to be identified and the type of mechanical workpiece are similar; when in other situations, it indicates that the angle of the mechanical workpiece to be identified and the type of mechanical workpiece are not similar.

[0021] Furthermore, the specific analysis process of identifying the mechanical workpiece to be identified by deep learning to obtain the identification result is: obtaining image data of each type of mechanical workpiece at various angles, and marking the image data of the same type of mechanical workpiece with the type of mechanical workpiece and the bounding box; preprocessing the image data; using the image data after preprocessing to train a mask region-based convolutional neural network to obtain a Mask R-CNN model; obtaining an image to be identified in which the mechanical workpiece to be identified exists, and inputting the image to be identified into the Mask R-CNN model to obtain the recognition result.

[0022] Furthermore, the specific analysis process of combining edge similarity and angle similarity to realize the identification of the mechanical workpiece to be identified is: obtaining edge similarity, angle similarity and identification result; when the edge similarity indicates that the edge of the mechanical workpiece to be identified is similar to that of the mechanical workpiece type and the angle similarity indicates that the angle of the mechanical workpiece to be identified is similar to that of the mechanical workpiece type, checking the type of the mechanical workpiece when the edge is similar, the type of the mechanical workpiece when the angle is similar and the type of the mechanical workpiece indicated by the identification result; when the type of the mechanical workpiece when the edge is similar, the type of the mechanical workpiece when the angle is similar and the type of the mechanical workpiece indicated by the identification result are the same, it indicates that the mechanical workpiece to be identified is the type of mechanical workpiece, and the identification of the mechanical workpiece to be identified is realized; when in other situations, repeating the process of obtaining the image to be identified by the intelligent collaborative robot to realize the identification of the mechanical workpiece to be identified by combining edge similarity and angle similarity until the identification of the mechanical workpiece to be identified is realized, and then ending the identification process.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0024] 1. Target recognition parameter data is obtained by acquiring data from the image to be identified, and the target recognition parameter data is analyzed to obtain the existence of the mechanical workpiece to be identified and locate it. After locating the mechanical workpiece to be identified, the edge similarity of the mechanical workpiece to be identified is analyzed through the target recognition parameter data, and the angle similarity of the mechanical workpiece to be identified is analyzed through the target recognition parameter data. The mechanical workpiece to be identified is identified through deep learning to obtain the identification result, thereby combining the edge similarity, angle similarity and recognition result to realize the identification of the mechanical workpiece to be identified, and then accurately identifying the mechanical workpiece to be identified from three aspects: edge similarity, angle similarity and recognition result, effectively solving the problem of difficulty in accurately and efficiently positioning and identifying mechanical workpieces in the prior art.

[0025] 2. By obtaining regional abnormality parameter data based on target identification parameter data, preprocessing the regional abnormality parameter data, and obtaining regional abnormality degree evaluation indicators based on the analysis of the preprocessed regional abnormality parameter data, the existence of the mechanical workpiece to be identified is analyzed and located according to the regional abnormality degree evaluation indicators, thereby achieving accurate positioning of the mechanical workpiece to be identified.

[0026] 3. Obtain workpiece edge similarity parameter data through target recognition parameter data, obtain workpiece edge similarity evaluation index based on workpiece edge similarity parameter data, and then analyze the edge similarity of the mechanical workpiece to be identified based on the workpiece edge similarity evaluation index, thereby realizing accurate analysis of the edge similarity between the mechanical workpiece to be identified and the type of mechanical workpiece. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flow chart of a target recognition and positioning method for an intelligent collaborative robot based on deep learning provided in an embodiment of the present application;

[0028] Figure 2 A graph of regional abnormality degree assessment indicators provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The embodiments of the present application solve the problem in the prior art that it is difficult to accurately and efficiently locate and identify mechanical workpieces by providing a target recognition and positioning method for an intelligent collaborative robot based on deep learning. Target recognition parameter data is obtained by acquiring data from an image to be identified, and the target recognition parameter data is analyzed to obtain the existence of the mechanical workpiece to be identified and locate it. After locating the mechanical workpiece to be identified, the edge similarity of the mechanical workpiece to be identified is analyzed through the target recognition parameter data, and the angle similarity of the mechanical workpiece to be identified is analyzed through the target recognition parameter data. The mechanical workpiece to be identified is identified through deep learning to obtain a recognition result, thereby combining the edge similarity, angle similarity and recognition result to realize the recognition of the mechanical workpiece to be identified, thereby realizing accurate recognition of the mechanical workpiece to be identified from three aspects: edge similarity, angle similarity and recognition result.

[0030] The technical solution in the embodiment of the present application is to solve the above-mentioned problem that it is difficult to accurately and efficiently locate and identify mechanical workpieces. The overall idea is as follows:

[0031] The image to be identified is acquired through an intelligent collaborative robot; data of the image to be identified is acquired to obtain target identification parameter data; regional abnormality parameter data is obtained based on the target identification parameter data; the regional abnormality parameter data is preprocessed; the regional abnormality degree evaluation index is obtained based on the analysis of the preprocessed regional abnormality parameter data; the existence of the mechanical workpiece to be identified is analyzed and located based on the regional abnormality degree evaluation index, and after locating the mechanical workpiece to be identified, the workpiece edge similarity parameter data is obtained through the target identification parameter data; the workpiece edge similarity evaluation index is obtained based on the workpiece edge similarity parameter data; the edge similarity of the mechanical workpiece to be identified is analyzed based on the workpiece edge similarity evaluation index. similarity; obtain inner angle similarity parameter data according to target recognition parameter data; normalize the inner angle similarity parameter data; analyze the normalized inner angle similarity parameter data to obtain the workpiece inner angle similarity evaluation index; analyze the angle similarity of the mechanical workpiece to be identified according to the workpiece inner angle similarity evaluation index; identify the mechanical workpiece to be identified through deep learning to obtain the recognition result, and the specific process is: obtain image data of each type of mechanical workpiece at various angles, and annotate the image data of the same type of mechanical workpiece with the type of mechanical workpiece and the bounding box; preprocess the image data; use the image data after preprocessing to train the Mask R-CNN model through the mask region-based convolutional neural network to obtain the Mask R-CNN model; obtain the image to be identified with the mechanical workpiece to be identified, and input the image to be identified into the Mask R-CNN model to obtain the recognition result; combine the edge similarity, angle similarity and recognition result to realize the recognition of the mechanical workpiece to be identified, and achieve the effect of accurately and efficiently positioning and identifying the mechanical workpiece.

[0032] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0033] like Figure 1 As shown, it is a flow chart of the target recognition and positioning method of the intelligent collaborative robot based on deep learning provided in an embodiment of the present application, and the method includes the following steps: acquiring an image to be recognized by an intelligent collaborative robot; acquiring data of the image to be recognized to obtain target recognition parameter data; analyzing the target recognition parameter data to obtain the existence of the mechanical workpiece to be recognized and positioning it; after locating the mechanical workpiece to be recognized, analyzing the edge similarity of the mechanical workpiece to be recognized by the target recognition parameter data; analyzing the angle similarity of the mechanical workpiece to be recognized by the target recognition parameter data; recognizing the mechanical workpiece to be recognized by deep learning to obtain a recognition result; and realizing the recognition of the mechanical workpiece to be recognized by combining the edge similarity, angle similarity and recognition result.

[0034] In this embodiment, the specific process of acquiring data of the image to be identified to obtain the target recognition parameter data is: obtaining the image to be identified through a camera, scanner or other image acquisition equipment, preprocessing the image to be identified, including adjusting the brightness, contrast, color balance, etc. of the image, segmenting the image to be identified into multiple parts or objects, extracting features that are helpful for identification and classification from the segmented image, including shape, texture, color and other features, and converting the extracted features into numerical values, so as to obtain the target recognition parameter data.

[0035] Furthermore, the specific analysis process of analyzing the target recognition parameter data to obtain the existence of the mechanical workpiece to be identified and positioning it is as follows: obtaining regional abnormality parameter data based on the target recognition parameter data; preprocessing the regional abnormality parameter data; obtaining a regional abnormality degree evaluation index based on the analysis of the preprocessed regional abnormality parameter data, and the regional abnormality degree evaluation index is used to reflect the possibility of the existence of the mechanical workpiece to be identified; analyzing the existence of the mechanical workpiece to be identified based on the regional abnormality degree evaluation index and positioning it.

[0036] In this embodiment, the regional abnormal situation parameter data is preprocessed, including: performing data cleaning on the regional abnormal situation parameter data, deleting missing data, ensuring that there is no duplicate data, identifying and processing abnormal or noise data; performing data conversion on the regional abnormal situation parameter data, scaling the data to a small range, such as between 0 and 1, to eliminate the influence of different dimensions. By preprocessing the regional abnormal situation parameter data, the accuracy and reliability of the regional abnormal situation parameter data are higher, which is conducive to obtaining a more accurate regional abnormal situation degree evaluation indicator.

[0037] Furthermore, the regional abnormality degree evaluation index can be obtained by a variety of methods, including but not limited to: obtaining a color image of the area to be identified, converting the color image into a grayscale image, converting the grayscale image into a binary image, and using methods such as Gaussian filtering to remove image noise, then using the Sobel operator to detect the edge in the image, extracting the shape features in the image through geometric invariant moments, and using local binary patterns (LBP) to extract texture features, and combining the above indicators for comprehensive evaluation to obtain the regional abnormality degree evaluation index. The regional abnormality degree evaluation index can also be obtained by calculation. The specific method for obtaining the regional abnormality degree evaluation index is: obtaining the number of images to be identified and numbering the images to be identified; dividing the images to be identified into regions to be identified and numbering the regions to be identified; performing multiple data collection on the regions to be identified and numbering the number of data collection times; obtaining regional abnormality parameter data; constructing a regional abnormality degree evaluation index calculation formula based on the regional abnormality parameter data; the specific regional abnormality degree evaluation index calculation formula is:

[0038]

[0039] In the formula, Expressed as s 0 The qth image to be recognized 0 The evaluation index of the degree of regional abnormality in the area to be identified, s 0 =1,2,...,s,s 0 represents the number of the image to be identified, s represents the total number of images to be identified, and q 0 =1,2,...,q,q 0 represents the number of the area to be identified, q represents the total number of areas to be identified, Expressed as s 0 The qth image to be recognized 0 The area to be identified is in the c 0 The number of pixels at the first brightness level during data collection, c 0 =1,2,...,c,c 0 represents the number of data collection times, c represents the total number of data collection times, Expressed as s 0 The qth image to be recognized 0 The area to be identified is in the c 0 The number of pixels at the second brightness level during the data collection, Expressed as s 0 The qth image to be recognized 0 The area to be identified is in the c 0 The number of pixels at the third brightness level during the data collection, Expressed as s 0The qth standard image corresponding to the image to be recognized 0 The standard value of the number of pixels of the first brightness level in the area to be identified, Expressed as s 0 The qth standard image corresponding to the image to be recognized 0 The standard value of the number of pixels of the second brightness level in the area to be identified, Expressed as s 0 The qth standard image corresponding to the image to be recognized 0 The standard value of the number of pixels at the third brightness level in the area to be identified, B 1 It is expressed as the weight ratio of the number of pixels at the first brightness level in the evaluation index of the degree of regional abnormality, B 2 It is expressed as the weight ratio of the number of pixels at the second brightness level in the evaluation index of the degree of regional abnormality, B 3 It is expressed as the weight ratio of the number of pixels at the third brightness level in the evaluation index of the degree of regional abnormality.

[0040] In this embodiment, by consulting experts in the field of image processing, weight ratios are assigned to the number of pixels at the first brightness level, the number of pixels at the second brightness level, and the number of pixels at the third brightness level according to the experts' knowledge and experience, and the obtained B 1 , B 2 and B 3 The specific value of B 1 , B 2 and B 3 The value range of is (0, 1) and satisfies B 1 , B 2 and B 3 The sum is equal to 1.

[0041] An image to be identified is obtained, which is a color image. Grayscale processing is performed on the image to be identified to obtain a grayscale image. Image processing is performed on the grayscale image to obtain a brightness histogram. The first brightness level, the second brightness level and the third brightness level are found from the brightness histogram, and the number of pixels corresponding to the first brightness level, the second brightness level and the third brightness level are read to obtain the number of pixels at the first brightness level, the number of pixels at the second brightness level and the number of pixels at the third brightness level. The actual area corresponding to the image to be identified may contain a mechanical workpiece to be identified. The actual area is kept in a state where there is no mechanical workpiece to be identified. A standard image is obtained under the same conditions as the image to be identified. The standard image is processed by the above method to obtain a standard value of the number of pixels at the first brightness level, the standard value of the number of pixels at the second brightness level and the standard value of the number of pixels at the third brightness level.

[0042] The value range of brightness level is [0, 255]. The number of pixels of the first brightness level, the number of pixels of the second brightness level and the number of pixels of the third brightness level are the number of pixels corresponding to the three brightness levels. For example, the brightness levels corresponding to the first brightness level, the second brightness level and the third brightness level can be 50, 120 and 200 respectively. For the same image, the total number of pixels is certain, and the sum of the number of pixels of each brightness level is the total number of pixels. Therefore, if one of the numbers of pixels of the first brightness level, the second brightness level and the third brightness level is large, the other numbers will be reduced, and vice versa. In this embodiment, the algorithm analyzes the regional abnormality degree evaluation index from three aspects: the number of pixels of the first brightness level, the number of pixels of the second brightness level and the number of pixels of the third brightness level, thereby avoiding the problem of large errors in the regional abnormality degree evaluation index due to incomplete consideration. In addition, the algorithm accurately obtains the regional abnormality degree evaluation index through the linear transformation of the hyperbolic sine function, thereby improving the accuracy and stability of the regional abnormality degree evaluation index.

[0043] like Figure 2 As shown, it is a curve chart of the regional abnormality degree evaluation index provided in the embodiment of the present application, with the number of pixels at the first brightness level as the independent variable and the regional abnormality degree evaluation index as the dependent variable. Through the increase and decrease analysis of Curve 1, Curve 2 and Curve 3, it can be known that when the number of pixels at the first brightness level is closer to the standard value of the number of pixels at the first brightness level, the larger the regional abnormality degree evaluation index is, and the higher the possibility that there is a mechanical artifact to be identified in the area to be identified; when the number of pixels at the first brightness level is smaller than the standard value of the number of pixels at the first brightness level, the smaller the regional abnormality degree evaluation index is, and the lower the possibility that there is a mechanical artifact to be identified in the area to be identified. The statistical table of relevant data of the regional abnormality degree evaluation index is shown in Table 1:

[0044] Table 1 Statistical table of relevant data of regional abnormal situation evaluation indicators

[0045]

[0046]

[0047] It can be seen from Table 1 that in Curve 1, when other parameters are consistent, the closer the number of pixels at the first brightness level is to the standard value of the number of pixels at the first brightness level, that is, when the number of pixels at the first brightness level is 35, 40, and 45, the regional abnormality degree evaluation index is 0.98, 1.00, and 1.02, respectively, and the regional abnormality degree evaluation index gradually increases; similarly, when the number of pixels at the first brightness level is smaller than the standard value of the number of pixels at the first brightness level, the smaller the regional abnormality degree evaluation index is, and the lower the possibility that there is a mechanical workpiece to be identified in the area to be identified. The above changes also apply to Curve 2 and Curve 3.

[0048] For curve one, curve two and curve three, when the number of pixels of the first brightness level of curve one, curve two and curve three is 40, when the number of pixels of the second brightness level and the number of pixels of the third brightness level of curve one, curve two and curve three are 50 and 25, 38 and 19, and 30 and 10 respectively, and other parameters are consistent, the regional abnormality degree evaluation indexes of curve one, curve two and curve three are 1.00, 0.94 and 0.89 respectively. It can be seen that when the number of pixels at the first brightness level is consistent with other parameters, the greater the difference between the number of pixels at the second brightness level and the number of pixels at the third brightness level and the standard value of the number of pixels at the second brightness level and the standard value of the number of pixels at the third brightness level, the smaller the regional abnormality degree evaluation index is.

[0049] Furthermore, the specific analysis process for analyzing the existence of the mechanical workpiece to be identified and locating it according to the regional abnormality degree evaluation index is as follows: obtaining the regional abnormality degree evaluation index, obtaining the first regional abnormality threshold and the second regional abnormality threshold, comparing the regional abnormality degree evaluation index with the first regional abnormality threshold and the second regional abnormality threshold; when the regional abnormality degree evaluation index is less than the first regional abnormality threshold or the regional abnormality degree evaluation index is greater than the second regional abnormality threshold, it means that the mechanical workpiece to be identified exists in the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index, and the mechanical workpiece to be identified is located, and the position of the mechanical workpiece to be identified is the actual position corresponding to the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index; when the regional abnormality degree evaluation index is not less than the first regional abnormality threshold and the regional abnormality degree evaluation index is not greater than the second regional abnormality threshold, it means that the mechanical workpiece to be identified does not exist in the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index, and subsequent analysis is stopped.

[0050] In this embodiment, the first regional abnormality threshold and the second regional abnormality threshold are obtained by consulting experts, and the first regional abnormality threshold and the second regional abnormality threshold are obtained. The first regional abnormality threshold and the second regional abnormality threshold are μ respectively. 1 =0.76 and μ2 =1.37, compare the regional abnormality level assessment index with the regional abnormality first threshold and the regional abnormality second threshold. When the regional abnormality level assessment index is less than the regional abnormality first threshold or the regional abnormality level assessment index is greater than the regional abnormality second threshold, that is, or It indicates that there is a mechanical workpiece to be identified in the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index, and the mechanical workpiece to be identified is located, and the position of the mechanical workpiece to be identified is the actual position corresponding to the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index; when the regional abnormality degree evaluation index is not less than the first regional abnormality threshold and the regional abnormality degree evaluation index is not greater than the second regional abnormality threshold, that is, When , it means that there is no mechanical workpiece to be identified in the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index, and subsequent analysis is stopped.

[0051] Furthermore, after locating the mechanical workpiece to be identified, the specific analysis process of analyzing the edge similarity of the mechanical workpiece to be identified through the target identification parameter data is as follows: obtaining the workpiece edge similarity parameter data through the target identification parameter data; obtaining the workpiece edge similarity evaluation index according to the workpiece edge similarity parameter data, and the workpiece edge similarity evaluation index is used to reflect the degree of similarity between the mechanical workpiece to be identified and the type of mechanical workpiece in terms of the edge; analyzing the edge similarity of the mechanical workpiece to be identified according to the workpiece edge similarity evaluation index.

[0052] In this embodiment, by obtaining the workpiece edge similarity evaluation index based on the workpiece edge similarity parameter data, the workpiece edge similarity evaluation index is considered more comprehensively, which is conducive to obtaining a more accurate workpiece edge similarity evaluation index and facilitating a more detailed analysis of edge similarity situations.

[0053] Furthermore, the workpiece edge similarity evaluation index can be obtained by the following method, including: obtaining a color image corresponding to the mechanical workpiece, converting the color image into a grayscale image, reducing the amount of data and highlighting the edge information, using Gaussian filtering to remove noise from the grayscale image, using the Sobel operator to detect the edge in the grayscale image, extracting edge strength, position, direction and other features, calculating the Euclidean distance between the features to evaluate the similarity, and obtaining the workpiece edge similarity evaluation index. The workpiece edge similarity evaluation index can also be obtained through a specific calculation process. The specific method for obtaining the workpiece edge similarity evaluation index is: obtaining the number of mechanical workpieces to be identified in the to-be-identified area of ​​the to-be-identified image, and numbering the mechanical workpieces to be identified in the to-be-identified area of ​​the to-be-identified image; performing multiple data collection on the to-be-identified area, and numbering the number of data collection times; obtaining all mechanical workpiece types, and numbering the mechanical workpiece types; obtaining workpiece edge similarity parameter data; constructing a workpiece edge similarity evaluation index calculation formula based on the workpiece edge similarity parameter data; the specific workpiece edge similarity evaluation index calculation formula is:

[0054]

[0055] In the formula, Denote the jth 0 The mechanical workpiece to be identified and the zth 0 The workpiece edge similarity evaluation index of the mechanical workpiece type, j 0 =1,2,...,j,j 0 represents the number of the mechanical workpiece to be identified, j represents the total number of mechanical workpieces to be identified, and z 0 =1,2,...,z,z 0 It represents the number of the mechanical workpiece type, z represents the total number of mechanical workpiece types, Denote the jth 0 The maximum edge gradient strength of the mechanical workpiece to be identified, Denote the jth 0 The minimum edge gradient strength of the mechanical workpiece to be identified, Denote the jth 0 The mechanical workpiece to be identified is in the c 0 The total edge length during data collection, c 0 =1,2,...,c,c 0 represents the number of data collection times, c represents the total number of data collection times, Represented as z 0 The maximum edge gradient strength reference value of the mechanical workpiece type, Represented as z 0 The minimum edge gradient strength reference value for the type of mechanical workpiece, Represented as z 0Reference value of total edge length of mechanical workpiece type, D 1 It is expressed as the weight ratio of the maximum edge gradient intensity in the workpiece edge similarity evaluation index, D 2 It is expressed as the weight ratio of the total edge length in the workpiece edge similarity evaluation index.

[0056] In this embodiment, workpiece historical data are collected, including the maximum edge gradient intensity, the minimum edge gradient intensity, the maximum edge gradient intensity reference value, the minimum edge gradient intensity reference value, the total edge length and the total edge length reference value, etc. The workpiece historical data are cleaned, missing values ​​and outliers are processed, and all workpiece historical data are converted into numerical data for quantitative analysis. Features that are helpful in analyzing the edge similarity of workpieces are extracted from the workpiece historical data, such as the maximum edge gradient intensity and the total edge length. An objective function is determined to measure the accuracy of the recommendation system, such as the mean square error (MSE) or the cross-entropy loss. An initial weight is randomly assigned to each feature, and the iterative optimization process is started using the gradient descent algorithm. In each iteration, the algorithm updates the weight according to the prediction error. If a feature contributes more to the workpiece edge similarity evaluation index, then the weight associated with the feature will be larger, thereby giving the feature a higher importance in future predictions. After each iteration, the performance of the recommendation system is evaluated by the value of the objective function. If the performance is no longer improved, the iteration can be stopped. Finally, a set of optimal weights is obtained, that is, D 1 and D 2 The specific value of D 1 and D 2 The value range of is (0, 1) and satisfies D 1 and D 2 The sum is equal to 1.

[0057] The image to be identified is converted into a grayscale image. Since the actual captured image may be affected by noise, a denoising algorithm (such as Gaussian filtering, median filtering, etc.) is needed to remove the noise points in the grayscale image. The edge detection algorithm is used to identify the edge in the image. Common edge detection algorithms include Sobel operator, etc. The specific process of using Sobel operator for edge recognition is as follows: create Sobel operators in horizontal and vertical directions, perform convolution operations on the image, calculate the horizontal and vertical components of the gradient respectively, merge the two components into an amplitude value, and adjust an appropriate threshold value based on experience and the specific content of the image to identify the edge and realize the edge recognition function of the image. The edge detection algorithm usually calculates the gradient intensity of each pixel in the image. The gradient intensity reflects the degree of change in the brightness of the pixel neighborhood. All gradient intensities are arranged in order from large to small, and the gradient intensities at the first and last place are taken as the maximum edge gradient intensity and the minimum edge gradient intensity. Use the edge detection algorithm to identify the edge in the image, obtain the position of the edge point, connect the detected edge points, and form an edge contour line. The total number of pixels in the edge contour line is the total edge length. By analyzing each type of mechanical workpiece according to the above process, the maximum edge gradient intensity reference value, the minimum edge gradient intensity reference value and the total edge length reference value are obtained.

[0058] The gradient intensity reflects the degree of change in brightness within the neighborhood of a pixel point. The faster the gradient intensity changes, the more obvious the contour of the mechanical workpiece is. Therefore, under normal circumstances, when the maximum edge gradient intensity is greater and the minimum edge gradient intensity is smaller, the total edge length is greater, and vice versa. The algorithm of this embodiment uses a nonlinear transformation of an exponential function to smooth the influence of outliers, reduce the distortion of the overall evaluation by extreme cases, and make the calculated workpiece edge similarity evaluation index more stable and reliable. In addition, the workpiece edge similarity evaluation index is obtained by analyzing the maximum edge gradient intensity, the minimum edge gradient intensity and the total edge length, so that the workpiece edge similarity evaluation index is more comprehensive.

[0059] Furthermore, the specific analysis process of analyzing the edge similarity of the mechanical workpiece to be identified based on the workpiece edge similarity evaluation index is: obtaining the workpiece edge similarity evaluation index, obtaining the first edge evaluation threshold and the second edge evaluation threshold, when the workpiece edge similarity evaluation index is less than the first edge evaluation threshold and the workpiece edge similarity evaluation index is greater than the second edge evaluation threshold, it indicates that the edge of the mechanical workpiece to be identified is similar to that of the mechanical workpiece type; when in other situations, it indicates that the edge of the mechanical workpiece to be identified is not similar to that of the mechanical workpiece type.

[0060] In this embodiment, the workpiece edge similarity evaluation index is explored through data visualization, statistical analysis and other methods to understand the distribution of the workpiece edge similarity evaluation index, including mean, median, standard deviation, quantile, etc., and the first edge evaluation threshold and the second edge evaluation threshold are obtained in combination with the distribution of the workpiece edge similarity evaluation index. The workpiece edge similarity evaluation index is obtained, and the first edge evaluation threshold and the second edge evaluation threshold are obtained. The first edge evaluation threshold and the second edge evaluation threshold are θ respectively. 1 and θ 2 , when the workpiece edge similarity evaluation index is less than the first edge evaluation threshold and the workpiece edge similarity evaluation index is greater than the second edge evaluation threshold, that is, When , it means that the edge of the mechanical workpiece to be identified is similar to that of the mechanical workpiece type; when it is in other situations, that is, or , it indicates that the edges of the mechanical workpiece to be identified are not similar to the type of mechanical workpiece.

[0061] Furthermore, the specific analysis process of analyzing the angle similarity of the mechanical workpiece to be identified through the target recognition parameter data is as follows: obtaining the inner angle similarity parameter data according to the target recognition parameter data; normalizing the inner angle similarity parameter data; analyzing the normalized inner angle similarity parameter data to obtain the workpiece inner angle similarity evaluation index, the workpiece inner angle similarity evaluation index is used to reflect the similarity degree between the mechanical workpiece to be identified and the type of mechanical workpiece in terms of angle; analyzing the angle similarity of the mechanical workpiece to be identified according to the workpiece inner angle similarity evaluation index; the specific analysis process of analyzing the angle similarity of the mechanical workpiece to be identified according to the workpiece inner angle similarity evaluation index is as follows: obtaining the workpiece inner angle similarity evaluation index, obtaining the first inner angle similarity evaluation threshold and the second inner angle similarity evaluation threshold, comparing the workpiece inner angle similarity evaluation index with the first inner angle similarity evaluation threshold and the second inner angle similarity evaluation threshold, when the workpiece inner angle similarity evaluation index is greater than the first inner angle similarity evaluation threshold and less than the second inner angle similarity evaluation threshold, it indicates that the angle of the mechanical workpiece to be identified and the type of mechanical workpiece are similar; when in other situations, it indicates that the angle of the mechanical workpiece to be identified and the type of mechanical workpiece are not similar.

[0062] In this embodiment, the specific process of normalizing the inner angle similarity parameter data is to use Min-Max Normalization or Z-Score Normalization to normalize the inner angle similarity parameter data, usually scaling the data to a smaller interval, such as [0, 1] or [-1, 1]. Different data in the inner angle similarity parameter data may have different dimensions and orders of magnitude, and normalization can eliminate this effect. Through normalization, excessive dependence on a certain data can be reduced. The normalized inner angle similarity parameter data is easier to interpret because the relationship between the data of the inner angle similarity parameter data is more consistent.

[0063] The workpiece internal angle similarity evaluation index can be obtained by the following methods, including: using a 3D scanner or camera array and other equipment to obtain the three-dimensional data of the mechanical workpiece, processing the three-dimensional data, reconstructing the three-dimensional model of the workpiece and extracting the internal angle features such as the size, distribution, and shape of the internal angle, comparing the internal angle features of the standard workpiece with the internal angle features of the workpiece to be evaluated, calculating the Euclidean distance between the internal angle features to evaluate the similarity, and obtaining the workpiece internal angle similarity evaluation index. The workpiece internal angle similarity evaluation index can also be obtained by calculation. The specific workpiece internal angle similarity evaluation index calculation formula is:

[0064]

[0065] In the formula, Denote the jth 0 The mechanical workpiece to be identified and the zth 0 Similarity evaluation index of workpiece inner angles of various types of mechanical workpieces, Denote the jth 0 The maximum internal angle of the mechanical workpiece to be identified, Denote the jth 0 The minimum internal angle of the mechanical workpiece to be identified, Denote the jth 0 The mechanical workpiece to be identified is in the c 0 The highest altitude at the time of data collection, Represented as z 0 The maximum internal angle reference value of the mechanical workpiece type, Represented as z 0 The minimum internal angle reference value of the mechanical workpiece type, Represented as z 0 The highest height reference value of the mechanical workpiece type, w 1 It is expressed as the weight ratio of the maximum inner angle in the workpiece inner angle similarity evaluation index, w 2 It is expressed as the weight ratio of the minimum internal angle in the workpiece internal angle similarity evaluation index, w3 It is expressed as the weight ratio of the highest height in the workpiece inner corner similarity evaluation index.

[0066] Collect a set of historical data, including the maximum internal angle, the minimum internal angle, the highest height, the maximum internal angle reference value, the minimum internal angle reference value and the highest height reference value, standardize the historical data, and use statistical software to apply the principal component analysis method (PCA) to analyze the standardized historical data. PCA will find the main components in the historical data, which can explain most of the data variation. PCA will output a component matrix showing the degree of association between each indicator and each component. The columns of the component matrix represent the components, and the rows represent the original indicators. The value of each element represents the contribution of the indicator to the corresponding component. The values ​​of each row of the component matrix can be regarded as the weight of the indicator, because they represent the relative importance of each indicator to the workpiece internal angle similarity evaluation index. The higher the weight, the greater the influence of the indicator on the workpiece internal angle similarity evaluation index. Through the above analysis process, the specific values ​​of the weight ratios of the maximum internal angle, the minimum internal angle and the highest height are obtained, that is, w 1 、w 2 and w 3 The specific value of w 1 、w 2 and w 3 The value range of is (0, 1), and satisfies w 1 、w 2 and w 3 The sum is equal to 1.

[0067] Before obtaining the image to be identified, 3D recognition technology is used to obtain specific data. The specific process is: by using a three-dimensional scanner to obtain the point cloud data of the workpiece, and then processing it through the point cloud processing software to obtain a three-dimensional model of the workpiece, and analyzing this model through computer-aided design software (Computer Aided Design, CAD) or three-dimensional measurement software to obtain the size, angle and curvature of the workpiece. Through the above process, the mechanical workpiece to be identified is analyzed to obtain all the internal angles, and all the internal angles are arranged in order from large to small. The first and last internal angles are used as the maximum internal angle and the minimum internal angle, that is, the maximum internal angle and the minimum internal angle are obtained. Through the above process, the mechanical workpiece to be identified is analyzed to obtain all heights, and the heights are arranged in order from large to small, and the first height is used as the maximum height. By analyzing each type of mechanical workpiece according to the same analysis process as the mechanical workpiece to be identified, the maximum internal angle reference value, the minimum internal angle reference value and the highest height reference value are obtained.

[0068] The maximum inner angle and the minimum inner angle reflect the shape characteristics of the mechanical workpiece to be identified to a certain extent. For example, the maximum inner angle and the minimum inner angle of a mechanical workpiece to be identified are both 90 degrees, and the maximum inner angle and the minimum inner angle of a mechanical workpiece type are both 120 degrees. The mechanical workpiece to be identified is extremely dissimilar to the mechanical workpiece type in terms of angle. By performing similarity analysis on the mechanical workpiece to be identified from the maximum inner angle, the minimum inner angle, and the highest height, the effect of analyzing the similarity between the mechanical workpiece to be identified and the mechanical workpiece type from a three-dimensional level is achieved. The algorithm of this embodiment uses the linear transformation of the hyperbolic cosine function to smooth the influence of outliers and reduce the distortion of the overall evaluation by extreme cases, so that the calculated workpiece inner angle similarity evaluation index is more stable and reliable, which is conducive to more accurately analyzing the similarity between the mechanical workpiece to be identified and the mechanical workpiece type in terms of angle according to the workpiece inner angle similarity evaluation index. In addition, multiple related but different measurement data are converted into a single progress index. This embodiment uses a quantitative method to evaluate and compare the similarity between different mechanical workpieces to be identified and the mechanical workpiece types, so that the evaluation method is more consistent.

[0069] The first and second thresholds of the similarity evaluation of the inner angles are obtained by quantile compression of the similarity evaluation index of the inner angles of the workpieces, and the first and second thresholds of the similarity evaluation of the inner angles are obtained. The first and second thresholds of the similarity evaluation of the inner angles are τ 1 and τ 2 , compare the workpiece inner corner similarity evaluation index with the first inner corner similarity evaluation threshold and the second inner corner similarity evaluation threshold. When the workpiece inner corner similarity evaluation index is greater than the first inner corner similarity evaluation threshold and less than the second inner corner similarity evaluation threshold, that is, When , it means that the angle of the mechanical workpiece to be identified is similar to that of the mechanical workpiece type; when in other situations, that is, or , it indicates that the angles of the mechanical workpiece to be identified and the type of mechanical workpiece are not similar.

[0070] Furthermore, the specific analysis process of identifying the mechanical workpiece to be identified through deep learning to obtain the identification result is: obtaining image data of each type of mechanical workpiece at various angles, and marking the image data of the same type of mechanical workpiece with the type of mechanical workpiece and the bounding box; preprocessing the image data; using the image data after preprocessing to train a mask region-based convolutional neural network to obtain a Mask R-CNN model; obtaining an image to be identified with a mechanical workpiece to be identified, and inputting the image to be identified into the Mask R-CNN model to obtain the recognition result.

[0071] In this embodiment, the English name of the mask region-based convolutional neural network is Mask R-CNN (Mask Region-based Convolutional Neural Network). The image data is preprocessed, for example, by resizing, normalizing, and cropping to meet the requirements of model training. In order to improve the generalization ability of the model, the image data is enhanced by random rotation, flipping, scaling, and cropping.

[0072] Furthermore, the specific analysis process for realizing the recognition of the mechanical workpiece to be identified by combining the edge similarity and the angle similarity is as follows: obtaining the edge similarity, the angle similarity and the recognition result; when the edge similarity indicates that the edge of the mechanical workpiece to be identified is similar to that of the mechanical workpiece type and the angle similarity indicates that the angle of the mechanical workpiece to be identified is similar to that of the mechanical workpiece type, checking the type of the mechanical workpiece when the edge is similar, the type of the mechanical workpiece when the angle is similar, and the type of the mechanical workpiece indicated by the recognition result; when the type of the mechanical workpiece when the edge is similar, the type of the mechanical workpiece when the angle is similar, and the type of the mechanical workpiece indicated by the recognition result are the same, it indicates that the mechanical workpiece to be identified is the type of mechanical workpiece, and the recognition of the mechanical workpiece to be identified is realized; when in other situations, repeating the process from obtaining the image to be identified by the intelligent collaborative robot to realizing the recognition of the mechanical workpiece to be identified by combining the edge similarity and the angle similarity until the recognition of the mechanical workpiece to be identified is realized, the recognition process is terminated.

[0073] In this embodiment, when the types of mechanical workpieces to be identified obtained in the three situations are the same, it means that the mechanical workpiece to be identified is accurately identified and does not need to be repeatedly identified. When the results of the three situations are inconsistent, it means that the mechanical workpiece to be identified is not accurately identified and needs to be repeatedly identified until it is accurately identified.

[0074] The technical solution in the above-mentioned embodiment of the present application has at least the following technical effects or advantages: by obtaining regional abnormality parameter data based on target identification parameter data, preprocessing the regional abnormality parameter data, and obtaining a regional abnormality degree evaluation index based on the analysis of the preprocessed regional abnormality parameter data, thereby analyzing the existence of the mechanical workpiece to be identified and locating it according to the regional abnormality degree evaluation index, thereby achieving accurate positioning of the mechanical workpiece to be identified.

[0075] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0079] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0080] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A target recognition and positioning method for an intelligent collaborative robot based on deep learning, characterized in that: The following steps are involved: Acquire the image to be recognized through the intelligent collaborative robot; Acquire data of the image to be identified to obtain target identification parameter data; Analyze the target recognition parameter data to obtain the existence of the mechanical workpiece to be identified and locate it; After locating the mechanical workpiece to be identified, analyzing the edge similarity of the mechanical workpiece to be identified through target identification parameter data; Analyze the angle similarity of the mechanical workpiece to be identified through target recognition parameter data; The mechanical workpiece to be identified is identified through deep learning to obtain the identification result; The mechanical workpiece to be identified is identified by combining edge similarity, angle similarity and recognition results; The specific analysis process of analyzing the target recognition parameter data to obtain the existence of the mechanical workpiece to be identified and locate it is as follows: Obtaining regional abnormality parameter data according to target recognition parameter data; Preprocessing of regional abnormality parameter data; Obtaining a regional abnormality degree evaluation index based on analysis of the pre-processed regional abnormality parameter data, wherein the regional abnormality degree evaluation index is used to reflect the possibility of the existence of the mechanical workpiece to be identified; Analyze the existence of the mechanical workpiece to be identified and locate it according to the evaluation index of the degree of regional abnormality; The specific analysis process of analyzing the existence of the mechanical workpiece to be identified and locating it according to the regional abnormality degree evaluation index is as follows: Obtain a regional abnormality degree assessment index, obtain a regional abnormality first threshold and a regional abnormality second threshold, compare the regional abnormality degree assessment index with the regional abnormality first threshold and the regional abnormality second threshold, when the regional abnormality degree assessment index is less than the regional abnormality first threshold or the regional abnormality degree assessment index is greater than the regional abnormality second threshold, it indicates that there is a mechanical workpiece to be identified in the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree assessment index, locate the mechanical workpiece to be identified, and the position of the mechanical workpiece to be identified is the actual position corresponding to the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree assessment index; When the regional abnormality degree evaluation index is not less than the first regional abnormality threshold and the regional abnormality degree evaluation index is not greater than the second regional abnormality threshold, it means that there is no mechanical workpiece to be identified in the to-be-identified area of ​​the to-be-identified image corresponding to the regional abnormality degree evaluation index, and subsequent analysis is stopped.

2. The target recognition and positioning method of the intelligent collaborative robot based on deep learning as claimed in claim 1, characterized in that: The specific method for obtaining the evaluation index of the degree of abnormality in the region is as follows: Obtain the number of images to be identified and number the images to be identified; Dividing the image to be identified into regions to be identified, and numbering the regions to be identified; Perform multiple data collection on the area to be identified, and number the data collection times; Obtain regional abnormal condition parameter data; Construct a calculation formula for the evaluation index of the degree of regional abnormality based on the regional abnormality parameter data; The specific calculation formula for the regional abnormality degree evaluation index is: ； In the formula, Expressed as The first image to be recognized The evaluation index of the degree of regional anomaly in the area to be identified, , Represents the number of the image to be identified. Represented as the total number of images to be recognized, , It represents the number of the area to be identified. It is represented as the total number of regions to be identified. Expressed as The first image to be recognized The area to be identified is The number of pixels at the first brightness level during data collection, , It represents the number of data collection times. It is expressed as the total number of data collection times, Expressed as The first image to be recognized The area to be identified is The number of pixels at the second brightness level during the data collection, Expressed as The first image to be recognized The area to be identified is The number of pixels at the third brightness level during the data collection, Expressed as The first standard image corresponding to the image to be recognized The standard value of the number of pixels of the first brightness level in the area to be identified, Expressed as The first standard image corresponding to the image to be recognized The standard value of the number of pixels of the second brightness level in the area to be identified, Expressed as The first standard image corresponding to the image to be recognized The standard value of the number of pixels at the third brightness level in the area to be identified, It is expressed as the weight ratio of the number of pixels at the first brightness level in the evaluation index of the degree of regional abnormality, It is expressed as the weight ratio of the number of pixels at the second brightness level in the evaluation index of the degree of regional abnormality, It is expressed as the weight ratio of the number of pixels at the third brightness level in the evaluation index of the degree of regional abnormality.

3. The target recognition and positioning method of the intelligent collaborative robot based on deep learning as claimed in claim 1, characterized in that: The specific analysis process of analyzing the edge similarity of the mechanical workpiece to be identified through the target recognition parameter data after locating the mechanical workpiece to be identified is: Obtain workpiece edge similarity parameter data through target recognition parameter data; Obtaining a workpiece edge similarity evaluation index according to the workpiece edge similarity parameter data, wherein the workpiece edge similarity evaluation index is used to reflect the similarity between the mechanical workpiece to be identified and the type of mechanical workpiece in terms of the edge; The edge similarity of the mechanical workpiece to be identified is analyzed based on the workpiece edge similarity evaluation index.

4. The target recognition and positioning method of an intelligent collaborative robot based on deep learning as claimed in claim 3, characterized in that: The specific method for obtaining the workpiece edge similarity evaluation index is: Acquire the number of mechanical workpieces to be identified in the area to be identified of the image to be identified, and number the mechanical workpieces to be identified in the area to be identified of the image to be identified; Perform multiple data collection on the area to be identified, and number the data collection times; Obtain all types of mechanical workpieces and number them; Obtaining workpiece edge similarity parameter data; According to the workpiece edge similarity parameter data, a workpiece edge similarity evaluation index calculation formula is constructed; The specific calculation formula of the workpiece edge similarity evaluation index is: ； In the formula, Expressed as The mechanical workpiece to be identified and The workpiece edge similarity evaluation index of the mechanical workpiece types, , Indicates the number of the mechanical workpiece to be identified. It is expressed as the total number of mechanical workpieces to be identified, , Indicates the number of the type of mechanical workpiece. Expressed as the total number of mechanical workpiece types, Expressed as The maximum edge gradient strength of the mechanical workpiece to be identified, Expressed as The minimum edge gradient strength of the mechanical workpiece to be identified, Expressed as The mechanical workpiece to be identified is The total edge length during data collection, , It represents the number of data collection times. It is expressed as the total number of data collection times, Expressed as The maximum edge gradient strength reference value of the mechanical workpiece type, Expressed as The minimum edge gradient strength reference value for the type of mechanical workpiece, Expressed as Reference value of total edge length of various types of mechanical workpieces, It is expressed as the weight ratio of the maximum edge gradient intensity in the workpiece edge similarity evaluation index. It is expressed as the weight ratio of the total edge length in the workpiece edge similarity evaluation index.

5. The target recognition and positioning method of an intelligent collaborative robot based on deep learning as claimed in claim 4, characterized in that: The specific analysis process of analyzing the edge similarity of the mechanical workpiece to be identified according to the workpiece edge similarity evaluation index is as follows: Obtaining a workpiece edge similarity evaluation index, obtaining a first edge evaluation threshold and a second edge evaluation threshold, when the workpiece edge similarity evaluation index is less than the first edge evaluation threshold and the workpiece edge similarity evaluation index is greater than the second edge evaluation threshold, it indicates that the edges of the mechanical workpiece to be identified are similar to the mechanical workpiece type; In other cases, it indicates that the edge of the mechanical workpiece to be identified is not similar to the mechanical workpiece type.

6. The target recognition and positioning method of an intelligent collaborative robot based on deep learning as claimed in claim 1, characterized in that: The specific analysis process of analyzing the angle similarity of the mechanical workpiece to be identified through the target identification parameter data is as follows: Obtaining inner angle similarity parameter data according to target recognition parameter data; Normalize the inner angle similarity parameter data; Analyzing the normalized internal angle similarity parameter data to obtain a workpiece internal angle similarity evaluation index, wherein the workpiece internal angle similarity evaluation index is used to reflect the similarity between the mechanical workpiece to be identified and the type of mechanical workpiece in terms of angle; Analyze the angle similarity of the mechanical workpiece to be identified based on the workpiece internal angle similarity evaluation index; The specific analysis process of analyzing the angle similarity of the mechanical workpiece to be identified according to the workpiece internal angle similarity evaluation index is as follows: Obtaining a workpiece inner angle similarity evaluation index, obtaining an inner angle similarity evaluation first threshold and an inner angle similarity evaluation second threshold, comparing the workpiece inner angle similarity evaluation index with the inner angle similarity evaluation first threshold and the inner angle similarity evaluation second threshold, when the workpiece inner angle similarity evaluation index is greater than the inner angle similarity evaluation first threshold and less than the inner angle similarity evaluation second threshold, it indicates that the angles of the mechanical workpiece to be identified and the type of the mechanical workpiece are similar; In other cases, it indicates that the angles of the mechanical workpiece to be identified and the type of mechanical workpiece are not similar.

7. The target recognition and positioning method of an intelligent collaborative robot based on deep learning as claimed in claim 1, characterized in that: The specific analysis process of obtaining the recognition result by deep learning for the mechanical workpiece to be recognized is as follows: Acquire image data of each type of mechanical workpiece at each angle, and annotate the image data of the same type of mechanical workpiece with the type of mechanical workpiece and a bounding box; Preprocess the image data; After preprocessing, the image data is used to train the Mask R-CNN model through a mask region-based convolutional neural network. An image to be identified containing a mechanical workpiece to be identified is obtained, and the image to be identified is input into the Mask R-CNN model to obtain a recognition result.

8. The target recognition and positioning method of an intelligent collaborative robot based on deep learning as claimed in claim 1, characterized in that: The specific analysis process of combining edge similarity and angle similarity to realize the recognition of the mechanical workpiece to be recognized is: Obtain edge similarity, angle similarity and recognition result. When edge similarity indicates that the edges of the mechanical workpiece to be recognized are similar to those of the mechanical workpiece type, and angle similarity indicates that the angles of the mechanical workpiece to be recognized are similar to those of the mechanical workpiece type, check the mechanical workpiece type when the edges are similar, the mechanical workpiece type when the angles are similar, and the mechanical workpiece type indicated by the recognition result. When the mechanical workpiece type when the edges are similar, the mechanical workpiece type when the angles are similar, and the mechanical workpiece type indicated by the recognition result are the same, it indicates that the mechanical workpiece to be recognized is the mechanical workpiece type, thereby realizing recognition of the mechanical workpiece to be recognized. When in other situations, the intelligent collaborative robot is used to repeatedly obtain the image to be identified and the process of identifying the mechanical workpiece to be identified is realized by combining the edge similarity and the angle similarity until the mechanical workpiece to be identified is identified, and then the identification process is terminated.

Images