A visual recognition method for laser cleaning parts
By dynamically adjusting the parameters and feature matching strategies, the difficulty of identifying the laser cleaning system in high stains and bright light environments is solved, and more efficient and accurate workpiece recognition is achieved.
Patent Information
- Application Number
- CN202410536834.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-30
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-04-30
AI Technical Summary
When identifying workpieces to be cleaned at the bottom of the rail train, existing laser cleaning systems face difficulties in identifying high stains and bright light environments, resulting in slow recognition speed and low accuracy.
A laser cleaning component visual recognition method is adopted, and the parameters are dynamically adjusted to improve the accuracy and efficiency of workpiece recognition through the steps of model library construction, feature point analysis, search path analysis, feature probability regulation, model matching update and local cleaning.
The speed and accuracy of workpiece recognition is improved, and efficient recognition in high stain and bright environments is achieved. It has faster model updates and higher registration efficiency than traditional methods.
Smart Images

Figure CN118447489B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of laser cleaning, and more particularly to a method for visually identifying parts cleaned by laser. Background Art
[0002] When rail transit vehicles reach a specified time of operation, the corresponding parts need to be repaired. The means of maintenance is cleaning. The purpose of cleaning is to remove oil, rust, etc. on the surface and inner wall. The current automated laser cleaning is applied to the visual system, robot-assisted system and laser generation system. The robot-assisted system operates the laser cleaning head to move according to the set cleaning trajectory.
[0003] At present, many workpieces to be cleaned need to be placed on the workbench and pushed into the cleaning room. The cleaning robot needs to identify the type, front and back sides, and contour boundaries of the workpieces to be cleaned. The existing identification method is relatively simple. The surface boundary coordinate points of the workpieces to be cleaned are located, and all coordinate points are connected to obtain the contour boundaries. However, when the surface of the workpieces to be cleaned at the bottom of the rail train is heavily stained or the light is bright, it is easy to make the contour boundaries of the workpieces to be cleaned on the workbench difficult to easily identify, which brings certain difficulties to the automatic cleaning of the cleaning robot. Summary of the invention
[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a laser cleaning parts visual recognition method, which can improve the subsequent workpiece recognition speed and ensure the accuracy of workpiece recognition.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] A method for visually identifying parts cleaned by laser, comprising the following steps:
[0007] The model library construction step divides the historical image dataset into proportions for training, testing, and validating various artifact models;
[0008] The feature point analysis step is to obtain a visually captured image of the workpiece to be cleaned as a target image, and obtain a key feature in the target image as a target analysis feature. The target analysis feature corresponds to a subdivision feature. The subdivision probability of each feature is calculated by a constant rate formula according to the target analysis feature. The feature subdivision probability reflects the probability of the target analysis feature matching each subdivision feature.
[0009] The search path analysis step selects several workpiece models in the model library corresponding to the maximum value of the feature segmentation probability, and selects other key features based on the target image to search for them in these workpiece models;
[0010] In the feature probability control step, if the search fails, the microscopic parameters in the constant rate formula are adjusted to reduce the probability value of the target analysis feature in the workpiece to be cleaned this time as the previous subdivision feature, and then the maximum probability of each subdivision feature after adjustment is re-selected, and the search path analysis step is repeated;
[0011] In the artifact model output step, if the search is successful, the artifact model is used as the target model.
[0012] Furthermore, it also includes a model matching update step. If the search fails in the retrieval path analysis step, the macro parameters in the constant rate formula are adjusted to reduce the probability value of the target analysis feature in the next workpiece to be cleaned being the last subdivided feature.
[0013] Furthermore, in the retrieval path analysis step, other key features in the target image are sorted by distance factors or significant factors to obtain a feature sorting table, and key features with higher rankings are preferentially searched in the workpiece model through a search strategy.
[0014] Furthermore, the search strategy includes a feature point type search sub-step and a feature point direction search sub-step.
[0015] The feature point type finding sub-step analyzes whether the workpiece model contains other key features in the feature sorting table. If the workpiece model contains all key features, the feature point direction finding sub-step is performed. If any key feature is missing in the workpiece model, the workpiece model is ignored. If all workpiece models are ignored, the feature probability control step is performed.
[0016] The feature point direction finding sub-step establishes a plane coordinate system in the target image, extracts the coordinate point information of the target analysis features and key features, compares the relative directions and distances of the key features in the target image and the target analysis features according to the coordinate point information, obtains the relative directions and distances of various other key features and subdivision features in the workpiece model, and compares the two. If they are all registered one by one, the workpiece model output step is performed. If any registration fails, the feature probability control step is performed.
[0017] Furthermore, the feature probability control step also includes a local cleaning sub-step.
[0018] In the local cleaning sub-step, if the result of the retrieval path analysis step is a search failure, the laser cleaning head is controlled to perform local cleaning on the target analysis feature, the visually captured image of the workpiece to be cleaned is re-acquired as an updated image, and the feature point analysis step is performed again.
[0019] Furthermore, it also includes a model synchronization matching step,
[0020] In the model synchronization matching step, the visual workpiece to be cleaned is framed according to the boundary to obtain a target frame, the target frame is used to preliminarily screen out a large group of matching workpiece models in the model library through a matching strategy, the intersections between the workpiece to be cleaned and the target frame are extracted, and the preliminarily screened workpiece models are rescreened again according to the number of the intersections to obtain a workpiece model group, the intersections are calculated through a calculation strategy to obtain a workpiece structure diagram, and the workpiece structure diagram and the workpiece model group are used to calculate the target model according to the similarity.
[0021] Furthermore, the matching strategy includes calculating the length-to-width ratio of the target frame line, and screening a large group of workpiece models that match the length-to-width ratio in the model library.
[0022] Furthermore, the calculation strategy includes connecting the intersections, calculating the distance between every two intersections, calculating the distance between vertices in the workpiece model group, and analyzing whether the ratio of the distance between the intersections is the same as the ratio of the distance between the vertices.
[0023] Furthermore, it also includes a model verification step, comparing the target model in the workpiece model output step with the target model in the model synchronization matching step, if the two are the same, then determine the output, if the two are not consistent, then record a feedback warning.
[0024] The beneficial effects of the present invention are as follows: the target analysis features in the target image are calculated by macro parameters and micro parameters to obtain the probability of meeting various subdivision features, and the macro parameters and micro parameters can be considered to be in a similarity relationship, and then the workpiece model containing the subdivision feature is extracted according to the subdivision feature with the highest probability, and other features are used for registration, the purpose of which is to identify the workpiece, and then the front and back sides of the workpiece can be identified according to the feature registration on this basis, wherein when the search fails during the registration process, the micro parameters can be adjusted dynamically and in real time, that is, when the subdivision feature probability is calculated again, the subdivision feature with the highest probability last time will have a lower probability to improve the recognition accuracy, and correspondingly, the target analysis features of the workpiece to be cleaned next time can be adjusted to the macro parameters of the subdivision feature last time to improve the recognition efficiency of subsequent workpieces to be cleaned, and compared with the existing model registration means, the model is updated quickly, and the registration efficiency and accuracy are high. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a method step diagram of the present invention;
[0026] Figure 2 It is a schematic diagram of feature search in the present invention;
[0027] Figure 3 It is a step diagram of synchronous verification in the present invention. DETAILED DESCRIPTION
[0028] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The same parts are represented by the same reference numerals. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "bottom surface" and "top surface", "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.
[0029] Since the existing workpiece recognition method is relatively simple, the surface boundary coordinate points of the workpiece to be cleaned are located, and all the coordinate points are connected to obtain the contour boundary. However, when the surface of the workpiece to be cleaned at the bottom of the rail train is highly stained, or the light is bright, it is easy to make the contour boundary of the workpiece to be cleaned on the workbench difficult to easily identify, which brings certain difficulties to the automatic cleaning of the cleaning robot. Therefore, the present invention designs this laser cleaning parts visual recognition method, such as Figure 1 As shown, the following steps are included:
[0030] The model library construction step is to divide the historical image data set into proportions for training, testing and verifying various workpiece models; specifically, ①, obtain a large number of data on the placement of parts to be cleaned in different states (front and back) on the workbench through visual sensors;
[0031] Different parts to be cleaned are fixed at their respective specific positions on the workbench, and the area array camera follows the robotic arm to collect data under the same field of view during the actual cleaning process.
[0032] ②、Component target detection data labeling;
[0033] During the laser cleaning process, the complete workbench is divided into four fields of view for trajectory design of the robot arm to perform cleaning work. These four fields of view overlap. In order to prevent the parts from being detected repeatedly, Label Img annotation software is used to annotate the category and position rectangular frame of each part to ensure that each part can be correctly identified as the foreground and classified. Note that the front and back sides of the same part should be marked into two categories during annotation. The Label Img output label format adopts the txt format of the YOLO detection model to record the category and normalized position coordinate information of each component to facilitate the call during model training.
[0034] ③. Production of information templates for each field of view component;
[0035] A complete set of views is selected as the position template of the workpiece to be cleaned, and the parts to be inspected in the four fields of view are determined respectively, that is, the parts to be inspected in field one are determined, and no longer inspected in other fields of view, and so on, to prevent state detection conflicts. In addition, the template also needs to record the position coordinate information of the front and back sides of each part and the part status category.
[0036] ④. Create a workpiece status detection data set;
[0037] The annotated txt tags and paired image data are randomly shuffled and divided into training set, validation set and test set in a ratio of 7:2:1 for training and evaluation of the component status detection model.
[0038] ⑤、Workpiece target detection model training;
[0039] First, build a target detection model, and use the data set described in ③ to train the target detection network to obtain the basic model of the component status detection model. The target detection network uses YOLOv8-S, which can use advanced features for detection at different scales, and uses a single-stage end-to-end framework based on a small number of parameters. It can ensure high efficiency while achieving accurate identification of targets of different sizes, and can achieve high-performance real-time detection.
[0040] The feature point analysis step obtains the visually captured image of the workpiece to be cleaned as the target image, obtains a key feature in the target image as the target analysis feature, the target analysis feature corresponds to a subdivision feature, and the subdivision probability of each feature is calculated according to the target analysis feature by a constant rate formula, and the feature subdivision probability reflects the probability that the target analysis feature matches each subdivision feature; assuming that the first identified key feature in the target image is hole a, the hole is used as the target analysis feature, but the hole types counted in the various workpiece models in the model library include hole A1, hole A2, hole A3, etc., and the constant rate formula includes a similarity formula or a fast approximation algorithm, etc. The purpose of the constant rate formula is to quickly give the probability values of hole a meeting hole A1, hole A2, and hole A3, respectively, and the probability values correspond to: hole A1-45%, hole A2-30%, hole A3-25%.
[0041] In the retrieval path analysis step, several workpiece models corresponding to the maximum value of feature subdivision probability are selected in the model library, and other key features are selected based on the target image to be searched in these workpiece models; that is, the workpiece model with the largest feature subdivision probability value should be hole A1-45%, and the corresponding workpiece model in the model library includes workpiece T1, workpiece T2, workpiece T3, etc., and other key features such as bosses and grooves are found in the target image. Similarly, the bosses, grooves and other features are analyzed by repeating the feature point analysis steps and searched in workpieces T1, workpiece T2, and workpiece T3. In addition, other key features can also be analyzed from workpieces T1, workpiece T2, and workpiece T3 respectively, and these key features are searched in the target image;
[0042] like Figure 2As shown, in the retrieval path analysis step, other key features in the target image are sorted by distance factors or significant factors to obtain a feature sorting table, and the key features with the highest sorting are preferentially searched in the workpiece model through a search strategy, wherein the search strategy includes a feature point type search sub-step and a feature point direction search sub-step. In the feature point type search sub-step, whether the workpiece model contains other key features in the feature sorting table is analyzed. If the workpiece model contains all the key features, the feature point direction search sub-step is performed. If any key feature is missing in the workpiece model, the workpiece model is ignored. If all workpiece models are ignored, the feature probability control step is performed.
[0043] The feature point direction finding sub-step establishes a plane coordinate system in the target image, extracts the coordinate point information of the target analysis features and key features, compares the relative direction and distance between each key feature in the target image and the target analysis features according to the coordinate point information, obtains the relative direction and distance of various other key features and subdivision features in the workpiece model, and compares the two. If they are all registered one by one, the workpiece model output step is performed. If any registration fails, the feature probability control step is performed;
[0044] Assuming that other key features in the target image include hole b, hole c and boss a, the distance values between other key features and the target analysis feature are sorted, and the order is: hole b, boss a, hole c. If they are sorted according to significant factors, the order is: hole b, hole c, boss a. When workpieces T1, workpiece T2, and workpiece T3 are looking for the above other key features, they are searched in order according to the sequence table. When the result is that workpiece T1 fails to find hole b, workpiece T1 is eliminated and ignored. At this time, workpiece T2 and workpiece T3 are searched. It can also be that workpiece T1, workpiece T2, and workpiece T3 are searched at the same time. When only workpiece T2 is successfully found, subsequent direction and position alignment is performed. According to the established plane coordinate system, the coordinate points of the target analysis feature and the coordinate points of hole b, boss a, and hole c are obtained, and then the direction angles between hole b, boss a, hole c and the target analysis feature are calculated, and then compared with the direction angles between the target analysis feature found in workpiece T2 and each key feature. If they are within the error value, it means that workpiece T2 is the target model to be found.
[0045] Correspondingly, when workpieces T1, T2, and T3 all fail in finding key features, or when the feature direction angle in the subsequent workpiece T2 is different from the direction angle in the target image, it means that workpieces T1, T2, and T3 all fail in finding, and then the subsequent feature probability control step is entered.
[0046] In the feature probability control step, if the search fails, the microscopic parameters in the constant rate formula are adjusted to reduce the probability value of the target analysis feature in the workpiece to be cleaned as the previous subdivision feature, and then the maximum probability of each subdivision feature after adjustment is re-selected, and the retrieval path analysis step is repeated; when the search fails, it means that the probability of hole a meeting hole A1 is too high, and the parameters in the constant rate formula need to be adjusted. The microscopic parameters are first adjusted, and the microscopic parameters can be understood as dynamic parameters. It is assumed that the probability values of hole a meeting hole A1, hole A2, and hole A3 are re-given at this time, that is, hole A1-27%, hole A2-30%, and hole A3-25%. At this time, only the probability of hole A1 is reduced, and the retrieval path analysis step is performed on hole A2. If it has not been found, the probability of hole A2 is reduced, and the retrieval path analysis step is performed on hole A3.
[0047] In addition, there is another processing logic. During the first feature point analysis step, the probabilities of hole A1, hole A2, and hole A3 are divided into hole A1-45%, hole A2-30%, and hole A3-25%. At this time, hole A1 is used as the basis for searching the model library, and workpiece T1, workpiece T2, workpiece T3...workpiece T6 are indexed, a total of 6 groups, and then a coefficient such as half of the number of workpieces is used for analysis. Then workpiece T1, workpiece T2, and workpiece T3 are analyzed. When the search fails in workpiece T1, workpiece T2, and workpiece T3, and the probability needs to be re-given, hole A1-35%, hole A2-30%, and hole A3-25%, then hole A1 is analyzed again, and the corresponding retrieval path analysis step is to analyze the remaining workpieces T4, workpiece T5, and workpiece T6. If one of the workpiece models is found, it is used as the target model, otherwise the probability of hole A1 is reduced again.
[0048] In the artifact model output step, if the search is successful, the artifact model is used as the target model.
[0049] Since the above-mentioned micro-parameter adjustment is for the recognition and registration of the same workpiece to be cleaned, but if the next workpiece to be cleaned is the same as the previous workpiece to be cleaned and is analyzed repeatedly, the recognition and registration accuracy of the model is relatively low, and it is difficult to meet the intelligent requirements of high precision and high efficiency. Therefore, a model matching update step is also included. If the search fails in the retrieval path analysis step, the macro parameters in the constant rate formula are adjusted to reduce the probability value of the target analysis feature in the next workpiece to be cleaned as the last subdivision feature; for example, the probability ratio of the target analysis feature hole a as hole A1 will be relatively reduced, that is, the macro parameters in the constant rate algorithm will be adjusted so that when the initial constant rate is set, the probability of hole a as hole A1 is reduced from the original 70% to 68%. The probability reduction of the macro parameters is much smaller than the reduction of the micro parameters. The purpose is to make the error of the same target analysis feature analysis smaller. When the probability of hole a as hole A1 is still the maximum value when the second analysis occurs, the macro parameters here can be adjusted without adjustment or with a smaller adjustment range.
[0050] The present invention continuously updates the model through a relatively complex analysis and processing process in the early stage. The target analysis features are more accurate when aligned with the subdivided features, which is convenient for the subsequent alignment of the workpiece model to be cleaned. Compared with the traditional workpiece contour line recognition method, the means of the present invention can more accurately identify and align when the boundary line is blurred due to accumulated stains. Compared with the traditional model trial and error update method, the means of the present invention achieves the adjustment of refined feature parameters during the update process, rather than blindly replacing old data or stacking data with new data.
[0051] Furthermore, in order to more accurately and efficiently identify the workpiece to be cleaned, the feature probability control step also includes a local cleaning sub-step.
[0052] In the local cleaning sub-step, if the result of the retrieval path analysis step is a search failure, the laser cleaning head is controlled to perform local cleaning on the target analysis feature, and the visually captured image of the workpiece to be cleaned is re-acquired as an updated image, and the feature point analysis step is performed again; assuming that the diameter of the hole is the basis for distinguishing the subdivision features of the hole, the hole a in the target image is determined as hole A2 due to the accumulation of stains when the rate is fixed, but it should actually be hole A1. Therefore, when a search fails, the laser cleaning head can be controlled to rinse the hole a, and then the image can be acquired for probability distribution, thereby improving the efficiency of the registration.
[0053] like Figure 3 As shown, it also includes a model synchronization matching step,
[0054] Model synchronization matching step, visually covering the workpiece to be cleaned with frame lines according to the boundary to obtain the target frame line, using the target frame line to preliminarily screen out a large group of matching workpiece models in the model library through a matching strategy, the matching strategy includes calculating the length-width ratio of the target frame line, using the length-width ratio to screen the large group of matching workpiece models in the model library, extracting the intersections between the workpiece to be cleaned and the target frame line, and rescreening the preliminarily screened workpiece models again based on the number of intersections to obtain a workpiece model group, that is, assuming that the vertices of the workpiece to be cleaned are cleaned, but the boundary lines are not clear, then the vertices are used as the target frame line, and the number of vertices is 3, then the workpiece model group is screened to obtain a workpiece model group, and the intersections are used through a calculation strategy to obtain a workpiece structure diagram, the calculation strategy includes connecting each intersection, calculating the distance value between each two intersections, calculating the distance value between each vertex in the large group of workpiece models, analyzing whether the ratio of the distance values between each intersection is the same as the ratio of the distance values between each vertex, and obtaining the target model based on the similarity calculation of the workpiece structure diagram and the workpiece model group;
[0055] It also includes a model verification step, which compares the target model in the artifact model output step with the target model in the model synchronization matching step. If the two are the same, the output is determined; if the two are inconsistent, a feedback warning is recorded.
[0056] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. A method for visual identification of laser-cleaned parts, characterized in that: The following steps are included: The model library construction step divides the historical image dataset into proportions for training, testing, and validating various artifact models in the target detection network; The feature point analysis step is to obtain a visually captured image of the workpiece to be cleaned as a target image, and obtain a key feature in the target image as a target analysis feature. The target analysis feature corresponds to a subdivision feature. The subdivision probability of each feature is calculated by a similarity formula according to the target analysis feature. The feature subdivision probability reflects the probability of the target analysis feature matching each subdivision feature. A search path analysis step, selecting several workpiece models in a model library corresponding to the maximum value of feature segmentation probability, and selecting other key features based on the target image to search among the several workpiece models; In the feature probability control step, if the search fails, the microscopic parameters in the similarity formula are controlled to reduce the probability value of the target analysis feature in the workpiece to be cleaned as the last subdivision feature, and then the maximum probability of each subdivision feature after the control is re-selected, and the search path analysis step is repeated; in the search path analysis step, other key features in the target image are sorted by distance factors or significant factors to obtain a feature sorting table, and the key features with the highest ranking are preferentially searched in the workpiece model through a search strategy, and the search strategy includes a feature point type search sub-step and a feature point direction search sub-step, The feature point type finding sub-step analyzes whether the workpiece model contains other key features in the feature sorting table. If the workpiece model contains all key features, the feature point direction finding sub-step is performed. If any key feature is missing in the workpiece model, the workpiece model is ignored. If all workpiece models are ignored, the feature probability control step is performed. The feature point direction finding sub-step is to establish a plane coordinate system in the target image, extract the coordinate point information of the target analysis feature and the key feature, compare the relative direction and distance between each key feature in the target image and the target analysis feature according to the coordinate point information, obtain the relative direction and distance of various other key features and subdivision features in the workpiece model, and compare the two. If they are all registered one by one, the workpiece model output step is performed. If any registration fails, the feature probability control step is performed; In the artifact model output step, if the search is successful, the artifact model is used as the target model.
2. A method for visually identifying parts cleaned by laser according to claim 1, characterized in that: It also includes a model matching update step. If the search fails in the retrieval path analysis step, the macro parameters in the similarity formula are adjusted to reduce the probability value of the target analysis feature in the next workpiece to be cleaned as the last segmentation feature.
3. The method for visually identifying parts cleaned by laser according to claim 1, characterized in that: The characteristic probability control step also includes a local cleaning sub-step, In the local cleaning sub-step, if the result of the retrieval path analysis step is a search failure, the laser cleaning head is controlled to perform local cleaning on the target analysis feature, the visually captured image of the workpiece to be cleaned is re-acquired as an updated image, and the feature point analysis step is performed again.
4. A method for visually identifying parts cleaned by laser according to claim 1 or 3, characterized in that: It also includes the model synchronization matching step, The model synchronization matching step includes: covering the visual workpiece to be cleaned with a frame line according to the boundary to obtain a target frame line, using the target frame line to preliminarily screen out a large group of workpiece models that match in the model library through a matching strategy, extracting the intersection points between the workpiece to be cleaned and the target frame line, and re-screening the preliminarily screened workpiece models according to the number of the intersection points to obtain a workpiece model group, obtaining a workpiece structure diagram through a calculation strategy for the intersection points, and obtaining a target model based on the similarity calculation between the workpiece structure diagram and the workpiece model group; The matching strategy includes calculating the length-width ratio of the target frame line, and selecting a matching workpiece model group in the model library based on the length-width ratio; The calculation strategy includes connecting each intersection point, calculating the distance value between each two intersection points, calculating the distance value between each vertex in the workpiece model group, and analyzing whether the ratio of the distance value between each intersection point is the same as the ratio of the distance value between each vertex.
5. A method for visually identifying parts cleaned by laser according to claim 4, characterized in that: It also includes a model verification step, comparing the target model in the workpiece model output step with the target model in the model synchronization matching step. If the two are the same, the output is determined; if the two are inconsistent, a feedback warning is recorded.
