Weld joint detection intelligent regulation and control system and method based on machine vision
Through the image and ultrasonic data of welding detection data, weld defects are identified by defect detection and identification models, and a regulation scheme is generated, which solves the problem of failure to identify unknown defect types in the prior art, and improves the accuracy and efficiency of weld detection .
Patent Information
- Application Number
- CN202411073436.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-08
AI Technical Summary
The existing intelligent control method for weld detection only considers the existing defect types, and lacks identification and control of unknown defect types, resulting in low accuracy and efficiency.
By obtaining the detection data at the welding point, using the defect detection model and defect type identification model, we can judge whether the image is a defect image, and judge the existence of the real defect based on the ultrasonic data, generate defect levels, categories and causes of formation, and adjust the initial regulation plan to respond. Unknown defect type.
It improves the accuracy and efficiency of weld detection, can timely identify and generate control plans, provide effective control for unrecorded defect types, and verify the effectiveness of control plans.
Smart Images

Figure CN118918093A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of weld detection and control, and specifically to a weld detection intelligent control system and method based on machine vision. Background Art
[0002] Traditional welding quality inspection methods have problems such as low efficiency and low accuracy, which are difficult to meet the high requirements of modern manufacturing industry for welding quality. In recent years, with the rapid development of machine vision technology, its application in the field of welding quality inspection and control has gradually attracted attention. The present invention aims to provide a weld inspection intelligent control system and method based on machine vision to improve the efficiency and accuracy of welding quality inspection.
[0003] The prior art (invention patent with announcement number CN116776103B) discloses a machine vision-based weld detection intelligent control system and method, including: a pipeline weld detection event marking module, a central database establishment module, a feature comparison module, a usage priority analysis module and an early warning control module; the pipeline weld detection event marking module is used to obtain pipeline weld detection events corresponding to pipeline welds of the same material type; the central database establishment module is used to build a central database for storing pipeline weld detection events; the feature comparison module is used to compare the target weld information with the weld feature information recorded in the central database; the usage priority analysis module is used to analyze the usage priority of the detection equipment configuration; the early warning control module is used to verify the real-time defect feature when the effective defect feature recorded in the target event is unique, and perform early warning control based on the verification result.
[0004] The above-mentioned weld detection intelligent control system records historical data on the use of ultrasonic flaw detectors to detect pipeline welds, builds a database of pipeline welds based on the historical data, detects ultrasonic data of the welding process, matches the data in the ultrasonic database, and provides defect solutions. However, since the above-mentioned scheme only considers the detection of pipeline welds with existing defect types and lacks identification and control of unknown defect types, the accuracy and efficiency of the weld detection intelligent control method are relatively low. Therefore, the weld detection intelligent control method still needs to be further improved. Summary of the invention
[0005] The present application aims to solve at least one of the technical problems existing in the prior art; to this end, the present application proposes a weld detection intelligent control system and method based on machine vision, which is used for the technical problem that the existing weld detection intelligent control method only considers the detection of existing defect types, lacks the identification and control of unknown defect types, and leads to low accuracy and efficiency of the weld detection intelligent control method. The present application detects whether there are real defects in the welding part corresponding to the original image by an ultrasonic flaw detection method; outputs a number of confidence levels and their corresponding defect types by training a defect type recognition model; generates defect levels, defect categories and formation causes of known or unknown defect types according to the confidence levels and their corresponding defect types, and generates an initial control plan based on this; determines whether the control plan is effective by obtaining the last defect level and comparing it with the current defect level, thereby adding a detection method for identifying unknown defect types and a corresponding control method, thereby solving the above-mentioned problems.
[0006] To achieve the above objectives, the first aspect of the present application provides a method for intelligent control of weld detection based on machine vision, comprising:
[0007] Step 1: Acquire detection data of the welding point, wherein the detection data includes an original image;
[0008] Step 2: Input the original image into the defect detection model to obtain the image label; determine whether the image label is a defect image; if yes, proceed to step 3; if no, proceed to step 1;
[0009] Step 3: Obtain ultrasonic detection data; determine whether there is a real defect in the welding part corresponding to the original image according to the ultrasonic data; if yes, proceed to step 4; if no, proceed to step 1;
[0010] Step 4: Input the original image into the defect type recognition model to obtain a number of confidence levels and their corresponding defect types; generate defect levels, defect categories and formation causes according to the confidence levels and their corresponding defect types;
[0011] Step 5: Generate an initial control plan according to the formation causes corresponding to the defect categories;
[0012] Step 6: Obtain the last defect level; mark it as the comparison defect level; adjust the initial control plan according to the comparison defect level and the defect level to obtain the control plan;
[0013] Step 7: Regulate the welding equipment according to the regulation plan.
[0014] The present application obtains the inspection data of the welding part, and the inspection data includes the original image; inputs the original image into the defect detection model to obtain the image label; determines whether the image label is a defect image; obtains the ultrasonic inspection data; determines whether there is a real defect in the welding part corresponding to the original image according to the ultrasonic data; inputs the original image into the defect type recognition model to obtain several confidences and their corresponding defect types; generates the defect level, defect category and formation cause according to the confidence and its corresponding defect type; generates the initial control plan according to the formation cause corresponding to the defect category; obtains the last defect level; and marks it as the comparison defect level; adjusts the initial control plan according to the comparison defect level and the defect level to obtain the control plan; controls the welding equipment according to the control plan;
[0015] In the present application, the original image is input into the defect type recognition model to obtain several confidence levels and their corresponding defect types; defect levels, defect categories and formation causes are generated according to the confidence levels and their corresponding defect types; the initial control plan is adjusted according to the compared defect levels and the defect levels to obtain the control plan, which increases the judgment and intelligent control for identifying unknown defect types, so that when unrecorded defects are detected, the control plan can be generated in time and the effectiveness of the control plan can be verified, thereby increasing the accuracy and efficiency of the intelligent control method for weld detection.
[0016] Furthermore, the construction of the defect detection model includes the following steps:
[0017] Obtaining a number of historical images and their corresponding image labels, where the image labels include defective images and normal images;
[0018] The historical images and their corresponding image labels are divided according to the corresponding proportions and integrated into training sets, validation sets, and test sets;
[0019] Select a neural network model as the basic network model; train the basic network model using the training set, and adjust the learning rate or other hyperparameters on the validation set to obtain a pre-trained model;
[0020] The pre-trained model is verified using the test set, and finally the defect detection model is obtained.
[0021] Furthermore, judging whether there is a real defect in the welding part corresponding to the original image according to the ultrasonic data includes:
[0022] Acquire a number of ultrasonic data in different directions; calculate the variance one of the peak data, the variance two of the number of peaks, and the variance three of the waveform width in each ultrasonic data;
[0023] Determine whether variance 1 is greater than variance threshold 1; if yes, set the peak state to unstable; if no, set the peak state to stable;
[0024] Determine whether variance 2 is greater than variance threshold 2; if yes, set the peak state to unstable; if no, set the peak state to stable;
[0025] Determine whether variance three is greater than variance threshold three; if yes, set the waveform state to unstable; if no, set the waveform state to stable;
[0026] When any of the peak state, wave crest state and waveform state is unstable, the welding part is a pseudo defect; otherwise, the welding part is a real defect;
[0027] The ultrasonic data includes reflected wave peak data and reflected wave morphology data in different directions; the reflected wave morphology data includes the number of peaks and waveform width.
[0028] The present application obtains ultrasonic data from several different directions; calculates the variance one of the high peak data, the variance two of the number of peaks, and the variance three of the waveform width in each ultrasonic data; determines the stability of each type of data by the variance, and then determines whether the welding part corresponding to the original image is a real defect; makes full use of each component of the ultrasonic data; and improves the accuracy of the judgment result.
[0029] Furthermore, the construction of the defect type identification model includes the following steps:
[0030] Obtain several real defect images corresponding to each defect type;
[0031] Divide the real defect images into training sets, validation sets, and test sets according to a certain ratio;
[0032] Preprocess the real defect images of the training set, validation set, and test set;
[0033] Select the convolutional neural network model VGGNet as the basic model;
[0034] The original output of the basic model was to directly give the defect type, but now it is modified to output all confidence levels and their corresponding defect types; the confidence level represents the degree of trustworthiness of the model for the prediction result;
[0035] Train the basic model with the training set, and adjust the learning rate or other hyperparameters on the validation set to obtain the pre-trained model;
[0036] By verifying the pre-trained model on the test set, a defect type recognition model is finally obtained, whose input is the original image and output is several confidence levels and their corresponding defect types.
[0037] Furthermore, the generation of defect level, defect category and formation cause according to the confidence level and its corresponding defect type includes:
[0038] Obtaining the maximum confidence value, and determining whether the confidence value is greater than a classification threshold of one;
[0039] If yes, then the defect type corresponding to the maximum confidence is obtained, and the defect level and formation cause corresponding to the defect type are searched in the defect type lookup table according to the defect type, and the defect level is used as the defect level of this time, and the formation cause is used as the formation cause of this time; the defect type is set as the defect category of this time;
[0040] No, when the confidence is greater than the classification threshold two; arrange the confidences from large to small; obtain several defect types corresponding to several confidences within the set ranking, and search for several defect levels and formation causes corresponding to the corresponding defect types in the defect type lookup table according to the corresponding defect types; and perform weighted summation of each defect level by confidence to obtain the current defect level; set the current formation cause to a mixed formation cause; and integrate several formation causes into the current formation cause; set the current defect category to a mixed defect; otherwise, set the defect category to none, the defect level to none, and the formation cause to none.
[0041] The present application obtains several confidence levels and corresponding defect types by inputting the original image into a defect type recognition model, selects the confidence level with the largest value to compare with the classification threshold, and uses corresponding methods to determine the defect level, defect category and formation cause of the current original image, thereby increasing the recognition of unknown defect types and corresponding control methods during weld detection, and improving the accuracy and efficiency of weld detection.
[0042] Furthermore, the defect type lookup table includes several defect levels, corresponding defect types and formation causes; the defect levels and defect types correspond one to one; the defect types and formation causes correspond one to one.
[0043] Furthermore, the defect level of this time is obtained by weighted summing up each defect level by confidence, including:
[0044] Get the confidence level Zi within the set ranking and the corresponding defect level Qi, where i is the number of the built-in confidence level of the set ranking; i=1, 2, ..., n, where n is the total number of built-in confidence levels of the set ranking;
[0045] The defect level of this time is calculated by the formula QD=∑Qi×Zi / ∑Zi.
[0046] Furthermore, the initial control scheme is generated according to the formation cause corresponding to the defect category, including:
[0047] Obtaining several formation causes and their corresponding standard control schemes; establishing a control lookup table according to the formation causes and the standard control schemes;
[0048] Determine whether the formation cause is a mixed formation cause; if yes, search the corresponding standard control scheme in the control lookup table according to each formation cause, and extract several confidences corresponding to the mixed cause; sort the corresponding standard control schemes in descending order of confidence, and integrate the sorted standard control schemes into the initial control scheme;
[0049] No, when the cause is none, an alarm signal is generated and an alarm operation is performed according to the alarm signal;
[0050] Otherwise, the corresponding standard control scheme is obtained by searching in the control lookup table according to the corresponding formation cause, and the standard control scheme is used as the initial control scheme.
[0051] Further, the control scheme is obtained by adjusting the initial control scheme according to the comparison defect level and the defect level, including:
[0052] Obtain the comparison defect level DD; calculate the effect level XD by the formula XD = DD-α×(1-exp(-β×DD))×DD; α is the proportional coefficient, β is the exponential coefficient, α and β∈(0,1);
[0053] Determine whether the current defect level is less than the effect level; if yes, obtain the last initial control plan and use it as the current control plan; and set the number of abnormal controls to zero;
[0054] If not, the initial control scheme is set as the control scheme; and the number of abnormal controls is increased by one; when the number of abnormal controls is greater than the control number threshold, an alarm signal is generated, and an alarm operation is performed according to the alarm signal.
[0055] This application obtains the previous defect level, and uses a formula to calculate the effect level that the control method of the previous defect category can theoretically achieve. By comparing the effect level with the current defect level, it is determined whether the control method of the previous defect category is effective. This adds verification of the effectiveness of the control method for the defect category and further improves the effectiveness of the intelligent control method for weld detection.
[0056] Another aspect of the present invention provides a machine vision-based weld detection intelligent control system, comprising: a data acquisition module, a defect detection module, a defect recognition module, a control module, an early warning module and a database;
[0057] Data acquisition module: acquires the detection data of the welding position, wherein the detection data includes the original image;
[0058] Defect detection module: extracts the original image from the detection data, inputs the original image into the defect detection model to obtain the corresponding image label; when the image label is a defect image, generates a verification signal;
[0059] Defect recognition module: collect ultrasonic detection data according to the verification signal; determine whether the original image has real defects according to the ultrasonic detection data; if yes, input the corresponding original image into the defect type recognition model to obtain several confidence levels and their corresponding defect types; generate defect levels, defect categories and formation causes according to the confidence levels and their corresponding defect types;
[0060] Control module: generate an initial control scheme according to the formation cause corresponding to the defect category; obtain the last defect level; mark it as the comparison defect level; adjust the initial control scheme according to the comparison defect level and the defect level to obtain a control scheme; control the welding equipment according to the control scheme;
[0061] Early warning module: obtains alarm signals and takes alarm actions.
[0062] Compared with the prior art, the beneficial effects of this application are:
[0063] 1. The present application obtains the inspection data of the welding part, and the inspection data includes the original image; inputs the original image into the defect detection model to obtain the image label; determines whether the image label is a defect image; obtains ultrasonic inspection data; determines whether there is a real defect in the welding part corresponding to the original image according to the ultrasonic data; inputs the original image into the defect type recognition model to obtain several confidences and the corresponding defect types; generates defect levels, defect categories and formation causes according to the confidences and the corresponding defect types; generates an initial control scheme according to the formation causes corresponding to the defect categories; obtains the last defect level; and marks it as a comparison defect level; adjusts the initial control scheme according to the comparison defect level and the defect level to obtain a control scheme; controls the welding equipment according to the control scheme; in the present application, the original image is input into the defect type recognition model to obtain several confidences and the corresponding defect types; generates defect levels, defect categories and formation causes according to the confidences and the corresponding defect types; adjusts the initial control scheme according to the comparison defect level and the defect level to obtain a control scheme, which increases the judgment and intelligent control for identifying unknown defect types, avoids the problem that only considering the detection of existing defect types, lacking the identification and control of unknown defect types, and resulting in low accuracy and efficiency of the intelligent control method for weld detection.
[0064] 2. This application obtains the last defect level and uses a formula to calculate the effect level that the control method of the last defect category can theoretically achieve. By comparing the effect level with the current defect level, it is determined whether the control method of the last defect category is effective. This adds verification of the effectiveness of the control method of the defect category and further improves the effectiveness of the intelligent control method for weld detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0066] Figure 1 This is a flow chart of a machine vision-based intelligent control method for weld detection in this application;
[0067] Figure 2 A flowchart of generating defect levels and defect categories and causes for this application;
[0068] Figure 3 This is a schematic diagram of the principle of a weld detection intelligent control system based on machine vision in this application. DETAILED DESCRIPTION
[0069] The technical solution of the present application will be described clearly and completely in conjunction with the embodiments below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.
[0070] See also Figure 1 The first aspect of the present application provides a method for intelligently controlling weld detection based on machine vision, comprising:
[0071] Step 1: Obtain the inspection data of the welding part, the inspection data includes the original image; the original image is the image of the welding part taken;
[0072] Step 2: Input the original image into the defect detection model to obtain the image label; determine whether the image label is a defective image; if yes, proceed to step 3; if no, proceed to step 1; the image labels are normal images and defective images. The normal image is an image with no problems in welding, and the defective image is an image with problems in the welding part;
[0073] Step 3: Obtain ultrasonic detection data; determine whether there is a real defect in the welding part corresponding to the original image according to the ultrasonic data; if yes, proceed to step 4; if no, proceed to step 1; ultrasonic detection data is the reflected wave data obtained by the ultrasonic flaw detection method, including the peak value of the reflected wave, the number of reflected wave peaks and the width of the reflected wave waveform; the real defect is a real defect;
[0074] Step 4: Input the original image into the defect type recognition model to obtain several confidence levels and their corresponding defect types; generate defect levels, defect categories and formation causes based on the confidence levels and their corresponding defect types; defect types are the types of defects that occur during welding, including pores, pits, and weld nodules; defect categories indicate the types of defects, and defect levels are the degree of harm of the defect category to actual welding. The higher the level, the greater the harm. The formation cause is the cause of the defect category, such as too high temperature or too fast speed, etc.;
[0075] Step 5: Generate an initial control plan based on the causes of the defect categories;
[0076] Step 6: Obtain the last defect level; mark it as the comparison defect level; adjust the initial control plan according to the comparison defect level and the defect level to obtain the control plan;
[0077] Step 7: Regulate the welding equipment according to the regulation plan.
[0078] This embodiment obtains the inspection data of the welding part, and the inspection data includes the original image; inputs the original image into the defect detection model to obtain the image label; determines whether the image label is a defect image; obtains ultrasonic inspection data; determines whether there is a real defect in the welding part corresponding to the original image according to the ultrasonic data; inputs the original image into the defect type recognition model to obtain a number of confidences and the corresponding defect types; generates defect levels, defect categories and formation causes according to the confidences and the corresponding defect types; generates an initial control plan according to the formation causes corresponding to the defect categories; obtains the last defect level; and marks it as a comparison defect level; adjusts the initial control plan according to the comparison defect level and the defect level to obtain a control plan; and controls the welding equipment according to the control plan;
[0079] In this embodiment, the original image is input into the defect type recognition model to obtain several confidence levels and their corresponding defect types; defect levels, defect categories and formation causes are generated according to the confidence levels and their corresponding defect types; the initial control scheme is adjusted according to the compared defect levels to obtain the control scheme, which increases the judgment and intelligent control for identifying unknown defect types, and avoids the problem of low accuracy and efficiency of the intelligent control method for weld detection due to the identification and control method that does not take into account the unknown defect types.
[0080] The construction of the defect detection model includes the following steps:
[0081] Obtaining a number of historical images and their corresponding image labels, where the image labels include defective images and normal images;
[0082] The historical images and their corresponding image labels are divided according to corresponding ratios and integrated into a training set, a validation set, and a test set. The division ratio in this embodiment is set based on experience, and the division ratio is 7:2:1.
[0083] Select a neural network model as the basic network model; train the basic network model using the training set, and adjust the learning rate or other hyperparameters on the validation set to obtain a pre-trained model;
[0084] The pre-trained model is verified using the test set, and finally the defect detection model is obtained.
[0085] Judging whether there are real defects in the welding part corresponding to the original image based on the ultrasonic data, including:
[0086] Acquire a number of ultrasonic data in different directions; calculate the variance one of the peak data, the variance two of the number of peaks, and the variance three of the waveform width in each ultrasonic data;
[0087] Determine whether variance 1 is greater than variance threshold 1; if yes, set the peak state to unstable; if no, set the peak state to stable;
[0088] Determine whether variance 2 is greater than variance threshold 2; if yes, set the peak quantity state to unstable; if no, set the peak quantity state to stable;
[0089] Determine whether variance three is greater than variance threshold three; if yes, set the waveform width state to unstable; if no, set the waveform width state to stable;
[0090] When any of the peak state, wave peak number state and waveform width state is unstable, the welding part is a pseudo defect; otherwise, the welding part is a real defect;
[0091] The ultrasonic data includes the peak data of reflected waves in different directions and the morphological data of reflected waves; the morphological data of reflected waves includes the number of peaks and the width of the waveform.
[0092] This embodiment obtains a group of ultrasonic data in different directions, including: high peak data {51, 52, 50, 52, 50} in decibels, the number of peaks {19, 20, 21, 21, 20} in pieces, and the waveform width {1.1, 1.4, 1.5, 1.2, 1.3} in meters. The corresponding variance one is 0.8, the variance two is 0.56, and the variance three is 0.02. The variance threshold one, the variance threshold two, and the variance threshold three are set to 1.0, 0.7, and 0.05 according to experience. Therefore, the high peak state, the peak number state, and the waveform width state are all stable, and the welding part is determined to be a real defect.
[0093] This embodiment obtains ultrasonic data from several different directions; calculates the variance one of the high peak data, the variance two of the number of peaks, and the variance three of the waveform width in each ultrasonic data; determines the stability of each type of data by the variance, and then determines whether the welding part corresponding to the original image is a real defect; makes full use of each component of the ultrasonic data; and improves the accuracy of the judgment result.
[0094] The construction of the defect type identification model includes the following steps:
[0095] Obtain several real defect images corresponding to each defect type;
[0096] The real defect images are divided into a training set, a validation set, and a test set according to a certain ratio; the division ratio in this embodiment is set based on experience, and the division ratio is 7:2:1;
[0097] Preprocess the real defect images of the training set, validation set, and test set;
[0098] Select the convolutional neural network model VGGNet as the basic model;
[0099] The original output of the basic model was to directly give the defect type, but now it is modified to output all confidence levels and their corresponding defect types; the confidence level represents the degree of trustworthiness of the model for the prediction result;
[0100] Train the basic model with the training set, and adjust the learning rate or other hyperparameters on the validation set to obtain the pre-trained model;
[0101] By verifying the pre-trained model on the test set, a defect type recognition model is finally obtained, whose input is the original image and output is several confidence levels and their corresponding defect types.
[0102] See also Figure 2 , based on the confidence level and its corresponding defect type, the defect level, defect category and formation cause are generated, including:
[0103] Get the maximum confidence value and determine whether the confidence value is greater than the classification threshold 1;
[0104] If yes, then obtain the defect type corresponding to the maximum confidence, find the defect level and formation cause corresponding to the defect type in the defect type lookup table according to the defect type, and use the defect level as the defect level of this time, and the formation cause as the formation cause of this time; set the defect type as the defect category of this time;
[0105] No, when the confidence is greater than the classification threshold 2; arrange the confidences from large to small; obtain several defect types corresponding to several confidences within the set ranking, and search for several defect levels and formation causes corresponding to the corresponding defect types in the defect type lookup table according to the corresponding defect types; and perform weighted summation of each defect level by confidence to obtain the current defect level; set the current formation cause to a mixed formation cause; and integrate several formation causes into the current formation cause; set the current defect category to a mixed defect; otherwise, set the defect category to none, the defect level to none, and the formation cause to none.
[0106] In this embodiment, by inputting three original images into the defect type recognition model, several defect types and their corresponding confidences corresponding to the original image 1 are obtained; they are {confidence: 0.85, defect type: pore, confidence: 0.1, defect type: pit, confidence: 0.05, defect type: weld nodule}, several defect types and their corresponding confidences corresponding to the original image 2 are {confidence: 0.65, defect type: pore, confidence: 0.3, defect type: pit, confidence: 0.05, defect type: weld nodule}, and several defect types and their corresponding confidences corresponding to the original image 3 are {confidence: 0.45, defect type: pore, confidence: 0.3, defect type: pit, confidence: 0. 25. Defect type: weld nodule. Classification threshold 1 is set to 0.8 based on experience, and classification threshold 2 is set to 0.5 based on experience. The maximum value of the confidence obtained for each original image is compared with the classification threshold. The defect category of original image 1 is pore defect, the defect category of original image 2 is mixed defect, and the defect category of original image 3 is none. The defect level and formation cause of the original image are found in the defect type lookup table. Finally, the defect level of original image 1 is 3, and the formation cause is 50 degrees Celsius high temperature; the defect level of original image 2 is 2.684, and the formation cause is 50 degrees Celsius high temperature and 30 cm / min slow speed; the defect level of original image 3 is none, and the formation cause is none; the defect type lookup table is as follows:
[0107] Defect level Defect Type Causes 3 Pores Temperature 50 degrees Celsius higher 2 Pits Slow speed 30 cm / min 1 Weld flash Slow speed 60 cm / min
[0108] In other embodiments, there are more defect types and more confidence levels, and the detection method is the same as the previous embodiment.
[0109] This embodiment obtains several confidence levels and corresponding defect types by inputting the original image into the defect type recognition model, selects the confidence level with the largest value to compare with the classification threshold, and uses the corresponding method to determine the defect level, defect category and formation cause of the current original image, thereby increasing the recognition of unknown defect types and corresponding control methods in the weld detection process, and improving the accuracy and efficiency of weld detection.
[0110] The defect type lookup table includes several defect levels, corresponding defect types and formation causes; defect levels and defect types correspond one to one; defect types and formation causes correspond one to one.
[0111] The defect level of this time is obtained by weighted summing up each defect level through confidence, including:
[0112] Get the confidence level Zi within the set ranking and the corresponding defect level Qi, where i is the number of the built-in confidence level of the set ranking; i=1, 2, ..., n, where n is the total number of built-in confidence levels of the set ranking;
[0113] The defect level of this time is calculated by the formula QD=∑Qi×Zi / ∑Zi.
[0114] In this embodiment, the top two confidences and defect types in the original image 2 are obtained {confidence: 0.65, defect type: pore, confidence: 0.3, defect type: pit}, wherein the defect level of the pore with a confidence of 0.65 is 3, and the defect level of the pit with a confidence of 0.3 is 2. The defect level of the original image 2 is calculated according to the formula QD=3×0.65 / 0.95+2×0.3 / 0.95=2.684; in other embodiments, the top several confidences can be obtained, and the calculation method is the same as this embodiment.
[0115] Generate an initial control plan based on the causes of the defect categories, including:
[0116] Obtaining several formation causes and their corresponding standard control schemes; establishing a control lookup table according to the formation causes and the standard control schemes;
[0117] Determine whether the formation cause is a mixed formation cause; if yes, search the corresponding standard control scheme in the control lookup table according to each formation cause, and extract several confidences corresponding to the mixed cause; sort the corresponding standard control schemes in descending order of confidence, and integrate the sorted standard control schemes into the initial control scheme;
[0118] No, when the cause is none, an alarm signal is generated and an alarm operation is performed according to the alarm signal;
[0119] Otherwise, the corresponding standard control scheme is searched in the control lookup table according to the corresponding formation cause, and the standard control scheme is used as the initial control scheme.
[0120] The control plan is obtained by comparing the defect level and adjusting the initial control plan according to the defect level, including:
[0121] Obtain the comparative defect level DD; calculate the effect level XD by the formula XD=DD-α×(1-exp(-β×DD))×DD; α is the proportional coefficient, β is the exponential coefficient, α and β∈(0,1); the effect level is an effect that can be achieved in theory by the control method of the last defect type. For the comparative defect level DD, the higher the DD, the better the effect after the control method, and the effect level that can be achieved in theory will be relatively low;
[0122] Determine whether the current defect level is less than the effect level; if yes, obtain the last initial control plan and use it as the current control plan; and set the number of abnormal controls to zero;
[0123] If not, the initial control scheme is set as the control scheme; and the number of abnormal controls is increased by one; when the number of abnormal controls is greater than the control number threshold, an alarm signal is generated, and an alarm operation is performed according to the alarm signal.
[0124] In this embodiment, two comparative defect levels {3, 2} are obtained, α and β are set to 0.4 and 0.1 according to experience, and the XD when the comparative defect level is 3 is calculated by the formula to be 2.689, the current defect level is 2.684, and the current defect level is less than the effect level. The previous initial control scheme is obtained and used as the control scheme for this time; the XD when the comparative defect level is 3 is calculated by the formula to be 1.855, the current defect level is 2.684, and the current defect level is greater than the effect level. The initial control scheme is set as the control scheme, and the number of abnormal controls is increased by one.
[0125] This embodiment obtains the previous defect level, and uses a formula to calculate the effect level that the control method of the previous defect category can theoretically achieve. By comparing the effect level with the current defect level, it is determined whether the control method of the previous defect category is effective. This adds verification of the effectiveness of the control method of the defect category and further improves the effectiveness of the intelligent control method for weld detection.
[0126] See also Figure 3 , a weld inspection intelligent control system based on machine vision, comprising: a data acquisition module, a defect detection module, a defect recognition module, a control module, an early warning module and a database;
[0127] Data acquisition module: acquires the detection data of the welding position, wherein the detection data includes the original image;
[0128] Defect detection module: extracts the original image from the detection data, inputs the original image into the defect detection model to obtain the corresponding image label; when the image label is a defect image, generates a verification signal;
[0129] Defect recognition module: collect ultrasonic detection data according to the verification signal; determine whether the original image has real defects according to the ultrasonic detection data; if yes, input the corresponding original image into the defect type recognition model to obtain several confidence levels and their corresponding defect types; generate defect levels, defect categories and formation causes according to the confidence levels and their corresponding defect types;
[0130] Control module: generate an initial control scheme according to the formation cause corresponding to the defect category; obtain the last defect level; mark it as the comparison defect level; adjust the initial control scheme according to the comparison defect level and the defect level to obtain a control scheme; control the welding equipment according to the control scheme;
[0131] Early warning module: obtains alarm signals and takes alarm actions.
[0132] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.
[0133] The working principle of the present application is as follows: by acquiring the original image of the welding part; inputting the original image into the defect detection model to determine whether the original image is a defect image; judging whether there is a real defect in the welding part corresponding to the original image through ultrasonic data; inputting the original image into the defect type recognition model to obtain several confidences and their corresponding defect types; generating defect levels, defect categories and formation causes according to the confidences and their corresponding defect types; generating an initial control scheme according to the formation causes corresponding to the defect categories; obtaining the last defect level as a comparison defect level; adjusting the initial control scheme according to the comparison defect level and the defect level to obtain a control scheme; regulating the welding equipment according to the control scheme; in the present application, the original image is input into the defect type recognition model to obtain several confidences and their corresponding defect types; generating defect levels, defect categories and formation causes according to the confidences and their corresponding defect types; adjusting the initial control scheme according to the comparison defect level and the defect level to obtain a control scheme, which increases the judgment and intelligent control for identifying unknown defect types, avoids the problem of low accuracy and efficiency of the intelligent control method for weld detection by only considering the detection of existing defect types and lacking the identification and control of unknown defect types.
[0134] The above embodiments are only used to illustrate the technical method of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical method of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present application.
Claims
1. A weld detection intelligent control method based on machine vision, characterized in that: include: Step 1: Acquire detection data of the welding point, wherein the detection data includes an original image; Step 2: Input the original image into the defect detection model to obtain the image label; determine whether the image label is a defect image; if yes, proceed to step 3; if no, proceed to step 1; Step 3: Obtain ultrasonic detection data; determine whether there is a real defect in the welding part corresponding to the original image according to the ultrasonic data; if yes, proceed to step 4; if no, proceed to step 1; Step 4: Input the original image into the defect type recognition model to obtain a number of confidence levels and their corresponding defect types; generate defect levels, defect categories and formation causes according to the confidence levels and their corresponding defect types; Step 5: Generate an initial control plan according to the formation causes corresponding to the defect categories; Step 6: Obtain the last defect level; mark it as the comparison defect level; adjust the initial control plan according to the comparison defect level and the defect level to obtain the control plan; Step 7: Regulate the welding equipment according to the regulation plan.
2. The intelligent control method for weld detection based on machine vision according to claim 1 is characterized in that: The construction of the defect detection model includes the following steps: Obtaining a number of historical images and their corresponding image labels, where the image labels include defective images and normal images; Divide historical images and their corresponding image labels and integrate them into training sets, validation sets, and test sets; Select a neural network model as the basic network model; train the basic network model using the training set, and adjust the learning rate or other hyperparameters on the validation set to obtain a pre-trained model; The pre-trained model is verified using the test set, and finally the defect detection model is obtained.
3. The intelligent control method for weld detection based on machine vision according to claim 1 is characterized in that: The step of judging whether there is a real defect in the welding part corresponding to the original image according to the ultrasonic data includes: Acquire a number of ultrasonic data in different directions; calculate the variance one of the peak data, the variance two of the number of peaks, and the variance three of the waveform width in each ultrasonic data; Determine whether variance 1 is greater than variance threshold 1; if yes, set the peak state to unstable; if no, set the peak state to stable; Determine whether variance 2 is greater than variance threshold 2; if yes, set the peak state to unstable; if no, set the peak state to stable; Determine whether variance three is greater than variance threshold three; if yes, set the waveform state to unstable; if no, set the waveform state to stable; When any of the peak state, wave crest state and waveform state is unstable, the welding part is a pseudo defect; otherwise, the welding part is a real defect; The ultrasonic data includes reflected wave peak data and reflected wave morphology data in different directions; the reflected wave morphology data includes the number of peaks and waveform width.
4. The intelligent control method for weld detection based on machine vision according to claim 1 is characterized in that: The construction of the defect type identification model includes the following steps: Obtain several real defect images corresponding to each defect type; Divide the real defect images into training sets, validation sets, and test sets; Preprocess the real defect images of the training set, validation set, and test set; Select the convolutional neural network model VGGNet as the basic model; Train the basic model with the training set, and adjust the learning rate on the validation set to obtain the pre-trained model; By verifying the pre-trained model on the test set, we finally obtain a defect type recognition model whose input is the original image and outputs several confidence levels and their corresponding defect types.
5. The intelligent control method for weld detection based on machine vision according to claim 1 is characterized in that: The generating of defect level, defect category and formation cause according to the confidence level and the corresponding defect type includes: Obtaining the maximum confidence value, and determining whether the confidence value is greater than a classification threshold of one; If yes, then the defect type corresponding to the maximum confidence is obtained, and the defect level and formation cause corresponding to the defect type are searched in the defect type lookup table according to the defect type, and the defect level is used as the defect level of this time, and the formation cause is used as the formation cause of this time; the defect type is set as the defect category of this time; No, when the confidence is greater than the classification threshold two; arrange the confidences from large to small; obtain several defect types corresponding to several confidences within the set ranking, and search for several defect levels and formation causes corresponding to the corresponding defect types in the defect type lookup table according to the corresponding defect types; and perform weighted summation of each defect level by confidence to obtain the current defect level; set the current formation cause to a mixed formation cause; and integrate several formation causes into the current formation cause; set the current defect category to a mixed defect; otherwise, set the defect category to none, the defect level to none, and the formation cause to none.
6. The intelligent control method for weld detection based on machine vision according to claim 5 is characterized in that: The defect type lookup table includes several defect levels, corresponding defect types and formation causes; the defect levels correspond to defect types one by one; the defect types correspond to formation causes one by one.
7. The intelligent control method for weld detection based on machine vision according to claim 5 is characterized in that: The defect level of this time is obtained by weighted summing of each defect level by confidence, including: Get the confidence level Zi within the set ranking and the corresponding defect level Qi, where i is the number of the built-in confidence level of the set ranking; i=1, 2, ..., n, where n is the total number of built-in confidence levels of the set ranking; The defect level of this time is calculated by the formula QD=∑Qi×Zi / ∑Zi.
8. The intelligent control method for weld detection based on machine vision according to claim 1, characterized in that: The initial control scheme is generated according to the formation causes corresponding to the defect categories, including: Obtaining several formation causes and their corresponding standard control schemes; establishing a control lookup table according to the formation causes and the standard control schemes; Determine whether the formation cause is a mixed formation cause; if yes, search the corresponding standard control scheme in the control lookup table according to each formation cause, and extract several confidences corresponding to the mixed cause; sort the corresponding standard control schemes in descending order of confidence, and integrate the sorted standard control schemes into the initial control scheme; No, when the cause is none, an alarm signal is generated and an alarm operation is performed according to the alarm signal; Otherwise, the corresponding standard control scheme is obtained by searching in the control lookup table according to the corresponding formation cause, and the standard control scheme is used as the initial control scheme.
9. The intelligent control method for weld detection based on machine vision according to claim 1, characterized in that: The control scheme is obtained by adjusting the initial control scheme according to the comparison defect level and the defect level, including: Obtain the comparison defect level DD; calculate the effect level XD by the formula XD = DD-α×(1-exp(-β×DD))×DD; α is the proportional coefficient, β is the exponential coefficient, α and β∈(0,1); Determine whether the current defect level is less than the effect level; if yes, obtain the last initial control plan and use it as the current control plan; and set the number of abnormal controls to zero; If not, the initial control scheme is set as the control scheme; and the number of abnormal controls is increased by one; when the number of abnormal controls is greater than the control number threshold, an alarm signal is generated, and an alarm operation is performed according to the alarm signal.
10. A weld detection intelligent control system based on machine vision, applied to a weld detection intelligent control method based on machine vision according to any one of claims 1 to 9, characterized in that: include: Data acquisition module, defect detection module, defect identification module, control module and database; Data acquisition module: acquires the detection data of the welding position, wherein the detection data includes the original image; Defect detection module: extracts the original image from the detection data, inputs the original image into the defect detection model to obtain the corresponding image label; when the image label is a defect image, generates a verification signal; Defect recognition module: collect ultrasonic detection data according to the verification signal; determine whether the original image has a real defect according to the ultrasonic detection data; if the original image is a real defect, input the corresponding original image into the defect type recognition model to obtain several confidence levels and their corresponding defect types; generate defect levels, defect categories and formation causes according to the confidence levels and their corresponding defect types; Control module: generates an initial control plan according to the formation cause corresponding to the defect category; obtains the last defect level; and marking it as a comparison defect level; adjusting the initial control plan according to the comparison defect level and the defect level to obtain a control plan; and controlling the welding equipment according to the control plan.
Citation Information
Cited By
Crystal defect detection method based on machine vision
CN120161068A
Display panel defect detection method and system
CN120510119A
Spherical tank welding seam intelligent detection system fusing dynamic convolution and three-dimensional reconstruction
CN121010566A
Vehicle weld defect detection method and device, program product and electronic equipment
CN121544552A
Weld defect color image detection method and system based on YOLO-World
CN121810699A
