Pipe fitting self-adaptive positioning and intelligent cutting method and system based on machine vision

Abstract

The invention discloses a pipe fitting self-adaptive positioning and intelligent cutting method and system based on machine vision, and relates to the field of intelligent control, and the method comprises the steps: collecting a pipe fitting image through a multi-view industrial camera, extracting three-dimensional features through a convolutional neural network, and building a complete pipe fitting three-dimensional model; constructing an adaptive positioning model, and solving accurate position coordinates and attitude angles of the pipe fittings by adopting a CNN-LSTM hybrid deep learning algorithm; according to cutting parameter requirements, an optimal cutting path is generated through an A * path planning algorithm, and the cutting robot is controlled to execute operation; images are collected in real time in the cutting process for quality monitoring, and if deviation is found, motion parameters are dynamically adjusted; and archiving and storing the whole-process data, and continuously optimizing the positioning model by applying an incremental learning technology. The method has the advantages that precise three-dimensional modeling and positioning of the pipe fitting are achieved by combining machine vision with deep learning, and intelligent high-quality cutting of the pipe fitting is completed through A * path planning, real-time vision monitoring, PID deviation correction and incremental learning optimization.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent control, in particular to a pipe adaptive positioning and intelligent cutting method and system based on machine vision. BACKGROUND

[0002] With the rapid development of industrial manufacturing towards intelligence and automation, traditional pipe cutting technology has been difficult to meet the processing needs of high precision and high efficiency due to its dependence on manual positioning and fixed parameter cutting. Especially in the fields of aerospace and automobile manufacturing, pipes often have complex curved surfaces or non-standard shapes, and manual measurement is prone to errors. Moreover, rigid cutting programs cannot adapt to the deviations of incoming materials, resulting in unstable processing quality, high material waste rate and other problems. The maturity of machine vision technology provides a new way to solve this problem.

[0003] The current pipe cutting methods on the market have many shortcomings in multiple dimensions. In the positioning link, traditional methods mostly rely on manual teaching or single laser positioning, using traditional visual technologies such as template matching, which are easily disturbed by low texture, reflection and environmental noise of the pipe, and the pose estimation error is often more than ±0.5mm. Moreover, it is difficult to construct a complete three-dimensional model and identify defects such as surface depressions and protrusions. In terms of path planning, most mainstream devices execute according to preset parameters without dynamic adjustment based on real-time defect data. For example, plasma and flame cutting often result in waste rates exceeding 20% due to the lack of ability to bypass defects, and the large heat-affected zone easily causes material deformation, requiring additional secondary processing. During the cutting process, most methods lack real-time visual closed-loop control and rely only on simple sensor feedback, which responds slowly to quality problems such as uneven cutting seam width and slag adhesion, resulting in a high rework rate of up to 20%. More importantly, most methods on the market are static models without incremental learning mechanism, and manual model changing and debugging are required for different specifications of pipes. Long-term use does not improve accuracy, making it difficult to adapt to flexible production needs. SUMMARY

[0004] To improve existing methods and systems, a pipe adaptive positioning and intelligent cutting method and system based on machine vision is provided. This method uses machine vision combined with deep learning to achieve accurate three-dimensional modeling and positioning of pipes, and through A* path planning, real-time visual monitoring + PID correction and incremental learning optimization, it completes intelligent high-quality cutting of pipes.

[0005] To achieve the above purposes, the technical solution adopted by the present application is as follows:

[0006] The pipe adaptive positioning and intelligent cutting method based on machine vision comprises:

[0007] The pipe to be cut is preliminarily fixed, the surface image of the pipe is collected by a visual acquisition system composed of an industrial area array camera, and the collected multi-view original image is preprocessed;

[0008] Based on the pre-processed multi-view images, a deep learning feature extraction algorithm based on convolutional neural network is used to extract the key features of the pipe fittings, and a complete three-dimensional feature model of the pipe fittings in the visual coordinate system is obtained.

[0009] An adaptive positioning model is constructed based on the three-dimensional feature model of the pipe fittings, and the deviation of the pipe fitting port circle center coordinates, the weld position and the preset cutting reference is minimized as the objective function. The actual position coordinates and attitude angle of the pipe fittings are obtained by solving the adaptive positioning model through the CNN-LSTM deep learning model.

[0010] Based on the cutting requirements of the pipe fittings, the cutting length, cutting angle and cutting seam width parameters are obtained. Based on the actual position coordinates and attitude angle of the pipe fittings, the motion path of the cutting head is planned through the A* path planning algorithm.

[0011] Based on the cutting path and the matched process parameters, the cutting robot drives the cutting head to move. During the cutting process, the cutting area image is collected in real time, the feature parameters are extracted and compared with the preset threshold, and the cutting quality is judged.

[0012] If the cutting quality is unqualified or the relative position deviation between the cutting head and the pipe fittings exceeds the threshold, a deviation signal is generated, and the motion parameters of the cutting robot are adjusted until the cutting process meets the quality requirements.

[0013] After cutting, the specification parameters, visual acquisition data, positioning results, cutting process parameters and cutting quality data of the pipe fittings are stored, and the adaptive positioning model is updated and optimized through the incremental learning algorithm.

[0014] Preferably, the to-be-cut pipe fittings are preliminarily fixed, the surface images of the pipe fittings are collected by a visual acquisition system composed of industrial area array cameras, and the pre-processing of the collected multi-view original images specifically includes:

[0015] The to-be-cut pipe fittings are preliminarily fixed, and the surface images of the pipe fittings are synchronously collected by a visual acquisition system composed of industrial area array cameras.

[0016] Gaussian filtering is performed on each original image, and a convolution kernel is selected according to the image noise level to smooth the image noise while retaining the pipe fitting edge details.

[0017] The brightness and color temperature of the light source are adjusted to monitor the gray value of the pipe fitting surface image in real time, and the histogram equalization algorithm is used to adjust the image gray scale distribution and enhance the contrast between the pipe fitting surface and the background.

[0018] The image edges are extracted by the Canny algorithm, and the edge contours of the pipe fitting port, weld and defects are selected and retained.

[0019] Preferably, the pre-processed multi-view image is used to extract the key features of the pipe fitting by a deep learning feature extraction algorithm based on a convolutional neural network, and a complete three-dimensional feature model of the pipe fitting in a visual coordinate system is obtained, which specifically includes:

[0020] Based on the pre-processed multi-view image, an improved ResNet50 deep learning algorithm is selected for key feature extraction, including the center coordinates and radius parameters of the pipe fitting port, the positioning weld position coordinates of the pipe fitting surface, and the contour coordinates of the concave and convex defects, forming a multi-dimensional feature point set;

[0021] Corresponding relationships are established for the same type of features collected by different cameras, and feature matching candidate pairs are labeled. A random sample consensus algorithm is introduced to filter out effective matching pairs by calculating the geometric constraint error between feature points.

[0022] The multi-view effective matching pair feature data of the camera is combined to generate a complete three-dimensional feature model of the pipe fitting in a visual coordinate system.

[0023] Preferably, the three-dimensional feature model of the pipe fitting is used to construct an adaptive positioning model, with the deviation of the pipe fitting port center coordinates and the weld position from the preset cutting reference as the objective function, and the CNN-LSTM deep learning model is used to solve the adaptive positioning model to obtain the actual position coordinates and attitude angle of the pipe fitting, which specifically includes:

[0024] The deviation of the pipe fitting port center coordinates and the attitude angle from the preset cutting reference and the shortest distance between the weld and the preset path are used as the optimization target to establish the objective function.

[0025] The trained CNN-LSTM model is called, the CNN module extracts the spatial structure features of the pipe fitting three-dimensional feature model, and the LSTM module processes the time sequence correlation data collected by the multi-view image.

[0026] The generated pipe fitting three-dimensional feature model is converted into a tensor format recognizable by the model and input into the CNN-LSTM model for feature mapping and time sequence analysis through the pre-trained network parameters.

[0027] The model calculates and outputs the actual position coordinates of the pipe fitting and the pitch angle around the X-axis, the yaw angle around the Y-axis, and the roll angle around the Z-axis.

[0028] Preferably, the cutting length, cutting angle, and cutting seam width parameters are obtained based on the cutting requirements of the pipe fitting, and the actual position coordinates and attitude angle of the pipe fitting are used to plan the motion path of the cutting head through the A* path planning algorithm, which specifically includes:

[0029] The preset cutting length, cutting angle, and cutting seam width parameters are obtained based on the pipe cutting task requirements.

[0030] The actual position coordinates and attitude angles of the pipe fitting are called, the pipe fitting material parameters are imported, and the space and material constraints of the path planning are constructed.

[0031] Through the A* path planning algorithm, based on real-time comparison of the defect area in the three-dimensional feature model of the pipe fitting, the defect position is bypassed, and the motion path of the cutting head is calculated by combining the Manhattan distance and the Euclidean distance of the heuristic function;

[0032] The pre-constructed process parameter database is called, based on the current pipe fitting material and the input cutting parameters, the corresponding cutting process parameters are matched, and a complete cutting execution scheme is formed.

[0033] Preferably, the cutting robot is controlled to move the cutting head based on the cutting path and the matched process parameters, and the cutting area image is collected in real time during the cutting process, the feature parameters are extracted and compared with the preset threshold, and the cutting quality is judged, which specifically includes:

[0034] Based on the obtained cutting head motion path and complete cutting execution scheme, the cutting robot is controlled to move the cutting head according to the path to perform pipe cutting operation;

[0035] The cutting area image is collected in real time during the cutting process, the cutting area image collected in real time is analyzed through image recognition algorithm, and the cutting seam width, edge flatness and slag adhesion amount parameters are extracted;

[0036] The extracted parameters are compared with the preset double threshold, if the parameters are within the qualified threshold, the cutting is normally carried out, if the parameters are unqualified or the position deviation exceeds the threshold, a deviation signal is generated.

[0037] Preferably, if the cutting quality is judged to be unqualified or the relative position deviation of the cutting head and the pipe exceeds the threshold, a deviation signal is generated, the motion parameters of the cutting robot are adjusted until the cutting process meets the quality requirements, which specifically includes:

[0038] The generated deviation signal is received in real time, and the deviation type is determined, including position deviation and process parameter deviation;

[0039] The deviation signal is input into the PID controller, the controller calls the parameters optimized by the particle swarm, and transmits them to the cutting robot and the cutting device;

[0040] The optimized parameters are specifically: if it is a position deviation, the motion parameters of the cutting robot are adjusted, and if it is a process parameter deviation, the cutting speed, power or gas pressure of the cutting device are corrected;

[0041] After the parameter adjustment is completed, the cutting area visual monitoring is performed again, and whether the cutting quality and the position deviation meet the requirements is judged, if not, the deviation adjustment process is repeated until the quality standard is met.

[0042] Preferably, after the cutting is completed, the specification parameters of the pipe fitting, the visual acquisition data, the positioning result, the cutting process parameters and the cutting quality data are stored, and the adaptive positioning model is updated and optimized through an incremental learning algorithm, and the specific steps include:

[0043] After the cutting is completed, the pipe fitting specification parameters, the positioning result, the cutting process parameter structured data and the visual acquisition original image, the preprocessed image unstructured data are classified and integrated and stored;

[0044] The sample data meeting the model optimization requirements are reserved through data integrity and consistency verification, the knowledge of the old CNN-LSTM model is transferred to the new model through the incremental learning algorithm based on knowledge distillation, and the new model learns the new data while retaining the old data learning achievements;

[0045] When the amount of new effective data in the database reaches a threshold value, the model optimization process is started.

[0046] Further, a pipe adaptive positioning and intelligent cutting system based on machine vision is proposed, which comprises:

[0047] The three-dimensional feature modeling module: the improved ResNet50 algorithm is used to extract the pipe fitting port, weld and defect features, and the complete three-dimensional feature model is constructed by combining multi-view matching point cloud data;

[0048] The adaptive positioning module: based on the CNN-LSTM hybrid model, the actual coordinates and attitude angle of the pipe fitting are solved, and the geometric deviation of the port center, the weld and the preset reference is minimized;

[0049] The path planning module: through the A* algorithm, the three-dimensional model of the pipe fitting and the material parameters are combined to plan the optimal cutting path around the defect area, and the process parameter library is matched to generate an execution scheme;

[0050] The cutting control module: the cutting robot is driven to move according to the planned path, and the cutting speed, power and gas pressure process parameters are adjusted in real time;

[0051] The dynamic correction module: the PID controller is used to correct the position deviation and process parameter deviation in real time, and the closed-loop control is performed until the cutting quality meets the standard;

[0052] The data storage and optimization module: the cutting whole-process data are stored in a structured manner, and the CNN-LSTM model parameters are updated regularly based on the incremental learning of knowledge distillation;

[0053] The processor: the processor is used for processing the calculation process of each formula and the construction calculation process of each model.

[0054] Compared with the prior art, the advantages of the present application are that:

[0055] The image is collected by an industrial area array camera, and the pipe fitting port, weld and defect features are accurately extracted through preprocessing and improved ResNet50 algorithm to construct a complete three-dimensional model; then, the CNN-LSTM model is relied on to solve positioning with the minimum deviation as the target, so that the adaptive and accurate capture of the pipe fitting position and attitude angle is realized, and the problem of large traditional positioning error is solved. In the path planning link, the A* algorithm is combined with the pipe fitting material and defect data to bypass the defect area and match the process parameter library to generate the optimal cutting path. The area image is monitored in real time during cutting, and the PID dynamic deviation correction mechanism is matched to timely adjust the robot movement or cutting process parameters, forming a quality closed-loop control. After cutting, the whole process data is stored, the positioning model is optimized through incremental learning, the long-term adaptability is improved, and the intelligent, efficient and high-quality positioning to cutting is realized, which greatly improves the cutting precision and stability. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The method proposed by the present application is shown in the schematic diagram;

[0057] Figure 2 The image acquisition and preprocessing schematic diagram proposed by the present application is shown in the schematic diagram;

[0058] Figure 3 The three-dimensional feature model of the pipe fitting is shown in the schematic diagram;

[0059] Figure 4 The pipe fitting position coordinates and attitude angle are shown in the schematic diagram;

[0060] Figure 5 The cutting head path planning schematic diagram proposed by the present application is shown in the schematic diagram;

[0061] Figure 6 The cutting deviation schematic diagram proposed by the present application is shown in the schematic diagram;

[0062] Figure 7 The cutting adjustment schematic diagram proposed by the present application is shown in the schematic diagram;

[0063] Figure 8 The model optimization schematic diagram proposed by the present application is shown in the schematic diagram. DETAILED DESCRIPTION

[0064] The following description is used to disclose the present application so that those skilled in the art can implement the present application. The preferred embodiments in the following description are only as examples, and other obvious modifications can be thought of by those skilled in the art.

[0065] The pipe fitting adaptive positioning and intelligent cutting system based on machine vision comprises:

[0066] The three-dimensional feature modeling module adopts the improved ResNet50 algorithm to extract the pipe fitting port, weld and defect features, and combines the multi-view matching point cloud data to construct a complete three-dimensional feature model;

[0067] Adaptive positioning module: based on CNN-LSTM hybrid model to solve the actual coordinates and attitude angle of the pipe fittings, minimize the geometric deviation of the port circle center, weld and the preset reference;

[0068] Path planning module: through A* algorithm combined with pipe fittings three-dimensional model and material parameters, plan the optimal cutting path to bypass the defect area, and match the process parameter library to generate the execution scheme;

[0069] Cutting control module: drive the cutting robot to move according to the planned path, and adjust the cutting speed, power and gas pressure process parameters in real time;

[0070] Dynamic correction module: use PID controller to correct position deviation and process parameter deviation in real time, and close loop control until the cutting quality meets the standard;

[0071] Data storage and optimization module: structured storage of cutting whole process data, incremental learning based on knowledge distillation to update CNN-LSTM model parameters regularly;

[0072] Processor: the processor is used for processing the calculation process of each formula and the construction calculation process of each model.

[0073] Referring to Figure 1 As shown, the pipe adaptive positioning and intelligent cutting method based on machine vision comprises:

[0074] Step one: fix the pipe to be cut, collect the pipe surface image through the visual acquisition system composed of industrial area array camera, and pretreat the collected multi-view original image;

[0075] Step two: based on the pretreated multi-view image, use the deep learning feature extraction algorithm based on convolutional neural network to extract the key features of the pipe, and obtain the complete three-dimensional feature model of the pipe in the visual coordinate system;

[0076] Step three: based on the three-dimensional feature model of the pipe, construct an adaptive positioning model, take the minimum deviation of the pipe port circle center coordinates, weld position and preset cutting reference as the objective function, solve the adaptive positioning model through CNN-LSTM deep learning model, and obtain the actual position coordinates and attitude angle of the pipe;

[0077] Step four: based on the cutting requirement of the pipe, obtain the cutting length, cutting angle and cutting seam width parameters, based on the actual position coordinates and attitude angle of the pipe, plan the motion path of the cutting head through A* path planning algorithm;

[0078] Step five: control the cutting robot to drive the cutting head to move based on the cutting path and matched process parameters, collect images of the cutting area in real time during the cutting process, extract feature parameters and compare them with preset threshold values to determine the cutting quality;

[0079] Step six: if the cutting quality is unqualified or the relative position deviation between the cutting head and the pipe exceeds the threshold value, generate a deviation signal, adjust the motion parameters of the cutting robot until the cutting process meets the quality requirements;

[0080] Step seven: after cutting is completed, store the specification parameters of the pipe, the visual acquisition data, the positioning results, the cutting process parameters, and the cutting quality data, and update and optimize the self-adaptive positioning model through an incremental learning algorithm.

[0081] Referring to Figure 2 As shown in the figure, the pipe to be cut is preliminarily fixed, the surface image of the pipe is collected by a visual acquisition system composed of an industrial area array camera, and the collected multi-view original image is preprocessed, which specifically includes:

[0082] The pipe to be cut is preliminarily fixed, and the surface image of the pipe is synchronously collected by a visual acquisition system composed of an industrial area array camera;

[0083] Gaussian filter processing is performed on each original image, and a convolution kernel is selected according to the image noise level to smooth the image noise while preserving the pipe edge details;

[0084] The brightness and color temperature of the light source are adjusted to monitor the gray value of the pipe surface image in real time, and the histogram equalization algorithm is used to adjust the image gray scale distribution to enhance the contrast between the pipe surface and the background;

[0085] The image edge is extracted by the Canny algorithm, and the edge profile of the pipe port, weld, and defect is selected and preserved.

[0086] Specifically, when using the Canny algorithm, the high threshold value is initially set to 100 and the low threshold value is set to 50. After applying the algorithm, the edge extraction result is checked. If the extracted edge is broken, the low threshold value needs to be gradually reduced. If there are a large number of miscellaneous edges, the high threshold value needs to be gradually increased until the complete pipe edge profile is extracted, including the circular profile of the port, the linear profile of the weld, and the irregular profile of the defect, without obvious breakage or miscellaneous edge interference, laying a foundation for subsequent feature matching.

[0087] Referring to Figure 3 Based on the preprocessed multi-view images, a deep learning feature extraction algorithm based on convolutional neural network is used to extract the key features of the pipe, and a complete three-dimensional feature model of the pipe in the visual coordinate system is obtained, which specifically includes:

[0088] Based on the pre-processed multi-view images, an improved ResNet50 deep learning algorithm is selected for key feature extraction, including the center coordinates of the pipe port, the radius parameter, the positioning weld position coordinates of the pipe surface, and the contour coordinates of the concave and convex defects, forming a multi-dimensional feature point set;

[0089] Corresponding relationships are established for the same type of features collected by different cameras, and feature matching candidate pairs are marked. A random sample consensus algorithm is introduced to filter out effective matching pairs by calculating the geometric constraint error between feature points.

[0090] The multi-view effective matching pair feature data of the camera is combined to generate a complete three-dimensional feature model of the pipe in the visual coordinate system.

[0091] Specifically, a plurality of edge points are marked along the arc trajectory of the port in the image. The approximate center region of the port is determined by the distribution trend of these points. Then the distance from each marked point to different candidate points in the region is measured, and the candidate point with the smallest distance deviation is selected as the center coordinate. If the distance difference of most marked points to the candidate point is less than 0.05mm, the candidate point is the accurate center. Then the distance from the center to any 10 marked points is calculated, and the average value is taken as the port radius. If the deviation of a single distance from the average value is greater than 0.1mm, the marked point is excluded and the radius is recalculated to ensure the accuracy of the radius.

[0092] By comparing the edge difference between the weld and the pipe base, the weld appears as a continuous linear protrusion or recess line in the edge image. First, the image is scanned along the axial direction of the pipe in segments to identify linear features that differ from the edge trend of the base. Then the start point, end point and intermediate key inflection point of the linear feature are marked. These points are connected in turn to form a complete weld trajectory, which is the weld position. If the weld has local fractures, transition points are added to make the weld trajectory continuous according to the trend of the lines on both sides of the fracture.

[0093] Focus on the irregular closed line region in the edge image. The concave defect appears as a closed edge that is inwardly recessed, and the convex defect appears as a closed edge that is outwardly protruding. First, mark the contour points along the closed line, and then record the coordinates of each point in the order of marking to form the complete contour of the defect. At the same time, distinguish the defect type to provide a basis for avoiding defects in subsequent path planning.

[0094] Referring to Figure 4 As shown in the figure, an adaptive positioning model is constructed based on the three-dimensional feature model of the pipe. The objective function is to minimize the deviation of the center coordinates of the pipe port, the weld position and the pre-set cutting reference. The CNN-LSTM deep learning model is used to solve the adaptive positioning model to obtain the actual position coordinates and attitude angle of the pipe, which specifically includes:

[0095] A target function is established with the deviation of the port center coordinates and attitude angle of the pipe fitting from the preset cutting reference and the shortest distance between the weld and the preset path as the optimization objectives;

[0096] The trained CNN-LSTM model is called, the CNN module extracts the spatial structure features of the pipe fitting three-dimensional feature model, and the LSTM module processes the time sequence correlation data collected by the multi-view images;

[0097] The generated pipe fitting three-dimensional feature model is converted into a tensor format recognizable by the model, and is input into the CNN-LSTM model for feature mapping and time sequence analysis through the pre-trained network parameters;

[0098] After the model calculation, the actual position coordinates of the pipe fitting and the pitch angle around the X-axis, the yaw angle around the Y-axis, and the roll angle around the Z-axis are output.

[0099] Specifically, the CNN module extracts the spatial features of the complete three-dimensional feature model of the pipe fitting, and disassembles it into multiple feature slices according to the spatial dimensions: for example, a cross-sectional slice is taken every 10 mm along the axial direction of the pipe fitting, and each slice contains spatial information such as the port profile, weld segment, and defect distribution at that position; the CNN module analyzes these slices layer by layer to identify the spatial correlation between features: for example, by comparing the positions of the port centers in different slices, it can determine whether the pipe fitting has an axial tilt; by capturing the orientation of the weld in each slice, it can sort out the overall spatial trajectory of the weld, and finally convert the three-dimensional features into structured spatial feature data to provide static spatial position basis for positioning;

[0100] The LSTM module processes time sequence information, as the multi-view images are collected synchronously in time sequence, and the images at different time points may have slight feature changes; the LSTM module organizes the camera's collection time sequence according to the time axis and learns the correlation rules in the time sequence data: for example, the position of the port center in the previous frame and the position change trend of the next frame, to determine whether the displacement is temporary interference or real deviation; by analyzing the stability of the weld trajectory in consecutive frames, it can filter out false deviations caused by image noise, and finally output dynamic time sequence correction basis.

[0101] Referring to Figure 5 Based on the cutting length, cutting angle, and cutting seam width parameters obtained based on the cutting requirements of the pipe fitting, and based on the actual position coordinates and attitude angle of the pipe fitting, the motion path of the cutting head is planned through the A* path planning algorithm, which specifically includes:

[0102] The preset cutting length, cutting angle, and cutting seam width parameters are obtained based on the cutting task requirements of the pipe fitting;

[0103] The actual position coordinates and attitude angle of the pipe fitting are called, the pipe fitting material parameters are imported, and the spatial and material constraints for path planning are constructed;

[0104] Through the A* path planning algorithm, based on real-time comparison of the defect area in the three-dimensional feature model of the pipe fitting, the defect position is bypassed, and the cutting head motion path is calculated by combining the Manhattan distance and the Euclidean distance of the heuristic function;

[0105] The pre-constructed process parameter database is called, and based on the current pipe fitting material and the input cutting parameters, the corresponding cutting process parameters are matched to form a complete cutting execution scheme.

[0106] Specifically, taking the actual position coordinates of the pipe fitting as the reference, the starting point and the ending point of each cutting segment are determined: if the first segment of 300mm is cut, the starting point is set as the coordinate corresponding to the center of the left end port of the pipe fitting, and the ending point is set as the coordinate after moving 300mm along the axial direction of the pipe fitting. For the bevel cutting requirement, the three-dimensional coordinate offset of the ending point needs to be calculated according to the angle;

[0107] The path nodes are set according to the length of the pipe fitting: one path node is set every 50mm, and straight lines or smooth curves are used to connect the nodes to avoid precision deviation caused by sudden turning of the cutting head; at the same time, the extracted defect features are referred to: if there is a concave defect with a diameter exceeding 2mm on the surface of the pipe fitting, the node will be offset away from the defect when planning the path of the corresponding segment to ensure that the distance between the cutting head motion trajectory and the edge of the defect is not less than 5mm, preventing uneven stress caused by defects during cutting;

[0108] For the turning or polyline area in the path, smoothing processing is performed: the right-angle turning is changed to arc transition to avoid the cutting head from stopping at the turning point due to sudden speed change; for long-distance straight-line cutting, a speed buffer segment is set at the middle node to ensure smooth running of the cutting head and reduce path deviation caused by inertia.

[0109] Referring to Figure 6 As shown in the figure, based on the cutting path and the matched process parameters, the cutting robot is controlled to drive the cutting head to move, and the cutting area image is collected in real time during the cutting process, the feature parameters are extracted and compared with the preset threshold to judge the cutting quality, which specifically includes:

[0110] Based on the obtained cutting head motion path and the complete cutting execution scheme, the cutting robot is controlled to drive the cutting head to move according to the path for pipe cutting operation;

[0111] The cutting area image is collected in real time during the cutting process, and the real-time collected cutting area image is analyzed through image recognition algorithm to extract the cutting seam width, edge flatness and slag adhesion amount parameters;

[0112] The extracted parameters are compared with the preset double threshold value: if the parameters are within the qualified threshold value, the cutting is normal, if the parameters are not qualified or the position deviation exceeds the threshold value, a deviation signal is generated.

[0113] Specifically, for each frame of collected image, three types of quality parameters are extracted: first, the slit width, by comparing the visual proportion of the slit in the image with the preset standard slit width, and observing whether the slit width is uniform; second, the edge flatness, observing whether the edge of the slit appears jagged ups and downs; third, the amount of slag adhesion, by identifying the color and shape of the slag in the image, the number of slag adhered around the slit is counted; during the extraction process, if the parameters of a certain frame of image are abnormal, the cutting position corresponding to the frame will be automatically marked, facilitating subsequent tracing of deviation reasons;

[0114] The extracted parameters are compared with the preset threshold values, and the quality is judged according to different cases: if the slit width is 0.4-0.6mm, the edge flatness fluctuation is less than 0.05mm, and the slag adhesion amount meets the requirements, it is determined to be qualified, and the cutting continues; if the parameters are in the early warning threshold, an early warning signal is sent, but the work is not suspended; if the parameters exceed the early warning threshold, it is immediately determined to be unqualified, the cutting work is suspended synchronously, and waits for subsequent deviation adjustment. During the judgment process, the images of unqualified frames and the corresponding cutting positions are automatically saved, providing a basis for deviation analysis, and ensuring that the quality of each cutting section is within a controllable range.

[0115] Referring to Figure 7 If the cutting quality is determined to be unqualified or the relative position deviation between the cutting head and the pipe exceeds the threshold value, a deviation signal is generated, the motion parameters of the cutting robot are adjusted until the cutting process meets the quality requirements, which specifically includes:

[0116] The generated deviation signal is received in real time, and the type of deviation is determined, including position deviation and process parameter deviation;

[0117] The deviation signal is input into the PID controller, the controller calls the parameters optimized by the particle swarm, and transmits them to the cutting robot and the cutting device;

[0118] The optimized parameters are specifically: if it is a position deviation, the motion parameters of the cutting robot are adjusted, and if it is a process parameter deviation, the cutting speed, power or gas pressure of the cutting device are corrected;

[0119] After the parameter adjustment is completed, the cutting area visual monitoring is performed again to determine whether the cutting quality and position deviation meet the requirements, and if not, the deviation adjustment process is repeated until the quality standards are met.

[0120] Specifically, for the position deviation, the motion parameters of the cutting robot are adjusted: if it is determined that the pipe is axially displaced, the cutting head is controlled to fine-tune the motion trajectory in the opposite direction, while the cutting speed is reduced to reduce the impact of inertia on the position; if it is determined that the cutting head motion is unstable, the motion acceleration is reduced to make the cutting head motion more gentle; adjustment follows the principle of small amplitude and multiple adjustments: the displacement is not more than 0.05mm, the speed is not more than 50mm / min, and the acceleration is not more than 10mm / s², to avoid new deviations caused by a single adjustment with too large amplitude;

[0121] For parameter deviation, the process parameters of the cutting device are adjusted: if the seam width exceeds the standard due to excessive speed, the speed is reduced by 50mm / min, and the seam width is reduced by about 0.05mm; if the flatness is poor due to insufficient power, the power is increased by 100W, and the cutting fusion effect is improved by about 5%; if there is too much slag due to low gas pressure, the pressure is increased by 0.1MPa, and the slag adhesion is reduced by about 30%; when adjusting, the recommended range of the process parameter database should be referred to to ensure that the adjusted parameters are still within the safe and effective range.

[0122] Referring to Figure 8 As shown in the figure, after cutting is completed, the specification parameters of the pipe, the visual acquisition data, the positioning results, the cutting process parameters, and the cutting quality data are stored, and the self-adaptive positioning model is updated and optimized through an incremental learning algorithm, which specifically includes:

[0123] After cutting is completed, the pipe specification parameters, positioning results, cutting process parameter structured data, and visual acquisition original image, preprocessed image unstructured data are classified and integrated for storage;

[0124] Through data integrity and consistency verification, sample data meeting the model optimization requirements are retained, and through the incremental learning algorithm based on knowledge distillation, the knowledge of the old CNN-LSTM model is transferred to the new model through KL divergence loss, and the new model learns the new data while retaining the learning achievements of the old data;

[0125] When the amount of effective data added to the database reaches the threshold, the model optimization process is started.

[0126] Specifically, based on the screened effective data set, the original positioning model is incrementally optimized, the core logic is to retain old knowledge and learn new knowledge, extract the core experience of the old model, for example, the positioning accuracy of the old model for stainless steel straight pipe is high, and this part of the ability needs to be retained during optimization to avoid the decline of the old scene accuracy due to learning new data; the new screened effective data is divided into a training subset and a validation subset, the training subset is used to learn the new scene rule, and the model parameters are adjusted gradually during the learning process to ensure that the model can adapt to the positioning needs of new materials and new types of pipe fittings; compare the prediction difference between new data and old model, if the positioning deviation of a certain type of pipe fitting in new data is large, then focus on adjusting the model for the feature data of this type of pipe fitting, improve the positioning sensitivity of thin-walled pipe fittings, and the KL divergence loss formula of incremental learning is:

[0127]

[0128] wherein, is the KL divergence loss value, the smaller the value represents the smaller the output distribution difference between the new model and the old model, is the sample data amount, is the output probability distribution of the old CNN-LSTM model for the ith sample, is the output probability distribution of the new CNN-LSTM model for the ith sample, is the minimum value, used to avoid calculation error caused by denominator being 0;

[0129] During optimization, the small amplitude iteration principle is followed, only 20%-30% of the parameters of the model are adjusted each time, to avoid the decline of the overall stability of the model caused by large modification; at the same time, the parameter adjustment trajectory is recorded, which is convenient for subsequent backtracking adjustment reason when the accuracy fluctuates.

[0130] It should be noted that: the above-mentioned sequence of the embodiments of the present application is only for description, not representing the advantages and disadvantages of the embodiments. And the above describes the specific embodiments of the present specification. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0131] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments.

[0132] The above only describes the preferred embodiments of the present application, and does not limit the present application, any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for machine vision-based adaptive positioning and intelligent cutting of pipe fittings, characterized in that, The application relates to a pipe cutting method and device based on deep learning. The application comprises the following steps: Preliminarily fixing the pipe to be cut, collecting the pipe surface image through a visual acquisition system composed of an industrial area array camera, and pre-processing the collected multi-view original image; Based on the pre-processed multi-view image, a deep learning feature extraction algorithm based on a convolutional neural network is used to extract the key features of the pipe, and a complete three-dimensional feature model of the pipe in the visual coordinate system is obtained; Based on the three-dimensional feature model of the pipe, an adaptive positioning model is constructed, the deviation of the pipe port center coordinates, the weld position and the preset cutting reference is minimized as the objective function, the adaptive positioning model is solved through a CNN-LSTM deep learning model, and the actual position coordinates and attitude angle of the pipe are obtained; Based on the cutting requirement of the pipe, the cutting length, cutting angle and cutting seam width parameters are obtained, and based on the actual position coordinates and attitude angle of the pipe, the motion path of the cutting head is planned through an A* path planning algorithm; Based on the cutting path and the matched process parameters, the cutting robot is controlled to drive the cutting head to move, the cutting area image is collected in real time during the cutting process, the feature parameters are extracted and compared with the preset threshold, and the cutting quality is judged; If the cutting quality is unqualified or the relative position deviation between the cutting head and the pipe exceeds the threshold, a deviation signal is generated, the motion parameters of the cutting robot are adjusted, and the cutting process is adjusted until the quality requirement is met; 2. The machine vision based pipe adaptive positioning and intelligent cutting method according to claim 1, wherein, After the cutting is completed, the specification parameters, visual acquisition data, positioning results, cutting process parameters and cutting quality data of the pipe are stored, and the adaptive positioning model is updated and optimized through an incremental learning algorithm. The preliminary fixation of the pipe to be cut, the collection of the pipe surface image through the visual acquisition system composed of the industrial area array camera and the pre-processing of the collected multi-view original image specifically comprises the following steps: The pipe to be cut is preliminarily fixed, and the pipe surface image is synchronously collected through the visual acquisition system composed of the industrial area array camera; Gaussian filter processing is performed on each original image, a convolution kernel is selected according to the image noise degree, and the image noise is smoothed while the pipe edge details are reserved; The brightness and color temperature of the light source are adjusted to monitor the pipe surface image gray value in real time, the image gray distribution is adjusted through a histogram equalization algorithm, and the contrast between the pipe surface and the background is enhanced; 3. The machine vision based pipe adaptive positioning and intelligent cutting method according to claim 1, wherein, The image edge is extracted through a Canny algorithm, and the edge profiles of the pipe port, the weld and the defects are screened and reserved. Based on the pre-processed multi-view image, the deep learning feature extraction algorithm based on the convolutional neural network is used to extract the key features of the pipe, and a complete three-dimensional feature model of the pipe in the visual coordinate system is obtained, which specifically comprises the following steps: Based on the pre-processed multi-view image, an improved ResNet50 deep learning algorithm is selected to extract key features, the key features include the center coordinates and radius parameters of the pipe port, the positioning weld position coordinates and the profile coordinates of the concave and convex defects on the pipe surface, and a multi-dimensional feature point set is formed; Corresponding relationships of the same type of features collected by different cameras are established, feature matching candidate pairs are marked, a random sample consensus algorithm is introduced, geometric constraint errors between feature points are calculated, and effective matching pairs are screened out. The multi-view effective matching feature data of the camera is combined to generate a complete three-dimensional feature model of the pipe fitting in a visual coordinate system.

4. The machine vision based pipe adaptive positioning and intelligent cutting method according to claim 1, wherein, The three-dimensional feature model of the pipe fitting constructs an adaptive positioning model, takes the deviation of the pipe fitting port circle center coordinates, the weld position and the preset cutting reference as the objective function, and solves the adaptive positioning model through a CNN-LSTM deep learning model to obtain the actual position coordinates and attitude angle of the pipe fitting, which specifically includes: Taking the deviation of the pipe fitting port circle center coordinates, the attitude angle and the preset cutting reference and the shortest distance between the weld and the preset path as the optimization target, a target function is established; The trained CNN-LSTM model is called, the CNN module extracts the spatial structure features of the pipe fitting three-dimensional feature model, and the LSTM module processes the time sequence correlation data collected by the multi-view image; The generated pipe fitting three-dimensional feature model is converted into a tensor format recognizable by the model, input into the CNN-LSTM model, and feature mapping and time sequence analysis are performed through the pre-trained network parameters; The model calculates and outputs the actual position coordinates of the pipe fitting and the pitch angle around the X-axis, the yaw angle around the Y-axis and the roll angle around the Z-axis.

5. The machine vision based pipe adaptive positioning and intelligent cutting method as claimed in claim 1, wherein, The cutting length, cutting angle and cutting seam width parameters are obtained based on the cutting requirements of the pipe fitting, and the motion path of the cutting head is planned through the A* path planning algorithm based on the actual position coordinates and attitude angle of the pipe fitting, which specifically includes: The preset cutting length, cutting angle and cutting seam width parameters are obtained based on the cutting requirements of the pipe fitting; The actual position coordinates and attitude angle of the pipe fitting are called, the pipe fitting material parameters are imported, and the spatial and material constraints of path planning are constructed; Based on the real-time comparison of the defect area in the pipe fitting three-dimensional feature model, the A* path planning algorithm is used to calculate the motion path of the cutting head by avoiding the defect position and using the Manhattan distance and Euclidean distance combined heuristic function; The pre-constructed process parameter database is called, the corresponding cutting process parameters are matched based on the current pipe fitting material and the input cutting parameters, and a complete cutting execution scheme is formed.

6. The machine vision based pipe adaptive positioning and intelligent cutting method as claimed in claim 1, wherein, Based on the obtained cutting head motion path and complete cutting execution scheme, the cutting robot is controlled to drive the cutting head to move according to the path to perform pipe cutting operation; During the cutting process, the cutting area image is collected in real time, the cutting seam width, edge flatness and slag adhesion parameters are extracted by analyzing the real-time collected cutting area image through the image recognition algorithm, and the extracted parameters are compared with the preset double threshold value. If the parameters are within the qualified threshold value, the cutting is normal, if they are unqualified or the position deviation exceeds the threshold value, a deviation signal is generated. If the cutting quality is judged to be unqualified or the relative position deviation of the cutting head and the pipe fitting exceeds the threshold value, a deviation signal is generated, and the motion parameters of the cutting robot are adjusted until the cutting process meets the quality requirements, which specifically includes:

7. The machine vision based pipe adaptive positioning and intelligent cutting method as claimed in claim 1, wherein, The generated deviation signal is received in real time to determine the type of deviation, including position deviation and process parameter deviation. ​ The deviation signal is input into a PID controller, the controller calls the parameters optimized by the particle swarm, and is transmitted to the cutting robot and the cutting device; The optimized parameters are as follows: if it is a position deviation, the motion parameters of the cutting robot are adjusted, and if it is a process parameter deviation, the cutting speed, power or gas pressure of the cutting device is corrected; After the parameter adjustment is completed, the cutting area visual monitoring is re-performed, and it is judged whether the cutting quality and the position deviation meet the requirements, and if not, the deviation adjustment process is repeated until the quality standard is met.

8. The machine vision based pipe adaptive positioning and intelligent cutting method as claimed in claim 1, wherein, After the cutting is completed, the specification parameters of the pipe fitting, the visual acquisition data, the positioning results, the cutting process parameters and the cutting quality data are stored, and the adaptive positioning model is updated and optimized through an incremental learning algorithm, specifically including: After the cutting is completed, the pipe fitting specification parameters, the positioning results, the cutting process parameter structured data and the visual acquisition original image, the preprocessed image unstructured data are classified and integrated and stored; Through data integrity and consistency verification, sample data meeting the model optimization requirements are reserved, and through the incremental learning algorithm based on knowledge distillation, the knowledge of the old CNN-LSTM model is transferred to the new model through KL divergence loss, and the new model learns the new data while retaining the old data learning results; When the amount of new effective data in the database reaches a threshold value, the model optimization process is started.

9. A machine vision based adaptive positioning and intelligent cutting system for realizing the machine vision based adaptive positioning and intelligent cutting method according to any one of claims 1-8, characterized in that, Including: The three-dimensional feature modeling module: the improved ResNet50 algorithm is used to extract the pipe fitting port, weld and defect features, and the complete three-dimensional feature model is constructed by combining multi-view matching point cloud data; The adaptive positioning module: based on the CNN-LSTM hybrid model, the actual coordinates and attitude angle of the pipe fitting are solved, and the geometric deviation of the port center, the weld and the preset reference is minimized; The path planning module: through the A* algorithm, the three-dimensional model of the pipe fitting and the material parameters are combined to plan the optimal cutting path around the defect area, and the process parameter library is matched to generate an execution scheme; The cutting control module: the cutting robot is driven to move according to the planned path, and the cutting speed, power and gas pressure process parameters are adjusted in real time; The dynamic correction module: a PID controller is used to correct the position deviation and the process parameter deviation in real time, and closed-loop control is performed until the cutting quality meets the standard; The data storage and optimization module: structured storage of cutting whole-process data, and periodic updating of CNN-LSTM model parameters based on knowledge distillation incremental learning; The processor: the processor is used to process the calculation process of each formula and the construction calculation process of each model.

Images