Coiled tubing pipe arrangement state recognition and adjustment method and device based on machine vision

Abstract

This application provides a machine vision-based method and apparatus for continuous pipe arrangement status recognition and adjustment. The method includes: real-time acquisition of pipe arrangement operation images in the roller pipe arrangement operation area, and image preprocessing of the pipe arrangement operation images; determining the image features of the preprocessed pipe arrangement operation images, and determining the corresponding current pipe arrangement status based on the image features and a preset pipe arrangement status recognition model; and performing corresponding roller pipe arrangement adjustment operations based on the current pipe arrangement status. This application can accurately identify the pipe arrangement status and automatically adjust it, reducing the risks of continuous pipe operation caused by human factors.

Description

Technical Field

[0001] This application relates to the field of computer vision, specifically to a method and apparatus for continuous pipe array state recognition and adjustment based on machine vision. Background Technology

[0002] Traditional hydraulic coiled tubing installers mainly consist of key components such as an injection head, control chamber, drum, and coiled tubing. The drum assembly, as one of the core control units of the coiled tubing installer, is primarily used for transporting the coiled tubing and laying it during the operation. During the initial tubing operation, the coiled tubing is wound into the drum via a guide under the combined action of the injection head and the drum. During the lowering operation, the drum motor provides a reverse torque to the drum to straighten the tubing, and the injection head pulls the tubing out of the drum. When the position of the tubing feeder on the drum of a traditional hydraulic coiled tubing installer deviates, the winding angle may be too large or too small during layer transitions on the drum. The size of the deviation angle directly affects the neatness of subsequent tubing laying. Disordered coiled tubing winding leads to external wear, reduced drum coil capacity, decreased tubing lifespan, and the need for repeated tubing laying.

[0003] Currently, most continuous tube winding systems using rollers, both domestically and internationally, employ a mechanical screw drive system. The drum rotates, driving the screw, and the winding trolley slides left and right with the screw to complete the winding process. In actual operation, the screw tongue is prone to wear and jamming. When the continuous tube transitions close to the edge of the drum spokes, the winding angle can easily deviate, causing subsequent winding disorder. This necessitates timely human intervention to anticipate potential risks and force manual winding. Subsequently, manufacturers have developed mechanical, hydraulic, and electrically controlled roller winding devices. While these devices offer better transmission accuracy and reliability, and alleviate winding problems, they still require manual intervention in actual operation, hindering true automation.

[0004] Current technology still relies on on-site manual identification, judgment, and forced pipe placement. The main drawbacks of this existing technology are:

[0005] ① It requires high skills from operators and carries the risk of misoperation;

[0006] ② Complex processes involve long single-well operation times, numerous monitoring points, complex operating procedures, and high labor intensity;

[0007] ③ Identifying abnormal operating conditions such as pipe laying relies on the operator's experience, which can easily lead to inaccurate predictions, improper handling, or untimely responses.

[0008] ④ The degree of automation is low, and the operation and control of equipment rely on manual labor, requiring a large number of people and resulting in high labor intensity.

[0009] Therefore, as the operational processes become increasingly complex and the requirements for precision in operations become ever higher, the importance of developing more stable and efficient new automated operating machines is becoming increasingly prominent. Exploring new technologies to automatically identify the status of the pipe arrangement and provide timely warnings will have significant practical implications for the automated, scientific, and efficient operation of continuous tube equipment. Summary of the Invention

[0010] To address the problems in the prior art, this application provides a machine vision-based method and apparatus for continuous pipe arrangement status identification and adjustment, which can accurately identify the pipe arrangement status and automatically adjust it, reducing the risks of continuous pipe operation caused by human factors.

[0011] To solve at least one of the above problems, this application provides the following technical solution:

[0012] In a first aspect, this application provides a method for continuous pipe row status identification and adjustment based on machine vision, including:

[0013] Real-time acquisition of pipe laying operation images in the roller pipe laying operation area, and image preprocessing of the pipe laying operation images;

[0014] The image features of the preprocessed pipe laying operation image are determined, and the current state of the pipe laying is determined based on the image features and the preset pipe laying state recognition model.

[0015] Perform the corresponding roller pipe adjustment operation according to the current state of the pipe arrangement.

[0016] Further, the image preprocessing of the pipe laying operation image includes:

[0017] The pipe laying operation image is segmented into regions according to a preset image segmentation algorithm to obtain the corresponding foreground region image;

[0018] The foreground region image is subjected to filtering, noise reduction, and image enhancement processing;

[0019] The foreground region image after filtering, noise reduction, and image enhancement is binarized to obtain the binarized image of the pipe laying operation image.

[0020] Furthermore, after obtaining the binarized image of the pipe laying operation image, the process further includes:

[0021] Perform image morphological closing operation on the binarized image of the pipe laying operation;

[0022] Edge detection is performed on the binarized image after image morphological closing operation according to the preset edge detection operator to obtain the edge detection result.

[0023] Further, determining the image features of the preprocessed pipe laying operation image includes:

[0024] Extract the shape features of the lead screw, oil pipe, pipe arm, and roller from the preprocessed pipe laying operation image;

[0025] The relative positions between the lead screw and the pipe arm, the oil pipe and the roller, and the lead screw and the oil pipe are determined based on the shape characteristics of the lead screw, the oil pipe and the roller, and the relative positions are determined as image features of the pipe laying operation image.

[0026] Further, before determining the current state of the corresponding pipe based on the image features and the preset pipe state recognition model, the following steps are included:

[0027] The preset historical image data is classified and labeled to obtain training sample data;

[0028] The training sample data is imported into a preset suggestion learning model and a preset overfitting optimization neural network for iterative training to obtain a pipe status recognition model.

[0029] Further, determining the current state of the corresponding pipe arrangement based on the image features and the preset pipe arrangement state recognition model includes:

[0030] The image features are input into a preset pipe laying status recognition model to obtain the expected incursion angle of the pipe laying control parameters output by the pipe laying status recognition model.

[0031] The current state of the pipe arrangement is determined by comparing the expected entrainment angle with the preset threshold value of the pipe arrangement control parameters.

[0032] Further, the step of performing the corresponding drum pipe adjustment operation according to the current state of the pipe includes:

[0033] Based on the current state of the pipe arrangement, the roller pipe arrangement actuator is driven to perform corresponding adjustment operations on the roller pipe arrangement trolley.

[0034] Secondly, this application provides a machine vision-based continuous pipe row status identification and adjustment device, comprising:

[0035] The image acquisition module is used to acquire real-time images of the pipe laying operation in the roller pipe laying operation area and to perform image preprocessing on the pipe laying operation images;

[0036] The image recognition module is used to determine the image features of the pipe laying operation image after image preprocessing, and to determine the corresponding current state of the pipe laying based on the image features and the preset pipe laying state recognition model.

[0037] An automatic adjustment module is used to perform corresponding roller pipe adjustment operations based on the current state of the pipe arrangement.

[0038] Furthermore, the image acquisition module includes:

[0039] The region segmentation unit is used to segment the pipe laying operation image into regions according to a preset image segmentation algorithm to obtain the corresponding foreground region image;

[0040] The noise reduction and enhancement unit is used to perform filtering, noise reduction, and image enhancement processing on the foreground region image.

[0041] The binarization unit is used to perform binarization conversion on the foreground region image after filtering, noise reduction and image enhancement processing to obtain the binarized image of the pipe laying operation image.

[0042] Furthermore, the image acquisition module also includes:

[0043] The morphological operation unit is used to perform image morphological closing operations on the binarized image of the pipe laying operation image;

[0044] The edge detection unit is used to perform edge detection on the binarized image after image morphological closing operation according to the preset edge detection operator, and obtain the edge detection result.

[0045] Furthermore, the image recognition module includes:

[0046] The shape feature extraction unit is used to extract the shape features of the lead screw, oil pipe, pipe arm and roller in the pipe laying operation image after image preprocessing;

[0047] The position feature determination unit is used to determine the relative positions between the lead screw and the pipe arm, the oil pipe and the drum, and the lead screw and the oil pipe based on the shape features of the lead screw, the oil pipe, the pipe arm and the drum, and to determine the relative positions as image features of the pipe laying operation image.

[0048] Furthermore, the image recognition module also includes:

[0049] The sample data construction unit is used to classify and label the preset historical image data to obtain training sample data;

[0050] The identification module training unit is used to import the training sample data into a preset suggestion learning model and a preset overfitting optimization neural network for iterative training to obtain a pipe status identification model.

[0051] Furthermore, the image recognition module also includes:

[0052] The pipe laying control parameter determination unit is used to input the image features into a preset pipe laying state recognition model to obtain the expected incursion angle of the pipe laying control parameters output by the pipe laying state recognition model.

[0053] The pipe arrangement current state determination unit is used to determine the corresponding pipe arrangement current state based on the comparison between the expected entrainment angle of the pipe arrangement control parameters and the preset threshold value.

[0054] Furthermore, the automatic adjustment module includes:

[0055] The system drive adjustment unit is used to drive the roller pipe laying actuator to perform corresponding adjustment operations on the roller pipe laying trolley according to the current state of the pipe laying.

[0056] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the machine vision-based continuous pipe array state recognition and adjustment method.

[0057] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the machine vision-based continuous pipe row state recognition and adjustment method.

[0058] Fifthly, this application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the machine vision-based continuous pipe row state recognition and adjustment method.

[0059] As can be seen from the above technical solution, this application provides a method and device for continuous pipe laying status recognition and adjustment based on machine vision. By using a pipe laying status recognition model, the pipe laying operation image of the drum pipe laying operation area can be accurately identified, thereby accurately identifying the pipe laying status and automatically adjusting it, reducing the risk of continuous pipe operation caused by human factors. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is one of the flowcharts illustrating the machine vision-based continuous pipe row status identification and adjustment method in the embodiments of this application;

[0062] Figure 2This is the second flowchart illustrating the machine vision-based continuous pipe row status identification and adjustment method in the embodiments of this application.

[0063] Figure 3 This is the third flowchart illustrating the machine vision-based continuous pipe row status identification and adjustment method in the embodiments of this application.

[0064] Figure 4 This is the fourth flowchart illustrating the machine vision-based continuous pipe row status identification and adjustment method in the embodiments of this application.

[0065] Figure 5 This is the fifth flowchart illustrating the machine vision-based continuous pipe row status identification and adjustment method in the embodiments of this application.

[0066] Figure 6 This is the sixth flowchart illustrating the machine vision-based continuous pipe row status identification and adjustment method in the embodiments of this application.

[0067] Figure 7 This is one of the structural diagrams of the machine vision-based continuous pipe row status recognition and adjustment device in the embodiments of this application;

[0068] Figure 8 This is the second structural diagram of the continuous pipe row status identification and adjustment device based on machine vision in the embodiments of this application;

[0069] Figure 9 This is the third structural diagram of the continuous pipe row status recognition and adjustment device based on machine vision in the embodiments of this application;

[0070] Figure 10 This is the fourth structural diagram of the continuous pipe row status identification and adjustment device based on machine vision in the embodiments of this application;

[0071] Figure 11 This is the fifth structural diagram of the continuous pipe row status identification and adjustment device based on machine vision in the embodiments of this application;

[0072] Figure 12 This is the sixth structural diagram of the continuous pipe row status identification and adjustment device based on machine vision in the embodiments of this application;

[0073] Figure 13 This is the seventh structural diagram of the continuous pipe row status identification and adjustment device based on machine vision in the embodiments of this application;

[0074] Figure 14 This is a schematic diagram of the overall structure of a continuous pipe row status recognition and adjustment method based on machine vision in a specific embodiment of this application.

[0075] Figure 15This is a schematic diagram of the overall process of a continuous pipe arrangement state recognition and adjustment method based on machine vision in a specific embodiment of this application.

[0076] Figure 16 This is a schematic diagram of image feature recognition in a specific embodiment of this application;

[0077] Figure 17 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0079] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0080] In view of the problems existing in the prior art, this application provides a method and device for continuous pipe laying status recognition and adjustment based on machine vision. The method and device accurately recognize the pipe laying operation image of the roller pipe laying operation area through the pipe laying status recognition model, thereby accurately identifying the pipe laying status and automatically adjusting it, reducing the risk of continuous pipe operation caused by human factors.

[0081] To accurately identify and automatically adjust the status of continuous pipe laying, thereby reducing the risks associated with continuous pipe operations due to human factors, this application provides an embodiment of a machine vision-based method for identifying and adjusting the status of continuous pipe laying. See [link to relevant documentation]. Figure 1 The machine vision-based method for continuous pipe row status recognition and adjustment specifically includes the following:

[0082] Step S101: Real-time acquisition of pipe laying operation images in the roller pipe laying operation area, and image preprocessing of the pipe laying operation images.

[0083] Optionally, this application can deploy multiple wide-angle industrial cameras in the roller pipe laying operation area to collect real-time images of the pipe laying operation, store the collected image data in the acquisition card, and finally transmit it to the PC host of the recognition system.

[0084] Optionally, after acquiring images of the pipe laying operation, this application can perform image preprocessing, which can utilize existing image processing algorithms.

[0085] Step S102: Determine the image features of the pipe laying operation image after image preprocessing, and determine the corresponding current pipe laying status based on the image features and the preset pipe laying status recognition model.

[0086] Optionally, this application can extract image features from the preprocessed pipe laying operation image and use the extracted image features as input to a preset pipe laying state recognition model. At the same time, the current state of the pipe laying can be determined according to the output of the preset pipe laying state recognition model (e.g., the expected inclination angle α of the output pipe laying control parameters).

[0087] Step S103: Perform the corresponding roller pipe adjustment operation according to the current state of the pipe.

[0088] Optionally, after determining the current state of the pipe laying, this application can drive the roller pipe laying actuator to adjust the roller pipe laying trolley in real time through the system, thereby realizing closed-loop automatic adjustment of the roller pipe laying.

[0089] As can be seen from the above description, the continuous pipe laying status recognition and adjustment method based on machine vision provided in this application embodiment can accurately identify the pipe laying operation image of the drum pipe laying operation area through the pipe laying status recognition model, thereby accurately identifying the pipe laying status and automatically adjusting it, reducing the risk of continuous pipe operation caused by human factors.

[0090] In order to accurately preprocess the images of pipe laying operations, in one embodiment of the continuous pipe laying state recognition and adjustment method based on machine vision in this application, see [reference needed]. Figure 2 The above step S101 may also specifically include the following:

[0091] Step S201: Perform region segmentation on the pipe laying operation image according to the preset image segmentation algorithm to obtain the corresponding foreground region image.

[0092] Step S202: Perform filtering, noise reduction, and image enhancement processing on the foreground region image.

[0093] Step S203: Perform binarization conversion on the foreground region image after filtering, noise reduction and image enhancement processing to obtain the binarized image of the pipe laying operation image.

[0094] Optionally, this application can use the Grabcut algorithm to segment and extract the foreground region of the image, and then perform filtering and adaptive CLAHE image enhancement algorithm on the extracted ROI foreground region in sequence to reduce the influence of noise factors. Finally, the enhanced image is subjected to OTSU binary processing to segment the image and return the image threshold.

[0095] To facilitate subsequent calculations, in one embodiment of the machine vision-based continuous pipe row state recognition and adjustment method of this application, see [link to relevant documentation]. Figure 3 Following step S203 above, the following additional content may be included:

[0096] Step S301: Perform image morphological closing operation on the binarized image of the pipe laying operation image.

[0097] Step S302: Perform edge detection on the binarized image after image morphological closing operation according to the preset edge detection operator to obtain the edge detection result.

[0098] Optionally, this application may perform image morphology closing operations on the binarized image to fill the unconnected "holes" in the foreground objects of the segmented binarized image.

[0099] Optionally, this application may also employ various algorithms (such as the Sobel operator, Robert operator, Prewitt operator, and Scharr operator based on first-order difference, and the Laplance operator based on second-order difference) to perform edge detection on images that have undergone morphological operations. For example, an improved Canny operator based on the non-maximum suppression iteration method can be established, which can effectively avoid the defects of manually selecting thresholds.

[0100] In order to accurately extract image features, in one embodiment of the continuous pipe row state recognition and adjustment method based on machine vision in this application, see [reference needed]. Figure 4 The above step S102 may also specifically include the following:

[0101] Step S401: Extract the shape features of the lead screw, oil pipe, pipe arm and roller in the preprocessed pipe laying operation image.

[0102] Step S402: Determine the relative positions of the lead screw and the pipe arm, the oil pipe and the roller, and the lead screw and the oil pipe based on the shape characteristics of the lead screw, the oil pipe, the pipe arm and the roller, and determine the relative positions as image features of the pipe laying operation image.

[0103] Optional, see Figure 16 This application can extract features from preprocessed images. It extracts the shape features of the lead screw, oil pipe, pipe arm, and drum, and based on these features, extracts feature points 1 and 2 of the lead screw and pipe arm, feature point 3 of the oil pipe and drum, and the relative position of feature point 4 of the lead screw and oil pipe. Finally, it extracts the required feature data as input data.

[0104] In order to accurately identify the status of continuous pipe arrangement, in one embodiment of the machine vision-based continuous pipe arrangement status identification and adjustment method of this application, see [link to relevant documentation]. Figure 5 Prior to step S102 above, the following content may also be included:

[0105] Step S501: Classify and label the preset historical image data to obtain training sample data.

[0106] Step S502: Import the training sample data into the preset suggestion learning model and the preset overfitting optimization neural network for iterative training to obtain the pipe status recognition model.

[0107] Optionally, before implementing the automated pipe-laying process, this application can first establish a pipe-laying machine learning recognition model. To address the convergence difficulties, vanishing gradients, or exploding gradients encountered during the training of traditional neural network models, VGG16 models, and ResNet models, this application can incorporate a Dropout network and a BatchNormalization network to optimize the network model. Furthermore, it employs transfer learning to achieve cross-domain learning objectives, improving learning performance and mitigating model overfitting. Historical image data is combined with on-site human experience, labeled, and classified as training sample data. This data is then imported into the established neural network model ("transfer learning ResNet50 model" + "Droupout optimization network") for extensive iterative training, ultimately yielding an optimized machine learning recognition model.

[0108] In order to accurately identify the status of continuous pipe arrangement, in one embodiment of the machine vision-based continuous pipe arrangement status identification and adjustment method of this application, see [link to relevant documentation]. Figure 6 The above step S102 may also specifically include the following:

[0109] Step S601: Input the image features into a preset pipe laying status recognition model to obtain the expected inclination angle of the pipe laying control parameters output by the pipe laying status recognition model.

[0110] Step S602: Determine the current state of the pipe arrangement based on the comparison between the expected entrainment angle of the pipe arrangement control parameters and the preset threshold value.

[0111] Optionally, this application can extract the required feature data as input data and import it into the pipe laying status recognition model. The trained and optimized model can identify image features based on the input data and output the expected entrainment angle α of the pipe laying control parameters under the current state. Based on the comparison between the expected entrainment angle α of the pipe laying control parameters and the preset threshold, the corresponding current state of the pipe laying can be determined.

[0112] In order to enable real-time adjustments, in one embodiment of the continuous pipe row state recognition and adjustment method based on machine vision in this application, the above step S103 may further include the following:

[0113] Based on the current state of the pipe arrangement, the roller pipe arrangement actuator is driven to perform corresponding adjustment operations on the roller pipe arrangement trolley.

[0114] Optionally, this application can drive the roller pipe laying actuator to adjust the roller pipe laying trolley in real time through the system, so as to realize closed-loop automatic adjustment of the roller pipe laying.

[0115] To accurately identify and automatically adjust the pipe arrangement status, thereby reducing the risks associated with continuous pipe operations due to human error, this application provides an embodiment of a machine vision-based continuous pipe arrangement status identification and adjustment device for implementing all or part of the aforementioned machine vision-based continuous pipe arrangement status identification and adjustment method. See [link to embodiment]. Figure 7 The machine vision-based continuous pipe row status identification and adjustment device specifically includes the following components:

[0116] The image acquisition module 10 is used to acquire images of the pipe laying operation in the roller pipe laying operation area in real time, and to perform image preprocessing on the pipe laying operation images.

[0117] The image recognition module 20 is used to determine the image features of the pipe laying operation image after image preprocessing, and to determine the corresponding current state of the pipe laying based on the image features and the preset pipe laying state recognition model.

[0118] The automatic adjustment module 30 is used to perform corresponding roller pipe adjustment operations according to the current state of the pipe.

[0119] As can be seen from the above description, the machine vision-based continuous pipe laying status recognition and adjustment device provided in this application embodiment can accurately identify the pipe laying operation image of the drum pipe laying operation area through the pipe laying status recognition model, thereby accurately identifying the pipe laying status and automatically adjusting it, reducing the risk of continuous pipe operation caused by human factors.

[0120] In order to accurately preprocess images of pipe laying operations, in one embodiment of the machine vision-based continuous pipe laying status recognition and adjustment device of this application, see [reference needed]. Figure 8 The image acquisition module 10 includes:

[0121] The region segmentation unit 11 is used to segment the pipe laying operation image into regions according to a preset image segmentation algorithm to obtain the corresponding foreground region image.

[0122] The noise reduction and enhancement unit 12 is used to perform filtering, noise reduction, and image enhancement processing on the foreground region image.

[0123] Binarization unit 13 is used to perform binarization conversion on the foreground region image after filtering, noise reduction and image enhancement processing to obtain the binarized image of the pipe laying operation image.

[0124] To facilitate subsequent calculations, in one embodiment of the machine vision-based continuous pipe row state recognition and adjustment device of this application, see [reference needed]. Figure 9 The image acquisition module 10 further includes:

[0125] The morphological operation unit 14 is used to perform image morphological closing operation on the binarized image of the pipe laying operation image.

[0126] The edge detection unit 15 is used to perform edge detection on the binarized image after image morphological closing operation according to the preset edge detection operator, and obtain the edge detection result.

[0127] In order to accurately extract image features, in one embodiment of the machine vision-based continuous pipe row state recognition and adjustment device of this application, see [reference needed]. Figure 10 The image recognition module 20 includes:

[0128] The shape feature extraction unit 21 is used to extract the shape features of the lead screw, oil pipe, pipe arm and roller in the pipe laying operation image after image preprocessing.

[0129] The position feature determination unit 22 is used to determine the relative positions between the lead screw and the pipe arm, the oil pipe and the drum, and the lead screw and the oil pipe based on the shape features of the lead screw, the oil pipe, the pipe arm and the drum, and to determine the relative positions as image features of the pipe laying operation image.

[0130] In order to accurately identify the status of the continuous pipe arrangement, in one embodiment of the machine vision-based continuous pipe arrangement status identification and adjustment device of this application, see [reference needed]. Figure 11 The image recognition module 20 further includes:

[0131] The sample data construction unit 23 is used to classify and label the preset historical image data to obtain training sample data.

[0132] The recognition module training unit 24 is used to import the training sample data into a preset suggestion learning model and a preset overfitting optimization neural network for iterative training to obtain a pipe status recognition model.

[0133] In order to accurately identify the status of the continuous pipe arrangement, in one embodiment of the machine vision-based continuous pipe arrangement status identification and adjustment device of this application, see [reference needed]. Figure 12 The image recognition module 20 further includes:

[0134] The pipe laying control parameter determination unit 25 is used to input the image features into a preset pipe laying state recognition model to obtain the expected inclination angle of the pipe laying control parameters output by the pipe laying state recognition model.

[0135] The pipe current state determination unit 26 is used to determine the corresponding pipe current state based on the comparison between the expected entrainment angle of the pipe control parameters and the preset threshold value.

[0136] In order to enable real-time adjustments, in one embodiment of the machine vision-based continuous pipe bundling status recognition and adjustment device of this application, see [reference needed]. Figure 13 The automatic adjustment module 30 includes:

[0137] The system drive adjustment unit 31 is used to drive the roller pipe laying actuator to perform corresponding adjustment operations on the roller pipe laying trolley according to the current state of the pipe laying.

[0138] To further illustrate this solution, this application also provides a specific application example of using the aforementioned machine vision-based continuous pipe array state recognition and adjustment device to implement the machine vision-based continuous pipe array state recognition and adjustment method. See [link to relevant documentation]. Figure 14 It consists of three parts: system acquisition hardware, system software, and system driver hardware.

[0139] The system's data acquisition hardware mainly includes an industrial camera and a data acquisition card. The system software runs on a PC host and is the core part of the pipe laying system, mainly including image processing algorithms and machine learning recognition algorithms. The system's driving hardware mainly includes a hydraulic system and a mechanical mechanism.

[0140] To more clearly illustrate the method for automatic continuous tube stacking of the present invention, assuming the identification and adjustment of the stacking status of a specific roller and a certain specification of continuous tube, the automatic continuous tube stacking method flow is as follows: Figure 15 As shown:

[0141] Step 01: Based on on-site human experience, label and classify historical image data as training sample data.

[0142] Step 02: Import the training data into the established machine learning model for extensive iterative training to finally obtain an optimized machine learning recognition model.

[0143] Step 03: Use hardware devices such as cameras and acquisition cards to collect real-time image data of the roller pipe laying system and transmit it to the main unit of the pipe laying system.

[0144] Step 04: Based on image processing technology, the acquired image data is processed by filtering and noise reduction, enhancement, image segmentation, morphological processing, edge detection and other methods to extract a binarized image.

[0145] Step 05: See Figure 16 The feature points 1 and 2 of the lead screw and pipe arm, the feature point 3 of the oil pipe and the drum, and the relative position of the feature point 4 of the lead screw and the oil pipe are quickly extracted from the preprocessed binarized image.

[0146] Step 06: The extracted feature data is used as input data and imported into the machine learning recognition model in Step 1. The trained and optimized model recognizes the input data and outputs the expected entanglement angle α of the pipe control parameters under the current state.

[0147] Step 07: Based on the pipe laying control parameter α, predict the current pipe laying status and drive the roller pipe laying actuator to adjust the roller pipe laying trolley in real time to achieve closed-loop automatic adjustment of the roller pipe laying.

[0148] As can be seen from the above, this application can also achieve at least the following technical effects:

[0149] 1. During the continuous tube winding process, this tube recognition system uses a camera to quickly acquire images of the tube winding on the drum in real time. It uses machine vision technology to preprocess the images to quickly extract features such as the drum edge contour, the position of the guide tube, and the entry point of the continuous tube winding into the drum. The system then identifies these features and assesses the tube winding status during continuous tube winding.

[0150] 2. This application can predict and avoid disordered tubing in a timely manner, effectively reduce the ineffective start-up and shutdown time of coiled tubing operations, reduce fatigue wear of coiled tubing, reduce the labor intensity of personnel, and improve the efficiency of coiled tubing use.

[0151] 3. This application can automatically correct the internal model parameters according to the properties of continuous pipes of different specifications and rollers of different models, ensuring the compatibility and practicality of the pipe recognition system.

[0152] 4. This application can achieve precise identification and adjustment of the pipe laying status, realize the "identification-predictive analysis-automatic adjustment" control closed loop in the continuous pipe laying process, and provide a foundation for subsequent high-precision unmanned automatic pipe laying technology.

[0153] From a hardware perspective, in order to accurately identify the status of the continuous pipe arrangement and automatically adjust it, thereby reducing the risks of continuous pipe operations caused by human factors, this application provides an embodiment of an electronic device for implementing all or part of the aforementioned machine vision-based continuous pipe arrangement status identification and adjustment method. The electronic device specifically includes the following components:

[0154] The system comprises a processor, memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to realize information transmission between the machine vision-based continuous pipe array status identification and adjustment device and core business systems, user terminals, and related databases and other related devices; the logic controller can be a desktop computer, tablet computer, or mobile terminal, etc., and this embodiment is not limited to these. In this embodiment, the logic controller can be implemented with reference to the embodiments of the machine vision-based continuous pipe array status identification and adjustment method and the embodiments of the machine vision-based continuous pipe array status identification and adjustment device, the contents of which are incorporated herein by reference, and repeated details will not be described again.

[0155] It is understood that the user terminal may include smartphones, tablet computers, network set-top boxes, portable computers, desktop computers, personal digital assistants (PDAs), in-vehicle devices, smart wearable devices, etc. Among these, the smart wearable devices may include smart glasses, smartwatches, smart bracelets, etc.

[0156] In practical applications, the machine vision-based continuous pipe array status recognition and adjustment method can be partially executed on the electronic device side as described above, or all operations can be completed in the client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed in the client device, the client device may further include a processor.

[0157] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0158] Figure 17 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 17 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 17 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0159] In one embodiment, the machine vision-based continuous pipe array status recognition and adjustment method can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control:

[0160] Step S101: Real-time acquisition of pipe laying operation images in the roller pipe laying operation area, and image preprocessing of the pipe laying operation images.

[0161] Step S102: Determine the image features of the pipe laying operation image after image preprocessing, and determine the corresponding current pipe laying status based on the image features and the preset pipe laying status recognition model.

[0162] Step S103: Perform the corresponding roller pipe adjustment operation according to the current state of the pipe.

[0163] As can be seen from the above description, the electronic device provided in this application embodiment can accurately identify the pipe laying operation image of the drum pipe laying operation area through the pipe laying status recognition model, thereby accurately identifying the pipe laying status and automatically adjusting it, reducing the risk of continuous pipe operation caused by human factors.

[0164] In another embodiment, the machine vision-based continuous tube array state recognition and adjustment device can be configured separately from the central processing unit 9100. For example, the machine vision-based continuous tube array state recognition and adjustment device can be configured as a chip connected to the central processing unit 9100, and the machine vision-based continuous tube array state recognition and adjustment method function can be realized through the control of the central processing unit.

[0165] like Figure 17 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 17 All components shown; in addition, the electronic device 9600 may also include Figure 17 For components not shown, please refer to existing technologies.

[0166] like Figure 17 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device 9600.

[0167] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0168] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0169] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0170] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device's communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0171] The communication module 9110 is a transmitter / receiver 9110 that transmits and receives signals via the antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0172] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 9130 is coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored sound via the speaker 9131.

[0173] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the machine vision-based continuous pipe array state identification and adjustment method with a server or client as the execution subject in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the machine vision-based continuous pipe array state identification and adjustment method with a server or client as the execution subject in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0174] Step S101: Real-time acquisition of pipe laying operation images in the roller pipe laying operation area, and image preprocessing of the pipe laying operation images.

[0175] Step S102: Determine the image features of the pipe laying operation image after image preprocessing, and determine the corresponding current pipe laying status based on the image features and the preset pipe laying status recognition model.

[0176] Step S103: Perform the corresponding roller pipe adjustment operation according to the current state of the pipe.

[0177] As can be seen from the above description, the computer-readable storage medium provided in the embodiments of this application can accurately identify the pipe laying operation image of the drum pipe laying operation area through the pipe laying status recognition model, thereby accurately identifying the pipe laying status and automatically adjusting it, reducing the risk of continuous pipe operation caused by human factors.

[0178] Embodiments of this application also provide a computer program product capable of implementing all steps in the machine vision-based continuous pipe array state identification and adjustment method described above, where the execution subject is a server or client. When executed by a processor, this computer program / instruction implements the steps of the machine vision-based continuous pipe array state identification and adjustment method. For example, the computer program / instruction implements the following steps:

[0179] Step S101: Real-time acquisition of pipe laying operation images in the roller pipe laying operation area, and image preprocessing of the pipe laying operation images.

[0180] Step S102: Determine the image features of the pipe laying operation image after image preprocessing, and determine the corresponding current pipe laying status based on the image features and the preset pipe laying status recognition model.

[0181] Step S103: Perform the corresponding roller pipe adjustment operation according to the current state of the pipe.

[0182] As can be seen from the above description, the computer program product provided in this application embodiment can accurately identify the pipe laying operation image of the drum pipe laying operation area through the pipe laying status recognition model, thereby accurately identifying the pipe laying status and automatically adjusting it, reducing the risk of continuous pipe operation caused by human factors.

[0183] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0184] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0185] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0186] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0187] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for continuous pipe row status recognition and adjustment based on machine vision, characterized in that, The method includes: Real-time acquisition of pipe laying operation images in the roller pipe laying operation area, and image preprocessing of the pipe laying operation images; The image features of the preprocessed pipe laying operation image are determined, and the current state of the pipe laying is determined based on the image features and the preset pipe laying state recognition model. Perform the corresponding drum pipe adjustment operation according to the current state of the pipe arrangement; The step of determining the image features of the pipe laying operation image after image preprocessing includes: extracting the shape features of the lead screw, oil pipe, pipe laying arm, and roller in the preprocessed pipe laying operation image; determining the relative positions between the lead screw and the pipe laying arm, the oil pipe and the roller, and the lead screw and the oil pipe based on the shape features of the lead screw, oil pipe, pipe laying arm, and roller, and determining the relative positions as the image features of the pipe laying operation image; The step of determining the current state of the corresponding pipe based on the image features and the preset pipe state recognition model includes: inputting the image features into the preset pipe state recognition model to obtain the expected entrainment angle of the pipe control parameters output by the pipe state recognition model; determining the current state of the corresponding pipe based on the comparison between the expected entrainment angle of the pipe control parameters and a preset threshold; the pipe state recognition model is obtained by training a ResNet50 model based on transfer learning and combined with a Dropout optimized neural network.

2. The method for continuous pipe row status identification and adjustment based on machine vision according to claim 1, characterized in that, The image preprocessing of the pipe laying operation image includes: The pipe laying operation image is segmented into regions according to a preset image segmentation algorithm to obtain the corresponding foreground region image; The foreground region image is subjected to filtering, noise reduction, and image enhancement processing; The foreground region image after filtering, noise reduction, and image enhancement is binarized to obtain the binarized image of the pipe laying operation image.

3. The method for continuous pipe row status identification and adjustment based on machine vision according to claim 2, characterized in that, After obtaining the binarized image of the pipe laying operation image, the process further includes: Perform image morphological closing operation on the binarized image of the pipe laying operation; Edge detection is performed on the binarized image after image morphological closing operation according to the preset edge detection operator to obtain the edge detection result.

4. The method for continuous pipe row status identification and adjustment based on machine vision according to claim 1, characterized in that, Before determining the current state of the corresponding pipe based on the image features and the preset pipe status recognition model, the process includes: The preset historical image data is classified and labeled to obtain training sample data; The training sample data is imported into a preset suggestion learning model and a preset overfitting optimization neural network for iterative training to obtain a pipe status recognition model.

5. The method for continuous pipe row status identification and adjustment based on machine vision according to claim 1, characterized in that, The step of performing the corresponding drum pipe adjustment operation according to the current state of the pipe includes: Based on the current state of the pipe arrangement, the roller pipe arrangement actuator is driven to perform corresponding adjustment operations on the roller pipe arrangement trolley.

6. A continuous pipe row status identification and adjustment device based on machine vision, characterized in that, include: The image acquisition module is used to acquire real-time images of the pipe laying operation in the roller pipe laying operation area and to perform image preprocessing on the pipe laying operation images; The image recognition module is used to determine the image features of the pipe laying operation image after image preprocessing, and to determine the corresponding current state of the pipe laying based on the image features and the preset pipe laying state recognition model. The automatic adjustment module is used to perform corresponding drum pipe adjustment operations according to the current state of the pipe arrangement; The image recognition module includes: The shape feature extraction unit is used to extract the shape features of the lead screw, oil pipe, pipe arm and roller in the pipe laying operation image after image preprocessing; The position feature determination unit is used to determine the relative positions between the lead screw and the pipe arm, the oil pipe and the drum, and the lead screw and the oil pipe based on the shape features of the lead screw, the oil pipe, the pipe arm and the drum, and to determine the relative positions as image features of the pipe laying operation image; The image recognition module further includes: The pipe laying control parameter determination unit is used to input the image features into a preset pipe laying state recognition model to obtain the expected incursion angle of the pipe laying control parameters output by the pipe laying state recognition model. The pipe current state determination unit is used to determine the corresponding pipe current state based on the numerical comparison between the expected entrainment angle of the pipe control parameters and a preset threshold; the pipe state recognition model is obtained by training a transfer learning ResNet50 model combined with a Drupout optimized neural network.

7. The continuous pipe row status identification and adjustment device based on machine vision according to claim 6, characterized in that, The image acquisition module includes: The region segmentation unit is used to segment the pipe laying operation image into regions according to a preset image segmentation algorithm to obtain the corresponding foreground region image; The noise reduction and enhancement unit is used to perform filtering, noise reduction, and image enhancement processing on the foreground region image. The binarization unit is used to perform binarization conversion on the foreground region image after filtering, noise reduction and image enhancement processing to obtain the binarized image of the pipe laying operation image.

8. The continuous pipe row status identification and adjustment device based on machine vision according to claim 6, characterized in that, The image acquisition module also includes: The morphological operation unit is used to perform image morphological closing operations on the binarized image of the pipe laying operation image; The edge detection unit is used to perform edge detection on the binarized image after image morphological closing operation according to the preset edge detection operator, and obtain the edge detection result.

9. The continuous pipe row status identification and adjustment device based on machine vision according to claim 6, characterized in that, The image recognition module further includes: The sample data construction unit is used to classify and label the preset historical image data to obtain training sample data; The identification module training unit is used to import the training sample data into a preset suggestion learning model and a preset overfitting optimization neural network for iterative training to obtain a pipe status identification model.

10. The continuous pipe row status identification and adjustment device based on machine vision according to claim 6, characterized in that, The automatic adjustment module includes: The system drive adjustment unit is used to drive the roller pipe laying actuator to perform corresponding adjustment operations on the roller pipe laying trolley according to the current state of the pipe laying.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the machine vision-based continuous pipe row status recognition and adjustment method as described in any one of claims 1 to 5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the machine vision-based continuous pipe row status identification and adjustment method as described in any one of claims 1 to 5.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the machine vision-based continuous pipe state identification and adjustment method according to any one of claims 1 to 5.

Images