Iron roughneck petroleum coupling state sound recognition method and system based on deep learning

Abstract

The invention relates to the technical field of oil drilling, in particular to an iron roughneck oil coupling state sound recognition method and system based on deep learning, and the method comprises the steps: S1, obtaining an audio signal segment of an iron roughneck oil coupling screwing-on process or a screwing-off process; s2, framing the audio signal segment according to a preset framing logic to obtain a multi-frame audio signal; s3, based on each frame of audio signal, obtaining the MFCC feature of each frame of audio signal, and determining coupling state information corresponding to the frame of audio signal according to the MFCC feature; s4, based on the MFCC feature and the coupling state information corresponding to each frame of audio signal and pre-obtained iron roughneck state data corresponding to the frame of audio signal, respectively obtaining a comprehensive feature vector corresponding to each frame of audio signal; and S5, obtaining a prediction result based on the comprehensive feature vector corresponding to each frame of audio signal and / or a pre-obtained first audio signal segment.

Description

Technical Field

[0001] The present invention relates to the field of oil drilling technology, and in particular to a method and system for sound recognition of oil coupling status of an iron roughneck based on deep learning. Background Art

[0002] At present, the automation and monitoring identification system of iron drillers in the process of pulling and unplugging couplings in the field of oil drilling has made significant progress. Through these systems, the process of pulling and unplugging couplings can be effectively controlled to improve work efficiency and safety. Coupling operation is a key link in the process of oil drilling. The different parts involved in connecting the drill string mainly include two forms: male buckle and female buckle. The male buckle is a part of the coupling, usually located at the end of the drill pipe or drill string, with external threads. During the coupling process, the male buckle is inserted into the female buckle and forms a tight connection by rotation. The female buckle is the part with internal threads, which is usually fixed on the other end of the drill pipe or the pipe string. It cooperates with the male buckle and rotates to make the two tightly combined to form a stable connection. Therefore, the process of pulling and unplugging is mainly divided into: pulling out: unplugging operation; drilling down: plugging operation. Traditional coupling status monitoring relies on data collected by physical sensors, such as pressure, torque, and rotation speed, and judges the state change through preset thresholds or patterns. Although these methods are effective to a certain extent, their accuracy and reliability will be greatly reduced in complex operating environments, such as sensor failure.

[0003] Existing physical sensor-based monitoring systems have limitations when dealing with complex environments and subtle state changes. First, physical sensors may be interfered with by environmental factors, resulting in inaccurate data. Second, these systems are often unable to process large amounts of data in real time, making it impossible to instantly identify faults or state changes. Most importantly, when the sensor itself fails, the reliability of the entire monitoring system will be affected. Although existing machine vision solutions can determine the height of the coupling and then perform the make-up and break-out operations, the error in target detection prevents the driller from perfectly screwing the male and female buckles out or in without error. Summary of the invention

[0004] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method and system for sound recognition of the status of an iron roughneck oil coupling based on deep learning.

[0005] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0006] In a first aspect, an embodiment of the present invention provides a method for identifying the state of an iron roughneck oil coupling based on deep learning, comprising:

[0007] S1, obtaining an audio signal segment of the process of making up or breaking out the oil coupling of the iron roughneck;

[0008] S2, dividing the audio signal segment into frames according to a preset frame division logic to obtain multiple frames of audio signals;

[0009] S3, based on each frame of audio signal, obtaining MFCC features of each frame of audio signal, and determining the hoop state information corresponding to the frame of audio signal according to the MFCC features;

[0010] S4, based on the MFCC features and coupling state information corresponding to each frame of the audio signal, and the pre-obtained iron roughneck state data corresponding to the frame of the audio signal, respectively obtain a comprehensive feature vector corresponding to each frame of the audio signal;

[0011] S5. Obtain a prediction result based on the comprehensive feature vector corresponding to each frame of the audio signal and / or the pre-acquired first audio signal segment;

[0012] The first audio signal segment is an audio signal of a pre-specified time period before the audio signal segment of the iron roughneck petroleum coupling make-up process or break-out process.

[0013] Preferably, the S2 specifically includes:

[0014] The audio signal segment is divided into multiple frames of audio signals according to a pre-specified frame length.

[0015] Preferably, the S3 specifically includes:

[0016] S31, performing feature extraction processing on each frame of audio signal to obtain MFCC features of each frame of audio signal;

[0017] S32, inputting the MFCC features of each frame of audio signal into a pre-trained hoop state detection model to obtain hoop state information corresponding to each frame of audio signal;

[0018] The coupling state information corresponding to the audio signal includes: the probability value and confidence level of each state label corresponding to the frame of audio signal;

[0019] The collar state detection model is trained using the first training data set to obtain a pre-trained collar state detection model;

[0020] The coupling state detection model is a convolutional neural network model;

[0021] The first training data set includes: audio signals of multiple time periods during the process of making up and breaking out the iron roughneck oil coupling, MFCC features corresponding to the audio signals of each time period, and pre-set state labels corresponding to the audio signals of each time period;

[0022] The status labels include: a first label for indicating that the audio signal of the time period corresponds to the normal operating state in the coupling; a second label for indicating that the audio signal of the time period corresponds to the background noise in the coupling; a third label for indicating that the audio signal of the time period corresponds to the tightening state in the coupling; a fourth label for indicating that the audio signal of the time period corresponds to the tightening completion state in the coupling; a fifth label for indicating that the audio signal of the time period corresponds to the tightening transition state in the coupling; a sixth label for indicating that the audio signal of the time period corresponds to the unscrewing state in the coupling; and a seventh label for indicating that the audio signal of the time period corresponds to the unscrewing completion state in the coupling.

[0023] Preferably, the S4 specifically includes:

[0024] S41, filtering the multi-frame audio signals obtained by dividing the audio signal into frames to obtain the filtered multi-frame audio signals;

[0025] S42, performing frame splicing processing on the filtered multiple frames of audio signals to obtain multiple audio sequences;

[0026] S43, based on the audio sequence and the MFCC features and coupling status information corresponding to each frame of audio signal and the iron roughneck status data corresponding to the frame of audio signal, obtain a comprehensive feature vector corresponding to the frame of audio signal.

[0027] Preferably, the S41 specifically includes:

[0028] Among the multiple frames of audio signals obtained after the audio signal is divided into frames, each frame of audio signal corresponding to a confidence level less than a preset value is removed to obtain a filtered multiple frame audio signal.

[0029] Preferably, the S42 specifically includes:

[0030] When n≤M-4, the nth audio sequence PT obtained after frame splicing n = n , P (n+2) , P (n+4)};

[0031] When M-4<n≤M-2, the nth audio sequence PT obtained after the frame splicing process n = n , P (n+2)};

[0032] When M-2<n≤M, the nth audio sequence PT obtained after frame splicing n = n};

[0033] Wherein, M is the number of frames of the filtered audio signal;

[0034] P n is the nth frame of audio signal in the filtered multi-frame audio signals.

[0035] Preferably,

[0036] The comprehensive feature vector is AS n ;

[0037] AS n =[F n , S [n] , C [n] , S.C. [n] ];

[0038] F n It is the confidence level corresponding to the nth frame of audio signal in the filtered multi-frame audio signal;

[0039] S [n] = {P n , P n The probability value of each state label corresponding to it, P n The confidence of each state label corresponding to it};

[0040] C [n] Includes: nth audio sequence PT n The corresponding confidence curve;

[0041] Among them, the nth audio sequence PT n The corresponding confidence curve is the nth audio sequence PT n A connecting line of the confidence of each state label corresponding to each frame of audio signal contained in ;

[0042] Iron Roughneck Status Data SC [n] is: any value among the first preset value, the second preset value and the third preset value;

[0043] Among them, the first preset value indicates that the iron roughneck is in an unknown state; the second preset value indicates that the iron roughneck is in a make-up state; and the third preset value indicates that the iron roughneck is in a make-up state.

[0044] Preferably, the S5 specifically includes:

[0045] S51, obtaining a comprehensive feature vector sequence based on the comprehensive feature vector corresponding to each frame of the audio signal;

[0046] AS seq ={AS1, AS2, AS3,...,AS M};

[0047] Among them, AS seq is the comprehensive feature vector sequence;

[0048] S52, the comprehensive feature vector sequence AS seq and / or a first audio signal segment acquired in advance, inputted into a pre-trained LSTM time series detection model to obtain a prediction result;

[0049] The second training data set is used to train the LSTM time series detection model to obtain a pre-trained LSTM time series detection model;

[0050] The second training data set includes: a first comprehensive feature vector sequence and a first audio label corresponding to the first comprehensive feature vector sequence; a second comprehensive feature vector sequence and a second audio label corresponding to the second comprehensive feature vector sequence;

[0051] The first comprehensive feature vector sequence is a comprehensive feature vector sequence corresponding to the first audio signal segment; the second comprehensive feature vector sequence is a comprehensive feature vector sequence corresponding to the second audio signal segment;

[0052] The first audio signal segment is an audio signal segment of the process from the iron roughneck tightening to the jamming or sudden whistling; the second audio signal segment is an audio signal segment of the process from the iron roughneck unscrewing to the unscrewing completion;

[0053] The first audio tag is an indication that an audio signal segment having the first audio tag will show a state where the iron roughneck is tightened to a stuck state or a sudden howling state;

[0054] The second audio tag is an identifier indicating that the unscrewing completion process state appears in the audio signal segment having the second audio tag.

[0055] Preferably,

[0056] The prediction result is the comprehensive feature vector sequence AS seq The corresponding audio signal segments of the iron roughneck oil coupling make-up process or break-out process correspond to the probability values ​​of the first audio label and the second audio label respectively.

[0057] On the other hand, this embodiment also provides a deep learning-based iron roughneck oil collar state sound recognition system, comprising:

[0058] A microphone array audio signal acquisition sensor, an embedded system, and an iron roughneck central control system connected in sequence;

[0059] Wherein, the microphone array audio signal acquisition sensor is arranged on the iron roughneck motion caliper, and is used to collect the audio signal segment of the iron roughneck oil coupling make-up process or make-out process;

[0060] The embedded system is used to adopt the deep learning-based sound recognition method for the state of the iron roughneck oil coupling as described in the first aspect, process the audio signal segment of the iron roughneck oil coupling make-up process or make-out process collected by the microphone array audio signal acquisition sensor, obtain a prediction result, and send the prediction result to the iron roughneck central control system.

[0061] The beneficial effects of the present invention are as follows: the method for identifying the state of an iron roughneck oil coupling based on deep learning of the present invention can accurately identify the state of the coupling process by extracting important features from the audio signal through a deep learning algorithm, as the state of the iron roughneck oil coupling is determined by the audio signal. This method has higher accuracy and faster response time than traditional mechanical sensors or manual judgment methods.

[0062] The present invention provides a method for recognizing the state of an iron roughneck oil coupling based on deep learning. The frame logic used can decompose continuous audio signals into small segments (i.e., frames). Each frame of audio is subjected to MFCC feature extraction and then compared and analyzed with preset state data. This frame-based analysis method can significantly improve the real-time performance of recognition because it does not need to wait for the entire audio segment to end before processing. At the same time, the combination of the state data of the iron roughneck can make the system more accurate in recognizing audio features and reduce misjudgments caused by environmental noise or other interference.

[0063] The invention discloses a method for sound recognition of the state of an iron roughneck oil coupling based on deep learning. The realization of the prediction function relies on the deep learning model's learning of historical audio data, as well as the processing and analysis of real-time audio data. Predictions are made based on the comprehensive feature vector of each frame of the audio signal, and the information in the first audio signal segment is combined to determine whether the current operating state meets expectations. Through this multi-level information processing, a more comprehensive judgment can be formed, and corresponding prompts or adjustments can be made before an abnormal situation occurs, relying on the model's predictive ability. This forward-looking early warning mechanism can significantly improve the safety and efficiency of operations.

[0064] The invention discloses a method for recognizing the state of an iron roughneck oil collar based on deep learning. MFCC features are a common means of processing audio signals, especially in the fields of speech recognition and sound classification. Applying them to the recognition of the state of an iron roughneck collar can extract the most discriminative part of the audio signal, ensuring the system's ability to capture sound features, so that even slight differences in sound can be accurately detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A flow chart of a method for sound recognition of an iron roughneck oil coupling state based on deep learning of the present invention;

[0066] Figure 2 Schematic diagram of a deep learning-based sound recognition system for an iron roughneck oil coupling status according to the present invention. DETAILED DESCRIPTION

[0067] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0068] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0069] See also Figure 1 This embodiment provides a method for recognizing the state of an iron roughneck oil collar based on deep learning, comprising:

[0070] S1, obtaining an audio signal segment of the process of making up or breaking out the oil coupling of the iron roughneck;

[0071] S2, dividing the audio signal segment into frames according to a preset frame division logic to obtain multiple frames of audio signals;

[0072] The step S2 specifically includes: dividing the audio signal segment into multiple frames of audio signals according to a pre-specified frame length. In this embodiment, the frame length of each frame of the audio signal is a pre-specified frame length.

[0073] S3, based on each frame of audio signal, obtaining MFCC features of each frame of audio signal, and determining the hoop state information corresponding to the frame of audio signal according to the MFCC features;

[0074] MFCC, the full name of which is Mel-Frequency Cepstral Coefficients, is a feature extraction technology widely used in the field of audio signal processing. MFCC features are mainly used to analyze the spectrum of audio signals, especially human voice signals, and have important applications in speech recognition, audio classification, sentiment analysis and other fields.

[0075] In the practical application of this embodiment, S3 specifically includes:

[0076] S31, performing feature extraction processing on each frame of audio signal to obtain MFCC features of each frame of audio signal;

[0077] S32, inputting the MFCC features of each frame of audio signal into a pre-trained hoop state detection model to obtain hoop state information corresponding to each frame of audio signal;

[0078] The coupling state information corresponding to the audio signal includes: the probability value and confidence level of each state label corresponding to the frame of audio signal;

[0079] The collar state detection model is trained using the first training data set to obtain a pre-trained collar state detection model;

[0080] The coupling state detection model is a convolutional neural network model;

[0081] The first training data set includes: audio signals of multiple time periods during the process of making up and breaking out the iron roughneck oil coupling, MFCC features corresponding to the audio signals of each time period, and pre-set state labels corresponding to the audio signals of each time period;

[0082] The status labels include: a first label for indicating that the audio signal of the time period corresponds to the normal operating state in the coupling; a second label for indicating that the audio signal of the time period corresponds to the background noise in the coupling; a third label for indicating that the audio signal of the time period corresponds to the tightening state in the coupling; a fourth label for indicating that the audio signal of the time period corresponds to the tightening completion state in the coupling; a fifth label for indicating that the audio signal of the time period corresponds to the tightening transition state in the coupling; a sixth label for indicating that the audio signal of the time period corresponds to the unscrewing state in the coupling; and a seventh label for indicating that the audio signal of the time period corresponds to the unscrewing completion state in the coupling.

[0083] In this embodiment, MFCC (Mel-Frequency Cepstral Coefficient) is used as a feature extraction method, which can accurately capture the spectral characteristics of audio signals, and is particularly suitable for analyzing complex industrial audio data such as oil couplings. This feature extraction method can improve the model's perception of sound details. In addition, the CNN model is used for coupling state detection in this embodiment, which can efficiently process complex audio signals and extract multi-level feature information. This model structure is suitable for processing spatiotemporal patterns in audio, thereby improving the classification accuracy of different coupling states. In this embodiment, each frame of audio signal not only outputs a state label, but also comes with a probability value and confidence, which makes the model more transparent and controllable when making state judgments. This can help identify possible uncertainties and provide a reference for subsequent processing.

[0084] In this embodiment, by subdividing the state corresponding to the audio signal into seven labels (such as normal operation state, background noise, tightening, tightening completion, etc.), each key state of the iron roughneck oil coupling process can be accurately classified. This refined label design helps to improve the accuracy of detection and can better reflect the actual coupling operation.

[0085] The first training data set comes directly from the actual oil coupling operation process, containing audio signals of multiple time periods and their corresponding state labels. The use of this training data ensures the practical application effect of the model, enabling the model to cope with complex audio signals in real environments.

[0086] S4, based on the MFCC features and coupling state information corresponding to each frame of the audio signal, and the pre-obtained iron roughneck state data corresponding to the frame of the audio signal, respectively obtain a comprehensive feature vector corresponding to each frame of the audio signal;

[0087] The iron roughneck status data in this embodiment is obtained from the iron roughneck central control system.

[0088] Specifically, the S4 specifically includes:

[0089] S41, filtering the multi-frame audio signals obtained by dividing the audio signal into frames to obtain the filtered multi-frame audio signals;

[0090] The S41 specifically includes: removing each audio frame whose corresponding confidence level is less than a preset value from the multiple audio frames obtained after the audio signal is divided into frames, to obtain the filtered multiple audio frames.

[0091] In this embodiment, by removing audio frames with confidence values ​​lower than a preset value, the quality of the audio data processed subsequently is ensured to be higher. This screening method can effectively reduce the interference of noise and low-quality data, so that the model can make state judgments based on more reliable data, thereby improving the overall accuracy.

[0092] S42, performing frame splicing processing on the filtered multiple frames of audio signals to obtain multiple audio sequences;

[0093] When n≤M-4, the nth audio sequence PT obtained after frame splicing n = n , P (n+2) , P (n+4)};

[0094] When M-4<n≤M-2, the nth audio sequence PT obtained after the frame splicing process n = n , P (n+2)};

[0095] When M-2<n≤M, the nth audio sequence PT obtained after frame splicing n = n};

[0096] Wherein, M is the number of frames of the filtered audio signal;

[0097] P n is the nth frame of audio signal in the filtered multi-frame audio signals.

[0098] Different frame splicing strategies are adopted according to the number of audio frames M. This flexible splicing mechanism ensures that a valid audio sequence can be formed regardless of the number of audio signal frames. This method improves the system's adaptability to audio signals of different lengths and enhances the system's robustness.

[0099] As the value of n increases, the audio sequence PT n It is gradually shortened from containing multiple frames to only one frame. This design makes the splicing processing more flexible and ensures that useful information can be extracted when the amount of data is small.

[0100] S43, based on the audio sequence and the MFCC features and coupling status information corresponding to each frame of audio signal and the iron roughneck status data corresponding to the frame of audio signal, obtain a comprehensive feature vector corresponding to the frame of audio signal.

[0101] Among them, the comprehensive feature vector is AS n ;

[0102] AS n =[F n , S [n] , C [n] , S.C. [n] ];

[0103] F n It is the confidence level corresponding to the nth frame of audio signal in the filtered multi-frame audio signal;

[0104] S [n] = {P n , P n The probability value of each state label corresponding to it, P n The confidence of each state label corresponding to it};

[0105] C [n] Includes: nth audio sequence PT n The corresponding confidence curve;

[0106] Among them, the nth audio sequence PT nThe corresponding confidence curve is the nth audio sequence PT n A connecting line of the confidence of each state label corresponding to each frame of audio signal contained in ;

[0107] Iron Roughneck Status Data SC [n] is: any value among the first preset value, the second preset value and the third preset value;

[0108] Among them, the first preset value indicates that the iron roughneck is in an unknown state; the second preset value indicates that the iron roughneck is in a make-up state; and the third preset value indicates that the iron roughneck is in a make-up state.

[0109] Comprehensive feature vector AS n The confidence, probability value of the state label, confidence curve and iron roughneck state data are combined to form a multi-dimensional feature representation. This multi-dimensional feature can more comprehensively reflect the state information of each frame of audio signal and help improve the accuracy of state detection.

[0110] By connecting the confidence of each frame of the audio signal in the audio sequence into a curve, the confidence change trend of the audio signal is further analyzed. This curve analysis can identify possible state transitions or abnormal situations, providing more detailed information for subsequent decision-making. The specific operating state of the iron roughneck (makeup, breakout, or unknown state) is introduced into the comprehensive feature vector as a key feature, so that the model can better combine the actual operating conditions and make more accurate state judgments.

[0111] In this embodiment, the combined processing of steps S41 to S43 can adapt to audio signals of different qualities and lengths, thereby adapting to a wider range of application scenarios. This design improves flexibility and applicability in practical applications.

[0112] S5. Obtain a prediction result based on the comprehensive feature vector corresponding to each frame of the audio signal and / or the pre-acquired first audio signal segment;

[0113] The first audio signal segment is an audio signal of a pre-specified time period before the audio signal segment of the iron roughneck petroleum coupling make-up process or break-out process.

[0114] In this embodiment, the pre-specified time period is 2000 ms.

[0115] In this embodiment, S5 specifically includes:

[0116] S51, obtaining a comprehensive feature vector sequence based on the comprehensive feature vector corresponding to each frame of the audio signal;

[0117] AS seq ={AS1, AS2, AS3,...,AS M};

[0118] Among them, AS seq is the comprehensive feature vector sequence;

[0119] In this embodiment, the comprehensive feature vector (AS n ) is constructed into a sequence (AS seq ), which can retain the timing information of the audio signal and the feature information of each frame. This integration method allows the system to capture the global features of the audio signal, especially the time-related feature changes, thereby improving the recognition accuracy of the audio signal state.

[0120] As a high-dimensional data expression, the comprehensive feature vector sequence not only contains the feature information of a single frame, but also systematically describes the characteristics of the entire audio signal segment. This design helps capture complex patterns in audio signals and improve the prediction ability of the model.

[0121] S52, the comprehensive feature vector sequence AS seq and / or a first audio signal segment acquired in advance, inputted into a pre-trained LSTM time series detection model to obtain a prediction result;

[0122] The second training data set is used to train the LSTM time series detection model to obtain a pre-trained LSTM time series detection model;

[0123] The second training data set includes: a first comprehensive feature vector sequence and a first audio label corresponding to the first comprehensive feature vector sequence; a second comprehensive feature vector sequence and a second audio label corresponding to the second comprehensive feature vector sequence;

[0124] The first comprehensive feature vector sequence is a comprehensive feature vector sequence corresponding to the first audio signal segment; the second comprehensive feature vector sequence is a comprehensive feature vector sequence corresponding to the second audio signal segment;

[0125] The first audio signal segment is an audio signal segment of the process from the iron roughneck tightening to the jamming or sudden whistling; the second audio signal segment is an audio signal segment of the process from the iron roughneck unscrewing to the unscrewing completion;

[0126] The first audio tag is an indication that an audio signal segment having the first audio tag will show a state where the iron roughneck is tightened to a stuck state or a sudden howling state;

[0127] The second audio tag is an identifier indicating that the unscrewing completion process state appears in the audio signal segment having the second audio tag.

[0128] The prediction result is the comprehensive feature vector sequence AS seqThe corresponding audio signal segments of the iron roughneck oil coupling make-up process or break-out process correspond to the probability values ​​of the first audio label and the second audio label respectively.

[0129] In this embodiment, LSTM (Long Short-Term Memory Network) has unique advantages in processing time series data and can remember long-term dependencies. Inputting the comprehensive feature vector sequence into the pre-trained LSTM time series detection model can effectively capture the potential time-dependent features in the audio signal, which is very important for accurately identifying abnormal conditions during the operation of the iron driller. By using the LSTM model, accurate prediction of audio signal segments can be achieved. The model can predict the corresponding audio tag probability value based on the feature vector sequence. This high-precision prediction result helps to improve reliability in practical applications.

[0130] In this embodiment, by using a training data set (second training data set) including different state labels, it is ensured that the LSTM model can identify the states corresponding to different audio signal segments. This training method enables the model to better cope with complex and changeable actual operating environments, thereby enhancing the generalization ability of the system. By providing clear label information (such as the first audio label and the second audio label) for different audio signal segments, the model can clearly distinguish different states during the prediction process. The clarity of this label improves the credibility of the model prediction results and helps to achieve more accurate state recognition.

[0131] In this embodiment, the prediction results are output in the form of probability values, so that the method in this embodiment can make more flexible decisions based on the probability values ​​of different state labels. This design allows adjustments to be made according to different thresholds or risk preferences in actual applications, thereby improving the practicality of the method in this embodiment.

[0132] In this embodiment, by carefully dividing the operation process of the iron roughneck (such as tightening to a jam, sudden whistling, unscrewing completion, etc.), it is possible to achieve refined identification of different operation processes. This refined identification can help operators better understand and control the operation process of the iron roughneck, and improve the safety and efficiency of the operation.

[0133] See also Figure 2 This embodiment also provides a deep learning-based iron roughneck oil collar state sound recognition system, including:

[0134] A microphone array audio signal acquisition sensor, an embedded system, and an iron roughneck central control system connected in sequence;

[0135] Wherein, the microphone array audio signal acquisition sensor is arranged on the iron roughneck motion caliper, and is used to collect the audio signal segment of the iron roughneck oil coupling make-up process or make-out process;

[0136] The embedded system is used to adopt the deep learning-based sound recognition method for the state of the iron roughneck oil coupling as described in the embodiment, process the audio signal segment of the iron roughneck oil coupling make-up process or make-out process collected by the microphone array audio signal acquisition sensor, obtain the prediction result, and send the prediction result to the iron roughneck central control system.

[0137] In this embodiment, except for the normal operation sound, the background noise used for feedback and the noise removal that can be filtered by the make-up and break-out information given by the iron roughneck, the rest of the process is:

[0138] In the tightening state, when the iron roughneck is being screwed on, the characteristics of the audio signal show a gradually increasing torque characteristic. The torque data provided by the iron roughneck central control system is within the normal range and the display gradually increases. No abnormal noise (such as jamming or howling) occurs.

[0139] The tightening is completed and the roughneck is in the process of making the screw. The audio signal characteristics show that the tightening action is completed and the torque reaches the preset value. The torque data of the roughneck central control system reaches the preset tightening completion torque value and no abnormal noise occurs.

[0140] In the over-tightening state, when the roughneck is in the make-up state, a sudden abnormal noise (such as jamming or howling) appears in the audio signal, the torque data of the roughneck central control system exceeds the preset safety range, and the audio characteristics match the known characteristics of over-tightening.

[0141] In the unscrewing state, the roughneck is in the process of shackle breaking, and the audio signal characteristics show the torque characteristics of gradually decreasing. The status provided by the roughneck central control system shows that the unscrewing operation is in progress and no abnormal noise occurs.

[0142] The unscrewing is completed and the hook is in the roughneck shackle. The audio signal characteristics show that the unscrewing action is completed and the torque is reduced to near zero. The status of the roughneck central control system shows that the unscrewing operation is completed and no abnormal noise occurs.

[0143] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0144] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0145] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0146] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0147] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for identifying the state of an iron roughneck oil collar based on deep learning, characterized in that: include: S1, obtaining an audio signal segment of the process of making up or breaking out the oil coupling of the iron roughneck; S2, dividing the audio signal segment into frames according to a preset frame division logic to obtain multiple frames of audio signals; S3, based on each frame of audio signal, obtaining MFCC features of each frame of audio signal, and determining the hoop state information corresponding to the frame of audio signal according to the MFCC features; S4, based on the MFCC features and coupling state information corresponding to each frame of the audio signal, and the pre-obtained iron roughneck state data corresponding to the frame of the audio signal, respectively obtain a comprehensive feature vector corresponding to each frame of the audio signal; S5. Obtain a prediction result based on the comprehensive feature vector corresponding to each frame of the audio signal and / or the pre-acquired first audio signal segment; The first audio signal segment is an audio signal of a pre-specified time period before the audio signal segment of the iron roughneck petroleum coupling make-up process or break-out process.

2. The method for identifying the state of an iron roughneck oil coupling based on deep learning according to claim 1, characterized in that: The S2 specifically includes: The audio signal segment is divided into multiple frames of audio signals according to a pre-specified frame length.

3. The deep learning-based sound recognition method for the state of an iron roughneck oil coupling according to claim 2, characterized in that: The S3 specifically includes: S31, performing feature extraction processing on each frame of audio signal to obtain MFCC features of each frame of audio signal; S32, inputting the MFCC features of each frame of audio signal into a pre-trained hoop state detection model to obtain hoop state information corresponding to each frame of audio signal; The coupling state information corresponding to the audio signal includes: the probability value and confidence level of each state label corresponding to the frame of audio signal; The collar state detection model is trained using the first training data set to obtain a pre-trained collar state detection model; The coupling state detection model is a convolutional neural network model; The first training data set includes: audio signals of multiple time periods during the process of making up and breaking out the iron roughneck oil coupling, MFCC features corresponding to the audio signals of each time period, and pre-set state labels corresponding to the audio signals of each time period; The status labels include: a first label for indicating that the audio signal of the time period corresponds to the normal operating state in the coupling; a second label for indicating that the audio signal of the time period corresponds to the background noise in the coupling; a third label for indicating that the audio signal of the time period corresponds to the tightening state in the coupling; a fourth label for indicating that the audio signal of the time period corresponds to the tightening completion state in the coupling; a fifth label for indicating that the audio signal of the time period corresponds to the tightening transition state in the coupling; a sixth label for indicating that the audio signal of the time period corresponds to the unscrewing state in the coupling; and a seventh label for indicating that the audio signal of the time period corresponds to the unscrewing completion state in the coupling.

4. The deep learning-based sound recognition method for the state of an iron roughneck oil coupling according to claim 3, characterized in that: The S4 specifically includes: S41, filtering the multi-frame audio signals obtained by dividing the audio signal into frames to obtain the filtered multi-frame audio signals; S42, performing frame splicing processing on the filtered multiple frames of audio signals to obtain multiple audio sequences; S43, based on the audio sequence and the MFCC features and coupling status information corresponding to each frame of audio signal and the iron roughneck status data corresponding to the frame of audio signal, obtain a comprehensive feature vector corresponding to the frame of audio signal.

5. The method for identifying the state of an iron roughneck oil coupling based on deep learning according to claim 4, characterized in that: The S41 specifically includes: Among the multiple frames of audio signals obtained after the audio signal is divided into frames, each frame of audio signal corresponding to a confidence level less than a preset value is removed to obtain a filtered multiple frame audio signal.

6. The deep learning-based sound recognition method for the state of an iron roughneck oil coupling according to claim 5, characterized in that: The S42 specifically includes: When n≤M-4, the nth audio sequence PT obtained after frame splicing n = n , P (n+2) , P (n+4) }; When M-4<n≤M-2, the nth audio sequence PT obtained after the frame splicing process n = n , P (n+2) }; When M-2<n≤M, the nth audio sequence PT obtained after frame splicing n = n }; Wherein, M is the number of frames of the filtered audio signal; P n is the nth frame of audio signal in the filtered multi-frame audio signals.

7. The deep learning-based sound recognition method for the state of an iron roughneck oil coupling according to claim 6, characterized in that: The comprehensive feature vector is AS n ; AS n =[F n ,S [n] ,C [n] ,SC [n] ]; F n It is the confidence level corresponding to the nth frame of audio signal in the filtered multi-frame audio signal; S [n] = {P n , P n The probability value of each state label corresponding to it, P n The confidence of each state label corresponding to it}; C [n] Includes: nth audio sequence PT n The corresponding confidence curve; Among them, the nth audio sequence PT n The corresponding confidence curve is the nth audio sequence PT n A connecting line of the confidence of each state label corresponding to each frame of audio signal contained in ; Iron Roughneck Status Data SC [n] is: any value among the first preset value, the second preset value and the third preset value; Among them, the first preset value indicates that the iron roughneck is in an unknown state; the second preset value indicates that the iron roughneck is in a make-up state; and the third preset value indicates that the iron roughneck is in a make-up state.

8. The deep learning-based sound recognition method for iron roughneck oil coupling status according to claim 7, characterized in that: The S5 specifically includes: S51, obtaining a comprehensive feature vector sequence based on the comprehensive feature vector corresponding to each frame of the audio signal; AS seq ={AS1、AS2、AS3、...、AS M }; Among them, AS seq is the comprehensive feature vector sequence; S52, the comprehensive feature vector sequence AS seq and / or a first audio signal segment acquired in advance, inputted into a pre-trained LSTM time series detection model to obtain a prediction result; The second training data set is used to train the LSTM time series detection model to obtain a pre-trained LSTM time series detection model; The second training data set includes: a first comprehensive feature vector sequence and a first audio label corresponding to the first comprehensive feature vector sequence; a second comprehensive feature vector sequence and a second audio label corresponding to the second comprehensive feature vector sequence; The first comprehensive feature vector sequence is a comprehensive feature vector sequence corresponding to the first audio signal segment; the second comprehensive feature vector sequence is a comprehensive feature vector sequence corresponding to the second audio signal segment; The first audio signal segment is an audio signal segment of the process from the iron roughneck tightening to the jamming or sudden whistling; the second audio signal segment is an audio signal segment of the process from the iron roughneck unscrewing to the unscrewing completion; The first audio tag is an indication that an audio signal segment having the first audio tag will show a state where the iron roughneck is tightened to a stuck state or a sudden howling state; The second audio tag is an identifier indicating that the unscrewing completion process state appears in the audio signal segment having the second audio tag.

9. The deep learning-based sound recognition method for the state of an iron roughneck oil coupling according to claim 8, characterized in that: The prediction result is the comprehensive feature vector sequence AS seq The corresponding audio signal segments of the iron roughneck oil coupling make-up process or break-out process correspond to the probability values ​​of the first audio label and the second audio label respectively.

10. A deep learning-based iron roughneck oil coupling status sound recognition system, characterized in that: include: A microphone array audio signal acquisition sensor, an embedded system, and an iron roughneck central control system connected in sequence; Wherein, the microphone array audio signal acquisition sensor is arranged on the iron roughneck motion caliper, and is used to collect the audio signal segment of the iron roughneck oil coupling make-up process or make-out process; The embedded system is used to adopt the deep learning-based sound recognition method for the state of the iron roughneck oil coupling as described in any one of claims 1 to 9, process the audio signal segments of the iron roughneck oil coupling make-up process or make-out process collected by the microphone array audio signal acquisition sensor, obtain a prediction result, and send the prediction result to the iron roughneck central control system.

Images