Drilling recognition fusion method and system for drilling site, and electronic device

Abstract

The present invention relates to the field of industrial control, and in particular, to a drilling recognition fusion method and system for a drilling site, and an electronic device. The method comprises the following steps: acquiring RFID tag information of each drill rod in real time, acquiring drilling video, and acquiring a drilling rig parameter; if AI drilling video recognition is unsuccessful, using an information analysis algorithm to analyze the RFID tag information so as to obtain a first drill rod counting result; if passive RFID drilling recognition is unsuccessful, using AI intelligent recognition technology to perform recognition and analysis on the drilling video so as to obtain a second drill rod counting result; and if both the AI drilling video recognition and the passive RFID drilling recognition are unsuccessful, using a parameter analysis algorithm to analyze the drilling rig parameter so as to obtain a third drill rod counting result. Fusing AI drilling video recognition, passive RFID drilling recognition, and drilling rig state recognition addresses the problem of recognition inaccuracies resulting from a single method.

Description

Drilling field drilling identification fusion method, system and electronic equipment TECHNICAL FIELD

[0001] The present application relates to the field of industrial control, in particular to a drilling field drilling identification fusion method, system and electronic equipment. BACKGROUND

[0002] Drilling is one of the key links of gas control and water exploration and drainage, and the real-time, accuracy and reliability of drilling records can ensure that the drilling construction quality meets the construction requirements, which is an important guarantee for coal mine safety production.

[0003] At present, due to the poor underground environment, the single drilling field drilling identification method cannot accurately identify and count the drilling. SUMMARY

[0004] The purpose of the present application is to provide a drilling field drilling identification fusion method, system and electronic equipment to solve the problem that the single drilling field drilling identification method cannot accurately identify and count the drilling.

[0005] The present application provides a drilling field drilling identification fusion method, which comprises the following steps: real-time acquisition of RFID tag information of each drill rod, acquisition of drilling video and acquisition of drilling machine parameters; if AI drilling video identification is unsuccessful, the RFID tag information is analyzed by using an information analysis algorithm to obtain a first drill rod count result; if passive RFID drilling identification is unsuccessful, the drilling video is identified and analyzed by using AI intelligent identification technology to obtain a second drill rod count result; if both the AI drilling video identification and the passive RFID drilling identification are unsuccessful, the drilling machine parameters are analyzed by using a parameter analysis algorithm to obtain a third drill rod count result, wherein the drilling machine parameters include feed pressure, pullout pressure and main pump pressure.

[0006] Optionally, the RFID tag information is analyzed by using the information analysis algorithm to obtain the first drill rod count result, which comprises: when a new RFID tag information is stored in the drilling tag queue, the drilling rod drilling count is increased by one to obtain the drilling rod drilling count result; or, when a new RFID tag information is stored in the drilling tag queue, the drilling rod drilling count is increased by one to obtain the drilling rod drilling count result.

[0007] Optionally, the drilling video is identified and analyzed by using the AI intelligent identification technology to obtain the second drill rod count result, which comprises: when an drilling key posture is identified, the drilling rod drilling count is increased by one to obtain the drilling rod drilling count result; or, when a drilling key posture is identified, the drilling rod drilling count is increased by one to obtain the drilling rod drilling count result.

[0008] Optionally, the drilling rig parameter is analyzed by using a parameter analysis algorithm to obtain a third drill pipe count result, including: when the feeding pressure is the same as the main pump pressure, a drill pipe drilling-in count is increased by one to obtain a drill pipe drilling-in count result; or, when the pulling pressure is the same as the main pump pressure, a drill pipe drilling-out count is increased by one to obtain a drill pipe drilling-out count result.

[0009] Optionally, the method further includes: when the AI drilling video recognition and the passive RFID drilling recognition are both successful, the first drill pipe count result and the second drill pipe count result are compared with each other.

[0010] Optionally, the method further includes: when the AI drilling video recognition is unsuccessful, the drilling rig parameter is analyzed by using a parameter analysis algorithm to obtain the third drill pipe count result; and the third drill pipe count result and the first drill pipe count result are compared with each other.

[0011] Optionally, the method further includes: when the passive RFID drilling recognition is unsuccessful, the drilling rig parameter is analyzed by using a parameter analysis algorithm to obtain the third drill pipe count result; and the third drill pipe count result and the second drill pipe count result are compared with each other.

[0012] Compared with the prior art, the drilling field drilling recognition fusion method provided by the application has the beneficial effects that:

[0013] The drilling field drilling recognition fusion method provided by the embodiment of the application acquires RFID tag information of each drill pipe, acquires drilling video, and acquires drilling rig parameters; if the AI drilling video recognition is unsuccessful, the RFID tag information is analyzed by using an information analysis algorithm to obtain a first drill pipe count result; if the passive RFID drilling recognition is unsuccessful, the drilling video is recognized and analyzed by using an AI intelligent recognition technology to obtain a second drill pipe count result; and if the AI drilling video recognition and the passive RFID drilling recognition are both unsuccessful, the drilling rig parameters are analyzed by using a parameter analysis algorithm to obtain a third drill pipe count result, wherein the drilling rig parameters include feeding pressure, pulling pressure, and main pump pressure. The AI drilling video recognition, the passive RFID drilling recognition, and the drilling rig state recognition are fused together, so that the problem of inaccurate recognition caused by a single mode is solved.

[0014] The embodiment of the application provides a drilling field drilling recognition fusion system, which is used for implementing the drilling field drilling recognition fusion method.

[0015] The drilling field drilling recognition fusion system provided by the application has the beneficial effects that the same technical effects as the drilling field drilling recognition fusion method are achieved, and details are not repeated here to avoid repetition.

[0016] The embodiment of the present application provides an electronic device, comprising a computer readable storage medium storing a computer program and a processor, and when the computer program is read and run by the processor, the drilling field drilling identification fusion method is realized.

[0017] The electronic device provided by the present application has the beneficial effect that the same technical effects as the drilling field drilling identification fusion method can be achieved, and to avoid repetition, the same technical effects will not be described here.

[0018] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and when the computer program is read and run by a processor, the drilling field drilling identification fusion method is realized.

[0019] The computer readable storage medium provided by the present application has the beneficial effect that the same technical effects as the drilling field drilling identification fusion method can be achieved, and to avoid repetition, the same technical effects will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or the prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0021] Fig. 1 is a schematic flow chart of a drilling field drilling identification fusion method provided by the embodiment of the present application;

[0022] Fig. 2 is a schematic diagram of three key frames in the embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings.

[0024] The artificial intelligence (AI) drilling video recognition may fail due to factors such as environmental light and water mist affecting the camera imaging effect. The passive radio frequency identification (RFID) may fail due to factors such as passive RFID tag damage and dirt affecting passive RFID recognition. The drilling rig state recognition may fail due to factors such as sensor failure and line failure affecting drilling rig state recognition. In view of the above situations, it can be determined whether the above recognition methods can be successfully recognized through manual evaluation or algorithm evaluation.

[0025] The embodiment of the present application provides a drilling field drilling recognition fusion method, and a schematic flow chart of the drilling field drilling recognition fusion method is shown in FIG. 1. The method comprises the following steps:

[0026] S110, real-time collection of RFID tag information of each drill rod, acquisition of drilling video and acquisition of drilling rig parameters.

[0027] S120, if the AI drilling video recognition is unsuccessful, the RFID tag information is analyzed by using an information analysis algorithm to obtain a first drill rod count result.

[0028] Optionally, the step S120 comprises: when a new RFID tag information is stored in the drilling-in tag queue, the drill rod drilling-in count is increased by one to obtain a drill rod drilling-in count result; or when a new RFID tag information is stored in the drilling-out tag queue, the drill rod drilling-out count is increased by one to obtain a drill rod drilling-out count result.

[0029] S130, if the passive RFID drilling recognition is unsuccessful, the drilling video is recognized and analyzed by using an AI intelligent recognition technology to obtain a second drill rod count result.

[0030] Optionally, the step S130 comprises: when a drilling-in key posture is recognized, the drill rod drilling-in count is increased by one to obtain a drill rod drilling-in count result; or when a drilling-out key posture is recognized, the drill rod drilling-out count is increased by one to obtain a drill rod drilling-out count result.

[0031] S140, if the AI drilling video recognition and the passive RFID drilling recognition are both unsuccessful, the drilling rig parameters are analyzed by using a parameter analysis algorithm to obtain a third drill rod count result.

[0032] The drilling rig parameters comprise feeding pressure, pulling pressure and main pump pressure.

[0033] Optionally, the step S140 comprises: when the feeding pressure is the same as the main pump pressure, the drill pipe feeding count is increased by one to obtain a drill pipe feeding count result; or, when the pulling pressure is the same as the main pump pressure, the drill pipe pulling count is increased by one to obtain a drill pipe pulling count result.

[0034] The drilling field drilling identification fusion method provided by the embodiment of the present application solves the problem of inaccurate drilling field drilling identification caused by poor imaging conditions by real-time collection of RFID tag information of each drill pipe, acquisition of drilling video, and acquisition of rig parameters; if AI drilling video identification is unsuccessful, the RFID tag information is analyzed by using an information analysis algorithm to obtain a first drill pipe count result; if passive RFID drilling identification is unsuccessful, the drilling video is identified and analyzed by using AI intelligent identification technology to obtain a second drill pipe count result; if both AI drilling video identification and passive RFID drilling identification are unsuccessful, the rig parameters are analyzed by using a parameter analysis algorithm to obtain a third drill pipe count result, wherein the rig parameters comprise feeding pressure, pulling pressure, and main pump pressure. The drilling field drilling identification fusion method fuses AI drilling video identification, passive RFID drilling identification, and rig state identification, and solves the problem of inaccurate identification caused by a single method.

[0035] Optionally, the method further comprises: when both AI drilling video identification and passive RFID drilling identification are successful, the first drill pipe count result and the second drill pipe count result are compared with each other.

[0036] Optionally, the method further comprises: when AI drilling video identification is unsuccessful, the rig parameters are analyzed by using a parameter analysis algorithm to obtain a third drill pipe count result; and the third drill pipe count result and the first drill pipe count result are compared with each other.

[0037] Optionally, the method further comprises: when passive RFID drilling identification is unsuccessful, the rig parameters are analyzed by using a parameter analysis algorithm to obtain a third drill pipe count result; and the third drill pipe count result and the second drill pipe count result are compared with each other.

[0038] Illustratively, when the two drill pipe count results are consistent, the drill pipe count result is correct; when the two drill pipe count results are inconsistent, the drilling field staff is notified, actual drilling field situation information is acquired for analysis, and corresponding measures are taken.

[0039] The embodiment of the present application further guarantees the accuracy of the drilling identification in the drilling field by comparing and verifying the drill pipe counting results in different ways, so that the identification result of the above drilling identification fusion method is more accurate.

[0040] The embodiment of the present application also provides a specific way of AI drilling video identification, which includes specific implementation modes of video stream transmission process, video stream processing, target detection and identification algorithm and other aspects.

[0041] 1) Video stream transmission process

[0042] 1. Build a Hypertext Transfer Protocol (HTTP) server: The server can receive the video stream address sent by the camera in the coal mine. The video content includes the start drilling, start drilling, end drilling instruction and the corresponding video.

[0043] 2. Start AI video analysis: After receiving the start command, the system starts the AI video analysis module. Due to the speed difference between AI video stream analysis and camera transmission video stream, the system uses a flexible buffer pool strategy to balance the speed difference between the two.

[0044] 3. Result pushing: After receiving the end command, the drilling result is pushed to the business end through the message queue (such as mqtt, kafka) or HTTP POST request.

[0045] 2) Video stream processing specific implementation

[0046] 1. HTTP server task:

[0047] Accept request: process the access request sent by the request end, and get the JS key value pair data (JavaScript Object Notation, JSON) in the POST request. The JSON data includes the Real Time Streaming Protocol (RTSP) stream address of the camera, the current request universal unique identifier (UUID, used for concurrent processing) and state identifier (state, used for controlling the opening or closing of the service).

[0048] Return data: the request side needs to get the push drill number count and the processed RTSP stream address. The server returns the current processing push drill number through streaming response, and pushes the processed RTSP stream to the request side.

[0049] 2. Media server: To achieve the above objectives, the system selects the mediamtx media server, which is ready-to-use and zero-dependent, allowing the publishing, reading, proxying, recording, and playing of video and audio streams. The requester obtains the RTSP stream on the mediamtx server through the RTSP stream address returned by the server.

[0050] 3. Strategy design: To achieve stable and low-delay RTSP streams, the system adopts a single-producer and multi-consumer strategy.

[0051] Specifically, single producer: RTSP video stream reading thread, continuously reading RTSP video stream and putting each frame with timestamp into the reading buffer pool; multi-consumer: model processing thread and push stream buffer pool (output buffer pool) thread. Model processing thread: if the speed of a single model processing cannot keep up with the reading stream speed, multiple 4 model processing threads are started for parallel processing. If resources are still insufficient, process one frame every few frames, and replace the remaining frames with the processed results. Pose estimation is performed on each frame of image, and threshold judgment is performed between consecutive frames. If the threshold is met, it is considered that a de-drilling is completed. Output buffer pool thread: ensures the correct order of frames and the stability of the pushed RTSP video stream. After sorting according to the timestamp, threshold judgment is performed. The buffer pool adjusts the sleep time of the thread according to the number of stored frames, ensures that there are always enough frames stored in the buffer pool to cope with sudden video stream interruption, and reduces the number of video stream stalls or interruptions.

[0052] Through the above design, the system can effectively realize the collection and processing of real-time video data, ensure the stable capture and analysis of miner de-drilling behavior information in complex mine environment, and efficiently push the results to the business end. This solution not only realizes efficient data processing and transmission, but also maintains the stability and low delay of the video stream, providing reliable data support for coal mine safety production.

[0053] 3) Target detection and recognition algorithm

[0054] The embodiment of the present application provides a target detection and recognition model based on miner key posture extraction and key point coordinate matching, to reduce the interference of irrelevant information in the coal mine underground monitoring video background, and to improve the accuracy of drill rod counting.

[0055] The key pose extraction extracts the skeleton modality data from the key pose frame by a lightweight pose estimation network named BlazePose. The key point coordinate matching uses the normalized Euclidean distance to calculate the similarity between behaviors, and when the similarity is less than a set threshold, it indicates that the action in the video is completed, and the count is increased by one, thereby realizing the coal mine drill rod operation counting. In order to be able to more comprehensively and accurately describe the operation of the coal mine worker, referring to the schematic diagram of three key frames shown in FIG. 2, the algorithm is intended to use three key frame actions to make a judgment, and the three key frames include a start frame, an execution frame and an end frame.

[0056] It should be noted that when a set of drilling start frames, drilling execution frames and drilling end frames are identified, it is identified as a drilling key pose; when a set of drilling start frames, drilling execution frames and drilling end frames are identified, it is identified as a drilling key pose.

[0057] When the matching degree of the behavior reaches the threshold value of the start frame, it is considered that the action starts, and then the similarity of the execution frame is evaluated. After the execution frame is completed, the threshold value of the end frame is identified, and the count is increased. Finally, the completed drill moving action recognition is realized. Here, the behavior matching degree recognition is identified by the traditional Euclidean distance, and the formula is as follows:

[0058]

[0059] Among them, is the matching degree; is the model output of the i-th sample; is the true output of the i-th sample. Exemplarily, the key frame recognition includes:

[0060] 1) Key frame action recognition network

[0061] 1) Key frame action recognition network

[0062] In order to effectively identify the actions of the miners operating the drill rod, a set of key frames that can fully represent these actions need to be manually selected from the video. These selected frames provide the information needed to train the pose encoding network. A lightweight pose estimation network named BlazePose is used, which combines key point heat maps, offset errors and key point regression. In the training process, the above BlazePose network uses the key point heat map regression method to improve the learning effect of the key point position. However, in actual use, this network only retains the key point coordinate regression part, thereby reducing the model parameters. In addition, other pose estimation models can also be used, such as the OpenPose model and the AlphaPose model.

[0063] ​In addition, the network also contains a tracker that identifies human key points in the current frame and enables behavior tracking. If key points are detected, the pose estimation model uses position estimation instead of re-detection, thereby speeding up inference.

[0064] 2) Keyframe percentage matching count

[0065] In embodiments of the present application, the correct key point percentage (PCK) metric provides a clear and intuitive measure of model accuracy suitable for pose estimation tasks. In addition, PCK can be used to compare the performance of different pose estimation models. A higher PCK value indicates better performance, as it represents more key points being accurately predicted within a specified threshold.

[0066] The use process of PCK is as follows:

[0067] 1. Key point detection: For each image or frame, the model predicts the positions of multiple key points of the human body. These key points usually include joints such as elbows, knees, shoulders, etc.

[0068] 2. Distance calculation: Calculate the Euclidean distance between each predicted key point and the corresponding true key point.

[0069] 3. Apply threshold: Compare each distance with a threshold. If the distance is less than or equal to the threshold, the key point is considered to be correctly predicted.

[0070] 4. Calculate the index: Calculate the PCK value, which is the ratio of the number of correctly predicted key points to the total number of key points.

[0071] 5. Behavior count: If the PCK value exceeds a certain value, it is counted as a behavior match. When three keyframe behaviors are matched, it is considered as a completed behavior, and the corresponding drill count is performed.

[0072] It should be noted that PCK is an index for evaluating the accuracy of pose estimation models. It measures the matching degree of the predicted human key points by the model and the true key points. The formula corresponding to PCK is as follows:

[0073]

[0074] where, is the total number of key points, is an indicator function, when is true, then ; when is false, then ; where is the Euclidean distance between the predicted position and the true position of the th key point. L is a reference length (e.g., length of a head or torso). L is a threshold value (e.g., 0.2 times the reference length).

[0075] The drilling field drilling identification fusion system provided by the embodiment of the present application has the beneficial effects as follows: the same technical effects as the drilling field drilling identification fusion method can be achieved.

[0076] The drilling field drilling identification fusion system provided by the embodiment of the present application has the beneficial effects as follows: the same technical effects as the drilling field drilling identification fusion method can be achieved.

[0077] The electronic device provided by the embodiment of the present application has the beneficial effects as follows: the same technical effects as the drilling field drilling identification fusion method can be achieved.

[0078] The electronic device provided by the embodiment of the present application has the beneficial effects as follows: the same technical effects as the drilling field drilling identification fusion method can be achieved.

[0079] The computer readable storage medium provided by the embodiment of the present application has the beneficial effects as follows: the same technical effects as the drilling field drilling identification fusion method can be achieved.

[0080] The computer readable storage medium provided by the embodiment of the present application has the beneficial effects as follows: the same technical effects as the drilling field drilling identification fusion method can be achieved.

[0081] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer degree to instruct a control device, and the program can be stored in a computer readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed, wherein the storage medium can be a memory, a disk, an optical disk, etc.

[0082] Finally, it needs to be pointed out that, in this document, the terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0083] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0084] Although the present application is disclosed as above, the present application is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and the protection scope of the present application should be defined by the scope defined in the claims.

Claims

1. A drilling site drilling identification fusion method, characterized in that, The method comprises the following steps: Real-time acquisition of RFID tag information of each drill rod, acquisition of drilling video and acquisition of drilling rig parameters; If AI drilling video recognition is unsuccessful, information analysis algorithm is used to analyze the RFID tag information to obtain a first drill rod count result; If passive RFID drilling recognition is unsuccessful, AI intelligent recognition technology is used to identify and analyze the drilling video to obtain a second drill rod count result; If both the AI drilling video recognition and the passive RFID drilling recognition are unsuccessful, parameter analysis algorithm is used to analyze the drilling rig parameters to obtain a third drill rod count result, wherein the drilling rig parameters include feed pressure, pullout pressure and main pump pressure.

2. The drilling field drilling identification fusion method according to claim 1, characterized in that, The use of information analysis algorithm to analyze the RFID tag information to obtain a first drill rod count result comprises: When a new RFID tag information is stored in the drilling-in tag queue, the drill rod drilling-in count is increased by one to obtain a drill rod drilling-in count result; Or, when a new RFID tag information is stored in the drilling-out tag queue, the drill rod drilling-out count is increased by one to obtain a drill rod drilling-out count result.

3. The drilling field drilling identification fusion method according to claim 1, characterized in that, The use of AI intelligent recognition technology to identify and analyze the drilling video to obtain a second drill rod count result comprises: When an in-drilling key posture is identified, the drill rod drilling-in count is increased by one to obtain a drill rod drilling-in count result; Or, when an out-drilling key posture is identified, the drill rod drilling-out count is increased by one to obtain a drill rod drilling-out count result.

4. The drilling field drilling identification fusion method according to claim 1, characterized in that, The use of parameter analysis algorithm to analyze the drilling rig parameters to obtain a third drill rod count result comprises: When the feed pressure is the same as the main pump pressure, the drill rod drilling-in count is increased by one to obtain a drill rod drilling-in count result; Or, when the pullout pressure is the same as the main pump pressure, the drill rod drilling-out count is increased by one to obtain a drill rod drilling-out count result.

5. The drilling field drilling identification fusion method according to claim 1, characterized in that, The method further comprises: When both the AI drilling video recognition and the passive RFID drilling recognition are successful, the first drill rod count result and the second drill rod count result are compared with each other for verification.

6. The drilling field drilling identification fusion method according to claim 1, characterized in that, The method further comprises: When the AI drilling video recognition is unsuccessful, parameter analysis algorithm is used to analyze the drilling rig parameters to obtain the third drill rod count result; The third drill rod count result and the first drill rod count result are compared with each other for verification.

7. The drilling field drilling identification fusion method according to claim 1, characterized in that, The method further comprises: When the passive RFID drilling recognition is unsuccessful, parameter analysis algorithm is used to analyze the drilling rig parameters to obtain the third drill rod count result; The third drill rod count result and the second drill rod count result are compared with each other for verification.

8. A drilling site drilling identification fusion system, characterized in that, The system is used to implement the method of any one of claims 1-7.

9. An electronic device, comprising: A computer readable storage medium storing a computer program and a processor, wherein the computer program is read and run by the processor to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer readable storage medium storing a computer program, wherein the computer program is read and run by a processor to implement the method of any one of claims 1-7.

Images