Drill site drilling identification fusion method and system and electronic equipment

Abstract

The invention relates to the field of industrial control, in particular to a drill site drilling identification fusion method and system and electronic equipment. The method comprises the following steps that RFID tag information of all drill rods is collected in real time, and a drilling video and drilling machine parameters are obtained; if the AI drilling video identification is unsuccessful, analyzing the RFID tag information by using an information analysis algorithm to obtain a first drill rod counting result; if the passive RFID drilling recognition is unsuccessful, the AI intelligent recognition technology is used for recognizing and analyzing the drilling video, and a second drill rod counting result is obtained; and if the AI drilling video identification and the passive RFID drilling identification are both unsuccessful, analyzing the drilling machine parameters by using a parameter analysis algorithm to obtain a third drilling rod counting result. By fusing AI drilling video identification, passive RFID drilling identification and drilling machine state identification, the problem of inaccurate identification caused by a single mode is solved.

Description

Technical Field

[0001] The present invention relates to the field of industrial control, and in particular to a drilling identification fusion method, system and electronic equipment. Background Art

[0002] Drilling is one of the key links in gas control and water exploration and drainage. The real-time, accuracy and reliability of drilling records can ensure that the quality of drilling construction meets the construction requirements and is an important guarantee for safe production in coal mines.

[0003] At present, due to the poor underground environment, it is impossible to accurately identify and count the drillings through a single drilling site drilling identification method. Summary of the invention

[0004] The purpose of the present invention is to provide a drilling site drilling identification fusion method, system and electronic equipment to solve the problem that drilling cannot be accurately identified and counted by a single drilling site drilling identification method.

[0005] An embodiment of the present invention provides a drilling identification fusion method in a drilling field, the method comprising the following steps: real-time collection of RFID tag information of each drill rod, acquisition of drilling video and acquisition of drilling rig parameters; if AI drilling video identification is unsuccessful, the RFID tag information is analyzed by using an information analysis algorithm to obtain a first drill rod counting result; if passive RFID drilling identification is unsuccessful, the drilling video is identified and analyzed by using AI intelligent identification technology to obtain a second drill rod counting result; if both the AI ​​drilling video identification and the passive RFID drilling identification are unsuccessful, the drilling rig parameters are analyzed by using a parameter analysis algorithm to obtain a third drill rod counting result, wherein the drilling rig parameters include feed pressure, pull-out pressure and main pump pressure.

[0006] Optionally, the RFID tag information is analyzed using an information analysis algorithm to obtain a first drill rod count result, including: when a new RFID tag information is stored in the drill entry tag queue, the drill rod drill entry count is increased by one to obtain a drill rod drill entry count result; or, when a new RFID tag information is stored in the drill withdrawal tag queue, the drill rod drill withdrawal count is increased by one to obtain a drill rod drill withdrawal count result.

[0007] Optionally, the drilling video is identified and analyzed using AI intelligent recognition technology to obtain a second drill rod counting result, including: when a key drilling posture is identified, the drill rod drilling count is increased by one to obtain a drill rod drilling count result; or, when a key drilling withdrawal posture is identified, the drill rod drilling withdrawal count is increased by one to obtain a drill rod drilling withdrawal count result.

[0008] Optionally, the drilling rig parameters are analyzed using a parameter analysis algorithm to obtain a third drill rod counting result, including: when the feed pressure is the same as the main pump pressure, the drill rod drilling count is increased by one to obtain a drill rod drilling count result; or, when the pull-out pressure is the same as the main pump pressure, the drill rod retraction count is increased by one to obtain a drill rod retraction count result.

[0009] Optionally, the method further includes: when both the AI ​​drilling video recognition and the passive RFID drilling recognition are successful, comparing and verifying the first drill rod counting result and the second drill rod counting result.

[0010] Optionally, the method further includes: when the AI ​​drilling video recognition is unsuccessful, analyzing the drilling rig parameters using a parameter analysis algorithm to obtain the third drill rod counting result; and comparing and verifying the third drill rod counting result with the first drill rod counting result.

[0011] Optionally, the method further includes: when passive RFID drilling identification is unsuccessful, analyzing the drilling rig parameters using a parameter analysis algorithm to obtain the third drill rod counting result; and comparing and verifying the third drill rod counting result with the second drill rod counting result.

[0012] Compared with the prior art, the drilling identification fusion method provided by the present invention has the following beneficial effects:

[0013] The drilling identification fusion method provided by the embodiment of the present invention collects the RFID tag information of each drill rod in real time, obtains the drilling video and the drilling rig parameters; if the AI ​​drilling video identification is unsuccessful, the information analysis algorithm is used to analyze the above RFID tag information to obtain the first drill rod counting result; if the passive RFID drilling identification is unsuccessful, the AI ​​intelligent identification technology is used to identify and analyze the above drilling video to obtain the second drill rod counting result; if both the AI ​​drilling video identification and the passive RFID drilling identification are unsuccessful, the parameter analysis algorithm is used to analyze the above drilling rig parameters to obtain the third drill rod counting result, wherein the above drilling rig parameters include the feed pressure, the pull-out pressure and the main pump pressure. By integrating the three methods of AI drilling video identification, passive RFID drilling identification and drilling rig status identification, the problem of inaccurate identification caused by a single method is solved.

[0014] An embodiment of the present invention provides a drilling site drilling identification fusion system, and the system is used to implement the drilling site drilling identification fusion method.

[0015] The drilling site drilling identification fusion system provided by the present invention has the beneficial effect of being able to achieve the same technical effect as the above-mentioned drilling site drilling identification fusion method, and will not be described again here to avoid repetition.

[0016] An embodiment of the present invention provides an electronic device, including a computer-readable storage medium storing a computer program and a processor. When the computer program is read and executed by the processor, the drilling identification fusion method in the drilling field is implemented.

[0017] The electronic device provided by the present invention has the beneficial effect of being able to achieve the same technical effect as the above-mentioned drilling identification fusion method in the drilling field, and will not be described here to avoid repetition.

[0018] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is read and executed by a processor, the drilling identification fusion method described above is implemented.

[0019] The computer-readable storage medium provided by the present invention has the beneficial effect of being able to achieve the same technical effect as the above-mentioned drilling field drilling identification fusion method, and will not be described here to avoid repetition. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0021] Figure 1 A schematic flow chart of a drilling identification fusion method provided by an embodiment of the present invention;

[0022] Figure 2 Schematic diagram of three key frames in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0024] Artificial Intelligence (AI) drilling video recognition may fail due to factors such as ambient light and water mist that affect the camera imaging effect; passive radio frequency identification (RFID) may fail due to factors such as passive RFID tags being damaged or dirty that affect passive RFID recognition; drilling rig status recognition may fail due to factors such as sensor failure and line failure that affect drilling rig status recognition. In response to the above situations, human evaluation or algorithm evaluation can be used to determine whether the above recognition methods can be successful.

[0025] The embodiment of the present invention provides a drilling identification fusion method for a drilling site, see Figure 1 A schematic flow chart of a drilling identification fusion method in a drilling field is shown, and the method comprises the following steps:

[0026] S110, collecting RFID tag information of each drill rod, obtaining drilling video and obtaining drilling rig parameters in real time.

[0027] S120: If the AI ​​drilling video recognition is unsuccessful, the RFID tag information is analyzed using an information analysis algorithm to obtain a first drill rod counting result.

[0028] Optionally, the above step S120 includes: when a new RFID tag information is stored in the drilling tag queue, the drill rod drilling count is increased by one to obtain the drill rod drilling count result; or, when a new RFID tag information is stored in the drill withdrawal tag queue, the drill rod drilling withdrawal count is increased by one to obtain the drill rod drilling withdrawal count result.

[0029] S130, if the passive RFID drilling recognition is unsuccessful, the drilling video is recognized and analyzed using AI intelligent recognition technology to obtain a second drill rod counting result.

[0030] Optionally, the above step S130 includes: when a key drilling posture is identified, the drill rod drilling count is increased by one to obtain a drill rod drilling count result; or, when a key drilling withdrawal posture is identified, the drill rod drilling withdrawal count is increased by one to obtain a drill rod drilling withdrawal count result.

[0031] S140, if both the AI ​​drilling video recognition and the passive RFID drilling recognition are unsuccessful, the above drilling rig parameters are analyzed using a parameter analysis algorithm to obtain a third drill rod counting result.

[0032] The above drilling rig parameters include feed pressure, pull-out pressure and main pump pressure.

[0033] Optionally, the above step S140 includes: when the above feed pressure is the same as the above main pump pressure, the drill rod drilling count is increased by one to obtain the drill rod drilling count result; or, when the above pull-out pressure is the same as the above main pump pressure, the drill rod drilling retraction count is increased by one to obtain the drill rod drilling retraction count result.

[0034] The drilling identification fusion method provided by the embodiment of the present invention collects the RFID tag information of each drill rod in real time, obtains the drilling video and obtains the drilling rig parameters; if the AI ​​drilling video identification is unsuccessful, the information analysis algorithm is used to analyze the above RFID tag information to obtain the first drill rod counting result, so as to solve the problem of inaccurate drilling identification in the drilling site due to poor imaging conditions; if the passive RFID drilling identification is unsuccessful, the AI ​​intelligent identification technology is used to identify and analyze the above drilling video to obtain the second drill rod counting result, so as to solve the problem of inaccurate drilling identification in the drilling site due to the inability to be correctly identified by the passive RFID tag; if both the AI ​​drilling video identification and the passive RFID drilling identification are unsuccessful, the parameter analysis algorithm is used to analyze the above drilling rig parameters to obtain the third drill rod counting result, wherein the above drilling rig parameters include feed pressure, pull-out pressure and main pump pressure, so as to solve the problem of inaccurate drilling identification in the drilling site due to poor imaging conditions and the inability to be correctly identified by the passive RFID tag. The above-mentioned drilling site drilling identification fusion method integrates three methods: AI drilling video recognition, passive RFID drilling recognition, and drilling rig status recognition, solving the problem of inaccurate recognition caused by a single method.

[0035] Optionally, the method further includes: when both the AI ​​drilling video recognition and the passive RFID drilling recognition are successful, comparing and verifying the first drill rod counting result and the second drill rod counting result.

[0036] Optionally, the method further includes: when the AI ​​drilling video recognition is unsuccessful, analyzing the drilling rig parameters using a parameter analysis algorithm to obtain a third drill rod counting result; and comparing and verifying the third drill rod counting result with the first drill rod counting result.

[0037] Optionally, the method further includes: when the passive RFID drilling identification is unsuccessful, analyzing the drilling rig parameters using a parameter analysis algorithm to obtain a third drill rod counting result; and comparing and verifying the third drill rod counting result with the second drill rod counting result.

[0038] For example, when the two drill rod counting results are consistent, the drill rod counting result is correct; when the two drill rod counting results are inconsistent, the drilling site staff is notified to obtain actual drilling site situation information for analysis and take corresponding measures.

[0039] The embodiment of the present invention further ensures the accuracy of drilling identification in a drilling field by comparing and verifying the drill rod counting results of different modes, thereby making the identification result of the drilling identification fusion method in a drilling field more accurate.

[0040] An embodiment of the present invention also provides a specific method for AI drilling video recognition, which includes specific implementation methods of multiple aspects such as video stream transmission process, video stream processing, and target detection and recognition algorithm.

[0041] 1) Video streaming process

[0042] 1. Build a Hypertext Transfer Protocol (HTTP) server: This server can receive the video stream address sent by the camera in the coal mine. The video content includes the instructions for starting drilling, starting drilling, and ending drilling, and the corresponding video.

[0043] 2. Start AI video analysis: After receiving the start command, the system starts the AI ​​video analysis module. Due to the difference in speed between AI video stream analysis and the video stream transmitted by the camera, the system uses a flexible buffer pool strategy to balance the speed difference between the two.

[0044] 3. Result push: After receiving the end command, the drill withdrawal result is pushed to the business end through the message queue (such as MQTT, Kafka) or the POST request in HTTP.

[0045] 2) Specific implementation of video stream processing

[0046] 1.HTTP server tasks:

[0047] Accept request: Process the access request sent by the requester and obtain the JS key-value pair data (JavaScript Object Notation, JSON) in the POST request. The JSON data contains the camera's Real-Time Streaming Protocol (RealTime Streaming Protocol, RTSP) stream address, the current request's universal unique identifier (UUID, used for concurrent processing) and the state identifier (state, used to control the opening or closing of the service).

[0048] Return data: The requester needs to obtain the push drill count and the processed RTSP stream address. The server continuously returns the current number of push drills processed through streaming response and pushes the processed RTSP stream to the requester.

[0049] 2. Media Server: To achieve the above goals, the system uses the mediamtx media server, which is ready-to-use and has zero dependencies, allowing the publishing, reading, proxying, recording and playing of video and audio streams. The requester obtains the RTSP stream on the mediamtx server through the RTSP stream address returned by the server.

[0050] 3. Strategy design: To achieve stable and low-latency RTSP streaming, the system adopts a single producer and multiple consumers strategy.

[0051] Specifically, single producer: read RTSP video stream thread, continuously read RTSP video stream, and add timestamp to each frame and put it into the read buffer pool; multiple consumers: model processing thread and push stream buffer pool (output buffer pool) thread. Model processing thread: If the processing speed of a single model cannot keep up with the speed of reading the stream, multiple 4 model processing threads are opened for parallel processing. If the resources are still insufficient, one frame is processed every few frames, and the rest of the frames are replaced by the processed results. Pose estimation is performed for each frame image, and threshold judgment is performed between the previous and next frames. When the threshold is met, it is considered that a de-drilling is completed. Output buffer pool thread: Ensure the correct order of frames and ensure the stability of the pushed RTSP video stream. Threshold judgment is performed after sorting according to the timestamp. The buffer pool adjusts the sleep time of the thread according to the number of stored frames to ensure that there are always enough frames stored in the buffer pool to cope with sudden video interruption and reduce the number of video stream freezes or interruptions.

[0052] Through the above design, the system can effectively realize the collection and processing of real-time video data, ensure the stable capture and analysis of miners' drill withdrawal behavior information in complex mine environments, and efficiently push the results to the business end. This solution maintains the stability and low latency of the video stream while achieving efficient data processing and transmission, providing reliable data support for coal mine safety production.

[0053] 3) Target detection and recognition algorithm

[0054] The embodiment of the present invention provides a target detection and recognition model based on the extraction of key postures of miners and the matching of key point coordinates to reduce the interference of irrelevant information in the background of underground coal mine monitoring videos, so as to improve the accuracy of drill rod counting.

[0055] Key pose extraction extracts skeletal modal data from key pose frames through a lightweight pose estimation network called BlazePose. Key point coordinate matching uses normalized Euclidean distance to calculate the similarity between behaviors. When the similarity is less than the set threshold, it means that the action in the video is completed, and the count is increased by one, thereby realizing the counting of coal mine drill rod operations. In order to describe the operations of coal mine miners more comprehensively and accurately, see Figure 2The schematic diagram of the three key frames shown in the figure, this algorithm intends to use three key frame actions for judgment, and the three key frames include a start frame, an execution frame, and an end frame.

[0056] It should be noted that when a group of drilling start frames, drilling execution frames and drilling end frames are identified, a drilling key posture is identified; when a group of drilling retraction start frames, drilling retraction execution frames and drilling retraction end frames are identified, a drilling retraction key posture is identified.

[0057] When the matching degree of the behavior reaches the threshold of the start frame, the action is considered to have started, and then the similarity of the execution frame is evaluated. After the execution frame is completed, the count is increased when the threshold of the end frame is recognized, and finally the recognition of the completed drilling action is realized. Here, the matching degree of the behavior is recognized by the traditional Euclidean distance, and its formula is as follows:

[0058]

[0059] Among them, matching rate is the matching degree; i is the model output of the i-th sample; T i is the true output of the i-th sample.

[0060] Exemplarily, key frame identification includes:

[0061] 1) Keyframe action recognition network

[0062] In order to effectively identify the actions of miners operating drill rods, a set of key frames that can fully represent these actions need to be manually selected from the video. The selected frames provide the information needed to train the pose encoding network. A lightweight pose estimation network called BlazePose is used, which combines key point heat maps, offset errors, and key point regression. During the training process, the above-mentioned BlazePose network uses the key point heat map regression method to improve the learning effect of the key point position. However, in actual use, this network only retains the key point coordinate regression part, thereby reducing the model parameters. In addition, other pose estimation models can also be used, such as the OpenPose model and the AlphaPose model.

[0063] In addition, the network includes a tracker that identifies key points of the human body in the current frame and enables action tracking. If key points are detected, the pose estimation model uses the position estimate instead of re-detection, which speeds up inference.

[0064] 2) Keyframe percentage match count

[0065] In an embodiment of the present invention, the percentage of correct key points (PCK) metric provides a clear and intuitive measure of model accuracy, which is suitable for pose estimation tasks. In addition, PCK can also be used to compare the performance of different pose estimation models. A higher PCK value indicates better performance because it represents that more key points are accurately predicted within the specified threshold.

[0066] The PCK usage process is as follows:

[0067] 1. Keypoint detection: For each image or frame, the model predicts the locations of multiple keypoints of the human body. These keypoints usually include joints such as elbows, knees, shoulders, etc.

[0068] 2. Distance calculation: Calculate the Euclidean distance between each predicted key point and the corresponding true key point.

[0069] 3. Apply threshold: Compare each distance to the threshold. If the distance is less than or equal to the threshold, the keypoint is considered to be correctly predicted.

[0070] 4. Calculate the indicator: Calculate the PCK value, which is the ratio of the number of correctly predicted key points to the total number of key points.

[0071] 5. Behavior count: When the PCK value exceeds a certain value, it is considered a behavior match. When all three key frame behaviors are matched, the behavior is considered completed and the corresponding drilling count is performed.

[0072] It should be noted that PCK is used as an indicator to evaluate the accuracy of the pose estimation model. It measures the degree of match between the human key points predicted by the model and the real key points. The formula corresponding to PCK is as follows:

[0073]

[0074] Where N is the total number of key points, δ i is the indicator function, when When true, then δ i =1; when When is false, then δ i = 0; where di is the Euclidean distance between the predicted position and the true position of the ith key point. d is the reference length (e.g., the length of the head or torso). T is the threshold (e.g., 0.2 times the reference length).

[0075] An embodiment of the present invention provides a drilling site drilling identification fusion system, which is used to implement the drilling site drilling identification fusion method.

[0076] The drilling site drilling identification fusion system provided by the present invention has the beneficial effect of being able to achieve the same technical effect as the above-mentioned drilling site drilling identification fusion method, and will not be described again here to avoid repetition.

[0077] An embodiment of the present invention provides an electronic device, including a computer-readable storage medium storing a computer program and a processor. When the computer program is read and executed by the processor, the drilling identification fusion method is implemented.

[0078] The electronic device provided by the present invention has the beneficial effect of being able to achieve the same technical effect as the above-mentioned drilling identification fusion method in the drilling field, and will not be described here to avoid repetition.

[0079] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is read and executed by a processor, the drilling identification fusion method is implemented.

[0080] The computer-readable storage medium provided by the present invention has the beneficial effect of being able to achieve the same technical effect as the above-mentioned drilling field drilling identification fusion method, and will not be described here to avoid repetition.

[0081] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the control device through a computer, and the program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the above-mentioned method embodiments, wherein the storage medium may be a memory, a disk, an optical disk, etc.

[0082] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0083] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0084] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A drilling identification and fusion method for a drilling site, characterized in that: The method comprises the following steps: Collect RFID tag information of each drill rod, obtain drilling video and drill rig parameters in real time; If the AI ​​drilling video recognition is unsuccessful, the RFID tag information is analyzed using an information analysis algorithm to obtain a first drill rod counting result; If the passive RFID drilling recognition is unsuccessful, the drilling video is identified and analyzed using AI intelligent recognition technology to obtain a second drill rod counting result; If both the AI ​​drilling video recognition and the passive RFID drilling recognition are unsuccessful, the drilling rig parameters are analyzed using a parameter analysis algorithm to obtain a third drill rod counting result, where the drilling rig parameters include feed pressure, pull-out pressure, and main pump pressure.

2. The drilling identification and fusion method according to claim 1, characterized in that: The analyzing the RFID tag information using an information analysis algorithm to obtain a first drill rod counting result includes: When a new RFID tag information is stored in the drilling tag queue, the drill rod drilling count is increased by one to obtain the drill rod drilling count result; Alternatively, when a new RFID tag information is stored in the drill rod withdrawal tag queue, the drill rod withdrawal count is increased by one to obtain a drill rod withdrawal count result.

3. The drilling identification and fusion method according to claim 1, characterized in that: The use of AI intelligent recognition technology to identify and analyze the drilling video to obtain a second drill rod counting result includes: When a key drilling posture is identified, the drill rod drilling count is increased by one to obtain the drill rod drilling count result; Alternatively, when a drill retraction key posture is identified, the drill rod retraction count is increased by one to obtain a drill rod retraction count result.

4. The drilling identification and fusion method according to claim 1, characterized in that: The method of analyzing the drilling rig parameters using a parameter analysis algorithm to obtain a third drill rod counting result includes: When the feed pressure is the same as the main pump pressure, the drill rod drilling count is increased by one to obtain the drill rod drilling count result; Alternatively, when the pull-out pressure is the same as the main pump pressure, the drill rod withdrawal count is increased by one to obtain a drill rod withdrawal count result.

5. The drilling identification and fusion method according to claim 1, characterized in that: The method further comprises: When both the AI ​​drilling video recognition and the passive RFID drilling recognition are successful, the first drill rod counting result and the second drill rod counting result are compared and verified with each other.

6. The drilling identification and fusion method according to claim 1, characterized in that: The method further comprises: When the AI ​​drilling video recognition is unsuccessful, the drilling rig parameters are analyzed using a parameter analysis algorithm to obtain the third drill rod counting result; The third drill rod counting result is compared and verified with the first drill rod counting result.

7. The drilling identification and fusion method according to claim 1, characterized in that: The method further comprises: When the passive RFID drilling identification is unsuccessful, the drilling rig parameters are analyzed using a parameter analysis algorithm to obtain the third drill rod counting result; The third drill rod counting result and the second drill rod counting result are compared and verified with each other.

8. A drilling identification and fusion system for a drilling site, characterized in that: The system is used to implement the method according to any one of claims 1 to 7.

9. An electronic device, characterized in that: The invention comprises a computer-readable storage medium storing a computer program and a processor, wherein when the computer program is read and executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is read and executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images