Loading and unloading source operation target detection method and related equipment

Abstract

The invention discloses a source loading and unloading operation target detection method and related equipment, and belongs to the field of machine vision. The method comprises the following steps: firstly, acquiring real-time video data of source loading and unloading operation of a logging team; inputting the real-time video data into a pre-constructed loading and unloading source operation target detection model, and finally outputting a detection result; the loading and unloading source operation target detection model adopts a Yolo-Pose key point target detection network model improved on the basis of yov5; different from a traditional target detection method, a Yolo-Pose key point target detection network model improved based on the yov5 not only can detect a target, but also can detect key point information of the target, such as the angle and line segment information of a radioactive source rod, can more accurately detect the position of the radioactive source rod in a picture and the relative position relation between the radioactive source rod and a human body, and improves the detection accuracy of the radioactive source rod. Therefore, the radioactive source rod is distinguished from other rod-shaped objects, compared with 2d frame information, the method is more accurate, multiple targets and target actions are recognized at the same time, and the requirement for detection accuracy can be met.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision, specifically to the field of object detection, and particularly relates to a method for detecting objects in the loading and unloading source operation and related equipment. Background Art

[0002] In the existing object detection methods, the Yolov5 network model is widely used as the basic network model. By training its own dataset, the object detection function of the specified type can be realized. According to the depth and width of the network structure, Yolov5 can be divided into five structures: Yolov5n, Yolov5s, Yolov5m, Yolov5l, and Yolov5x. The main framework of each network structure is basically the same; in the network structure configuration parameters, the depth and width of the network are flexibly configured through two parameters, depth_multiple and width_multiple, so as to obtain network models of different sizes.

[0003] In the scenario of loading and unloading source operation, multi-object detection such as human detection, handheld radiation source rod detection, lead clothing detection, anti-drop chuck detection, and buffer pad detection, as well as key point detection of the radiation rod, are required to determine whether the loading and unloading source operation is standardized. There are various rod-shaped objects such as columns, poles, and sticks at the operation site, and these objects are easily confused with the radiation source rods that also present a rod shape. Therefore, using the ordinary Yolov5 object detection model, only the 2D box information of the object can be obtained, and it is very difficult to distinguish other rod-shaped objects from the radiation source rods only through the 2D box information. In this scenario where both multi-object recognition and object action recognition are required, a suitable model cannot be selected from the existing network structures of Yolov5.

[0004] It can be seen that the existing method for detecting objects in the loading and unloading source operation cannot guarantee the accuracy of object detection in the scenario where multi-object and object action need to be recognized simultaneously. Summary of the Invention

[0005] To overcome the shortcomings of the above technology, the present invention provides a method for detecting objects in the loading and unloading source operation and related equipment, which can solve the technical problem that the existing method for detecting objects in the loading and unloading source operation cannot meet the accuracy requirement of object detection because it needs to recognize multi-object and object action simultaneously.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A method for detecting objects in the loading and unloading source operation includes:

[0008] Obtaining real-time video data of the loading and unloading source operation of the logging team;

[0009] Input the real-time video data into the loading and unloading source operation target detection model, and output the target detection result;

[0010] Among them, the loading and unloading source operation target detection model uses a Yolo-Pose key point target detection network model improved based on yolov5.

[0011] Furthermore, the construction steps of the loading and unloading source operation target detection model are as follows:

[0012] Collect the real-time video data of the loading and unloading source operation and screen out the images containing the detection targets;

[0013] Divide the images containing the detection targets into a training set and a test set, and construct a Yolo-Pose key point target detection network model improved based on yolov5;

[0014] Input the training set into the Yolo-Pose key point target detection network model improved based on yolov5 to obtain the loading and unloading source operation target detection model, and use the test set to verify the loading and unloading source operation target detection model, and output the loading and unloading source operation target detection model.

[0015] Furthermore, among them, extract the image frames from the collected real-time video data of the loading and unloading source operation, screen out the images containing the detection targets, use annotation software to annotate the images containing the detection targets, and establish a data set; divide the data set into a training set and a test set.

[0016] Furthermore, among them, divide the data set into a training set and a test set according to a ratio of 9:1.

[0017] Furthermore, among them, the Yolo-Pose key point target detection network model improved based on yolov5 includes:

[0018] Adopt CSPDarknet backbone as the Backbone part; adopt PANet as the Neck part.

[0019] Furthermore, among them, the real-time video data includes images containing personnel, lead aprons on the personnel, anti-drop chucks, buffer pads, and radioactive source rods.

[0020] Furthermore, the target detection result includes personnel, lead aprons on the personnel, anti-drop chucks, buffer pads, radioactive source rods, and the target names corresponding to the above detection targets.

[0021] A loading and unloading source operation target detection system for implementing the steps of the above loading and unloading source operation target detection method, including:

[0022] A data acquisition module for acquiring the real-time video data of the logging crew's loading and unloading source operation;

[0023] A target detection module, configured to input real-time video data into a loading and unloading source operation target detection model and output a target detection result; wherein, the loading and unloading source operation target detection model adopts a Yolo-Pose key point target detection network model improved based on yolov5.

[0024] A device, comprising:

[0025] A memory, configured to store a computer program;

[0026] A processor, configured to implement the steps of the above-mentioned loading and unloading source operation target detection method when executing the computer program.

[0027] A computer-readable storage medium, storing a computer program, which is used to implement the steps of the above-mentioned loading and unloading source operation target detection method when executed by a processor.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention also provides a loading and unloading source operation target detection method. This method first obtains real-time video data of the loading and unloading source operation of a logging crew; then inputs the real-time video data into a pre-constructed loading and unloading source operation target detection model, and finally outputs a detection result; the loading and unloading source operation target detection model adopts a Yolo-Pose key point target detection network model improved based on yolov5; different from traditional target detection methods, the Yolo-Pose key point target detection network model improved based on yolov5 can not only detect the target, but also detect the key point information of the target, such as the angle and line segment information of the radiation source rod, and can more accurately detect the position of the radiation source rod in the picture and the relative position relationship between the radiation source rod and the human body, so as to distinguish the radiation source rod from other rod-shaped objects, which is more accurate than 2d box information. Using this method to simultaneously identify multiple targets and target actions can meet the requirements of detection accuracy. Description of the Drawings

[0030] Figure 1 It is a flowchart of a loading and unloading source operation target detection method provided by an embodiment of the present invention;

[0031] Figure 2 It is a structural diagram of a Yolo-Pose key point target detection network model improved based on yolov5 provided by an embodiment of the present invention;

[0032] Figure 3 It is a flowchart of a loading and unloading source operation target detection method provided by the present invention;

[0033] Figure 4Schematic diagram of a loading and unloading source operation target detection system provided by the present invention;

[0034] Figure 5 Schematic diagram of the detection principle provided by an embodiment of the present invention. Detailed implementation manners

[0035] The present invention provides a method for detecting loading and unloading source operation targets, as Figure 3 shown, including the following steps:

[0036] S1: Obtain real-time video data of the loading and unloading source operation of the logging team.

[0037] S2: Input the real-time video data into the loading and unloading source operation target detection model, and output the target detection result.

[0038] Here, the loading and unloading source operation target detection model adopts a Yolo-Pose key point target detection network model improved based on yolov5.

[0039] Among them, the real-time video data includes images of personnel, lead clothing on the personnel, anti-falling chucks, buffer pads, and radioactive source rods.

[0040] The target detection result includes the target names corresponding to the above detection targets such as personnel, lead clothing on the personnel, anti-falling chucks, buffer pads, and radioactive source rods.

[0041] Specifically, the construction steps of the loading and unloading source operation target detection model are as follows:

[0042] S201: Collect real-time video data of the loading and unloading source operation and screen out the images containing detection targets;

[0043] S202: Divide the images containing detection targets into a training set and a test set, and construct a Yolo-Pose key point target detection network model improved based on yolov5;

[0044] S203: Input the training set into the Yolo-Pose key point target detection network model improved based on yolov5 to obtain the loading and unloading source operation target detection model, and use the test set to verify the loading and unloading source operation target detection model, and output the loading and unloading source operation target detection model.

[0045] In S201, image frames are extracted from the collected real-time video data of the loading and unloading source operation, the images containing detection targets are screened out, and the images containing detection targets are labeled using labeling software to establish a data set; the data set is divided into a training set and a test set according to a ratio of 9:1.

[0046] In S202, the structure of the Yolo-Pose key point object detection network model improved based on yolov5 includes: using CSPDarknet backbone as the Backbone part; using PANet as the Neck part.

[0047] As Figure 4 shown, the present invention also provides a loading and unloading source operation target detection system, including: a data acquisition module for acquiring real-time video data of the logging team's loading and unloading source operation; a target detection module for inputting the real-time video data into the loading and unloading source operation target detection model and outputting a target detection result; wherein, the loading and unloading source operation target detection model adopts a Yolo-Pose key point object detection network model improved based on yolov5.

[0048] The present invention also provides a device, including: a memory for storing a computer program; a processor for implementing the steps of the above-mentioned loading and unloading source operation target detection method when executing the computer program.

[0049] When the processor executes the computer program, it implements the above-mentioned steps of loading and unloading source operation target detection, for example: acquiring real-time video data of the logging team's loading and unloading source operation; inputting the real-time video data into the loading and unloading source operation target detection model and outputting a target detection result; wherein, the loading and unloading source operation target detection model adopts a Yolo-Pose key point object detection network model improved based on yolov5.

[0050] Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-mentioned system, for example: a data acquisition module for acquiring real-time video data of the logging team's loading and unloading source operation; a target detection module for inputting the real-time video data into the loading and unloading source operation target detection model and outputting a target detection result; wherein, the loading and unloading source operation target detection model adopts a Yolo-Pose key point object detection network model improved based on yolov5.

[0051] Exemplarily, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of completing preset functions, and the instruction segments are used to describe the execution process of the computer program in the loading and unloading source operation target detection device. For example, the computer program can be divided into a data acquisition module and a target detection module; the specific functions of each module are as follows: the data acquisition module is used to acquire real-time video data of the loading and unloading source operation of the logging crew; the target detection module is used to input the real-time video data into the loading and unloading source operation target detection model and output the target detection result; wherein, the loading and unloading source operation target detection model adopts a Yolo-Pose key point target detection network model improved based on yolov5.

[0052] The loading and unloading source operation target detection device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The loading and unloading source operation target detection device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above are examples of the loading and unloading source operation target detection device, which do not constitute a limitation on the loading and unloading source operation target detection device, and may include more components than the above, or combine some components, or different components. For example, the loading and unloading source operation target detection device may further include an input / output device, a network access device, a bus, etc.

[0053] The so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the loading and unloading source operation target detection, and uses various interfaces and lines to connect all parts of the loading and unloading source operation target detection device.

[0054] The memory can be used to store the computer program and / or module. The processor realizes various functions of the loading and unloading source operation target detection device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory.

[0055] The memory may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0056] The present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the method for detecting a loading and unloading source operation target are implemented.

[0057] If the modules / units integrated in the loading and unloading source operation target detection system are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0058] Based on such an understanding, all or part of the processes in the method for detecting a loading and unloading source operation target of the present invention can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the method for detecting a loading and unloading source operation target can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, an executable file, or a preset intermediate form, etc.

[0059] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0060] It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0061] The present invention will be further described below in conjunction with embodiments and the accompanying drawings:

[0062] Example 1

[0063] As described in the background art, in the scenario where it is required to identify multiple targets and target actions, it is impossible to select a suitable model from the existing Yolov5 network structure. That is, the existing target detection method for loading and unloading source operations cannot guarantee the accuracy of target detection because it is necessary to identify multiple targets and target actions simultaneously. Or rather, the existing recognition methods cannot meet the detection requirements of multiple targets and target actions on the premise of ensuring the accuracy of target detection.

[0064] To solve the above problems, this embodiment provides a target detection method for loading and unloading source operations, which can improve the Yolo-Pose key point target detection network model based on the yolov5 structure in the scenario of identifying multiple targets and target actions, combine the advantages of the two models, and ensure the accuracy of target detection, as Figure 5 shown.

[0065] As Figure 1 shown, this embodiment provides a target detection method for loading and unloading source operations, and the specific steps are as follows:

[0066] Step 1: Install a network monitoring camera in front of the loading and unloading source operation to collect video data of the real-time work of the loading and unloading source operation.

[0067] Step 2: Extract image frames from the video data collected in Step 1, and use marking software to mark the specified type of targets that appear in the images to establish a data set.

[0068] Step 3: Use the data set to construct a Yolo-Pose key point target detection network model improved based on yolov5;

[0069] Step 4: Use the training set in Step 2 to train the model constructed in Step 3 to obtain a trained target detection model (i.e., the target detection model for loading and unloading source operations), and use the test set in Step 2 to verify the trained target detection model;

[0070] Step 5: Input the real-time monitoring video stream during the real-time work of the loading and unloading source operation into the trained target detection model in Step 4 to detect the specified type of targets. If the targets corresponding to the specified type are found, mark the target positions and send out prompt information.

[0071] Among them, the frame rate of the network monitoring camera used in Step 1 is not less than 30 frames per second;

[0072] Step 2 specifically includes extracting each frame of image from the video stream, and screening out the images containing the specified type of target; splitting the screened images into a training set and a test set according to a ratio of 9:1; numbering the pictures in the training set and the test set, and using a marking software to perform rectangular box annotation on the specified type of target part of the pictures in the training set and the validation set, and obtaining the corresponding coordinates of the rectangular box.

[0073] In step 3, as Figure 2 shown, Figure 2 is the specific model structure of the Yolo-Pose key point target detection network model improved based on yolov5. Yolo-Pose improved based on Yolov5 is a new joint detection method without Heatmap, which is different from the existing two-stage methods based on Heatmap. YOLO-Pose can train the model end-to-end and optimize the OKS metric itself. The model learns to jointly detect the bounding boxes of multiple people and their corresponding two-dimensional poses in a single forward pass, thus exceeding the best results of both top-down and bottom-up methods. The backbone of Yolo-Pose uses the CSPDarknet backbone, the Neck part uses PANet, followed by four detection heads with different scales, and two decoupled heads are used to predict the boxes and key points. YOLO-Pose does not require post-processing of the bottom-up method to group the detected key points into a skeleton, because each bounding box has a related pose, resulting in an inherent grouping of key points. Different from the top-down method, multiple forward propagations are eliminated because the poses of all people are localized.

[0074] In step 5, the frame frequency of the network monitoring camera used is not less than 30 frames per second. After detecting the specified target, annotate the target with a square box and display the name information of the target beside it.

[0075] This embodiment provides a method for detecting the target of a loading and unloading source, and its detection principle is as follows:

[0076] Due to the presence of various rod-shaped objects such as columns, poles, and sticks at the operation site, these objects are easily confused with the radiation source rod which also presents a rod shape; using an ordinary yolov5 object detection model, only the 2D box information of the target can be obtained. And only through the 2D box information, it is very difficult to distinguish other rod-shaped objects from the radiation source rod. For multi-object detection, the requirement of detection accuracy cannot be guaranteed. In this method, the Yolo-Pose key point object detection network model improved based on yolov5 is used. When detecting the radiation source rod, the key point information of the rod can also be obtained, so as to calculate the angular orientation of the radiation source rod, and a line segment can be used to replace the radiation source rod as the position. These information are more accurate than the ordinary 2D box information. Using the angular and line segment key point information of the radiation source rod, the position of the radiation source rod in the picture and the relative position relationship between the radiation source rod and the human body can be detected more accurately, so as to distinguish the radiation source rod from other rod-shaped objects. For example, when the on-site operator operates the radiation source rod, the radiation source rod is held horizontally in the operator's hand, while other rod-shaped objects are standing upright in the picture, and the relative position relationship between other rod-shaped objects and the human body is completely different from that of the radiation source rod; through these key point information, this method can detect the radiation source rod more accurately, so as to more accurately judge whether there is a person holding the radiation source rod on the site.

[0077] Embodiment 2

[0078] This embodiment provides a method for detecting the target of the source loading and unloading operation, and the specific steps are as follows:

[0079] Step 1: Use the network monitoring camera installed in front of the source loading and unloading operation at the wellhead of the logging to collect the data of the source loading and unloading operation work video. The frame rate of the video collected by the camera is 30 frames / s;

[0080] Step 2: Extract the image frames from the video collected in Step 1. About 14,000 images containing the source loading and unloading operation are screened out from the extracted images, and they are split into a training set and a validation set according to a ratio of 9:1, that is, 12,600 images are used as the training set, and the remaining 1,400 images are used as the test set. Use the annotation software to annotate the detected target personnel, the lead clothing on the personnel, the anti-drop chuck, the buffer pad, and the radiation source rod target in the dataset, and obtain the corresponding coordinates of the rectangular frames of the detected target personnel, the lead clothing on the personnel, the anti-drop chuck, the buffer pad, and the radiation source rod target.

[0081] Step 3: Build a Yolo-Pose key point object detection network model improved based on yolov5;

[0082] Step 4: Use the labeled training set in Step 2 to train the Yolo-Pose key point object detection network model improved based on yolov5 constructed in Step 3 to obtain a trained object detection model, and use the validation set in Step 2 to verify the trained object detection model;

[0083] Step 5: Input the real-time monitoring video stream of the network monitoring camera installed above the logging wellhead loading and unloading source operation machine into the trained object detection model in Step 4. When detecting target personnel, the lead clothing on the personnel, the anti-falling chuck, the buffer pad, and the radiation source rod during the loading and unloading of the source, rectangular frames are used to mark the targets of the personnel, the lead clothing on the personnel, the anti-falling chuck, the buffer pad, and the radiation source rod on the image, and the target names are displayed beside them, that is, the target names corresponding to the personnel, the lead clothing on the personnel, the anti-falling chuck, the buffer pad, and the radiation source rod.

[0084] In summary, the present invention provides a method for detecting the target of loading and unloading the source. Compared with the existing detection methods, it has the following advantages:

[0085] This method first obtains the real-time video data of the logging team's loading and unloading source operation; then inputs the real-time video data into the pre-constructed object detection model for loading and unloading the source, and finally outputs the detection result; the object detection model for loading and unloading the source uses a Yolo-Pose key point object detection network model improved based on yolov5; different from the traditional object detection method, the Yolo-Pose key point object detection network model improved based on yolov5 can not only detect the target, but also detect the key point information of the target, such as the angle and line segment information of the radiation source rod, and can more accurately detect the position of the radiation source rod in the picture and the relative position relationship between the radiation source rod and the human body, so as to distinguish the radiation source rod from other rod-shaped objects, which is more accurate than the 2d frame information. Using this method to simultaneously identify multiple targets and target actions can meet the requirements of detection accuracy.

[0086] The above embodiments are only one of the implementation manners that can realize the technical solution of the present invention. The scope of protection required by the present invention is not only limited by this embodiment, but also includes any changes, substitutions and other implementation manners that are easily conceivable by those skilled in the art within the technical scope disclosed by the present invention.

Claims

1. A method for detecting the operation target of a loading and unloading source, characterized in that, Including: Obtain real-time video data of the operation of loading and unloading the source by the logging crew; Input the real-time video data into the target detection model for loading and unloading the source, and output the target detection result; Among them, the target detection model for loading and unloading the source adopts a Yolo-Pose key point target detection network model improved based on yolov5.

2. The method for detecting the operation target of a loading and unloading source according to claim 1, characterized in that, The construction steps of the target detection model for loading and unloading the source are as follows: Collect real-time video data of the operation of loading and unloading the source and screen out the images containing detection targets; Divide the images containing detection targets into a training set and a test set, and construct a Yolo-Pose key point target detection network model improved based on yolov5; Input the training set into the Yolo-Pose key point target detection network model improved based on yolov5 to obtain the target detection model for loading and unloading the source, and use the test set to verify the target detection model for loading and unloading the source, and output the target detection model for loading and unloading the source.

3. The method for detecting the operation target of a loading and unloading source according to claim 2, characterized in that, Among them, Extract image frames from the collected real-time video data of the operation of loading and unloading the source, screen out the images containing detection targets, use annotation software to annotate the images containing detection targets, and establish a data set; Divide the data set into a training set and a test set.

4. The method for detecting the operation target of a loading and unloading source according to claim 3, characterized in that, Among them, Divide the data set into a training set and a test set according to a ratio of 9:

1.

5. The method for detecting the operation target of a loading and unloading source according to claim 2, characterized in that, Among them, The Yolo-Pose key point target detection network model improved based on yolov5 includes: Adopt CSPDarknet backbone as the Backbone part; adopt PANet as the Neck part.

6. The method for detecting the operation target of a loading and unloading source according to claim 1, characterized in that, Among them, The real-time video data includes images containing personnel, lead clothing on the personnel, anti-falling chucks, buffer pads, and radioactive source rods.

7. The method for detecting the operation target of a loading and unloading source according to claim 6, characterized in that, The target detection result includes personnel, lead clothing on the personnel, anti-falling chucks, buffer pads, radioactive source rods, and the target names corresponding to the above detection targets.

8. A system for detecting the operation target of a loading and unloading source, which is used to implement the steps of the method for detecting the operation target of a loading and unloading source according to any one of claims 1-7, characterized in that, Including: A data acquisition module for obtaining real-time video data of the operation of loading and unloading the source by the logging crew; A target detection module for inputting the real-time video data into the target detection model for loading and unloading the source and outputting the target detection result; among them, the target detection model for loading and unloading the source adopts a Yolo-Pose key point target detection network model improved based on yolov5.

9. A device, characterized in that, Including: A memory for storing computer programs; A processor for implementing the steps of the target detection method for loading and unloading the source according to any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium, the computer-readable storage medium stores a computer program, characterized in that, The computer program is used to implement the steps of the target detection method for loading and unloading the source according to any one of claims 1-7 when executed by the processor.

Images