Method and device for detecting rotation radius of person entering machine, medium and equipment

Abstract

The invention discloses a method and device for detecting the rotation radius of a person entering a machine, a medium and equipment. The method comprises the following steps: acquiring a monitoring video of mechanical operation; screening out an image containing a mechanical operation scene from the monitoring video; carrying out manual annotation on the image containing the mechanical operation scene to obtain annotation data, and dividing the annotation data; constructing a target detection model, and obtaining a trained target detection model; obtaining a to-be-detected picture; inputting a to-be-detected picture into the trained target detection model, inputting the to-be-detected picture into a target detection network after image decoding and image preprocessing operation, and obtaining coordinates, categories and confidence coefficients of the personnel and the rotating machinery based on a detection result; and calculating the central point distance between the person and the rotating machine and the mechanical arm length of the rotating machine based on the detection result of the single image, and judging whether the person is within the rotating radius of the rotating machine. According to the method, whether a person is within the rotation radius of the rotating machine can be judged in real time, and the image quality is improved.

Description

Technical Field

[0001] This invention relates to the field of mechanical inspection technology, and in particular to methods, devices, media and equipment for detecting the radius of rotation of machinery when personnel enter. Background Technology

[0002] Oilfield operations commonly involve excavation and hoisting, with excavators, cranes, and other rotating machinery posing a risk of personnel entering the rotating radius and sustaining mechanical injuries. A crucial safety measure is ensuring timely detection when personnel are within the machinery's rotation radius, allowing for the implementation of necessary safety measures to prevent accidents. Traditional methods typically rely on hardware such as mechanical sensors, photoelectric sensors, or lidar, but these can be limited by environmental noise, equipment malfunctions, or high maintenance costs. Effective video surveillance is urgently needed to identify and mitigate such risks.

[0003] With the rapid development of video detection technology, intelligent monitoring methods based on image and video data are becoming increasingly attractive. By using deep learning models such as deep convolutional neural networks (CNNs), it is possible to detect whether workers have entered the rotation radius of machinery in real-time video streams, thereby improving workplace safety. This approach not only reduces reliance on additional sensors but also provides higher accuracy and real-time performance.

[0004] However, introducing video detection into the field of mechanical rotation radius detection faces several challenges, such as poor image quality, and detection errors caused by changes in illumination, target size, and pose. Summary of the Invention

[0005] The main objective of this invention is to provide a method, apparatus, and equipment for detecting the radius of rotation of a machine when a person enters it, aiming to solve the technical problems of poor image quality and detection errors caused by changes in illumination, target size, and posture in the prior art.

[0006] To achieve the above objectives, the present invention provides a method for detecting the radius of rotation of machinery when personnel enter it. This method includes: S10, acquiring monitoring video V of the machinery operation based on a monitoring camera connected to the work site. i S20, from the monitoring video V i Images containing mechanical operation scenes were selected from the images. i S30, for the image I containing the mechanical operation scene i Perform manual annotation to obtain annotation data I ann and the labeled data I ann S40, Perform segmentation; S40, Construct target detection model, and process the labeled data I ann Perform training to obtain a trained object detection model; S50, obtain the image to be detected.test S60, the image to be detected I test The trained object detection model is input into the object detection network after image decoding and preprocessing. Based on the detection results, the coordinates, categories, and confidence scores of the personnel and rotating machinery are obtained. S70: Based on the detection results of a single image, the distance D between the center point of the personnel and the rotating machinery is calculated. cent Given the length W of the robotic arm of the rotating machinery, determine whether the person is within the rotation radius of the rotating machinery. If D cent If <= 1.5*W, then the personnel are within the rotation radius of the rotating machinery.

[0007] In some embodiments, step S30 includes: S310, processing the image I containing the mechanical operation scene. i Perform manual annotation to obtain annotation data I ann S320, the labeled data I ann Divided into training samples I train and validation sample I valid .

[0008] In some embodiments, step S40 includes: S410, constructing a deep learning-based object detection model; S420, processing the training samples I. train and validation sample I valid Perform Mosaic data augmentation training; S430, augment the training sample I with Mosaic data. train and validation sample I valid Train the model to obtain a well-trained object detection model.

[0009] In some embodiments, step S50 includes: S510, extracting frames from the video stream of the monitoring camera at the access work site using FFmpeg, extracting 1 frame every N frames as a detection image I. test .

[0010] In some embodiments, before step S60, the following step is further included: S520, inputting the test images belonging to the same test group into the trained target detection model to extract the coordinates of the target.

[0011] In some embodiments, step S60 includes: S610, transferring the image to be detected I... test The trained object detection model is input into the object detection network after image decoding and preprocessing; S620, the detection results are filtered for redundant and duplicate bounding boxes using Non-Maximum Suppression (NMS); S630, the coordinates of personnel and rotating machinery are obtained based on the detection results. ,category and confidence level , where n is the number of people or rotating machinery detected in the image.

[0012] In some embodiments, step S70 includes: S710, determining whether there are people and rotating machinery in the detection result of a single image; S720, if they exist, calculating the area ratio of people to rotating machinery. S730, If the area of ​​the personnel is 10% to 30% of the area of ​​the rotating machinery, then determine that the personnel and the rotating machinery are around the rotating machinery; S740, calculate the distance D between the center point of the personnel and the rotating machinery. cent The length W of the robotic arm of the rotating machinery is 750, and if D... cent If <= 1.5*W, then the personnel are within the rotation radius of the rotating machinery.

[0013] Furthermore, to achieve the above objectives, embodiments of this application also provide a detection device for personnel entering the rotation radius of machinery. This device includes a video acquisition module, used to acquire monitoring video V of the machinery operation based on a monitoring camera connected to the work site. i Image filtering module, used to filter images from the surveillance video V i Images containing mechanical operation scenes were selected from the images. i The manual annotation module is used to annotate the images I containing the mechanical operation scene. i Perform manual annotation to obtain annotation data I ann and the labeled data I ann The system performs segmentation; a model building and training module is used to build an object detection model and train the labeled data I. ann The training module is used to obtain a trained object detection model; the image acquisition module is used to acquire the image to be detected. test； The detection module is used to detect the image I. test The trained object detection model is input into the object detection network after image decoding and preprocessing. Based on the detection results, the coordinates, categories, and confidence scores of personnel and rotating machinery are obtained. A calculation module is used to calculate the distance D between the center point of the personnel and the center point of the rotating machinery based on the detection results of a single image. cent Given the length W of the robotic arm of the rotating machinery, determine whether the person is within the rotation radius of the rotating machinery. If D cent If <= 1.5*W, then the personnel are within the rotation radius of the rotating machinery.

[0014] Furthermore, to achieve the above objectives, embodiments of this application also provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform a method for detecting the radius of rotation of a machine entered by a person, as described in any embodiment of this application.

[0015] Furthermore, to achieve the above objectives, embodiments of this application also provide a computing device, which includes at least one processor, a memory, and an input / output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the method for detecting the radius of rotation of a person entering a machine according to any embodiment of this application.

[0016] The method for detecting personnel entering the rotation radius of machinery provided in this application involves manually annotating images containing mechanical operation scenes to obtain annotation data; inputting the image to be detected into a trained target detection model, followed by image decoding and preprocessing, and then inputting it into a target detection network; obtaining the coordinates, category, and confidence level of the personnel and rotating machinery based on the detection results, thereby improving image quality and greatly reducing detection errors caused by changes in illumination, target size, and posture; and calculating the distance between the center point of the personnel and the rotating machinery and the length of the mechanical arm of the rotating machinery based on the detection results of a single image, thereby determining in real time whether the personnel are within the rotation radius of the rotating machinery. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a method for detecting the radius of rotation of a machine when a person enters it, provided in one or more embodiments of this application; Figure 2 A structural block diagram of a detection device for detecting the radius of rotation of a machine for personnel entering, provided in an embodiment of this application; Figure 3 Another flowchart of a method for detecting the radius of rotation of a machine when a person enters it, as provided in an embodiment of this application; Figure 4 The original Mosaic graphic enhancement stitching order provided for one or more embodiments of this application; Figure 5 An improved Mosaic graphics enhancement stitching order is provided for one embodiment of this application; Figure 6 A schematic diagram illustrating the algorithm training for personnel entering the mechanical rotation radius provided in one or more embodiments of this application; Figure 7 A schematic diagram of an AF-FPN network provided for one or more embodiments of this application; Figure 8 Another flowchart of a method for detecting the radius of rotation of a machine for personnel entering, provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a medium provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the application. Rather, these embodiments are provided to make the disclosure more thorough and complete, and to fully convey the scope of the disclosure to those skilled in the art.

[0020] Those skilled in the art will understand that the embodiments of this application can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this application can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0021] This application provides a method for detecting the radius of rotation of machinery when a person enters it. This method can be executed by a computer, such as... Figure 1 , Figure 3 and Figure 8 As shown, the method may include the following steps: S10, acquire monitoring video of mechanical operation based on the monitoring cameras accessing the work site. i .

[0022] Among them, surveillance cameras are semiconductor imaging devices with advantages such as high sensitivity, resistance to strong light, low distortion, small size, long life and vibration resistance.

[0023] S20, from the monitoring video V i Images containing mechanical operation scenes were selected from the images. i .

[0024] S30, the image I containing the mechanical operation scene... i Perform manual annotation to obtain annotation data I ann and the labeled data I ann Divide into sections.

[0025] In an exemplary embodiment, step S30 may specifically include the following steps: S310, the image I containing the mechanical operation scene... i Perform manual annotation to obtain annotation data I ann ; S320, the labeled data I ann Divided into training samples I train and validation sample I valid .

[0026] Specifically, training samples are datasets used to train machine learning models, while validation samples are datasets used to validate the performance of the models.

[0027] S40, Construct a target detection model and process the labeled data I. ann Train the model to obtain a well-trained object detection model.

[0028] In an exemplary embodiment, step S40 may specifically include the following steps: S410, Construct a deep learning-based object detection model; S420, for the training sample I train and validation sample I valid Perform Mosaic data augmentation training; S430, the training sample I after Mosaic data augmentation train and validation sample I valid Train the model to obtain a well-trained object detection model.

[0029] In the data preprocessing stage, Mosaic data augmentation is used to enrich the dataset. The original Mosaic data augmentation includes... Figure 4 As shown, four images are randomly cropped, scaled, and then stitched together onto a single mosaic image, with the remaining portion filled with gray borders. This algorithm modifies the Mosaic data, allowing up to nine images to be randomly cropped, scaled, and then stitched together into a single mosaic image, in the following stitching order: Figure 5 As shown, the improved Mosaic data augmentation has three advantages: First, it increases the batch training size; second, it minimizes the area of ​​gray border padding, as excessive gray borders can cause the model to learn irrelevant feature information, reducing the efficiency of model training; and third, because the scaled images are closer in size to the small targets, it expands the small target dataset, enhances the robustness of the model, and effectively improves the model's ability to detect small targets.

[0030] Furthermore, for training sample I train and validation sample I valid The training process includes the following key steps: Data collection: i.e., collecting training samples I train and validation sample I valid ; Data preprocessing: The collected data needs to be cleaned and organized, such as removing duplicate information, correcting spelling errors, filtering out irrelevant or low-quality data, and converting text into a format that the model can understand, such as converting text into numerical representation. Tokenization and tokenization: The language model will segment the input text into words, converting them into "word fragments" or "sub-words". This process is called tokenization. For example, "learning" can be broken down into "learn" and "practice", or processed into smaller units.

[0031] In addition, this embodiment uses Darknet53 as the backbone network and YOLOv5 model with a feature pyramid structure as the detection head, and modifies the feature pyramid structure of YOLOv5 by adopting the AF-FPN structure.

[0032] Specifically, YOLOv5 is a deep learning-based object detection model. Compared to YOLOv4, YOLOv5 has significantly improved in both object detection accuracy and speed, making it one of the most popular models in the current object detection field. Developed using PyTorch, YOLOv5 utilizes modules such as a backbone network, a detection head, and a loss function to achieve rapid detection and localization of multiple targets in an image.

[0033] Furthermore, AF-FPN is a neural network architecture for object detection and semantic segmentation tasks, designed to improve the performance of Feature Pyramid Network (FPN). FPN is a widely used method for handling multi-scale targets; it generates feature pyramids at different network layers, enabling the model to effectively handle targets of varying sizes.

[0034] This embodiment of AF-FPN introduces an Adaptive Attention Module (AAM) and a Feature Enhancement Module (FEM). The AAM reduces feature channels, minimizing the loss of contextual information in high-level feature maps. The FEM enhances the representation of the feature pyramid, improving inference speed while achieving state-of-the-art performance. The AF-FPN structure is as follows: Figure 7 As shown.

[0035] The input image is processed through multiple convolutional layers to generate feature maps {C1, C2, C3, C4, C5}. C5 is then processed through AAM to generate feature map M6. M6 is summed with M5 and fused with other features at lower levels via a top-down propagation approach. PANet shortens the information path between lower and higher level features by performing special fusion of P3, P4, and P5.

[0036] The operation of AAM can be divided into two steps: (1) Multiple contextual features at different scales are obtained through an adaptive average pooling layer. The pooling coefficient is [0.1, 0.5], which is adaptively varied according to the target size of the dataset.

[0037] (2) A spatial weight map is generated for each feature map through a spatial attention mechanism. The contextual features are fused through the weight map to generate a new feature map containing multi-scale contextual information.

[0038] The introduction of AF-FPN aims to further improve the performance of object detection and semantic segmentation models, especially in handling multi-scale and complex scenes. This adaptive feature selection mechanism helps the network better capture features at different scales in the image, thereby improving the model's performance.

[0039] Step S50, obtain the image to be detected I test .

[0040] In an exemplary embodiment, reference is made to Figure 8 Step S50 may include the following steps: S510, based on FFmpeg, perform frame extraction on the video stream from the monitoring camera at the access work site, extracting 1 frame every N frames as the detection image I. test .

[0041] In an exemplary embodiment, the following steps may be included before step S60: S520 inputs the test images belonging to the same test group into the trained target detection model to extract the target coordinates.

[0042] S60, the image to be detected I test The trained target detection model is input into the target detection network after image decoding and image preprocessing. Based on the detection results, the coordinates, categories, and confidence scores of personnel and rotating machinery are obtained.

[0043] Confidence level, in this context, represents the probability of making the correct choice under specific conditions and based on given data. It's a concept of "possibility" given by people based on scientific data when there is no definitive fact. Confidence level is usually expressed as a percentage within a certain range, such as 95%, or as a p-value. The p-value represents the degree of uncertainty regarding a fact; for example, p < 0.05 means there is no greater than 0.05 uncertainty, i.e., the probability of the fact occurring is 95%.

[0044] In an exemplary embodiment, step S60 may include the following steps: S610, the image to be detected I test The trained object detection model is input into the object detection network after image decoding and image preprocessing. S620 filters out redundant and duplicate boxes by using NMS (non-maximum suppression) to detect the results; S630, obtains the coordinates of personnel and rotating machinery based on the detection results. ,category and confidence level , where n is the number of people or rotating machinery detected in the image.

[0045] Specifically, in step S610, the rectangular box loss algorithm in the general algorithm is replaced with the EIOU loss algorithm based on the general YOLOv5 detection model.

[0046] Furthermore, YOLOv5's loss function includes bounding box loss (lossrect), confidence loss (lossobj), and classification loss (lossclc), with the bounding box loss employing CIOU Loss. While CIOU Loss considers the overlapping area, center point distance, and aspect ratio of the bounding boxes, the difference in aspect ratio reflected by 'v' in its formula, rather than the true difference between width and height and their confidence scores, can sometimes hinder the model's effective optimization of similarity. To address this issue, this patent uses EIOU as the IOU loss, replacing the bounding box loss in the YOLOv5 model.

[0047] The EIOU algorithm formula is as follows: (1) The loss function consists of three parts: overlap loss L IOU Center distance loss L dis Width and height loss L asp The width-to-height loss directly minimizes the difference in height and width between the predicted bounding box and the ground truth bounding box, resulting in faster convergence and better localization results.

[0048] 1. Overlap Loss L IOU The calculation formula is as follows: (2) (3) Where IOU represents the intersection-union ratio of A and B, and A and B represent the bounding boxes (i.e., the polygons of the detected target bounding boxes) of the predicted and ground truth boxes, respectively, in the form of: [x1,y1,x2,y2]. Here, (x1,y1) represents the top-left corner of the rectangle, and (x2,y2) represents the top-right corner of the rectangle.

[0049] 2. Center distance loss L dis The calculation formula is as follows: (4) in, represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, where b represents the center point coordinates of the predicted bounding box. Represents the coordinates of the center point of the true bounding box. It is the diagonal distance encompassing the smallest bounding box. These represent the width and height of the minimum bounding box, respectively.

[0050] 3. Width and height loss L asp The calculation formula is as follows: (5) in, This represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. The distance between the center points of the predicted bounding box and the center point of the ground truth bounding box is represented by , w represents the width of the predicted bounding box, and h represents the length of the predicted bounding box. This represents the width of the actual bounding box. This indicates the length of the actual bounding box.

[0051] S70, Calculate the distance D between the person and the center point of the rotating machinery based on the detection results of a single image. cent Given the length W of the robotic arm of the rotating machinery, determine whether the person is within the rotation radius of the rotating machinery. If D cent If <= 1.5*W, then the personnel are within the rotation radius of the rotating machinery.

[0052] In an exemplary embodiment, reference is made to Figure 3 Step S70 may include the following steps: S710, determines whether there are people and rotating machinery in the detection results of a single image; S720, if it exists, calculate the area ratio of personnel to rotating machinery; S730, if the area of ​​personnel is 10% to 30% of the area of ​​rotating machinery, then it is determined that the personnel and rotating machinery are around the rotating machinery. S740, calculate the distance D between the operator and the center point of the rotating machinery. cent The length W of the robotic arm of rotating machinery; S750, if D cent If <= 1.5*W, then the personnel are within the rotation radius of the rotating machinery.

[0053] refer to Figure 6 During algorithm training, the input consists of sample images labeled "person" and "robotic arm," with the person already within the robotic arm's rotation range. After MOSC enhancement of the algorithm images, feature pyramid enhancement is applied to the "robotic arm" element. A loss function is calculated based on the distance between the two elements. Following this feature enhancement, the algorithm iteratively learns using the general YOLO V5 darknet53 backbone network until it reaches the preset accuracy.

[0054] When the algorithm is used, the improved algorithm is used as the target detector. If personnel and rotating machinery are detected, and the area threshold between personnel and machinery is between 10% and 30%, and the distance between the center point of the person and the rotating machinery and the length W of the mechanical arm of the rotating machinery are less than or equal to 1.5*W, then it can be determined that the person is within the rotation radius of the rotating machinery.

[0055] In one or more embodiments of this method, labeled data is obtained by manually annotating the images containing the mechanical operation scene; the images to be detected are input into a trained target detection model, and then input into the target detection network after image decoding and image preprocessing; the coordinates, categories, and confidence scores of personnel and rotating machinery are obtained based on the detection results, which improves the image quality and greatly reduces the detection errors caused by changes in illumination, target size, and posture; the distance between the center point of the personnel and the rotating machinery and the length of the mechanical arm of the rotating machinery are calculated by the detection results of a single image, which can determine in real time whether the personnel are within the rotation radius of the rotating machinery.

[0056] Next, the method of the above embodiment was verified using videos collected on-site. Three test scenarios were selected to obtain sufficient images for testing. Under the conditions of an Intel Core i7 CPU, 4GB of RAM, and an NVIDIA GeForce 2080Ti dedicated graphics card, a dataset of construction site scenes collected by the user was used for training and testing. During the training process of this model, the object detection model needed to be retrained. Finally, sufficient images were obtained for testing in 1-3 test scenarios. The specific results are shown in Table 1.

[0057] Table 1

[0058] Based on the above embodiments, refer to Figure 2 Another embodiment of this application also provides a detection device for personnel entering the rotation radius of a machine. The detection device 200 for personnel entering the rotation radius of a machine may include the following modules: Video acquisition module 210 is used to acquire monitoring video V of mechanical operation based on the monitoring camera connected to the work site. i ; Image filtering module 220, used to filter images from the surveillance video V i Images containing mechanical operation scenes were selected from the images. i ; The manual annotation module 230 is used to annotate the image I containing the mechanical operation scene. i Perform manual annotation to obtain annotation data I ann and the labeled data I ann Divide; The model building and training module 240 is used to build an object detection model and train the labeled data I. ann Train the model to obtain a well-trained object detection model. Image acquisition module 250 is used to acquire the image to be detected, I. test ; Detection module 260 is used to detect the image I to be detected. test The trained target detection model is input into the target detection network after image decoding and image preprocessing. Based on the detection results, the coordinates, categories, and confidence scores of personnel and rotating machinery are obtained. Calculation module 270 is used to calculate the distance D between the person and the center point of the rotating machinery based on the detection results of a single image. cent Given the length W of the robotic arm of the rotating machinery, determine whether the person is within the rotation radius of the rotating machinery. If D cent If the value is less than or equal to 1.5 * W, then the personnel are within the rotation radius of the rotating machinery.

[0059] Based on the above embodiments, this application also provides a computer-readable storage medium, see reference. Figure 9 The computer-readable storage medium shown is an optical disc 90, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it implements the steps described in the above method implementation, for example, acquiring monitoring video V of mechanical operation based on a monitoring camera connected to the work site. i From the surveillance video V i Images containing mechanical operation scenes were selected from the images. i ; For the image I containing the mechanical operation scene i Perform manual annotation to obtain annotation data I ann and the labeled data I ann The data is divided into segments; a target detection model is constructed, and the labeled data I is processed. ann Train the object detection model to obtain a trained object detection model; obtain the image to be detected (I). test； The image to be detected I test The trained object detection model is input into the object detection network after image decoding and preprocessing. Based on the detection results, the coordinates, categories, and confidence scores of personnel and rotating machinery are obtained. The distance D between the center point of the personnel and the center point of the rotating machinery is calculated based on the detection results of a single image. cent Given the length W of the robotic arm of the rotating machinery, determine whether the person is within the rotation radius of the rotating machinery. If D cent If the radius is less than or equal to 1.5 * W, then the personnel are within the rotation radius of the rotating machinery. The specific implementation methods of each step will not be repeated here.

[0060] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0061] Furthermore, based on the above embodiments, this application also provides a computing device. Figure 10 A block diagram is shown of an exemplary computing device 60 suitable for implementing embodiments of the present application. The computing device 60 may be a computer system or a server. Figure 10 The computing device 60 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0062] like Figure 10 As shown, the components of computing device 60 may include, but are not limited to: one or more processors or processing units 601, system memory 602, and bus 603 connecting different system components (including system memory 602 and processing unit 601).

[0063] The computing device 60 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computing device 60, including volatile and non-volatile media, removable and non-removable media.

[0064] System memory 602 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 6021 and / or cache memory 6022. Computing device 60 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, ROM 6023 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 10 (Not shown in the image, usually referred to as "hard drive"). Although not shown in... Figure 10 The diagram illustrates that a disk drive for reading and writing to removable non-volatile disks (e.g., "floppy disks") and an optical disk drive for reading and writing to removable non-volatile optical disks (e.g., CD-ROMs, DVD-ROMs, or other optical media) can be provided. In these cases, each drive can be connected to a bus 603 connecting different system components via one or more data media interfaces. The system memory 602 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0065] A program / utility 6025 having a set (at least one) of program modules 6024 may be stored, for example, in system memory 602, and such program modules 6024 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment. Program modules 6024 typically perform the functions and / or methods described in the embodiments of this application.

[0066] The computing device 60 can also communicate with one or more external devices 604 (such as a keyboard, pointing device, display, etc.). This communication can be performed via input / output (I / O) interface 605. Furthermore, the computing device 60 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 606. Figure 10 As shown, network adapter 606 communicates with other modules of computing device 60 (such as processing unit 601, etc.) via bus 603, which connects different system components. It should be understood that, although... Figure 10 Other hardware and / or software modules may be used in conjunction with computing device 60, as not shown in the diagram.

[0067] Processing unit 601 executes various functional applications and data processing by running programs stored in system memory 602, such as acquiring monitoring video V of mechanical operations based on monitoring cameras connected to the work site. i From the surveillance video V i Images containing mechanical operation scenes were selected from the images. i ; For the image I containing the mechanical operation scene i Perform manual annotation to obtain annotation data I ann and the labeled data I ann The data is divided into segments; a target detection model is constructed, and the labeled data I is processed. ann Train the object detection model to obtain a trained object detection model; obtain the image to be detected (I). test； The image to be detected I test The trained object detection model is input into the object detection network after image decoding and preprocessing. Based on the detection results, the coordinates, categories, and confidence scores of personnel and rotating machinery are obtained. The distance D between the center point of the personnel and the center point of the rotating machinery is calculated based on the detection results of a single image. cent Given the length W of the robotic arm of the rotating machinery, determine whether the person is within the rotation radius of the rotating machinery. If D centIf the radius is less than or equal to 1.5 * W, then the person is within the rotation radius of the rotating machinery. The specific implementation methods of each step will not be repeated here. It should be noted that although several units / modules or sub-units / sub-modules of the detection device for detecting personnel entering the machinery's rotation radius are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0068] In the description of this application, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0069] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0070] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0071] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0072] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0073] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

[0075] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

Claims

1. A method for detecting the radius of rotation of a machine when personnel enter it, characterized in that, The method includes the following steps: S10, acquire monitoring video of mechanical operation based on the monitoring cameras accessing the work site. i ; S20, from the monitoring video V i Images containing mechanical operation scenes were selected from the data. i ; S30, the image I containing the mechanical operation scene... i Perform manual annotation to obtain annotation data I ann and the labeled data I ann Divide; S40, Construct a target detection model and process the labeled data I. ann Train the model to obtain a well-trained object detection model. S50, Obtain the image to be detected (I) test ; S60, the image to be detected I test The trained target detection model is input into the target detection network after image decoding and image preprocessing. Based on the detection results, the coordinates, categories, and confidence scores of personnel and rotating machinery are obtained. S70, Calculate the distance D between the person and the center point of the rotating machinery based on the detection results of a single image. cent Given the length W of the robotic arm of the rotating machinery, determine whether the person is within the rotation radius of the rotating machinery. If D cent If <= 1.5*W, then the personnel are within the rotation radius of the rotating machinery.

2. The method for detecting the radius of rotation of a machine when a person enters, as described in claim 1, is characterized in that... Step S30 includes: S310, the image I containing the mechanical operation scene... i Perform manual annotation to obtain annotation data I ann ; S320, the labeled data I ann Divided into training samples I train and validation sample I valid .

3. The method for detecting the radius of rotation of a machine when a person enters, as described in claim 2, is characterized in that... Step S40 includes: S410, constructing a deep learning-based object detection model; S420, for the training sample I train and validation sample I valid Perform Mosaic data augmentation training; S430, the training sample I after Mosaic data augmentation train and validation sample I valid Train the model to obtain a well-trained object detection model.

4. The method for detecting the radius of rotation of a machine when a person enters, as described in claim 1, is characterized in that... Step S50 includes: S510, based on FFmpeg, perform frame extraction on the video stream from the monitoring camera at the access work site, extracting 1 frame every N frames as the detection image I. test .

5. The method for detecting the radius of rotation of a machine when a person enters, as described in claim 1, is characterized in that... Before step S60, the method further includes: S520 inputs the test images belonging to the same test group into the trained target detection model to extract the target coordinates.

6. The method for detecting the radius of rotation of a machine when a person enters, as described in claim 1, is characterized in that... Step S60 includes: S610, the image to be detected I test The trained object detection model is input into the object detection network after image decoding and image preprocessing. S620 filters out redundant and duplicate boxes by using NMS (non-maximum suppression) to detect the results; S630, obtains the coordinates of personnel and rotating machinery based on the detection results. ,category and confidence level , where n is the number of people or rotating machinery detected in the image.

7. The method for detecting the radius of rotation of a machine when a person enters, as described in claim 1, is characterized in that... Step S70 includes: S710, determines whether there are people and rotating machinery in the detection results of a single image; S720, if it exists, calculate the area ratio of personnel to rotating machinery; S730, if the area of ​​personnel is 10% to 30% of the area of ​​rotating machinery, then it is determined that the personnel and rotating machinery are around the rotating machinery. S740, calculate the distance D between the operator and the center point of the rotating machinery. cent The length W of the robotic arm of rotating machinery; S750, if D cent If <= 1.5*W, then the personnel are within the rotation radius of the rotating machinery.

8. A detection device for the radius of rotation of a machine when personnel enter, characterized in that, include: The video acquisition module is used to acquire monitoring video of mechanical operations based on the surveillance cameras connected to the work site. i ; Image filtering module, used to filter images from the surveillance video V i Images containing mechanical operation scenes were selected from the data. i ; The manual annotation module is used to annotate the image I containing the mechanical operation scene. i Perform manual annotation to obtain annotation data I ann and the labeled data I ann Divide; The model building and training module is used to build an object detection model and train the labeled data I. ann Train the model to obtain a well-trained object detection model. The image acquisition module is used to acquire the image to be detected (I). test ; The detection module is used to detect the image I. test The trained target detection model is input into the target detection network after image decoding and image preprocessing. Based on the detection results, the coordinates, categories, and confidence scores of personnel and rotating machinery are obtained. The calculation module is used to calculate the distance D between the person and the center point of the rotating machinery based on the detection results of a single image. cent Given the length W of the robotic arm of the rotating machinery, determine whether the person is within the rotation radius of the rotating machinery. If D cent If <= 1.5*W, then the personnel are within the rotation radius of the rotating machinery.

9. A computer-readable storage medium, characterized in that, It includes instructions that, when executed on a computer, cause the computer to perform the detection method for detecting personnel entering the radius of rotation of a machine as described in any one of claims 1 to 7.

10. A computing device, characterized in that, The computing device includes: At least one processor, memory, and input / output unit; The memory is used to store computer programs, and the processor is used to call the computer programs stored in the memory to execute the method for detecting the radius of rotation of a person entering a machine as described in any one of claims 1 to 7.

Images