Rolling equipment control method and device based on multi-target detection

Abstract

The invention provides a rolling equipment control method and device based on multi-target detection, and relates to the technical field of target detection, and the method comprises the steps: obtaining image data collected by camera equipment for rolling equipment; multi-target detection is carried out on the image data through a pre-trained target detection model to obtain a target detection result corresponding to the image data, and the target detection result comprises one or more of target edge contours corresponding to the living body of the operator, the hand region of the operator and the region of gloves worn by the operator; determining a current state instance corresponding to the target detection result, wherein the current state instance represents a physical monitoring area in which a target edge contour contained in the target detection result falls; and determining a target control strategy according to the equipment state of the rolling equipment and the current state instance so as to execute a control action corresponding to the target control strategy on the rolling equipment. The method can effectively prevent and intervene high-risk conditions in the operation of the rolling equipment in real time.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a control method and apparatus for a roller pressing equipment based on multi-target detection. Background Technology

[0002] Roll forming equipment, a key molding device in the processing of polymer materials such as rubber and plastics, works by applying pressure to materials through one or more pairs of counter-rotating rollers, stretching them into sheets of the required thickness and width. In industrial production, roll forming equipment typically features large roller diameters, high linear speeds, and powerful extrusion pressures. The operation involves several high-risk areas, including the roller gap interlocking zone, exposed drive chain / gear areas, loading and unloading stations, and equipment maintenance passages. To ensure operator safety, current technologies generally employ a safety protection system combining human and technological safeguards.

[0003] Regarding human safety, workers must receive regular safety training, which covers the operating procedures of the roller pressing equipment (including standard procedures for starting, stopping, adjusting the distance, and cleaning the rollers), daily inspection and basic maintenance skills, the proper wearing requirements for personal protective equipment (such as cut-resistant gloves, impact-resistant shoes, and goggles), on-site safety sign recognition, emergency response procedures (such as handling mechanical injuries and first aid for high-temperature burns), and warning education on typical accident cases. At the same time, the company strengthens personnel's safety awareness and behavioral constraints by establishing management mechanisms such as pre-shift safety confirmation, two-person mutual protection, and incentives for reporting hidden dangers.

[0004] In terms of technical protection, the current mainstream configuration includes two types of physical protection facilities: pull rope emergency stop device and through-beam light grid. The pull rope is laid along the entire length of the dangerous area on both sides of the roller pressing equipment and is connected to the emergency stop switch by a high-strength steel wire rope. The operator can pull it at any position to cut off the main power supply and brake the roller. The through-beam light grid is installed in front of the roller gap, at the feed inlet and at the main passage. It consists of an infrared transmitter and a receiver to form an invisible light curtain. When a person or limb enters the set protective surface, it will trigger the interlock to stop the machine.

[0005] However, in actual operation, human-based safety measures are easily affected by factors such as insufficient ambient light, high temperature and humidity, continuous noise interference, personnel fatigue, emotional fluctuations, and individual distraction, leading to operational negligence or delayed response. When multiple people are working together, poor communication or unclear responsibilities may also lead to misoperation. In terms of technical safety measures, the pull rope can only be passively triggered when danger has occurred or is imminent, relying on subjective judgment and immediate action of personnel, making it difficult to achieve pre-emptive prevention. Although the through-beam grating has real-time intrusion detection capabilities, in the rubber calendering process, due to the high viscosity of the rubber material, its tendency to splash, and its easy adhesion of dust, the beam is often mistakenly blocked, causing unplanned shutdowns. Moreover, its protection range is limited to the set plane and cannot cover three-dimensional spaces such as above the equipment, at the ends of the rollers, and below the maintenance platform. More importantly, when the operator is outside the grating protection range (such as during high-altitude maintenance) and other personnel mistakenly issue a start command, the system cannot recognize this abnormal human-machine coexistence state, posing a serious risk of loss of control. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a control method and device for roller pressing equipment based on multi-target detection, which can effectively prevent and intervene in high-risk situations in the operation of roller pressing equipment in real time.

[0007] In a first aspect, the present invention provides a control method for a roller pressing equipment based on multi-target detection, wherein at least two camera devices are deployed at a designated location in the area where the roller pressing equipment is located, and each camera device has a pre-defined physical monitoring area within its field of view. The method includes: Acquire image data captured by the camera equipment on the roller pressing equipment; By using a pre-trained target detection model, multi-target detection is performed on the image data to obtain the target detection results corresponding to the image data. The target detection results include one or more of the target edge contours corresponding to the worker's living body, the target edge contours corresponding to the worker's hand area, and the target edge contours corresponding to the glove area worn by the worker. Determine the current state instance corresponding to the target detection result. The current state instance represents the physical monitoring area into which the target edge contour contained in the target detection result falls. The target control strategy is determined based on the equipment status and current status instance of the roller pressing equipment, so as to execute the control action corresponding to the target control strategy on the roller pressing equipment.

[0008] In one implementation, the target detection model employs a neural network model based on the YOLOv5 framework. The target detection model includes a backbone network, a feature fusion network, and a multi-target detection head. Through a pre-trained target detection model, multi-target detection is performed on the image data to obtain the target detection results corresponding to the image data, including: The basic features corresponding to the image data are extracted through the backbone network; The feature fusion network is used to fuse basic features to obtain a multi-size fused feature map that matches the multi-target detection head, and the multi-size fused feature map is filtered by background features to obtain a multi-size target feature map. Using a multi-target detection head, target detection is performed on multi-size target feature maps based on their associated static liveness anchor boxes, dynamic hand anchor boxes, and dynamic glove anchor boxes to obtain the initial detection results corresponding to the image data. The OpenPose model is called to extract key points corresponding to the image data, and the initial detection results are corrected using the key points to obtain the target detection results corresponding to the image data.

[0009] In one embodiment, the multi-target detection head includes a liveness detection head, a hand detection head, and a glove detection head; The liveness detection head is associated with a static liveness anchor frame; The hand detection head includes multiple hand sub-detection heads of different sizes, and the multiple hand sub-detection heads of different sizes are associated with at least one dynamic hand anchor frame; The glove detection head includes multiple glove detection heads of different sizes, and the multiple glove detection heads of different sizes are associated with at least one dynamic glove anchor frame.

[0010] In one implementation, a multi-target detection head performs target detection on multi-size target feature maps based on its associated static liveness anchor frame, dynamic hand anchor frame, and dynamic glove anchor frame to obtain initial detection results corresponding to the image data, including: Using a liveness detection head, target detection is performed on multi-size target feature maps based on its associated static liveness anchor frame to obtain the initial edge contour corresponding to the live worker. The hand detection head includes multiple hand sub-detection heads, which perform target detection on multi-size target feature maps based on their respective associated dynamic hand anchor boxes to obtain the initial edge contour corresponding to the worker's hand area. The glove detection head includes multiple glove detection heads, which perform target detection on multi-size target feature maps based on their respective associated dynamic glove anchor frames to obtain the initial edge contour corresponding to the glove area of ​​the worker.

[0011] In one implementation, a hand detection head includes multiple hand sub-detection heads, which perform target detection on multi-size target feature maps based on their respective associated dynamic hand anchor frames to obtain the initial edge contour corresponding to the worker's hand region, including: Feature extraction is performed on the image data to obtain the region of interest and its size range corresponding to the worker's hand area; The following operations are performed by each hand sub-detector contained in the hand detection head: Determine the region of interest corresponding to the worker's hand area, and the intersection-union ratio between it and multiple dynamic hand anchor frames associated with the hand sub-detection head; Based on the cross-union ratio and the cross-union ratio threshold corresponding to the size range, determine whether the region of interest corresponding to the worker's hand area matches the dynamic hand anchor frame; Under the condition of matching, the initial edge contour corresponding to the area of ​​the worker's hand is obtained.

[0012] In one implementation, the OpenPose model is invoked to extract key points corresponding to the image data, and the initial detection results are corrected using these key points, including: Determine whether the initial detection result contains the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area; If so, the OpenPose model is invoked, and key points are extracted from the arm sub-region output by the target detection model, using the center coordinates of the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area as the reference. The key points are then used to correct the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area.

[0013] In one implementation, the method further includes: Obtain the labeled image data corresponding to the roller pressing equipment. The labeled image data includes the live operator, the operator's hand area, and the area of ​​the glove worn by the operator. From the labeled image data, extract the sub-images corresponding to the worker's hand area and the area of ​​the glove worn by the worker: For sub-images corresponding to regions of the same type, multiple target sizes are randomly determined from the sizes corresponding to the sub-images of that type of region; Clustering is performed on the sub-images corresponding to the target size and the sub-images corresponding to other sizes to obtain multiple target clusters; Determine the matching degree between each size contained in the target cluster and the size of the multi-target detection head. Based on the matching degree, the sub-images corresponding to each size contained in the target cluster are used as dynamic anchor boxes associated with the multi-target detection head. The dynamic anchor boxes are dynamic hand anchor boxes or dynamic glove anchor boxes.

[0014] In one implementation, clustering is performed based on clustering parameters, as well as the Euclidean distance and cross-union ratio distance between the sub-images corresponding to the target size and the sub-images corresponding to other sizes, to obtain multiple clusters. Determine the weighted distance variance value for each cluster; If the weighted distance variance is greater than a preset threshold, the clustering parameters are adaptively adjusted. Based on the adaptively adjusted clustering parameters, as well as the Euclidean distance and cross-union distance between the sub-images corresponding to the target size and the sub-images corresponding to other sizes, the clusters are re-divided until the preset clustering stopping condition is met, resulting in multiple target clusters.

[0015] In one implementation, a target control strategy is determined based on the equipment status and current state instance of the roller pressing equipment, including: Determine whether the current status instance matches the warning / alarm status instance associated with the equipment status of the roller press; If so, the control strategy corresponding to the matched warning / alarm status instance will be used as the target control strategy for the roller pressing equipment.

[0016] Secondly, the present invention also provides a control device for a roller pressing equipment based on multi-target detection, wherein at least two camera devices are deployed at a designated location in the area where the roller pressing equipment is located, and each camera device has a pre-defined physical monitoring area within its field of view. The device includes: The image acquisition module is used to acquire image data collected by the camera device from the roller pressing device; The multi-target detection module is used to perform multi-target detection on image data through a pre-trained target detection model to obtain the target detection results corresponding to the image data. The target detection results include one or more of the target edge contours corresponding to the worker's living body, the target edge contours corresponding to the worker's hand area, and the target edge contours corresponding to the glove area worn by the worker. The state instance determination module is used to determine the current state instance corresponding to the target detection result. The current state instance represents the physical monitoring area into which the target edge contour contained in the target detection result falls. The control strategy determination module is used to determine the target control strategy based on the equipment status and current status instance of the roller pressing equipment, so as to execute the control action corresponding to the target control strategy on the roller pressing equipment.

[0017] Thirdly, the present invention also provides an electronic device including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement any of the methods provided in the first aspect.

[0018] Fourthly, the present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement any of the methods provided in the first aspect.

[0019] This invention provides a control method and apparatus for a roller pressing machine based on multi-target detection. At least two camera devices are deployed at designated locations within the area where the roller pressing machine is located. Each camera device has a pre-defined physical monitoring area within its field of view. First, image data collected by the camera devices targeting the roller pressing machine is acquired. Then, a pre-trained target detection model is used to perform multi-target detection on the image data to obtain target detection results. These results include one or more of the following: the target edge contour corresponding to the worker's body, the target edge contour corresponding to the worker's hand area, and the target edge contour corresponding to the area of ​​the glove worn by the worker. Next, a current state instance corresponding to the target detection results is determined. This current state instance represents the physical monitoring area into which the target edge contours included in the target detection results fall. Finally, a target control strategy is determined based on the equipment state of the roller pressing machine and the current state instance, and the control actions corresponding to the target control strategy are executed on the roller pressing machine. This method, by deploying multiple camera devices in the area of ​​the roller pressing machine, defining a physical monitoring area within its field of view, and using a pre-trained target detection model to perform multi-target detection on the image data, identifies the positions of the worker's body, hands, and gloves. Based on the detection results, the current state instance is determined. Combined with the equipment status of the roller pressing equipment, the corresponding control strategy is intelligently generated and executed. This invention realizes real-time monitoring and risk warning for operators. It can automatically take control measures before potential dangers occur, significantly improve the safety of roller pressing equipment operation, reduce the risks caused by human negligence and misoperation, and ensure personnel safety in the production process.

[0020] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a control method for a roller pressing equipment based on multi-target detection, provided in an embodiment of the present invention; Figure 2A schematic diagram of a system architecture for implementing a multi-target detection-based control method for a roller press is provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the deployment of a camera device provided in an embodiment of the present invention; Figure 4 This is a technical framework diagram of a method for controlling a roller pressing device based on multi-target detection, provided by an embodiment of the present invention. Figure 5 A schematic diagram of a control device for a roller pressing equipment based on multi-target detection is provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Currently, existing safety protection systems for roller pressing equipment have many limitations in terms of human and technical safeguards. For example, environmental factors affect the operator's response capabilities, pull-rope emergency stop devices rely on subjective judgment, and through-beam gratings are easily obstructed and have limited protection range, making it impossible to effectively prevent and respond to high-risk situations in real time. Therefore, this invention provides a roller pressing equipment control method and device based on multi-target detection, which can effectively prevent and intervene in high-risk situations during roller pressing equipment operation in real time.

[0026] To facilitate understanding of this embodiment, a detailed description of a multi-target detection-based control method for a rolling mill disclosed in this embodiment of the invention will be provided first. A calender is a type of rolling mill. At least two camera devices are deployed at designated locations within the area where the rolling mill is located. Each camera device has a pre-defined physical monitoring area within its field of view, including a safety zone, a warning zone, and an alarm zone. (See [link to relevant documentation]). Figure 1 The diagram shows a flow chart of a control method for a roller pressing equipment based on multi-target detection. The method mainly includes the following steps S102 to S108: Step S102: Obtain image data collected by the camera device from the roller pressing device.

[0027] Step S104: Perform multi-target detection on the image data using a pre-trained target detection model to obtain the target detection results corresponding to the image data.

[0028] The target detection model can employ a neural network model based on the YOLOv5 framework. The target detection results include one or more of the following: the target edge contour corresponding to the live worker, the target edge contour corresponding to the worker's hand area, and the target edge contour corresponding to the area of ​​the glove worn by the worker. In one example, the target detection model includes a backbone network, a feature fusion network, and a multi-target detection head. These components sequentially perform feature extraction, feature fusion, background filtering, and target detection on the input image data to obtain the target edge contours corresponding to the live worker, the worker's hand area, and the area of ​​the glove worn by the worker.

[0029] Step S106: Determine the current state instance corresponding to the target detection result.

[0030] Here, the current state instance represents the physical monitoring area into which the target edge contour contained in the target detection result falls. In one example, the pixel coordinates of the target edge contour are mapped to a three-dimensional Cartesian coordinate system with the camera optical center as the origin by a geometric back-projection transformation driven by calibration parameters; further, the preset physical monitoring area (including safe area, warning area and alarm area) is modeled as a three-dimensional spatial polyhedron with depth constraints in the camera coordinate system (e.g., defined by the vertex set of the XY plane polygon and the effective range of the Z axis); finally, based on the relative positional relationship between the minimum enclosing convex hull spanned by the target edge contour in the camera coordinate system and the three-dimensional spatial polyhedron, it is determined whether the target edge contour is in a no-area, safe area, warning area or alarm area.

[0031] Step S108: Determine the target control strategy based on the equipment status and current status instance of the roller pressing equipment, and execute the control action corresponding to the target control strategy on the roller pressing equipment.

[0032] The equipment status includes a standby state and an operating state. The target control strategy is used to control the audible and visual alarms and the start and stop of the roller pressing equipment. In one embodiment, if the current state instance matches the warning / alarm state instance associated with the equipment status of the roller pressing equipment, the control strategy corresponding to the matched warning / alarm state instance is used as the target control strategy for the roller pressing equipment, and then the control action corresponding to the target control strategy is executed on the roller pressing equipment.

[0033] The multi-target detection-based control method for roller pressing equipment provided in this invention deploys multiple cameras in the roller pressing equipment area and delineates a physical monitoring area within the field of view. A pre-trained target detection model is used to perform multi-target detection on the image data, identifying the location of the worker's body, hands, and gloves. Based on the detection results, the current state instance is determined. Combined with the equipment status of the roller pressing equipment, a corresponding control strategy is intelligently generated and executed. This invention achieves real-time monitoring and risk warning for workers, automatically taking control measures before potential dangers occur. This significantly improves the safety of roller pressing equipment operation, reduces the risks caused by human negligence and misoperation, and ensures personnel safety during the production process.

[0034] To facilitate understanding, the embodiments of the present invention first explain the system architecture for implementing the multi-target detection-based control method for roller pressing equipment, see [link to relevant documentation]. Figure 2 The diagram illustrates a system architecture for implementing a multi-target detection-based control method for roller pressing equipment. It includes a field PLC control system, a main controller, a host computer, a switch, and a camera. The main controller is communicatively connected to the PLC control system, the host computer, and the switch. The switch is communicatively connected to the camera. The host computer is configured with AI algorithms (i.e., a neural network model based on the YOLOv5 framework and an OpenPose model). See also... Figure 3 The diagram illustrates the deployment of a camera system. By installing one camera on each side of the roller pressing equipment, omnidirectional, blind-spot-free visual coverage is achieved, effectively avoiding obstruction of view caused by equipment and raw materials. Preferably, each camera's field of view is pre-defined with a safety zone, a warning zone, and an alarm zone.

[0035] In one implementation, the main control unit acquires image data from the camera device via a switch and uploads it to the host computer. The host computer executes a multi-target detection-based control method for the roller pressing equipment to obtain a target control strategy and feeds it back to the main control unit. The main control unit then sends control commands to the field PLC control system, enabling the field PLC control system to control the start and stop of the roller pressing equipment. See also... Figure 4 The diagram illustrates a technical framework for a multi-target detection-based control method for roller pressing equipment, including processes such as image acquisition, AI recognition, region determination, command control of the on-site PLC control system, and system recording. Furthermore... Figure 4 It also shows that the on-site PLC control system has functions such as feedback of equipment status, equipment reverse control, and audible and visual alarms.

[0036] Based on this, embodiments of the present invention provide a specific implementation of a host computer-based control method for a roller pressing device based on multi-target detection, including: (a) Acquire image data collected by camera equipment from the roller pressing equipment.

[0037] (ii) Multi-object detection is performed on the image data using a pre-trained object detection model to obtain the object detection results corresponding to the image data. The object detection model adopts a neural network model based on the YOLOv5 framework.

[0038] Considering that operators need to manually push the rubber for orderly calendering during the roller rotation process, this can easily cause the rubber to stick to the operator's hands and pull limbs into the equipment, thus leading to safety accidents. Simply relying on liveness detection cannot provide effective protection. Therefore, this embodiment of the invention proposes to detect the operator's live body, hands, and the gloves they are wearing.

[0039] In its specific implementation, the target detection model based on the YOLOv5 framework includes a backbone network, a feature fusion network, and a multi-target detection head. The multi-target detection head includes a liveness detection head, a hand detection head, and a glove detection head. The liveness detection head is associated with a static liveness anchor box. The hand detection head includes multiple hand sub-detectors of different sizes (such as ultra-small, small, medium, and large sizes), and each of these multiple hand sub-detectors is associated with at least one dynamic hand anchor box. The glove detection head includes multiple glove sub-detectors of different sizes (such as ultra-small, small, medium, and large sizes), and each of these multiple glove sub-detectors is associated with at least one dynamic glove anchor box.

[0040] The present invention first explains the process of generating the anchor frame.

[0041] (1) Static live anchor frame: For the anchor frame used by the live operator, the embodiment of the present invention adopts the traditional static anchor frame, and the specific details will not be repeated here.

[0042] (2) Dynamic hand anchor frame: For the anchor frame used in the area where the worker wears gloves, the embodiments of the present invention adopt a dynamic anchor frame based on clustering and adaptive adjustment, as detailed in (2.1) to (2.7) below: (2.1) Obtain the labeled image data corresponding to the roller pressing equipment. The labeled image data includes the live body of the operator, the area of ​​the operator's hands, and the area of ​​the gloves worn by the operator.

[0043] (2.2) Extract the sub-images corresponding to the worker's hand area and the glove area worn by the worker from the labeled image data.

[0044] (2.3) For sub-images corresponding to regions of the same type, multiple target sizes are randomly determined from the sizes corresponding to the sub-images corresponding to that type of region. Here, sub-images corresponding to regions of the same type refer to: all sub-images corresponding to the worker's hand region, or all sub-images corresponding to the glove region worn by the worker. In this embodiment of the invention, the worker's hand region is used as an example, and multiple target sizes are randomly determined from the sizes corresponding to all sub-images corresponding to the worker's hand region.

[0045] (2.4) Based on the clustering parameters, as well as the Euclidean distance and cross-union distance between the sub-images corresponding to the target size and the sub-images corresponding to other sizes, clustering is performed to obtain multiple clusters.

[0046] Continuing with the example of the worker's hand area, it includes: Initialization parameters: Based on the hand size distribution of the four hand sub-detectors, core clustering parameters are preset, and no new parameters are needed. Specific parameters are as follows: Number of clusters K: 9; Adaptive step size: initial step size set to 1.0; Adaptive distance threshold: initial set to 15.0 pixels; Intra-cluster variance threshold: initial set to 8.0; Clustering stopping condition: In 5 consecutive iterations, the change in the center point of all clusters is ≤1 pixel, or the number of iterations is ≥100.

[0047] Initial clustering: Nine hand sub-images of different sizes are randomly selected from the sub-images and used as the centroids of nine initial clusters. The Euclidean distance and Cross-Union Ratio (CIRR) distance between each sub-image to be clustered and the nine centroids are calculated. A weighted distance is obtained by weighting the sum of the Euclidean and CIRR distances, and the sub-image is assigned to the cluster with the smallest weighted distance. This process completes the initial clustering, resulting in nine initial clusters, each containing multiple highly similar sub-images.

[0048] Euclidean distance is a pixel-level distance used to measure the similarity of pixel distribution in sub-images and can be directly calculated using the Euclidean distance formula. Intersection over Union (IOU) distance is a region-level distance used to measure the similarity of sub-image contours. The determination process for IOU distance is as follows: First, calculate the IOU, which measures the degree of overlap between the effective regions of two sub-images by dividing the number of pixels in the intersection of their effective regions by the number of pixels in the union. Then, subtract the IOU from 1 to obtain the IOU distance. In other words, the IOU distance equals 1 minus the ratio of the intersection to the union of the effective regions of two sub-images. A smaller IOU distance indicates a higher degree of overlap in the effective regions (hand contours) of the two sub-images, meaning a higher similarity in gestures and dimensions; a larger IOU distance indicates a lower degree of overlap in the effective regions of the two sub-images, meaning a greater difference in gestures and dimensions.

[0049] (2.5) Determine the weighted distance variance value corresponding to each cluster.

[0050] For each initial cluster, the weighted distance variance among all sub-images within the cluster is calculated. The weighted distance variance is used to measure the dispersion of these sub-images in terms of features. (2.6) When the weighted distance variance is greater than the preset threshold, the clustering parameters are adaptively adjusted, and based on the adaptively adjusted clustering parameters, as well as the Euclidean distance and cross-union distance between the sub-images corresponding to the target size and the sub-images corresponding to other sizes, the clusters are re-divided until the preset clustering stopping condition is met, and multiple target clusters are obtained.

[0051] If the weighted distance variance of a cluster exceeds a preset threshold of 8.0, it means that the sub-images within that cluster are highly differentiated and may contain multiple different gesture types, such as both clenched fists and outstretched hands. In this case, we will mark such a cluster as a "cluster to be optimized" for further processing.

[0052] For clusters marked as "clusters to be optimized", the clustering parameters will be adjusted based on the weighted distance variance of the clusters: Iteration step size adjustment: If the weighted distance variance within a cluster is large (indicating large differences between sub-images), the iteration step size is increased to 1.2 times to speed up the update of cluster centers. Conversely, if the variance is small (indicating small differences between sub-images), the step size is reduced to 0.8 times to avoid redundancy within the cluster due to excessive iteration.

[0053] Cluster center point correction: Instead of simply taking the average pixel value of all sub-images within a cluster as the cluster center point, the sub-image with the smallest weighted distance within the cluster is selected as the new cluster center point. This ensures that the cluster center point is more representative and better reflects the gesture and size characteristics of the cluster. In addition, sub-images corresponding to frequently occurring hand gestures (such as clenched fists and half-clenched fists) are given priority as cluster center points.

[0054] Based on the adjusted clustering parameters and cluster centroids, the weighted distance between each sub-image and the nine cluster centroids is recalculated, and the cluster division is updated: Reclassification: Each hand sub-image is reassigned to the cluster with the smallest weighted distance; Cluster merging: If the weighted distance between the centroids of two clusters is less than the adaptive distance threshold (e.g., 15.0), it indicates that the sub-images in these two clusters are highly similar (similar gestures and sizes), so these two clusters are merged into one cluster; Cluster splitting: If the weighted distance variance within a cluster is consistently greater than the threshold (e.g., 8.0), and the number of sub-images contained in the cluster reaches or exceeds 100 (sufficient sample size), the cluster is split into two sub-clusters, and centroids are reselected to ensure that the sub-image similarity within each cluster meets the standard.

[0055] Finally, through the above adjustments and updates, the number of clusters was stabilized at K=9 to meet the division of labor requirements of the four sub-detection heads in the scheme. The signing process was repeated until the clustering stopping condition was met.

[0056] (2.7) Determine the matching degree between each size contained in the target cluster and the size of the multi-target detection head, so as to use the sub-images corresponding to each size contained in the target cluster as dynamic anchor boxes associated with the multi-target detection head based on the matching degree. The dynamic anchor boxes are dynamic hand anchor boxes or dynamic glove anchor boxes.

[0057] 1) Preliminary preparation: For all target clusters, extract the actual dimensions (width, height) of all sub-images within each cluster. At the same time, define the size range of multiple hand sub-detection heads. The ultra-small detector head is adapted to a size range of <64×64 pixels (core adaptation for small hands / small gloves), the small detector head is adapted to a size range of 64-128×64-128 pixels (adaptation for regular hands / gloves), the medium detector head is adapted to a size range of 128-256×128-256 pixels (adaptation for extended gestures / large gloves), and the large detector head is adapted to a size range of >256×256 pixels (backup, adaptation for extremely special large targets).

[0058] 2) Matching Degree Calculation: For each size within each target cluster, the matching degree is calculated with the size ranges of the four or more hand sub-detectors. The core metric for matching degree is the intersection-over-union ratio (IoU), which is calculated as follows: The "middle value" of the size range of the hand sub-detection head is used as the reference size of the hand sub-detection head (e.g., the reference size of the ultra-small detection head is 50×50, and the reference size of the small detection head is 96×96). Calculate the cross-union ratio (CURRY) between a single size within a cluster (e.g., 30×50 within a hand cluster) and the reference size of the hand sub-detector head (the higher the CURRY, the better the match). The supplementary dimensional deviation rate calculation involves calculating the absolute values ​​of the width and height deviations between the cluster dimensions and the reference dimensions of the hand sub-detection head, and then dividing them by the reference dimensions of the detection head. A smaller deviation rate indicates a higher degree of matching between the cluster dimensions and the reference dimensions of the detection head. The specific steps are as follows: Width Deviation: Calculate the absolute value of the difference between the width of the cluster dimensions and the width of the reference dimensions of the hand sub-detection head; Height Deviation: Calculate the absolute value of the difference between the height of the cluster dimensions and the height of the reference dimensions of the hand sub-detection head; Deviation Rate: Divide the width deviation and height deviation by the width and height of the reference dimensions of the hand sub-detection head, respectively, to obtain the width deviation rate and height deviation rate; Overall Deviation Rate: The average of the width deviation rate and height deviation rate can be taken as the final dimensional deviation rate.

[0059] The final matching degree is obtained by weighted summation of the crossover ratio and the dimensional deviation rate.

[0060] 3) Based on the final matching degree and the anchor frame matching threshold (for example, the anchor frame matching threshold for ultra-small and small sizes is 0.45, and the anchor frame matching threshold for medium sizes is 0.5), determine the clusters matched by the hand sub-detection head, and use the sub-images contained in the clusters as the dynamic hand anchor frames associated with the hand sub-detection head.

[0061] (3) Dynamic glove anchor frame: For the anchor frame used in the glove area worn by the operator, the embodiment of the present invention adopts a dynamic anchor frame based on clustering and adaptive adjustment. For details, please refer to the specific implementation method given in the section on dynamic hand anchor frames. The embodiment of the present invention will not elaborate further on this.

[0062] After determining the anchor boxes associated with each detection head, the steps for multi-object detection of the image data can be performed, including: Step 1: Extract basic features corresponding to the image data through the backbone network. In one implementation, the backbone network adopts the CSPDarknet architecture to perform multi-scale feature extraction on the input image data: progressive downsampling is achieved through cascaded Conv convolutional layers and CSP (Cross Stage Partial) modules, which enhances gradient flow and feature reuse while maintaining computational efficiency; finally, three layers of basic features with decreasing spatial resolution are output, where the high-resolution layer focuses on preserving the fine structure of the hand, providing strong discriminative feature support for subsequent small target detection and fine localization.

[0063] Step 2: The basic features are fused through a feature fusion network to obtain a multi-size fused feature map that matches the multi-target detection head, and the multi-size fused feature map is filtered by background features to obtain a multi-size target feature map.

[0064] The feature fusion network adopts the PANet architecture, which fuses the basic features output by the backbone network. Through upsampling and downsampling, it finally outputs four basic features at four different scales to meet the input requirements of four sub-detection heads.

[0065] The fused feature maps at four scales output from the Neck layer are fed into the background feature filtering layer. The similarity between each feature point and the rubber background feature is calculated. If the similarity exceeds a rubber background feature threshold (e.g., 80%), the feature point is suppressed (set to 0). If the similarity is less than 80%, the feature point is retained and considered a suspected target feature. The filtered target feature maps at four scales are output, retaining only features related to people, hands, and gloves, eliminating most of the rubber background interference, thereby reducing the probability of subsequent false detections.

[0066] Step 3: Using a multi-target detection head, based on its associated static liveness anchor boxes, dynamic hand anchor boxes, and dynamic glove anchor boxes, target detection is performed on the multi-size target feature maps to obtain the initial detection results corresponding to the image data. Specifically, this includes: Branch 1: Using a liveness detection head, target detection is performed on multi-size target feature maps based on its associated static liveness anchor frame to obtain the initial edge contour corresponding to the live worker.

[0067] Branch 2: Using multiple hand sub-detection heads included in the hand detection head, target detection is performed on multi-size target feature maps based on their respective associated dynamic hand anchor boxes to obtain the initial edge contour corresponding to the worker's hand region.

[0068] Specifically, this includes: extracting features from image data to obtain the region of interest (ROI) and its size range corresponding to the worker's hand area; performing the following operations through each hand sub-detector included in the hand detection head: determining the ROI corresponding to the worker's hand area and the cross-union ratio (CUP) between it and multiple dynamic hand anchor frames associated with that hand sub-detector; determining whether the ROI corresponding to the worker's hand area matches the dynamic hand anchor frames based on the CUP and the CUP threshold corresponding to the size range; and obtaining the initial edge contour corresponding to the worker's hand area if a match is found.

[0069] In one implementation, a dynamic anchor box matching strategy is employed, feeding the filtered multi-size target feature maps into four sub-detection heads. Each sub-detection head calls its corresponding dynamic hand anchor box to initially determine the size of the suspected target, then calculates the intersection-union ratio (IU) between the target feature map and the dynamic hand anchor box, adaptively matching the optimal dynamic hand anchor box. If the IU is greater than or equal to the corresponding size's IU threshold, a successful match is determined. Specifically, the ultra-small target detection head (160×160) is used to detect hands or gloves smaller than 64×64 pixels, calling anchor boxes (20×40, 30×50) and outputting a small target detection box. The small target detection head (80×80) is used to detect hands or gloves between 64 and 128 pixels, calling anchor boxes (30×50, 40×60) and outputting a small target detection box. The medium target detection head (40×40) is used to detect people (upper body) between 128 and 256 pixels, calling a general anchor box (adapted to human body size) and outputting a human body detection box. Large target detection head (20×40): Used to detect people (full body) larger than 256 pixels, serving as a backup detection head to ensure no human body is missed. For the initial edge contours output by each sub-detection head, initial edge contours with a confidence level greater than or equal to 0.5 are selected, and low-confidence initial edge contours are discarded to initially reduce the false detection rate.

[0070] Branch 3: Using multiple glove detection heads included in the glove detection head, target detection is performed on multi-size target feature maps based on their respective associated dynamic glove anchor frames to obtain the initial edge contour corresponding to the worker's glove region. The process of extracting the initial edge contour corresponding to the worker's glove region can be referred to the aforementioned Branch 2, and will not be elaborated further in this embodiment of the invention.

[0071] Step 4: The OpenPose model is invoked to extract key points corresponding to the image data. These key points are then used to correct the initial detection results, yielding the target detection results corresponding to the image data. Specifically, this includes: Determine whether the initial detection result contains the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area; If so, the OpenPose model is invoked. Using the center coordinates of the initial edge contour corresponding to the worker's hand region or the worker's glove region as a reference, key points are extracted from the arm sub-region output by the target detection model. These key points are then used to correct the initial edge contour corresponding to the worker's hand region or the worker's glove region. The specific implementation process is as follows: Key points are extracted from the arm sub-region using the OpenPose model. These key points include the wrist, finger joints, etc. Based on the extracted key points, the initial edge contour is corrected. The specific steps are as follows: Find key points related to the hand or glove, such as the wrist and finger joints; use the positional information of these key points to adjust the initial edge contour to more accurately match the actual hand or glove shape; smooth the adjusted edge contour to remove possible noise and irregularities, ensuring the continuity and smoothness of the contour, thus obtaining the corrected target edge contour.

[0072] This invention employs two deep learning models, YOLOv5 and OpenPose, and incorporates a large number of samples for training. In the initial stages of the project, the optimal installation height, angle, and focal length of the camera were calculated based on the on-site optical environment. After a month-long video collection process, over 200,000 valid sample images were gathered. The collected dataset underwent preprocessing operations such as image / video scaling, cropping, and normalization. Data augmentation techniques, such as random rotation, flipping, and brightness adjustment, were also applied to prepare training data and improve model performance. Practical verification demonstrated that the model met the design requirements, achieving an accuracy of 99.9%, and exhibiting excellent performance in terms of performance, recall, and precision.

[0073] (iii) Determine the current state instance corresponding to the target detection result.

[0074] In practical implementation, blue warning zones and red alarm zones are defined within the field of view. People, hands, and gloves have four positional states: no position, safe zone, warning zone, and alarm zone. Each object's position is independent, therefore the combinations of their states are Cartesian products. For example, assuming there are n states for an object and a total of k objects, the number of all possible combinations is: Where C represents the total number of combinations, n represents the number of states for each object, and k represents the number of objects. In this embodiment of the invention, objects include a person, a hand, and a glove, and states include no location, safe zone, warning zone, and alarm zone. Therefore, the total number of current state instances is: .

[0075] Table 1 Current Status Examples

[0076] (iv) Determine the target control strategy based on the equipment status and current status instance of the roller pressing equipment, including: determining whether the current status instance matches the warning / alarm status instance associated with the equipment status of the roller pressing equipment; if so, take the control strategy corresponding to the matching warning / alarm status instance as the target control strategy corresponding to the roller pressing equipment.

[0077] In practical applications, devices have two states: ready to run and already running, resulting in a total of 128 possible combinations of current state instances. Device reverse control assumes someone is within the field of view, therefore, cases where a person is not present need to be excluded, leading to 72 possible logical processing combinations.

[0078] There are 36 possible combinations when the device is in standby mode and a person is not in a location-free area. In this case, the device is not allowed to start if the target object appears in the warning or alarm zone.

[0079] There are 36 possible combinations when the equipment is running and a person is not in a no-position area. If the target object appears in the warning area but not in the alarm area, only an audible and visual alarm will be triggered; if the target object appears in the alarm area, not only will an audible and visual alarm be triggered, but the machine will also be shut down.

[0080] In summary, the embodiments of the present invention have at least the following characteristics: Using AI vision to empower equipment safety management on the rolling mill avoids the need for close-range installation of photoelectric gratings, as rubber extension during processing may accidentally trigger the gratings, causing the equipment to stop prematurely. It also avoids the need for long-distance installation of photoelectric gratings, as they are ineffective in protecting personnel crossing them during processing. Furthermore, this solution eliminates the need for workers to wear additional sensing devices.

[0081] Based on the foregoing embodiments, this invention provides a control device for a roller pressing equipment based on multi-target detection. At least two camera devices are deployed at designated locations within the area where the roller pressing equipment is located. Each camera device has a pre-defined physical monitoring area within its field of view. (See [link to relevant documentation]). Figure 5 The diagram shows a structural schematic of a roller pressing equipment control device based on multi-target detection. The device mainly includes the following parts: Image acquisition module 502 is used to acquire image data collected by the camera device from the roller pressing device; The multi-target detection module 504 is used to perform multi-target detection on image data through a pre-trained target detection model to obtain the target detection results corresponding to the image data. The target detection results include one or more of the target edge contours corresponding to the worker's living body, the target edge contours corresponding to the worker's hand area, and the target edge contours corresponding to the glove area worn by the worker. The state instance determination module 506 is used to determine the current state instance corresponding to the target detection result. The current state instance represents the physical monitoring area into which the target edge contour contained in the target detection result falls. The control strategy determination module 508 is used to determine the target control strategy based on the equipment status and current status instance of the roller pressing equipment, so as to execute the control action corresponding to the target control strategy on the roller pressing equipment.

[0082] The multi-target detection-based control device for roller pressing equipment provided in this invention deploys multiple cameras in the roller pressing equipment area and delineates a physical monitoring area within the field of view. A pre-trained target detection model performs multi-target detection on the image data, identifying the location of the worker's body, hands, and gloves. Based on the detection results, the current state instance is determined. Combined with the equipment status of the roller pressing equipment, corresponding control strategies are intelligently generated and executed. This invention achieves real-time monitoring and risk warning for workers, automatically taking control measures before potential dangers occur. This significantly improves the safety of roller pressing equipment operation, reduces the risks caused by human negligence and misoperation, and ensures personnel safety during the production process.

[0083] In one implementation, the target detection model employs a neural network model based on the YOLOv5 framework. The target detection model includes a backbone network, a feature fusion network, and a multi-target detection head; the multi-target detection module 504 is specifically used for: The basic features corresponding to the image data are extracted through the backbone network; The feature fusion network is used to fuse basic features to obtain a multi-size fused feature map that matches the multi-target detection head, and the multi-size fused feature map is filtered by background features to obtain a multi-size target feature map. Using a multi-target detection head, target detection is performed on multi-size target feature maps based on their associated static liveness anchor boxes, dynamic hand anchor boxes, and dynamic glove anchor boxes to obtain the initial detection results corresponding to the image data. The OpenPose model is called to extract key points corresponding to the image data, and the initial detection results are corrected using the key points to obtain the target detection results corresponding to the image data.

[0084] In one embodiment, the multi-target detection head includes a liveness detection head, a hand detection head, and a glove detection head; The liveness detection head is associated with a static liveness anchor frame; The hand detection head includes multiple hand sub-detection heads of different sizes, and the multiple hand sub-detection heads of different sizes are associated with at least one dynamic hand anchor frame; The glove detection head includes multiple glove detection heads of different sizes, and the multiple glove detection heads of different sizes are associated with at least one dynamic glove anchor frame.

[0085] In one implementation, the multi-target detection module 504 is specifically used for: Using a liveness detection head, target detection is performed on multi-size target feature maps based on its associated static liveness anchor frame to obtain the initial edge contour corresponding to the live worker. The hand detection head includes multiple hand sub-detection heads, which perform target detection on multi-size target feature maps based on their respective associated dynamic hand anchor boxes to obtain the initial edge contour corresponding to the worker's hand area. The glove detection head includes multiple glove detection heads, which perform target detection on multi-size target feature maps based on their respective associated dynamic glove anchor frames to obtain the initial edge contour corresponding to the glove area of ​​the worker.

[0086] In one implementation, the multi-target detection module 504 is specifically used for: Feature extraction is performed on the image data to obtain the region of interest and its size range corresponding to the worker's hand area; The following operations are performed by each hand sub-detector contained in the hand detection head: Determine the region of interest corresponding to the worker's hand area, and the intersection-union ratio between it and multiple dynamic hand anchor frames associated with the hand sub-detection head; Based on the cross-union ratio and the cross-union ratio threshold corresponding to the size range, determine whether the region of interest corresponding to the worker's hand area matches the dynamic hand anchor frame; Under the condition of matching, the initial edge contour corresponding to the area of ​​the worker's hand is obtained.

[0087] In one implementation, the multi-target detection module 504 is specifically used for: Determine whether the initial detection result contains the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area; If so, the OpenPose model is invoked, and key points are extracted from the arm sub-region output by the target detection model, using the center coordinates of the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area as the reference. The key points are then used to correct the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area.

[0088] In one implementation, an anchor frame clustering module is also included, for: Obtain the labeled image data corresponding to the roller pressing equipment. The labeled image data includes the live operator, the operator's hand area, and the area of ​​the glove worn by the operator. From the labeled image data, extract the sub-images corresponding to the worker's hand area and the area of ​​the glove worn by the worker: For sub-images corresponding to regions of the same type, multiple target sizes are randomly determined from the sizes corresponding to the sub-images of that type of region; Clustering is performed on the sub-images corresponding to the target size and the sub-images corresponding to other sizes to obtain multiple target clusters; Determine the matching degree between each size contained in the target cluster and the size of the multi-target detection head. Based on the matching degree, the sub-images corresponding to each size contained in the target cluster are used as dynamic anchor boxes associated with the multi-target detection head. The dynamic anchor boxes are dynamic hand anchor boxes or dynamic glove anchor boxes.

[0089] In one implementation, the anchor frame clustering module is specifically used for: Based on clustering parameters, as well as the Euclidean distance and cross-union distance between the sub-images corresponding to the target size and the sub-images corresponding to other sizes, clustering is performed to obtain multiple clusters; Determine the weighted distance variance value for each cluster; If the weighted distance variance is greater than a preset threshold, the clustering parameters are adaptively adjusted. Based on the adaptively adjusted clustering parameters, as well as the Euclidean distance and cross-union distance between the sub-images corresponding to the target size and the sub-images corresponding to other sizes, the clusters are re-divided until the preset clustering stopping condition is met, resulting in multiple target clusters.

[0090] In one implementation, the control strategy determination module 508 is specifically used for: Determine whether the current status instance matches the warning / alarm status instance associated with the equipment status of the roller press; If so, the control strategy corresponding to the matched warning / alarm status instance will be used as the target control strategy for the roller pressing equipment.

[0091] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0092] This invention provides an electronic device, specifically, the electronic device includes a processor and a memory; the memory stores a computer program, which, when run by the processor, executes the method described in any of the above embodiments.

[0093] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 100 includes: a processor 60, a memory 61, a bus 62, and a communication interface 63. The processor 60, the communication interface 63, and the memory 61 are connected through the bus 62. The processor 60 is used to execute executable modules, such as computer programs, stored in the memory 61.

[0094] The memory 61 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 63 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0095] Bus 62 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0096] The memory 61 is used to store programs. After receiving an execution instruction, the processor 60 executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 60 or implemented by the processor 60.

[0097] Processor 60 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 60 or by instructions in software form. Processor 60 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 61. Processor 60 reads the information in memory 61 and, in conjunction with its hardware, completes the steps of the above method.

[0098] The computer program product of the readable storage medium provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the foregoing method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0099] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A control method for a roller pressing equipment based on multi-target detection, characterized in that, At least two camera devices are deployed at a designated location within the area where the roller pressing equipment is located. Each camera device has a pre-defined physical monitoring area within its field of view. The method includes: Acquire image data captured by the camera device for the roller pressing device; The image data is subjected to multi-target detection using a pre-trained target detection model to obtain the target detection results corresponding to the image data. The target detection results include one or more of the following: the target edge contour corresponding to the worker's living body, the target edge contour corresponding to the worker's hand area, and the target edge contour corresponding to the glove area worn by the worker. Determine the current state instance corresponding to the target detection result, wherein the current state instance represents the physical monitoring area into which the target edge contour contained in the target detection result falls; A target control strategy is determined based on the equipment status of the roller pressing equipment and the current status instance, so as to execute the control action corresponding to the target control strategy on the roller pressing equipment; The target detection model employs a neural network model based on the YOLOv5 framework. This model includes a backbone network, a feature fusion network, and a multi-target detection head. The pre-trained target detection model performs multi-target detection on the image data to obtain the target detection results corresponding to the image data, including: The basic features corresponding to the image data are extracted through the backbone network; The feature fusion network is used to fuse the basic features to obtain a multi-size fused feature map that matches the multi-target detection head, and the multi-size fused feature map is then filtered by background features to obtain a multi-size target feature map. Using the multi-target detection head, target detection is performed on the multi-size target feature map based on its associated static liveness anchor frame, dynamic hand anchor frame, and dynamic glove anchor frame to obtain the initial detection result corresponding to the image data; The OpenPose model is invoked to extract key points corresponding to the image data, and the initial detection results are corrected using the key points to obtain the target detection results corresponding to the image data.

2. The control method for roller pressing equipment based on multi-target detection according to claim 1, characterized in that, The multi-target detection head includes a liveness detection head, a hand detection head, and a glove detection head; The liveness detection head is associated with a static liveness anchor frame; The hand detection head includes multiple hand sub-detection heads of different sizes, and the multiple hand sub-detection heads of different sizes are associated with at least one dynamic hand anchor frame; The glove detection head includes multiple glove detection heads of different sizes, and the multiple glove detection heads of different sizes are associated with at least one dynamic glove anchor frame.

3. The control method for roller pressing equipment based on multi-target detection according to claim 2, characterized in that, Using the multi-target detection head, based on its associated static liveness anchor frame, dynamic hand anchor frame, and dynamic glove anchor frame, target detection is performed on the multi-size target feature map to obtain the initial detection result corresponding to the image data, including: Using the liveness detection head, target detection is performed on the multi-size target feature map based on the associated static liveness anchor frame to obtain the initial edge contour corresponding to the live worker. The hand detection head includes multiple hand sub-detection heads, which perform target detection on the multi-size target feature map based on their respective associated dynamic hand anchor frames to obtain the initial edge contour corresponding to the worker's hand area. The glove detection head includes multiple glove detection heads, which perform target detection on the multi-size target feature map based on their respective associated dynamic glove anchor frames to obtain the initial edge contour corresponding to the worker's glove area.

4. The control method for roller pressing equipment based on multi-target detection according to claim 3, characterized in that, The hand detection head includes multiple hand sub-detection heads, which perform target detection on the multi-size target feature map based on their respective associated dynamic hand anchor frames to obtain the initial edge contour corresponding to the worker's hand region, including: Feature extraction is performed on the multi-size target feature map to obtain the region of interest and its size range corresponding to the worker's hand area; The following operations are performed by each of the hand sub-detection heads included in the hand detection head: Determine the region of interest corresponding to the worker's hand area, and the intersection-union ratio (CUI) among the multiple dynamic hand anchor frames associated with the hand sub-detection head; Based on the intersection-union ratio and the intersection-union ratio threshold corresponding to the size range, determine whether the region of interest corresponding to the worker's hand area matches the dynamic hand anchor frame; The initial edge contour corresponding to the worker's hand area is obtained under the condition of matching.

5. The control method for roller pressing equipment based on multi-target detection according to claim 1, characterized in that, The OpenPose model is invoked to extract key points corresponding to the image data, and these key points are used to correct the initial detection results, including: Determine whether the initial detection result includes the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area; If so, the OpenPose model is invoked, and key points are extracted from the arm sub-region output by the target detection model, using the center coordinates of the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area as a reference. The key points are then used to correct the initial edge contour corresponding to the worker's hand area or the initial edge contour corresponding to the worker's glove area.

6. The control method for roller pressing equipment based on multi-target detection according to claim 1, characterized in that, The method further includes: Obtain the labeled image data corresponding to the roller pressing equipment. The labeled image data includes the live body of the operator, the operator's hand area, and the area of ​​the glove worn by the operator. From the labeled image data, extract the sub-images corresponding to the worker's hand area and the glove area worn by the worker: For the sub-images corresponding to the same type of region, multiple target sizes are randomly determined from the sizes corresponding to the sub-images corresponding to that type of region; Clustering is performed on the sub-image corresponding to the target size and the sub-images corresponding to other sizes to obtain multiple target clusters; The matching degree between each size contained in the target cluster and the size of the multi-target detection head is determined, so that the sub-images corresponding to each size contained in the target cluster are used as dynamic anchor frames associated with the multi-target detection head based on the matching degree. The dynamic anchor frames are the dynamic hand anchor frames or the dynamic glove anchor frames.

7. The control method for roller pressing equipment based on multi-target detection according to claim 6, characterized in that, Clustering is performed on the sub-image corresponding to the target size and the sub-images corresponding to other sizes to obtain multiple target clusters, including: Based on clustering parameters, and the Euclidean distance and intersection-union ratio distance between the sub-image corresponding to the target size and the sub-image corresponding to other sizes, clustering is performed to obtain multiple clusters; Determine the weighted distance variance value corresponding to each of the said clusters; If the weighted distance variance is greater than a preset threshold, the clustering parameters are adaptively adjusted, and based on the adaptively adjusted clustering parameters, as well as the Euclidean distance and intersection-union distance between the sub-image corresponding to the target size and the sub-image corresponding to other sizes, the clusters are re-divided until the preset clustering stopping condition is met, resulting in multiple target clusters.

8. The control method for roller pressing equipment based on multi-target detection according to claim 1, characterized in that, The target control strategy is determined based on the equipment status of the roller pressing equipment and the current status instance, including: Determine whether the current state instance matches the warning / alarm state instance associated with the equipment state of the roller pressing equipment; If so, the control strategy corresponding to the matched warning / alarm status instance will be used as the target control strategy for the roller pressing equipment.

9. A control device for a roller pressing equipment based on multi-target detection, characterized in that, At least two camera devices are deployed at a designated location within the area where the roller pressing equipment is located. Each camera device has a pre-defined physical monitoring area within its field of view. The device includes: The image acquisition module is used to acquire image data collected by the camera device from the roller pressing device; The multi-target detection module is used to perform multi-target detection on the image data using a pre-trained target detection model to obtain the target detection result corresponding to the image data. The target detection result includes one or more of the following: the target edge contour corresponding to the worker's living body, the target edge contour corresponding to the worker's hand area, and the target edge contour corresponding to the glove area worn by the worker. The state instance determination module is used to determine the current state instance corresponding to the target detection result, wherein the current state instance represents the physical monitoring area into which the target edge contour contained in the target detection result falls; The control strategy determination module is used to determine a target control strategy based on the equipment status of the roller pressing equipment and the current status instance, so as to execute the control action corresponding to the target control strategy on the roller pressing equipment; The target detection model employs a neural network model based on the YOLOv5 framework, comprising a backbone network, a feature fusion network, and a multi-target detection head; the multi-target detection module is specifically used for: The basic features corresponding to the image data are extracted through the backbone network; The feature fusion network is used to fuse the basic features to obtain a multi-size fused feature map that matches the multi-target detection head, and the multi-size fused feature map is then filtered by background features to obtain a multi-size target feature map. Using the multi-target detection head, target detection is performed on the multi-size target feature map based on its associated static liveness anchor frame, dynamic hand anchor frame, and dynamic glove anchor frame to obtain the initial detection result corresponding to the image data; The OpenPose model is invoked to extract key points corresponding to the image data, and the initial detection results are corrected using the key points to obtain the target detection results corresponding to the image data.

Images