Multi-working-plane monitoring method and device, deploy and control ball and computer readable storage medium

Abstract

The embodiment of the invention provides a multi-working-plane monitoring method and device, a deployment and control ball and a computer readable storage medium, and relates to the technical field of monitoring management. The method comprises the following steps: identifying the types of a plurality of target objects in a panoramic image through a monitoring ball, dividing a spatial range where the target objects of the same type are located into working planes corresponding to the target objects based on a target detection result, and sequentially determining the working planes divided according to the panoramic image as to-be-monitored target working planes, and the deployment and control ball controls the pan-tilt camera to automatically monitor each target working plane in a polling manner so as to continuously monitor whether construction violation behaviors exist in the multiple working planes or not. Under the condition that manual intervention is not needed, the deployment and control ball utilizes target object detection to automatically recognize a plurality of working planes and complete coverage type monitoring on a plurality of working scenes, so that the problem that a single deployment and control ball has a monitoring blind area in a multi-working-plane scene is effectively solved, and the overall safety supervision efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of monitoring and management technology, and more specifically, to a multi-workface monitoring method, device, surveillance sphere, and computer-readable storage medium. Background Technology

[0002] In power construction operations, to ensure safety, power companies typically require a "no work without video surveillance" policy. Therefore, during routine safety supervision, each work order is equipped with a surveillance camera. This camera captures real-time video of the work on-site and uploads the video content to the safety supervision center via local recording and network transmission.

[0003] Existing surveillance cameras have significant limitations when deployed at work sites. A single surveillance camera can only monitor one work area. When there are multiple work areas (such as high-altitude operations, ground operations, crane lifting operations, etc.) under the management of a work ticket, the remaining work areas are likely to become blind spots for monitoring, posing safety hazards. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a multi-workface monitoring method, device, monitoring ball, and computer-readable storage medium, which can effectively solve the monitoring blind spot problem of a single monitoring ball in a multi-workface scenario.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a multi-face monitoring method, applied to a surveillance sphere, the method comprising: The panoramic image is identified according to multiple target object types to obtain target detection results; If the target detection result contains at least one target object type, the work surface corresponding to the target object belonging to the same target object type in the target detection result is determined as the target work surface; different target object types correspond to different work surfaces. Control the pan-tilt camera to track the target work surface and monitor whether there are any construction violations on the target work surface.

[0006] In an optional implementation, the target object types include transmission tower types, heavy equipment types, and construction worker types; construction workers are non-high-altitude workers wearing safety equipment; the step of identifying the panoramic image according to multiple target object types to obtain target detection results includes: The pre-trained target detection model is used to detect whether there are power transmission towers, heavy equipment, or construction workers in the panoramic image. If the transmission tower exists, add the type of the transmission tower and the tower information of at least one transmission tower in the panoramic image to the target detection result; If the heavy equipment exists, add the type of the heavy equipment and the equipment information of at least one heavy equipment in the panoramic image to the target detection result; If the construction worker exists, the construction worker type and the construction worker information of at least one construction worker in the panoramic image are added to the target detection result.

[0007] In an optional implementation, if the target detection result contains at least one target object type, determining the work surface corresponding to the target object belonging to the same target object type in the target detection result as the target work surface includes: If the target detection result only contains the transmission tower type, the transmission tower working surface is determined as the target working surface; the transmission tower working surface includes the working surfaces corresponding to all transmission towers identified in the panoramic image; If the target detection result contains only the heavy equipment type, the heavy equipment work surface is determined as the target work surface; the heavy equipment work surface includes all work surfaces corresponding to heavy equipment identified in the panoramic image; If the target detection result contains only the construction worker type, the construction worker's work surface is determined as the target work surface; the construction worker's work surface includes the work surfaces corresponding to all construction workers identified in the panoramic image; If the target detection result includes the transmission tower type and the heavy equipment type, the working surface corresponding to the transmission tower type and the working surface corresponding to the heavy equipment type are respectively determined as the target working surface.

[0008] In an optional implementation, the control of the pan-tilt camera to track the target work surface and monitor whether there are any construction violations on the target work surface includes: If the target working surface is the working surface of the transmission tower, the target tower is determined from all the transmission towers identified in the panoramic image; Adjust the direction of the gimbal and the zoom level of the gimbal camera according to the tower information corresponding to the target tower, so that the target tower is located in the monitoring screen of the gimbal camera; The gimbal camera is controlled to scan the target tower step by step, and multiple images of the target tower are captured. Based on the captured images, the number of workers and their work attire information on the target tower are identified. If the number of workers on the target tower exceeds the number threshold or the work attire information indicates that the workers on the target tower are not wearing safety equipment, it is determined that there is a construction violation on the target tower.

[0009] In an optional implementation, the control of the pan-tilt camera to track the target work surface and monitor whether there are any construction violations on the target work surface includes: If the target work surface is the work surface of the heavy equipment, the target equipment is determined from all the heavy equipment identified in the panoramic image; Adjust the direction of the gimbal and the zoom level of the gimbal camera according to the device information corresponding to the target device, so that the target device is located in the detection field of the gimbal camera; The gimbal camera is controlled to scan the target device step by step to acquire multiple captured images of the target device. Based on the captured images, the boom detection frame, the relative depth of the boom, the personnel detection frame, and / or the relative depth of the personnel of the target device are identified. The presence of construction violations on the target work surface is identified based on the target equipment's boom detection frame, the target equipment's boom relative depth, the personnel detection frame, and / or the personnel relative depth.

[0010] In an optional implementation, identifying whether there are construction violations at the target work surface based on the target equipment's boom detection frame, the target equipment's boom relative depth, the personnel detection frame, and / or the personnel relative depth includes: If the personnel detection frame exists, determine whether the crane detection frame of the target equipment and the personnel detection frame intersect; If the boom detection frame of the target equipment intersects with the personnel detection frame, and the difference between the boom depth value and the personnel depth value corresponding to the intersecting personnel detection frame is less than a depth threshold, it is determined that there is a construction violation at the target work surface.

[0011] In an optional implementation, the control of the pan-tilt camera to track the target work surface and monitor whether there are any construction violations on the target work surface includes: If the target work area is the work area of ​​the construction personnel, the target construction personnel are determined from all the construction personnel identified in the panoramic image; Adjust the direction of the gimbal and the zoom level of the gimbal camera according to the construction personnel information corresponding to the target construction personnel, so that the target construction personnel are located in the monitoring screen of the gimbal camera; The PTZ camera is controlled to scan the target construction worker step by step, and multiple images of the target construction worker are captured. Based on the captured images, the work clothing information and fence crossing detection information of the target construction worker are identified. If the work attire information of the target construction worker indicates that the target construction worker is not wearing safety equipment, or if the fence crossing detection information of the target construction worker indicates that the target construction worker has crossed the fence, it is determined that the target construction worker has committed a construction violation.

[0012] In an optional implementation, the fence crossing detection information includes a person frame and a fence frame, and the method further includes: If the person frame and the fence frame intersect, the behavior of crossing the fence is detected based on the head depth value of the head region in the person frame and the fence depth value of the fence frame; the fence depth value is determined based on the depth value of the interval with the highest distribution concentration in the fence frame.

[0013] In an optional implementation, the method further includes: If the target detection result does not include any of the target object types, control the gimbal camera to scan the non-high-altitude construction site, obtain multiple snapshot images of the non-high-altitude construction site, and identify whether there are construction personnel based on the snapshot images of the non-high-altitude construction site. If no construction workers are present, then a construction violation is deemed to have occurred. If construction workers are present, the system identifies whether they are wearing safety equipment and whether they are crossing the fence based on the captured images of the non-high-altitude construction site. If the construction workers are not wearing safety equipment or cross the fence, it is determined that there is a construction violation.

[0014] Secondly, the present invention provides a multi-face monitoring device for use in a surveillance system, the device comprising: The detection module is used to identify panoramic images according to multiple target object types and obtain target detection results; The monitoring module is used to determine the work surface corresponding to the target object of the same target object type in the target detection result as the target work surface if the target detection result contains at least one target object type; the work surfaces corresponding to target objects of different target object types are different; control the pan-tilt camera to track the target work surface and monitor whether there are any construction violations on the target work surface.

[0015] Thirdly, the present invention provides a control ball, including a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the multi-workface monitoring method described in any of the foregoing embodiments.

[0016] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the multi-workface monitoring method as described in any of the foregoing embodiments.

[0017] Compared to existing technologies, the multi-workface monitoring method, device, surveillance sphere, and computer-readable storage medium provided in this invention identify multiple target object types in a panoramic image using a surveillance sphere. Based on the target detection results, the spatial range of the same type of target object is divided into corresponding workfaces. Due to the different operational behavior characteristics of different target object types, each type of workface has its own monitoring focus and logical processing path. The workfaces divided according to the panoramic image are sequentially determined as target workfaces to be monitored. The surveillance sphere controls the pan-tilt camera to automatically patrol and monitor each target workface to continuously monitor whether there are any construction violations on multiple workfaces. Without manual intervention, the surveillance sphere automatically identifies multiple workfaces using target object detection and completes comprehensive monitoring of multiple work scenarios, thereby effectively solving the monitoring blind spot problem of a single surveillance sphere in multi-workface scenarios and improving the overall safety supervision efficiency.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a multi-workface monitoring method provided in an embodiment of the present invention is shown.

[0021] Figure 2 This diagram illustrates another flowchart of the multi-workface monitoring method provided in an embodiment of the present invention.

[0022] Figure 3 A block diagram of a multi-workface monitoring device provided in an embodiment of the present invention is shown.

[0023] Figure 4 A block diagram of a control ball provided in an embodiment of the present invention is shown.

[0024] Icons: 200-Multi-face monitoring device; 201-Detection module; 202-Monitoring module; 203-Transmission module; 300-Control ball; 310-Processor; 320-Memory. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0027] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0028] Currently, a standard surveillance camera is typically deployed at power work sites to collect on-site video images, which are then transmitted to the safety supervision center via a dedicated 4G wireless network. Supervision center personnel can view the video feed in real time on the platform to monitor compliance with regulations. The platform's AI system can also perform frame-by-frame analysis of the uploaded videos to identify potential violations and push the results to supervisors for confirmation.

[0029] However, in real-world scenarios where a single work order corresponds to multiple work areas, such as simultaneous high-altitude work, ground work, and crane lifting operations, the inventors discovered that ordinary surveillance cameras lack end-side intelligent analysis capabilities. They can only cover one work area and cannot autonomously identify and cycle through multiple work areas, leaving the remaining work areas as monitoring blind spots and making it difficult to detect safety hazards in a timely manner.

[0030] It should be understood that a work area refers to the physical space area in a construction site related to a specific work activity. For example, a high-altitude work area refers to the area around a power transmission tower where climbing may be involved, a ground work area refers to the area where construction workers are not working at height, and a crane lifting work area refers to the area where the crane boom is operating.

[0031] In addition, the existing surveillance cameras mainly focus on video acquisition and remote transmission. Violation identification relies entirely on the AI ​​processing in the background of the security supervision center, resulting in a high dependence on network bandwidth and cloud computing power, low response efficiency, and difficulty in meeting the comprehensive and real-time security supervision needs in complex operation scenarios.

[0032] Based on this, embodiments of the present invention provide a multi-workface monitoring method, device, surveillance sphere, and computer-readable storage medium. The method uses the surveillance sphere to identify multiple target object types in a panoramic image and, based on the target detection results, divides the spatial range of the same type of target object into corresponding workfaces. Due to the different operational behavior characteristics of different target object types, each type of workface has its own monitoring focus and logical processing path. The workfaces divided according to the panoramic image are sequentially determined as the target workfaces to be monitored. The surveillance sphere controls a pan-tilt camera to automatically patrol and monitor each target workface to continuously monitor whether there are any construction violations on multiple workfaces. Without manual intervention, the surveillance sphere automatically identifies multiple workfaces using target object detection and completes comprehensive monitoring of multiple work scenarios, thereby effectively solving the monitoring blind spot problem of a single surveillance sphere in multi-workface scenarios and improving overall safety supervision efficiency.

[0033] In this embodiment of the invention, the monitoring sphere is a multi-view panoramic intelligent monitoring sphere, which includes a multi-view fixed-focus camera and a pan-tilt unit. The fixed-focus camera is located at the base of the multi-view panoramic intelligent monitoring sphere, achieving full coverage of a 360-degree horizontal field of view. The pan-tilt unit includes a pan-tilt camera, which, while controlling the pan-tilt unit to adjust its direction, uses the pan-tilt camera to poll and scan each work surface to identify whether there are any construction violations on the currently monitored work surface.

[0034] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0035] Please refer to Figure 1 , Figure 1 A schematic flowchart of a multi-workface monitoring method provided by an embodiment of the present invention is shown. The method includes the following steps: Step S100: Identify the panoramic image according to multiple target object types to obtain target detection results.

[0036] In this embodiment of the invention, the surveillance sphere utilizes its built-in artificial intelligence algorithm module to perform target recognition processing on panoramic images captured by multi-view panoramic cameras, thereby obtaining a target detection result that may contain multiple target object types. The target object types mentioned here specifically include, but are not limited to, heavy equipment such as power transmission towers and cranes, as well as construction workers wearing safety equipment. The presence of these target objects usually indicates the existence of a specific type of construction work at the construction site.

[0037] Furthermore, the monitoring sphere divides the area containing all target objects of the same type from the target detection results into several corresponding work surfaces, where the work surfaces corresponding to different target object types have spatial differences. This can be understood as each target object type corresponding to one or more work spaces with different physical locations, and the work space corresponding to each target object type is the work surface.

[0038] Step S110: If the target detection result contains at least one target object type, the work surface corresponding to the target object belonging to the same target object type in the target detection result is determined as the target work surface; the work surfaces corresponding to target objects of different target object types are different.

[0039] In this embodiment of the invention, when the target detection result contains at least one target object type, the control ball sets the work surfaces corresponding to these target objects as target work surfaces in sequence according to the preset business logic, and starts the PTZ camera to automatically track and monitor them in a predetermined order.

[0040] For example, if a power transmission tower and a crane are detected simultaneously in a panoramic image, the monitoring system will treat the work surfaces corresponding to these two target objects as two independent target work surfaces, and switch to the corresponding target work surfaces in turn for in-depth monitoring based on priority or time polling strategies.

[0041] Step S120: Control the PTZ camera to track the target work surface and monitor whether there are any construction violations on the target work surface.

[0042] In this embodiment of the invention, the gimbal camera is controlled to adjust the shooting angle and focal length so that the currently selected target work surface is located in the center area of ​​the gimbal camera's image. By continuously tracking the target work surface and combining image analysis technology, it is determined in real time whether the target work surface has any behavior that violates safety regulations.

[0043] Therefore, by sequentially setting the work surfaces corresponding to the identified different target objects as target work surfaces and conducting round-robin monitoring, the surveillance sphere can autonomously complete the safety monitoring tasks of multiple work surfaces without human intervention, avoiding the blind spot problem caused by traditional single-view surveillance spheres that can only cover a single work surface. This mechanism not only improves the comprehensiveness and timeliness of on-site supervision, but also significantly reduces the dependence on back-end manual monitoring resources, realizing the effective utilization of the surveillance sphere's artificial intelligence capabilities.

[0044] In summary, the multi-workface monitoring method provided in this invention identifies multiple target object types in a panoramic image using a PTZ camera. Based on the target detection results, the spatial range of the same type of target object is divided into corresponding workfaces. Due to the different operational behavior characteristics of different target object types, each type of workface has its own monitoring focus and logical processing path. The workfaces divided according to the panoramic image are sequentially determined as the target workfaces to be monitored. The PTZ camera controls the PTZ camera to automatically patrol and monitor each target workface to continuously monitor whether there are any construction violations on multiple workfaces. Without manual intervention, the PTZ camera automatically identifies multiple workfaces using target object detection and completes comprehensive monitoring of multiple work scenarios, thereby effectively solving the monitoring blind spot problem of a single PTZ camera in multi-workface scenarios and improving the overall safety supervision efficiency.

[0045] Optionally, in practical applications, target object types include transmission towers, heavy equipment, and construction workers. Construction workers refer to non-high-altitude workers wearing safety equipment. The following provides a possible implementation method for generating target detection results based on transmission towers, heavy equipment, and construction workers. Figure 1 The sub-steps of step S100 may include: Step S100-1: Use a pre-trained target detection model to detect whether there are power transmission towers, heavy equipment, or construction workers in the panoramic image.

[0046] In this embodiment of the invention, the transmission tower type is used to characterize high-altitude operation scenarios in power construction sites, the heavy equipment type is used to characterize large machinery operation scenarios in power construction sites, and the construction worker type is used to characterize ordinary ground operation scenarios in power construction sites.

[0047] The surveillance camera utilizes its built-in artificial intelligence algorithm module and a pre-trained target detection model to analyze panoramic images captured by multi-view panoramic cameras to determine whether power transmission towers, heavy equipment, or construction workers are present in the images. This target detection model, trained on a large number of construction site image samples, can accurately identify the key features of these three types of targets and output corresponding detection results.

[0048] Step S100-2: If power transmission towers exist, add the power transmission tower type and tower information of at least one power transmission tower in the panoramic image to the target detection result.

[0049] In this embodiment of the invention, during the detection process, if a power transmission tower is detected, the tower type along with the tower information of each detected tower is added to the target detection result. The tower information includes the tower detection frame, the tower's relative depth, and the number of tower pixels. The tower's relative depth characterizes the distance between the power transmission tower and the monitoring sphere; the number of tower pixels characterizes the number of pixels the power transmission tower occupies in the panoramic image.

[0050] Step S100-3: If heavy equipment exists, add the type of heavy equipment and the equipment information of at least one heavy equipment in the panoramic image to the target detection result.

[0051] Step S100-4: If construction workers exist, add the construction worker type and the construction worker information of at least one construction worker in the panoramic image to the target detection result.

[0052] In this embodiment of the invention, if heavy equipment is detected, the heavy equipment type along with the equipment information of each detected heavy equipment is added to the target detection result. If construction workers are detected, the construction worker type along with the construction worker information of each detected construction worker is added to the target detection result. The equipment information includes a heavy equipment detection box, and the construction worker information includes a construction worker detection box.

[0053] Understandably, by classifying and identifying target objects according to preset types and structurally recording the existence status and corresponding information of each type of target object, the monitoring sphere can accurately divide the corresponding work areas based on this information and sequentially patrol and monitor multiple work areas. This design not only enhances the monitoring sphere's adaptability to complex construction site environments but also provides a reliable data foundation for PTZ tracking and violation detection, thereby improving the intelligence level and regulatory efficiency of multi-work area monitoring.

[0054] As can be seen, by introducing specific types of target objects and adopting a structured information addition mechanism, the embodiments of the present invention enable the system to acquire key visual information related to different work scenarios more efficiently, thereby providing accurate data support for subsequent division of work surfaces and patrol monitoring based on target object types, and improving the environmental perception capability and task execution targeting of the deployment ball in complex construction sites.

[0055] Alternatively, the following is one possible implementation method for determining the target work surface to be monitored. Figure 1 The sub-steps of step S110 may include: Step S110-1: If the target detection result only contains the type of transmission tower, the transmission tower working surface is determined as the target working surface; the transmission tower working surface includes the working surfaces corresponding to all transmission towers identified in the panoramic image.

[0056] In this embodiment of the invention, it is determined whether the target detection result only includes transmission tower types. If this condition is met, it indicates that all target objects identified in the current panoramic image are transmission towers, and the transmission tower working surface is set as the target working surface. The transmission tower working surface includes the spatial area corresponding to all transmission towers identified in the panoramic image; these areas typically represent high-altitude work scenarios.

[0057] Step S110-2: If the target detection result only contains heavy equipment type, the heavy equipment working surface is determined as the target working surface; the heavy equipment working surface includes the working surfaces corresponding to all heavy equipment identified in the panoramic image.

[0058] In this embodiment of the invention, if the target detection result only includes heavy equipment, it indicates that there are ground operations at the construction site dominated by large machinery such as cranes, and the heavy equipment work surface is set as the target work surface. The heavy equipment work surface covers the locations of all identified heavy equipment in the panoramic image, and is used for subsequent focused monitoring of related high-risk behaviors, such as people standing under crane booms.

[0059] Step S110-3: If the target detection result only contains the type of construction workers, the construction worker work surface is determined as the target work surface; the construction worker work surface includes the work surfaces corresponding to all construction workers identified in the panoramic image.

[0060] In this embodiment of the invention, if the target detection result only includes the type of construction worker, it indicates that the construction site is mainly engaged in ordinary ground operations, and the construction worker's work area is set as the target work area. The construction worker's work area covers the area where all identified construction workers are located, serving as the basis for detecting the wearing of safety equipment and crossing of fences.

[0061] Step S110-4: If the target detection result includes transmission tower type and heavy equipment type, determine the working surface corresponding to the transmission tower type and the working surface corresponding to the heavy equipment type as the target working surface respectively.

[0062] In this embodiment of the invention, if the target detection result includes both transmission tower type and heavy equipment type, it means that there are multiple parallel and different types of work activities at the construction site. In order to ensure the comprehensiveness and effectiveness of supervision, the work surfaces corresponding to the transmission tower type and the work surfaces corresponding to the heavy equipment type are respectively included in the target work surface set, thereby realizing the alternating patrol monitoring of the two types of work scenarios.

[0063] This can be understood as follows: through this work area division mechanism based on the combination of target object types, the monitoring ball can automatically adjust the monitoring strategy according to the actual situation on site, avoid safety blind spots caused by fixed perspectives or single work area monitoring, and improve the adaptability and response efficiency to different types of work.

[0064] As can be seen, the embodiments of the present invention, by determining the working surfaces corresponding to multiple target objects of the same type in sequence as target working surfaces based on the combination of target object types in the target detection results, can achieve automatic identification and monitoring range division of different types of work activities at the construction site without manual intervention, providing a clear operational basis for subsequent PTZ camera patrol tracking, thereby improving the autonomous judgment capability and monitoring coverage of the PTZ camera in complex multi-operation scenarios.

[0065] Alternatively, the following is a possible implementation method for monitoring construction violations at the working face of power transmission towers. Figure 1 The sub-steps of step S120 may include: Step S120-1: If the target working surface is a power transmission tower working surface, determine the target tower from all power transmission towers identified in the panoramic image.

[0066] In this embodiment of the invention, when it is determined that the current target working surface is a transmission tower working surface, the monitoring mode is switched to the tower working mode, and the target tower is determined from all transmission towers identified in the panoramic image based on the relative depth of the tower and / or the number of tower pixels.

[0067] If a single transmission tower exists in the panoramic image and the number of tower pixels exceeds a pixel threshold, that tower is designated as the target tower. If multiple transmission towers exist in the panoramic image, the towers with a pixel count exceeding the pixel threshold are selected as candidate towers. The target tower is then determined from the candidate towers based on relative depth, and this process continues until all candidate towers have been scanned to complete the current monitoring task for the transmission tower work area. For example, candidate towers can be sorted based on relative depth, and then selected as target towers sequentially from near to far or from far to near.

[0068] Step S120-2: Adjust the direction of the gimbal and the zoom of the gimbal camera according to the tower information corresponding to the target tower, so that the target tower is in the monitoring screen of the gimbal camera.

[0069] In this embodiment of the invention, the tower detection frame in the tower information corresponding to the target tower is mapped to the three-dimensional coordinates of the gimbal. The direction of the gimbal and the zoom of the gimbal camera are adjusted according to the three-dimensional coordinates corresponding to the target tower. The direction of the gimbal is finely adjusted so that the base of the target tower is located at the starting position of the monitoring screen corresponding to the target tower scanned by the gimbal camera.

[0070] Step S120-3: Control the PTZ camera to scan the target tower step by step, acquire multiple capture images of the target tower, and identify the number of workers and their clothing information on the target tower based on the capture images.

[0071] In this embodiment of the invention, the gimbal camera is controlled to scan the target tower step by step, and the personnel detection boxes corresponding to the workers are obtained based on the images captured on the target tower. The number of workers on the target tower is determined according to the number of personnel detection boxes. If the number of workers exceeds the personnel threshold, the images captured on the target tower are determined as the capture images of the target tower.

[0072] If the number of workers does not exceed the personnel threshold, after the PTZ camera scans each worker on the target tower, the zoom of the PTZ camera is finely adjusted so that the proportion of workers in the monitoring screen of the PTZ camera meets the preset ratio requirement; the workers are continuously tracked for a preset time and their images are captured; the built-in image model is used to identify whether the workers are wearing safety equipment in the captured images, and the workers' work attire information is obtained.

[0073] Each worker is assigned a specific work attire information record, which indicates whether the worker is wearing safety equipment, such as a helmet and seatbelt. The preset time can be set according to recognition efficiency and real-time requirements; this invention does not limit this setting. It should be noted that the built-in image model determines whether a worker is wearing a helmet or seatbelt by detecting local features (such as the head and waist) of the worker in the monitoring image.

[0074] Step S120-4: If the number of workers on the target tower exceeds the number threshold or the work attire information indicates that the workers on the target tower are not wearing safety equipment, it is determined that there is a construction violation on the target tower.

[0075] In this embodiment of the invention, if the number of workers on the target tower exceeds a certain threshold, it is determined that the target tower is in violation of construction regulations due to overstaffing, and the captured image of the target tower is identified as an abnormal image. If the workers on the target tower are not wearing safety equipment, it is determined that the target tower is in violation of construction regulations due to a lack of safety equipment, and the captured image of the target tower is identified as an abnormal image.

[0076] Upon detecting construction violations, an alarm is issued at the construction site of the transmission tower work area based on the target tower. Simultaneously, abnormal images are uploaded to the artificial intelligence platform server (i.e., the safety supervision center) for secondary verification and analysis. The results of this secondary verification and analysis are then presented to supervisors for final confirmation. This process improves efficiency and ensures comprehensive monitoring and analysis coverage even with limited computing resources on the artificial intelligence server.

[0077] As can be seen, the embodiments of the present invention track each target tower in the panoramic image by adjusting the direction of the gimbal and the zoom of the gimbal camera, and determine whether there are violations of safety regulations such as overloading or failure to wear safety equipment in the transmission tower working face based on the number of workers on the target tower and the workers' work clothing information. This realizes automatic tracking and identification of violations of transmission tower working scenarios without human intervention, and improves the automatic monitoring capability and supervision accuracy of the control ball in complex construction sites.

[0078] Alternatively, the following is a possible implementation method for monitoring construction violations at heavy equipment work sites. Figure 1 The sub-steps of step S120 may include: Step S120-5: If the target work surface is a heavy equipment work surface, determine the target equipment from all the heavy equipment identified in the panoramic image.

[0079] In this embodiment of the invention, when the current target work surface is determined to be a heavy equipment work surface, the monitoring mode is switched to heavy equipment work mode. Target equipment is sequentially determined from all heavy equipment identified in the panoramic image; that is, heavy equipment meeting preset screening conditions is sequentially identified as target equipment. This process continues until all heavy equipment meeting the preset screening conditions has been scanned, thus completing the current monitoring task for the heavy equipment work surface. The preset screening conditions can be relative depth and pixel count, or the size of the heavy equipment detection frame, and can be set according to the actual application scenario; this invention is not limited to these settings.

[0080] Step S120-6: Adjust the direction of the gimbal and the zoom of the gimbal camera according to the device information corresponding to the target device, so that the target device is located in the detection field of the gimbal camera.

[0081] Step S120-7: Control the PTZ camera to scan the target device step by step, acquire multiple capture images of the target device, and identify the target device's boom detection frame, the target device's boom relative depth, the personnel detection frame, and / or the personnel relative depth based on the capture images.

[0082] In this embodiment of the invention, detecting heavy equipment typically indicates the possibility of high-risk hoisting operations. Similarly, referring to the scanning process of the target tower, the direction and zoom of the pan-tilt unit can be adjusted according to the heavy equipment detection frame in the equipment information corresponding to the target equipment, so that the target equipment is located in the center area of ​​the pan-tilt camera's image and presented at an appropriate scale, thereby ensuring the clarity of subsequent image acquisition and the accuracy of analysis.

[0083] The system continuously tracks the target device at preset intervals, acquiring multiple snapshots of the device during the tracking and scanning process. Target detection is performed on these snapshots to identify the boom detection box and personnel detection boxes. Monocular depth estimation is then used to learn the relative depths of the boom and personnel from the snapshots. Specifically, the boom detection box represents the bounding box of the target device's boom in the snapshot image, and the relative depth value of the boom represents the spatial position of the boom relative to the gimbal camera. The personnel detection box represents the bounding box of personnel appearing in the snapshot image, and the relative depth value of the personnel represents the spatial position of the personnel relative to the gimbal camera.

[0084] Step S120-8: Identify whether there are any construction violations on the target work surface based on the target equipment boom detection frame, the target equipment boom relative depth, the personnel detection frame, and / or the personnel relative depth.

[0085] In this embodiment of the invention, it is determined whether a person is standing under the boom of the target equipment based on the boom detection frame, the boom depth value, the personnel detection frame, and / or the personnel depth value. If a person is standing under the boom of the target equipment, the captured image of the target equipment is identified as an abnormal image, and it is determined that there is a construction violation at the target work surface. If no one is standing under the boom of any of the target equipment, it is determined that there is no construction violation at the target work surface.

[0086] Upon detecting construction violations, an alarm is issued at the construction site of the heavy equipment operation area based on the target equipment. Simultaneously, abnormal images are uploaded to the artificial intelligence platform server (i.e., the safety supervision center) for secondary verification and analysis. The results of this secondary verification and analysis are then presented to supervisors for final confirmation. This process improves efficiency and ensures comprehensive monitoring and analysis coverage even with limited computing resources on the artificial intelligence server.

[0087] As can be seen, the embodiments of the present invention track each target device in the panoramic image by adjusting the direction of the gimbal and the zoom of the gimbal camera, and determine whether there is any violation of safety regulations by standing under the boom of the heavy equipment in the working face based on the boom detection frame of the target device, the relative depth of the boom of the target device, the personnel detection frame and / or the relative depth of the personnel. This realizes automatic tracking and identification of violations of heavy equipment working scenes without human intervention, and improves the automatic monitoring capability and supervision accuracy of the surveillance ball in complex construction sites.

[0088] Optionally, regarding how to identify construction violations by personnel leaving heavy equipment, the following is a possible implementation method. Sub-steps of step S120-8 may include: If a personnel detection frame exists, determine whether the target equipment's boom detection frame and the personnel detection frame intersect. If the target equipment's boom detection frame and the personnel detection frame intersect and the difference between the boom depth value and the personnel depth value corresponding to the intersecting personnel detection frame is less than a depth threshold, it is determined that there is a construction violation at the target work surface.

[0089] In this embodiment of the invention, the presence of a personnel detection frame in the captured image is determined based on the target device. If no personnel detection frame is found, it is determined that no one is standing under the boom of the target device, and therefore, no construction violation is occurring on the target device. If no one is standing under the boom of any of the target devices, it is determined that no construction violation is occurring on the target work surface.

[0090] If a personnel detection box exists, further analyze the spatial relationship between the crane detection box and the personnel detection box in the captured image (i.e., the two-dimensional image) to determine whether the crane detection box and the personnel detection box intersect, that is, whether there is an overlapping area between the two detection boxes in the image coordinate system. If there is an overlapping area, it means that the personnel are within the range of the crane's movement trajectory in the two-dimensional space.

[0091] If there is spatial overlap between personnel and the crane boom in two-dimensional space, the difference between the relative depth of the crane boom and the relative depth of the personnel in the corresponding personnel detection frame is compared. When the difference is less than a preset depth threshold, it indicates that the distance between the crane boom and the personnel in three-dimensional space is too close, posing a safety hazard, and it is determined that there is a construction violation at the current target work surface. The difference between the relative depth of the crane boom and the relative depth of the personnel can be the minimum difference between the crane boom and the personnel, or it can be the average difference between the crane boom and the personnel; this invention does not limit this.

[0092] As can be seen, by comparing the two-dimensional spatial information and three-dimensional depth information of the boom of heavy equipment and personnel, the embodiments of the present invention can comprehensively analyze the relative relationship between the boom of heavy equipment and personnel, effectively identify construction violations by personnel standing under the boom of heavy equipment, and improve the accuracy and robustness of the monitoring ball in identifying construction violations.

[0093] Alternatively, the following is a possible implementation method for monitoring construction violations by workers on the work surface. Figure 1 The sub-steps of step S120 may include: Step S120-9: If the target work surface is the work surface of the construction personnel, determine the target construction personnel from all the construction personnel identified in the panoramic image.

[0094] In this embodiment of the invention, when it is determined that the current target work surface is the work surface of the construction personnel, the construction personnel identified from the panoramic image are sequentially determined as the target construction personnel, and each target construction personnel is continuously tracked and monitored.

[0095] It should be noted that construction workers refer to those who are not working at heights and are wearing safety equipment. In other words, on a construction site, people who are not wearing safety equipment are not considered construction workers, and workers on power transmission towers are not considered construction workers, while workers on the ground who are wearing safety equipment are considered construction workers.

[0096] Step S120-10: Adjust the direction of the PTZ and the zoom of the PTZ camera according to the construction personnel information corresponding to the target construction personnel, so that the target construction personnel are in the monitoring screen of the PTZ camera.

[0097] Step S120-11: Control the PTZ camera to scan the target construction workers step by step, acquire multiple snapshot images of the target construction workers, and identify the work clothing information and fence crossing detection information of the target construction workers based on the snapshot images.

[0098] In this embodiment of the invention, the direction of the gimbal and the zoom of the gimbal camera are adjusted based on the construction personnel detection frame in the construction personnel information corresponding to the target construction personnel. The gimbal camera is used to continuously scan the target construction personnel according to a preset time to capture multiple images of the target construction personnel.

[0099] By utilizing a built-in image model to detect local features (such as head and waist) of workers in captured images, the system determines whether the target construction workers are wearing safety helmets or safety belts, thus obtaining the workers' work attire information. Based on the target construction workers and the fence, target detection is performed on the captured images to obtain information on whether the target construction workers crossed the fence.

[0100] Step S120-12: If the target construction worker's work attire information indicates that the target construction worker is not wearing safety equipment or the target construction worker's fence crossing detection information indicates that the target construction worker has crossed the fence, it is determined that the target construction worker has violated construction regulations.

[0101] In this embodiment of the invention, if the work attire information shows that the target construction worker is not wearing safety equipment, or if the fence crossing detection information indicates that the target construction worker has crossed the fence, then the target construction worker is determined to have committed a construction violation, and the captured image of the target construction worker is identified as an abnormal image. An alarm is issued to the construction site at the target construction worker's work area based on the abnormal image. Simultaneously, the abnormal image is uploaded to the artificial intelligence platform server (i.e., the safety supervision center) so that the artificial intelligence server can perform secondary verification and analysis. The results of this secondary verification and analysis are presented to the supervisors for final confirmation, which improves efficiency and achieves comprehensive monitoring and analysis coverage even with limited computing resources on the artificial intelligence server.

[0102] It should be understood that if at least one construction worker in the panoramic image exhibits a construction violation, then the construction worker's work surface (i.e., the target work surface) is determined to have a construction violation. If none of the construction workers in the panoramic image exhibit a construction violation, then the construction worker's work surface (i.e., the target work surface) is determined to have no construction violation.

[0103] As can be seen, the embodiments of the present invention utilize a gimbal camera to automatically locate, capture images, and analyze the behavior of each construction worker on the work surface, in order to determine whether the construction workers have violated construction regulations by removing safety equipment or crossing fences. This achieves automated supervision of the construction workers' work behavior, improves the autonomous identification and rain warning capabilities of the control ball in ordinary ground operation scenarios, and effectively ensures the safety management efficiency of the construction site.

[0104] Optionally, the fence crossing detection information includes a person frame and a fence frame. The following is a possible implementation method for identifying fence crossing violations. The method also includes the following steps: If the person frame and the fence frame intersect, the behavior of crossing the fence is detected based on the head depth value of the head region in the person frame and the fence depth value of the fence frame; the fence depth value is determined based on the depth value of the interval with the highest distribution concentration in the fence frame.

[0105] In this embodiment of the invention, if only the frame of the target construction worker is detected in the captured image, it is directly determined that the target construction worker did not cross the fence. If both the frame of the target construction worker and the fence frame are detected in the captured image, it is determined whether there is an overlapping area between the frame and the fence frame in the two-dimensional image space. Here, the frame of the worker refers to the bounding box of the target construction worker in the captured image identified by the target detection algorithm, and the fence frame refers to the bounding box of the fence area in the captured image.

[0106] If the person frame and the fence frame overlap in two-dimensional space, the head depth value of the head region in the person frame and the fence depth value of the corresponding fence frame are further extracted. The fence depth value is not directly taken as the average depth of all pixels within the fence frame, but is determined based on the depth value of the interval with the highest depth distribution concentration within the fence frame. This method effectively eliminates interference from fence edges or background, improving the accuracy of depth calculation.

[0107] Calculate the depth difference between the head depth value and the fence depth value. If the depth difference is less than or equal to the preset fence depth threshold, it is determined that there is a fence crossing behavior; otherwise, it is determined that there is no fence crossing behavior.

[0108] It should be understood that the head depth value reflects how close the construction worker's body is to the camera, while the fence depth value represents the spatial position of the fence plane. When the difference between the two depths is less than or equal to the preset fence depth threshold, it indicates that the target construction worker's body has crossed the plane of the fence, that is, the behavior of crossing the fence has occurred; otherwise, it is considered that he has not crossed the fence.

[0109] As can be seen, the embodiments of the present invention effectively improve the accuracy of fence crossing behavior recognition by combining the spatial intersection judgment of the person frame and the fence frame in the two-dimensional image with the three-dimensional depth information analysis through a dual judgment mechanism. This avoids the risk of misjudgment caused by relying solely on image coordinate judgment and provides reliable technical support for the safety supervision of ordinary work surfaces at construction sites.

[0110] Optionally, in practical applications, there may be situations where the panoramic camera does not scan the specified type of target object. Regarding how to utilize a gimbal camera to further monitor construction violations at non-high-altitude construction sites, the following is a possible implementation method. Please refer to... Figure 2 The method may also include the following steps: Step S130: If the target detection result does not contain any target object type, control the gimbal camera to scan the non-high-altitude construction site, obtain multiple snapshot images of the non-high-altitude construction site, and identify whether construction personnel exist based on the snapshot images of the non-high-altitude construction site.

[0111] In this embodiment of the invention, if the target detection result does not contain any target object type, it means that no target object was detected in the panoramic image, and the "No workers in the vicinity" label is superimposed on the panoramic image.

[0112] When no target object is detected in the panoramic image, it indicates that there is no clearly delineated work area in the panoramic image. At this time, the PTZ camera controlled by the control ball actively scans the non-high-altitude construction site area (i.e., the ordinary work area) around the perimeter and continuously acquires multiple snapshot images of this area as the data source for subsequent analysis. After acquiring the snapshot images, target detection is performed on the snapshot images to identify whether construction personnel are present.

[0113] Step S140: If no construction personnel are present, it is determined that there is a construction violation.

[0114] In this embodiment of the invention, if no construction workers are identified from the captured image, it is determined that there is a construction violation of "no workers present", the captured image is identified as an abnormal image, and a "no workers present in the vicinity" label is superimposed on the abnormal image.

[0115] Step S150: If construction workers are present, the system identifies whether they are wearing safety equipment and whether they are crossing the fence based on the captured images of the non-high-altitude construction site.

[0116] Step S160: If the construction workers are not wearing safety equipment or are crossing the fence, it is determined that there is a construction violation.

[0117] In this embodiment of the invention, if construction workers are present, their work behavior is analyzed in depth to determine whether they have removed their safety equipment or crossed fences. If it is detected that a construction worker has not worn safety equipment throughout the entire process or has crossed fences, a construction violation is determined, and an alarm mechanism is triggered to upload the abnormal image to the artificial intelligence platform server. This allows the artificial intelligence platform server to perform secondary verification and analysis of the abnormal image, improving efficiency and achieving comprehensive monitoring and analysis coverage even with limited computing resources on the artificial intelligence server.

[0118] As can be seen, by introducing an active scanning and in-depth analysis process for "blank scenes without workers in a panoramic view", the embodiments of the present invention effectively expand the monitoring coverage of the surveillance ball in complex environments and improve the autonomous judgment capability and safety supervision integrity in the case of obvious target objects.

[0119] Based on the same inventive concept, the basic principle and technical effects of the multi-workface monitoring device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments.

[0120] Please refer to Figure 3 , Figure 3This is a block diagram of a multi-workface monitoring device 200 provided in an embodiment of the present invention. The multi-workface monitoring device 200 is applied to a control ball and includes a detection module 201 and a monitoring module 202.

[0121] The detection module 201 is used to identify panoramic images according to multiple target object types and obtain target detection results. The monitoring module 202 is used to determine the work surface corresponding to the target object of the same target object type in the target detection result as the target work surface if the target detection result contains at least one target object type; the work surfaces corresponding to target objects of different target object types are different; control the pan-tilt camera to track the target work surface and monitor whether there are any construction violations on the target work surface.

[0122] In summary, the multi-workface monitoring device provided in this embodiment of the invention identifies multiple target object types in a panoramic image using a PTZ camera. Based on the target detection results, it divides the spatial range of the same type of target object into corresponding workfaces. Due to the different operational behavior characteristics of different target object types, each type of workface has its own monitoring focus and logical processing path. The workfaces divided according to the panoramic image are sequentially determined as the target workfaces to be monitored. The PTZ camera controls the PTZ camera to automatically patrol and monitor each target workface to continuously monitor whether there are any construction violations on multiple workfaces. Without manual intervention, the PTZ camera automatically identifies multiple workfaces using target object detection and completes comprehensive monitoring of multiple work scenarios, thereby effectively solving the monitoring blind spot problem of a single PTZ camera in multi-workface scenarios and improving the overall safety supervision efficiency.

[0123] Optionally, the target object types include transmission tower types, heavy equipment types, and construction worker types, where construction workers are non-high-altitude workers wearing safety equipment. The detection module 201 is specifically used to detect whether transmission towers, heavy equipment, or construction workers exist in the panoramic image using a pre-trained target detection model; if transmission towers exist, the transmission tower type and tower information of at least one transmission tower in the panoramic image are added to the target detection result; if heavy equipment exists, the heavy equipment type and equipment information of at least one heavy equipment in the panoramic image are added to the target detection result; if construction workers exist, the construction worker type and construction worker information of at least one construction worker in the panoramic image are added to the target detection result.

[0124] Optionally, the monitoring module 202 is specifically configured to: if the target detection result contains only the type of transmission tower, determine the transmission tower working surface as the target working surface; the transmission tower working surface includes all working surfaces corresponding to all transmission towers identified in the panoramic image; if the target detection result contains only the type of heavy equipment, determine the heavy equipment working surface as the target working surface; the heavy equipment working surface includes all working surfaces corresponding to all heavy equipment identified in the panoramic image; if the target detection result contains only the type of construction personnel, determine the construction personnel working surface as the target working surface; the construction personnel working surface includes all working surfaces corresponding to all construction personnel identified in the panoramic image; if the target detection result contains both the type of transmission tower and the type of heavy equipment, determine the working surface corresponding to the type of transmission tower and the working surface corresponding to the type of heavy equipment as the target working surface, respectively.

[0125] Optionally, the monitoring module 202 is specifically used to: if the target work area is a power transmission tower work area, determine the target tower from all power transmission towers identified in the panoramic image; adjust the direction of the pan-tilt unit and the zoom level of the pan-tilt camera according to the tower information corresponding to the target tower so that the target tower is located in the monitoring frame of the pan-tilt camera; control the pan-tilt camera to gradually scan the target tower and acquire multiple captured images of the target tower; identify the number of workers and their work attire information on the target tower based on the captured images; if the number of workers on the target tower exceeds the number threshold or the work attire information indicates that the workers on the target tower are not wearing safety equipment, determine that there is a construction violation on the target tower.

[0126] Optionally, the monitoring module 202 is specifically used to: if the target work surface is a heavy equipment work surface, determine the target equipment from all heavy equipment identified in the panoramic image; adjust the direction of the pan-tilt unit and the zoom level of the pan-tilt camera according to the equipment information corresponding to the target equipment so that the target equipment is located in the detection field of the pan-tilt camera; control the pan-tilt camera to gradually scan the target equipment, acquire multiple captured images of the target equipment, and identify the boom detection frame, relative depth of the boom, personnel detection frame and / or relative depth of the personnel based on the captured images; and identify whether there are any construction violations on the target work surface based on the boom detection frame, relative depth of the boom, personnel detection frame and / or relative depth of the personnel.

[0127] Optionally, the monitoring module 202 is specifically used to determine whether the boom detection frame and the personnel detection frame of the target equipment intersect if a personnel detection frame exists; if the boom detection frame and the personnel detection frame of the target equipment intersect and the difference between the boom depth value and the personnel depth value corresponding to the intersecting personnel detection frame is less than the depth threshold, it is determined that there is a construction violation on the target work surface.

[0128] Optionally, the monitoring module 202 is specifically used to: if the target work surface is a construction worker's work surface, identify the target construction worker from all construction workers identified in the panoramic image; adjust the direction of the pan-tilt unit and the zoom level of the pan-tilt camera according to the construction worker information corresponding to the target construction worker, so that the target construction worker is located in the monitoring frame of the pan-tilt camera; control the pan-tilt camera to gradually scan the target construction worker, acquire multiple captured images of the target construction worker, and identify the target construction worker's work attire information and fence crossing detection information based on the captured images; if the target construction worker's work attire information indicates that the target construction worker is not wearing safety equipment or the target construction worker's fence crossing detection information indicates that the target construction worker has crossed the fence, determine that the target construction worker has committed a construction violation.

[0129] Optionally, the monitoring module 202 is specifically used to detect fence crossing behavior if the person frame and the fence frame intersect, based on the head depth value of the head region in the person frame and the fence depth value of the fence frame; the fence depth value is determined based on the depth value of the interval with the highest distribution concentration in the fence frame.

[0130] Optionally, the monitoring module 202 is specifically used to control the pan-tilt camera to scan the non-high-altitude construction site and acquire multiple snapshot images of the non-high-altitude construction site if the target detection result does not contain any target object type. Based on the snapshot images of the non-high-altitude construction site, it identifies whether construction personnel are present. If no construction personnel are present, it determines that there is a construction violation. If construction personnel are present, it identifies whether the construction personnel are wearing safety equipment and whether they have crossed the fence based on the snapshot images of the non-high-altitude construction site. If the construction personnel are not wearing safety equipment or have crossed the fence, it determines that there is a construction violation.

[0131] Optionally, the multi-workface monitoring device 200 also includes a transmission module 203, which is used to upload abnormal images to the artificial intelligence platform server so that the artificial intelligence platform server can perform secondary verification and analysis on the abnormal images.

[0132] As one possible implementation, the multi-workface monitoring device includes a detection module, a monitoring module, and a transmission module. The detection module includes a multi-view panoramic fixed-focus camera video acquisition and AI module. This module achieves comprehensive 360° horizontal field of view coverage through multiple fixed-focus cameras installed on the base of the control sphere. To reduce the complexity of video viewing for inspectors, the multiple cameras are stitched together using software technology to display a single 360° view or two 180° views. Simultaneously, the panoramic fixed-focus camera video acquisition and AI module incorporates AI computing power and algorithms, enabling it to automatically identify the transmission towers and the relative depths of multiple transmission towers.

[0133] The monitoring module includes a gimbal camera zoom video acquisition and AI module and a gimbal control and motion module. The gimbal camera zoom video acquisition and AI module controls the optical zoom gimbal camera using an autofocus algorithm, and combines this with a built-in computing power algorithm for personnel recognition, locking the target personnel within a suitable proportion of the frame for video and image acquisition. Simultaneously, the built-in algorithm monitors and judges in real time whether the personnel are operating according to regulations. The gimbal control and motion module uses a microcontroller, drive control circuit, and horizontal and vertical motor transmission system to achieve the horizontal and vertical movement of the control ball.

[0134] The transmission module includes a main control communication module, which interacts with the AI ​​platform server via 4G or 5G communication. This includes interface and parsing of the motor B interface or GB28181 protocol, as well as encrypted interface communication. The AI ​​platform server can control and operate the surveillance camera through the main control communication module, while also coordinating the interaction between various modules and managing power distribution.

[0135] Please refer to Figure 4 , Figure 4 This is a block diagram of a deployment ball 300 provided in an embodiment of the present invention. The deployment ball 300 includes a processor 310 and a memory 320. The memory 320 can implement the multi-workface monitoring method disclosed in the above embodiments under the control of the processor 310. This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor 310, implements the multi-workface monitoring method disclosed in the above embodiments.

[0136] This invention provides a program product that, when executed by processor 310, implements the multi-workface monitoring method disclosed in the above embodiments.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0138] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0139] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring multiple work surfaces, characterized in that, Applied to a control ball, the method includes: The panoramic image is identified according to multiple target object types to obtain target detection results; If the target detection result contains at least one target object type, the work surface corresponding to the target object belonging to the same target object type in the target detection result is determined as the target work surface; different target object types correspond to different work surfaces. Control the pan-tilt camera to track the target work surface and monitor whether there are any construction violations on the target work surface.

2. The multi-workface monitoring method according to claim 1, characterized in that, The target object types include transmission tower types, heavy equipment types, and construction personnel types; The construction workers are those wearing safety equipment and not engaged in high-altitude work; the panoramic image is identified according to multiple target object types to obtain target detection results, including: The pre-trained target detection model is used to detect whether there are power transmission towers, heavy equipment, or construction workers in the panoramic image. If the transmission tower exists, add the type of the transmission tower and the tower information of at least one transmission tower in the panoramic image to the target detection result; If the heavy equipment exists, add the type of the heavy equipment and the equipment information of at least one heavy equipment in the panoramic image to the target detection result; If the construction worker exists, the construction worker type and the construction worker information of at least one construction worker in the panoramic image are added to the target detection result.

3. The multi-workface monitoring method according to claim 2, characterized in that, If the target detection result contains at least one target object type, the work surface corresponding to the target object belonging to the same target object type in the target detection result is determined as the target work surface, including: If the target detection result only contains the transmission tower type, the transmission tower working surface is determined as the target working surface; the transmission tower working surface includes the working surfaces corresponding to all transmission towers identified in the panoramic image; If the target detection result contains only the heavy equipment type, the heavy equipment work surface is determined as the target work surface; the heavy equipment work surface includes all work surfaces corresponding to heavy equipment identified in the panoramic image; If the target detection result contains only the construction worker type, the construction worker's work surface is determined as the target work surface; the construction worker's work surface includes the work surfaces corresponding to all construction workers identified in the panoramic image; If the target detection result includes the transmission tower type and the heavy equipment type, the working surface corresponding to the transmission tower type and the working surface corresponding to the heavy equipment type are respectively determined as the target working surface.

4. The multi-workface monitoring method according to claim 3, characterized in that, The control pan-tilt camera tracks the target work surface and monitors whether there are any construction violations on the target work surface, including: If the target working surface is the working surface of the transmission tower, the target tower is determined from all the transmission towers identified in the panoramic image; Adjust the direction of the gimbal and the zoom level of the gimbal camera according to the tower information corresponding to the target tower, so that the target tower is located in the monitoring screen of the gimbal camera; The gimbal camera is controlled to scan the target tower step by step, and multiple images of the target tower are captured. Based on the captured images, the number of workers and their work attire information on the target tower are identified. If the number of workers on the target tower exceeds the number threshold or the work attire information indicates that the workers on the target tower are not wearing safety equipment, it is determined that there is a construction violation on the target tower.

5. The multi-workface monitoring method according to claim 3, characterized in that, The control pan-tilt camera tracks the target work surface and monitors whether there are any construction violations on the target work surface, including: If the target work surface is the work surface of the heavy equipment, the target equipment is determined from all the heavy equipment identified in the panoramic image; Adjust the direction of the gimbal and the zoom level of the gimbal camera according to the device information corresponding to the target device, so that the target device is located in the detection field of the gimbal camera; The gimbal camera is controlled to scan the target device step by step to acquire multiple captured images of the target device. Based on the captured images, the boom detection frame, the relative depth of the boom, the personnel detection frame, and / or the relative depth of the personnel of the target device are identified. The presence of construction violations on the target work surface is identified based on the target equipment's boom detection frame, the target equipment's boom relative depth, the personnel detection frame, and / or the personnel relative depth.

6. The multi-face monitoring method according to claim 5, characterized in that, The step of identifying whether there are construction violations on the target work surface based on the target equipment's boom detection frame, the target equipment's boom relative depth, the personnel detection frame, and / or the personnel relative depth includes: If the personnel detection frame exists, determine whether the crane detection frame of the target equipment and the personnel detection frame intersect; If the boom detection frame of the target equipment intersects with the personnel detection frame, and the difference between the relative depth of the boom and the relative depth of the personnel corresponding to the intersecting personnel detection frame is less than a depth threshold, it is determined that there is a construction violation at the target work surface.

7. The multi-workface monitoring method according to claim 3, characterized in that, The control pan-tilt camera tracks the target work surface and monitors whether there are any construction violations on the target work surface, including: If the target work area is the work area of ​​the construction personnel, the target construction personnel are determined from all the construction personnel identified in the panoramic image; Adjust the direction of the gimbal and the zoom level of the gimbal camera according to the construction personnel information corresponding to the target construction personnel, so that the target construction personnel are located in the monitoring screen of the gimbal camera; The PTZ camera is controlled to scan the target construction worker step by step, and multiple images of the target construction worker are captured. Based on the captured images, the work clothing information and fence crossing detection information of the target construction worker are identified. If the work attire information of the target construction worker indicates that the target construction worker is not wearing safety equipment, or if the fence crossing detection information of the target construction worker indicates that the target construction worker has crossed the fence, it is determined that the target construction worker has committed a construction violation.

8. The multi-workface monitoring method according to claim 7, characterized in that, The fence crossing detection information includes a person frame and a fence frame, and the method further includes: If the person frame and the fence frame intersect, the behavior of crossing the fence is detected based on the head depth value of the head region in the person frame and the fence depth value of the fence frame; the fence depth value is determined based on the depth value of the interval with the highest distribution concentration in the fence frame.

9. The multi-face monitoring method according to claim 2 or 3, characterized in that, The method further includes: If the target detection result does not include any of the target object types, control the gimbal camera to scan the non-high-altitude construction site, obtain multiple snapshot images of the non-high-altitude construction site, and identify whether there are construction personnel based on the snapshot images of the non-high-altitude construction site. If no construction workers are present, then a construction violation is deemed to have occurred. If construction workers are present, the system identifies whether they are wearing safety equipment and whether they are crossing the fence based on the captured images of the non-high-altitude construction site. If the construction workers are not wearing safety equipment or cross the fence, it is determined that there is a construction violation.

10. A multi-workface monitoring device, characterized in that, Applied to a control ball, the device includes: The detection module is used to identify panoramic images according to multiple target object types and obtain target detection results; The monitoring module is used to determine the work surface corresponding to the target object of the same target object type in the target detection result as the target work surface if the target detection result contains at least one target object type; the work surfaces corresponding to target objects of different target object types are different; control the pan-tilt camera to track the target work surface and monitor whether there are any construction violations on the target work surface.

11. A control ball, characterized in that, It includes a processor and a memory, the memory storing a computer program that can be executed by the processor to implement the multi-workface monitoring method according to any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-workface monitoring method as described in any one of claims 1-9.

Images