Intelligent mobile control ball video monitoring direction automatic adjustment method

Abstract

This invention relates to an automatic direction adjustment method for intelligent mobile PTZ video surveillance, comprising the following steps: acquiring images of the work site; determining the priority of the work area in the image, only checking if it is the highest priority; if so, no adjustment is needed; otherwise, finding the optimal work area; rotating the PTZ lens according to a preset rotation rule, capturing several images for target feature recognition, combining the target features in each image to determine the current work scene type, and calculating the score of the work area contained in each image according to a formula, selecting the image with the highest score as the optimal work area; rotating the PTZ lens to the lens angle corresponding to the image with the highest score, aligning it with the optimal work area. The advantages of this invention are: timed triggering of PTZ rotation, capturing images of the work site from different directions, performing target feature recognition and analysis of the work site, selecting the optimal work area at the current site, and achieving automatic focusing of the PTZ video surveillance image.

Description

Technical Field

[0001] This invention relates to the field of on-site security monitoring, and in particular to a method for automatically adjusting the direction of intelligent mobile surveillance PTZ video monitoring. Background Technology

[0002] With the steady advancement of new power system construction, the power grid structure is becoming increasingly complex. Power grid construction sites are numerous and widespread, with complex and ever-changing environments, varying personnel quality and skill levels, and a complex interplay of risks, hazards, and accidents, making on-site safety risk prevention and control extremely difficult. Therefore, "cloud-based intelligent analysis + remote supervision" has become a crucial means of on-site safety management. This involves deploying mobile surveillance cameras and other video monitoring equipment at the work site to collect real-time video data and transmit it to a cloud platform. Then, firstly, violation identification models are deployed in the cloud for violations such as the absence of safety fences or the failure to wear insulated gloves during voltage testing, using the video and images collected by the surveillance cameras for violation analysis and alarms. Secondly, safety inspectors can remotely review on-site videos to supervise the implementation of safety measures and the compliance of personnel's work behaviors. However, to avoid remote video monitoring by safety inspectors, on-site workers intentionally point the surveillance cameras towards the sky, walls, or other non-work areas, or obstruct them, resulting in a large amount of "invalid" video footage being received by the cloud. This affects the effectiveness of intelligent violation identification and requires remote inspectors to manually adjust the monitoring angle to promptly understand the actual on-site work situation.

[0003] Currently, there are numerous patents related to automatic target tracking and intelligent identification of violations by surveillance cameras at work sites. A smaller number of patents combine third-party devices such as positioning terminals to automatically adjust the optimal shooting angle of the surveillance camera, maximizing the number of targets tracked on-site. For example, the invention patent CN118972700A, "A Target Tracking Method, Device, and Computer Medium for Surveillance Cameras," converts the personnel location coordinates collected by the positioning terminal worn by on-site workers into surveillance camera coordinates. Then, using the center of the surveillance camera as the origin, it calculates the optimal horizontal and vertical rotation angles of the camera with the highest number of target locations within the camera's field of view, thereby maximizing the number of work targets tracked by the surveillance camera. This method relies on the positioning terminal to provide on-site personnel location information for target identification, and its main purpose is to ensure that as many work targets as possible appear within the surveillance camera's monitoring field of view. However, in actual power work sites, the area with the most workers is not necessarily the highest priority monitoring area. For example, in scenarios such as high-altitude operations and hoisting operations, the surveillance camera should prioritize monitoring the movement of workers and suspended loads on the towers. Therefore, the method proposed in this patent has high preconditions and a limited scope of application. For example, the invention patent CN113593177A, "A Video Alarm Linkage Implementation Method Based on High-Precision Positioning and Image Recognition," uses an edge computing device as its core and BeiDou high-precision positioning technology to realize a digital electronic fence. It uses image analysis to identify violations such as not wearing a safety helmet or safety belt, as well as the identification of reference points. When a violation occurs, it can automatically and promptly control the monitoring ball to move to the location of the violating worker based on high-precision positioning data, meeting the requirements for tracking and evidence collection. This method mainly relies on positioning data and uses the edge computing device to adjust the monitoring direction of the monitoring ball to the location of the violation for evidence collection, but it does not solve the problem of whether the monitoring ball is aligned with the optimal work area.

[0004] In summary, current patents related to intelligent control of surveillance cameras mainly focus on dynamic tracking of on-site work targets. These patents achieve dynamic target tracking by using pure visual image analysis algorithms or by adjusting the monitoring angle of the surveillance camera in conjunction with a positioning terminal. However, no related patents have been found to enable the surveillance camera to automatically adjust its monitoring angle to find the optimal working surface and improve the monitoring effectiveness of the surveillance camera. Summary of the Invention

[0005] To address the aforementioned issues, the present invention aims to provide an intelligent mobile surveillance sphere video monitoring direction automatic adjustment method. By analyzing the target characteristics at the power operation site, a site feature recognition model is constructed. Based on the feature recognition model, the video stream captured by the surveillance sphere is frame-by-frame analyzed. Combined with the automatic focusing strategy of the surveillance sphere video image, the optimal work surface is calculated and the angle of the surveillance sphere is adjusted to focus on the work surface, thereby achieving automatic focusing of the on-site surveillance sphere video monitoring image.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A method for automatically adjusting the direction of intelligent mobile PTZ video surveillance includes the following steps: Step 1: Obtain images of the work site: Extract images of the work site from the video stream of the surveillance camera in the form of frame-by-frame extraction; Step 2: Determine the priority of the work area in the image: Use the work site feature recognition model to identify target features in the image, combine the identified target features with the predetermined work area priority feature rules to determine whether it is the highest priority work area. If yes, end the automatic adjustment task; otherwise, proceed to step 3. Step 3: Find the optimal work area: Rotate the control ball lens according to the preset rotation rules, capture several images for target feature recognition, combine the target features in each image to determine the current work scene type, and calculate the work area score contained in each image according to the predetermined formula. The image with the highest score is taken as the optimal work area. Step 4: Orient the control ball lens toward the lens angle corresponding to the image with the highest score, and align it with the optimal working area.

[0007] The present invention has the following beneficial effects: 1. This invention proposes an automatic adjustment method for the direction of intelligent mobile surveillance sphere video monitoring. According to a predetermined strategy, the sphere is triggered to rotate horizontally and vertically at regular intervals to capture images of the work site from different directions. The method then identifies and analyzes the characteristic targets of the work site, selects the optimal work area based on the predetermined work area priority and the image work area score calculation formula, and adjusts the sphere lens to align with the optimal work area. This solves the problem that the sphere may capture a large number of "invalid" video images at power work sites due to subjective or objective factors, which affects the recognition effect of the intelligent violation recognition model.

[0008] 2. Before focusing on the optimal working area, this invention diagnoses the quality of the video images captured by the control ball, identifying phenomena such as images that are too dark, black and white images, stripe interference, blurred images, overly bright images, and image occlusion, thereby avoiding invalid rotation of the control ball, reducing the use of computing resources by the control ball, and improving the battery life of the control ball.

[0009] 3. This invention proposes a scoring formula for the work area image based on the target weight coefficient of "operator characteristics + work scene characteristics". It calculates and scores targets under the same scene type and sorts them to obtain the optimal work area image under the same scene. Compared with a single judgment method based on work scene priority, the accuracy of optimal work area recognition is improved by more than 30%.

[0010] 4. After adjusting the control ball to align with the work area, this invention proposes a control ball fine-tuning method to center the video image of the work target, so that the key work target can always be presented in the control ball image, enabling continuous monitoring of the work target and improving the visibility of the on-site image. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a schematic diagram of the horizontal rotation of the control ball according to the present invention; Figure 3 This is a schematic diagram of the vertical rotation of the control ball according to the present invention; Figure 4 This is a schematic diagram showing the target of the operation being adjusted to the center of the screen according to the present invention. Detailed Implementation

[0012] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] See Figure 1 A method for automatic direction adjustment of intelligent mobile PTZ video surveillance, comprising the following steps: Step 1: Extract images of the work site from the video stream of the surveillance camera. For example, extract N frames (N is configurable, e.g., 10 frames) at 1-second intervals from the surveillance camera's video stream. Then, send the extracted N frames to the video quality diagnostic model for analysis. The diagnostic results are then statistically analyzed. If 2 / 3 of the images are abnormal, the current surveillance camera video is considered abnormal, a warning is issued, the current adjustment is stopped, and the process waits for the next adjustment. If the current surveillance camera video is normal, proceed to Step 2.

[0014] By collecting typical abnormal images from mobile surveillance cameras used in power operation sites, such as images that are too dark, black and white, striped interference, blurred, too bright, and obstructed, a video quality diagnostic model was trained using the MobileNet network architecture. In real-world scenarios, machine learning or deep learning algorithms such as SVM and ResNet can be selected to train the video quality diagnostic model, depending on factors such as the computing power of the surveillance camera, sample collection conditions, and accuracy.

[0015] Images extracted from the video stream of the surveillance camera are fed into a video quality diagnostic model for analysis. If the diagnostic results include one or more of the following: image too dark, black and white image, stripe interference, image blur, image too bright, or image occlusion, then the current surveillance camera video is determined to have an anomaly. The time of image acquisition, anomaly type, and image itself are uploaded to the cloud platform (mobile surveillance cameras used in power work sites have built-in power-specific 4G / 5G SIM cards, enabling communication with the power intranet and data interaction between the cloud and the surveillance camera). Simultaneously, a voice alarm is triggered on the surveillance camera, such as "The surveillance camera image appears to be abnormal; please have the person in charge check and adjust it promptly," reminding the person in charge to check and adjust the surveillance camera accordingly.

[0016] Step 2: Use the on-site feature recognition model to identify target features. Combine the identified target features with the predetermined work area priority feature rules to determine whether it is the highest priority work area. If yes, end the automatic adjustment task; otherwise, proceed to Step 3. The work area priority feature rules include the work scenario type, priority level, and related target features. The list of on-site target features is shown in Table 1, and the list of work area monitoring priorities is shown in Table 2.

[0017] Table 1:

[0018] Table 2:

[0019] In this embodiment, the optimal work area and the highest priority work area are defined as follows: Optimal work area: Power work sites are numerous and widespread, often involving multiple operations simultaneously, such as working at heights and working on the ground. Different types of operations have different potential risk levels, with working at heights often posing a greater risk than working on the ground. The optimal work area for a work site is determined based on characteristics such as the level of operational risk and the density of workers. N types of work areas are categorized and assigned priority. The optimal work area is then determined based on the highest comprehensive score, according to the actual conditions of the work site.

[0020] Highest Priority Work Area: Work areas with a priority of 1, including areas for work at height. Personal safety is the primary factor in on-site safety management; therefore, this invention designates tower climbing, work inside boom lifts, and work on ladders—work areas requiring personnel to work at height—as the highest priority work areas. When this type of work occurs, the surveillance camera should be pointed at this work area in real time.

[0021] In addition, due to the complex environment of power operation sites, it is sometimes impossible to find a control point that can cover the entire operation area. Furthermore, due to factors such as the control distance and height of the control ball, as well as obstruction, the feature targets in the operation site images collected by the control ball may not be complete. In addition, the intelligent recognition model may have false alarms and missed alarms. Therefore, ordinary operation scenarios are set into three categories to ensure that the control ball can still focus on the effective operation area in complex environments, rather than aiming at non-operation areas such as the sky and grass.

[0022] Step 3: Finding the optimal work area. Due to the diverse types of work at power sites, multiple work areas may exist within the same site, and a single work area may encompass various types of work, such as ground-based work and work at heights. Images captured by the monitoring sphere during its rotation at a certain angle may cover the same work type, or a single image may cover multiple work types. Therefore, this invention uses a comprehensive scoring method to find the optimal work area, which improves the model's fault tolerance and accuracy. Specifically: Image capture: The control lens is rotated according to a preset rotation rule to capture several images for target feature recognition. The target features in each image are combined to determine the current operation scene type. The score of the operation area contained in each image is calculated according to a predetermined formula, and the image with the highest score is taken as the optimal operation area.

[0023] The specific steps for rotating the control PTZ camera according to the preset rotation rules are as follows: First, rotate the control ball horizontally 360°, with each rotation step being 60°: (Settings) The range of values ​​is [ , The corresponding rotation angle range is [ , ]; The range of values ​​is [ , The corresponding rotation angle range is [ , Using the current orientation of the PT (Positioning Point) camera as the rotation base point, the current PT value is set to... , The corresponding horizontal and vertical angle values ​​are set to , The calculation formula is as follows: ; ; Horizontal angle exist Based on this, rotate one full circle in increments of 60°, vertical angle Keeping the position constant, capture one image of the operation at each 60° horizontal rotation, and record the orientation angle of the control ball corresponding to the image. A total of 6 images are captured. The formulas for calculating the horizontal and vertical angles after each rotation are as follows: ; ; In the formula, i takes the value [1, 6]. When i = 1, When horizontally continuously superimposed during a 60° rotation, Exceeding [ , When within the range, for Adjustments were made to ensure that its value falls within [ , Within the specified range, the adjustment formula is as follows: ; mod means taking the remainder, which is the remainder after dividing by 360°.

[0024] The formulas for calculating the horizontal and vertical angles for each image are as follows: ; In the formula, i takes the value [1, 6], and the formula for calculating the PT value corresponding to i is as follows: ; ; The PTZ parameters are changed by calling the pan-tilt-zoom (PTZ) control interface provided by the surveillance camera, where the parameters are continuously changed. , The Z parameter remains constant, thus controlling the rotation of the control ball.

[0025] Calculate the work area score: The 12 captured images are sequentially fed into the work site feature recognition model for feature recognition. The types of recognized target features include worker features and work scene features. The current work scene type is determined by combining the target features in each image. If a highest priority work area is found, step 4 is executed. If no highest priority work area is found, the work area score is calculated for each image separately. The calculation formula is as follows: ; In the formula, The score is given to the job area covered by the i-th image. The weight coefficient value of the m-th target feature among the identified worker characteristics. The total number of the m-th target feature among the worker characteristics. The weight coefficient value of the nth target feature in the identified work scene features. This represents the total number of the nth target feature identified from the characteristics of the work scene. Determine the image number corresponding to the highest-scoring task area: Calculate the image with the highest score in the task area based on the above formula, i.e. , where t is the image number with the highest score in the task area, with a value of [1, 12].

[0026] Step 4: Obtain the angle of the PTZ camera when acquiring the image with that image number. Rotate the PTZ camera to the desired angle. That is, to align with the optimal work area.

[0027] Step 5: Fine-tuning the image of the optimal work area. When the PTZ camera is turned to the optimal work area, if key targets such as on-site personnel and large machinery are located at the edge of the video frame rather than in the center, the monitoring effect will be affected. Therefore, it is necessary to fine-tune the video image of the focused work area by adjusting the PTZ parameters of the PTZ camera. The specific steps are as follows: Step 51: When setting the image offset of the control ball, calculate the linear relationship between image pixels and PTZ parameters, and determine the P value P1 corresponding to a 1px horizontal movement of the lens and the T value T1 corresponding to a 1px vertical movement, as follows: Set the resolution of the video feed captured by the surveillance camera to w h refers to the number of pixels on the horizontal and vertical axes, measured in pixels (px).

[0028] First, calculate the P value corresponding to a horizontal movement of 1px. (See also...) Figure 2 Assuming the target A's position remains unchanged, the control ball's TZ value remains constant, only the P value is changed, controlling the control ball to rotate horizontally to the left. The P value of the target A's initial position A1 is denoted as... The P value at the target end position A2 is denoted as The formula for calculating the P-value corresponding to a horizontal movement of 1px by the control ball is as follows: .

[0029] Then, calculate the T value corresponding to a vertical movement of 1px. For example... Figure 3 As shown: Assuming the target A's position remains unchanged, the control ball's PZ value remains constant, only the T value is changed, controlling the control ball to rotate vertically upwards. The T value for the initial target position A1 is denoted as... The T value at the target end position A2 is denoted as The formula for calculating the T value corresponding to a horizontal movement of 1px by the control ball is as follows: ; Step 52: Calculate the PT parameter offset of the control ball to adjust the on-site operation target to the center area of ​​the screen. For example... Figure 4 As shown, assuming target A needs to be moved to the center point O of the image, target A here could actually be a single target such as a worker or a crane, or it could be a set of targets composed of multiple targets such as workers and cranes. The power operation site feature recognition model mentioned above can identify feature targets such as workers and cranes on site, and return the top-left corner coordinates (x, y) and size (w, h) of the target on the image.

[0030] Given the pixel coordinates of feature target A in the image ( , The size of feature target A in the image is ( ). , ), calculate the coordinates of the center point of target A ( , The calculation formula is as follows: ; ; Center point operation () , The calculation formula is as follows: ; ; The formula for calculating the offset of the PT parameter of the control ball is as follows: The center point of target A is moved to the center point of the image. ; ; The formula for calculating the final PT parameters of the adjusted control ball is as follows: ; ; in , The PT parameter values ​​were set before the PT was adjusted; finally, the PTZ parameters were changed by calling the PT's pan-tilt control interface. , The Z parameter remains unchanged, thus moving the feature target A to the center of the image.

[0031] Finally, the image corresponding to the reference angle before the control ball rotates, the 12 images captured during the rotation process, the image of the final alignment with the work area, and the time information are uploaded to the cloud platform as evidence of the automatic adjustment of the control ball, which facilitates the verification of the subsequent adjustment of the control ball and the analysis of its performance and effectiveness.

[0032] This invention proposes an automatic direction adjustment method for intelligent mobile surveillance PTZ cameras. By deploying an automatic focusing work area service within the mobile surveillance PTZ camera, the method triggers the camera to rotate 360 ​​degrees horizontally and vertically according to a predetermined strategy at regular intervals. The system captures images of the work site from different directions, identifies and analyzes the features of the work site, and selects the optimal work area based on the predetermined work area priority and image work area score calculation formula. It then adjusts the camera of the surveillance ball to align with the optimal work area. This solves the problem that, due to subjective or objective factors, the surveillance ball at the power work site often has its camera pointing towards the sky, grass, walls, non-core work areas, or intentionally obstructing the view, resulting in a large number of "invalid" video images being collected. This affects the recognition effect of the intelligent violation identification model and the efficiency of remote supervision personnel in conducting on-site inspections.

[0033] The above description is merely a specific embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for automatic direction adjustment of intelligent mobile surveillance PTZ video monitoring, characterized in that: Includes the following steps: Step 1: Obtain images of the work site: Extract images of the work site from the video stream of the surveillance camera in the form of frame-by-frame extraction; Step 2: Determine the priority of the work area in the image: Use the work site feature recognition model to identify target features in the image, combine the identified target features with the predetermined work area priority feature rules to determine whether it is the highest priority work area. If yes, end the automatic adjustment task; otherwise, proceed to step 3. Step 3: Find the optimal work area: Rotate the control ball lens according to the preset rotation rules, capture several images for target feature recognition, combine the target features in each image to determine the current work scene type, and calculate the work area score contained in each image according to the predetermined formula. The image with the highest score is taken as the optimal work area. Step 4: Orient the control ball lens toward the lens angle corresponding to the image with the highest score, and align it with the optimal working area.

2. The method for automatic direction adjustment of intelligent mobile deployment ball video surveillance according to claim 1, characterized in that: The process includes the following steps: Before performing step 2, the extracted images of the work site are input into the video quality diagnostic model to determine whether they are abnormal images. If so, a warning is issued, the current adjustment is terminated, and the system waits for the images extracted in the next cycle. If not, step 2 is performed.

3. The method for automatic direction adjustment of intelligent mobile deployment ball video surveillance according to claim 1, characterized in that: In step 3, the identified target features include worker features and work scene features. The current work scene type is determined by combining the target features in each image. If a highest priority work area is found, step 4 is executed. If no highest priority work area is found, a work area score is calculated for each image. The calculation formula is as follows: ； In the formula, The score is given to the job area covered by the i-th image. The weight coefficient value of the m-th target feature among the identified worker characteristics. The total number of the m-th target feature among the worker characteristics. The weight coefficient value of the nth target feature in the identified work scene features. This represents the total number of the nth target feature identified from the characteristics of the work scene. Based on the above formula, the image with the highest score in the image task region is calculated, i.e. Where t is the image number with the highest score in the work area, and then the angle of the PTZ camera when acquiring the image with that number is obtained based on the image number. The angle of the lens That is, focus on the optimal work area.

4. The method for automatic direction adjustment of intelligent mobile deployment ball video surveillance according to claim 1, characterized in that: It also includes step 5, fine-tuning the screen of the optimal work area: When setting the offset of the control ball screen, the formula for calculating the linear relationship between image pixels and PTZ parameters is used to determine the P value P1 corresponding to 1px horizontal movement of the lens and the T value T1 corresponding to 1px vertical movement of the lens. Calculate the PT parameter offset of the control sphere to adjust the on-site operation target to the center area of ​​the screen: Given the pixel coordinates of feature target A in the image ( , The size of feature target A in the image is ( ). , ), calculate the coordinates of the center point of target A ( , The calculation formula is as follows: ； ； Center point operation () , The calculation formula is as follows: ; The screen size is (w, h). The formula for calculating the offset of the PT parameter of the control ball is as follows: The center point of target A is moved to the center point of the image. ； ； The formula for calculating the final PT parameters of the adjusted control ball is as follows: ； ； in , The PT parameter values ​​were set before the PT was adjusted; finally, the PTZ parameters were changed by calling the PT's pan-tilt control interface. , The Z parameter remains unchanged, thus moving the feature target A to the center of the image.

5. The method for automatic direction adjustment of intelligent mobile deployment PTZ video surveillance according to claim 1, characterized in that: The specific steps for rotating the control ball lens according to the preset rotation rules in step 3 are as follows: First, rotate the control ball horizontally 360°: Set The range of values ​​is [ , The corresponding rotation angle range is [ , ]; The range of values ​​is [ , The corresponding rotation angle range is [ , ]; Using the current orientation of the PT (Positioning Point) camera as the rotation base point, the current PT value is set to... , The corresponding horizontal and vertical angle values ​​are set to , The calculation formula is as follows: ； ； Horizontal angle exist Based on this, rotate one full circle in increments of 60°, vertical angle Keeping the position constant, capture one image of the operation at each 60° horizontal rotation, and record the orientation angle of the control ball corresponding to the image. A total of 6 images are captured. The formulas for calculating the horizontal and vertical angles after each rotation are as follows: ； ； In the formula, i takes the value [1, 6]. When i = 1, When horizontally continuously superimposed during a 60° rotation, Exceeding [ , When within the range, for Adjustments were made to ensure that its value falls within [ , Within the specified range, the adjustment formula is as follows: ； The formulas for calculating the horizontal and vertical angles for each image are as follows: ； In the formula, i takes the value [1, 6], and the formula for calculating the PT value corresponding to i is as follows: ； ； The PTZ parameters are changed by calling the pan-tilt-zoom (PTZ) control interface provided by the surveillance camera, where the parameters are continuously changed. , The Z parameter remains constant, thus controlling the rotation of the control ball.

6. The method for automatic direction adjustment of intelligent mobile deployment ball video surveillance according to claim 1, characterized in that: The priority feature rules for the work area in step 2 include the work scenario type, priority level, and target features involved.

Images