Bird activity scene-based camera intelligent field of view angle switching method and system

Abstract

This application relates to a method and system for intelligent field-of-view switching of cameras based on bird activity scenes. The method involves acquiring video streams through cameras deployed on bird feeders, preprocessing them to generate standardized image sequences, detecting and tracking individual birds within the image sequences, extracting motion features to form a feature set, determining the type of bird activity scene and acquiring interaction information, inputting the feature set, scene type, and interaction information into an angle decision model, outputting candidate shooting angles and estimated value scores, calculating an urgency score to select a target shooting angle, and generating control commands based on the target shooting angle to drive the gimbal to that angle, thus completing the automatic switching of the field of view. This solution enables intelligent adjustment of the camera's field of view, improving the completeness and accuracy of bird behavior photography.

Description

Technical Field

[0001] This application relates to the fields of camera and information technology, and in particular to a method and system for intelligent field-of-view switching of cameras based on bird activity scenes. Background Technology

[0002] In the field of animal behavior research, the observation and recording of bird behavior is an important subfield. To obtain high-quality video data of bird behavior, traditional methods primarily rely on manually operated cameras. Researchers need to remain at the observation site for extended periods, manually adjusting the camera's field of view to capture various bird activities. This method allows researchers to choose appropriate shooting angles based on their experience and judgment, thus meeting, to a certain extent, the basic needs of bird behavior research.

[0003] However, manually operating cameras has many drawbacks. Because bird activity is random and unpredictable, researchers find it difficult to accurately capture crucial moments of bird behavior at all times. To address this issue, some researchers have attempted to use cameras at fixed angles for continuous recording. Fixed-angle cameras can continuously record the area of ​​bird activity, avoiding omissions that may occur with manual operation.

[0004] However, fixed-angle cameras also have significant limitations. Their field of view is fixed and cannot be dynamically adjusted based on the real-time activity and behavioral changes of the birds. When the birds' activity range exceeds the camera's field of view, their behavior cannot be fully recorded, resulting in incomplete image data and affecting the accuracy and depth of bird behavior research. Summary of the Invention

[0005] The main purpose of this application is to provide a method for intelligent field-of-view switching of cameras based on bird activity scenes, which can automatically switch the camera field of view according to the bird activity scene and behavioral characteristics, thereby improving the completeness and accuracy of bird behavior shooting.

[0006] To achieve the above objectives, embodiments of the present invention provide a method for intelligent field-of-view switching of a camera based on bird activity scenes, the method comprising the following steps: Video streams are captured by cameras deployed on bird feeders to obtain raw image data, and the raw image data is preprocessed to generate a standardized image sequence. The standardized image sequence contains image frames arranged by timestamps, and each image frame is associated with its acquisition timestamp and the spatial orientation parameters of the camera. Bird individual detection and continuous tracking are performed on the standardized image sequence. Based on the tracking results, the motion features of each tracked bird individual are identified and extracted to form a set of bird motion features. The motion features include at least head motion features that characterize the trajectory and frequency of head movement, leg motion features that characterize the leg movement pattern and posture, and body motion features that characterize the overall movement state of the body. Based on the tracking results of the birds, the current bird activity scene type is determined, and bird interaction information is obtained based on the bird activity scene type. The bird activity scene type includes single bird activity scene and multiple bird activity scene. The bird motion feature set, the scene type, and the bird interaction information are input into a pre-trained angle decision model. The angle decision model performs fusion analysis on the input information and outputs a candidate set containing at least one candidate shooting angle and its corresponding estimated value score. Based on the scene type and the bird interaction information, an urgency score representing the priority of the shooting task is calculated; and a target shooting angle is selected from the candidate set according to the urgency score. A control command is generated based on the target shooting angle. The control command includes the identifier of the target camera and the angle parameters to be adjusted. The control command is sent to the corresponding camera gimbal driver module to drive the gimbal to move to the target shooting angle and complete the automatic switching of the field of view.

[0007] In summary, the technical solution of this application preprocesses the raw image data acquired by the camera to generate standardized image sequences, providing high-quality data for subsequent bird detection and tracking. Detecting and continuously tracking individual birds and extracting their motion features allows for accurate understanding of their behavioral states. Interaction information is obtained based on the bird activity scene type, and combined with the motion feature set input into the angle decision model, which outputs candidate shooting angles and their estimated value scores. The target shooting angle is then selected by calculating the urgency score, and finally, the camera gimbal is controlled to move to that angle, achieving automatic switching of the field of view. This solution enables intelligent adjustment of the camera's field of view, allowing for timely and accurate capture of key bird behaviors, improving the quality and efficiency of bird behavior photography, and providing richer and more accurate image data for bird behavior research. Attached Figure Description

[0008] Figure 1 This is a scene diagram illustrating the intelligent field-of-view switching method for cameras based on bird activity scenes in this application embodiment; Figure 2 A flowchart is provided for an embodiment of this application to illustrate a method for intelligent field-of-view switching of a camera based on bird activity scenes; Figure 3A schematic diagram illustrating the process of generating urgency scores provided in this application embodiment; Figure 4 This is a schematic diagram illustrating the process of generating a set of bird motion features provided in an embodiment of this application. Figure 5 This is a schematic diagram illustrating the process of generating a set of candidate shooting angles provided in an embodiment of this application. Figure 6 A schematic diagram illustrating the process of generating an initial list of candidate shooting angles for embodiments of this application; Figure 7 This is a schematic diagram of the bird interaction information processing provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a camera intelligent field-of-view switching system based on bird activity scenes provided in an embodiment of this application. Detailed Implementation The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0009] This application provides a method and system for intelligent field-of-view switching of a camera based on bird activity scenes, which will be described in detail below.

[0010] like Figure 1 As shown, a scenario for intelligent field-of-view switching of a camera based on bird activity scenes is provided. This scenario includes a camera 100 deployed on a bird feeder, a camera gimbal driver module (built into the bird feeder), a shooting control platform, and an angle decision model. The camera, camera gimbal driver module, shooting control platform, and angle decision model are connected via a network.

[0011] Taking a feeding area at a bird research base as an example, researchers installed a camera on top of the bird feeder to observe and record bird behavior. This camera covers the feeder and a certain area around it to capture various bird activities.

[0012] A camera fixed to a bird feeder continuously captures real-time video streams of the area surrounding the feeder. For example, at different times of day, various bird species come to the feeder to forage and roost. The camera records these bird activities as a video stream. The captured video stream is analyzed frame by frame to extract each image. Due to environmental factors and the camera itself, noise interference may exist in the images; therefore, denoising processing is required to generate a preliminary denoised image frame sequence. Next, a frame sequence alignment operation is performed on the preliminary denoised image frame sequence to ensure temporal and spatial consistency of image frames captured at different times. Then, brightness and contrast normalization processing is performed on the aligned image frame sequence to bring the brightness and contrast of the images to a uniform standard. Finally, the processed image frame sequences are integrated into a standardized image sequence and uploaded to the shooting control platform.

[0013] The image capture control platform performs bird detection on each frame of a standardized image sequence, identifying individual birds and labeling their locations. For example, multiple birds in different locations may be detected in a single frame. Based on this location information, continuous tracking is performed on the individual birds, generating a bird movement path that includes the tracking trajectory. From the tracking trajectory, motion features such as head movement trajectory and frequency, leg movement patterns and posture changes, and overall body movement can be extracted, forming a set of bird motion features.

[0014] Based on bird tracking results, the current bird activity scenario type is determined. If only one bird is active near the feeder at a given time, it is classified as a single-bird activity scenario; if multiple birds are active near the feeder simultaneously, it is classified as a multi-bird activity scenario. When classified as a multi-bird activity scenario, bird interaction information is calculated based on the spatial relative positions and movement trajectories of each individual bird; when classified as a single-bird activity scenario, the bird interaction information is set to a preset default value.

[0015] The shooting control platform also inputs bird motion feature sets, scene types, and bird interaction information into a pre-trained angle decision model. The angle decision model performs a fusion analysis of the input information and outputs a candidate set containing at least one candidate shooting angle and its corresponding estimated value score. For example, the model may evaluate the value of different shooting angles for capturing key bird behaviors based on bird behavior and scene type, and give corresponding scores.

[0016] Based on scene type and bird interaction information, an urgency score representing the priority of the shooting task is calculated. If the birds in the current scene are active and interact frequently, the urgency score of the shooting task will be higher; conversely, if the birds are quiet and interact less, the urgency score will be lower. Target shooting angles are selected from the candidate set based on the urgency score.

[0017] Finally, the shooting control platform generates control commands based on the target shooting angle. These commands include the target camera's identifier and the required angle adjustment parameters. The control commands are then sent to the corresponding camera gimbal driver module, which moves the gimbal to the target shooting angle, completing the automatic switching of the field of view. For example, once the target shooting angle is determined, the camera gimbal driver module will precisely adjust the gimbal angle according to the control commands, ensuring the camera's field of view is aligned with the target shooting angle, thus clearly capturing the bird's activity. refer to Figure 2 , Figure 2 This is a flowchart illustrating a method for intelligent field-of-view switching of a camera based on bird activity scenes, provided in an embodiment of this application. The execution subject of this method can be a computer device (such as a shooting control platform), like a server. The intelligent field-of-view switching method for a camera based on bird activity scenes provided in this embodiment specifically includes: S10: Acquire video streams by a camera deployed on a bird feeder to obtain raw image data, and perform preprocessing operations on the raw image data to generate a standardized image sequence; the standardized image sequence contains image frames arranged by timestamps, and each image frame is associated with its acquisition timestamp and the spatial orientation parameters of the camera.

[0018] In this embodiment, the raw image data is the initial image information of bird activity captured by the camera. The video stream is dynamic image data composed of a series of consecutive image frames. Preprocessing operations involve denoising, aligning, and normalizing the raw image data to improve image quality and data consistency. The standardized image sequence is a set of image frames arranged according to timestamps after preprocessing. Each image frame is associated with its acquisition timestamp and the camera's spatial orientation parameters, which accurately record the image acquisition time and the camera's location information.

[0019] Denoising refers to removing unnecessary information from an image caused by environmental interference, camera noise, and other factors, making the image clearer. Frame sequence alignment aligns image frames acquired at different times in time and space, ensuring synchronization and consistency between image frames. Brightness and contrast normalization adjusts the brightness and contrast of the image to a uniform standard, facilitating subsequent analysis and processing.

[0020] In one embodiment, step S10 can be implemented as follows: A1: Real-time video streams of the area surrounding the bird feeder are captured using cameras located at fixed positions.

[0021] In this embodiment, a fixed-position camera is mounted on the bird feeder, ensuring a relatively stable monitoring field of view around the feeder. The real-time video stream is dynamic image data composed of continuous image frames, capable of recording the activities of birds around the feeder. The fixed position of the camera needs to be carefully selected to ensure sufficient coverage of the area around the feeder where birds may be active.

[0022] In one embodiment, the camera can be mounted at the top center of the bird feeder, a position that provides a relatively wide field of view covering the feeder and an area within a certain radius around it. The camera continuously captures a video stream at a certain frame rate (e.g., 25 frames per second), comprehensively recording various activities of the birds around the feeder, such as their arrival, perching, feeding, and departure. By continuously capturing real-time video streams, rich raw data is provided for subsequent analysis of bird behavior.

[0023] A2: Analyze the video stream frame by frame, extract each frame image and remove noise interference to generate a preliminary denoised image frame sequence.

[0024] The denoising operation aims to improve the clarity and quality of the image, making the bird features in the image more prominent.

[0025] In one embodiment, video decoding technology can be used to parse the acquired video stream frame by frame, converting the video stream into a series of individual image frames. For each image frame, a bilateral filtering algorithm can be used to remove noise. By calculating the spatial distance and grayscale value difference between each pixel and its neighboring pixels, a weighted average is applied to the neighboring pixels, thereby achieving the effect of noise reduction. After bilateral filtering, a preliminary denoising image frame sequence is generated.

[0026] A3: Perform a frame sequence alignment operation on the initially denoised image frame sequence to generate an aligned image frame sequence.

[0027] In this embodiment, the image frame sequence after initial denoising may exhibit temporal and spatial inconsistencies due to slight camera shake, environmental factors, etc. Frame sequence alignment adjusts these image frames in time and space to achieve synchronization and consistency.

[0028] In one embodiment, firstly, the image frames are sorted chronologically by analyzing their timestamp information to ensure they are arranged in the order they were acquired. Then, spatial alignment is performed using a feature point matching method. Feature points, such as corner points and edge points, are extracted from each image frame. By matching feature points between adjacent image frames, the translation, rotation, and scaling relationships between the image frames are calculated. Based on these transformation relationships, the image frames are transformed accordingly to align them spatially. For example, the SIFT (Scale Invariant Feature Transform) algorithm is used to extract feature points, and the RANSAC (Random Sample Consensus) algorithm is used for feature point matching and transformation parameter calculation. After frame sequence alignment, an aligned image frame sequence is generated. Frame sequence alignment improves the consistency between image frames, providing more reliable data for subsequent analysis.

[0029] A4: Perform brightness and contrast normalization processing on the aligned image frame sequence to obtain the processed image frame sequence.

[0030] In this embodiment, the aligned image frame sequence may exhibit differences in brightness and contrast between different image frames due to variations in ambient light. Brightness and contrast normalization adjusts the brightness and contrast of these image frames to a uniform standard, resulting in a more consistent visual effect. The processed image frame sequence is a set of image frames that have undergone brightness and contrast normalization. Brightness and contrast normalization improves the visual quality of the image, making bird features in the image clearer and facilitating subsequent analysis and identification.

[0031] In one embodiment, an adaptive gamma correction algorithm is used to process each frame of the image. This algorithm dynamically adjusts the gamma value based on the local brightness information of the image, enhancing image contrast while avoiding over-enhancing or under-enhancing the brightness of certain areas. First, the image is divided into multiple small blocks, and the average brightness of each block is calculated. The gamma value of each block is then adjusted based on the average brightness value, and gamma correction is performed on each block. Finally, the processed blocks are merged into a complete image. By performing adaptive gamma correction on each frame, a processed image frame sequence is obtained. Brightness and contrast normalization improves image quality and consistency, providing better conditions for subsequent bird detection and feature extraction.

[0032] A5: Integrate the aforementioned image frame sequence into a standardized image sequence.

[0033] In this embodiment, the image frame sequence, after denoising, alignment, and brightness and contrast normalization, already possesses good quality and consistency. Integrating these image frame sequences into a standardized image sequence involves organizing and managing the processed image frames according to certain rules, making it an ordered whole that facilitates subsequent processing and analysis. The standardized image sequence contains image frames arranged by timestamps, with each image frame associated with its acquisition timestamp and the spatial orientation parameters of the camera.

[0034] The generation of standardized image sequences facilitates the unified management and use of image frames, providing a unified data format for subsequent bird individual detection, tracking, and behavioral analysis.

[0035] In one embodiment, the processed image frame sequence is integrated into a standardized image sequence. A corresponding acquisition timestamp and camera spatial orientation parameters are added to each image frame. The acquisition timestamp can be obtained from the video stream's metadata, and the camera's spatial orientation parameters can be determined based on the camera's installation location and angle. All image frames are sorted according to their timestamps to form an ordered image frame sequence. This sequence is stored in a specific data structure, simultaneously associating the acquisition timestamp of each image frame with the camera's spatial orientation parameters to generate the standardized image sequence. For example, a database can be used to store the standardized image sequence, with each record containing the binary data of the image frame, the acquisition timestamp, and the camera's spatial orientation parameters.

[0036] S20: Perform bird individual detection and continuous tracking on the standardized image sequence, identify and extract the motion features of each tracked bird individual based on the tracking results, and form a bird motion feature set; the motion features include at least head motion features representing the trajectory and frequency of head movement, leg motion features representing the leg movement pattern and posture, and body motion features representing the overall movement state of the body.

[0037] In this embodiment, bird individual detection refers to the process of identifying individual birds in an image sequence. Continuous tracking involves continuously tracking the detected individual birds across different image frames to obtain their movement trajectories. Motion features are key information describing bird movement behavior; head motion features can reflect the bird's attention direction and alertness, leg motion features can reflect the bird's movement mode and posture, and body motion features can represent the bird's overall movement state.

[0038] Among these, head movement trajectory refers to the path of a bird's head in space, and frequency refers to the speed of head movement. Leg movement patterns include walking, jumping, etc., and posture refers to the bending, extension, and other states of the legs. Overall body movement states include standing still, flying, running, etc.

[0039] In one embodiment, bird detection is performed on each image frame in the standardized image sequence. This can be achieved using a deep learning-based object detection algorithm, such as YOLO (You Only Look Once). YOLO is a real-time object detection algorithm that can quickly and accurately detect individual birds in an image and label their location information, obtaining bird location data. Continuous tracking of the birds based on this location data can be performed using a Kalman filter algorithm. Kalman filtering is an optimal estimation algorithm that can predict the next position of a bird based on its historical location information, thereby enabling continuous tracking and generating a bird movement path containing the tracking trajectory.

[0040] S30: Based on the tracking results of the birds, determine the current bird activity scene type, and obtain bird interaction information based on the bird activity scene type. The bird activity scene type includes single bird activity scene and multiple bird activity scene.

[0041] In this embodiment, the bird activity scene types are categorized based on the number and activity patterns of birds. A single-bird activity scene refers to a scene where only one bird is active within the activity area at any given time; a multi-bird activity scene refers to a scene where two or more birds are simultaneously active within the activity area. Bird interaction information describes the relationships and behavioral interactions between birds.

[0042] In scenarios involving multiple birds, there may be interactive behaviors such as competition, cooperation, and communication among the birds. These behaviors can be reflected through the spatial relative positions and movement trajectories of individual birds.

[0043] In one embodiment, step S30, which involves obtaining bird interaction information based on the bird activity scene type, may include the following steps: B1. When the scene is determined to be a scene with multiple birds, calculate the bird interaction information based on the spatial relative position and movement trajectory relationship between each individual bird.

[0044] In this embodiment, in a scenario involving multiple birds, the spatial relative positions and movement trajectories of individual birds reflect their interactions. Spatial relative positions indicate the distance between birds, while movement trajectory relationships indicate whether their movement directions are consistent or intersecting. Bird interaction information is a quantitative representation of these interactions.

[0045] The inter-individual distance distribution matrix describes the distance relationships between multiple individual birds. Distance distribution features are extracted from the matrix to reflect the patterns of distance distribution. The trajectory intersection distribution features are a quantitative description of the intersection of bird trajectories. For specific methods of calculating bird interactions, please refer to [link to relevant documentation]. Figure 7 The method for generating interactive information about birds is shown.

[0046] B2: When the scene is determined to be a single bird activity, the bird interaction information is set to a preset default value, which is used to indicate a no-interaction state.

[0047] In this embodiment, in a single bird activity scenario, since there is only one bird and no interaction with other birds, the bird interaction information is set to a preset default value. The preset default value is a predefined numerical value or vector used to represent a no-interaction state.

[0048] The "no-interaction" state indicates that the bird did not have direct contact, competition, or cooperation with other birds during its activity.

[0049] In one embodiment, when the scenario is determined to be a single bird activity, the bird interaction information is directly set to a preset default value. The preset default value can be set according to the actual situation; for example, it can be a vector of all zeros, representing no interaction features. By setting the bird interaction information to a default value, the processing flow in single bird activity scenarios can be simplified, and the system efficiency can be improved.

[0050] For example, the default value is [0, 0, 0]. When the scene is determined to be a single bird activity, the bird interaction information is directly assigned the value [0, 0, 0], indicating that the bird is in a non-interactive state. S40: The bird action feature set, the scene type, and the bird interaction information are input into a pre-trained angle decision model. The angle decision model performs fusion analysis on the input information and outputs a candidate set containing at least one candidate shooting angle and its corresponding estimated value score.

[0051] S40: Input the bird motion feature set, the scene type, and the bird interaction information into a pre-trained angle decision model, and perform fusion analysis on the input information through the angle decision model to output a candidate set containing at least one candidate shooting angle and its corresponding estimated value score.

[0052] In this embodiment, the angle decision model is a trained machine learning model that can analyze the advantages and disadvantages of different shooting angles based on the input set of bird movement features, scene type, and bird interaction information, and provide an estimated value score. Candidate shooting angles are the potentially suitable shooting angles recommended by the model, and the estimated value score is an evaluation of the quality and value of each candidate shooting angle.

[0053] Fusion analysis refers to the comprehensive processing of different types of information to extract more valuable features and information.

[0054] S50: Based on the scene type and the bird interaction information, calculate the urgency score that represents the priority of the shooting task; and select the target shooting angle from the candidate set according to the urgency score.

[0055] In this embodiment, the urgency score is an indicator for measuring the priority of the shooting task, reflecting the urgency and importance of the current shooting task. The target shooting angle is the angle most suitable for the current shooting task selected from the candidate set.

[0056] The urgency of a shooting task is influenced by the type of scene and information about bird interactions. For example, the urgency of a shooting task may be higher in scenes with multiple birds active and interacting frequently.

[0057] In one embodiment, selecting a target shooting angle from the candidate set based on the urgency score in step S50 may include: The urgency score is compared with a preset threshold; if the urgency score is greater than the preset threshold, the candidate shooting angle with the highest estimated value score is selected from the candidate set as the target shooting angle; if the urgency score is not greater than the preset threshold, the candidate shooting angle with the highest priority in the candidate set is selected as the target shooting angle.

[0058] In this embodiment, the preset threshold is a pre-set critical value used to determine the urgency of the shooting task. The comparison between the urgency score and the preset threshold determines the method for selecting the target shooting angle from the candidate set. The estimated value score is an evaluation of the quality and value of the candidate shooting angles, and the priority is the ranking of the candidate shooting angles according to actual needs.

[0059] When the urgency score is greater than the preset threshold, it indicates that the shooting task is relatively urgent and the shooting angle that can obtain high-quality images should be selected first. When the urgency score is not greater than the preset threshold, it indicates that the urgency of the shooting task is relatively low and the shooting angle can be selected according to priority.

[0060] In one embodiment, the preset threshold can be adjusted based on past shooting experience and actual needs. If the urgency score is greater than the preset threshold, the candidate shooting angle with the highest estimated value score is searched from the candidate set. This can be achieved by traversing the candidate set, comparing the estimated value scores of each candidate shooting angle, and finding the angle with the highest score as the target shooting angle. If the urgency score is not greater than the preset threshold, the candidate shooting angle with the highest priority is selected from the candidate set as the target shooting angle according to a pre-set priority rule. The priority setting can consider various factors, such as shooting history, the rarity of the bird, and the uniqueness of the shooting angle. By selecting the target shooting angle based on the urgency score and the preset threshold, the urgency of the shooting and the quality of the shooting angle can be reasonably balanced under different shooting task conditions.

[0061] For example, if the preset threshold is 80 and the calculated urgency score is 90, which is greater than the preset threshold, and there are three candidate shooting angles in the candidate set with estimated value scores of 85, 92, and 88, the candidate shooting angle with the highest estimated value score of 92 is selected as the target shooting angle. If the urgency score is 70, which is not greater than the preset threshold, assuming that the first candidate shooting angle in the candidate set has the highest priority according to the priority rule, then that angle is selected as the target shooting angle.

[0062] In one embodiment, if there are many birds in the scene and they are densely distributed, the scene complexity score will be higher. When the scene type is a scene with multiple birds, the distance distribution and movement direction correlation in the bird interaction information are analyzed to generate an interaction intensity score. If the birds are close to each other and their movement directions are highly correlated, the interaction intensity score will be higher. The scene complexity score and the interaction intensity score are fused and calculated, for example, using a weighted summation method, to generate an initial urgency score. The initial urgency score is adjusted by combining the current field of view coverage of the camera with the spatial position relationship of the target bird. If the target bird is outside the current field of view coverage, requiring a larger adjustment of the camera angle, the urgency score will increase accordingly, forming the final urgency score. The urgency score is compared with a preset threshold. If the urgency score is greater than the preset threshold, it indicates that the shooting task is relatively urgent and the key image needs to be captured as soon as possible. In this case, the candidate shooting angle with the highest estimated value score is selected from the candidate set as the target shooting angle; if the urgency score is not greater than the preset threshold, the candidate shooting angle with the highest priority in the candidate set is selected as the target shooting angle. Priority settings can be adjusted according to actual needs, such as based on shooting history and the rarity of the bird species. By calculating a urgency score and selecting target shooting angles accordingly, shooting tasks can be rationally arranged in different bird activity scenarios, improving shooting efficiency and quality.

[0063] S60: Generate a control command based on the target shooting angle. The control command includes the identifier of the target camera and the angle parameters to be adjusted. Send the control command to the corresponding camera gimbal driver module to drive the gimbal to move to the target shooting angle and complete the automatic switching of the field of view.

[0064] In this embodiment, the control command is used to control the movement of the camera pan-tilt unit, and it includes the identifier of the target camera and the angle parameters to be adjusted. The camera pan-tilt drive module is the device responsible for receiving the control command and driving the pan-tilt unit to move.

[0065] The identifier of the target camera is used to uniquely identify the camera whose field of view needs to be adjusted. The angle parameters include information such as the horizontal and vertical angles that the gimbal needs to rotate.

[0066] In one embodiment, firstly, the identifier of the target camera is determined. This identifier can be a unique identifier such as the camera's number or name. Then, the required angle parameters for adjustment are calculated based on the target shooting angle, for example, calculating the angles the gimbal needs to rotate horizontally and vertically. The target camera's identifier and angle parameters are combined into a control command. The control command is sent to the corresponding camera gimbal driver module. Data transmission can be performed via a wired or wireless network to ensure the control command reaches the driver module accurately. After receiving the control command, the camera gimbal driver module parses the command and extracts the target camera's identifier and angle parameters. Based on the angle parameters, the gimbal is driven to rotate to the target shooting angle. During the gimbal movement, a closed-loop control method can be used to monitor the actual position of the gimbal in real time and compare it with the target position. Adjustments are made based on the comparison results to ensure the gimbal accurately moves to the target shooting angle. After the gimbal movement is completed, the camera's field of view automatically switches to the target shooting angle, thus enabling clear capture of bird activity. By generating control commands and driving the gimbal movement, automatic switching of the camera's field of view is achieved, improving the automation and accuracy of the shooting.

[0067] In one embodiment, reference Figure 3 Step S50, which calculates the urgency score representing the priority of the shooting task based on the scene type determination result and the bird interaction information, may include steps S51-S54, which will be described in detail below: S51: Analyze the scene type determination result, extract the number and distribution density of individual birds in the current scene, and generate a scene complexity score.

[0068] In this embodiment, the scene type determination result includes information on whether the current bird activity scene is a single bird activity scene or a multiple bird activity scene. By parsing this result, the number and distribution density of individual birds in the current scene can be obtained. The scene complexity score is a quantitative assessment of the complexity of the current scene, and it is related to the number and distribution density of individual birds.

[0069] The number of individual birds reflects the quantity of birds in the scene, while the distribution density reflects the spatial distribution of birds. If there are many individual birds and they are densely distributed, the scene complexity will be high.

[0070] In one embodiment, if the scene is determined to be a single bird activity, the number of individual birds is 1, the distribution density is low, and the scene complexity score is relatively low. If the scene is determined to be a multi-bird activity, the number of individual birds is counted. This can be obtained by detecting and counting individual birds in the image sequence. The distribution density of the birds is calculated; for example, the bird activity area can be divided into several small regions, the number of birds in each small region can be counted, and then the average density can be calculated. A scene complexity score is generated based on the number of individual birds and the distribution density. A linear function can be used, for example, scene complexity score = a * number of individual birds + b * distribution density, where a and b are pre-set weighting coefficients. By generating a scene complexity score, the complexity of the current scene can be better understood.

[0071] For example, in a scenario with multiple birds, the number of individual birds is counted to be 10. The bird activity area is divided into 5 smaller areas, with the number of birds in each area being 2, 3, 2, 2, and 1 respectively. The calculated average distribution density is 2. Assuming a = 0.6 and b = 0.4, the scenario complexity score calculated using the formula is 0.6 * 10 + 0.4 * 2 = 6.8.

[0072] S52: When the scene type is a scene with multiple birds, analyze the distance distribution and movement direction correlation in the bird interaction information to generate an interaction intensity score.

[0073] In this embodiment, in a scenario involving multiple birds, the distance distribution and movement direction correlation in the bird interaction information reflect the intensity of interaction between birds. The distance distribution describes the distance between individual birds, while the movement direction correlation describes the consistency of their movement directions. The interaction intensity score is a quantitative assessment of the degree of interaction between birds. Specifically, if birds are close together and have a high correlation in their movement directions, their interaction intensity is considered high; conversely, their interaction intensity is considered low.

[0074] In one embodiment, when the scene type is a scenario involving multiple birds, the distance distribution in the bird interaction information is analyzed. Statistics such as the mean and variance of the distance distribution can be calculated to describe the concentration and dispersion of the distances. The correlation of movement directions can be analyzed by calculating the cosine of the angle between the bird movement direction vectors. The closer the cosine value is to 1, the more consistent the movement directions are, and the higher the correlation. An interaction intensity score is generated based on the distance distribution and the correlation of movement directions. A comprehensive calculation formula can be used, for example, Interaction Intensity Score = c * Distance Distribution Score + d * Movement Direction Correlation Score, where c and d are pre-set weighting coefficients. The distance distribution score can be calculated based on the statistics of the distance distribution, and the movement direction correlation score can be calculated based on the cosine of the angle. By generating an interaction intensity score, a better understanding of the degree of interaction between multiple birds can be achieved, providing more accurate information for calculating the urgency score.

[0075] For example, the calculated mean of the distance distribution is 0.5 and the variance is 0.1. According to the preset rules, the distance distribution score is set to 0.6. The average cosine of the angle between the bird movement direction vectors is calculated to be 0.8, and the movement direction correlation score is set to 0.8. Assuming c = 0.5 and d = 0.5, the interaction strength score is calculated as 0.5 * 0.6 + 0.5 * 0.8 = 0.7.

[0076] S53: The scene complexity score and the interaction intensity score are fused together to generate an initial urgency score.

[0077] In one embodiment, the scene complexity score and the interaction intensity score are fused together. A weighted summation method can be used, for example, initial urgency score = e * scene complexity score + f * interaction intensity score, where e and f are pre-set weight coefficients. The weight coefficients can be adjusted according to the actual situation; for example, if more emphasis is placed on scene complexity, the value of e can be set larger; if more emphasis is placed on the interaction intensity between birds, the value of f can be set larger. Generating an initial urgency score through fusion calculation can comprehensively consider multiple factors of the scene and more accurately assess the priority of the shooting task.

[0078] For example, if the scene complexity score is 6.8 and the interaction intensity score is 0.7, assuming e = 0.7 and f = 0.3, the initial urgency score calculated according to the formula is 0.7 * 6.8 + 0.3 * 0.7 = 4.76 + 0.21 = 4.97.

[0079] S54: Combine the current field of view coverage of the camera with the spatial position of the target bird, adjust the initial urgency score to form the final urgency score.

[0080] In this embodiment, the relationship between the current field of view of the camera and the spatial position of the target bird affects the ease and urgency of the shooting. If the target bird is outside the current field of view, a greater adjustment of the camera angle is required, increasing the urgency of the shooting task. By incorporating this relationship into the initial urgency score, a final urgency score that better reflects the actual situation can be obtained.

[0081] In one embodiment, the current field of view coverage of the camera is determined. This can be determined through the camera's parameter settings and installation location. The spatial coordinates of the target bird are obtained. The spatial position of the target bird is compared with the current field of view coverage of the camera. If the target bird is within the field of view coverage, the initial urgency score can remain unchanged or be slightly adjusted; if the target bird is outside the field of view coverage, the initial urgency score needs to be increased based on factors such as the distance between the target bird and the field of view boundary. For example, an adjustment coefficient can be used, the size of which is determined based on the distance, and then the initial urgency score is multiplied by the adjustment coefficient to obtain the final urgency score. By adjusting the initial urgency score to form the final urgency score, the actual priority of the shooting task can be more accurately reflected, providing a more reliable basis for the selection of the target shooting angle.

[0082] For example, if the initial urgency score is 4.97, and after comparison it is found that the target bird is outside the current camera's field of view and far from the edge of the field of view, the adjustment factor is determined to be 1.3. Then the final urgency score = 4.97 * 1.3 = 6.461.

[0083] In one embodiment, reference Figure 4 Step S20 may include steps S21-S26, which will be described in detail below: S21: Perform bird individual detection operation on each image frame in the standardized image sequence, identify the bird individuals present and label their location information to obtain bird individual location data.

[0084] In this embodiment, the bird individual detection operation is the process of finding individual birds within image frames. By detecting each image frame in a standardized image sequence, the individual birds present can be identified, and their location information can be labeled. Bird individual location data records the information of the individual bird's position in the image frame, and can be represented using coordinates or other methods.

[0085] In one embodiment, bird detection is performed on each image frame in a standardized image sequence. A deep learning-based object detection algorithm, such as Faster R-CNN (Region-based Convolutional Neural Networks), can be used. Faster R-CNN is a two-stage object detection algorithm that first generates candidate regions that may contain the target through a Region Proposal Network (RPN), and then classifies and locates these candidate regions. After identifying individual birds in the image frames, their location information is labeled. A bounding box can be used to label the location of each bird; the coordinates of the top-left and bottom-right corners of the box represent the bird's location. The location information of each bird in each image frame is recorded to obtain bird location data. This is achieved by performing bird detection and labeling the location information.

[0086] For example, in a certain image frame, the Faster R-CNN algorithm detects three individual birds. For the first bird, its location information is labeled with the coordinates of the top-left corner (x1, y1) and the bottom-right corner (x2, y2) of the bounding box; for the second bird, it is labeled with (x3, y3) and (x4, y4); and for the third bird, it is labeled with (x5, y5) and (x6, y6). This location information is recorded to form the bird location data for that image frame.

[0087] S22: Perform continuous tracking operation on the individual bird based on the individual bird location data to generate the individual bird movement path containing the tracking trajectory.

[0088] In this embodiment, continuous tracking is the process of continuously tracking detected individual birds across different image frames. Based on the individual bird's location data, the position of the individual bird in each image frame can be determined, thereby generating its movement path. The individual bird's movement path contains the tracking trajectory of the individual bird and can reflect information such as the bird's direction of movement and speed.

[0089] In one embodiment, continuous tracking of individual birds is performed based on their location data. A method combining Kalman filtering and the Hungarian algorithm can be employed. Kalman filtering predicts the bird's position in the next image frame, while the Hungarian algorithm matches the predicted position with the actual detected position. In each image frame, based on the Kalman filtering prediction, potential bird individuals are searched for in the vicinity. The cost matrix between the predicted and detected positions is calculated using the Hungarian algorithm, and then matched to determine the corresponding position of each bird individual. The positions of the birds in different image frames are sequentially concatenated to generate a bird movement path containing the tracking trajectory.

[0090] For example, consider tracking an individual bird across a series of image frames. In the first frame, its position is (x1, y1). A Kalman filter is used to predict its position in the second frame. Then, in the second frame, a Hungarian algorithm is used to match its actual position to (x2, y2), and so on. Connecting these positions (x1, y1), (x2, y2), (x3, y3), and so on, forms the bird's movement path.

[0091] S23: Extract the head movement trajectory and frequency from the tracking trajectory to form head movement features.

[0092] In this embodiment, the head movement trajectory reflects the movement path of the bird's head in space, and the frequency reflects the speed of the head movement. Head movement characteristics are a comprehensive description of the head movement trajectory and frequency, which can reflect the bird's attention direction and alertness state, etc.

[0093] In one embodiment, firstly, the key points of the head in the tracking trajectory are continuously located. This can be achieved by identifying and tracking feature points of the bird's head in image frames, generating a set of head movement trajectory coordinates containing a time series. Based on this set of coordinates, the angle of change in head movement direction and the displacement distance are calculated to form a direction change feature. For example, the angle of change in direction is obtained by calculating the vector angle between the head positions at two adjacent time points, and the displacement distance is obtained by calculating the distance between the two points. The number of repetitive movements of the head trajectory per unit time is counted to determine the frequency value of the head movement. The direction change feature and the frequency value are then integrated, for example, by combining them into a single vector to form a head action feature vector. By extracting head movement trajectories and frequencies to form head action features, more detailed information can be provided for bird behavior analysis.

[0094] For example, over a period of time, key points on a bird's head are located, resulting in a series of coordinates (x1, y1), (x2, y2), (x3, y3), etc. The directional change angle and displacement distance between adjacent coordinates are calculated to obtain directional change characteristics. The number of times the head repeats its movements within 10 seconds is counted as 5, determining the head movement frequency to be 0.5 times / second. The directional change characteristics and frequency value are then combined to form a head motion feature vector.

[0095] S24: Extract leg movement patterns and posture changes from the tracking trajectory to form leg movement features.

[0096] In this embodiment, the tracking trajectory contains the movement information of an individual bird, from which leg movement patterns and posture changes can be extracted. Leg movement patterns include walking, jumping, etc., and posture changes include leg bending, extension, etc. Leg movement characteristics are a comprehensive description of leg movement patterns and posture changes, which can reflect the bird's mobility and behavioral habits.

[0097] In one embodiment, leg movement patterns and posture changes are extracted from the tracking trajectory. Leg keypoints in the tracking trajectory are identified and tracked to determine the leg's position and posture in different image frames. The leg movement pattern is determined by analyzing the distance and angle changes between the leg keypoints. For example, if the distance between leg keypoints changes regularly and the angle changes conform to walking characteristics, it is determined to be a walking pattern. Simultaneously, the leg posture, such as bending or extending, is determined based on the position and angle of the leg keypoints. The leg movement patterns and posture changes are integrated to form leg motion features. These features can be represented by a vector or matrix, where different elements represent different movement patterns and postures.

[0098] For example, key points on a bird's legs are tracked across a series of image frames. Regular changes in the distances and angles between these key points are observed, consistent with jumping characteristics, leading to the identification of the leg movement pattern as jumping. Simultaneously, bending and extending posture changes in the legs are observed during the jump. Integrating the jumping pattern and these posture changes forms the leg movement characteristics.

[0099] S25: Extract the overall body motion state from the tracking trajectory to form body movement features.

[0100] In this embodiment, the tracking trajectory records the overall movement information of an individual bird, from which its overall body movement state can be extracted. The overall body movement state includes states such as stationary, flying, and running. Body movement characteristics are a quantitative description of the overall body movement state, reflecting the bird's activity level and behavioral patterns.

[0101] In one embodiment, the overall body motion state is extracted from the tracking trajectory. The positional changes and velocity information of key points on the bird's body in the tracking trajectory are analyzed. If the position of a key point remains essentially unchanged for a period of time and its velocity is close to zero, the overall body motion state is determined to be stationary. If the positional changes of the key points conform to the characteristics of flight, such as a large displacement and significant altitude change in a short period of time, it is determined to be in flight. If the positional changes of the key points conform to the characteristics of running, such as a rapid horizontal displacement on the ground, it is determined to be in running. The overall body motion state is quantified to form body movement features. These features can be represented by a numerical value or a vector, with different values ​​representing different motion states.

[0102] For example, observe the tracking trajectory of a bird over a period of time. If the positions of its key body points remain largely unchanged and its speed is close to zero, the overall state of its body movement can be judged as static. This static state is quantified as 0, forming the body movement characteristics.

[0103] S26: Integrate the head movement features, leg movement features and body movement features to form a set of bird movement features.

[0104] In one embodiment, the head motion feature vector, leg motion feature vector, and body motion feature vector can be concatenated sequentially to form a longer vector, which serves as the bird motion feature set. Alternatively, they can be combined into a matrix, where each row or column represents a motion feature. For example, the head motion feature vector might be [0.2, 0.3, 0.5], the leg motion feature vector [0.6, 0.4], and the body motion feature vector [0]. By concatenating these vectors, a bird motion feature set [0.2, 0.3, 0.5, 0.6, 0.4, 0] can be created.

[0105] In one embodiment, reference Figure 5 Step S40 may include steps S41-S45, which will be described in detail below: S41: Input the head movement features, leg movement features and body movement features from the bird movement feature set into the feature extraction module of the angle decision model to extract a multi-dimensional behavior feature vector containing movement behavior complexity features, posture dynamic change features and overall movement stability features.

[0106] In this embodiment, the feature extraction module of the angle decision model is used to extract more valuable features from the input bird movement features. Head movement features, leg movement features, and body movement features reflect the movement of different parts of the bird, respectively. Through the feature extraction module, these features can be further transformed into a multi-dimensional behavioral feature vector that includes movement behavior complexity features, posture dynamic change features, and overall movement stability features.

[0107] Among them, the complexity of movement behavior reflects the complexity of bird movement, the dynamic change of posture reflects the changes in bird posture, and the overall movement stability represents the stability of bird movement.

[0108] In one embodiment, head movement features, leg movement features, and body movement features from a bird movement feature set are input into the feature extraction module of the angle decision model. The feature extraction module can employ a multilayer perceptron (MLP). An MLP is a feedforward artificial neural network consisting of an input layer, hidden layers, and an output layer. The input layer receives head movement features, leg movement features, and body movement feature vectors. The hidden layer performs nonlinear transformations on the input features, learning the complex relationships between them. Through the calculations of the hidden layer, features of movement behavior complexity, dynamic posture changes, and overall movement stability are extracted. These features are then combined into a multi-dimensional behavior feature vector. For example, if the head movement feature vector has a dimension of 3, the leg movement feature vector has a dimension of 2, and the body movement feature vector has a dimension of 1, after processing by the feature extraction module, a multi-dimensional behavior feature vector with a dimension of 5 is obtained.

[0109] For example, the input head motion feature vector is [0.1, 0.2, 0.3], the leg motion feature vector is [0.4, 0.5], and the body motion feature vector is [0.6]. After processing by the feature extraction module, a multi-dimensional behavioral feature vector [0.2, 0.3, 0.4, 0.5, 0.6] is obtained, which respectively represent different aspects of motion behavior complexity features, posture dynamic change features, and overall motion stability features.

[0110] S42: Input the scene type and the bird interaction information into the classification module of the angle decision model to generate a classification label for the scene type and a quantitative representation of the interaction information.

[0111] In this embodiment, the classification module of the angle decision model processes the input scene type and bird interaction information. Scene types are categorized into single-bird activity scenes and multi-bird activity scenes, while bird interaction information reflects the interactions between birds. The classification module converts scene types into classification labels and quantifies the bird interaction information.

[0112] In one embodiment, scene type and bird interaction information are input into the classification module of the angle decision model. The classification module can employ a logistic regression algorithm. For scene types, if it is a scene with a single bird, it is encoded as classification label 0; if it is a scene with multiple birds, it is encoded as classification label 1. For bird interaction information, a mapping function is used to convert it into numerical values ​​based on features such as distance distribution and movement trajectory intersection point distribution. For example, the distance distribution features and movement trajectory intersection point distribution features can be weighted and summed to obtain a comprehensive numerical value as a quantitative representation of the bird interaction information. By generating classification labels for scene types and quantitative representations of interaction information, different types of information can be unified into numerical forms, facilitating subsequent fusion analysis.

[0113] For example, consider a scene with multiple birds active, categorized as 1. The distance distribution feature in the bird interaction information is 0.3, and the intersection point distribution feature of the movement trajectory is 0.4. Assuming the weighting coefficients are 0.6 and 0.4 respectively, the quantified representation of the bird interaction information is calculated as 0.3 * 0.6 + 0.4 * 0.4 = 0.34.

[0114] S43: Through the fusion analysis module of the angle decision model, the multi-dimensional behavioral feature vector, classification label and quantitative representation are comprehensively processed to generate an initial list of candidate shooting angles.

[0115] The initial list of candidate shooting angles contains angles that may be suitable for shooting, which forms the basis for subsequent selection of target shooting angles.

[0116] In one embodiment, a fully connected neural network can be used to comprehensively process multi-dimensional behavioral feature vectors, classification labels, and quantization representations through the fusion analysis module of the angle decision model. The input layer of the fully connected neural network receives the multi-dimensional behavioral feature vectors, classification labels, and quantization representations as input. The hidden layer performs nonlinear transformations and feature fusion on the input information, learning the correlations between different pieces of information. The output layer generates an initial list of candidate shooting angles based on the calculation results of the hidden layer. When generating the initial list, some possible shooting angles can be selected according to pre-defined rules, such as the range of angle values ​​and the intervals between angles. For example, assuming the shooting angle range is from 0 to 360 degrees with an interval of 10 degrees, some angles are selected as the initial list of candidate shooting angles based on the fusion analysis results.

[0117] For example, the input multi-dimensional behavioral feature vector is [0.2, 0.3, 0.4, 0.5, 0.6], the classification label is 1, and the quantization representation is 0.34. After calculation by the fully connected neural network, according to the pre-set rules, the initial list of candidate shooting angles is generated as [30, 60, 90, 120, 150].

[0118] In one embodiment, reference Figure 6 Step S43 may include steps S431-S434, which will be described in detail below: S431: The multi-dimensional behavioral feature vector is concatenated with the classification label to form a fused feature vector.

[0119] In this embodiment, the multi-dimensional behavioral feature vector contains various features of bird movement, while the classification label represents the current scene type. Concatenating these features integrates information from different sources.

[0120] In one embodiment, a multi-dimensional behavioral feature vector is concatenated with a classification label. Assume the multi-dimensional behavioral feature vector is [a1, a2, a3, a4, a5], and the classification label is 1 (representing a scene with multiple birds). The classification label is added to the multi-dimensional behavioral feature vector in an appropriate form to form a fused feature vector. Alternatively, the classification label can be added as a new element to the end of the vector, resulting in the fused feature vector [a1, a2, a3, a4, a5, 1]. This concatenation process effectively integrates different types of information.

[0121] For example, the multi-dimensional behavioral feature vector is [0.2, 0.3, 0.4, 0.5, 0.6], the classification label is 1, and after concatenation, a fused feature vector is formed [0.2, 0.3, 0.4, 0.5, 0.6, 1].

[0122] S432: Perform a weighted superposition operation on the fused feature vector and the quantized representation to generate a comprehensive feature representation.

[0123] In this embodiment, the fused feature vector includes bird movement features and scene type information, and the quantized representation is a numerical description of bird interaction information. Performing a weighted overlay operation on these features is to integrate them according to the importance of different information, generating a comprehensive feature representation that better reflects the overall situation.

[0124] In one embodiment, a weighted summation operation is performed on the fused feature vector and the quantized representation. Assume the fused feature vector is [b1, b2, b3, b4, b5, b6], the quantized representation is q, and the weights of the fused feature vector and quantized representation are pre-set as w1 and w2, respectively, with w1 + w2 = 1. Each element in the fused feature vector is multiplied by w1, and the quantized representation is multiplied by w2, then the results are added together. For example, the first element of the comprehensive feature representation is w1 * b1 + w2 * q, and each element is calculated sequentially to obtain the comprehensive feature representation. Generating a comprehensive feature representation through a weighted summation operation allows for the effective integration of information from different sources based on its importance, providing a more accurate basis for subsequently generating candidate shooting angles.

[0125] For example, if the fused feature vector is [0.2, 0.3, 0.4, 0.5, 0.6, 1], and the quantization is 0.34, and w1 = 0.7 and w2 = 0.3, then the first element of the comprehensive feature representation is 0.7 * 0.2 + 0.3 * 0.34 = 0.242. The complete comprehensive feature representation is obtained by calculating this value sequentially.

[0126] S433: Based on the comprehensive feature representation, call the preset angle generation rule base to generate a preliminary set of candidate shooting angles.

[0127] In this embodiment, the comprehensive feature representation integrates bird movement characteristics, scene type information, and bird interaction information. The preset angle generation rule base is a pre-defined series of rules used to generate candidate shooting angles based on the comprehensive feature representation. By calling the rule base based on the comprehensive feature representation, a preliminary set of candidate shooting angles that match the current bird activity can be generated.

[0128] The rules in the angle generation rule base can be formulated based on a large amount of experimental data and practical experience to ensure that the generated candidate shooting angles have high feasibility and effectiveness.

[0129] In one embodiment, a preset angle generation rule base is invoked based on the comprehensive feature representation. The rule base may contain conditional judgment rules; for example, if an element in the comprehensive feature representation is greater than a certain threshold, a specific set of angles is selected as candidate shooting angles. Based on the specific value of the comprehensive feature representation, the corresponding rules in the rule base are matched to generate a preliminary set of candidate shooting angles. For example, the rule base may specify that when an element in the comprehensive feature representation is greater than 0.5, 30 degrees, 60 degrees, and 90 degrees are selected as candidate shooting angles. By invoking the preset angle generation rule base to generate a preliminary set of candidate shooting angles, multiple possible shooting angles can be quickly obtained.

[0130] For example, if a key element in the comprehensive feature representation is 0.6, the initial set of candidate shooting angles generated according to the rule base is [30, 60, 90].

[0131] S434: Perform geometric feasibility verification on the candidate shooting angles in the preliminary set, eliminate unfeasible angles, and obtain an initial list of candidate shooting angles.

[0132] In one embodiment, the camera's installation location and the gimbal's range of motion are first determined. For each candidate shooting angle in the initial set, it is calculated whether the camera's field of view at that angle will be obstructed by obstacles, and whether the gimbal can rotate to that angle. If a candidate shooting angle would obstruct the camera's field of view or prevent the gimbal from reaching that angle, it is discarded. For example, if the initial set is [30, 60, 90, 120, 150], after geometric feasibility verification, it is found that the 120-degree angle would obstruct the camera's field of view by nearby trees, and the 150-degree angle exceeds the gimbal's range of motion; these two angles are discarded, resulting in an initial list of candidate shooting angles [30, 60, 90]. Obtaining an initial list of candidate shooting angles through geometric feasibility verification ensures that the subsequently selected target shooting angles are practically feasible, improving the success rate of the shooting operation.

[0133] S44: Perform a predicted value score calculation for each candidate shooting angle in the initial list to generate a candidate set containing the score.

[0134] In this embodiment, calculating a predicted value score for each candidate shooting angle in the initial list is to evaluate the merits of each candidate angle. The predicted value score reflects the value of that angle for photographing bird behavior. By calculating the score, each candidate shooting angle is combined with its corresponding score to generate a candidate set containing the scores.

[0135] In one embodiment, a predicted value score is calculated for each candidate shooting angle in the initial list. A linear regression model can be used. The linear regression model establishes a linear equation to calculate the predicted value score based on information such as multi-dimensional behavioral feature vectors, classification labels, and quantization representations. For example, the predicted value score = a * an element of the multi-dimensional behavioral feature vector + b * classification label + c * quantization representation + d, where a, b, c, and d are pre-trained coefficients. For each candidate shooting angle in the initial list, the relevant information is substituted into the linear equation to calculate its predicted value score. Each candidate shooting angle is combined with its corresponding predicted value score to form a candidate set containing scores. By generating a candidate set containing scores, a quantitative basis can be provided for subsequent selection of target shooting angles.

[0136] For example, the candidate shooting angles in the initial list are [30, 60, 90, 120, 150]. Assume the coefficients of the linear regression model are a = 0.2, b = 0.3, c = 0.4, and d = 0.1. For the candidate shooting angle 30, substituting the relevant information into the linear equation, the estimated value score is calculated as: 0.2 * 0.2 + 0.3 * 1 + 0.4 * 0.34 + 0.1 = 0.636. The score for each candidate shooting angle is calculated sequentially, generating a candidate set containing the scores: [[30, 0.636], [60, 0.7], [90, 0.68], [120, 0.72], [150, 0.65]].

[0137] S45: Sort the candidate shooting angles in the candidate set according to the estimated value score to generate the final candidate set.

[0138] In this embodiment, the candidate shooting angles in the candidate set are sorted according to their estimated value scores to more clearly demonstrate the order of merit of each candidate shooting angle. Sorting facilitates the subsequent selection of a target shooting angle from the candidate set based on factors such as urgency scores. In the final candidate set, the candidate shooting angles are arranged from high to low or from low to high according to their estimated value scores.

[0139] The sorting process can employ sorting algorithms, such as bubble sort and quick sort.

[0140] In one embodiment, the candidate shooting angles in the candidate set are sorted according to their estimated value scores. The quicksort algorithm is used, which is an efficient sorting algorithm with an average time complexity of O(n log n). The quicksort algorithm divides the candidate set into two parts by selecting a pivot element, such that all elements in the left part are less than or equal to the pivot element, and all elements in the right part are greater than or equal to the pivot element. Then, the left and right parts are recursively sorted. The candidate shooting angles in the candidate set are sorted from highest to lowest according to their estimated value scores to generate the final candidate set. For example, given the candidate set [[30, 0.636], [60, 0.7], [90, 0.68], [120, 0.72], [150, 0.65]], after quicksort, the final candidate set is generated as [[120, 0.72], [60, 0.7], [90, 0.68], [150, 0.65], [30, 0.636]]. Generating the final candidate set through sorting provides a more intuitive reference for selecting the target shooting angle, improving the efficiency and accuracy of the selection process.

[0141] In one embodiment, reference Figure 7 Step S31 calculates bird interaction information based on the spatial relative positions and movement trajectories of individual birds. This may include steps S311-S314, which will be described in detail below: S311: When the scene is determined to be a scene with multiple birds, calculate the spatial relative position between each individual bird and generate a distance distribution matrix between individuals.

[0142] In one embodiment, step S311 can be implemented as follows: H1: Based on the tracking results of the birds, obtain the spatial coordinates of each individual bird.

[0143] In this embodiment, the bird tracking results record the bird's position information in different image frames. Based on these tracking results, the spatial coordinates of each individual bird are obtained, which forms the basis for subsequent operations such as calculating the distance between individuals.

[0144] Among them, spatial location coordinates can be represented in a pre-defined three-dimensional coordinate system. Accurate coordinate information helps to precisely analyze the spatial relationships between birds.

[0145] In one embodiment, the spatial coordinates of each individual bird are obtained based on the bird tracking results. A three-dimensional Cartesian coordinate system can be established in the scene where the images are acquired, with a fixed point as the origin. For each individual bird, its position in each image frame is found in the tracking results. Using the geometric relationships of the image and the camera parameters, the position in the image is converted into spatial coordinates in the three-dimensional coordinate system. For example, the bird's coordinates in three-dimensional space can be calculated using the camera calibration parameters and the pixel positions of the birds in the image, based on the principle of triangulation.

[0146] For example, in a bird feeding area, a three-dimensional Cartesian coordinate system is established with the center of the feeder as the origin. By tracking a bird and determining its pixel position in a certain image frame, and combining the camera's calibration parameters, the spatial coordinates of the bird in the three-dimensional coordinate system are calculated as (2, 3, 4).

[0147] H2: Calculate the Euclidean distance between any two individual birds and generate the initial inter-individual distance matrix.

[0148] In this embodiment, after obtaining the spatial coordinates of each individual bird, the Euclidean distance between any two individual birds can be calculated. The Euclidean distance is the straight-line distance between two points in three-dimensional space, intuitively reflecting the spatial distance between individual birds. The calculated distances are then filled into a matrix to generate an initial inter-individual distance matrix. This initial inter-individual distance matrix is ​​a symmetric matrix, with diagonal elements being 0, indicating that the distance between the same individual bird and itself is 0.

[0149] In one embodiment, the Euclidean distance between any two individual birds is calculated to generate an initial inter-individual distance matrix. Assume there are n birds with spatial coordinates (x1, y1, z1), (x2, y2, z2), ..., (xn, yn, zn). For any two birds i and j, the Euclidean distance d_ij = √((xj - xi)^2 + (yj - yi)^2 + (zj - zi)^2) between them is calculated. These distances are then filled into an n×n matrix to obtain the initial inter-individual distance matrix. For example, consider three birds A, B, and C with coordinates (1, 2, 3), (4, 5, 6), and (7, 8, 9) respectively. The calculated values ​​are: d_AB = √((4 - 1)^2 + (5 - 2)^2 + (6 - 3)^2) = √27, d_AC = √((7 - 1)^2 + (8 - 2)^2 + (9 - 3)^2) = √72, and d_BC = √((7 - 4)^2 + (8 - 5)^2 + (9 - 6)^2) = √27. This generates an initial inter-individual distance matrix [[0, √27, √72], [√27, 0, √27], [√72, √27, 0]]. This initial inter-individual distance matrix visually represents the spatial distance relationships between individual birds.

[0150] H3: Normalize the initial inter-individual distance matrix to eliminate scale differences caused by different camera perspectives.

[0151] In one embodiment, the initial inter-individual distance matrix is ​​normalized. First, the maximum element `max_d` in the initial inter-individual distance matrix is ​​found. For each element `d_ij` in the matrix, normalization is calculated using the formula `d_ij_normalized = d_ij / max_d` to obtain the normalized elements. After normalizing all elements, a normalized distance matrix is ​​formed. For example, if the initial inter-individual distance matrix is ​​[[0, 3, 5], [3, 0, 4], [5, 4, 0]], the maximum element `max_d` = 5, and the normalized matrix is ​​[[0, 0.6, 1], [0.6, 0, 0.8], [1, 0.8, 0]]. Normalization eliminates scale differences caused by different camera perspectives.

[0152] H4: Calculate the frequency of each distance value in the normalized distance matrix to generate the distance distribution characteristics between individuals.

[0153] The distribution characteristics of distances between individuals can be represented by a frequency distribution table or histogram, reflecting the central tendency and dispersion of distances between bird individuals.

[0154] In one embodiment, the frequency of occurrence of each distance value in the normalized distance matrix is ​​statistically analyzed to generate inter-individual distance distribution features. The normalized distance matrix is ​​traversed, and the number of occurrences of each different distance value is recorded. Then, the frequency of each distance value is calculated as: frequency = number of occurrences / total number of non-zero elements in the matrix. For example, in a normalized distance matrix [[0, 0.6, 1], [0.6, 0, 0.8], [1, 0.8, 0]], the total number of non-zero elements is 6. The distance value 0.6 occurs 2 times, with a frequency of 2 / 6 = 1 / 3; the distance value 0.8 occurs 2 times, with a frequency of 2 / 6 = 1 / 3; and the distance value 1 occurs 2 times, with a frequency of 2 / 6 = 1 / 3.

[0155] S312: Analyze and process the distance distribution matrix between individuals to extract the distance distribution features.

[0156] In this embodiment, the distance distribution matrix between individuals contains distance information between multiple bird individuals. Analyzing and processing this matrix to extract distance distribution features allows for a deeper understanding of the distance distribution patterns among bird individuals. These distance distribution features may include statistical measures such as average distance, minimum distance, maximum distance, and distance variance.

[0157] In one embodiment, the distance distribution matrix between individuals is analyzed to extract its distance distribution characteristics. The average distance is obtained by calculating the mean of all non-zero elements in the matrix. The minimum non-zero element in the matrix is ​​found to obtain the minimum distance. The maximum element in the matrix is ​​found to obtain the maximum distance. The variance of all non-zero elements in the matrix is ​​calculated to reflect the dispersion of the distance. For example, for the distance distribution matrix between individuals [[0, 3, 5], [3, 0, 4], [5, 4, 0]], the average distance = (3 + 5 + 3 + 4 + 5 + 4) / 6 = 4, the minimum distance = 3, the maximum distance = 5, and the variance is calculated using the corresponding formula.

[0158] S313: Based on the tracking trajectory, calculate the number of intersections of the movement trajectories of each individual bird and generate the distribution characteristics of the movement trajectory intersections.

[0159] In this embodiment, the tracking trajectory records the movement paths of individual birds. Calculating the number of intersections in the movement trajectories of each individual bird can reflect the movement interactions between birds. By generating the distribution characteristics of the intersection points of movement trajectories, this interaction can be quantified.

[0160] In one embodiment, based on the tracking trajectories, the number of intersections in the movement trajectories of each individual bird is calculated. The tracking trajectory can be viewed as a series of point sets. For any two bird tracking trajectories, it is checked whether their point sets have identical or very close points; these points are considered intersections. The total number of intersections between all individual bird movement trajectories is then counted. Alternatively, time can be divided into different time periods, and the number of intersections in each time period can be counted to form a curve showing the change in the number of intersections over time. For example, analyzing the tracking trajectories of three birds over a period of time reveals that their movement trajectories have 5 intersections. The relevant information of these 5 intersections is then compiled into a movement trajectory intersection distribution characteristic.

[0161] S314: Integrate the distance distribution features with the movement trajectory intersection point distribution features to generate bird interaction information.

[0162] In one embodiment, distance distribution features and trajectory intersection point distribution features are integrated to generate bird interaction information. The distance distribution feature vector and the trajectory intersection point distribution feature vector can be concatenated to form a longer vector as the bird interaction information. For example, if the distance distribution feature vector is [0.2, 0.3, 0.5] and the trajectory intersection point distribution feature vector is [0.6, 0.4], concatenating them yields the bird interaction information [0.2, 0.3, 0.5, 0.6, 0.4]. Alternatively, a weighted summation method can be used, assigning different weights to the distance distribution features and the trajectory intersection point distribution features, and then summing them to obtain the bird interaction information.

[0163] Accordingly, to better implement the above methods, this application also provides a camera intelligent field-of-view switching system based on bird activity scenes. For example... Figure 8 As shown, the intelligent field-of-view switching system 80 based on bird activity scenes includes: The acquisition module 801 is used to acquire video streams through a camera deployed on a bird feeder, obtain raw image data, and perform preprocessing operations on the raw image data to generate a standardized image sequence; the standardized image sequence contains image frames arranged by timestamps, and each image frame is associated with its acquisition timestamp and the spatial orientation parameters of the camera; The feature processing module 802 is used to perform bird individual detection and continuous tracking on the standardized image sequence, identify and extract the motion features of each tracked bird individual based on the tracking results, and form a bird motion feature set; the motion features include at least head motion features representing the trajectory and frequency of head movement, leg motion features representing the leg movement pattern and posture, and body motion features representing the overall movement state of the body. The scene recognition module 803 is used to determine the current bird activity scene type based on the bird tracking results, and to obtain bird interaction information based on the bird activity scene type. The bird activity scene type includes single bird activity scene and multiple bird activity scene. Angle analysis module 804 is used to input the bird action feature set, the scene type and the bird interaction information into a pre-trained angle decision model, and to perform fusion analysis on the input information through the angle decision model to output a candidate set containing at least one candidate shooting angle and its corresponding estimated value score. Angle selection module 805 is used to calculate an urgency score representing the priority of the shooting task based on the scene type and the bird interaction information; and select a target shooting angle from the candidate set according to the urgency score. The control module 806 is used to generate control commands based on the target shooting angle. The control commands include the identifier of the target camera and the angle parameters to be adjusted. The control commands are sent to the corresponding camera gimbal drive module to drive the gimbal to move to the target shooting angle and complete the automatic switching of the field of view.

[0164] The implementation details of each module are provided in the preceding method embodiments and will not be repeated here. The technical effects achieved by each module and device are described in the foregoing method embodiments.

[0165] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application are still within the scope of this application.

Claims

1. A method for intelligent field-of-view switching of a camera based on bird activity scenes, characterized in that, Includes the following steps: Video streams are captured by cameras deployed on bird feeders to obtain raw image data, and the raw image data is preprocessed to generate a standardized image sequence. The standardized image sequence contains image frames arranged by timestamps, and each image frame is associated with its acquisition timestamp and the spatial orientation parameters of the camera. Bird individual detection and continuous tracking are performed on the standardized image sequence. Based on the tracking results, the motion features of each tracked bird individual are identified and extracted to form a set of bird motion features. The motion features include at least head motion features that characterize the trajectory and frequency of head movement, leg motion features that characterize the leg movement pattern and posture, and body motion features that characterize the overall movement state of the body. Based on the tracking results of the birds, the current bird activity scene type is determined, and bird interaction information is obtained based on the bird activity scene type. The bird activity scene type includes single bird activity scene and multiple bird activity scene. The bird motion feature set, the scene type, and the bird interaction information are input into a pre-trained angle decision model. The angle decision model performs fusion analysis on the input information and outputs a candidate set containing at least one candidate shooting angle and its corresponding estimated value score. Based on the scene type and the bird interaction information, an urgency score representing the priority of the shooting task is calculated; And select the target shooting angle from the candidate set based on the urgency score; A control command is generated based on the target shooting angle. The control command includes the identifier of the target camera and the angle parameters to be adjusted. The control command is sent to the corresponding camera gimbal driver module to drive the gimbal to move to the target shooting angle and complete the automatic switching of the field of view.

2. The method according to claim 1, characterized in that, The step of obtaining bird interaction information based on the bird activity scene type includes: When the scene is determined to be a scene with multiple birds, the bird interaction information is calculated based on the spatial relative position and movement trajectory relationship between the individual birds. When the scene is determined to be a single bird activity, the bird interaction information is set to a preset default value, which is used to indicate a no-interaction state.

3. The method according to claim 2, characterized in that, The step of selecting a target shooting angle from the candidate set based on the urgency score includes: The urgency score is compared with a preset threshold; if the urgency score is greater than the preset threshold, the candidate shooting angle with the highest estimated value score is selected from the candidate set as the target shooting angle; if the urgency score is not greater than the preset threshold, the candidate shooting angle with the highest priority in the candidate set is selected as the target shooting angle.

4. The method according to claim 3, characterized in that, The calculation of an urgency score, representing the priority of the shooting task, based on the scene type determination result and the bird interaction information includes: The scene type determination result is analyzed to extract the number and distribution density of individual birds in the current scene and generate a scene complexity score; When the scenario type is a scenario with multiple birds, analyze the correlation between the distance distribution and the direction of movement in the bird interaction information to generate an interaction intensity score; The scenario complexity score and the interaction intensity score are fused together to generate an initial urgency score; The initial urgency score is adjusted by combining the current field of view coverage of the camera with the spatial position of the target bird individual to form the final urgency score.

5. The method according to claim 4, characterized in that, The process involves performing individual detection and continuous tracking on the standardized image sequence, extracting the motion features of the tracked bird individuals, and forming a bird motion feature set, including: Perform bird individual detection on each image frame in the standardized image sequence to identify the existing bird individuals and label their location information to obtain bird individual location data; Based on the individual bird location data, a continuous tracking operation is performed on the individual bird to generate a bird movement path containing the tracking trajectory; The head movement trajectory and frequency are extracted from the tracking trajectory to form head movement features; Leg movement patterns and posture changes are extracted from the tracking trajectory to form leg movement features; The overall body motion state is extracted from the tracking trajectory to form body movement characteristics; The head movement features, leg movement features, and body movement features are integrated and processed to form a set of bird movement features.

6. The method according to claim 4, characterized in that, The process involves inputting the bird motion feature set, the scene type, and the bird interaction information into a pre-trained angle decision model. The angle decision model then performs a fusion analysis on the input information, outputting a candidate set containing at least one candidate shooting angle and its corresponding estimated value score, including: The head movement features, leg movement features, and body movement features in the bird movement feature set are input into the feature extraction module of the angle decision model to extract a multi-dimensional behavior feature vector that includes movement behavior complexity features, posture dynamic change features, and overall movement stability features. The scene type and the bird interaction information are input into the classification module of the angle decision model to generate a classification label for the scene type and a quantitative representation of the interaction information. The fusion analysis module of the angle decision model comprehensively processes the multi-dimensional behavioral feature vector, classification label and quantitative representation to generate an initial list of candidate shooting angles. For each candidate shooting angle in the initial list, a predicted value score is calculated to generate a candidate set containing the score; The candidate shooting angles in the candidate set are sorted according to the estimated value score to generate the final candidate set.

7. The method according to claim 6, characterized in that, The fusion analysis module of the angle decision model comprehensively processes the multi-dimensional behavioral feature vector, classification label, and quantization representation to generate an initial list of candidate shooting angles, including: The multi-dimensional behavioral feature vector is concatenated with the classification label to form a fused feature vector; A weighted superposition operation is performed on the fused feature vector and the quantized representation to generate a comprehensive feature representation; Based on the comprehensive feature representation, a preset angle generation rule base is invoked to generate a preliminary set of candidate shooting angles; Geometric feasibility is verified on the candidate shooting angles in the preliminary set, and unfeasible angles are eliminated to obtain an initial list of candidate shooting angles.

8. The method according to claim 2, characterized in that, Based on the spatial relative positions and movement trajectories of individual birds, bird interaction information is calculated, including: When the scene is determined to be a scene with multiple birds, the spatial relative positions between individual birds are calculated to generate a distance distribution matrix between individuals. The distance distribution matrix between individuals is analyzed and processed to extract the distance distribution features. Based on the tracking trajectory, the number of intersections of the movement trajectories of each individual bird is calculated, and the distribution characteristics of the movement trajectory intersections are generated. The distance distribution features and the intersection point distribution features of the movement trajectory are integrated and processed to generate bird interaction information.

9. The method according to claim 8, characterized in that, The calculation of the spatial relative positions between individual birds to generate an inter-individual distance distribution matrix includes: Based on the tracking results of the birds, the spatial coordinates of each individual bird are obtained; Calculate the Euclidean distance between any two individual birds to generate an initial inter-individual distance matrix; The initial inter-individual distance matrix is ​​normalized to eliminate scale differences caused by different camera perspectives; The frequency of each distance value in the normalized distance matrix is ​​statistically analyzed to generate the distance distribution characteristics between individuals.

10. A camera intelligent field-of-view switching system based on bird activity scenes, characterized in that, The system includes: The acquisition module is used to acquire video streams through cameras deployed on bird feeders, obtain raw image data, and perform preprocessing operations on the raw image data to generate a standardized image sequence; the standardized image sequence contains image frames arranged by timestamps, and each image frame is associated with its acquisition timestamp and the spatial orientation parameters of the camera; The feature processing module is used to perform bird individual detection and continuous tracking on the standardized image sequence, identify and extract the motion features of each tracked bird individual based on the tracking results, and form a bird motion feature set; the motion features include at least head motion features that characterize the trajectory and frequency of head movement, leg motion features that characterize the leg movement pattern and posture, and body motion features that characterize the overall movement state of the body. The scene recognition module is used to determine the current bird activity scene type based on the bird tracking results, and to obtain bird interaction information based on the bird activity scene type. The bird activity scene type includes single bird activity scene and multiple bird activity scene. An angle analysis module is used to input the bird action feature set, the scene type, and the bird interaction information into a pre-trained angle decision model. The angle decision model performs fusion analysis on the input information and outputs a candidate set containing at least one candidate shooting angle and its corresponding estimated value score. An angle selection module is used to calculate an urgency score that represents the priority of the shooting task based on the scene type and the bird interaction information; and select a target shooting angle from the candidate set according to the urgency score. The control module is used to generate control commands based on the target shooting angle. The control commands include the identifier of the target camera and the angle parameters to be adjusted. The control commands are sent to the corresponding camera gimbal drive module to drive the gimbal to move to the target shooting angle and complete the automatic switching of the field of view.

Images