Bird observation method, device and equipment based on camera group and readable medium

Abstract

The embodiment of the invention discloses a bird observation method, device and equipment based on a camera group and a readable medium. A specific embodiment of the method comprises the following steps: respectively initializing camera parameters of a main camera and an auxiliary camera included in the bird observation system; controlling a main camera to collect an environment video corresponding to the first target area; based on the environment video, executing a bird target recognition task to generate a bird target recognition result; according to the bird target identification result, determining a movement track corresponding to the target bird to generate bird movement track information; generating a second target area corresponding to the target bird, and controlling the auxiliary camera to execute a target tracking task corresponding to the target bird based on the second target area to obtain a bird tracking video; and transmitting the environment video and the bird tracking video to an associated target terminal for synchronous display. According to the embodiment, the situation that panoramic information and observation information are lost during bird observation is avoided.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a bird observation method, apparatus, device, and readable medium based on a camera group. Background Art

[0002] With the advancement of 4K high-definition camera technology and intelligent detection algorithms, remote observation of wildlife, especially birds, has become a popular application area for science education and ecological research. Currently, remote bird observation typically involves using a single camera to track the bird's main area through digital zoom or pan / tilt rotation, combined with target detection-based local selection and display to enhance visualization of local details.

[0003] However, when using the above method for remote bird observation, the following technical problems often arise:

[0004] When remotely observing birds using a single camera, zooming in on the camera image will result in a loss of the overall view, making it impossible to observe both the overall environment and local details at the same time. Furthermore, zooming in on the image can easily lead to a decrease in image quality, which in turn can cause a loss of observation information.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the Invention

[0006] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure propose a bird observation method, device, electronic device, and computer-readable medium based on a camera group to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a bird observation method based on a camera group, the method comprising: respectively initializing the camera parameters of the main camera and the auxiliary camera included in the above-mentioned bird observation system, wherein the above-mentioned camera parameters include: zoom ratio, timestamp and exposure focus parameters; controlling the above-mentioned main camera to collect the environmental video corresponding to the first target area; based on the above-mentioned environmental video, performing a bird target recognition task to generate a bird target recognition result; according to the above-mentioned bird target recognition result, determining the corresponding movement trajectory of the target bird to generate bird movement trajectory information; generating a second target area corresponding to the above-mentioned target bird, and based on the above-mentioned second target area, controlling the above-mentioned auxiliary camera to perform the target tracking task corresponding to the above-mentioned target bird to obtain a bird tracking video; transmitting the above-mentioned environmental video and the above-mentioned bird tracking video to the associated target terminal for synchronous display.

[0009] In a second aspect, some embodiments of the present disclosure provide a bird observation device based on a camera group, the device comprising: an initialization unit, configured to respectively initialize the camera parameters of a main camera and an auxiliary camera included in the bird observation system, wherein the above-mentioned camera parameters include: zoom ratio, timestamp and exposure focus parameters; a control unit, configured to control the above-mentioned main camera to collect an environmental video corresponding to a first target area; an execution unit, configured to perform a bird target recognition task based on the above-mentioned environmental video to generate a bird target recognition result; a determination unit, configured to determine the movement trajectory corresponding to the target bird according to the above-mentioned bird target recognition result to generate bird movement trajectory information; a generation unit, configured to generate a second target area corresponding to the above-mentioned target bird, and based on the above-mentioned second target area, control the above-mentioned auxiliary camera to perform a target tracking task corresponding to the above-mentioned target bird to obtain a bird tracking video; a transmission unit, configured to transmit the above-mentioned environmental video and the above-mentioned bird tracking video to an associated target terminal for synchronous display.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0012] The above-described embodiments of the present disclosure have the following advantageous effects: The camera group-based bird observation method of some embodiments of the present disclosure avoids the loss of panoramic information and observation information. Specifically, the reason for the loss of panoramic information and observation information is that when remotely observing birds using a single camera, zooming in on the camera image can lead to the loss of panoramic information, making it impossible to simultaneously observe the overall environment and local details. Furthermore, zooming in on the image can easily lead to a decrease in image quality, which in turn leads to the loss of observation information. Based on this, the camera group-based bird observation method of some embodiments of the present disclosure first initializes the camera parameters of the main camera and auxiliary camera included in the bird observation system. Thus, the camera group can be initialized according to pre-set camera parameters. Next, the main camera is controlled to capture an environmental video corresponding to a first target area. This allows the capture of panoramic environmental information for bird observation. Then, based on the environmental video, a bird target recognition task is performed to generate a bird target recognition result. This allows the captured bird to be identified. Then, based on the bird target recognition result, the movement trajectory of the target bird is determined to generate bird movement trajectory information. This allows the movement trajectory of the observed bird to be determined. Then, a second target area corresponding to the target bird is generated, and based on the second target area, the auxiliary camera is controlled to perform the target tracking task corresponding to the target bird, thereby obtaining a bird tracking video. In this way, the auxiliary camera can be controlled to accurately observe the bird, and at the same time, the loss of observation details can be avoided when the content captured by the auxiliary camera is magnified. Finally, the environmental video and the bird tracking video are transmitted to the associated target terminal for synchronous display. In this way, the captured panoramic video and bird video can be displayed synchronously, avoiding the loss of panoramic information and observation information. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of a bird observation method based on a camera group according to the present disclosure;

[0015] Figure 2 is a schematic structural diagram of some embodiments of a bird observation device based on a camera group according to the present disclosure;

[0016] Figure 3 It is a structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0019] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 The process 100 of some embodiments of the bird observation method based on a camera group according to the present disclosure is shown. The bird observation method based on a camera group includes the following steps:

[0024] Step 101: Initialize the camera parameters of the main camera and auxiliary camera included in the bird observation system respectively.

[0025] In some embodiments, the execution entity (e.g., a server) of the camera group-based bird observation method can initialize the camera parameters of the primary and secondary cameras included in the bird observation system. The camera parameters include: zoom ratio, timestamp, and exposure and focus parameters. The zoom ratio can be the zoom ratio between the camera-captured image and the real image. The timestamp can indicate the time when the video was captured.

[0026] Step 102: Control the main camera to capture the environment video corresponding to the first target area.

[0027] In some embodiments, the execution entity may control the main camera to capture an environment video corresponding to a first target area, wherein the first target area may be an area that can be captured by the main camera.

[0028] Optionally, after step 201, the following steps are further included:

[0029] In the first step, video frame extraction processing is performed on the above-mentioned environmental video to generate an environmental video frame sequence.

[0030] In some embodiments, the execution subject may perform a video frame extraction process on the environment video to generate an environment video frame sequence, wherein the video frame extraction process may be to extract each environment video frame constituting the environment video.

[0031] In the second step, key frame extraction processing is performed on the above-mentioned environment video frame sequence to generate environment key frames, and the environment key frame sequence is obtained as the environment video.

[0032] In some embodiments, the execution entity may perform key frame extraction processing on the environmental video frame sequence to generate environmental key frames, thereby obtaining the environmental key frame sequence as the environmental video. The key frame extraction processing may be performed on the environmental video frame sequence using a key frame extraction algorithm based on OpenCV.

[0033] Step 103: Based on the above environment video, perform a bird target recognition task to generate a bird target recognition result.

[0034] In some embodiments, the execution entity may perform a bird target recognition task based on the environment video to generate a bird target recognition result. In practice, the execution entity may perform the bird target recognition task on each frame of the environment video to generate a bird target recognition result.

[0035] In practice, the bird target recognition task can be performed by following the steps below:

[0036] In the first step, the video frames included in the environmental video are input into a pre-trained bird target recognition model to generate a bird recognition video frame as a bird target recognition result. The bird recognition video frame is displayed with a bird detection frame. The bird target recognition model can be a lightweight model for bird recognition based on YOLOv8.

[0037] The second step is to expand the bird detection frame displayed in the bird recognition video frame to generate an expanded bird recognition video frame. In practice, the bird detection frame can be expanded by extending the side length of the bird detection frame according to a predetermined pixel length to expand the bird detection frame displayed in the bird recognition video frame, thereby generating an expanded bird recognition video frame.

[0038] Step 104 : determining the movement trajectory of the target bird according to the above bird target recognition result to generate bird movement trajectory information.

[0039] In some embodiments, the execution entity may determine the movement trajectory corresponding to the target bird according to the bird target recognition result to generate bird movement trajectory information.

[0040] In some optional implementations of some embodiments, the execution entity may determine the movement trajectory corresponding to the target bird through the following steps:

[0041] In the first step, in response to the bird target recognition result indicating that a bird has been recognized, a preset number of video frames are selected from the environment video as a target video frame sequence. The preset number may be a pre-set number of selected video frames. As an example, the preset number may be 15 frames.

[0042] In the second step, each target video frame included in the target video frame sequence is filtered to generate filtered video frames, thereby obtaining a filtered video frame sequence. The filtering process may be performed using a trajectory smoothing algorithm based on Savitzky-Golay filtering.

[0043] The third step is to perform positioning processing on the filtered video frame sequence to generate a trajectory coordinate sequence. Here, positioning processing can be performed on the bird shown in each filtered video frame in the filtered video frame sequence to generate the trajectory coordinates.

[0044] The fourth step is to determine the target bird's corresponding movement speed based on the environmental video and the trajectory coordinate sequence. In practice, the target bird's corresponding movement speed can be determined based on the duration between each two frames in the environmental video and the distance between each two trajectory coordinates in the trajectory coordinate sequence. The movement speed is represented by a velocity vector.

[0045] In the fifth step, based on the smoothed trajectory coordinate sequence and the movement speed, trajectory prediction processing is performed on the target bird to generate a trajectory prediction result. Here, trajectory prediction processing can be performed on the target bird using an extended Kalman filter using a linear motion model to generate a trajectory prediction result. The trajectory prediction result can represent the predicted position of the bird in the five frames following the last frame of the environment video.

[0046] The sixth step is to combine the above trajectory prediction results and the above smoothed trajectory coordinate sequence into the bird movement trajectory information.

[0047] Step 105 : generating a second target area corresponding to the target bird, and based on the second target area, controlling the auxiliary camera to perform a target tracking task corresponding to the target bird, thereby obtaining a bird tracking video.

[0048] In some embodiments, the execution entity may generate a second target area corresponding to the target bird, and based on the second target area, control the auxiliary camera to perform a target tracking task corresponding to the target bird, thereby obtaining a bird tracking video. The second target area may be the range of a circumscribed matrix of a circle with the target bird as the center and a predetermined radius.

[0049] In practice, the following steps may be performed to generate a second target area corresponding to the target bird, and based on the second target area, control the auxiliary camera to perform the target tracking task corresponding to the target bird:

[0050] In the first step, for each track coordinate included in the bird movement track information, the track coordinate is mapped to the coordinate system of the auxiliary camera to generate a mapped track coordinate.

[0051] The second step is to generate a second target region based on the generated mapped trajectory coordinates. In practice, for each of the mapped trajectory coordinates, a circumscribed matrix corresponding to the coordinate can be determined, and the circumscribed matrix is ​​defined as the second target sub-region. The determined second target sub-regions are then merged into the second target region.

[0052] The third step is to determine the shooting parameters corresponding to each of the mapped trajectory coordinates, wherein the shooting parameters may include a pitch angle and a horizontal angle.

[0053] The fourth step is to control the auxiliary camera to perform the target tracking task corresponding to the target bird based on the generated shooting parameters and the above-mentioned moving speed, so as to obtain a bird tracking video.

[0054] Step 106: Transmit the environmental video and the bird tracking video to an associated target terminal for synchronous display.

[0055] In some embodiments, the execution entity may transmit the environmental video and the bird tracking video to an associated target terminal for synchronous display.

[0056] While implementing technical solutions to the aforementioned technical problem, the following technical issues often arise: A delay between the primary and secondary cameras prevents the synchronized display of the surrounding video and the bird tracking video. Furthermore, since bird observation equipment is typically deployed outdoors, the bandwidth required to transmit large video streams in real time is insufficient. Given these technical issues and the current state of technology, the following solutions were chosen.

[0057] In practice, the environmental video and the bird tracking video can be transmitted to the associated target terminal for synchronous display by the following steps:

[0058] In the first step, metadata of the environment video and the bird tracking video are extracted to generate a metadata set, wherein the metadata may be timestamp (PTS) metadata.

[0059] The second step is to align the timestamps of the environment video and the bird tracking video based on the metadata groups. In practice, first, the video frames in the environment video and the bird tracking video can be sorted according to the corresponding metadata groups to obtain the environment video frame sequence and the bird tracking video frame sequence. Second, the timestamps of the environment video frame sequence and the bird tracking video frame sequence can be aligned using the timestamps represented by the corresponding metadata.

[0060] In a third step, in response to a delay between the bird tracking video and the environment video being greater than a preset delay, at least one delayed video frame in the environment video is deleted. The preset delay may be a predetermined delay of the bird tracking video. For example, the preset delay may be 40 ms.

[0061] The fourth step is to adjust the layout of the bird tracking video and the environment video in the preset rendering template based on the preset rendering template to obtain a rendered image. In practice, the bird tracking video and the environment video can be rendered according to a preset scaling ratio according to the preset rendering template.

[0062] The fifth step is to determine the terminal bandwidth corresponding to the target terminal in real time.

[0063] In a sixth step, in response to the terminal bandwidth being less than or equal to the preset bandwidth, the video resolution of the bird tracking video displayed on the rendering screen is adjusted.

[0064] In the seventh step, the rendered image is sent to the target terminal for synchronous display. Here, a variety of screen layouts can be set. As an example, it can be picture-in-picture (PIP), top-bottom split screen, left-right split screen, etc. The bird tracking video can be 20%-30% of the size of the environment video. The aspect ratio of the bird tracking video to the environment video can also be dynamically adjusted based on the size of the bird or its flight status.

[0065] Optionally, after step 7, the bird tracking video and the environment video are sent to the same playback buffer pool, and the bird tracking video is played in the playback buffer pool using the timestamp corresponding to each frame of the environment video. This can prevent screen tearing and audio-video asynchrony.

[0066] Steps 1 through 7, as an inventive feature of an embodiment of the present disclosure, address the technical problem that "the delay between the primary and secondary cameras prevents the synchronized display of the ambient video and the bird-tracking video. Furthermore, since bird-observing equipment is typically located outdoors, the bandwidth connected to the equipment cannot support real-time transmission of the large video stream, resulting in a delay in the transmitted video." The reasons for the synchronized display of the ambient video and the bird-tracking video, as well as the delay in the transmitted video, are as follows: the delay between the primary and secondary cameras prevents the synchronized display of the ambient video and the bird-tracking video. Furthermore, since bird-observing equipment is typically located outdoors, the bandwidth connected to the equipment cannot support real-time transmission of the large video stream. Resolving these factors ensures synchronized display of the ambient video and the bird-tracking video, while avoiding delays in the transmitted video. To achieve this, the present disclosure first extracts metadata from the ambient video and the bird-tracking video to generate metadata sets. This allows the determination of timestamp metadata corresponding to each video frame. Second, based on the metadata sets, timestamps of the ambient video and the bird-tracking video are aligned. This allows the timestamp alignment to be performed using metadata. Third, in response to the delay between the bird tracking video and the environment video being greater than a preset delay, at least one delayed video frame in the environment video is deleted.

[0067] In the process of adopting technical solutions to solve the above-mentioned technical problem 1, the following problems often arise: when the image taken by the auxiliary camera is enlarged, the resolution of the auxiliary camera is insufficient, resulting in unclear bird images after enlargement. In response to the above technical problems, the conventional solution is generally to use an auxiliary camera with higher resolution for shooting and acquisition. However, the above-mentioned conventional solution still has the following problems: the cost of auxiliary cameras with higher resolution is high, and more shooting equipment is required when shooting birds, resulting in higher costs for bird observation. In the face of the above-mentioned technical problems, combined with the current technical status, it can be decided to adopt the following solutions.

[0068] Optionally, after step 106, the following steps are further included:

[0069] In the first step, in response to detecting a zoom-in operation of the target terminal on the bird tracking video, a zoom-in parameter corresponding to the zoom-in operation is determined.

[0070] In some embodiments, the execution entity may, in response to detecting a zoom-in operation on the bird tracking video by the target terminal, determine a magnification parameter corresponding to the zoom-in operation. The zoom-in parameter includes a selection range and a magnification factor. The selection range may be a matrix range with a predetermined length and a center of the bird tracking video as the matrix center. The magnification factor may be the magnification factor of the zoom-in operation on the bird tracking video.

[0071] The second step is to capture a selected range corresponding to the above-mentioned magnification parameters from the above-mentioned bird tracking video to generate a captured image.

[0072] In some embodiments, the execution entity may capture a selected range corresponding to the zoom parameter from the bird tracking video to generate a captured image.

[0073] The third step is to perform normalization processing on each pixel included in the above-mentioned intercepted image to generate a normalized image.

[0074] In some embodiments, the execution entity may perform normalization processing on each pixel included in the captured image to generate a normalized image, wherein the normalization processing may be normalizing the pixel value of each pixel of the captured image to a range of [-1, 1].

[0075] The fourth step is to determine the hardware parameter information corresponding to the auxiliary camera.

[0076] In some embodiments, the execution entity may determine hardware parameter information corresponding to the auxiliary camera, wherein the hardware parameter information may be resolution information of the auxiliary camera.

[0077] In the fifth step, based on the hardware parameter information, super-resolution processing is performed on the normalized image to generate a super-resolution image.

[0078] In some embodiments, the execution entity may perform super-resolution processing on the normalized image based on the hardware parameter information to generate a super-resolution image. In practice, the super-resolution processing may be performed on the normalized image using a super-resolution reconstruction algorithm. Here, in response to the hardware parameter information being less than or equal to a preset resolution, the super-resolution processing may be performed on the normalized image to generate a super-resolution image.

[0079] In the sixth step, an unsharp mask process is performed on the super-resolution image to generate a masked image.

[0080] In some embodiments, the execution entity may perform unsharp masking on the super-resolution image to generate a masked image.

[0081] In the seventh step, edge fusion processing is performed on the masked image to generate an edge fused masked image.

[0082] In some embodiments, the execution entity may perform edge blending on the masked image to generate an edge-blended masked image. The edge blending process may include feathering and dynamic masking. The feathering process may include feathering the region boundaries of the masked image using Gaussian blur.

[0083] In the eighth step, the edge-fused mask image is sent to the target terminal for display.

[0084] In some embodiments, the execution entity may send the edge-fused mask image to the target terminal for display.

[0085] Steps 1 to 8, as an inventive feature of an embodiment of the present disclosure, address the technical problem: "When zooming in on the image captured by the auxiliary camera, the auxiliary camera's resolution is insufficient, resulting in an unclear magnified bird image." The unclear magnified bird image is caused by the following: When zooming in on the image captured by the auxiliary camera, the auxiliary camera's resolution is insufficient, resulting in an unclear magnified bird image. Resolving these factors can prevent the unclear magnified bird image. To achieve this, the present disclosure firstly, in response to detecting a zoom operation on the bird tracking video by the target terminal, determines a zoom parameter corresponding to the zoom operation. This allows the user's zoom parameter for the image to be determined. Second, extracts a selected range from the bird tracking video corresponding to the zoom parameter to generate a captured image. This allows the user to capture the image range desired for magnification. Third, normalizes each pixel in the captured image to generate a normalized image; determines hardware parameter information corresponding to the auxiliary camera; and, based on the hardware parameter information, performs super-resolution processing on the normalized image to generate a super-resolution image. Thus, when the resolution of the auxiliary camera is low, super-resolution reconstruction can be used to improve the clarity of the details after magnification. Fourth, the super-resolution image is subjected to unsharp masking to generate a masked image; and the masked image is subjected to edge fusion processing to generate an edge-fused masked image. Thus, the naturalness of the image can be improved by unsharp masking and edge fusion, and the disconnection with the surrounding image can be avoided, thereby avoiding the inconsistency of contextual information. Fifth, the edge-fused masked image is sent to the target terminal for display. Thus, the unclear image of the bird after magnification is avoided.

[0086] The above-described embodiments of the present disclosure have the following advantageous effects: The camera group-based bird observation method of some embodiments of the present disclosure avoids the loss of panoramic information and observation information. Specifically, the reason for the loss of panoramic information and observation information is that when remotely observing birds using a single camera, zooming in on the camera image can lead to the loss of panoramic information, making it impossible to simultaneously observe the overall environment and local details. Furthermore, zooming in on the image can easily lead to a decrease in image quality, which in turn leads to the loss of observation information. Based on this, the camera group-based bird observation method of some embodiments of the present disclosure first initializes the camera parameters of the main camera and auxiliary camera included in the bird observation system. Thus, the camera group can be initialized according to pre-set camera parameters. Next, the main camera is controlled to capture an environmental video corresponding to a first target area. This allows the capture of panoramic environmental information for bird observation. Then, based on the environmental video, a bird target recognition task is performed to generate a bird target recognition result. This allows the captured bird to be identified. Then, based on the bird target recognition result, the movement trajectory of the target bird is determined to generate bird movement trajectory information. This allows the movement trajectory of the observed bird to be determined. Then, a second target area corresponding to the target bird is generated, and based on the second target area, the auxiliary camera is controlled to perform the target tracking task corresponding to the target bird, thereby obtaining a bird tracking video. In this way, the auxiliary camera can be controlled to accurately observe the bird, and at the same time, the loss of observation details can be avoided when the content captured by the auxiliary camera is magnified. Finally, the environmental video and the bird tracking video are transmitted to the associated target terminal for synchronous display. In this way, the captured panoramic video and bird video can be displayed synchronously, avoiding the loss of panoramic information and observation information.

[0087] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a bird observation device based on a camera group. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the bird observation device based on the camera group can be specifically applied to various electronic devices.

[0088] like Figure 2As shown, some embodiments of a bird observation device 200 based on a camera group include: an initialization unit 201, a control unit 202, an execution unit 203, a determination unit 204, a generation unit 205, and a transmission unit 206. The initialization unit 201 is configured to respectively initialize the camera parameters of the main camera and the auxiliary camera included in the bird observation system, wherein the camera parameters include: zoom ratio, timestamp, and exposure and focus parameters; the control unit 202 is configured to control the main camera to capture an environmental video corresponding to a first target area; the execution unit 203 is configured to perform a bird target recognition task based on the environmental video to generate a bird target recognition result; the determination unit 204 is configured to determine the movement trajectory corresponding to the target bird based on the bird target recognition result to generate bird movement trajectory information; the generation unit 205 is configured to generate a second target area corresponding to the target bird, and based on the second target area, control the auxiliary camera to perform a target tracking task corresponding to the target bird to obtain a bird tracking video; and the transmission unit 206 is configured to transmit the environmental video and the bird tracking video to an associated target terminal for synchronous display.

[0089] It is understandable that the units described in the bird watching device 200 based on the camera group are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the bird watching device 200 based on the camera group and the units contained therein, and will not be repeated here.

[0090] Reference below Figure 3 , which shows a schematic structural diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0091] like Figure 3As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0092] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0093] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0094] It should be noted that the computer-readable medium described in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0095] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0096] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: initializes the camera parameters of the main camera and auxiliary camera included in the bird observation system, respectively, wherein the camera parameters include: zoom ratio, timestamp, and exposure and focus parameters. Controls the main camera to capture an environmental video corresponding to a first target area. Based on the environmental video, performs a bird target recognition task to generate a bird target recognition result. Based on the bird target recognition result, determines the movement trajectory corresponding to the target bird to generate bird movement trajectory information. Generates a second target area corresponding to the target bird, and based on the second target area, controls the auxiliary camera to perform a target tracking task corresponding to the target bird to obtain a bird tracking video. Transmits the environmental video and the bird tracking video to an associated target terminal for synchronous display.

[0097] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0099] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor. For example, they may be described as follows: a processor including an initialization unit, a control unit, an execution unit, a determination unit, a generation unit, and a transmission unit. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, the initialization unit may also be described as a "unit for respectively initializing the camera parameters of the main camera and the auxiliary camera included in the bird observation system."

[0100] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0101] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A bird observation method based on a camera group, applied to a bird observation system, wherein: The bird observation system includes a main camera and an auxiliary camera, and the method includes: Initializing the camera parameters of the main camera and the auxiliary camera included in the bird observation system respectively, wherein the camera parameters include: zoom ratio, timestamp, and exposure and focus parameters; Controlling the main camera to capture an environmental video corresponding to the first target area; Based on the environment video, performing a bird target recognition task to generate a bird target recognition result; Determining the movement trajectory of the target bird according to the bird target recognition result to generate bird movement trajectory information; generating a second target area corresponding to the target bird, and controlling the auxiliary camera to perform a target tracking task corresponding to the target bird based on the second target area to obtain a bird tracking video; The environmental video and the bird tracking video are transmitted to an associated target terminal for synchronous display.

2. The method according to claim 1, wherein After controlling the main camera to capture the environment video corresponding to the first target area, the method further includes: Performing video frame extraction processing on the environmental video to generate an environmental video frame sequence; A key frame extraction process is performed on the environmental video frame sequence to generate environmental key frames, and an environmental key frame sequence is obtained as the environmental video.

3. The method according to claim 1, wherein The performing bird target recognition processing based on the environmental video to generate a bird target recognition result includes: Inputting the video frames included in the environment video into a pre-trained bird target recognition model to generate a bird recognition video frame as a bird target recognition result, wherein the bird recognition video frame displays a bird detection frame; The bird detection frame displayed in the bird recognition video frame is expanded to generate an expanded bird recognition video frame.

4. The method according to claim 1, wherein The step of determining the movement trajectory of the target bird according to the bird target recognition result to generate bird movement trajectory information includes: In response to the bird target recognition result indicating that a bird is recognized, selecting a preset number of video frames from the environment video as a target video frame sequence; Performing filtering on each target video frame included in the target video frame sequence to generate filtered video frames, thereby obtaining a filtered video frame sequence; performing positioning processing on the filtered video frame sequence to generate a trajectory coordinate sequence; Determining a moving speed corresponding to the target bird based on the environmental video and the trajectory coordinate sequence; performing trajectory prediction processing on the target bird according to the smoothed trajectory coordinate sequence and the moving speed to generate a trajectory prediction result; The trajectory prediction result and the smoothed trajectory coordinate sequence are combined into bird movement trajectory information.

5. The method according to claim 4, wherein The generating of a second target area corresponding to the target bird, and controlling the auxiliary camera to perform a target tracking task corresponding to the target bird based on the second target area to obtain a bird tracking video, includes: For each track coordinate included in the bird movement track information, mapping the track coordinate to the coordinate system of the auxiliary camera to generate a mapped track coordinate; generating a second target area based on the generated coordinates of each mapped trajectory; For each of the mapped trajectory coordinates, determining a shooting parameter corresponding to the mapped trajectory coordinate, wherein the shooting parameter includes: a pitch angle and a horizontal angle; Based on the generated shooting parameters and the moving speed, the auxiliary camera is controlled to perform a target tracking task corresponding to the target bird to obtain a bird tracking video.

6. A bird observation device based on a camera group, comprising: An initialization unit is configured to respectively initialize camera parameters of a main camera and an auxiliary camera included in the bird observation system, wherein the camera parameters include: a zoom ratio, a timestamp, and exposure and focus parameters; a control unit configured to control the main camera to capture an environment video corresponding to the first target area; an execution unit, configured to execute a bird target recognition task based on the environment video to generate a bird target recognition result; a determining unit configured to determine a movement trajectory corresponding to the target bird according to the bird target recognition result to generate bird movement trajectory information; a generating unit configured to generate a second target area corresponding to the target bird, and based on the second target area, control the auxiliary camera to perform a target tracking task corresponding to the target bird, thereby obtaining a bird tracking video; The transmission unit is configured to transmit the environment video and the bird tracking video to an associated target terminal for synchronous display.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images