Target counting method, target counting system and computer equipment

Abstract

The invention provides a target counting method, a target counting system and computer equipment. The method comprises the following steps: controlling a camera group to move; acquiring video data acquired by each camera of the camera group in the moving process and splicing the video data to obtain a panoramic video covering the monitoring area; and identifying and tracking targets in the panoramic video, and determining the number of the targets in the monitoring area. Wherein the camera group comprises a plurality of cameras which are arranged at intervals in the first direction of the top of the monitoring area and have downward lenses, each camera is configured to be capable of moving towards the second direction of the top of the monitoring area at the same time, the first direction is different from the second direction, and the field angle of the camera group covers the range of the first direction of the monitoring area; and the maximum moving distance of the camera group enables the acquisition range of each camera in the time dimension to cover the range of the second direction of the monitoring area. By optimizing the layout of the camera group, the accuracy of counting targets based on videos is improved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to a target counting method, a target counting system, and a computer device. Background Art

[0002] In the livestock and poultry breeding industry, counting the number of livestock and poultry is an important part of daily management.

[0003] With the development of technology, setting up cameras in livestock and poultry farms, collecting images by the cameras and counting the number of livestock and poultry through image recognition technology has become an efficient and potential solution. However, the livestock and poultry house scene is usually large, the environment is complex, the lighting conditions vary, and the density of the livestock and poultry group is different. These factors greatly increase the difficulty of image recognition, resulting in the accuracy of target counting being affected. Summary of the Invention

[0004] To solve the existing technical problems, the present application provides a target counting method, a target counting system, and a computer device with relatively high accuracy.

[0005] In a first aspect, a target counting method is provided. The method includes:

[0006] Controlling the movement of a camera group; the camera group includes a plurality of cameras with downward-facing lenses spaced in a first direction at the top of a monitoring area, and each of the cameras is configured to be movable simultaneously in a second direction at the top of the monitoring area. The first direction and the second direction are different. The field of view angle of the camera group covers the range in the first direction of the monitoring area, and the maximum movement distance of the camera group enables the acquisition ranges of each of the cameras in the time dimension to cover the range in the second direction of the monitoring area;

[0007] Obtaining and stitching the video data collected by each camera of the camera group during the movement to obtain a panoramic video covering the monitoring area;

[0008] Identifying and tracking the targets in the panoramic video to determine the number of targets in the monitoring area.

[0009] In a second aspect, a target counting system is provided, including a camera group, a motion device, and a computer device; the camera group includes a plurality of cameras with downward-facing lenses spaced in a first direction at the top of a monitoring area, and each of the cameras is configured to be movable simultaneously in a second direction at the top of the monitoring area. The first direction and the second direction are different. The field of view angle of the camera group covers the range in the first direction of the monitoring area, and the maximum movement distance of the camera group enables the acquisition ranges of each of the cameras in the time dimension to cover the range in the second direction of the monitoring area;

[0010] The computer device includes a processor and a memory connected to the processor. A computer program executable by the processor is stored on the memory. When the computer program is executed by the processor, the steps of the above-mentioned target counting method are implemented.

[0011] In a third aspect, a computer device is provided, which includes a processor and a memory connected to the processor. A computer program executable by the processor is stored on the memory. When the computer program is executed by the processor, the steps of the above-mentioned target counting method are implemented.

[0012] For the target counting method provided in the above embodiment, a camera group composed of multiple cameras is utilized. The camera group is arranged in the first direction at the top of the monitoring area, and the multiple cameras are spaced apart so that the imaging range of the camera group in a single-frame dimension covers the range of the first direction of the monitoring area. Each camera of the camera group is configured to be movable in the second direction of the monitoring area simultaneously, and its maximum moving distance enables the acquisition range of each camera in the time dimension to cover the range of the second direction of the monitoring area. In this way, by stitching the video data of each camera of the camera group, a panoramic video covering the monitoring area can be obtained. Then, target recognition and tracking processing are performed on the panoramic video to determine the number of targets in the monitoring area. This method can cover a relatively large monitoring area with a limited number of cameras by optimizing the layout of the camera group, and then use the panoramic video to identify and count the targets in the monitoring area. In this method, the cameras face down to shoot the monitoring area, which can reduce the probability of distortion and occlusion and improve the accuracy of counting targets based on the video. This method can be applied to the livestock counting scenario in a livestock breeding site with a large site area.

[0013] The target counting system provided in the above embodiment belongs to the same concept as the corresponding target counting method embodiment, and thus has the same technical effects as the corresponding target counting method embodiment, which will not be elaborated here. Description of the Drawings

[0014] Figure 1 It is a schematic structural diagram of the target counting system in an embodiment.

[0015] Figure 2 It is a flowchart of the target counting method in an embodiment.

[0016] Figure 3 It is a schematic diagram of the relative positions of the monitoring area, the effective recognition area, and the acquisition range of the field of view angle in an embodiment.

[0017] Figure 4 It is a schematic diagram of the relative position relationship between the effective recognition area and the counting line in an embodiment.

[0018] Figures 5 to 7Schematic diagram of tracking target count through counting lines in an embodiment.

[0019] Figure 8 Schematic diagram of the effect of distortion correction in an embodiment. Detailed implementation manners

[0020] The technical solution of the present invention will be further elaborated in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0021] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the drawings. The described embodiments should not be regarded as limitations of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0022] In the following description, the expression "some embodiments" is involved, which describes a subset of all possible embodiments. It should be noted that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0023] In the following description, the terms "first, second, third" involved are only used to distinguish similar objects, and do not represent a specific order for the objects. It can be understood that "first, second, third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0024] In the livestock and poultry breeding industry, breeders need to often count the number of livestock and poultry. Although target detection algorithms have been relatively mature, for the application of livestock and poultry breeding, due to reasons such as the large area of the breeding site and the high density of livestock and poultry, some current livestock and poultry counting algorithms and devices do not consider the problem of the too large livestock and poultry house scene. In the scenario where the livestock and poultry house scene is too large, although tilting the camera to take pictures can cover the whole area, there is a large amount of occlusion, resulting in a poor recognition effect. Some proposed intelligent methods, such as image recognition, video counting, aisle counting, etc., will have low recognition rates or inconvenient operations due to image distortion, livestock and poultry occlusion, or the need to drive livestock and poultry to specific positions, and do not have wide applicability.

[0025] For livestock and poultry counting, especially for free-range livestock and poultry, the breeding site area is large. Even using a wide-angle or fish-eye lens, it is difficult to cover the whole area, and the images will also have large distortions, affecting the counting accuracy. For example, a single pigsty is usually about 8*4 meters, and a free-range chicken house is usually 6*20 meters. There is no single camera that can cover the whole livestock and poultry house and clearly capture the livestock and poultry without occlusion. As a result, the target detection algorithm cannot be well utilized in the livestock and poultry counting scenario.

[0026] A target counting system, such as Figure 1 shown, includes a camera group 10, a moving device 20, and a computer device 30. Among them, the camera group 10 includes multiple cameras with their lenses facing downwards and arranged at intervals in a first direction at the top of the monitoring area, such as Figure 1 shown, the camera group 10 includes 4 cameras of the spacing device. The moving device 20 is connected to the camera group 10 and is used to drive each camera of the camera group to move simultaneously in a second direction at the top of the monitoring area. Among them, the first direction and the second direction are different.

[0027] In one embodiment, the moving device 20 includes a crossbar and a moving component. Each camera of the camera group 10 is fixed on the crossbar. When the moving component moves, it drives each camera on the crossbar to move in the second direction. The moving device 20 can be a rail machine or a conveyor belt, etc. In one embodiment, there can also be multiple moving devices, and each moving device 20 drives one camera to move in the second direction.

[0028] By reasonably arranging the spacing and installation height of the cameras according to the size of the monitoring area and the field of view angles of the cameras in the camera group 10, the field of view angles of the camera group 10 can cover the range in the first direction of the monitoring area, and the maximum moving distance of the camera group 10 enables the acquisition ranges of the cameras in terms of time to cover the range in the second direction of the monitoring area. Finally, the imaging range of the camera group 10 covers the entire monitoring area.

[0029] By arranging the camera group 10 and the moving device 20 at the top of the monitoring area, a relatively large monitoring area can be covered with a limited number of cameras.

[0030] Among them, the lenses of the multiple cameras face downwards and are perpendicular to the top of the monitoring area, so that the cameras shoot vertically downwards, greatly reducing the occurrence of distortion and target occlusion, and providing high-quality images of the monitoring area.

[0031] The multiple cameras of the camera group 10 collect video data during the movement process and send it to the computer device 30. The computer device 30 implements the target counting method of the present application to determine the number of targets in the monitoring area.

[0032] Among them, the computer device 30 can be the control device of the moving device 20, used to send a moving instruction to the moving device 20 to control the movement of the camera group 10. The computer device 30 can configure a target counting algorithm or call the target counting algorithm of the server in the cloud to determine the number of targets in the monitoring area.

[0033] Such as Figure 2 shown, a target counting method includes:

[0034] Step 202, control the movement of the camera group; the camera group includes multiple cameras with downward-facing lenses arranged at intervals in a first direction at the top of the monitoring area. Each camera is configured to be movable simultaneously in a second direction at the top of the monitoring area. The first direction and the second direction are different. The field of view angle of the camera group covers the range in the first direction of the monitoring area, and the maximum movement distance of the camera group enables the acquisition range of each camera in the time dimension to cover the range in the second direction of the monitoring area.

[0035] The camera uses a camera group 10 as shown in Figure 1 to achieve covering a relatively large monitoring area with a limited number of cameras. The camera group 10 includes multiple cameras with downward-facing lenses arranged at intervals in a first direction at the top of the monitoring area. The combined field of view angles of the multiple cameras in the first direction cover the range in the first direction of the monitoring area, such that the imaging range of the camera group 10 in a single frame dimension covers the range in the first direction of the monitoring area. Each camera of the camera group 10 is configured to be movable simultaneously in a second direction at the top of the monitoring area. The maximum movement distance of the camera group 10 enables the acquisition range of each camera in the time dimension to cover the range in the second direction of the monitoring area, ultimately enabling the imaging range of the camera group to cover the entire monitoring area. Among them, the first direction and the second direction are different and can be set according to the planar shape of the monitoring area.

[0036] Preferably, the first direction can be parallel to one boundary of the plane of the monitoring area, and the second direction is perpendicular to the first direction. In this way, the maximum imaging range of the camera group in the monitoring area can be achieved.

[0037] Taking the monitoring area as a livestock and poultry breeding site as an example, in one embodiment, the planar shape of the livestock and poultry breeding site is a matrix, then the camera group can be arranged in the longitudinal direction at its top. Arranging multiple cameras with downward-facing lenses at intervals in the longitudinal direction at the top of the livestock and poultry breeding site can enable the imaging range of the camera group to cover the longitudinal range of the livestock and poultry breeding site in the longitudinal direction. The second direction is the transverse direction at the top. The camera group is controlled to move in the transverse direction at the top, such that the acquisition range of each camera of the camera group in the time dimension covers the transverse range of the livestock and poultry breeding site. Thus, the imaging range of the camera group can cover the entire plane of the monitoring area.

[0038] In one embodiment, the planar shape of the livestock and poultry breeding site is a triangle, then the camera group can be arranged parallel to the boundary of the longest side of the livestock and poultry breeding site, and the imaging range of the camera group covers the boundary of the longest side of the livestock and poultry breeding site in this direction. The second direction is perpendicular to the first direction. The camera group is controlled to move in the second direction at the top, such that the acquisition range of the cameras of the camera group in the time dimension covers the range in the second direction of the livestock and poultry breeding site. Thus, the imaging range of the camera group can cover the entire plane of the monitoring area.

[0039] In one embodiment, the camera group may adopt a fish-eye lens with a field of view angle of 180 degrees. Through calculation, when the lens is 2 meters above the ground, there is a good recognition accuracy within a range of 5 meters for each camera. In order to ensure low distortion and an overlapping area between images, the distance between the two lenses is set to 3 meters. Therefore, the combination of the two lenses can cover a range of 8 meters wide. When the number of fish-eye lenses is 2, it can meet the needs of counting and recognition in most livestock and poultry breeding sites, such as the requirements of fattening pig pens of 8 * 4 ㎡ and flat-layer chicken coops of 6 * 20 ㎡. The camera group 10 moves forward driven by the motion device 20, so as to achieve full coverage of the livestock and poultry breeding site.

[0040] Step 204: Obtain the video data collected by each camera of the camera group during the movement and splice them to obtain a panoramic video covering the monitoring area.

[0041] Specifically, the acquisition range of each camera of the camera group in the time dimension covers the range in the second direction of the monitoring area. The field of view angle of the camera group covers the range in the first direction of the monitoring area. By splicing the videos of each camera, a panoramic video covering the monitoring area can be obtained.

[0042] In one embodiment, to obtain a high-quality panoramic video, the video data of each camera can be frame-synchronized, and after distortion correction, splicing processing is performed.

[0043] Step 206: Identify and track the targets in the panoramic video to determine the number of targets in the monitoring area.

[0044] In one embodiment, a target recognition and tracking model can be pre-trained, and by identifying and tracking the targets in the panoramic video, the number of targets in the monitoring area can be determined.

[0045] The target counting method of the present application utilizes a camera group composed of multiple cameras. The camera group is arranged in the first direction at the top of the monitoring area, and the multiple cameras are spaced so that the camera range of the camera group in the single-frame dimension covers the range of the first direction of the monitoring area. Each camera of the camera group is configured to be movable simultaneously in the second direction of the monitoring area, and its maximum moving distance enables the acquisition range of each camera in the time dimension to cover the range of the second direction of the monitoring area. In this way, by splicing the video data of each camera of the camera group, a panoramic video covering the monitoring area can be obtained. Then, target recognition and tracking processing are performed on the panoramic video to determine the number of targets in the monitoring area. By optimizing the layout of the camera group, this method can cover a large monitoring area with a limited number of cameras, and then use the panoramic video to identify and count the targets in the monitoring area. In this method, the cameras face down to shoot the monitoring area, which can reduce the probability of distortion and occlusion and improve the accuracy of counting targets based on the video. This method can be applied to the livestock counting scenario in a large livestock and poultry breeding site with a large site area.

[0046] To improve the quality of video acquisition and reduce the probability of distortion and occlusion of targets, in one embodiment, an effective recognition area in the video frame is first set according to the distortion range and / or the center of the field of view angle. Then, when controlling the movement of the camera, as much as possible, the starting boundary and the ending boundary of the monitoring area in the first direction are made to be located in the effective recognition area of the camera, so that any area of the monitoring area can appear in the effective recognition area of a certain frame of the panoramic video.

[0047] Generally speaking, the effective recognition area is an area with less distortion in the video frame or the center area of the field of view angle of the camera group 10.

[0048] In this embodiment, when the starting boundary of the monitoring area in the first direction is located in the effective recognition area of the camera, the camera is controlled to move in the second direction until the ending boundary of the monitoring area in the first direction is located in the effective recognition area, so that the entire range of the monitoring area is located in the effective recognition area. Specifically, as Figure 3 shown, according to the moving direction of the camera group 10, the monitoring area 300 has a starting boundary 301 in the first direction and an ending boundary 302 in the first direction. In the acquisition range 320 of the field of view angle of the camera group 10, there is an effective recognition area 321 related to the distortion range and / or the center of the field of view angle. The image distortion in the effective recognition area 321 is small and close to the vertical direction of the camera, so the probability of distortion and occlusion of targets in the effective recognition area is reduced.

[0049] To ensure that the entire monitoring area is within the effective recognition area, that is, the effective recognition area fully covers the monitoring area in the vertical range with small distortion, the camera group 10 is controlled to move starting from the moving starting point. Among them, at the moving starting point, the starting boundary 301 of the monitoring area 300 in the first direction coincides with or does not exceed the first boundary 31 of the effective recognition area 321. The camera is controlled to move in the second direction, and when the first boundary 31 of the effective recognition area 321 coincides with the termination boundary 302 of the monitoring area 300 in the first direction, or exceeds the termination boundary 302 of the monitoring area in the first direction by a preset distance, the camera is controlled to stop moving. In this way, the entire range of the monitoring area can be located within the effective recognition area.

[0050] In this embodiment, when the starting boundary of the monitoring area in the first direction is within the effective recognition area of the camera, the camera is controlled to move in the second direction until the end boundary of the monitoring area in the first direction is within the effective recognition area of the camera. In this way, it can ensure that the effective recognition area completely covers the monitoring area in the panoramic video, and an image with small distortion and low occlusion probability corresponding to the monitoring area can be obtained, thereby improving the accuracy of target recognition in the monitoring area.

[0051] In one embodiment, considering the moving direction and the center of the field of view angle of the camera, the center of the field of view angle can be deviated from the center of the field of view angle of the camera group by a preset range in the moving direction of the monitoring area towards the end boundary, and the effective recognition area can be determined.

[0052] As Figure 4 shown, the coordinate range of the area corresponding to the center of the field of view angle of the camera group is deviated by a preset distance in the second direction (as Figure 4 shown, deviated to the right), and the effective recognition area is obtained. The size and shape of the effective recognition area remain unchanged relative to the center of the field of view angle of the camera group. Only the position of the coordinate range of the effective recognition area is offset by a preset distance in the second direction relative to the coordinate range of the area corresponding to the center of the field of view angle of the camera group, and the offset direction is the same as the moving direction of the camera. Therefore, the moving distance of the camera group can be reduced, and the efficiency of video acquisition can be improved.

[0053] In one embodiment, the targets in the panoramic video are identified and tracked, and the number of targets in the monitoring area is determined, including: identifying the targets in the panoramic video and tracking the targets; determining the number of targets in the monitoring area according to the number of non-repeating targets in the effective recognition area of the video frames of the panoramic video.

[0054] As mentioned above, the effective recognition area of the camera has small distortion and is perpendicular or nearly perpendicular to the shooting direction of the camera. Therefore, the possibility of distortion and occlusion is low, and the image quality is high.

[0055] By identifying the target in the panoramic video and tracking the target. Determine the target data according to the data of non-repeating targets in the valid recognition area of the video frame. For example, in the second video frame, targets 1, 2, and 3 are detected in the valid recognition area. As the camera moves in the second direction, the monitoring area corresponding to the valid recognition area in the fifth video frame also changes, detecting target 3, and targets 4 and 5 are detected for the first time. Then the currently detected number of targets is 5.

[0056] In this embodiment, by using the target recognition and tracking of the panoramic video, and determining the number of targets in the monitoring area according to the data of non-repeating targets in the valid recognition area with higher image quality, the accuracy of target detection can be improved.

[0057] In one embodiment, determining the number of targets in the monitoring area according to the number of non-repeating targets in the valid recognition area of the video frame of the panoramic video includes:

[0058] Determine the identification and center coordinates of the targets in the valid recognition area of the video frame;

[0059] Track the center coordinates of the targets with identification in the image coordinates;

[0060] When the center coordinates of the targets with identification exceed the counting line and have not been counted, accumulate the number of the corresponding targets; the counting line coincides with at least the left boundary of the valid recognition area or deviates from the left boundary to the left by a preset distance.

[0061] In this embodiment, considering the moving direction of the camera group and the mobility of the targets, in order to achieve accurate counting of the targets, by using the left boundary of the valid recognition area as the counting line, or setting the counting line to deviate from the left boundary of the valid recognition area to the left by a preset distance to track the identified targets. After the targets are identified, they will be given an identification, such as a number. Since the distortion of the valid recognition area is small and the occlusion is less, the accuracy of target recognition in the valid recognition area is high, and the possibility of identification change of the targets in the valid recognition area can be reduced. When the targets move with the camera group and exceed the counting line (changing from the coordinates on the right side of the counting line to the coordinates on the left side of the counting line), and the targets have not been counted, accumulate the number of the corresponding targets until every frame of the panoramic video is traversed to obtain the number of targets in the monitoring area. By using the counting line and the valid recognition area, accurate counting of the targets can be achieved.

[0062] As Figure 4 shown, set the counting line to deviate from the left boundary of the valid recognition area to the left by a preset distance. When controlling the camera group to move in the second direction, the counting line can be made to coincide with the starting boundary in the first direction of the monitoring area to improve the accuracy of counting.

[0063] As Figure 5As shown, in the current frame, targets 1, 2, and 3 are recognized in the effective recognition area 320. The coordinates of all three targets do not exceed the counting line 501, and no counting is performed at this time. As the camera group moves in the second direction, it is tracked that in Figure 6 the current frame shown, the center coordinates of targets 1 and 2 exceed the counting line 501 and have not been counted yet, so the current counted target number is 2. And in Figure 5 the current frame shown, new targets 3 and 4 are detected. As the camera group moves in the second direction, it is tracked that in Figure 7 the current frame shown, the center coordinates of targets 3 and 1 exceed the counting line, but target 1 has been counted (because target 1 is redetected as it moves and exceeds the counting line), so Figure 6 for the current frame of , the statistical result of new target 3 is added, and the current counted target number is 3.

[0064] In this way, even if the targets in the monitoring area move, the non-repeated targets in the effective recognition area can still be accurately counted.

[0065] In one embodiment, considering that the position of the effective recognition area is unchanged in each video frame, but as the camera group moves in the second direction, the actual monitoring area corresponding to the effective recognition area of each video frame changes. Therefore, the number of targets in the monitoring area can also be determined by counting the non-repeated targets in different actual monitoring areas corresponding to the effective recognition area.

[0066] Specifically, to identify and track the targets in the panoramic video and determine the number of targets in the monitoring area, it includes: determining the identifiers and center coordinates of the targets in the effective recognition area of the video frame; converting the coordinates of the effective recognition area and the center coordinates of the targets into the world coordinate system to determine the current actual area corresponding to the effective recognition area in the world coordinate system; adding the number of new targets in the current actual area to the historical statistical result.

[0067] In one embodiment, it is necessary to splice the video data of each camera of the camera group to obtain a panoramic video, and then perform target detection and tracking based on the panoramic video to achieve target counting. Therefore, the quality of video splicing affects the counting accuracy. Due to the time difference between multiple camera videos and the installation process cannot ensure that the cameras are completely horizontal, frame synchronization and distortion correction processing are performed before the splicing calculation in this embodiment to improve the video splicing quality. Among them, frame synchronization is used to solve the video time difference of different cameras and avoid the problems of artifacts or disappearance caused by the running of livestock and poultry. Distortion correction can solve the problem of high distortion of cameras with large field of view angles, which is more in line with the human eye's perception and will also reduce distortion and improve the algorithm recognition rate. Splicing calculation is used to splice the images of multiple cameras into a complete image and calculate the offset, tilt angle, scaling ratio, etc. between the images.

[0068] In one embodiment, obtaining the video data collected by each camera of the camera group during movement and stitching them to obtain a panoramic video covering the monitoring area includes:

[0069] Obtaining the video data collected by each camera of the camera group during movement;

[0070] Performing frame synchronization and distortion correction processing on the video data of each camera;

[0071] Calculating the homography transformation matrix between the video frames of each camera, and converting the video frames of each camera to the same spatial coordinates according to the homography transformation matrix;

[0072] Respectively intercepting and stitching the video frames of adjacent cameras to obtain a panoramic video of the monitoring area.

[0073] Specifically, frame synchronization is used to solve the video time difference of different cameras and avoid artifacts or disappearance problems caused by the running of livestock and poultry. In this embodiment, the video is synchronized more precisely by using the timestamp synchronization method. The specific steps are as follows:

[0074] (1) The cameras perform time calibration from the time calibration server so that the times of multiple cameras are consistent.

[0075] (2) Each camera performs video recording and obtains the timestamp corresponding to each frame when taking pictures. Select one video as video A, and align other videos to video A.

[0076] (3) The steps for aligning other videos to video A are:

[0077] ① Let each frame image of video A be The timestamp is Each frame of video B is The timestamp is where i represents the index number of the frame.

[0078] ② Set the search range of the frame to 5, that is, the frame of video B needs to be matched with the to 11 frames of video A for the closest time match, and at the same time return the matched image of A, so as to achieve frame synchronization.

[0079] Since the fish-eye has a large distortion, distortion correction is required. In this application, the longitude and latitude distortion correction method is used. The effect after distortion correction is as Figure 8 shown. Compared with the checkerboard calibration method, the livestock and poultry farther away are less deformed.

[0080] Image stitching errors can affect the counting accuracy. After frame synchronization, there are overlapping regions in the images, as well as camera tilts and non - horizontal issues between cameras caused by installation and design processes. The main implementation steps of a stitching method are as follows:

[0081] 1) Randomly select multiple frames of synchronized images, and use the OmniGlue algorithm based on the large - model DINOv2 to perform feature - point matching calculations on the synchronized images. The calculation will obtain a homography matrix H, which contains information such as the position offset, tilt angle, and scaling ratio between image A and image B.

[0082] 2) Use the homography matrix H to stitch image A and image B. The stitching steps are as follows:

[0083] ① Perform a perspective transformation on B to place B in the same spatial coordinate system as A, obtaining

[0084]

[0085] Select the left - hand half of image A, Select the right - hand half of the image to form a new image. The stitching algorithm is as follows:

[0086] A[width / / 2:] = 0

[0087]

[0088] In this embodiment, by performing frame synchronization and distortion processing on the videos collected by each camera during movement, the video quality can be improved, and artifacts and image distortion can be reduced. Furthermore, the processed videos are stitched to obtain a high - quality panoramic video of the monitoring area.

[0089] In one embodiment, the steps of identifying targets in the panoramic video and tracking the targets can be processed using an AI model.

[0090] Specifically, first prepare a training set and train to obtain a target recognition and tracking model.

[0091] Among them, the target recognition and tracking model can adopt the Yolov11 + ByteTrack framework.

[0092] Extract frames from the panoramic video of the monitoring area, and use labelImg to label the livestock and poultry targets in it.

[0093] For the stitched panoramic video, perform frame - rate reduction processing, and extract 5 frames per second for recognition and tracking. This frame rate can reduce the pressure on the server and will not cause the tracked targets to be lost.

[0094] Input the panoramic video into the Yolov11 model for training to identify the positions of targets in the video frames. After training, splice the ByteTrack model. The result recognized by Yolov11 is the position of each livestock and poultry, and then it is passed to ByteTrack to track the livestock and poultry, and assign an ID to each livestock and poultry to avoid duplicate counting.

[0095] In one embodiment, taking the livestock and poultry breeding site as a pigsty and the target as a pig for example, by installing the following in the pigsty Figure 1 The target counting system includes a camera group 10, a moving device 20 and a computer device 30.

[0096] Specifically, the camera group 10 includes a plurality of cameras with the lenses facing downwards and perpendicular to the top in the longitudinal direction at the top of the pigsty, and is configured to be able to move simultaneously in the lateral direction of the top of the pigsty by using the moving device. The field of view angle of the camera group covers the range in the longitudinal direction of the pigsty, and the maximum moving distance of the camera group enables the acquisition range of each camera in the time dimension to cover the range in the lateral direction of the pigsty. By adopting the layout of the camera group 10, a pigsty image with lower distortion can be obtained, and by driving the movement of the camera group 20 by the moving device 10, full coverage of the pigsty can be achieved. Subsequently, through the target counting method of the present application, high-quality spliced video and recognition accuracy can be achieved.

[0097] The computer device communicates with the server in the cloud. The cloud server deploys a trained target recognition and tracking model, and this model can adopt the Yolov11+ByteTrack framework. By adopting cloud deployment, a large amount of livestock and poultry data can be collected and accumulated, and the algorithm can be continuously optimized to continuously improve the recognition effect. The computer device can call the target recognition and tracking model in the cloud to implement the target calculation method.

[0098] The method includes:

[0099] Step 1: Control the camera group to move in the lateral direction of the pigsty.

[0100] Step 2: Obtain the video data collected by each camera of the camera group during the movement and splice it to obtain a panoramic video covering the monitoring area.

[0101] The quality of video stitching affects the counting accuracy. Due to the time difference between multiple camera videos and the installation process not being able to ensure that the cameras are completely horizontal, after frame synchronization and distortion correction in the present invention, the video images of each camera are stitched to improve the video stitching quality. Among them, frame synchronization is used to solve the time difference of videos from different cameras and avoid artifacts or disappearance caused by the running of livestock and poultry. Distortion correction can solve the problem of high distortion of cameras with large field of view angles, making it more in line with the human eye's perception and also reducing distortion to improve the algorithm recognition rate. Stitching calculation is used to stitch the images of multiple cameras into a complete image and calculate the offset, tilt angle, scaling ratio, etc. between the images.

[0102] Step 3: Identify and track the targets in the panoramic video and determine the number of targets in the monitoring area.

[0103] Among them, when the pigs are blocked, it may cause incorrect model recognition, and the same pig is repeatedly assigned an identifier, resulting in inaccurate counting.

[0104] In this embodiment, an effective recognition area is set. According to the center of the field of view angle of the camera group and the second direction of movement of the camera group, the area where the center of the field of view angle in the video frame deviates from the second direction by a preset range is determined as the effective recognition area of the video frame.

[0105] As Figure 4 shown, the effective recognition area is located at the position to the right of the center of the video frame image.

[0106] Identify the targets in the panoramic video, track the targets, and determine the identifiers and center coordinates of the targets in the effective recognition area of the video frame; track the center coordinates of the targets with identifiers in the image coordinates; when the center coordinates of the targets with identifiers exceed the counting line and have not been counted, accumulate the quantity of the corresponding targets; the counting line coincides with at least the left boundary of the effective recognition area or deviates leftward from the left boundary by a preset distance.

[0107] In this embodiment, the counting line and the effective recognition area are set so that the counting only focuses on the area with high accuracy, improving the accuracy of target recognition and thus the accuracy of statistics.

[0108] On the other hand, an embodiment of the present application further provides a computer device, which includes a processor and a memory connected to the processor. A computer program executable by the processor is stored on the memory. The computer program is implemented by the processor to perform each process of the above-mentioned target counting method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0109] In another aspect of the embodiments of the present application, a computer-readable storage medium is further provided. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, it implements each process of the above-mentioned target counting method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here. Among them, the computer-readable storage medium includes, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0110] In another aspect of the embodiments of the present application, a computer program product is further provided, including a computer program. When the computer program is executed by a processor, it implements each process of the target counting method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0111] It should be noted that in this article, the term "including", "comprising", or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including that element.

[0112] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0113] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A target counting method, characterized in that: The method comprises: Control the movement of a camera group; the camera group includes a plurality of cameras with downward lenses arranged at intervals in a first direction at the top of the monitoring area, each of the cameras being configured to simultaneously move toward a second direction at the top of the monitoring area, the first direction and the second direction being different, the field of view of the camera group covering the range of the first direction of the monitoring area, and the maximum moving distance of the camera group making the acquisition range of each of the cameras in the time dimension cover the range of the second direction of the monitoring area; Acquire and stitch video data collected by each camera of the camera group during movement to obtain a panoramic video covering the monitoring area; Identify and track targets in the panoramic video, and determine the number of targets in the monitoring area.

2. The target counting method according to claim 1, characterized in that: The controlling the movement of the camera group comprises: When the starting boundary of the monitoring area in the first direction is located in the effective recognition area of ​​the camera, the camera is controlled to move in the second direction until the ending boundary of the monitoring area in the first direction is located in the effective recognition area of ​​the camera; the effective recognition area is related to the distortion range and / or the center of the field of view angle.

3. The target counting method according to claim 2, characterized in that: The method further comprises: The effective recognition area is obtained by deviating the coordinate range of the area corresponding to the center of the field of view angle toward the second direction by a preset distance.

4. The target counting method according to claim 2 or 3, characterized in that: The identifying and tracking the target in the panoramic video and determining the number of targets in the monitoring area includes: Identifying a target in the panoramic video and tracking the target; The number of targets in the monitoring area is determined according to the number of non-repetitive targets in the effective identification area in the video frame of the panoramic video.

5. The target counting method according to claim 4, characterized in that: The determining the number of targets in the monitoring area according to the number of non-repetitive targets in the effective identification area in the video frame of the panoramic video includes: Determining the identification and center coordinates of the target in the effective recognition area in the video frame; Tracking the center coordinates of the target with the mark in the image coordinates; When the center coordinates of the target with the mark exceed the counting line and have not been counted, the number of corresponding targets is accumulated; the counting line at least coincides with the left boundary of the effective identification area or deviates from the left boundary to the left by a preset distance.

6. The target counting method according to claim 2 or 3, characterized in that: The identifying and tracking the target in the panoramic video and determining the number of targets in the monitoring area includes: Determine the identification of the target in the effective identification area in the video frame and the center coordinates of the target; Convert the coordinates of the effective identification area and the center coordinates of the target into a world coordinate system, and determine a current actual area corresponding to the effective identification area in the world coordinate system; The number of new targets in the current actual area is added to the historical statistical results.

7. The target counting method according to claim 1, characterized in that: The step of acquiring and stitching video data collected by each camera of the camera group during movement to obtain a panoramic video covering the monitoring area includes: Acquire video data collected by each camera of the camera group during movement; Performing frame synchronization and distortion correction processing on the video data of each camera; Calculating the homography change matrix between the video frames of each camera, and transforming the video frames of each camera to the same spatial coordinates according to the homography transformation matrix; The video frames of adjacent cameras are respectively intercepted and spliced ​​to obtain a panoramic video of the monitoring area.

8. A target counting system, characterized in that: The invention comprises a camera group, a motion device and a computer device; the camera group comprises a plurality of cameras with lenses facing downwards arranged at intervals in a first direction at the top of a monitoring area, and is configured to be simultaneously movable toward a second direction at the top of the monitoring area, the first direction and the second direction are different, the field of view of the camera group covers the range of the first direction of the monitoring area, and the maximum moving distance of the camera group enables the acquisition range of each camera in the time dimension to cover the range of the second direction of the monitoring area; The computer device includes a processor and a memory connected to the processor, the memory stores a computer program executable by the processor, and the computer program, when executed by the processor, implements the steps of the target counting method as described in any one of claims 1 to 7.

9. The target counting system according to claim 8, characterized in that: The lenses of the multiple cameras face downward and are perpendicular to the top of the monitoring area.

10. A computer device, characterized in that: It comprises a processor and a memory connected to the processor, wherein the memory stores a computer program executable by the processor, and when the computer program is executed by the processor, the steps of the target counting method as described in any one of claims 1 to 7 are implemented.

Images