Method and apparatus for identifying individual dead farmed animals in a video based on vit technology

Abstract

The application provides a method and device for identifying dead farmed animal individuals in a video based on a ViT technology, a storage medium and a terminal, and belongs to the technical field of computers. The method is applied to a terminal. The terminal selects key frames from a short video segment containing dead farmed animals, and synthesizes pseudo-color three-channel images from the key frames. The pseudo-color three-channel images are data-labeled to label dead farmed animal individuals and non-dead farmed animal individuals in the images, and a training sample set is obtained. The training sample set is input into a ViT target detection model to train the model. The scheme converts target detection in a video into target detection in an image, thereby reducing the time cost of labeling work. Whether the farmed animals are alive or not is inferred by distinguishing the motion and stillness of the animals in the short video segment. When there are tens of thousands or even millions of animals in a large-scale farm, the automatic algorithm can replace the frequent patrol cost of manual work.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a method, apparatus, storage medium and terminal for identifying dead farmed animals in videos based on ViT technology. Background Technology

[0002] Object detection is a classic task in computer vision. In recent years, deep neural networks have rapidly gained popularity in image object detection, achieving excellent results compared to traditional methods. Following their success in the image field, deep learning-based video object detection algorithms have also developed rapidly, mainly falling into two categories: deep learning video object detection algorithms based on detection and tracking, and deep learning object detection algorithms based on dynamic information such as optical flow.

[0003] With the widespread application of high-definition cameras in livestock farming, animal behavior recognition based on video analysis can use the aforementioned video target detection algorithms to mine typical behaviors of animals such as lying down, standing, walking, running, and jumping. This makes it possible to automatically identify key healthy physiological behaviors and abnormal pathological behaviors during the farming process, providing timely warnings of animal health status and abnormal behaviors, and offering technical support for large-scale precision farming.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, storage medium, and terminal for identifying dead farmed animals in videos based on ViT technology. In large-scale farms with tens of thousands or even millions of animals, this automated algorithm can replace the cost of frequent manual patrols. The technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a method for identifying dead farmed animals in videos based on ViT technology. The method includes: recording farmed animals using a camera and accumulating M short video segments; selecting all keyframes from each short video segment and synthesizing them into three images, which are used as RGB three channels to obtain one pseudo-color image; and marking the positions of static and dynamic farmed animals in the image with bounding boxes to obtain a training sample set. The training sample set is then input into a ViT object detection model for training. Next, recording farmed animals using a camera and accumulating P short video segments of dead animals to be identified; selecting all keyframes from each short video segment and synthesizing them into three images, which are used as RGB three channels to obtain one pseudo-color image; and inputting this image into the trained ViT object detection model, which calculates the positions of static (dead) and dynamic (surviving) farmed animals in the video segments.

[0007] The above process involves extracting all 3*N keyframes from any short video clip and synthesizing them into a pseudo-color three-channel image, where N is a positive integer. For the 1st to Nth keyframes, the average value is taken to synthesize a grayscale image, which serves as the red band in the pseudo-color three-channel image. Similarly, for the (N+1)th to 2*Nth keyframes, the average value is taken to synthesize a grayscale image, which serves as the green band in the pseudo-color three-channel image. The average value is taken for the 2*N+1th to 3*Nth keyframes, which serves as the blue band in the pseudo-color three-channel image. This process is repeated to obtain M pseudo-color three-channel images from all M short video clips.

[0008] The above method involves annotating M pseudo-color three-channel images synthesized from M short video clips, using bounding boxes to enclose each individual farmed animal in the image. The bounding boxes are then assigned category attributes to distinguish between static and dynamic farmed animals. This yields the training sample set.

[0009] The above process involves inputting the training sample set into the ViT object detection model to train the model.

[0010] The above method involves extracting all 3*N keyframes from any short video clip of a dead animal to be identified, and synthesizing them into a pseudo-color three-channel image, where N is a positive integer. For keyframes 1 to N, the average value is taken to synthesize a grayscale image, which serves as the red band in the pseudo-color three-channel image. For keyframes N+1 to 2*N, the average value is taken to synthesize a grayscale image, which serves as the green band in the pseudo-color three-channel image. For keyframes 2*N+1 to 3*N, the average value is taken to synthesize a grayscale image, which serves as the blue band in the pseudo-color three-channel image. This process is repeated to obtain P pseudo-color three-channel images from all P short video clips.

[0011] The above-mentioned P images corresponding to the P short video segments of the dead animals to be identified are input into the trained ViT object detection model. The model calculates the positions of static farmed animals (dead animals) and dynamic farmed animals (surviving animals) in the video segments, and encloses each farmed animal in the image with a rectangle of different category attributes.

[0012] Secondly, embodiments of this application provide a device for identifying deceased farmed animals based on deep learning video analysis, the device comprising:

[0013] The first pseudo-color image determination module is used to: synthesize M pseudo-color three-channel images from M short video clips used for training the model, where M is a positive integer; the annotation module is used to: annotate the M pseudo-color three-channel images corresponding to the M short video clips, using rectangles to enclose each individual farmed animal in the image, and assigning category attributes to the rectangles to label static and dynamic farmed animal individuals, thus obtaining a training sample set; the training module is used to: input the training sample set into the ViT object detection model to train the model; the second pseudo-color image determination module is used to: synthesize P pseudo-color three-channel images from P short video clips of the dead animals to be identified, where P is a positive integer; the object recognition module is used to: input the P pseudo-color three-channel images corresponding to the P short video clips of the dead animals to be identified into the trained ViT object detection model, thereby obtaining the static and dynamic farmed animal individuals corresponding to the video clips, i.e., the dead and non-dead farmed animal individuals.

[0014] Thirdly, embodiments of this application provide a terminal, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for identifying dead farmed animals in a video based on ViT technology as described in the first aspect.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the method for identifying dead farmed animals in a video based on ViT technology as described in the first aspect.

[0016] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following:

[0017] In the implementation of this application's embodiment, the terminal synthesizes several pseudo-color three-channel images from short video clips used for training and to identify individual dead farmed animals. The pseudo-color three-channel images synthesized from the video clips are then input into the trained target detection model to identify the individual dead farmed animals. This solution employs a novel video analysis and recognition strategy, transforming the video target detection problem into an image target detection problem. This avoids the difficulties in data acquisition, the lack of sufficient labeled video samples, and the high computational cost of video target detection, thus reducing the time cost of labeling. By distinguishing the movement of animals in short video clips, it infers whether the farmed animals are alive. In large-scale farms with tens of thousands or even millions of animals, this automated algorithm can replace the cost of frequent manual patrols.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 A flowchart illustrating an exemplary embodiment of this application shows an algorithm training method for identifying dead farmed animals in videos based on ViT technology;

[0021] Figure 2 A flowchart of a pseudo-color three-channel image determination method provided by an exemplary embodiment of this application is shown;

[0022] Figure 3 A schematic diagram of a pseudo-color three-channel image provided by an exemplary embodiment of this application is shown;

[0023] Figures 4a to 4d A schematic diagram of pseudo-color three-channel image data annotation provided in an exemplary embodiment of this application is shown;

[0024] Figure 5 A structural diagram of the ViT target detection model (Deformable DETR) provided in an exemplary embodiment of this application is shown;

[0025] Figure 6 A schematic diagram of the ViT Target Detection Model (DETR) provided in an exemplary embodiment of this application is shown;

[0026] Figure 7 The illustration shows a schematic diagram of deformable convolution provided in an embodiment of this application;

[0027] Figure 8 The flowchart illustrates a target detection method for identifying dead farmed animals in a video based on ViT technology, according to an embodiment of this application.

[0028] Figure 9 This illustration shows a schematic diagram of the results of identifying individual dead farmed animals based on video analysis using ViT technology, according to an embodiment of this application.

[0029] Figure 10 This schematic diagram illustrates the structure of a device for detecting dead farmed animals based on video analysis using ViT technology, according to an embodiment of this application.

[0030] Figure 11 The diagram illustrates a block diagram of a terminal provided in one embodiment of this application. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0032] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0033] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0034] Video analytics-based animal behavior recognition focuses on typical daily behaviors of animals. However, animal mortality is relatively rare, making it extremely difficult to collect video data samples containing images of deceased animals. Furthermore, in livestock farming settings, deceased animals are easily obscured by other animals, hindering the collection of video data samples containing such images. Additionally, labeling video data samples is very labor-intensive. Therefore, obtaining the necessary video data samples and labeled video samples for video analytics-based animal behavior recognition is inefficient and scarce. Moreover, video object detection algorithms in related technologies require significant computation and have poor real-time performance, making them difficult to apply in real-world scenarios, such as integrating them into livestock farming management systems.

[0035] To address the aforementioned problems, this application provides a method, apparatus, storage medium, and terminal for identifying dead farmed animals in videos based on ViT technology. This transforms target detection in videos into target detection in images, reducing the time cost of annotation. By distinguishing the movement of animals in short video clips, it infers whether the farmed animals are alive. In large-scale farms with tens of thousands or even millions of animals, this automated algorithm can replace the cost of frequent manual patrols. The following will describe the steps of this exemplary implementation scheme in more detail with reference to the accompanying drawings and embodiments.

[0036] in, Figure 1 The flowchart illustrates an algorithm training method for identifying dead farmed animals in a video based on ViT technology, according to an exemplary embodiment of this application.

[0037] The following example illustrates the identification of dead and live animals in a farm. Figure 1 The illustrated embodiments are described in detail below:

[0038] S101, synthesize M pseudo-color three-channel images from the M short video clips used to train the model, where M is a positive integer.

[0039] In an exemplary embodiment, to better collect data samples, PTZ cameras can be deployed according to the farming environment in the farm, or mobile cameras mounted on inspection robots can be used to simultaneously cover multiple different farming units from different perspectives. This allows for the efficient collection of as diverse data samples as possible, saving hardware and labor costs. After video data is collected through video acquisition devices such as PTZ cameras, the video data will include video clips of a certain farmed animal. For example, taking chickens, the next step is to identify dead and surviving individuals among the chickens in the video data.

[0040] In an exemplary embodiment, all keyframes of any short video clip are extracted and synthesized into a pseudo-color three-channel image. For example, each short video clip is synthesized into an RGB image, i.e., an image with three channels: red (R), green (G), and blue (B). This transforms the processing of video data into the processing of image data.

[0041] In an exemplary embodiment, as a specific implementation of S101, Figure 2 A flowchart illustrating a pseudo-color three-channel image determination method in an exemplary embodiment of this application is shown schematically. (Reference) Figure 2 The embodiment shown in the figure includes S201-S202.

[0042] S201. For all M short video segments, obtain all 3*N keyframes in each short video segment, where M is a positive integer and N is a positive integer.

[0043] S202, for each short video segment with 3*N keyframes, the average value of keyframes 1 to N is used to synthesize a grayscale image, which serves as the red band in the pseudo-color three-channel image; the average value of keyframes N+1 to 2*N is used to synthesize a grayscale image, which serves as the green band in the pseudo-color three-channel image; the average value of keyframes 2*N+1 to 3*N is used to synthesize a grayscale image, which serves as the blue band in the pseudo-color three-channel image. This yields a pseudo-color three-channel image synthesized from 3*N keyframes. All M short video segments are synthesized to obtain M pseudo-color three-channel images.

[0044] In an exemplary embodiment, during the process of obtaining a single-channel image of a specific band (red, green, blue), for each N keyframes, the method for synthesizing the single-channel image can be, in addition to taking the average value, unsupervised learning methods such as Principal Component Analysis (PCA) can also be selected, without limitation.

[0045] For example, taking N=5, each short video segment includes 15 keyframes. The first 5 keyframes are averaged to create a single grayscale image, which serves as the red band in the pseudo-color three-channel image. The 6th to 10th keyframes are averaged to create another grayscale image, which serves as the green band in the pseudo-color three-channel image. The 11th to 15th keyframes are averaged to create yet another grayscale image, which serves as the blue band in the pseudo-color three-channel image. This results in a pseudo-color three-channel image synthesized from 15 keyframes. In the resulting pseudo-color three-channel image, the edges of moving objects will have colored shadows, while the edges of stationary objects will not, thus distinguishing between static and dynamic targets. Figure 3As shown, for example, in this pseudo-color three-channel image, the edges of the bodies of live chickens in a normal state will have colored light and shadow, indicating that they are in a moving state, while dead chickens will have no such light and shadow at all, thus distinguishing dead individuals from living individuals.

[0046] S102, perform data annotation on M pseudo-color three-channel images synthesized from M short video clips, enclose each individual farmed animal in the image with a rectangle, and assign category attributes to the rectangle to label static and dynamic farmed animal individuals, thus obtaining the training sample set.

[0047] In an exemplary embodiment, after distinguishing between statically raised animal individuals (dead) and dynamically raised animal individuals (surviving) using pseudo-color three-channel images, the next step is to annotate these pseudo-color three-channel images. For example, typical samples containing both dead and surviving individuals can be selected from M pseudo-color three-channel images, resulting in 2000 samples containing both dead and surviving individuals, and 3000 samples containing only surviving individuals. These 5000 samples are then annotated to identify the dead and surviving individuals, thereby obtaining a training sample set. The above sample number is merely an example and is not specifically limited in this embodiment.

[0048] In an exemplary embodiment, reference is made to Figures 4a to 4d The diagram shown illustrates the annotation of pseudo-color three-channel image data. The annotation method can be... Figures 4a to 4d The diagram shows the use of different colored selection boxes to label static and dynamic individuals as different categories. For example, taking chickens in a cage as an example, red rectangles are used to represent dead chickens, and green rectangles are used to represent live chickens. The data labeling method is not limited in this embodiment. Furthermore, the pseudo-color three-channel image can be masked beforehand, allowing focus to be placed only on the chickens inside the cage, without interference from individuals in other areas, thus facilitating data labeling.

[0049] In an exemplary embodiment, the above steps convert video data annotation into image data annotation. The image object detection method addresses the problems of difficult data acquisition, insufficient video annotation samples, and massive computational demands inherent in video object detection. Image data is more readily available in terms of quantity compared to video data, and image data annotation is simpler and more efficient than video data annotation. This significantly reduces the time cost of annotation work and the subsequent time cost of training the object detection model.

[0050] S103, input the training sample set into the object detection model to train the object detection model.

[0051] In exemplary embodiments, for object detection models, traditional RCNN (Regions with CNN features) and YOLO (You Only Look Once) object detection algorithms cannot achieve satisfactory recognition results. DETR based on Vision Transformer (ViT) technology shows some improvement in recognition performance, but it is still insufficient for application. Ultimately, Deformable DETR, also based on ViT technology, was selected, referencing... Figure 5 The diagram shown illustrates the structure of the ViT object detection model (DeformableDETR). The DeformableDETR model is developed based on the DETR model, borrowing the idea of ​​Deformable Convolution and applying it to the attention mechanism. Figure 5 As shown, the DeformableDETR model uses a multi-scale deformable self-attention module in the encoder to transform the feature map of the input image, obtaining a multi-scale feature map. Then, it combines a multi-scale deformable cross-attention module in the decoder and a transformer self-attention module in the decoder to process the object query sequence, finally obtaining the bounding box prediction results.

[0052] In an exemplary embodiment, reference is made to Figure 6 The diagram shows a ViT object detection model (DETR). Figure 6As shown, DETR uses a traditional Convolutional Neural Network (CNN) skeleton to embed the input image (set of image features) to obtain a corresponding two-dimensional representation. This two-dimensional representation is then converted into a one-dimensional representation, and combined with positional encoding, the output is passed to a Transformer Encoder to obtain a set of predicted object queries of length N. The Transformer Decoder performs parallel decoding on the obtained predicted object sequences to obtain output sequences. These output sequences are then passed to shared Feed-Forward Network (FFN) prediction heads, which can predict either a detection result (including class and bounding box) or a "no objection" class.

[0053] In an exemplary embodiment, reference is made to Figure 7 The diagram shows a deformable convolution. Figure 7 In the image, the left side represents the standard convolution operation, while the right side represents the deformable convolution operation. The receptive field of deformable convolution can adaptively adjust according to the proportion and shape of the object. Therefore, when the standard convolution operation on the left processes sheep in an image, it cannot adjust the receptive field according to the object's proportion and shape, and thus cannot effectively distinguish between different individuals. Figure 7 As shown, the receptive field of the sheep on the left is incomplete, and the receptive field of the sheep on the right is also incomplete and exceeds the range of the target, covering other targets; while the receptive field of the deformable convolution operation on the right almost perfectly covers the target sheep, which helps to improve the accuracy of target detection.

[0054] In an exemplary embodiment, compared to the global and dense attention mechanism of DETR, the Deformable DETR model focuses on only a set of sampling points in its neighborhood for each reference point. The positions of these sampling points are not fixed but learnable, thus achieving a local and sparse efficient attention mechanism. In animal husbandry scenarios, there are many instances of mutual occlusion among individual animals. Traditional object detection models require post-processing operations such as Non-Maximum Suppression (NMS) to output the final detection result, and the effectiveness of NMS is highly affected by mutual occlusion. The DETR model and the Deformable DETR model apply Vision Transformer (ViT) technology to the object detection model, successfully enabling the model to directly output detection results end-to-end, replacing the role of NMS in object detection. Compared to the DETR model, the Deformable DETR model incorporates a Deformable Convolution operator into its network structure. In the context of farmed animal identification, for poultry, the Deformable DETR model can effectively incorporate the deformation effect of bird feathers into the algorithm, thus achieving results that are comprehensively superior to the DETR model. The exploration from the DETR model to the Deformable DETR model has enhanced the expressive power of object detection models, resulting in more accurate detection results and more efficient model training.

[0055] In an exemplary embodiment, the above-mentioned training sample set is input into the object detection model to train the object detection model, thereby obtaining an object detection model for video object detection through image object detection.

[0056] This application addresses the problems existing in video object detection in related technologies by employing a novel video analysis and recognition strategy, namely, a training method for the object detection model. This method transforms video object detection into image object detection, bypassing direct computation on massive video data and circumventing a series of problems associated with video object detection. It processes consecutive single-frame images from the video data to fuse them into a pseudo-color image, and then uses a Deformable DETR object detection model based on Vision Transformer (ViT) technology to identify static targets (dead farmed animals) in the pseudo-color image. During the conversion of video data into image data, this application, after comparing several unsupervised learning algorithms and methods that directly calculate averages, demonstrates that the provided fusion processing method can efficiently and accurately generate the fused pseudo-color image. This ensures consistent brightness and hue across all regions of the image while significantly highlighting active farmed animals and "suppressing" the colors of dead farmed animals and irrelevant backgrounds. For poultry, the Deformable DETR model can also address the deformation effect of feathers. The object detection model trained using the above-mentioned training method can achieve the effects of low computational cost, low memory and graphics card resource consumption, better real-time performance, and significantly reduce the number of labeled samples required, as well as accelerate the training speed of the object detection model.

[0057] The following are examples of object detection methods:

[0058] Figure 8 The flowchart illustrating a target detection method for identifying individual dead farmed animals in a video based on ViT technology according to an exemplary embodiment of this application is shown below. Figure 8 The illustrated embodiments are described in detail below; please refer to them. Figure 8 .

[0059] S801, synthesize P pseudo-color three-channel images from the P short video segments to be identified, where P is a positive integer.

[0060] In an exemplary embodiment, all keyframes of any short video clip are extracted and synthesized into a pseudo-color three-channel image. For example, each short video clip is synthesized into an RGB image, i.e., an image with three channels: red (R), green (G), and blue (B). This transforms the processing of video data into the processing of image data.

[0061] S802, input the P pseudo-color three-channel images synthesized from the P short video clips into the target detection model to obtain the static and dynamic farmed animal individuals in the video clips.

[0062] In an exemplary embodiment, P pseudo-color three-channel images are input into a trained object detection model to obtain static and dynamic farmed animal individuals corresponding to video clips. The object detection model uses Deformable DETR, and the training method for the model can be referred to the above embodiment, and will not be repeated here.

[0063] For example, taking the identification of dead and surviving chickens in a poultry farm as an example, after the video clips are processed through a series of operations in the above embodiments, their corresponding pseudo-color image sets are input into the above target detection model. The target detection model will eventually output the positions of static targets (dead chickens) and dynamic targets (surviving chickens). (Reference) Figure 9 The diagram shown illustrates the results of identifying dead farmed animals based on ViT technology video analysis. For example, the target detection model can use different colored bounding boxes to mark and distinguish static targets (dead chickens) from dynamic targets (surviving chickens). For instance, red rectangles represent dead chickens, and green rectangles represent live chickens. The way the positions of static and dynamic targets are output is merely an example and is not limited in this embodiment.

[0064] This application utilizes keyframe extraction and unsupervised learning fusion methods to convert video data into image data. Deep learning is then employed to perform deep feature learning and extraction on the image data. After training the deep neural network, the video segments to be identified are converted into fused pseudo-color images, which are then input into the deep neural network to complete the identification and differentiation of static and dynamic targets. The fusion method provided in this application can efficiently and accurately generate pseudo-color images, thereby ensuring consistent brightness and hue across all regions of the image while significantly highlighting active farmed animals and "suppressing" the colors of dead farmed animals and irrelevant backgrounds.

[0065] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0066] in, Figure 10 A structural diagram of an apparatus for identifying dead farmed animals in a video based on ViT technology, according to an exemplary embodiment of this application, is shown.

[0067] The device 1000 for identifying dead farmed animals in videos based on ViT technology in this application embodiment includes: a first pseudo-color image determination module 1001, a labeling module 1002, a training module 1003, a second pseudo-color image determination module 1004, and a target recognition module 1005, wherein:

[0068] The first pseudo-color image determination module 1001 is used to synthesize M pseudo-color three-channel images from M short video clips used to train the model, where M is a positive integer.

[0069] The annotation module 1002 is used to: annotate M pseudo-color three-channel images corresponding to M short video segments, using bounding boxes to enclose each individual farmed animal in the image; and assign category attributes to the bounding boxes to distinguish between static and dynamic farmed animal individuals, thus obtaining a training sample set.

[0070] Training module 1003 is used to: input the training sample set into the ViT object detection model to train the model.

[0071] The second pseudo-color image determination module 1004 is used to synthesize P short video clips of the dead animal to be identified into P pseudo-color three-channel images, where P is a positive integer.

[0072] The target recognition module 1005 is used to: input P pseudo-color three-channel images corresponding to P short video segments of the dead animals to be identified into the trained ViT target detection model to obtain the static and dynamic farmed animal individuals corresponding to the video segments, that is, the farmed animal individuals that are dead and not dead.

[0073] It should be noted that the device for identifying dead farmed animals in videos based on ViT technology provided in the above embodiments is only illustrated by the division of the above functional modules when executing the method for identifying dead farmed animals in videos based on ViT technology. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device for identifying dead farmed animals in videos based on ViT technology and the method for identifying dead farmed animals in videos based on ViT technology provided in the above embodiments belong to the same concept. Therefore, for details not disclosed in the device embodiments of this application, please refer to the embodiments of the method for identifying dead farmed animals in videos based on ViT technology described above, which will not be repeated here.

[0074] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0075] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the methods described in the foregoing embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0076] This application also provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of any of the methods described above.

[0077] Figure 11 This diagram schematically illustrates the structure of a terminal according to an exemplary embodiment of this application. Please refer to... Figure 11 As shown, terminal 1100 includes a processor 1101 and a memory 1102.

[0078] In this embodiment, processor 1101 is the control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine. Processor 1101 may include one or more processing cores, such as a 4-core processor or an 8-core processor. Processor 1101 can be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 1101 may also include a main processor and a coprocessor. The main processor is used to process data in the wake-up state, also known as the Central Processing Unit (CPU); the coprocessor is a low-power processor used to process data in the standby state.

[0079] In this embodiment of the application, the processor 1101 is specifically used for:

[0080] Video recordings of farmed animals were captured using cameras, accumulating M short video segments. Keyframes from each video segment were extracted and synthesized into three images, which were then used as the RGB three-channel array to obtain a single pseudo-color image. Bounding boxes were used to label the locations of static and dynamic farmed animals within this image, resulting in a training sample set. This training sample set was then input into the ViT object detection model for training.

[0081] Specifically, the process involves extracting all 3*N keyframes from any given short video clip and synthesizing them into a pseudo-color three-channel image, where N is a positive integer; averaging the values ​​of keyframes 1 to N to synthesize a grayscale image, which serves as the red band in the pseudo-color three-channel image; averaging the values ​​of keyframes N+1 to 2*N to synthesize a grayscale image, which serves as the green band in the pseudo-color three-channel image; averaging the values ​​of keyframes 2*N+1 to 3*N to synthesize a grayscale image, which serves as the blue band in the pseudo-color three-channel image; and so on, resulting in M ​​pseudo-color three-channel images from all M short video clips.

[0082] Specifically, this involves annotating the M pseudo-color three-channel images synthesized from the M short video clips, and enclosing each individual farmed animal in the image with a bounding box. The bounding boxes are then assigned category attributes to distinguish between static and dynamic farmed animals. This yields the training sample set.

[0083] Specifically, this includes inputting the training sample set into the ViT object detection model to train the model.

[0084] The aforementioned processor 1101 is also specifically used for:

[0085] The farmed animals are recorded using cameras, and P short videos of dead animals to be identified are accumulated. All keyframes in each short video segment are selected and synthesized into 3 images, which are used as RGB three channels to obtain a pseudo-color image. This image is then input into the ViT object detection model that has been trained. The model calculates the positions of static farmed animals (dead animals) and dynamic farmed animals (surviving animals) in the video segments.

[0086] Specifically, the process involves extracting all 3*N keyframes from any short video clip of a dead animal to be identified, and synthesizing them into a pseudo-color three-channel image, where N is a positive integer; averaging the values ​​of keyframes 1 to N to synthesize a grayscale image, which serves as the red band in the pseudo-color three-channel image; averaging the values ​​of keyframes N+1 to 2*N to synthesize a grayscale image, which serves as the green band in the pseudo-color three-channel image; averaging the values ​​of keyframes 2*N+1 to 3*N to synthesize a grayscale image, which serves as the blue band in the pseudo-color three-channel image; and so on, resulting in P pseudo-color three-channel images from all P short video clips.

[0087] Specifically, the above-mentioned method involves inputting P images corresponding to P short video segments of the dead animals to be identified into the trained ViT object detection model. The model calculates the positions of static farmed animal individuals (dead animals) and dynamic farmed animal individuals (surviving animals) in the video segments, and encloses each farmed animal individual in the image with a rectangle of different category attributes.

[0088] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage terminals or flash memory terminals. In some embodiments of this application, the non-transitory computer-readable storage media in the memory 1102 are used to store at least one instruction, which is executed by the processor 1101 to implement the methods in the embodiments of this application.

[0089] In some embodiments, terminal 1100 further includes a peripheral terminal interface 1103 and at least one peripheral terminal. The processor 1101, memory 1102, and peripheral terminal interface 1103 can be connected via a bus or signal line. Each peripheral terminal can be connected to peripheral terminal interface 1103 via a bus, signal line, or circuit board. Specifically, the peripheral terminal includes at least one of a display screen 1104, a camera 1105, and an audio circuit 1106.

[0090] The peripheral terminal interface 1103 can be used to connect at least one input / output (I / O) related peripheral terminal to the processor 1101 and the memory 1102. In some embodiments of this application, the processor 1101, memory 1102, and peripheral terminal interface 1103 are integrated on the same chip or circuit board; in some other embodiments of this application, any one or two of the processor 1101, memory 1102, and peripheral terminal interface 1103 can be implemented on separate chips or circuit boards. This application does not specifically limit the implementation in this regard.

[0091] Display screen 1104 is used to display a user interface (UI). The UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1104 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1101 for processing. In this case, display screen 1104 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments of this application, there may be one display screen 1104, which serves as the front panel of terminal 1100; in other embodiments, there may be at least two display screens 1104, respectively disposed on different surfaces of terminal 1100 or in a folded design; in still other embodiments, display screen 1104 may be a flexible display screen, disposed on a curved or folded surface of terminal 1100. Furthermore, display screen 1104 may also be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. The display screen 1104 can be made of materials such as liquid crystal display (LCD) or organic light-emitting diode (OLED).

[0092] Camera 1105 is used to capture images or videos. Optionally, camera 1105 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusing the main camera and the depth-sensing camera, panoramic shooting by fusing the main camera and the wide-angle camera, virtual reality (VR) shooting, or other fusion shooting functions. In some embodiments of this application, camera 1105 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0093] The audio circuit 1106 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 1101 for processing. For stereo sound acquisition or noise reduction purposes, there may be multiple microphones, each located at a different part of the terminal 1100. The microphone may also be an array microphone or an omnidirectional microphone.

[0094] Power supply 1107 is used to power the various components in terminal 1100. Power supply 1107 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1107 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0095] The terminal structure block diagram shown in the embodiments of this application does not constitute a limitation on the terminal 1100. The terminal 1100 may include more or fewer components than shown, or combine certain components, or adopt different component arrangements.

[0096] In this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or order; the term "multiple" refers to two or more unless otherwise expressly defined. The terms "install," "connect," "link," "fix," etc., should be interpreted broadly. For example, "connect" can be a fixed connection, a detachable connection, or an integral connection; "link" can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0097] In the description of this application, it should be understood that the terms "upper" and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or unit referred to must have a specific orientation or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0098] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, equivalent variations made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. A method for identifying dead farmed animals in videos based on ViT technology, characterized in that, include: Video recordings are made of farmed animals using cameras, and M short video segments are accumulated. Each short video segment is 3 to 10 seconds long. All key frames in each short video segment are selected. For any short video segment in the M segments, 3*N key frames are extracted and synthesized into a pseudo-color three-channel image to obtain M pseudo-color three-channel images corresponding to the M short videos. M is a positive integer and N is a positive integer with N greater than 1. The positions of static and dynamic farmed animal individuals in M ​​pseudo-color three-channel images are marked with bounding boxes to obtain a training sample set. The training sample set is then input into the ViT object detection model to train the ViT object detection model. The farmed animals are recorded using cameras, and P short videos of dead animals to be identified are accumulated. Each short video is 3 to 10 seconds long. All key frames in each short video are selected. For any short video in the P short videos, 3*N key frames are extracted and synthesized into a pseudo-color three-channel image to obtain P pseudo-color three-channel images corresponding to the P short videos, where P is a positive integer. P pseudo-color three-channel images are input into the trained ViT object detection model to obtain the positions of static and dynamic farmed animal individuals in each short video segment. The static farmed animal individuals are the farmed animal individuals that have died, and the dynamic farmed animal individuals are the farmed animal individuals that have not died. The process of extracting 3*N keyframes from any one of the M short video segments and synthesizing a pseudo-color three-channel image includes: averaging the values ​​of keyframes 1 to N to synthesize a grayscale image, which serves as the red band in the pseudo-color three-channel image; averaging the values ​​of keyframes N+1 to 2*N to synthesize a grayscale image, which serves as the green band in the pseudo-color three-channel image; averaging the values ​​of keyframes 2*N+1 to 3*N to synthesize a grayscale image, which serves as the blue band in the pseudo-color three-channel image; and generating the pseudo-color three-channel image based on the red, green, and blue bands in the pseudo-color three-channel image. The keyframes are used to represent the clearest images in each short video segment. The time interval between the keyframes is 0.1 to 0.5 seconds. If the farmed animal shows movement in multiple keyframes with intervals of several seconds, it means that it is alive and moving. If the farmed animal shows a completely still static state, without even any body movements caused by breathing, it means that it is likely dead.

2. The method for identifying dead farmed animals in videos based on ViT technology according to claim 1, characterized in that, The method of marking the positions of static and dynamic farmed animal individuals in M ​​pseudo-color three-channel images with bounding boxes includes: Data annotation is performed on the M pseudo-color three-channel images corresponding to the M short videos. Each individual farmed animal in the image is enclosed in a rectangle, and the rectangle is assigned a category attribute to label the individual farmed animal as either static or dynamic.

3. The method for identifying dead farmed animals in videos based on ViT technology according to claim 1, characterized in that, The process of extracting 3*N keyframes from any one of the P-segment short videos and synthesizing them into a pseudo-color three-channel image includes: For the first to N keyframes, take the average value and synthesize a grayscale image as the red band in the pseudo-color three-channel image; For the N+1 to 2*N keyframes, take the average value and synthesize a grayscale image as the green band in the pseudo-color three-channel image; For the 2*N+1 to 3*N keyframes, take the average value and synthesize a grayscale image as the blue band in the pseudo-color three-channel image; The pseudo-color three-channel image is generated based on the red band, green band, and blue band in the pseudo-color three-channel image.

4. The method for identifying dead farmed animals in videos based on ViT technology according to claim 1, characterized in that, The step of inputting P pseudo-color three-channel images into the trained ViT object detection model to obtain the positions of static and dynamic farmed animal individuals in each short video segment includes: P pseudo-color three-channel images corresponding to P short videos of the dead animals to be identified are input into the trained ViT object detection model. The trained ViT object detection model calculates the positions of static and dynamic farmed animals in the images and encloses each farmed animal in the image with a rectangle of different category attributes.

5. A device for identifying dead farmed animals in videos based on ViT technology, characterized in that, include: The first pseudo-color image determination module is used to: record videos of farmed animals using cameras and accumulate M short video segments, each with a length of 3 to 10 seconds; select all keyframes in each short video segment; extract 3*N keyframes from any one of the M short video segments and synthesize a pseudo-color three-channel image to obtain M pseudo-color three-channel images corresponding to the M short video segments, where M is a positive integer and N is a positive integer and N is greater than 1; The annotation module is used to: annotate the M pseudo-color three-channel images corresponding to the M short videos, enclose each individual farmed animal in the image with a rectangle, and assign category attributes to the rectangle to label the static farmed animal individuals and the dynamic farmed animal individuals, thereby obtaining a training sample set; The training module is used to: input the training sample set into the ViT target detection model to train the ViT target detection model; The second pseudo-color image determination module is used to: record videos of farmed animals using cameras and accumulate P short videos of dead animals to be identified, each video being 3 to 10 seconds long; select all keyframes from each video; extract 3*N keyframes from any one of the P short videos; and synthesize a pseudo-color three-channel image to obtain P pseudo-color three-channel images corresponding to the P short videos, where P is a positive integer. The target recognition module is used to: input P pseudo-color three-channel images corresponding to P short videos of the dead animals to be identified into the trained ViT target detection model, and obtain the positions of static and dynamic farmed animal individuals corresponding to the videos, wherein the static farmed animal individuals are dead farmed animal individuals and the dynamic farmed animal individuals are non-dead farmed animal individuals; The process of extracting 3*N keyframes from any one of the M short video segments and synthesizing a pseudo-color three-channel image includes: averaging the values ​​of keyframes 1 to N to synthesize a grayscale image, which serves as the red band in the pseudo-color three-channel image; averaging the values ​​of keyframes N+1 to 2*N to synthesize a grayscale image, which serves as the green band in the pseudo-color three-channel image; averaging the values ​​of keyframes 2*N+1 to 3*N to synthesize a grayscale image, which serves as the blue band in the pseudo-color three-channel image; and generating the pseudo-color three-channel image based on the red, green, and blue bands in the pseudo-color three-channel image. The keyframes are used to represent the clearest images in each short video segment. The time interval between the keyframes is 0.1 to 0.5 seconds. If the farmed animal shows movement in multiple keyframes with intervals of several seconds, it means that it is alive and moving. If the farmed animal shows a completely still static state, without even any body movements caused by breathing, it means that it is likely dead.

6. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for identifying dead farmed animals in a video based on ViT technology as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for identifying dead farmed animals in a video based on ViT technology as described in any one of claims 1 to 4.

Images