Livestock farm video image refining method, device, equipment, medium and program product

Abstract

The present application provides a livestock farm video image refining method, device, equipment, medium and program product, the method comprises: obtaining a to-be-refined video image set comprising a plurality of to-be-processed images; based on the content difference degree between the to-be-processed images, the plurality of to-be-processed images are marked to obtain a first marking result; through an open set recognition algorithm, target detection is performed on the to-be-processed images to obtain a target detection result, based on the target detection result, the plurality of to-be-processed images are marked to obtain a second marking result and a third marking result, the second marking result and the third marking result respectively reflect the detection difficulty and the motion change degree of the target in the to-be-processed images; based on the first marking result, the second marking result and the third marking result, a data refining result is obtained. The present application can extract difficult example images, more representative images and images with higher value from a large number of monitoring video images, reduce the annotation amount of the training data set, and improve the quality of the training data set.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to methods, apparatus, equipment, media, and program products for refining video images from livestock farms. Background Technology

[0002] In traditional livestock farm environments, monitoring animal behavior and assessing health largely rely on manual observation. This method is not only inefficient but also easily influenced by the observer's subjective judgment. Currently, video surveillance systems are widely used in livestock farms, and machine vision algorithms can replace traditional manual supervision to achieve intelligent identification of targets such as animals, people, vehicles, and objects, thereby meeting the needs of intelligent livestock management. The processing accuracy of machine vision algorithms depends on the quality of the training dataset. Existing livestock farm machine vision algorithms rely on manual annotation of a large number of livestock farm monitoring video images for their training datasets.

[0003] However, livestock farm surveillance videos are characterized by high similarity and repetitiveness. If the training dataset for machine vision algorithms used in practical applications contains too many similar images or easily recognizable legends, and the content of the dataset is not balanced, the trained machine vision algorithm will encounter bottlenecks and problems in actual use in livestock farm scenarios. These problems may include difficulty in recognition and poor recognition stability. Furthermore, because livestock farm surveillance images contain a large number of individual animals, the annotation of the image dataset is extremely time-consuming and labor-intensive. Manual annotation can easily lead to visual fatigue for annotators, and the quality of the annotated data cannot be guaranteed. Therefore, in existing technologies, the quality of training datasets for livestock farm machine vision algorithms is not high. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, medium, and program product for refining livestock farm video images, which addresses the shortcomings of low-quality training datasets in existing livestock farm machine vision algorithms, and enables the refinement of livestock farm monitoring video images to generate high-quality training datasets.

[0005] This invention provides a method for refining video images from livestock farms, comprising: Obtain a set of video images to be refined, which includes multiple images to be processed; Based on the content difference between the images to be processed, multiple images to be processed are labeled to obtain a first labeling result; The image to be processed is subjected to target detection by an open set recognition algorithm to obtain the target detection results in the image to be processed. Based on the target detection results, multiple images to be processed are labeled to obtain a second labeling result and a third labeling result. The second labeling result reflects the difficulty of target detection in the image to be processed, and the third labeling result reflects the degree of motion change of the target in the image to be processed. Based on the first labeling result, the second labeling result, and the third labeling result, the data refining result of the video image set to be refined is obtained.

[0006] The livestock farm video image refining method provided by the present invention The step of labeling multiple images to be processed based on the target detection results to obtain a second labeling result and a third labeling result includes: The second labeling result of the image to be processed is determined based on the number of target detection boxes and the confidence level in the target detection results of the image to be processed.

[0007] According to the livestock farm video image refining method provided by the present invention, determining the second labeling result of the image to be processed based on the number and confidence of the target detection boxes in the target detection result of the image to be processed includes: The second labeling result of the image to be processed is determined based on the number of target detection boxes in the target detection results of the image to be processed, the degree of change in the number of target detection boxes in the neighboring images of the image to be processed, and the confidence level of the target detection boxes in the image to be processed.

[0008] According to the livestock farm video image refining method provided by the present invention, the step of labeling multiple images to be processed based on the target detection result to obtain a second labeling result and a third labeling result includes: Based on the target detection results, target segmentation is performed on the image to be processed to obtain the segmentation mask corresponding to the image to be processed. The third labeling result of the image to be processed is determined based on the segmentation mask of the image to be processed.

[0009] According to the livestock farm video image refinement method provided by the present invention, the step of performing target detection on the image to be processed using an open set recognition algorithm to obtain the target detection result in the image to be processed includes: The image to be processed is input into the open set recognition algorithm model to obtain the initial detection result output by the open set recognition algorithm model; Based on the confidence level and size of the detection boxes in the initial detection results, the number of detection boxes in the initial detection results is reduced to obtain the target detection results in the image to be processed.

[0010] According to the livestock farm video image refining method provided by the present invention, the step of obtaining the set of video images to be refined includes: The original surveillance video is acquired, and the original surveillance video is processed by extracting frames according to a preset frame extraction rule to obtain multiple images to be processed, thereby generating the set of video images to be refined.

[0011] The present invention also provides a livestock farm video image refining device, comprising: The image acquisition module is used to acquire a set of video images to be refined, which includes multiple images to be processed. The difference calculation module is used to label multiple images to be processed based on the content difference between the images to be processed, and obtain a first labeling result; The open set detection module is used to perform target detection on the image to be processed using an open set recognition algorithm, obtain the target detection results in the image to be processed, and label multiple images to be processed based on the target detection results to obtain a second labeling result and a third labeling result. The second labeling result reflects the difficulty of detecting the target in the image to be processed, and the third labeling result reflects the degree of motion change of the target in the image to be processed. The selection module is used to obtain the data refining result of the video image set to be refined based on the first marking result, the second marking result, and the third marking result.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the livestock farm video image refining method as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the livestock farm video image refining method as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the livestock farm video image refining method as described above.

[0015] The present invention provides a method, apparatus, equipment, medium, and program product for refining video images from livestock farms. The method includes: acquiring a set of video images to be refined, the set including multiple images to be processed; labeling the multiple images to be processed based on the content differences between them, obtaining a first labeling result; performing target detection on the images to be processed using an open set recognition algorithm, obtaining target detection results in the images to be processed; labeling the multiple images to be processed based on the target detection results, obtaining a second labeling result and a third labeling result, the second labeling result reflecting the difficulty of target detection in the images to be processed, and the third labeling result reflecting the degree of motion change of the targets in the images to be processed; and obtaining a data refinement result for the set of video images to be refined based on the first labeling result, the second labeling result, and the third labeling result.

[0016] This invention calculates the content difference between surveillance video images to be processed and uses an open set recognition algorithm to detect targets in the surveillance video images to be processed. Based on the target detection results of the open set recognition algorithm, it judges the difficulty of target detection and the degree of target motion change in the image to be processed. It can extract difficult images, more representative images and higher value images from a large number of surveillance video images, reduce the amount of annotation in the training dataset and improve the quality of the training dataset. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a method for refining video images from a livestock farm, provided by the present invention.

[0019] Figure 2 This is a flowchart illustrating the construction of an image dataset in a livestock farm video image refining method provided by the present invention.

[0020] Figure 3 This is a flowchart of image selection in a livestock farm video image refining method provided by the present invention.

[0021] Figure 4 This is a schematic diagram comparing the image dataset construction process in the livestock farm video image refining method provided by the present invention with traditional methods.

[0022] Figure 5 This is a schematic diagram illustrating the effect of a livestock farm video image refining method provided by the present invention. Figure 1 .

[0023] Figure 6 This is a schematic diagram illustrating the effect of a livestock farm video image refining method provided by the present invention. Figure 2 .

[0024] Figure 7 This is a schematic diagram of the structure of the livestock farm video image refining device provided by the present invention.

[0025] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] like Figure 1 As shown, the present invention provides a method for refining livestock farm video images, comprising the following steps: S110. Obtain the set of video images to be refined, which includes multiple images to be processed. S120. Label multiple images to be processed based on the content difference between the images to be processed, and obtain the first labeling result; S130. Target detection is performed on the image to be processed using an open set recognition algorithm to obtain the target detection results in the image to be processed. Based on the target detection results, multiple images to be processed are labeled to obtain a second labeling result and a third labeling result. The second labeling result reflects the difficulty of target detection in the image to be processed, and the third labeling result reflects the degree of motion change of the target in the image to be processed. S140. Based on the first labeling result, the second labeling result, and the third labeling result, the data refining result of the video image set to be refined is obtained.

[0028] The method provided by this invention calculates the content differences between surveillance video images to be processed and uses an open-set recognition algorithm to detect targets in these images. Based on the target detection results of the open-set recognition algorithm, it determines the detection difficulty and the degree of motion change of the targets in the images to be processed. This allows for the extraction of difficult, more representative, and higher-value images from a large number of surveillance video images, reducing the amount of annotation required in the training dataset and improving its quality. Training a machine vision algorithm suitable for livestock farm scenarios using a higher-quality training dataset avoids overfitting and poor robustness, thereby improving the algorithm's performance, accuracy, and stability in livestock farm scenarios.

[0029] In one possible implementation, the set of video images to be refined can be all frames from the original surveillance video. However, frames in livestock farm surveillance videos are characterized by high similarity and repetition. Therefore, in one implementation, all frames from the original surveillance video are first extracted to obtain the set of video images to be refined, and then further refinement processing is performed. Specifically, obtaining the set of video images to be refined includes: The original surveillance video is acquired, and then frame-by-frame processing is performed on the original surveillance video according to the preset frame-by-frame extraction rules to obtain multiple images to be processed, generating a set of video images to be refined.

[0030] The raw surveillance video is captured by the monitoring equipment without frame reduction. The number of frames in the raw video is related to the image acquisition settings (frame rate, etc.) of the monitoring equipment. From the raw video, frames are extracted according to a preset frame extraction rule to obtain the individual images to be processed in the set of video images to be refined. The frame extraction rule can define the interval time, frame extraction range, frame extraction ratio, etc. Through frame extraction, the size of the raw data can be initially reduced.

[0031] After obtaining the set of video images to be refined, such as Figure 2 As shown, the processing of the video image set to be refined is divided into two parallel branches: one branch calculates the content difference of the images to be processed included therein to obtain the first label result, and the other branch inputs the images to be processed included therein into the open set recognition algorithm model, and performs target detection on the images to be processed through the open set recognition algorithm to obtain the target detection result in the images to be processed.

[0032] Specifically, the method provided by this invention is based on three factors to achieve the recognition of images that can provide higher algorithmic learning value in the refined video image set: the content difference between the image and other images, the difficulty of detecting the target in the image, and the degree of motion change of the target in the image.

[0033] The content difference between an image and other images reflects the degree of change in scene, lighting conditions, and content. A larger difference indicates that the image has undergone significant changes within a short period. Image difference can help filter out images with significant scene changes over a period of time. These images usually contain more information, helping to improve the diversity and representativeness of the dataset. In a livestock farm, this method can capture changes in the environment and scene, thus reflecting the actual situation more comprehensively. Image filtering based on the difficulty of detecting targets in the images helps to identify and retain video images with significant changes in the number of targets. In a livestock farm, significant changes in the number of animals may indicate the occurrence of special events or behaviors, or correspond to scenes where the recognition algorithm performs poorly. Capturing these changes is crucial for behavioral analysis and anomaly detection. Therefore, filtering these images into the training dataset can reveal the performance of the detection algorithm in complex situations, helping to optimize the accuracy and robustness of the model. Significant activity changes indicate that animals exhibit relatively obvious behaviors within a short period of time. Capturing these short-term activity changes is crucial for behavioral analysis and event monitoring, as it helps to accurately record and analyze animal dynamics. In livestock farms, capturing these intense activities helps monitor animal health and behavioral changes, providing a basis for further management and decision-making. Therefore, selecting these images for the training dataset can improve the model's ability to capture animal behavior and enhance model performance.

[0034] The first labeling result includes adding a first label and not adding a second label. The image to be processed with the first label has a greater image difference relative to its neighboring images than the image to be processed without the first label. The content difference between images to be processed can be measured using perceptual hash values ​​(pHash). For example, the perceptual hash value of the image to be processed can be calculated and compared with the adjacent previous image. If the difference exceeds a preset hash value difference threshold, it is determined that the content difference between the image to be processed and the adjacent previous image is greater than the difference threshold, and the first label is added to both the image to be processed and the adjacent previous image. This ensures that frames with significant changes in scene and lighting conditions are captured, enhancing the scene diversity of the data. Of course, it is understandable that other metrics can also be used to measure the content difference between images to be processed, such as structural similarity metrics (SSIM) and MAD (Mean Absolute Difference). The difficulty of detecting targets and the degree of motion change in the images to be processed can be judged based on the detection results of open-set recognition.

[0035] Open-set recognition algorithms are contrasted with closed-set recognition algorithms. Closed-set recognition algorithms know all target categories and can only identify predetermined targets, while open-set recognition algorithms can identify targets of known categories as well as those of unknown categories. Large open-set recognition models can effectively identify various targets in images even with zero-shot learning. The open-set recognition algorithm is used to perform target detection on the image being processed, obtaining the target detection results, including: The image to be processed is input into the open set recognition algorithm model to obtain the initial detection result output by the open set recognition algorithm model; Based on the confidence level and size of the detection boxes in the initial detection results, the number of detection boxes in the initial detection results is reduced to obtain the target detection results in the image to be processed.

[0036] like Figure 2 As shown, while inputting the image to be processed into the open-set recognition algorithm model, the cue words for the detected target can also be input simultaneously. The open-set recognition algorithm model can output initial detection results. Although the open-set recognition algorithm can detect targets of unknown categories, its detection accuracy is not high enough, and it may also detect objects in the image that are not targets, generating detection boxes. The initial detection results output by the open-set recognition algorithm model include detection boxes and their confidence scores. The method provided by this invention reduces the number of detection boxes in the initial detection results based on the confidence scores and sizes of the detection boxes, thereby improving the efficiency and accuracy of subsequent processing.

[0037] Specifically, the number of detection boxes in the initial detection results is reduced based on the confidence level and size of the detection boxes, including: Delete detection boxes whose confidence level is lower than the preset confidence threshold in the initial detection results; The intersection-union ratio (CUI) of the remaining detection boxes and the reference detection box is obtained sequentially. Detection boxes with CUI ratios higher than the preset CUI threshold are deleted. The reference detection box is the detection box with the highest confidence in the initial detection results.

[0038] The above method can control the output of the open set recognition algorithm model. By adjusting the size of the recognition result box, confidence score, and non-maximum suppression of multiple recognition boxes for the same target, the number of target object recognition boxes can be controlled to be slightly larger than the actual box size. This reduces the number of new labeling operations and slightly increases the number of label deletion operations in the subsequent manual correction process, thus reducing the difficulty of manual annotation.

[0039] Specifically, based on the target detection results, multiple images to be processed are labeled to obtain a second labeling result and a third labeling result, including: Based on the number of target detection boxes and the confidence level in the target detection results of the image to be processed, the second labeling result of the image to be processed is determined; Based on the target detection results, target segmentation is performed on the image to be processed to obtain the segmentation mask corresponding to the image to be processed. The third labeling result of the image to be processed is determined based on the segmentation mask of the image to be processed.

[0040] The second labeling result includes adding the second label and not adding the second label, and the third labeling result includes adding the third label and not adding the third label. The target detection difficulty in the image to be processed with the second label is higher than that in the image to be processed without the second label; the degree of motion change of the target in the image to be processed with the third label is higher than that in the image to be processed without the third label.

[0041] Specifically, based on the number of target detection boxes and the confidence level in the target detection results of the image to be processed, the second labeling result of the image to be processed is determined, including: The second labeling result of the image to be processed is determined based on the number of target detection boxes in the target detection results of the image to be processed, the degree of change in the number of target detection boxes in the neighboring images of the image to be processed, and the confidence level of the target detection boxes in the image to be processed.

[0042] like Figure 3 As shown, when the number of target detection boxes in the image to be processed is equal to 0 or very small (e.g., less than a preset minimum value), a second label is added to the image to be processed. If the number of target detection boxes in the image to be processed is equal to 0 or very small, it means that the target detection in the image to be processed may be difficult. Selecting it can avoid missing potentially important information in the video image.

[0043] After obtaining the target detection results of the image to be processed, target segmentation is performed on the image to be processed based on the target detection results to obtain a segmentation mask corresponding to the image to be processed. The segmentation mask corresponding to the image to be processed reflects the region occupied by the target in the image to be processed. Based on the number of target detection boxes in the target detection results of the image to be processed, the degree of change in the number of target detection boxes in the neighboring images of the image to be processed, and the confidence of the target detection boxes in the image to be processed, the second labeling result of the image to be processed is determined, which also includes: If the average confidence score of the target detection boxes in the image to be processed is lower than the preset average threshold, a second label is added to the image to be processed. Based on the number of target detection boxes in the image to be processed and the images before and after it, the average value of the detection boxes in the image to be processed is calculated. If the difference between the average values ​​of the detection boxes corresponding to two adjacent images to be processed is greater than a preset difference threshold, then a second marker is added to the two adjacent images to be processed.

[0044] The images to be processed in the video image set to be refined can be arranged in chronological order of acquisition time, with adjacent images representing images acquired consecutively in the same timeframe. Adding a second label based on the average confidence score of the target detection boxes in the images ensures that the dataset includes more challenging detection scenarios. These labeled images can subsequently improve the robustness of the model. Adding a second label based on the variation in the number of target detection boxes in the images ensures that scenes with significant changes in the number of targets are captured, reflecting the dynamic behavior of the animals and the complexity of the scene, and identifying data where the model's performance is unstable.

[0045] The third labeling result of the image to be processed is determined based on the segmentation mask of the image to be processed, including: Obtain the similarity coefficient between the segmentation masks of adjacent images to be processed, and add a third label to two images to be processed whose similarity coefficient is lower than a preset value.

[0046] The similarity coefficient between the segmentation masks of adjacent images to be processed can be calculated using various known similarity calculation methods (such as the Jaccard similarity coefficient calculation method). By obtaining the similarity coefficient between the segmentation masks of adjacent images to be processed, activity changes at the recognition result level can be captured, image changes caused by non-targets can be excluded, scenes of drastic changes in animal activity can be captured, and the diversity and representativeness of the data can be improved, capturing data covering animal behavior. Based on the first, second, and third labeling results, the selection of images to be processed can be achieved by adding images with the first labeling result to the first set, and adding images with the second and third labeling results to the second set. The union of the first and second sets can be directly taken as the data refinement result, or a score can be calculated for the images to be processed with the second and / or third labeling results, and the images in the second set can be further selected before taking the union with the first set. Specifically, the calculation results based on confidence level, number of detection boxes, and segmentation mask can be normalized by applying weights to obtain a comprehensive score for each image to be processed. Images exceeding the comprehensive score threshold are retained in a second set, effectively separating key data from the video content that vary due to scene changes, changes in recognition difficulty, and changes in animal activity. Furthermore, as... Figure 2 As shown, the data refinement result is obtained after passing through the data selector (equivalent to...) Figure 2After the "secondary selection of images" in the data refinement results, the image to be processed and the target detection results in the image to be processed (i.e., Figure 2 The "coarse annotation results" are submitted for manual review and correction, ultimately resulting in a training dataset that can be used for machine vision algorithms in livestock farm monitoring scenarios. Figure 4 As shown, compared to existing annotation methods that rely entirely on manual intervention, the method provided by this invention reduces human involvement in the data mining and labeling process, significantly reducing labor costs, improving the objectivity and standardization of data screening, accelerating the dataset construction process, and increasing data utilization efficiency. Furthermore, the screening method based on image similarity and model recognition results can uncover portions of the data with significant differences and high algorithmic recognition difficulty, balancing the content and difficulty of the dataset, and offering advantages such as greater objectivity, stability, and controllability.

[0047] An experiment was conducted using an image dataset constructed from 12 hours of surveillance video from a large-scale beef cattle farm to support a machine vision algorithm, verifying the effectiveness of the method provided in this invention. The original data contains a total of 670,978 frames, such as... Figure 5 As shown, according to the frame extraction rule of "taking 30 seconds every 600 seconds, and 5 frames per second", the initial sampled image contains 11,400 frames. The data selector intelligently selects from the initial sampled image, retaining 1,552 frames. Among these 1,552 selected frames, a total of 25,000 cattle appear. Traditional data annotation methods would require 870 man-hours to complete the labeling of individuals and outlines. By debugging a livestock farm-specific open-set recognition algorithm, the data in this example was automatically labeled. The quality of the automated labeling results was manually reviewed and corrected, such as... Figure 6 As shown, the labelers no longer perform the traditional task of adding outlines and borders to each target. Instead, they delete, correct, and add new elements to the completed results. Through the combination of automated labeling and manual correction, labor costs have been reduced to 65 man-hours, and the cost of dataset labeling has been reduced by 90%.

[0048] The livestock farm video image refining apparatus provided by the present invention is described below. The livestock farm video image refining apparatus described below can be referred to in correspondence with the livestock farm video image refining method described above. Figure 7 As shown, the livestock farm video image refining device provided by the present invention includes: The image acquisition module 710 is used to acquire a set of video images to be refined, wherein the set of video images to be refined includes multiple images to be processed; The difference calculation module 720 is used to label multiple images to be processed based on the content difference between the images to be processed, and obtain a first labeling result; The open set detection module 730 is used to perform target detection on the image to be processed using an open set recognition algorithm, obtain the target detection result in the image to be processed, and label multiple images to be processed based on the target detection result to obtain a second labeling result and a third labeling result. The second labeling result reflects the difficulty of detecting the target in the image to be processed, and the third labeling result reflects the degree of motion change of the target in the image to be processed. Selection module 740 is used to obtain the data refining result of the video image set to be refined based on the first marking result, the second marking result and the third marking result.

[0049] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a livestock farm video image refining method. The livestock farm video image refining method includes: acquiring a set of video images to be refined, which includes multiple images to be processed; labeling the multiple images to be processed based on the content differences between them, obtaining a first labeling result; performing target detection on the images to be processed using an open set recognition algorithm, obtaining target detection results in the images to be processed; labeling the multiple images to be processed based on the target detection results, obtaining a second labeling result and a third labeling result. The second labeling result reflects the difficulty of target detection in the images to be processed, and the third labeling result reflects the degree of motion change of the targets in the images to be processed; and obtaining a data refining result for the set of video images to be refined based on the first labeling result, the second labeling result, and the third labeling result.

[0050] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0051] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the above-mentioned livestock farm video image refining method. The livestock farm video image refining method includes: acquiring a set of video images to be refined, which includes multiple images to be processed; marking the multiple images to be processed based on the content differences between the images to be processed to obtain a first marking result; performing target detection on the images to be processed using an open set recognition algorithm to obtain target detection results in the images to be processed; marking the multiple images to be processed based on the target detection results to obtain a second marking result and a third marking result, whereby the second marking result reflects the difficulty of detecting targets in the images to be processed, and the third marking result reflects the degree of motion change of targets in the images to be processed; and obtaining a data refining result for the set of video images to be refined based on the first marking result, the second marking result, and the third marking result.

[0052] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described livestock farm video image refining method. The livestock farm video image refining method includes: acquiring a set of video images to be refined, the set of video images to be refined including multiple images to be processed; labeling the multiple images to be processed based on the content differences between the images to be processed to obtain a first labeling result; performing target detection on the images to be processed using an open set recognition algorithm to obtain target detection results in the images to be processed; labeling the multiple images to be processed based on the target detection results to obtain a second labeling result and a third labeling result, the second labeling result reflecting the difficulty of target detection in the images to be processed, and the third labeling result reflecting the degree of motion change of the targets in the images to be processed; and obtaining a data refining result of the set of video images to be refined based on the first labeling result, the second labeling result, and the third labeling result.

[0053] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0054] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for refining video images from a livestock farm, characterized in that, include: Obtain a set of video images to be refined, which includes multiple images to be processed; Based on the content difference between the images to be processed, multiple images to be processed are labeled to obtain a first labeling result. The first labeling result includes adding a first label and not adding a second label. The image to be processed with the first label added has a greater image difference relative to its neighboring images than the image to be processed without the first label added. The image to be processed is subjected to target detection using an open set recognition algorithm to obtain target detection results. Based on the target detection results, multiple images to be processed are labeled to obtain a second labeling result and a third labeling result. The second labeling result reflects the difficulty of target detection in the image to be processed, and the third labeling result reflects the degree of motion change of the target in the image to be processed. The second labeling result includes adding a second label and not adding a second label, and the third labeling result includes adding a third label and not adding a third label. The difficulty of target detection in the image to be processed with the second label is higher than that in the image to be processed without the second label; the degree of motion change of the target in the image to be processed with the third label is higher than that in the image to be processed without the third label. Based on the first labeling result, the second labeling result, and the third labeling result, the data refining result of the video image set to be refined is obtained.

2. The method for refining livestock farm video images according to claim 1, characterized in that, The step of labeling multiple images to be processed based on the target detection results to obtain a second labeling result and a third labeling result includes: The second labeling result of the image to be processed is determined based on the number of target detection boxes and the confidence level in the target detection results of the image to be processed.

3. The method for refining livestock farm video images according to claim 2, characterized in that, The determination of the second labeling result of the image to be processed based on the number of target detection boxes and the confidence level in the target detection results of the image to be processed includes: The second labeling result of the image to be processed is determined based on the number of target detection boxes in the target detection results of the image to be processed, the degree of change in the number of target detection boxes in the neighboring images of the image to be processed, and the confidence level of the target detection boxes in the image to be processed.

4. The livestock farm video image refining method according to claim 1, characterized in that, The step of labeling multiple images to be processed based on the target detection results to obtain a second labeling result and a third labeling result includes: Based on the target detection results, target segmentation is performed on the image to be processed to obtain the segmentation mask corresponding to the image to be processed. The third labeling result of the image to be processed is determined based on the segmentation mask of the image to be processed.

5. The method for refining livestock farm video images according to claim 1, characterized in that, The step of performing target detection on the image to be processed using an open set recognition algorithm to obtain the target detection result in the image to be processed includes: The image to be processed is input into the open set recognition algorithm model to obtain the initial detection result output by the open set recognition algorithm model; Based on the confidence level and size of the detection boxes in the initial detection results, the number of detection boxes in the initial detection results is reduced to obtain the target detection results in the image to be processed.

6. The method for refining livestock farm video images according to any one of claims 1-5, characterized in that, The acquisition of the set of video images to be refined includes: The original surveillance video is acquired, and the original surveillance video is processed by extracting frames according to a preset frame extraction rule to obtain multiple images to be processed, thereby generating the set of video images to be refined.

7. A livestock farm video image refining device, characterized in that, include: The image acquisition module is used to acquire a set of video images to be refined, which includes multiple images to be processed. The difference calculation module is used to mark multiple images to be processed based on the content difference between the images to be processed, and obtain a first marking result. The first marking result includes adding a first mark and not adding a second mark. The image difference of the image to be processed relative to the adjacent images is greater than that of the image to be processed relative to the adjacent images without adding a first mark. The open set detection module is used to perform target detection on the image to be processed using an open set recognition algorithm, obtain target detection results in the image to be processed, and label multiple images to be processed based on the target detection results to obtain a second labeling result and a third labeling result. The second labeling result reflects the difficulty of target detection in the image to be processed, and the third labeling result reflects the degree of motion change of the target in the image to be processed. The second labeling result includes adding a second label and not adding a second label, and the third labeling result includes adding a third label and not adding a third label. The difficulty of target detection in the image to be processed with the second label is higher than that in the image to be processed without the second label; the degree of motion change of the target in the image to be processed with the third label is higher than that in the image to be processed without the third label. The selection module is used to obtain the data refining result of the video image set to be refined based on the first marking result, the second marking result, and the third marking result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the livestock farm video image refining method as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the livestock farm video image refining method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the livestock farm video image refining method as described in any one of claims 1-6.

Images