Livestock farm video image refining method, device, equipment, medium and program product

By calculating the content difference of video images and using open set recognition algorithm to screen the object detection results, a high-quality training data set is generated, which solves the problem of low quality of training data sets in the existing technology and improves the animal farm machine vision algorithm Performance and identification accuracy.

CN118939833AActive Publication Date: 2024-11-12CHINA AGRI UNIV +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410928517.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-12
Estimated Expiration
2044-07-11

AI Technical Summary

Technical Problem

The quality of the training data set of existing animal farm machine vision algorithms is not high, which leads to difficulty in identification and poor recognition stability. The manual labeling process is time-consuming and labor-intensive, resulting in the inability to guarantee the quality of the labeling data.

Method used

By obtaining the video image set to be refined, the content difference between images is calculated, and the object detection is used to detect objects using open set recognition algorithms, and the object detection results are obtained. Based on these results, the images are marked and difficult images and representative images are selected. Images, generate high-quality training datasets.

Benefits of technology

The amount of labeling of the training data set is reduced, the quality of the data set is improved, the problems of overfitting and poor robustness are avoided, and the performance, recognition accuracy and stability of machine vision algorithms in livestock farm scenarios are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939833A_ABST
    Figure CN118939833A_ABST
Patent Text Reader

Abstract

The invention provides a livestock farm video image refining method, device and equipment, a medium and a program product. The method comprises the following steps: acquiring a to-be-refined video image set comprising a plurality of to-be-processed images; marking the plurality of to-be-processed images based on the content difference degree between the to-be-processed images to obtain a first marking result; performing target detection on the to-be-processed images through an open set recognition algorithm to obtain a target detection result, and marking the plurality of to-be-processed images based on the target detection result to obtain a second marking result and a third marking result, the second marking result and the third marking result respectively reflect the detection difficulty and the motion change degree of the target in the to-be-processed image; and obtaining a data refining result based on the first marking result, the second marking result and the third marking result. According to the method, the difficult image, the more representative image and the image with the higher value can be extracted from a large number of monitoring video images, the labeling amount of the training data set is reduced, and the quality of the training data set is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a livestock farm video image refining method, device, equipment, medium and program product. Background Art

[0002] In traditional livestock farm environments, animal behavior monitoring and health assessment mostly rely on manual observation, which is not only inefficient but also easily affected by the observer's judgment. At present, video surveillance systems are widely used in livestock farms. Machine vision algorithms can replace traditional manual supervision to achieve intelligent recognition of targets such as animals, people, vehicles, and objects, thereby meeting the needs of intelligent breeding management. The processing accuracy of machine vision algorithms depends on the quality of the training data set. The training data set of existing livestock farm machine vision algorithms relies on manual annotation of a large number of livestock farm monitoring video images.

[0003] However, the content of livestock farm monitoring videos has the characteristics of high similarity and strong repetitiveness. If the training data set constructed for the actual application of machine vision algorithms contains too many similar images or easily recognizable legends, and the content in the data set is not balanced, the trained machine vision algorithm will have bottlenecks and problems in the actual use of livestock farm scenes, such as difficulty in recognition and poor recognition stability. At the same time, since livestock farm monitoring images contain a large number of individual animals, the labeling of image data sets is very time-consuming and labor-intensive. In the manual labeling process, it is easy to cause visual fatigue of the labelers, and the quality of the labeled data cannot be guaranteed. Therefore, in the prior art, the quality of the training data set for livestock farm machine vision algorithms is not high. Summary of the invention

[0004] The present invention provides a livestock farm video image refining method, device, equipment, medium and program product to solve the defect of low quality of training data sets of livestock farm machine vision algorithms in the prior art, and to refine livestock farm monitoring video images to generate high-quality training data sets.

[0005] The present invention provides a livestock farm video image refining method, comprising: Acquire a video image set to be refined, wherein the video image set to be refined includes a plurality of images to be processed; Marking the plurality of images to be processed based on the content differences between the images to be processed to obtain a first marking result; Performing target detection on the image to be processed by an open set recognition algorithm to obtain a target detection result in the image to be processed, marking a plurality of the images to be processed based on the target detection result to obtain a second marking result and a third marking result, wherein the second marking result reflects the detection difficulty of the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; A data refinement result of the video image set to be refined is obtained based on the first labeling result, the second labeling result and the third labeling result.

[0006] According to the livestock farm video image refining method provided by the present invention,

[0007] The step of marking the plurality of images to be processed based on the target detection result to obtain a second marking result and a third marking result includes: The second labeling result of the image to be processed is determined based on the number and confidence of the target detection boxes in the target detection result of the image to be processed.

[0008] According to the livestock farm video image refining method provided by the present invention, the second labeling result of the image to be processed is determined based on the number and confidence of the target detection frames in the target detection result of the image to be processed, including: The second labeling result of the image to be processed is determined based on the number of target detection frames in the target detection result of the image to be processed, the degree of change in the number of the target detection frames in the adjacent images of the image to be processed, and the confidence of the target detection frames in the image to be processed.

[0009] According to the livestock farm video image refining method provided by the present invention, the multiple images to be processed are marked based on the target detection result to obtain the second marking result and the third marking result, including: Performing target segmentation on the image to be processed based on the target detection result to obtain a segmentation mask corresponding to the image to be processed; The third labeling result of the image to be processed is determined based on the segmentation mask of the image to be processed.

[0010] According to the livestock farm video image refining method provided by the present invention, the target detection is performed on the image to be processed by an open set recognition algorithm to obtain the target detection result in the image to be processed, including: Inputting the image to be processed into an open set recognition algorithm model to obtain an initial detection result output by the open set recognition algorithm model; The number of detection frames in the initial detection result is reduced based on the confidence of the detection frames in the initial detection result and the size of the detection frames to obtain the target detection result in the image to be processed.

[0011] According to the livestock farm video image refining method provided by the present invention, the step of obtaining a video image set to be refined comprises: The original surveillance video is obtained, and frame extraction is performed on the original surveillance video according to a preset frame extraction rule to obtain a plurality of images to be processed, thereby generating the video image set to be refined.

[0012] The present invention also provides a livestock farm video image refining device, comprising: An image acquisition module, used for acquiring a video image set to be refined, wherein the video image set to be refined includes a plurality of images to be processed; A difference calculation module, used for marking a plurality of images to be processed based on the content difference between the images to be processed to obtain a first marking result; an open set detection module, configured to perform target detection on the image to be processed by using an open set recognition algorithm, obtain a target detection result in the image to be processed, and mark a plurality of the images to be processed based on the target detection result to obtain a second marking result and a third marking result, wherein the second marking result reflects the detection difficulty of the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; The selection module is used to obtain a data refining result of the video image set to be refined based on the first marking result, the second marking result and the third marking result.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the livestock farm video image refining methods described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the livestock farm video image refining method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for refining a livestock farm video image as described above is implemented.

[0016] The present invention provides a livestock farm video image refining method, device, equipment, medium and program product, wherein the method comprises: obtaining a video image set to be refined, wherein the video image set to be refined includes multiple images to be processed; marking the multiple images to be processed based on the content difference between the images to be processed to obtain a first marking result; performing target detection on the images to be processed by an open set recognition algorithm to obtain the target detection result in the images to be processed, marking the multiple images to be processed based on the target detection result to obtain a second marking result and a third marking result, wherein the second marking result reflects the detection difficulty of the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; based on the first marking result, the second marking result and the third marking result, obtaining a data refining result of the video image set to be refined.

[0017] The present invention calculates the content difference between the surveillance video images to be processed, and uses the open set recognition algorithm to perform target detection on the surveillance video images to be processed. Based on the target detection results of the open set recognition algorithm, the detection difficulty of the target in the image to be processed and the degree of motion change of the target are judged. Difficult images, more representative images and images with higher value can be extracted from a large number of surveillance video images, thereby reducing the annotation amount of the training data set and improving the quality of the training data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 The present invention is a schematic flow chart of a livestock farm video image refining method provided by the present invention.

[0020] Figure 2 It is a flow chart of constructing an image data set in a livestock farm video image refining method provided by the present invention.

[0021] Figure 3 The present invention provides a flowchart of image selection in a livestock farm video image refining method.

[0022] Figure 4 It is a schematic diagram comparing the construction process of an image data set in a livestock farm video image refining method provided by the present invention with a traditional method.

[0023] Figure 5 This is a schematic diagram of the effect of a livestock farm video image refining method provided by the present invention. Figure 1 .

[0024] Figure 6 This is a schematic diagram of the effect of a livestock farm video image refining method provided by the present invention. Figure 2 .

[0025] Figure 7 It is a structural schematic diagram of a livestock farm video image refining device provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] like Figure 1 As shown, the present invention provides a livestock farm video image refining method, comprising the steps of: S110, obtaining a video image set to be refined, where the video image set to be refined includes a plurality of images to be processed; S120, marking a plurality of images to be processed based on the content differences between the images to be processed to obtain a first marking result; S130, performing target detection on the image to be processed by an open set recognition algorithm, obtaining a target detection result in the image to be processed, marking a plurality of images to be processed based on the target detection result, and obtaining a second marking result and a third marking result, wherein the second marking result reflects the detection difficulty of the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; S140 . Obtain a data refining result of the video image set to be refined based on the first marking result, the second marking result, and the third marking result.

[0029] The method provided by the present invention calculates the content difference between the surveillance video images to be processed, and uses the open set recognition algorithm to perform target detection on the surveillance video images to be processed, and judges the detection difficulty of the target in the image to be processed and the degree of motion change of the target based on the target detection result of the open set recognition algorithm. It can extract difficult images, more representative images and images with higher value from a large number of surveillance video images, reduce the annotation amount of the training data set, and improve the quality of the training data set. The machine vision algorithm suitable for the livestock farm scene can be obtained by training with a higher quality training data set, which can avoid the problems of overfitting and poor robustness, thereby improving the performance of the algorithm in the livestock farm scene, and improving the recognition accuracy and recognition stability.

[0030] In a possible implementation, the video image set to be refined may be all frames in the original surveillance video. However, the frames in the livestock farm surveillance video have the characteristics of high similarity and strong repetitiveness. Therefore, in an implementation, all frames in the original surveillance video are first extracted to obtain the video image set to be refined, and then further refined. Specifically, obtaining the video image set to be refined includes: The original surveillance video is obtained, and frame extraction is performed on the original surveillance video according to a preset frame extraction rule to obtain multiple images to be processed, thereby generating a video image set to be refined.

[0031] The original surveillance video is obtained by the surveillance equipment without frame deletion. The number of frames in the original surveillance video is related to the image acquisition settings (frame rate, etc.) of the surveillance equipment. In the original surveillance video, the frames are extracted according to the preset frame extraction rules to obtain the images to be processed in the video image set to be refined. The frame extraction rules can define the interval time, frame extraction range, frame extraction ratio, etc. Through frame extraction processing, the amount of original data can be initially reduced.

[0032] After obtaining the video image set to be refined, Figure 2 As shown, the processing of the refined video image set is divided into two parallel branches: one is to calculate the content difference of the images to be processed therein to obtain a first labeling result, and the other is to input the images to be processed therein into an open set recognition algorithm model, perform target detection on the images to be processed through the open set recognition algorithm, and obtain the target detection result in the images to be processed.

[0033] Specifically, the method provided by the present invention realizes the recognition of images that can provide higher algorithm learning value in the refined video image set based on three factors: the content difference between the image and other images, the difficulty of detecting the target in the image, and the degree of motion change of the target in the image.

[0034] The content difference between an image and other images reflects the degree of change in the scene, lighting conditions, and shooting content of the image. A larger difference means that the image has changed significantly in a short period of time. Image difference can help filter out images with large scene changes over a period of time. These images usually contain more information and help improve the diversity and representativeness of the data set. In livestock farms, this method can capture changes in the environment and scenes, thereby more comprehensively reflecting the actual situation. Image screening based on the difficulty of detecting targets in the image helps to identify and retain video images with significant changes in the number of targets. In livestock farms, significant changes in the number of animals may mean the occurrence of special events or behaviors or scenes where the corresponding recognition algorithm performs unstablely. Capturing these changes is very important for behavioral analysis and anomaly detection. Therefore, filtering these images into the training data set can reveal the performance of the detection algorithm in complex situations and help optimize the accuracy and robustness of the model. Larger activity changes mean that the animal has more obvious behavior in a short period of time. Capturing the activity changes of animals in a short period of time is very important for behavioral analysis and event monitoring. It helps to accurately record and analyze the dynamic behavior of animals. In livestock farms, capturing these intense activities helps monitor the health and behavioral changes of animals and provide a basis for further management and decision-making. Therefore, filtering these images into the training data set can improve the model's capture of animal behavior and improve model performance.

[0035] The first marking result includes adding the first mark and not adding the second mark, and the image difference between the image to be processed with the first mark added and the adjacent image is greater than the image difference between the image to be processed without the first mark added and the adjacent image. The content difference between the images to be processed can be measured by the perceptual hash value (pHash), for example, the perceptual hash value of the image to be processed and the adjacent previous image to be processed can be calculated and compared. If the difference exceeds the preset hash value difference threshold, it is determined that the content difference between the image to be processed and the adjacent previous image to be processed is greater than the difference threshold, and the first mark is added to the image to be processed and the adjacent previous image to be processed. This ensures that frames with significant changes in scene and light conditions are captured, and enhances the scene diversity of the data. Of course, it is understandable that other indicators can also be used to measure the content difference between the images to be processed, such as the structural similarity index (SSIM), MAD (Mean Absolute difference, average absolute difference), etc. The detection difficulty of the target in the image to be processed and the degree of motion change can be judged based on the detection results of open set recognition.

[0036] The open set recognition algorithm is relative to the closed set recognition algorithm. In the closed set recognition algorithm, the algorithm knows all the target categories and can only recognize the predetermined targets, while the open set recognition algorithm can not only recognize the targets of known categories, but also the targets of unknown categories. The open set recognition large model can effectively recognize various targets in the image under zero-sample learning. The open set recognition algorithm is used to perform target detection on the image to be processed, and the target detection results in the image to be processed are obtained, including: Inputting the image to be processed into the open set recognition algorithm model to obtain the initial detection result output by the open set recognition algorithm model; The number of detection frames in the initial detection result is reduced based on the confidence of the detection frame and the size of the detection frame to obtain the target detection result in the image to be processed.

[0037] like Figure 2 As shown, when the image to be processed is input into the open set recognition algorithm model, the prompt word of the detection target can be input at the same time, and the open set recognition algorithm model can output the initial detection result. Although the open set recognition algorithm can realize the detection of unknown category targets, its detection accuracy is not high enough, and objects that are not targets in the image may also be detected to generate a detection frame. The initial detection result output by the open set recognition algorithm model includes a detection frame and the confidence of the detection frame. The method provided by the present invention reduces the number of detection frames in the initial detection result based on the confidence of the detection frame in the initial detection result and the size of the detection frame, thereby improving the subsequent processing efficiency and processing accuracy.

[0038] Specifically, the number of detection boxes in the initial detection result is reduced based on the confidence of the detection box and the size of the detection box in the initial detection result, including: Delete the detection boxes whose confidence level in the initial detection results is lower than a preset confidence threshold; The IoU ratios of the remaining detection frames and the reference detection frame are obtained in turn, and the detection frames with IoU ratios higher than the preset IoU ratio threshold are deleted. The reference detection frame is the detection frame with the highest confidence in the initial detection result.

[0039] Through the above method, it is possible to control the output results of the open set recognition algorithm model. By controlling the size of the result box after recognition, the confidence score, the non-maximum suppression of multiple recognition boxes of the same target, etc., the number of target object recognition boxes can be controlled to be slightly larger than the actual box. In this way, in the subsequent manual correction process, the number of operations for adding new marks can be reduced, the number of operations for deleting marks can be slightly increased, and the difficulty of manual labeling can be reduced.

[0040] Specifically, marking the plurality of images to be processed based on the target detection result to obtain a second marking result and a third marking result includes: Determine a second labeling result of the image to be processed based on the number and confidence of the target detection frames in the target detection result of the image to be processed; Performing target segmentation on the image to be processed based on the target detection result to obtain a segmentation mask corresponding to the image to be processed; A third labeling result of the image to be processed is determined based on the segmentation mask of the image to be processed.

[0041] The second marking result includes adding the second mark and not adding the second mark, and the third marking result includes adding the third mark and not adding the third mark. The detection difficulty of the target in the image to be processed with the second mark added is higher than that of the image to be processed without the second mark added; the degree of motion change of the target in the image to be processed with the third mark added is higher than that of the image to be processed without the third mark added.

[0042] Specifically, determining the second labeling result of the image to be processed based on the number and confidence of the target detection frames in the target detection result of the image to be processed includes: A second labeling result of the image to be processed is determined based on the number of target detection frames in the target detection result of the image to be processed, the degree of change in the number of target detection frames in adjacent images of the image to be processed, and the confidence of the target detection frames in the image to be processed.

[0043] like Figure 3 As shown, when the number of target detection frames of the image to be processed is equal to 0 or very small (for example, less than a preset minimum value), a second mark is added to the image to be processed. If the number of target detection frames of the image to be processed is equal to 0 or very small, it means that the target detection in the image to be processed may be difficult. Selecting it can avoid missing potential important information in the video image.

[0044] After obtaining the target detection result of the image to be processed, target segmentation is performed on the image to be processed based on the target detection result to obtain a segmentation mask corresponding to the image to be processed, and the segmentation mask corresponding to the image to be processed reflects the area occupied by the target in the image to be processed. Based on the number of target detection frames in the target detection result of the image to be processed, the degree of change in the number of target detection frames in the adjacent images of the image to be processed, and the confidence of the target detection frame in the image to be processed, the second labeling result of the image to be processed is determined, and it also includes: If the average confidence value of the target detection frame in the image to be processed is lower than a preset average value threshold, a second mark is added to the image to be processed; Based on the number of target detection frames in the image to be processed and the images before and after it, the average value of the detection frames corresponding to the image to be processed is calculated; If the difference between the average values ​​of the detection frames corresponding to two adjacent images to be processed is greater than a preset difference threshold, a second mark is added to the two adjacent images to be processed.

[0045] The images to be processed included in the video image set to be refined can be arranged in order of acquisition time, and adjacent images to be processed are images with continuous acquisition time in the video image set to be refined. Adding a second tag based on the average confidence value of the target detection box in the image to be processed can ensure that the data set contains more challenging detection scenes. The image data with the first tag added can subsequently improve the robustness of the model. Adding a second tag based on the change in the number of target detection boxes in the image to be processed can ensure that scenes with significant changes in the number of targets are captured, reflecting the dynamic behavior of animals and the complexity of the scene, and identifying data where the model is unstable.

[0046] Determining a third labeling result of the image to be processed based on the segmentation mask of the image to be processed includes: The similarity coefficients between the segmentation masks of adjacent images to be processed are obtained, and a third mark is added to two images to be processed whose similarity coefficients are lower than a preset value.

[0047] The similarity coefficients between the segmentation masks of adjacent images to be processed can be calculated using a variety of known similarity calculation methods (such as Jaccard similarity coefficient calculation methods). By obtaining the similarity coefficients between the segmentation masks of adjacent images to be processed, it is possible to capture activity changes at the level of recognition results, exclude image changes caused by non-targets, capture scenes where animal activities change dramatically, improve the diversity and representativeness of data, and capture data covering animal behavior. Based on the first marking result, the second marking result, and the third marking result, the image to be processed is selected, which can be to add the image to be processed with the first marking result added to the first set, and to add the image to be processed with the second marking result and the third marking result added to the second set. The union of the first set and the second set can be directly taken as the data refinement result, or the score calculation can be performed on the image to be processed with the second marking result and / or the third marking result added, and the image in the second set is further selected and then taken into the union with the first set. Specifically, the calculation results based on confidence, number of detection boxes and segmentation masks can be normalized by applying weights to obtain a comprehensive score for each image to be processed, and the images to be processed that exceed the comprehensive score threshold are retained in the second set, so as to effectively separate the key data in the video content due to scene changes, changes in recognition difficulty and changes in animal activities. Figure 2 As shown, after the data selector obtains the data refinement result (equivalent to Figure 2After the "Secondary Selection Image" in the data refinement result, the image to be processed and the target detection result in the image to be processed (i.e. Figure 2 The “rough annotation results” in the figure are submitted to manual review and correction, and finally a training data set that can be used for machine vision algorithms in livestock farm monitoring scenarios is obtained. Figure 4 As shown, compared with the existing full-process manual labeling method, the method provided by the present invention reduces the human participation in the data mining and labeling process, greatly reduces the labor cost, improves the objectivity and standardization of data screening, accelerates the data set construction process, and improves the efficiency of data utilization. In addition, according to the screening method of image similarity and model recognition results, it is possible to mine the parts of the data with large differences and greater algorithm recognition difficulty, and balance the content and difficulty of the data set, which has the advantages of being more objective, stable, and controllable.

[0048] An experiment was conducted using a 12-hour surveillance video of a large-scale beef farm to construct an image dataset for a machine vision algorithm to verify the effectiveness of the method provided by the present invention. The original data contains a total of 670,978 frames of content, such as Figure 5 As shown, according to the frame extraction rule of "30 seconds every 600 seconds, 5 frames per second", the initial sampling image contains 11,400 frames. The data selector made an intelligent selection based on the initial sampling image and retained the content of 1,552 frames. In the selected content of 1,552 frames, a total of 25,000 cows appeared. The traditional data labeling method requires a total of 870 working hours to complete the labeling of individuals and contours. By debugging the open set recognition algorithm dedicated to livestock farms, the data in this example is automatically labeled. For the results of automated labeling, the quality was manually reviewed and corrected, as shown in the figure below. Figure 6 As shown in the figure, the labeling staff no longer need to add outlines and borders to each object in the traditional process, but need to delete, correct, and add completed results. Through the combination of automatic labeling and manual correction, the labor cost was reduced to 65 working hours, and the cost of data set labeling was reduced by 90%.

[0049] The livestock farm video image refining device provided by the present invention is described below. The livestock farm video image refining device described below and the livestock farm video image refining method described above can be referred to each other. Figure 7 As shown, the livestock farm video image refining device provided by the present invention comprises: An image acquisition module 710 is used to acquire a video image set to be refined, wherein the video image set to be refined includes a plurality of images to be processed; A difference calculation module 720 is used to mark the multiple images to be processed based on the content difference between the images to be processed to obtain a first marking result; An open set detection module 730 is used to perform target detection on the image to be processed by an open set recognition algorithm, obtain a target detection result in the image to be processed, and mark a plurality of the images to be processed based on the target detection result to obtain a second marking result and a third marking result, wherein the second marking result reflects the detection difficulty of the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; The selection module 740 is configured to obtain a data refining result of the video image set to be refined based on the first marking result, the second marking result and the third marking result.

[0050] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the livestock farm video image refining method, which includes: obtaining a video image set to be refined, wherein the video image set to be refined includes a plurality of images to be processed; marking the plurality of images to be processed based on the content difference between the images to be processed to obtain a first marking result; performing target detection on the images to be processed by an open set recognition algorithm, obtaining target detection results in the images to be processed, marking the plurality of images to be processed based on the target detection results, obtaining a second marking result and a third marking result, wherein the second marking result reflects the detection difficulty of the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; and obtaining a data refining result of the video image set to be refined based on the first marking result, the second marking result and the third marking result.

[0051] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0052] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the livestock farm video image refining method provided above. The livestock farm video image refining method includes: obtaining a set of video images to be refined, wherein the set of video images to be refined includes multiple images to be processed; marking the multiple images to be processed based on the content difference between the images to be processed to obtain a first marking result; performing target detection on the images to be processed by an open set recognition algorithm to obtain target detection results in the images to be processed, marking the multiple images to be processed based on the target detection results to obtain a second marking result and a third marking result, wherein the second marking result reflects the difficulty of detecting the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; based on the first marking result, the second marking result and the third marking result, a data refining result of the video image set to be refined is obtained.

[0053] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the livestock farm video image refining method provided above, the livestock farm video image refining method comprising: obtaining a set of video images to be refined, the set of video images to be refined including a plurality of images to be processed; marking the plurality of images to be processed based on the content difference between the images to be processed, and obtaining a first marking result; performing target detection on the images to be processed by an open set recognition algorithm, obtaining target detection results in the images to be processed, marking the plurality of images to be processed based on the target detection results, and obtaining a second marking result and a third marking result, the second marking result reflecting the difficulty of detecting the target in the image to be processed, and the third marking result reflecting the degree of motion change of the target in the image to be processed; obtaining a data refining result of the video image set to be refined based on the first marking result, the second marking result and the third marking result.

[0054] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0055] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A livestock farm video image refining method, characterized in that: include: Acquire a video image set to be refined, wherein the video image set to be refined includes a plurality of images to be processed; Marking the plurality of images to be processed based on the content differences between the images to be processed to obtain a first marking result; Performing target detection on the image to be processed by an open set recognition algorithm to obtain a target detection result in the image to be processed, marking a plurality of the images to be processed based on the target detection result to obtain a second marking result and a third marking result, wherein the second marking result reflects the detection difficulty of the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; A data refinement result of the video image set to be refined is obtained based on the first labeling result, the second labeling result and the third labeling result.

2. The livestock farm video image refining method according to claim 1, characterized in that: The step of marking the plurality of images to be processed based on the target detection result to obtain a second marking result and a third marking result includes: The second labeling result of the image to be processed is determined based on the number and confidence of the target detection boxes in the target detection result of the image to be processed.

3. The livestock farm video image refining method according to claim 2, characterized in that: The determining the second labeling result of the image to be processed based on the number and confidence of the target detection frames in the target detection result of the image to be processed includes: The second labeling result of the image to be processed is determined based on the number of target detection frames in the target detection result of the image to be processed, the degree of change in the number of the target detection frames in the adjacent images of the image to be processed, and the confidence of the target detection frames in the image to be processed.

4. The livestock farm video image refining method according to claim 1, characterized in that: The step of marking the plurality of images to be processed based on the target detection result to obtain a second marking result and a third marking result includes: Performing target segmentation on the image to be processed based on the target detection result to obtain a segmentation mask corresponding to the image to be processed; The third labeling result of the image to be processed is determined based on the segmentation mask of the image to be processed.

5. The livestock farm video image refining method according to claim 1, characterized in that: The performing target detection on the image to be processed by using an open set recognition algorithm to obtain a target detection result in the image to be processed includes: Inputting the image to be processed into an open set recognition algorithm model to obtain an initial detection result output by the open set recognition algorithm model; The number of detection frames in the initial detection result is reduced based on the confidence of the detection frames in the initial detection result and the size of the detection frames to obtain the target detection result in the image to be processed.

6. The livestock farm video image refining method according to any one of claims 1 to 5, characterized in that: The step of obtaining a video image set to be refined includes: The original surveillance video is obtained, and frame extraction is performed on the original surveillance video according to a preset frame extraction rule to obtain a plurality of images to be processed, thereby generating the video image set to be refined.

7. A video image refining device for a livestock farm, characterized in that: include: An image acquisition module, used for acquiring a video image set to be refined, wherein the video image set to be refined includes a plurality of images to be processed; A difference calculation module, used for marking a plurality of the images to be processed based on the content difference between the images to be processed to obtain a first marking result; an open set detection module, configured to perform target detection on the image to be processed by using an open set recognition algorithm, obtain a target detection result in the image to be processed, and mark a plurality of the images to be processed based on the target detection result to obtain a second marking result and a third marking result, wherein the second marking result reflects the detection difficulty of the target in the image to be processed, and the third marking result reflects the degree of motion change of the target in the image to be processed; The selection module is used to obtain a data refining result of the video image set to be refined based on the first marking result, the second marking result and the third marking result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the livestock farm video image refining method as described in any one of claims 1-6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the livestock farm video image refining method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the livestock farm video image refining method as described in any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Image processing method and device, electronic equipment, storage medium and computer product

    CN114120068A

  • Video data set identification method and device

    CN117953317A

  • Automatic construction method of target detection class data set and related equipment thereof

    CN117975198A

  • Motion detection method and system, electronic equipment and storage medium

    CN118196136A