Target detection method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of image processing, and provides a target detection method and device, equipment, a storage medium and a program product. The target detection method comprises the steps of determining whether a suspected abnormal object exists or not according to a first video frame sequence of a to-be-detected video; under the condition that the suspected abnormal object exists, a to-be-detected sub-video is intercepted from the to-be-detected video frame sequence according to the first video frame sequence, and a time period corresponding to the first video frame sequence partially coincides with a time period corresponding to the to-be-detected sub-video; and carrying out classification processing on the to-be-detected sub-videos to obtain an anomaly identification result of the suspected abnormal object. According to the invention, under the condition that the existence of the suspected abnormal object is determined, the video clips with the suspected abnormal object are classified to obtain the abnormal recognition result of the suspected abnormal object, so that the recognition accuracy of the abnormal object can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a target detection method and apparatus, equipment, storage medium and program product. Background Art

[0002] Object detection can find important and key objects in an image, and detecting and identifying abnormal objects can find unknown abnormal situations and take effective measures in time to avoid more adverse effects. Therefore, detecting and identifying abnormal objects has become an important branch of the field of object detection. Summary of the invention

[0003] The purpose of the present disclosure is to provide a target detection method and apparatus, equipment, storage medium and program product, thereby improving the recognition accuracy of abnormal objects at least to a certain extent.

[0004] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0005] According to a first aspect of the present disclosure, a target detection method is provided, comprising: determining whether there is a suspected abnormal object based on a first video frame sequence of a video to be detected; in the case where the suspected abnormal object exists, cutting out a sub-video to be detected from the video frame sequence to be detected based on the first video frame sequence, wherein a time period corresponding to the first video frame sequence partially overlaps with a time period corresponding to the sub-video to be detected; and classifying the sub-video to be detected to obtain an abnormality recognition result of the suspected abnormal object.

[0006] According to the second aspect of the present disclosure, a target detection device is provided, comprising: a suspected abnormal object determination module, configured to determine whether there is a suspected abnormal object based on a first video frame sequence of a video to be detected; a video interception module, configured to intercept a sub-video to be detected from the video frame sequence to be detected based on the first video frame sequence when the suspected abnormal object exists, wherein the time period corresponding to the first video frame sequence partially overlaps with the time period corresponding to the sub-video to be detected; a classification module, configured to classify the sub-video to be detected to obtain an abnormal identification result of the suspected abnormal object. According to the third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the target detection method as described in the first aspect of the above-mentioned embodiment.

[0007] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the target detection method as described in the first aspect is implemented.

[0008] According to a fifth aspect of the present disclosure, a computer program product containing instructions is provided, which, when executed on a computer, enables the computer to execute the steps of the target detection method as described in the first aspect. It can be seen from the above technical solutions that the target detection method, target detection device, and computer-readable storage medium, computer program product, and electronic device implementing the target detection method in the exemplary embodiments of the present disclosure have at least the following advantages and positive effects:

[0009] In the technical solutions provided by some embodiments of the present disclosure, when it is determined that there is a suspected abnormal object according to the first video frame sequence of the video to be detected, the sub-video to be detected is intercepted from the video to be detected according to the first video frame sequence, and then the sub-video to be detected containing the suspected abnormal object is classified and processed, and finally the abnormal recognition result of the suspected abnormal object is obtained. In this way, on the one hand, when it is determined that there is a suspected abnormal object, the present disclosure intercepts the sub-video, and further judges the abnormal state of the suspected abnormal object according to the classification result of the sub-video. Since the video can reflect the continuous state of the object over a period of time, the probability of false detection can be reduced and the accuracy of abnormal object detection can be improved; on the other hand, the present disclosure first determines that there is a suspected abnormal object through the first video frame sequence, then intercepts the sub-video, and then performs video classification processing on the sub-video to obtain the abnormal recognition result, instead of directly classifying the entire video, so the detection efficiency of abnormal objects can be improved.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0012] Figure 1 A schematic diagram showing an exemplary system architecture to which embodiments of the present disclosure may be applied;

[0013] Figure 2 A schematic diagram showing a flow chart of a target detection method in an exemplary embodiment of the present disclosure;

[0014] Figure 3 A flow chart showing a method for determining whether there is a suspected abnormal object in an exemplary embodiment of the present disclosure;

[0015] Figure 4 A schematic diagram showing a flow chart of a method for determining a sub-video to be detected in an exemplary embodiment of the present disclosure;

[0016] Figure 5 A schematic flow chart showing a method for obtaining a sub-video to be detected according to a sub-video to be cut in an exemplary embodiment of the present disclosure is shown;

[0017] Figure 6 A schematic flow chart showing a method for classifying a sub-video to be detected in an exemplary embodiment of the present disclosure;

[0018] Figure 7 A schematic diagram showing a flow chart of a training process of a video classification model in an exemplary embodiment of the present disclosure;

[0019] Figure 8 A flow chart showing a method for performing video acquisition based on an automatic feeder to identify abnormal animals in an exemplary embodiment of the present disclosure;

[0020] Fig. 9 A schematic diagram showing a flow chart of a training process of a target detection model in an exemplary embodiment of the present disclosure;

[0021] Fig.10 A schematic flow chart showing a method for training a video classification model and a method for training an object detection model in an exemplary embodiment of the present disclosure is shown;

[0022] Fig.11 A schematic flow chart showing a method for abnormal animal detection in an exemplary embodiment of the present disclosure;

[0023] Fig.12 A schematic diagram showing an abnormal animal prompt information pushed to farmers in an exemplary embodiment of the present disclosure;

[0024] Fig.13 A schematic diagram showing the composition of a target detection device in an exemplary embodiment of the present disclosure;

[0025] Fig.14 A schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0026] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0027] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0028] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0029] Object detection can find important and key objects in an image, and detecting and identifying abnormal objects can find unknown abnormal situations and take effective measures in time to avoid more adverse effects. Therefore, detecting and identifying abnormal objects has become an important branch of the field of object detection.

[0030] In a related technology, a thermal infrared camera is used to determine whether the temperature of a living target is normal, and at the same time, target detection technology is used to detect the position of each living target in the current RGB (red, green, and blue color channels) image to determine whether the living target is in motion. Finally, based on the temperature and motion state of the living target, it is determined whether there are any abnormal target objects among the living targets.

[0031] However, this method requires the use of thermal infrared information and RGB information to judge abnormal objects. The equipment cost of thermal infrared sensors is relatively high, which increases the equipment cost in actual production. Secondly, when using target detection to detect abnormal objects, since the training phase is based on single-frame images, the timing information is lost, which can easily cause the detector to confuse positive and negative samples in the inference phase, resulting in a decrease in the accuracy of abnormal object detection.

[0032] In another related technology, the RGB image information is obtained by using the installed USB (Universal Serial Bus) camera device, and then the abnormal object is identified by the relevant features of the abnormal object in the image information. For example, for abnormal chickens, the abnormal chickens are judged by the abnormal chicken related features such as the comb color, comb size and number of eggs in the image. The whole device is an integrated inspection device, and the algorithm is installed on the robot, and the image acquisition and abnormal object detection are performed through the information exchange between the motion chassis and the algorithm.

[0033] However, this method is also based on the detection of abnormal objects based on a single-frame image, and it also loses the timing information, so the recognition accuracy is low. Secondly, this method needs to redefine different abnormal object features for different abnormal objects, and is not applicable to all abnormal objects. For example, the above-mentioned features related to abnormal chickens cannot be applied to the detection of abnormal pigs. Therefore, the scalability of this method is poor. Furthermore, this method is equipped with a mobile inspection platform. In addition to being affected by the detection accuracy of the algorithm, it is also limited by charging restrictions, the fault tolerance of path planning, road obstacle avoidance and other technologies, and the technical maintenance cost is also high, which is not suitable for promotion and use in practice.

[0034] Based on this, the present disclosure first determines whether there is a suspected abnormal object based on multiple frames of images through the first video frame sequence. When it is determined that there is a suspected abnormal object, the video segment containing the suspected abnormal object is intercepted to obtain a sub-video segment, and then the sub-video segment is classified and processed. The abnormal identification result of the suspected abnormal object is determined based on the classification processing result of the sub-video segment. Through the first video frame sequence and the sub-video segment, it is possible to determine whether the object is abnormal based on the continuous state of the object within a period of time, thereby reducing the false detection probability of abnormal object detection and improving the accuracy of abnormal object detection.

[0035] The target detection method and device provided by the present disclosure can be applied to Figure 1 In the system architecture of the exemplary application environment shown.

[0036] like Figure 1As shown, the system architecture 100 may include a terminal device 110 and a server 120. The terminal device 110 may be a terminal device such as a smart phone, a tablet computer, a desktop computer, a laptop computer, a smart wearable device, etc. The server 120 generally refers to a background system that provides services related to the target detection and recognition method in this exemplary embodiment, and may be a server or a cluster formed by multiple servers. The terminal 110 and the server 120 may be connected via a wired or wireless communication link to exchange data.

[0037] In an exemplary embodiment, the above-mentioned target detection method can be performed by the terminal device 110. Accordingly, the target detection device can be set in the terminal device 110 to implement the corresponding module functions. For example, the user uses the terminal device 110 to obtain the video to be detected, and the terminal device 110 determines whether there is a suspected abnormal object based on the first video frame sequence of the video to be detected. If it is determined that there is a suspected abnormal object, a sub-video is cut out, and the abnormal recognition result of the suspected abnormal object is determined based on the classification processing result of the sub-video.

[0038] In an exemplary embodiment, the above-mentioned target detection method can be performed by the server 120. Accordingly, the target detection device can be set in the server 120 to implement the corresponding module functions. For example, the server 120 obtains the video to be detected, determines whether there is a suspected abnormal object according to the first video frame sequence of the video to be detected, and if it is determined that there is a suspected abnormal object, cuts out a sub-video, and determines the abnormal recognition result of the suspected abnormal object according to the classification processing result of the sub-video.

[0039] In an exemplary embodiment, the above-mentioned target detection method can also be performed jointly by the terminal device 110 and the server 120. Accordingly, some modules of the target detection device can be set in the terminal device 110, and some modules can be set in the server 120 to realize the corresponding module functions. For example, the terminal device 110 can determine whether there is a suspected abnormal object based on the first video frame sequence of the video to be detected. When it is determined that there is a suspected abnormal object, the sub-video to be detected is intercepted and sent to the server 120. The server 120 classifies the sub-video to be detected, obtains the abnormal recognition result of the suspected abnormal object, and sends the abnormal recognition result to the terminal device 110.

[0040] It should be understood that Figure 1The number of terminal devices and servers in the example is only illustrative. Any number of terminal devices and servers may be provided as required. For example, the server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0041] It is easy for those skilled in the art to understand that the above application scenarios are only used as examples and are not limited to the present exemplary embodiment.

[0042] Figure 2 FIG. 1 is a flow chart showing a target detection method in an exemplary embodiment of the present disclosure. Figure 2 , the method comprising:

[0043] Step S210, determining whether there is a suspected abnormal object according to the first video frame sequence of the video to be detected;

[0044] Step S220: if the suspected abnormal object exists, extracting a sub-video to be detected from the video to be detected according to the first video frame sequence, wherein a time period corresponding to the first video frame sequence partially overlaps with a time period corresponding to the sub-video to be detected;

[0045] Step S230: classify the sub-video to be detected to obtain an abnormality recognition result of the suspected abnormal object.

[0046] exist Figure 2 In the technical solution provided by the illustrated embodiment, when it is determined that there is a suspected abnormal object according to the first video frame sequence of the video to be detected, the sub-video to be detected is intercepted from the video to be detected according to the first video frame sequence, and then the sub-video to be detected containing the suspected abnormal object is classified and processed, and finally the abnormal recognition result of the suspected abnormal object is obtained. In this way, on the one hand, the present disclosure intercepts the sub-video when it is determined that there is a suspected abnormal object, and further judges the abnormal state of the suspected abnormal object according to the classification result of the sub-video. Since the video can reflect the continuous state of the object over a period of time, the probability of false detection can be reduced and the accuracy of abnormal object detection can be improved; on the other hand, the present disclosure first determines that there is a suspected abnormal object through the first video frame sequence, then intercepts the sub-video, and then performs video classification processing on the sub-video to obtain the abnormal recognition result, instead of directly classifying the entire video, so the detection efficiency of abnormal objects can be improved.

[0047] The following Figure 2The specific implementation methods of each step in the embodiment shown are described in detail:

[0048] In step S210, it is determined whether there is a suspected abnormal object according to a first video frame sequence of the video to be detected.

[0049] In an exemplary embodiment, the video to be detected in the present disclosure includes one or more objects to be identified. The objects to be identified may include any biological objects, such as chickens, ducks, geese, etc.

[0050] Among them, abnormal status can be customized according to needs. For example, for chickens, abnormal chickens can include sick chickens, dead chickens, and for ducks, they can be sick ducks, dead ducks, etc.

[0051] In an exemplary embodiment, a method for determining the first video frame sequence includes: when a candidate abnormal object is detected in a video frame of a video to be detected, determining the abnormal time at which the candidate abnormal object is detected; and determining the first video frame sequence based on a first video frame in the video frame of the video to be detected that is later than the abnormal time.

[0052] For example, in the present disclosure, abnormal object detection can be performed in a single frame image corresponding to a video frame of a video to be detected according to a target detection model, and the abnormal object detected by the target detection model is determined as a candidate abnormal object. When a candidate abnormal object is detected, the time corresponding to the video frame in which the candidate abnormal object is detected is determined as the abnormal time, and a first video frame sequence is determined based on the first video frame in the video frame of the video to be detected that is later than the abnormal time.

[0053] For example, there are 100 video frames in the video to be detected, and these 100 video frames are arranged in chronological order. According to the chronological order of the video frames, target detection can be performed sequentially in the video frames of the video to be detected to determine whether there are candidate abnormal objects. For example, if an abnormal object is detected in the 30th video frame, the abnormal object detected in the 30th video frame is determined as the candidate abnormal object, and the 30th video frame is the video frame in which the candidate abnormal object is detected. The time corresponding to the 30th video frame is the abnormal time. Because the 100 video frames are arranged in chronological order, the first video frame in the 100 video frames that is later than the abnormal time includes the 31st frame to the 100th frame, then the 31st frame to the 100th frame can be understood as the first video frame, and the video frame sequence composed of the 31st frame to the 100th frame can be understood as the first video frame sequence.

[0054] For example, Figure 3 A flow chart showing a method for determining whether a suspected abnormal object exists in an exemplary embodiment of the present disclosure is shown, with reference to Figure 3, the method may include steps S310 to S320, wherein:

[0055] In step S310, target tracking is performed on candidate abnormal objects in a first video frame sequence.

[0056] In an exemplary embodiment, multiple candidate abnormal objects may be detected in the video frame of the video to be detected, so the candidate abnormal objects can be tracked according to a multi-target tracking model, wherein the multi-target tracking model may include any machine learning model capable of multi-target tracking, such as a bytetrack tracking model, a target tracking model based on a convolutional neural network, and the like.

[0057] In the present disclosure, a multi-target tracking model trained according to other data sets can be directly used to perform multi-target tracking on candidate abnormal objects. Of course, an object tracking data set can also be generated based on a video containing the object to be detected, and the existing trained multi-target tracking model can be fine-tuned based on the object tracking data set to obtain a multi-target tracking model suitable for tracking the object to be detected. Taking the case where the object to be detected is chickens, the chicken video can be annotated to generate an abnormal chicken video tracking data set, and the multi-target tracking model trained according to other data sets can be fine-tuned based on the tracking data set to obtain a multi-target tracking model for abnormal chicken tracking. This exemplary embodiment does not make any special limitations on this.

[0058] Continuing with the case where the object to be detected is a chicken, the abnormal chicken is a weak dead chicken, and the multi-target tracking model is a bytetrack tracking model, for the current Nth video frame, target detection is performed to obtain a target detection result rectangular box of a weak dead chicken, which can characterize the position of the weak dead chicken in the detection video image frame by the center position coordinates, width and height of the rectangular box. According to the target detection result, the target detection result is used as the position of the target to be tracked in the initial frame, and bytetrack is used to perform multi-target tracking to obtain track_list (i.e., tracking list), which is used to store the tracking identifier of each frame. Bytytrack needs to initialize the tracker and will track the detected target in each frame of the sequence. The bytetrack algorithm uses a simple and efficient association strategy, that is, based on the IOU (Intersection over Union) tracking strategy, to associate the target detected in the current frame with the target already being tracked. The algorithm calculates the IOU between each newly detected target and the existing tracked target, and updates the tracking status based on this information. Bytetrack maintains a state for each tracked target, such as "active" or "lost". When a target is not successfully associated for several consecutive frames, it may be marked as "lost", and when it is redetected, the tracking state of the target can be reactivated. The bytetrack algorithm assigns a unique ID (Identity document, i.e. the tracking identifier mentioned above) to each tracked target. Even if the target disappears and reappears in the field of view, the algorithm can maintain the consistency of the ID, thereby achieving effective tracking of the target. Finally, bytetrack outputs the ID of each target and the corresponding bounding box position, providing real-time information of all tracked targets in each frame.

[0059] For example, the video frame in which the candidate abnormal object is detected and the first video frame sequence can be input into the multi-target tracking model. The video frame in which the candidate abnormal object is detected is the initial frame of the multi-target tracking model, and the video frames in the first video frame sequence can be understood as the tracking frames of the multi-target tracking model. According to the position of the candidate abnormal object in the initial frame, it can be determined which candidate abnormal object to be tracked is, so that the candidate abnormal object can be tracked in the tracking frame.

[0060] During the target tracking process, the multi-target tracking model generates the candidate position of the current frame based on the position of the candidate abnormal object in the previous frame, such as generating a rectangular box of the candidate position through Gaussian distribution near the position of the previous frame, and then calculating the intersection and union ratio of the rectangular box of the previous frame position and the rectangular box of the candidate position of the current frame, and selecting the candidate position with the largest intersection and union ratio and greater than the preset value as the position of the candidate abnormal object in the current frame, thereby realizing target tracking of the candidate abnormal object.

[0061] Next, in step S320, based on the situation that the candidate abnormal object is tracked in the first video frame sequence, it is determined whether the candidate abnormal object is a suspected abnormal object.

[0062] Exemplarily, a specific implementation of step S320 may include: when the number of times the candidate abnormal object is tracked in the first video frame sequence reaches a first preset value, determining that the candidate abnormal object is the suspected abnormal object.

[0063] Exemplarily, another specific implementation of step S320 may include: when the duration for which the candidate abnormal object is tracked in the first video frame sequence reaches a first preset duration, determining that the candidate abnormal object is the suspected abnormal object.

[0064] For example, when the duration during which the candidate abnormal object is tracked in the first video frame sequence reaches a first preset duration and / or when the number of times the candidate abnormal object is tracked in the first video frame sequence reaches a first preset value, tracking of the candidate abnormal object can be stopped, and the candidate abnormal object can be determined to be a suspected abnormal object, that is, it is determined that a suspected abnormal object exists.

[0065] For example, it is possible to determine whether the candidate abnormal object is the same according to the ID assigned to each candidate abnormal object by the multi-target tracking model, and to determine whether the number of times the same ID appears in the tracking frame reaches a first preset value or reaches a first preset duration.

[0066] Among them, the first preset value and the first preset duration can be determined according to experience or demand, and the values ​​of the first preset value and the first preset duration may also be related to the sequence length of the tracking video. If the sequence length of the tracking video is long, the first preset value or the first preset duration may be large; if the sequence length of the tracking video is short, the first preset value or the first preset duration may be short. This exemplary embodiment does not make any special limitation on this.

[0067] Next, continue to refer to Figure 2 In step S220, when the suspected abnormal object exists, a sub-video to be detected is cut out from the video to be detected according to the first video frame sequence.

[0068] In an exemplary implementation, the time period corresponding to the first video frame sequence partially overlaps with the time period corresponding to the sub-video to be detected.

[0069] For example, the sub-video to be detected may be a video segment in the first video frame sequence. Figure 4 A schematic diagram showing a flow chart of a method for determining a sub-video to be detected in an exemplary embodiment of the present disclosure. Figure 4, the method may include steps S410 to S430. Wherein:

[0070] In step S410, an initial video frame is determined from the video frames of the video to be detected according to the video frame in which the candidate abnormal object is detected.

[0071] Continuing with the example of 100 video frames in the above-mentioned video to be detected, if an abnormal object is detected in the 30th video frame, the 30th video frame can be understood as the initial video frame.

[0072] In step S420, based on the video frames in which the candidate abnormal object is tracked a number of times reaching a first preset value in the first video frame sequence, a cutoff video frame is determined from the video frames of the video to be detected.

[0073] Continuing with the example of 100 video frames in the above-mentioned video to be detected, and detecting an abnormal object in the 30th video frame, if the number of times the candidate abnormal object is tracked reaches a first preset value when tracking to the 50th frame, the 50th frame can be understood as the cutoff video frame.

[0074] In step S430, video frames from the initial video frame to the cutoff video frame are intercepted from the video to be detected to obtain the sub-video to be detected.

[0075] Continuing with the example where the video to be detected has 100 video frames, an abnormal object is detected in the 30th video frame, and the number of times the abnormal object detected in the 50th frame is tracked reaches a first preset value, then the video segment composed of the video frames from the 30th frame to the 50th frame can be the sub-video to be detected.

[0076] In another exemplary implementation, the video frame captured from the video to be detected may be cropped to obtain the sub-video to be detected. Based on this, another specific implementation of step S430 may refer to Figure 5 shown. Figure 5 A flow chart of a method for obtaining a sub-video to be detected based on a sub-video to be cut in an exemplary embodiment of the present disclosure is shown. Figure 5 , the method may include steps S510 to S540. Wherein:

[0077] In step S510, video frames from the initial video frame to the cutoff video frame are intercepted from the video to be detected to determine a sub-video to be cropped.

[0078] For example, a video segment directly cut out from the video to be detected can be used as the sub-video to be cut. For example, the video segment consisting of the 30th frame to the 50th frame can be used as the sub-video to be cut.

[0079] In step S520, for an initial video frame in the sub-video to be cropped, the initial video frame is cropped according to the detected position of the candidate abnormal object in the initial video frame.

[0080] For example, since target detection has been performed in the initial video frame, the position of the candidate abnormal object in the initial frame can be determined according to the result of the target detection, so the initial video frame can be cropped according to the position of the candidate abnormal object in the initial video frame. For example, taking the position of the candidate abnormal object in the initial video frame represented by a rectangular frame as an example, the central coordinates of the rectangular frame representing the position of the candidate abnormal object in the initial video frame can be used as the center point, and the image area within the range of the preset distance value from the center point can be retained, and other image areas can be cropped, so that the size of the cropped video frame is smaller than the size of the video frame before cropping, thereby realizing the cropping of the initial video frame.

[0081] In step S530, for a second video frame in the sub-video to be cropped, the second video frame is cropped according to the tracked position of the candidate abnormal object in the second video frame.

[0082] Wherein, the second video frame includes other video frames except the initial video frame in the sub-video to be cut. Because the candidate abnormal object is tracked in the second video frame, the second video frame can be cut according to the position of the tracked candidate abnormal object. For example, the center point of the rectangular frame representing the position of the tracked candidate abnormal object can be used as the center, and the image area within the range of the preset distance value from the center point is retained, and other image areas are cut off, so that the size of the cut video frame is smaller than the size of the video frame before cutting.

[0083] Exemplarily, a specific implementation of step S530 may include: determining a first target video frame based on a second video frame in which the candidate abnormal object is tracked, and determining a second target video frame based on a second video frame in which the candidate abnormal object is not tracked; cropping the first target video frame based on the position of the tracked candidate abnormal object in the first target video frame; determining a reference video frame based on a first target video frame that is before the second target video frame and closest to the second target video frame; and cropping the second target video frame based on the position of the tracked candidate abnormal object in the reference video frame.

[0084] Taking the video to be detected in the disclosure as a video of a certain group as an example, in the group, a certain abnormal object may be blocked by other objects and then reappear, so when tracking, there will be a situation where the candidate abnormal object tracking is lost and then reappears. For example, the abnormal object is detected in the 30th frame, and the candidate abnormal object is also tracked from the 31st frame to the 34th frame, but the candidate abnormal object is blocked by the normal object in the 35th frame, resulting in loss, and reappearing in the 36th frame. For the 35th frame, when cutting, the position of the candidate abnormal object tracked in the 34th frame can be referenced for cutting, such as the position tracked in the 34th frame can be used as the position tracked in the 35th frame, and for other video frames that track the candidate abnormal object, the video screen is directly cut according to the position of the candidate abnormal object tracked.

[0085] Next, in step S540, the sub-video to be detected is obtained according to the sub-video to be cropped after the video frame is cropped.

[0086] For example, after the video frames in the sub-video to be cropped are cropped according to the above method, the sub-video to be cropped after the video frames are cropped can be used as the sub-video to be detected. That is, the video frames in the sub-video to be detected are the video frames obtained after the cropping according to the above steps S520 and S530.

[0087] Through the above steps S510 to S540, when the sub-video is captured, because the detection and tracking has been performed, the original video image frame can be cropped according to the detection and tracking results, such as cropping a fixed size position with the center position coordinates of the abnormal chicken rectangular frame detected and tracked as the center, so as to obtain the sub-video to be detected. In this way, since the detection and tracking provides the prior knowledge of the abnormal object, after cropping, the area to be identified becomes smaller, the amount of calculation is smaller, and the efficiency of the subsequent classification processing of the sub-video to be detected can be improved without losing the recognition accuracy, thereby improving the efficiency of target detection.

[0088] Of course, the sub-video to be detected can be obtained by other methods, such as directly using the first video frame sequence as the sub-video to be detected, or sequentially intercepting a preset number of video frames from the initial video frame in the video to be detected to obtain the sub-video to be detected, which is not particularly limited in this exemplary embodiment.

[0089] Continue to refer Figure 2 Next, in step S230, the sub-video to be detected is classified to obtain an abnormality recognition result of the suspected abnormal object.

[0090] For example, Figure 6 FIG. 1 is a flow chart showing a method for classifying a sub-video to be detected in an exemplary embodiment of the present disclosure. Figure 6 , the method may include steps S610 to S620. Wherein:

[0091] In step S610, the sub-video to be detected is input into a pre-trained video classification model to output the video category of the sub-video to be detected.

[0092] In an exemplary embodiment, the candidate abnormal subjects in the present disclosure include candidate abnormal animals. Figure 7 A flowchart of a video classification model training process in an exemplary embodiment of the present disclosure is shown. Figure 7 , the method may include steps S710 to S730. Wherein:

[0093] In step S710, in response to the start-up of the animal feeding device, an image acquisition device disposed on the feeding device is turned on to record an animal video to obtain an animal video data set.

[0094] Taking group-raising animals as an example, when the animals are raised in a group, the animals can be fed through an animal feeding device. Based on this, when the animal feeding device is started, animal videos can be collected to obtain an animal video data set.

[0095] The group breeding method in the present disclosure may include cage breeding, the animal feeding device may include an automatic feeder, and the image acquisition device may include an RGB camera. For example, the animals in the cages can be fed by the automatic feeder, and the automatic feeder can be braked by an acceleration sensor. One automatic feeder corresponds to one acceleration sensor, and one row of cages corresponds to one RGB camera. The RGB camera is installed on the automatic feeder. In this way, the RGB camera can move with the automatic feeder to shoot videos of the animals in the corresponding row of cages.

[0096] When the automatic feeder is started, the component of acceleration in the direction of motion will have a steep slope change. Therefore, the component threshold of acceleration in the direction of motion can be set to determine whether the automatic feeder is started. For example, when the component of acceleration in the direction of motion is greater than or equal to the first acceleration threshold, it is determined that the automatic feeder is started. In response to the automatic feeder starting, the camera set above the feeder can be turned on to enable the camera to record the animal video during feeding.

[0097] In an exemplary embodiment, when the feeder stops, the camera installed on the feeder can be turned off or put in standby mode, so that the camera stops recording, and then the video of the animal during feeding taken by the camera is obtained. Similarly, when the acceleration indicated by the acceleration sensor corresponding to the feeder decreases to less than or equal to the second acceleration threshold, it can be determined that the automatic feeder stops, and then the camera is controlled to stop recording the video.

[0098] In another exemplary embodiment, when the abnormal objects in the present disclosure include abnormal objects in caged animals, the running time of the automatic feeder from one side (head) to the other side (tail) of the cage can be determined according to the length of a row of cages corresponding to the camera and the running speed of the automatic feeder, and then, after the camera is started, the timing starts, and when the time reaches the running time, the camera is turned off or put into standby mode so that the camera stops recording, thereby obtaining a video of the caged animals taken by the camera during feeding.

[0099] Of course, for the free-range method, the free-range animals can also be fed through an automatic feeding device. Similarly, when the animal feeding device is started, the free-range animal video can be collected to obtain an animal video data set.

[0100] In an exemplary embodiment, an animal video dataset may be obtained based on animal videos recorded by a camera during multiple feedings.

[0101] In step S720, the animal videos in the animal video data set are segmented to obtain an animal sub-video data set, and the animal sub-videos in the animal sub-video data set are annotated with video categories to obtain an animal sub-video label data set.

[0102] Exemplarily, before segmenting the animal videos in the animal video dataset, the picture area can be cropped first to retain the video picture information of the target area, thereby obtaining a target animal video dataset, wherein the target area can include the area with the largest animal proportion, thereby avoiding interference of surrounding background information on abnormal object recognition.

[0103] For example, the animal video screen recorded by the camera can be divided into a preset number of equal parts, and then the area with the largest animal proportion is taken as the target area, and the other areas are cut to generate a target animal video data set. For example, the animal video screen recorded by the camera can be divided into three equal parts, where most of the animals are located in the middle area, then the middle area is taken as the target area, and the rest of the area is cut off to obtain a target animal video data set.

[0104] Among them, the preset number can be determined based on experience or after analyzing a large number of animal videos. For example, by analyzing the picture taken by the camera, it can be divided into 3 equal parts, 5 equal parts, 7 equal parts, etc., and then statistics are counted under which division method, the video picture that contains the most animals and the least other content can be cropped out from more video frames, so as to obtain the preset number and the method for determining the target area corresponding to the preset number.

[0105] For example, taking 3 equal parts and 5 equal parts as an example, we analyzed 1,000 animal images taken by cameras and found that the middle area of ​​3 equal parts can contain animals to the greatest extent in 900 animal images and contain other content to the least extent, and the areas corresponding to the middle 3 parts of 5 equal parts can contain animals to the greatest extent in 800 animal images and contain other content to the least extent. Therefore, the animal images in the animal video dataset can be divided into 3 equal parts and the middle area can be taken as the target area. The video images of the target area can be retained, and the images of other areas can be cropped to obtain the target animal video dataset.

[0106] By cropping the video footage captured by the camera, we can identify the footage containing animals. In subsequent processing, we can avoid interference from excessive background information in the identification of abnormal animals, reduce the computational cost of subsequent analysis and reasoning, and improve recognition accuracy.

[0107] Of course, the animal video images captured by the camera may not be cropped according to needs or actual conditions, and this exemplary embodiment does not specifically limit this.

[0108] After obtaining the target animal video dataset, each target animal video in the target animal video dataset may be segmented to obtain an animal sub-video dataset.

[0109] In an exemplary implementation, each target animal video in the target animal video dataset may be segmented according to a preset length to obtain a plurality of animal sub-videos.

[0110] For example, the animal video recorded by the camera can be sampled according to the first preset frame rate to obtain an animal video frame sequence, and every N frames in the animal video frame sequence can be taken as an animal sub-video, so as to divide the target animal video data set into multiple animal sub-videos. It is also possible to take an animal video within a T time length as an animal sub-video, such as taking an animal sub-video every T time length, so as to divide the target animal video in the target animal video data set into multiple animal sub-videos. The division results of the target animal videos in the target animal video data set are summarized to obtain an animal sub-video data set.

[0111] In an exemplary implementation, for the animal sub-videos in the animal sub-video dataset, the video categories of the animal sub-videos are annotated according to whether there are abnormal animals in the animal sub-videos to obtain the animal sub-video category label dataset.

[0112] The video category includes a first video category and a second video category. The first video category indicates that there are abnormal animals in the animal sub-video, and the second video category indicates that there are no abnormal animals in the animal sub-video.

[0113] In another exemplary implementation, abnormal animals may also be marked in the animal sub-video frame sequence corresponding to the animal sub-video to obtain an animal sub-video label dataset.

[0114] For example, each animal sub-video may have two labels, one of which is whether the category of the animal sub-video is the first video category or the second video category mentioned above, and the other label is the position of the abnormal animal included in each sub-video frame in the animal sub-video, which can be represented by a rectangular box.

[0115] In an exemplary embodiment, for each animal sub-video, video frames can be sampled according to a second preset frame rate to obtain an animal sub-video frame sequence. The second preset frame rate and the first preset frame rate can be the same or different. Of course, sampling may not be performed for each animal sub-video, and this exemplary embodiment does not specifically limit this.

[0116] Each video frame in the animal sub-video frame sequence can be named and saved according to its timing information and the information of the animal sub-video in which it is located. For example, the first video frame of the first animal sub-video can be named 0101, the third video frame of the first animal sub-video can be named 0103, the second video frame of the second animal sub-video can be named 0202, etc. In this way, an animal sub-video signature data set containing timing information can be obtained, and when the animal sub-video is read subsequently, it can also be read based on the timing information according to the naming method.

[0117] In step S730, supervised learning training is performed on the video classification model to be trained according to the animal sub-video label data set to obtain a trained video classification model.

[0118] In an exemplary embodiment, the video classification model in the present disclosure may include any machine learning model that can be used for video classification, such as a slow-fast video classification model, a video classification model based on a convolutional neural network, etc., which is not particularly limited in this exemplary embodiment.

[0119] For example, the animal sub-video data in the animal sub-video label data set can be divided into a training set and a test set, and the video classification model to be trained is trained through supervised learning based on the training set. Then, the trained video classification model is tested based on the test set. When model evaluation indicators, such as accuracy and recall rate, do not meet the indicator threshold, the training set and the test set can be re-divided and re-trained until the model evaluation indicators meet the indicator threshold during the test. In this way, the training is terminated, thereby obtaining the trained video classification model.

[0120] In an exemplary embodiment, during training, the ratio between the number of first videos of animal sub-videos with abnormal animals and the number of second videos of animal sub-videos without abnormal animals can meet a preset condition, which makes the difference in number between the two smaller. For example, the ratio of the number of first videos to the number of second videos can be 1:1, so that the positive samples with abnormal animals and the negative samples without abnormal animals are balanced, avoiding the problem of poor model performance due to imbalance between positive and negative samples.

[0121] In the real world, the number of videos with abnormal objects is definitely much smaller than the number of videos without abnormal objects. This is also the primary problem that needs to be overcome when using video classification methods for abnormal object recognition, that is, how to obtain enough videos with abnormal objects for training video classification models.

[0122] In the present disclosure, for the detection of abnormal animals, firstly, by arranging a camera on an automatic feeder, the camera is automatically controlled to record the video of animals during feeding by starting and stopping the automatic feeder, thereby improving the efficiency of collecting animal videos, and obtaining enough videos of animal activities in the real world. Secondly, in the present disclosure, considering that abnormal animals may exist in different video segments of animal videos, the number of videos containing abnormal animals can be expanded by segmenting the collected real animal videos. For example, in a video, different abnormal animals may exist at the 2nd second, the 6th second, and the 10th second respectively. By segmenting them, 3 video clips containing abnormal animals can be obtained, thereby effectively solving the problem of imbalance in the number of videos of abnormal objects and normal objects. Furthermore, a video classification model can be trained based on a large number of abnormal object video clips and normal object video clips, so that the video classification task can be applied to abnormal object recognition, so as to determine the abnormal object according to the continuous state of the object over a period of time based on the strong spatiotemporal information of the video, thereby improving the accuracy of abnormal object recognition.

[0123] Exemplarily, after obtaining the trained video classification model, when performing abnormal object detection, the sub-video to be detected can be input into the trained video classification model. The video classification model can extract features of the objects in the sub-video to be detected, and thus determine the video category of the sub-video to be detected based on the extracted features.

[0124] Take the trained video classification model as the above slow-fast video classification model and the sub-video to be detected as the sub-video within the T period in the video to be detected as an example. The slow-fast video classification model includes two paths, one for capturing fast actions in the video (Fast path) and the other for capturing slow actions in the video (Slow path) to extract fast frames and slow frames. Among them, the slow path usually samples fewer frames, such as sampling one frame every 16 frames, to capture a wider context and slow actions, and the fast path samples more frames, such as sampling one frame every 4 frames, to capture rapid motion changes. The pre-trained slow-fast network is used to extract features from the frames of each path, and the video frames are converted into high-dimensional feature representations. These feature representations can capture the action information in the video. Combining the features from the slow and fast paths, the slow-fast model classifies the video clips through its deep neural network structure and identifies the behaviors in the video. The final classifier layer outputs the probability distribution of each behavior category, thereby determining the main behavior in the video, that is, selecting the behavior category with the highest probability as the classification category of the video.

[0125] In step S620, an abnormality recognition result of the suspected abnormal object is obtained according to the video category.

[0126] In an exemplary embodiment, when the result of the classification process indicates that an abnormal object exists in the sub-video to be detected, the abnormality recognition result of the suspected abnormal object is determined to be abnormal.

[0127] In other words, when the video category of the sub-video to be detected containing the suspected abnormal object is the first video category, the abnormal identification result of the suspected abnormal object is determined to be abnormal, and when the video category of the sub-video to be detected containing the suspected abnormal object is the second video category, the abnormal identification result of the suspected abnormal object is determined to be normal. The first video category is used to indicate that there is an abnormal object in the sub-video to be detected, and the second video category is used to indicate that there is no abnormal object in the sub-video to be detected.

[0128] In the case where the candidate abnormal objects in the present disclosure include abnormal animals, the video to be detected in the present disclosure includes a video of the animal to be detected, and the method for obtaining the video of the animal to be detected may include: in response to the start of the animal feeding device, starting the image acquisition device set on the animal feeding device to record the video of the animal to be detected. The specific implementation method of recording the video of the animal to be detected can refer to the relevant content in the above-mentioned step S710, which will not be repeated here.

[0129] In an exemplary embodiment, the animal video recorded by the camera is sampled based on a first preset frame rate to obtain the video of the animal to be detected. In this way, while retaining the timing information of the video, the efficiency of subsequent video processing can be improved due to the reduction in the number of frames.

[0130] Of course, it is also possible not to sample the animal video recorded by the camera, and this exemplary embodiment does not specifically limit this.

[0131] In a caged animal scenario disclosed herein, taking into account a situation where a camera suddenly malfunctions and then quickly recovers on its own, or when feeding, the animals in a cage are resting and do not move to the front of the cage, resulting in the camera not capturing video containing animals during a period or moment, all or part of the video frames corresponding to the video of the animal to be detected contain animals.

[0132] In the present disclosure, when the abnormal object is an abnormal animal in a group of animals, by setting a camera on an automatic feeder, based on the operating status of the automatic feeder necessary for large-scale breeding, such as the start and stop of the automatic feeder, the video of the animal during feeding can be automatically recorded, so that the abnormal animal can be detected and identified later based on the video recorded by the camera. On the one hand, there is no need to manually turn on or off the camera, which improves efficiency and saves labor time costs; on the other hand, there is no need for robot inspections or infrared cameras to judge abnormal animals, and the technical cost is low, which is suitable for promotion and use in actual breeding; on the other hand, when the animals are more active during feeding, the movement differences of the animals are more obvious, so that abnormal animals can be identified more accurately, improving the accuracy of abnormal animal detection and identification.

[0133] For example, Figure 8 A flow chart showing a method for abnormal animal identification based on video acquisition by an automatic feeder in an exemplary embodiment of the present disclosure is shown. Figure 8, the method may include steps S801 to S808. Wherein: in step S801, the acceleration of the acceleration sensor corresponding to the automatic feeder is detected; in step S802, whether the automatic feeder is started is determined according to the acceleration, if not, then go to step S801, if yes, then go to step S803; in step S803, wake up the RGB camera; in step S804, the RGB camera records the animal video; in step S805, whether the automatic feeder is stopped is determined according to the acceleration, if not, then go to step S804, so that the RGB camera continues to record the animal video, if yes, then go to step S806; in step S806, the RGB camera stops recording the animal video and saves the recorded animal video; in step S807, the RGB camera sends the saved animal video to the abnormal animal identification terminal, so that the abnormal animal identification terminal identifies the abnormal animal in the animal video; in step S808, the abnormal animal identification terminal outputs the abnormal animal identification result.

[0134] For example, as mentioned above, in the caged abnormal animal detection scenario, the animals in the cages are fed by automatic feeders. One automatic feeder corresponds to one acceleration sensor, and one row of cages corresponds to one RGB camera, which is installed on the automatic feeder. Normally, feeding can be performed by starting the automatic feeder at a fixed time, such as feeding 3-4 times a day, that is, feeding once every 6 hours or 8 hours. When the automatic feeder is started, the value of the acceleration sensor on the y-axis will change steeply. By setting the acceleration threshold, it can be used as a switch to determine whether the automatic feeder is started. When the acceleration of the acceleration sensor gradually increases to a large acceleration threshold, it is determined that the automatic feeder is started. When it is determined that the automatic feeder is started, the RGB camera can be awakened, wireless charging can be stopped, and the video recording function can be turned on to start recording animal videos. When the automatic feeder is finished feeding, the acceleration of the acceleration sensor gradually decreases to near 0. When the acceleration of the acceleration sensor gradually decreases to an acceleration threshold close to 0, it is determined that the automatic feeder stops feeding. When the automatic feeder stops feeding, the RGB camera is controlled to stop video recording and save the recorded video, and wireless charging starts. The animal video saved by the RGB camera is sent to the abnormal animal recognition terminal, and abnormal animal detection and analysis is performed based on the abnormal target detection method in the present disclosure. The target detection method disclosed in the present invention can locate the moment when an abnormal object appears and output a screenshot corresponding to the moment when the abnormal object appears. For example, when the abnormal recognition result of a suspected abnormal object is determined to be abnormal, the video frame in which the candidate abnormal object determined as the suspected abnormal object appears can be located.

[0135] In the present disclosure, as described above, an abnormal object detection can be performed in a video frame of a video to be detected by using a target detection model, and the detected abnormal object is determined as a candidate abnormal object. Fig. 9 A flowchart showing a training process of a target detection model in an exemplary embodiment of the present disclosure is shown. Fig. 9 The training process of the target detection model includes steps S910 to S950. Among them:

[0136] In step S910, in response to the start-up of the animal feeding device, an image acquisition device disposed on the feeding device is turned on to record an animal video to obtain an animal video data set.

[0137] The specific implementation of step S910 may refer to the relevant description of the specific implementation of step S710 mentioned above, and will not be repeated here.

[0138] In step S920, video frames of the animal videos in the animal video dataset are sampled based on a preset frame rate to obtain an animal video frame sequence dataset.

[0139] In an exemplary embodiment, the animal video in the animal video dataset obtained in step S910 may be cropped according to the method in step S720 to obtain a target animal video dataset, and then the target animal video dataset may be sampled according to a preset frame rate to obtain an animal video frame sequence dataset.

[0140] In step S930, video frames that do not contain abnormal animals in the animal video frame sequence data set are deleted to obtain an abnormal animal image frame data set.

[0141] For example, when there are abnormal animals in the video frames in the animal video frame sequence, they can be retained, and when there are no abnormal animals, they can be deleted, thereby obtaining an abnormal animal image frame dataset containing abnormal animals. In other words, each image frame in the abnormal animal image frame dataset contains at least one abnormal animal.

[0142] In step S940, for the image frames in the abnormal animal image frame dataset, abnormal animals in the image frames in the abnormal animal image frame dataset are labeled to obtain an abnormal animal image frame label dataset.

[0143] For example, the position information of the abnormal animal in the abnormal animal image frame may be marked, such as the abnormal animal in the abnormal animal image frame may be marked in the form of a rectangular frame, thereby obtaining an abnormal animal image frame label data set.

[0144] In step S950, supervised learning training is performed on the target detection model to be trained based on the abnormal animal image frame label dataset to obtain a trained target detection model.

[0145] In an exemplary embodiment, the target detection model may include any machine learning model capable of performing target detection, such as a yolo (you only look once) target detection model, a fast-rcnn (fast-region CNN) target detection model, etc. This exemplary embodiment does not impose any feature limitations on this.

[0146] Among them, the training method of the target detection model can refer to the above-mentioned training method of the video classification model, that is, the data set is divided into a training set and a test set for model training, which will not be repeated here.

[0147] Through the above-mentioned steps S810 to S850, an abnormal animal target detection model can be trained according to the collected real animal videos, thereby realizing the detection of abnormal animals and obtaining candidate abnormal animals.

[0148] Taking caged animals as an example, Fig.10 A flowchart of a method for training a video classification model and a method for training an object detection model in an exemplary embodiment of the present disclosure is shown. Fig.10 , the method may include steps S1001 to S1009. Wherein: in step S1001, a caged animal video recorded by a camera is obtained; in step S1002, the video screen of the caged animal video recorded by the camera is cropped, and the central area of ​​the video screen is retained to obtain a target caged animal video; in step S1003, the target caged animal video is cut to obtain a target caged animal sub-video segment; in step S1004, the target caged animal sub-video segment is labeled with a video category; in step S1005, the labeled target caged animal sub-video segment is sampled with a video image frame to obtain a target caged animal sub-video image frame sequence label data set; in step S1006, a video classification model is trained according to the sub-video image frame sequence label data set; in step S1007, video image frames that do not contain abnormal animals in the target caged animal sub-video image frame sequence are deleted to obtain an abnormal animal image frame data set; in step S1008, abnormal animal rectangular frames are labeled in the abnormal animal image frame data set to obtain an abnormal animal image frame label data set; in step S1009, a target detection model is trained according to the abnormal animal image frame label data set.

[0149] For example, after obtaining the caged animal video data recorded by the camera, in order to avoid the surrounding redundant information interfering with the final judgment result, the central area of ​​the picture is cropped (cropping the area of ​​1 / 3 width on the left and right, and only retaining the video picture of the middle 1 / 3 area), and the preset length of 5 seconds can be used as the segmentation parameter, for example, 5 seconds. The preset length can be determined according to the running speed of the feeder. The video is cut into N sub-video segments. Since there is an imbalance of positive and negative samples between abnormal animals and normal animal sub-segments, in order to avoid the impact of positive and negative sample imbalance on model training, the ratio of the number of normal animal video sub-segments and abnormal animal video sub-segments in the label data set can be 1:1, and then the video sub-segments are cut at 30 frames per second to generate image frame data. The naming rule of the image frame data saves the timing information and the information of the video sub-segment, and obtains the data set 1 containing the timing information. All the image frames that do not contain abnormal animals in the data set 1 are deleted to obtain the abnormal animal image frame data set 2. The video classification model is trained using the data set 1, such as the slow-fast model, and the target detection model is trained using the data set 2, such as the yolo target detection model.

[0150] In an exemplary embodiment of the present disclosure, abnormal object detection may also be performed based on video category alone, that is, video classification is performed only on the video to be detected. When the video category indicates the presence of an abnormal object, the abnormal object recognition result of the video to be detected is determined to be the presence of an abnormal object. Otherwise, the abnormal object recognition result of the video to be detected is determined to be the absence of an abnormal object.

[0151] In another exemplary embodiment, the detection result of the abnormal object can also be determined based on the detection and tracking result alone. That is, detection and tracking are performed only on the video to be detected. When the detection and tracking result indicates that there is an abnormal object, the abnormal object recognition result of the video to be detected is determined to be that there is an abnormal object. When the detection and tracking result indicates that there is no abnormal object, the abnormal object recognition result is determined to be that there is no abnormal object.

[0152] In another exemplary implementation, video classification and abnormal object detection and tracking may be performed on the video to be detected at the same time, and the two may be combined to detect abnormal objects.

[0153] For example, the video to be detected can be simultaneously classified and detected and tracked for abnormal objects. When the video category is the first video category mentioned above or the detection and tracking result indicates that there are abnormal objects in the video to be detected (i.e., when it is determined that there are suspected abnormal objects in the first video frame sequence), it is determined that there are abnormal objects in the video to be detected. In other words, both methods simultaneously identify abnormal objects. When any method determines that there are abnormal objects in the video to be detected, it is considered that there are abnormal objects in the video to be detected. In this way, the abnormal objects can be identified more comprehensively, and the occurrence of missed identification can be avoided as much as possible.

[0154] Alternatively, when the video category is the first video category and the detection and tracking result indicates that the video to be detected has an abnormal object, it is determined that the video to be detected has an abnormal object. In other words, the two methods simultaneously perform abnormal object identification, and only when the two methods simultaneously determine that the video to be detected has an abnormal object, it is considered that the video to be detected has an abnormal object. In this way, the occurrence of misidentification can be avoided as much as possible, and the accuracy of abnormal object identification can be improved.

[0155] Based on this, for example, Fig.11 A flow chart showing a method for detecting abnormal animals in an exemplary embodiment of the present disclosure is shown. Fig.11 , the method may include steps S1101 to S1109. Wherein: in step S1101, input the video of the animal to be detected; in step S1102, segment the video of the animal to be detected, and obtain the current image frame t in the current T period; in step S1103, perform abnormal animal detection on the current image frame t to obtain the detected abnormal animal rectangular frame; in step S1104, perform target tracking based on the detected abnormal animal rectangular frame to obtain the abnormal animal detection and tracking result; in step S1105, obtain the video segment in the current T period; in step S1106, perform video classification on the video segment in the current T period; in step S1107, obtain the video classification result of the video segment in the current T period; in step S1108; determine the abnormal animal detection result of the video segment in the current T period according to the abnormal animal detection and tracking result and the video classification result; in step S1109, output the video segment with abnormal animals and push the prompt information of the abnormal animal to the client where the breeder is located.

[0156] For example, the video that needs to be used for abnormal animal recognition is input into the terminal for inference. N frames are taken as a video segment, and the video within the T time period (0 < t < N) is used for video classification inference to obtain the video binary classification result, that is, the result of whether there are abnormal animals in the current T time period. At the same time, object detection is performed for each frame at time t in the T time period. If there are abnormal animals, a rectangular box of the abnormal animal target of box(x, y, w, h) will be obtained. Then, based on the object detection result, object tracking is performed. The object tracking algorithm can adopt the above-mentioned bytetrack algorithm, and the information of track(box, track_id, cls) is obtained through tracking, where box represents the position of the tracked rectangular box, track_id represents the tracking target identifier, and cls represents the category of the tracking target. In the present disclosure, cls is the abnormal animal. Taking chickens as an example, cls is the abnormal chicken. The results with the same track_id are stored in the track_list. After the detection and tracking of the video segment within the current T time period are completed, according to the detection and tracking results and the video classification result, it is jointly determined whether there are abnormal animals in the video segment within the current T time period. For example, when the video classification result is true (i.e., the first video category) and the number of occurrences of the same track_id in the track_list is greater than the threshold track_thre (1 < track_thre < N), it is considered that there are abnormal animals in the current time segment. Another example is that it can also be considered that there are abnormal animals in the current time segment when the video classification result is true (i.e., the first video category) or the number of occurrences of the same track_id in the track_list is greater than the threshold track_thre (1 < track_thre < N).

[0157] In an exemplary application scenario of the present disclosure, taking caged chickens as an example, the video of caged chickens can be collected by a camera set on the feeder of the caged chickens, so as to obtain a real video of caged chickens. Data processing is performed on the real video of caged chickens, such as the above-mentioned video segmentation processing, adding video category label processing, marking the position of the rectangular box of abnormal chickens, etc., to obtain a video classification training data set and an abnormal chicken detection data set, and then training a video classification model and an object detection model for abnormal chicken detection. After obtaining the trained video classification model and object detection model, the video of chickens to be detected can be used for abnormal chicken detection.

[0158] For example, a caged chicken video can be collected by a camera set on a caged chicken feeder, and the collected video can be cropped to remove other non-chicken background information as much as possible, retain the chicken information, and obtain the target caged chicken video. The target caged chicken video is segmented according to the duration T to obtain the target caged chicken video clips, and then based on the trained target detection model, abnormal chicken detection and tracking is performed for the current time t to obtain the abnormal chicken detection and tracking results, and based on the trained video classification model, the target caged chicken video clips in the T period to which the current time t belongs are classified to obtain the video classification results. Finally, for the T period to which the current time t belongs, according to the video classification results and the number of occurrences of the same abnormal chicken in the tracking frame, it is determined whether there is an abnormal chicken in the current T period, and the abnormal chicken results are pushed to the farmer's client, so that the farmer can deal with the abnormal chicken in time.

[0159] For example, Fig.12 As shown, the abnormal chicken results pushed to the farmer may include an image frame screenshot of the moment the abnormal chicken is detected, in which information such as the number of abnormal chickens and the cages where the abnormal chickens are located is displayed, and this exemplary embodiment does not make any special limitation on this.

[0160] In the present disclosure, for abnormal animal identification, a camera is set on the automatic feeder, and a gravity sensor is used to monitor whether the feeder is feeding, so as to automatically obtain animal video information during feeding based on the automatic feeder equipment that is necessary for large-scale breeding. From a biological point of view, feeding is when animals are more active, and the movement differences of animals are more obvious, making it easier to accurately locate abnormal animals. Compared with robot inspections or abnormal animal identification solutions based on thermal infrared temperature sensors, the abnormal animal identification method in the present disclosure has low technical cost, low promotion cost, and high accuracy.

[0161] In addition, it is difficult for the human eye to judge whether an object is an abnormal object through a single-frame image, such as whether a chicken is a sick or weak chicken or a dead chicken. Therefore, abnormal object recognition based on single-frame image detection is definitely not accurate enough. Even if the abnormal object is tracked, since the tracking is based on the detection results, inaccurate detection will also lead to unreliable tracking results.

[0162] In the present disclosure, in order to solve this problem, when it is determined that there is a suspected abnormal object according to the first video frame sequence of the video to be detected, the sub-video to be detected is intercepted from the video to be detected according to the first video frame sequence, and then the sub-video to be detected containing the suspected abnormal object is classified and processed, and finally the abnormal recognition result of the suspected abnormal object is obtained. In this way, on the one hand, when it is determined that there is a suspected abnormal object, the abnormal state of the suspected abnormal object is further judged by intercepting the sub-video according to the classification result of the sub-video. Since the video can reflect the continuous state of the object over a period of time, the probability of false detection can be reduced and the accuracy of abnormal object detection can be improved; on the other hand, the present disclosure first determines that there is a suspected abnormal object through the first video frame sequence, then intercepts the sub-video, and then performs video classification processing on the sub-video to obtain the abnormal recognition result, instead of directly classifying the entire video. The efficiency of classifying a shorter video segment is definitely higher than the efficiency of classifying a longer video segment. Therefore, the detection efficiency of abnormal objects can be improved.

[0163] In addition, it should be noted that the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0164] Furthermore, an exemplary embodiment of the present disclosure also provides a target detection device. Fig.13 As shown, the target detection device includes the following program modules: a suspected abnormal object determination module 1310, configured to determine whether there is a suspected abnormal object based on a first video frame sequence of the video to be detected; a video interception module 1320, configured to intercept a sub-video to be detected from the video frame to be detected according to the first video frame sequence when the suspected abnormal object exists, wherein the time period corresponding to the first video frame sequence partially overlaps with the time period corresponding to the sub-video to be detected; a classification module 1330, configured to classify the sub-video to be detected to obtain an abnormality recognition result of the suspected abnormal object.

[0165] In an exemplary embodiment, based on the aforementioned embodiment, the method for determining the first video frame sequence includes: when a candidate abnormal object is detected in a video frame of a video to be detected, determining the abnormal time at which the candidate abnormal object is detected, and determining the first video frame sequence based on the first video frame in the video frame of the video to be detected that is later than the abnormal time.

[0166] In an exemplary embodiment, based on the aforementioned embodiment, determining whether there is a suspected abnormal object based on the first video frame sequence of the video to be detected includes: performing target tracking on the candidate abnormal object in the first video frame sequence; and determining whether the candidate abnormal object is a suspected abnormal object based on the target tracking situation when the candidate abnormal object is tracked in the first video frame sequence.

[0167] In an exemplary embodiment, based on the aforementioned embodiment, determining whether the candidate abnormal object is a suspected abnormal object according to the target tracking situation of the candidate abnormal object being tracked in the first video frame sequence includes: when the number of times the candidate abnormal object is tracked in the first video frame sequence reaches a first preset value, determining that the candidate abnormal object is the suspected abnormal object; or when the duration for which the candidate abnormal object is tracked in the first video frame sequence reaches a first preset duration, determining that the candidate abnormal object is a suspected abnormal object.

[0168] In an exemplary embodiment, based on the aforementioned embodiment, the method of extracting a sub-video to be detected from the video to be detected according to the first video frame sequence includes: determining an initial video frame from the video frames of the video to be detected according to the video frame in which the candidate abnormal object is detected; determining a cutoff video frame from the video frames of the video to be detected based on the video frame in which the number of times the candidate abnormal object is tracked in the first video frame sequence reaches a first preset value; and intercepting video frames from the initial video frame to the cutoff video frame in the video to be detected to obtain the sub-video to be detected.

[0169] In an exemplary embodiment, based on the aforementioned embodiment, the video frames from the initial video frame to the cutoff video frame in the video to be detected to obtain the sub-video to be detected include: intercepting the video frames from the initial video frame to the cutoff video frame in the video to be detected to determine the sub-video to be cut; for the initial video frame in the sub-video to be cut, according to the position of the detected candidate abnormal object in the initial video frame, the initial video frame is cut; for the second video frame in the sub-video to be cut, according to the tracked position of the candidate abnormal object in the second video frame, the second video frame is cut; according to the sub-video to be cut after the video frame is cut, the sub-video to be detected is obtained.

[0170] In an exemplary embodiment, based on the aforementioned embodiment, the second video frame is cropped according to the position of the tracked candidate abnormal object in the second video frame; including: determining a first target video frame according to the second video frame in which the candidate abnormal object is tracked, and determining a second target video frame according to the second video frame in which the candidate abnormal object is not tracked; cropping the first target video frame according to the position of the tracked candidate abnormal object in the first target video frame; determining a reference video frame according to the first target video frame that is before the second target video frame and closest to the second target video frame; and cropping the second target video frame based on the position of the tracked candidate abnormal object in the reference video frame.

[0171] In an exemplary embodiment, based on the aforementioned embodiment, the classification processing of the sub-video to be detected to obtain the abnormal recognition result of the suspected abnormal object includes: when the result of the classification processing indicates that there is an abnormal object in the sub-video to be detected, determining that the abnormal recognition result of the suspected abnormal object is abnormal.

[0172] In an exemplary embodiment, based on the aforementioned embodiment, the classification processing of the sub-video to be detected to obtain the abnormal recognition result of the suspected abnormal object includes: inputting the sub-video to be detected into a pre-trained video classification model to output the video category of the sub-video to be detected; and obtaining the abnormal recognition result of the suspected abnormal object according to the video category.

[0173] In an exemplary embodiment, based on the aforementioned embodiment, the abnormal objects include abnormal animals, and the training process of the video classification model includes the following steps: in response to the start-up of the animal feeding device, turning on the image acquisition device provided on the feeding device to record animal videos to obtain an animal video data set; segmenting the animal videos in the animal video data set to obtain an animal sub-video data set, and annotating the animal sub-videos in the animal sub-video data set with video categories to obtain an animal sub-video label data set; and performing supervised learning training on the video classification model to be trained according to the animal sub-video label data set to obtain a trained video classification model.

[0174] The specific details of each part of the above-mentioned device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0175] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0176] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0177] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned target detection method is implemented.

[0178] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0179] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0180] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, Python, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0181] Computer programs can be carried or transmitted through electrical, magnetic, optical, electromagnetic, infrared and other signals. Electronic devices can convert signals carrying computer programs into digital signals to run computer programs. When a computer program runs on an electronic device, its code is used to enable the electronic device to execute (more specifically, it can enable the processor of the electronic device to execute) the method steps of various exemplary embodiments of the present disclosure, such as the target detection method described above, which includes the following steps: determining whether there is a suspected abnormal object based on the first video frame sequence of the video to be detected; in the case of the presence of the suspected abnormal object, cutting out a sub-video to be detected from the video to be detected based on the first video frame sequence, wherein the time period corresponding to the first video frame sequence partially overlaps with the time period corresponding to the sub-video to be detected; classifying the sub-video to be detected to obtain an abnormal recognition result of the suspected abnormal object.

[0182] The above method steps are executed by a computer program. On the one hand, when it is determined that there is a suspected abnormal object, a sub-video is intercepted, and the abnormal state of the suspected abnormal object is further judged according to the classification result of the sub-video. Since the video can reflect the continuous state of the object over a period of time, the probability of false detection can be reduced and the accuracy of abnormal object detection can be improved. On the other hand, when it is determined that there is a suspected abnormal object through the first video frame sequence, a sub-video is intercepted, and then the sub-video is subjected to video classification processing to obtain an abnormal recognition result, instead of directly classifying the entire video. Therefore, the detection efficiency of abnormal objects can be improved.

[0183] The exemplary embodiments of the present disclosure also provide an electronic device, such as the terminal device 110 or the server 120 described above. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as a computer program. The processor executes the method steps of various exemplary embodiments of the present disclosure by executing the executable instructions. In addition, the electronic device may also include a display for displaying a graphical user interface.

[0184] Reference below Fig.14 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Fig.14 The electronic device 1400 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0185] like Fig.14 As shown, the electronic device 1400 may include: a processor 1410 , a memory 1420 , a bus 1430 , an I / O (input / output) interface 1440 , a network adapter 1450 , and a display 1460 .

[0186] The memory 1420 may include a volatile memory, such as a RAM 1421, a cache unit 1422, and may also include a non-volatile memory, such as a ROM 1423. The memory 1420 may also include one or more program modules 1424, such program modules 1424 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 1424 may include each module in the above-mentioned device.

[0187] Processor 1410 may include one or more processing units. For example, processor 1410 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0188] The processor 1410 can be used to execute executable instructions stored in the memory 1420, such as the above-mentioned target detection method, which includes the following steps: determining whether there is a suspected abnormal object based on the first video frame sequence of the video to be detected; in the case of the existence of the suspected abnormal object, cutting out a sub-video to be detected from the video to be detected based on the first video frame sequence, wherein the time period corresponding to the first video frame sequence partially overlaps with the time period corresponding to the sub-video to be detected; and classifying the sub-video to be detected to obtain an abnormality recognition result of the suspected abnormal object.

[0189] The above method is implemented through a computer program. On the one hand, when it is determined that there is a suspected abnormal object, a sub-video is intercepted, and the abnormal state of the suspected abnormal object is further judged according to the classification result of the sub-video. Since the video can reflect the continuous state of the object over a period of time, the probability of false detection can be reduced and the accuracy of abnormal object detection can be improved. On the other hand, when it is determined that there is a suspected abnormal object through the first video frame sequence, a sub-video is intercepted, and then the sub-video is classified to obtain an abnormal recognition result, instead of directly classifying the entire video. Therefore, the detection efficiency of abnormal objects can be improved.

[0190] The bus 1430 is used to realize the connection between different components of the electronic device 1400, and may include a data bus, an address bus, and a control bus.

[0191] The electronic device 1400 can communicate with one or more external devices 1500 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 1440 .

[0192] The electronic device 1400 can communicate with one or more networks through the network adapter 1450. For example, the network adapter 1450 can provide mobile communication solutions such as 3G / 4G / 5G, or provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 1450 can communicate with other modules of the electronic device 1400 through the bus 1430.

[0193] The electronic device 1400 may display a graphical user interface through the display 1460 , such as an interface corresponding to abnormal object prompt information pushed to farmers.

[0194] although Fig.14 Not shown, other hardware and / or software modules may also be provided in the electronic device 1400, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0195] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0196] As can be seen from the above, the technical solution of the present disclosure can be implemented as a method, an apparatus, a system, a computer program product, a storage medium, an electronic device, etc. Those skilled in the art can understand that various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, such as being respectively referred to as a "circuit", "module" or "system".

[0197] It should be understood that the present disclosure is not limited to the specific method steps or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. Those skilled in the art will easily think of other embodiments based on the specific embodiments provided by the present disclosure. Therefore, the specific embodiments provided by the present disclosure are only exemplary, and the scope and spirit of the present disclosure are indicated by the claims, and any variations, uses or adaptive changes of the present disclosure should be covered, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the present disclosure.

Claims

1. A target detection method, characterized in that: include: Determine whether there is a suspected abnormal object according to a first video frame sequence of the video to be detected; In the case where the suspected abnormal object exists, a sub-video to be detected is cut out from the video to be detected according to the first video frame sequence, wherein a time period corresponding to the first video frame sequence partially overlaps with a time period corresponding to the sub-video to be detected; The sub-video to be detected is classified and processed to obtain an abnormality recognition result of the suspected abnormal object.

2. The target detection method according to claim 1, characterized in that: The method for determining the first video frame sequence includes: In the case where a candidate abnormal object is detected in a video frame of the video to be detected, determining an abnormal time at which the candidate abnormal object is detected; The first video frame sequence is determined according to a first video frame whose time is later than the abnormal time in the video frames of the video to be detected.

3. The target detection method according to claim 2, characterized in that: The determining whether there is a suspected abnormal object according to the first video frame sequence of the video to be detected includes: Performing target tracking on the candidate abnormal object in the first video frame sequence; According to the situation that the candidate abnormal object is tracked in the first video frame sequence, it is determined whether the candidate abnormal object is a suspected abnormal object.

4. The target detection method according to claim 3, characterized in that: The determining, according to the situation that the candidate abnormal object is tracked in the first video frame sequence, whether the candidate abnormal object is a suspected abnormal object comprises: When the number of times that the candidate abnormal object is tracked in the first video frame sequence reaches a first preset value, determining that the candidate abnormal object is the suspected abnormal object; and / or When the duration during which the candidate abnormal object is tracked in the first video frame sequence reaches a first preset duration, the candidate abnormal object is determined to be the suspected abnormal object.

5. The target detection method according to claim 3, characterized in that: The extracting a sub-video to be detected from the video to be detected according to the first video frame sequence includes: According to the video frame in which the candidate abnormal object is detected, determining an initial video frame from the video frames of the video to be detected; Based on the video frames in which the candidate abnormal object is tracked a number of times reaching a first preset value in the first video frame sequence, determining a cutoff video frame from the video frames of the video to be detected; Video frames from the initial video frame to the cutoff video frame are intercepted in the video to be detected to obtain the sub-video to be detected.

6. The target detection method according to claim 5, characterized in that: The intercepting of video frames from the initial video frame to the cutoff video frame in the video to be detected to obtain the sub-video to be detected comprises: Intercepting video frames from the initial video frame to the cutoff video frame in the video to be detected to determine a sub-video to be cut; For an initial video frame in the sub-video to be cropped, cropping the initial video frame according to the detected position of the candidate abnormal object in the initial video frame; For a second video frame in the sub-video to be cropped, cropping the second video frame according to a position of the tracked candidate abnormal object in the second video frame; The sub-video to be detected is obtained according to the sub-video to be cropped after the video frame is cropped.

7. The target detection method according to claim 6, characterized in that: The method of cropping the second video frame according to the tracked position of the candidate abnormal object in the second video frame comprises: Determine a first target video frame according to a second video frame in which the candidate abnormal object is tracked, and determine a second target video frame according to a second video frame in which the candidate abnormal object is not tracked; Cropping the first target video frame according to the position of the candidate abnormal object in the first target video frame; Determine a reference video frame according to a first target video frame that is before the second target video frame and closest to the second target video frame; The second target video frame is cropped based on the position of the candidate abnormal object in the reference video frame.

8. The target detection method according to claim 1, characterized in that: The classifying process of the sub-video to be detected to obtain the abnormality recognition result of the suspected abnormal object includes: Inputting the sub-video to be detected into a pre-trained video classification model to output the video category of the sub-video to be detected; An abnormality recognition result of the suspected abnormal object is obtained according to the video category.

9. The target detection method according to claim 8, characterized in that: The abnormal objects include abnormal animals, and the training process of the video classification model includes the following steps: In response to the activation of the animal feeding device, an image acquisition device disposed on the feeding device is activated to record an animal video to obtain an animal video data set; Segmenting the animal videos in the animal video dataset to obtain an animal sub-video dataset, and annotating the animal sub-videos in the animal sub-video dataset with video categories to obtain an animal sub-video label dataset; The video classification model to be trained is trained through supervised learning according to the animal sub-video label dataset to obtain a trained video classification model.

10. A target detection device, characterized in that: include: A suspected abnormal object determination module is configured to determine whether there is a suspected abnormal object according to a first video frame sequence of the video to be detected; a video interception module configured to intercept a sub-video to be detected from the video frame to be detected according to the first video frame sequence when the suspected abnormal object exists, wherein a time period corresponding to the first video frame sequence partially overlaps with a time period corresponding to the sub-video to be detected; The classification module is configured to perform classification processing on the sub-video to be detected to obtain an abnormality recognition result of the suspected abnormal object.

11. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the target detection method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the target detection method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the target detection method according to any one of claims 1 to 9 is implemented.

Images