Livestock behavior identification method and system based on artificial intelligence

Abstract

The invention relates to a livestock behavior recognition method and system based on artificial intelligence, and the method comprises the steps: obtaining the target positions of livestock in a current frame of a livestock video through an encoder-decoder network, and the deviation of each target position, and determining the range in an apparent feature map based on the target positions and the deviation, determining an apparent characteristic matrix of the target by using the apparent characteristic vector and the heat map in the range; determining weights of an IOU matrix and an apparent feature similarity matrix based on the number of successful track continuous matching times, calculating the similarity between each track and each target by using the weights, and matching the tracks with the targets by adopting a Hungary matching mode; and inputting the historical apparent feature matrix set of the track in the confirmed state into a time sequence network to obtain behavior classification of the livestock. According to the method, the problem that tracking and behavior recognition of different individuals are disordered due to the fact that the livestock hair colors and the individuals are extremely similar is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an artificial intelligence-based livestock behavior recognition method and system. Background Art

[0002] In the past, people mainly observed abnormal behaviors of livestock through the observation of breeders, such as poor mental state and poor appetite. On the one hand, this depends on the experience of breeders. Inexperienced breeders may not be able to accurately identify. On the other hand, only when breeders observe, abnormalities will be found. If they cannot be identified in the early stage, subsequent treatment will be more troublesome. With the development of deep learning technology, livestock behavior recognition methods based on computer vision can monitor livestock behavior in real time. By using deep neural networks (DNN) and image processing-based models in video surveillance systems, livestock behavior characteristics can be efficiently extracted, such as common behaviors such as walking, resting, eating, and drinking. These behavior patterns are not only closely related to the growth and development and reproductive cycle of livestock, but also directly affect the breeding benefits. However, when using traditional behavior recognition methods to identify livestock behaviors, R&D personnel found that because the hair color and individual size of the same type of livestock, such as pigs and cattle, are relatively similar, traditional behavior recognition models cannot distinguish between different individuals of the same type of livestock, resulting in confusion in behavior recognition. Summary of the invention

[0003] In view of the problem that different individuals of the same species of livestock have similar coat colors and are difficult to distinguish, resulting in confusion in behavior recognition, in a first aspect, the present invention provides a livestock behavior recognition method based on artificial intelligence, the method comprising: Obtaining the target position of livestock in the current frame of the livestock video and the offset of each target position through an encoder-decoder network, determining the range in the apparent feature map based on the target position and the offset, and determining the apparent feature matrix of the target using the apparent feature vector and the heat map within the range; The weights of the IOU matrix and the apparent feature similarity matrix are determined based on the number of successful consecutive track matchings, and the similarity between each track and each target is calculated using the weights, and the tracks and targets are matched using the Hungarian matching method; The historical appearance feature matrix set of the confirmed state trajectory is input into the time series network to obtain the behavior classification of livestock.

[0004] In a possible implementation manner, determining the range in the apparent feature map based on the position of the target and the offset is specifically as follows: Get the length and width of the target, and convert them to the length and width on the heat map according to the current frame image size and heat map size; The position of the target plus the offset position is taken as the center point of the rectangle, the length and width of the rectangle are set as the length and width on the heat map, and the area of ​​the rectangle on the apparent feature map is taken as the range.

[0005] In a possible implementation manner, the use of the apparent feature vectors and the heat map within the range to determine the apparent feature matrix of the target is specifically: In the heat map, the points within the range are sorted according to the values ​​of the points in the heat map, and a preset number of points are selected from the sorted point set; Determine and select the positions of a preset number of points in the appearance feature map, and combine the appearance feature vectors of the positions into an appearance feature matrix of the target.

[0006] In a possible implementation, the weights of the IOU matrix and the appearance feature similarity matrix are determined based on the number of successful consecutive trajectory matching, specifically: Obtain the maximum number of trajectory mismatch retention times. If the number of consecutive successful matches is less than the maximum number of retention times, the weights of the IOU matrix and the apparent feature similarity matrix are the same; When the number of consecutive successful matches is greater than or equal to the maximum retention number, the weight of the IOU matrix increases by a preset value, and the weight of the apparent feature similarity matrix decreases by a preset value every time the number of consecutive successful matches increases by the maximum retention number; the weight of the IOU matrix and the weight of the apparent feature similarity matrix are between [0,1], and the sum of the two is 1.

[0007] In a possible implementation manner, after matching the trajectory and the target using the Hungarian matching method, the method further includes: If the target is not matched successfully, a new trajectory is created according to the target; If the track does not match successfully, if the track is in a confirmed state and the number of times the track has not been matched successfully is greater than the maximum number of retention times, the track will be deleted; if the track is in a confirmed state and the number of times the track has not been matched successfully is not greater than the maximum number of retention times, the track will be retained; if the track is in an unconfirmed state, the track will be deleted.

[0008] In another aspect, the present invention provides a livestock behavior recognition system based on artificial intelligence, the system comprising: A feature acquisition module is used to obtain the target position of livestock and the offset of each target position in the current frame of the livestock video through an encoder-decoder network, determine the range in the apparent feature map based on the target position and the offset, and determine the apparent feature matrix of the target using the apparent feature vector and the heat map within the range; A matching module is used to determine the weights of the IOU matrix and the apparent feature similarity matrix based on the number of consecutive successful track matching, calculate the similarity between each track and each target using the weights, and match the track and the target using the Hungarian matching method; The behavior recognition module is used to input the historical appearance feature matrix set of the confirmed state trajectory into the time series network to obtain the behavior classification of livestock.

[0009] In a possible implementation manner, determining the range in the apparent feature map based on the position of the target and the offset is specifically as follows: Get the length and width of the target, and convert them to the length and width on the heat map according to the current frame image size and heat map size; The position of the target plus the offset position is taken as the center point of the rectangle, the length and width of the rectangle are set as the length and width on the heat map, and the area of ​​the rectangle on the apparent feature map is taken as the range.

[0010] In a possible implementation manner, the use of the apparent feature vectors and the heat map within the range to determine the apparent feature matrix of the target is specifically: In the heat map, the points within the range are sorted according to the values ​​of the points in the heat map, and a preset number of points are selected from the sorted point set; Determine and select the positions of a preset number of points in the appearance feature map, and combine the appearance feature vectors of the positions into an appearance feature matrix of the target.

[0011] In a possible implementation, the weights of the IOU matrix and the appearance feature similarity matrix are determined based on the number of successful consecutive trajectory matching, specifically: Obtain the maximum number of trajectory mismatch retention times. If the number of consecutive successful matches is less than the maximum number of retention times, the weights of the IOU matrix and the apparent feature similarity matrix are the same; When the number of consecutive successful matches is greater than or equal to the maximum retention number, the weight of the IOU matrix increases by a preset value, and the weight of the apparent feature similarity matrix decreases by a preset value every time the number of consecutive successful matches increases by the maximum retention number; the weight of the IOU matrix and the weight of the apparent feature similarity matrix are between [0,1], and the sum of the two is 1.

[0012] In a possible implementation manner, after matching the trajectory and the target using the Hungarian matching method, the method further includes: If the target is not matched successfully, a new trajectory is created according to the target; If the track does not match successfully, if the track is in a confirmed state and the number of times the track has not been matched successfully is greater than the maximum number of retention times, the track will be deleted; if the track is in a confirmed state and the number of times the track has not been matched successfully is not greater than the maximum number of retention times, the track will be retained; if the track is in an unconfirmed state, the track will be deleted.

[0013] In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method provided in any one of the embodiments of the first aspect.

[0014] In a fourth aspect, a computing device is provided, comprising a memory and a processor. The memory stores executable code, and when the processor executes the executable code, the method provided in any one of the embodiments of the first aspect is implemented.

[0015] In order to solve the problem of confusion in livestock behavior recognition due to the extremely similar fur color and size of livestock, the present invention determines the target's apparent feature matrix based on the range determined by the target position and offset in the apparent feature map, using the apparent feature vector and heat map within the range, and can provide more and more refined features.

[0016] In addition, since livestock individuals are extremely similar, which easily leads to the problem of identity switching during the tracking process, the present invention dynamically adjusts the weights of the IOU matrix and the apparent feature similarity matrix based on the number of consecutive successful trajectory matches. As the tracking time increases, if the number of consecutive successful matches is large and the trajectory prediction is more accurate, the weight of the IOU matrix is ​​gradually increased, thereby reducing the possibility of identity switching. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of Embodiment 1; Figure 2 It is the structural diagram of the feature extraction module; Figure 3 A schematic diagram showing the selection of a preset number of points according to the values ​​on the heat map is shown; Figure 4 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0019] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0021] Figure 1 A flowchart of a livestock behavior recognition method based on artificial intelligence is provided for one embodiment disclosed in this specification. The execution subject of the method can be a device with processing capabilities: a server or a system or a device or a software platform.

[0022] like Figure 1 As shown, the method specifically includes steps S110 to S130: Step S110, obtaining the target position of the livestock in the current frame of the livestock video and the offset of each target position through an encoder-decoder network, determining the range in the appearance feature map based on the target position and the offset, and determining the appearance feature matrix of the target using the appearance feature vector and the heat map within the range; Specifically, the encoder-decoder network of the FairMOT model is used to encode and decode the current frame of the input livestock video to obtain the heat map, the length and width feature map of the target box, the center point offset and the appearance feature map, such as Figure 2 As shown in the figure, the heat map, target box length and width feature map, center point offset and appearance feature map have the same length and width, but different number of channels. Specifically, the number of channels of the heat map is 1, and the value of each point in the heat map indicates the possibility of the existence of the target. The larger the value, the greater the possibility of the existence of the target. The target box length and width feature map indicates the length and width of the target at the corresponding position in the heat map, and its number of channels is 2. The number of channels of the center point offset is also 2, indicating the center offset of the target point in the heat map. The appearance feature map is a multi-channel feature map such as a 256 or 512-channel feature map, and each point represents the appearance feature (Re-ID feature) of the corresponding position in the heat map.

[0023] Since the coat color, size, etc. of the same kind of livestock are very similar, the present invention uses the apparent features of multiple positions in the apparent feature map to form the apparent feature matrix of the target to obtain more accurate apparent features. Specifically, the range of the apparent feature map is determined based on the position of the target and the offset, and the apparent feature matrix of the target is determined using the apparent feature vector and the heat map within the range. The apparent features of each target no longer use a one-dimensional apparent feature vector, but use multiple one-dimensional apparent feature vectors to form an apparent feature matrix. For example, the apparent feature matrix of a target is 2×2×256, with a length and width of 2 respectively, and the number of channels is 256, then there are 4 apparent feature vectors with a length of 256, wherein the apparent feature vector comes from the apparent feature map.

[0024] In one embodiment, the range in the apparent feature map is determined based on the position of the target and the offset, specifically: obtaining the length and width of the target, converting the length and width to the length and width on the heat map according to the current frame image size and the heat map size; taking the position of the target plus the offset as the center point of the rectangle, setting the length and width of the rectangle as the length and width of the heat map, and taking the area of ​​the rectangle on the apparent feature map as the range. Specifically, after obtaining the target from the heat map, the value of the target in the same position in the heat map in the length and width feature map of the target frame is obtained from the length and width feature map of the target frame, and the obtained value is the length h and width w of the target frame. After encoding and decoding, the size of the current frame image and the heat map are different. For example, the size of the current frame image is 1024×1024, and the size of the heat map is 128×128. Since the length h and width w are the values ​​of the target frame in the current frame image, they are converted to the size corresponding to the heat map, for example, the length h is 80, and the size converted to the heat map is 8. Similarly, the width is also converted. The position of the target on the heat map is added with an offset to obtain the accurate position of the target frame, and then a rectangle is determined with the accurate position of the target frame as the center and the length and width after conversion in the apparent feature map, and the rectangle is the range. In one example, the target frame size obtained from the target frame length and width feature map is 80×160, which is 10×20 after conversion to the heat map. If the position of a target on the heat map is (16, 36), and the offset is (15, 38), then the range is a rectangle with a length of 10 and a width of 20 as the center (15, 38). In one embodiment, the length and width of the rectangle are set as the length and width transferred to the heat map. Specifically, the horizontal coordinate of the center point plus half of the height after conversion is used as the upper side of the quadrilateral, the horizontal coordinate of the center point minus half of the height after conversion is used as the lower side of the quadrilateral, the vertical coordinate of the center point plus half of the width after conversion is used as the left side of the quadrilateral, and the vertical coordinate of the center point minus half of the width after conversion is used as the right side of the quadrilateral.

[0025] Since there may be many points in the heat map within the range, the subsequent calculation amount is relatively large, and the existence of points with low target probability may interfere with the results. In one embodiment, the apparent feature vectors and heat map within the range are used to determine the apparent feature matrix of the target, specifically: sorting the points within the range in the heat map according to the values ​​of the points in the heat map, and selecting a preset number of points from the sorted point set; determining the positions of the selected preset number of points in the apparent feature map, and combining the apparent feature vectors of the positions into the apparent feature matrix of the target. After determining the range, sort the points within the range in the heat map according to the values ​​of the points in the heat map from large to small. In an example, assuming that there are 6 points in the range, the sorted points are point 4 (6, 6), point 5 (5, 5), point 1 (5, 6), point 3 (7, 6), point 6 (6, 5), and point 2 (7, 5). If the preset number is 4, the apparent feature vectors of the first 4 points on the apparent feature map are obtained. Figure 3 The result of sorting the points in the range according to the values ​​on the heat map is shown, forming an apparent feature matrix of size 2×2, and the number of channels of the apparent feature matrix is ​​the same as the number of channels of the apparent feature map.

[0026] Step S120, determining the weights of the IOU matrix and the apparent feature similarity matrix based on the number of consecutive successful track matching, calculating the similarity between each track and each target using the weights, and matching the track and the target using the Hungarian matching method; In multi-target tracking, it is necessary to associate the targets detected in consecutive frames with the tracks (Tracks) of the previous frames to determine which target frames belong to the same target. This process usually uses a cost matrix to measure the degree of match between each track and each detection frame. However, livestock individuals are relatively similar, and the traditional multi-target tracking first performs appearance matching and then performs IOU matching, which is not a good practical effect. In the present invention, the weights of the IOU matrix and the apparent feature similarity matrix are determined based on the number of successful continuous track matching. Preferably, the more successful continuous track matching times, the greater the weight of the IOU matrix, and the smaller the weight of the apparent feature similarity matrix. Among them, the IOU matrix is ​​a matrix composed of the IOU values ​​of the target frames of each target and each track, the apparent feature similarity matrix is ​​a matrix composed of the similarity of the appearance feature matrices of each target and each track, and the IOU matrix and the apparent feature similarity matrix have the same size. Preferably, the number of columns of the IOU matrix and the apparent feature similarity matrix is ​​the number of tracks, and the number of rows is the number of targets, as shown in Table 1. In an optional embodiment, the appearance feature similarity is calculated using a Triplet network; in another embodiment, the maximum value of the similarities between each point in the appearance feature matrix of the target and all the points in the appearance feature matrix of the trajectory is used as the similarity of the point in the appearance feature matrix of the target, and the average value of the similarities of each point in the appearance feature matrix of the target is used as the appearance feature similarity between the target and the trajectory. Preferably, the first appearance feature matrix in the historical appearance feature matrix set of the trajectory is used as the appearance feature matrix of the trajectory.

[0027] Table 1

[0028] In one embodiment, the weights of the IOU matrix and the apparent feature similarity matrix are determined based on the number of successful consecutive trajectory matches, specifically: Get the maximum number of track mismatch retention times. If the number of consecutive successful matches is less than the maximum number of retention times, the weights of the IOU matrix and the apparent feature similarity matrix are the same; where the maximum number of track mismatch retention times is the number of consecutive frames in which a track does not match the target, then the track will be deleted. For example, if the maximum number of retention times is 3, a track will be deleted if it does not match the target for 3 consecutive frames. When a track is just beginning to be tracked, or when it is re-matched after a short mismatch during the tracking process, the weights of the IOU matrix and the apparent feature similarity matrix are the same value. Preferably, the weights of both are 0.5.

[0029] When the number of consecutive successful matches is greater than or equal to the maximum number of retentions, the weight of the IOU matrix increases by a preset value, and the weight of the apparent feature similarity matrix decreases by a preset value for each increase in the number of consecutive successful matches by the maximum number of retentions; the weight of the IOU matrix and the weight of the apparent feature similarity matrix are between [0,1], and the sum of the two is 1. If the number of consecutive successful matches is greater than or equal to the maximum number of retentions, it means that the trajectory is relatively stable and the prediction of the trajectory is relatively accurate. At this time, the weight of the IOU matrix is ​​set to be larger, so that it depends more on the predicted position rather than the appearance features, which can reduce mismatches caused by the same appearance. When the number of consecutive track matches exceeds the maximum number of retentions, each time the number of consecutive matches increases by the maximum number of retentions, the weight of the IOU matrix will increase by a preset value, such as 0.05 or 0.1, and the weight of the apparent feature similarity matrix will decrease by the same preset value accordingly, until the weight of the IOU matrix reaches the maximum value of 1.

[0030] In addition, after obtaining the IOU matrix, the appearance feature similarity matrix and the corresponding weights, the IOU matrix and the appearance feature similarity matrix are weighted by the weights and added bit by bit, and after normalization, a matching matrix is ​​constructed, and the trajectory and the target are matched using the Hungarian matching algorithm. Since the Hungarian algorithm has the lowest cost, the Hungarian algorithm can be improved to maximize the cost as the optimal matching result.

[0031] In another embodiment, after the Hungarian matching method is used to match the track and the target, it also includes: if the target is not matched successfully, a new track is created according to the target; if the track is not matched successfully, if the track is in a confirmed state and the number of consecutive unmatched tracks is greater than the maximum number of retention times, the track is deleted; if the track is in a confirmed state and the number of consecutive unmatched tracks is not greater than the maximum number of retention times, the track is retained; if the track is in an unconfirmed state, the track is deleted. When the newly detected target does not match any existing track successfully, a new track is created for the target, indicating that a new target begins to enter the tracking field of view. On the contrary, if an existing track does not match any target in the current frame, if the track is already in a confirmed state, and the number of consecutive unmatched tracks exceeds the preset maximum number of retention times, it is considered that the target has left the field of view and the track is deleted; if the track is in a confirmed state, but the number of consecutive mismatches has not exceeded the maximum number of retention times, the track is temporarily retained. Finally, if the track is still in an unconfirmed state, the track will be directly deleted.

[0032] Step S130, inputting the historical appearance feature matrix set of the confirmed state trajectory into the time series network to obtain the behavior classification of the livestock.

[0033] When the trajectory matches the target, the target's apparent feature matrix will be added to the historical apparent feature matrix set of the trajectory. The historical apparent feature matrix set is set to a fixed length, such as 20 or 30, and the apparent feature matrices in the historical apparent feature matrix set are sorted according to the time sequence of the apparent feature matrix. When a new apparent feature matrix is ​​added, if the historical apparent feature matrix set is full, the earliest historical apparent feature matrix is ​​deleted, and then all the apparent feature matrices are moved back one position. The trajectory of the confirmation state is relatively stable, and the historical apparent feature matrix set of the confirmation state is input into a time series network such as LSTM, GRU, RNN, etc. to identify the behavior of livestock, such as standing, walking, eating, etc. In one embodiment, each historical apparent feature matrix in the historical apparent feature matrix set is reduced in dimension, and then the historical apparent feature matrix set is reduced in dimension and input into LSTM or GRU. The dimensionality reduction method includes but is not limited to downsampling, convolution or C3D convolution.

[0034] Corresponding to the livestock behavior recognition method, the embodiments disclosed in the specification of the present invention further provide a livestock behavior recognition system, such as Figure 4 As shown, the system comprises: A feature acquisition module 410 is used to obtain the target position of the livestock in the current frame of the livestock video and the offset of each target position through an encoder-decoder network, determine the range in the apparent feature map based on the target position and the offset, and determine the apparent feature matrix of the target using the apparent feature vector and the heat map within the range; A matching module 420 is used to determine the weights of the IOU matrix and the apparent feature similarity matrix based on the number of consecutive successful track matching, calculate the similarity between each track and each target using the weights, and match the track and the target using the Hungarian matching method; The behavior recognition module 430 is used to input the historical appearance feature matrix set of the confirmed state trajectory into the time series network to obtain the behavior classification of the livestock.

[0035] In a possible implementation manner, determining the range in the apparent feature map based on the position of the target and the offset is specifically as follows: Get the length and width of the target, and convert them to the length and width on the heat map according to the current frame image size and heat map size; The position of the target plus the offset position is taken as the center point of the rectangle, the length and width of the rectangle are set as the length and width on the heat map, and the area of ​​the rectangle on the apparent feature map is taken as the range.

[0036] In a possible implementation manner, the use of the apparent feature vectors and the heat map within the range to determine the apparent feature matrix of the target is specifically: In the heat map, the points within the range are sorted according to the values ​​of the points in the heat map, and a preset number of points are selected from the sorted point set; Determine and select the positions of a preset number of points in the appearance feature map, and combine the appearance feature vectors of the positions into an appearance feature matrix of the target.

[0037] In a possible implementation, the weights of the IOU matrix and the appearance feature similarity matrix are determined based on the number of successful consecutive trajectory matching, specifically: Obtain the maximum number of trajectory mismatch retention times. If the number of consecutive successful matches is less than the maximum number of retention times, the weights of the IOU matrix and the apparent feature similarity matrix are the same; When the number of consecutive successful matches is greater than or equal to the maximum retention number, the weight of the IOU matrix increases by a preset value, and the weight of the apparent feature similarity matrix decreases by a preset value every time the number of consecutive successful matches increases by the maximum retention number; the weight of the IOU matrix and the weight of the apparent feature similarity matrix are between [0,1], and the sum of the two is 1.

[0038] In a possible implementation manner, after matching the trajectory and the target using the Hungarian matching method, the method further includes: If the target is not matched successfully, a new trajectory is created according to the target; If the track does not match successfully, if the track is in a confirmed state and the number of times the track has not been matched successfully is greater than the maximum number of retention times, the track will be deleted; if the track is in a confirmed state and the number of times the track has not been matched successfully is not greater than the maximum number of retention times, the track will be retained; if the track is in an unconfirmed state, the track will be deleted.

[0039] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the multiple embodiments disclosed in this specification can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0040] The specific implementation methods described above further illustrate in detail the purposes, technical solutions and beneficial effects of the multiple embodiments disclosed in this specification. It should be understood that the above description is only the specific implementation methods of the multiple embodiments disclosed in this specification, and is not used to limit the protection scope of the multiple embodiments disclosed in this specification. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the multiple embodiments disclosed in this specification should be included in the protection scope of the multiple embodiments disclosed in this specification.

Claims

1. A livestock behavior recognition method based on artificial intelligence, characterized in that: The method comprises: Obtaining the target position of livestock in the current frame of the livestock video and the offset of each target position through an encoder-decoder network, determining the range in the apparent feature map based on the target position and the offset, and determining the apparent feature matrix of the target using the apparent feature vector and the heat map within the range; The weights of the IOU matrix and the apparent feature similarity matrix are determined based on the number of successful consecutive track matchings, and the similarity between each track and each target is calculated using the weights, and the tracks and targets are matched using the Hungarian matching method; The historical apparent feature matrix set of the confirmed state trajectory is input into the time series network to obtain the behavior classification of livestock.

2. The method according to claim 1, characterized in that The determining of the range in the apparent feature map based on the position of the target and the offset is specifically: Get the length and width of the target, and convert them to the length and width on the heat map according to the current frame image size and heat map size; The position of the target plus the offset position is taken as the center point of the rectangle, the length and width of the rectangle are set as the length and width on the heat map, and the area of ​​the rectangle on the apparent feature map is taken as the range.

3. The method according to claim 1, characterized in that The apparent feature matrix of the target is determined by using the apparent feature vector and the heat map within the range, specifically: In the heat map, the points within the range are sorted according to the values ​​of the points in the heat map, and a preset number of points are selected from the sorted point set; Determine and select the positions of a preset number of points in the appearance feature map, and combine the appearance feature vectors of the positions into an appearance feature matrix of the target.

4. The method according to claim 1, characterized in that The weights of the IOU matrix and the apparent feature similarity matrix are determined based on the number of successful consecutive trajectory matching, specifically: Obtain the maximum number of trajectory mismatch retention times. If the number of consecutive successful matches is less than the maximum number of retention times, the weights of the IOU matrix and the apparent feature similarity matrix are the same; When the number of consecutive successful matches is greater than or equal to the maximum retention number, the weight of the IOU matrix increases by a preset value, and the weight of the apparent feature similarity matrix decreases by a preset value every time the number of consecutive successful matches increases by the maximum retention number; the weight of the IOU matrix and the weight of the apparent feature similarity matrix are between [0,1], and the sum of the two is 1.

5. The method according to claim 1, characterized in that After the trajectory and the target are matched by the Hungarian matching method, the method further includes: If the target is not matched successfully, a new trajectory is created according to the target; If the track does not match successfully, if the track is in a confirmed state and the number of times the track has not been matched successfully is greater than the maximum number of retention times, the track will be deleted; if the track is in a confirmed state and the number of times the track has not been matched successfully is not greater than the maximum number of retention times, the track will be retained; if the track is in an unconfirmed state, the track will be deleted.

6. A livestock behavior recognition system based on artificial intelligence, characterized in that: The system comprises: A feature acquisition module is used to obtain the target position of livestock and the offset of each target position in the current frame of the livestock video through an encoder-decoder network, determine the range in the apparent feature map based on the target position and the offset, and determine the apparent feature matrix of the target using the apparent feature vector and the heat map within the range; A matching module is used to determine the weights of the IOU matrix and the apparent feature similarity matrix based on the number of consecutive successful track matching, calculate the similarity between each track and each target using the weights, and match the track and the target using the Hungarian matching method; The behavior recognition module is used to input the historical appearance feature matrix set of the confirmed state trajectory into the time series network to obtain the behavior classification of livestock.

7. The system according to claim 6, characterized in that The determining of the range in the apparent feature map based on the position of the target and the offset is specifically: Get the length and width of the target, and convert them to the length and width on the heat map according to the current frame image size and heat map size; The position of the target plus the offset position is taken as the center point of the rectangle, the length and width of the rectangle are set as the length and width on the heat map, and the area of ​​the rectangle on the apparent feature map is taken as the range.

8. The system according to claim 6, characterized in that The apparent feature matrix of the target is determined by using the apparent feature vector and the heat map within the range, specifically: In the heat map, the points within the range are sorted according to the values ​​of the points in the heat map, and a preset number of points are selected from the sorted point set; Determine and select the positions of a preset number of points in the appearance feature map, and combine the appearance feature vectors of the positions into an appearance feature matrix of the target.

9. The system according to claim 6, characterized in that The weights of the IOU matrix and the apparent feature similarity matrix are determined based on the number of successful consecutive trajectory matching, specifically: Obtain the maximum number of trajectory mismatch retention times. If the number of consecutive successful matches is less than the maximum number of retention times, the weights of the IOU matrix and the apparent feature similarity matrix are the same; When the number of consecutive successful matches is greater than or equal to the maximum retention number, the weight of the IOU matrix increases by a preset value, and the weight of the apparent feature similarity matrix decreases by a preset value every time the number of consecutive successful matches increases by the maximum retention number; the weight of the IOU matrix and the weight of the apparent feature similarity matrix are between [0,1], and the sum of the two is 1.

10. The system according to claim 6, characterized in that After the trajectory and the target are matched by the Hungarian matching method, the method further includes: If the target is not matched successfully, a new trajectory is created according to the target; If the track does not match successfully, if the track is in a confirmed state and the number of times the track has not been matched successfully is greater than the maximum number of retention times, the track will be deleted; if the track is in a confirmed state and the number of times the track has not been matched successfully is not greater than the maximum number of retention times, the track will be retained; if the track is in an unconfirmed state, the track will be deleted.

Images