Under-film cotton seedling tracking and counting system and method
By introducing a self-attention pooling network and prior content information into the cotton seedling tracking and counting system, the problems of slow training and identity confusion in existing methods are solved, and efficient and accurate cotton seedling counting is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-04-03
AI Technical Summary
Existing cotton seedling counting methods based on large language model converters suffer from slow training convergence speed, unstable detection results, and severe identity confusion, especially under occlusion scenarios.
A tracking and counting system and method for cotton seedlings under film is adopted. Video frames are acquired through data acquisition equipment, and convolution processing, global context-enhanced encoding processing, self-attention pooling processing, and joint detection and association processing of identity and bounding box are performed using the tracking and counting equipment. Combined with prior content information, the detection capability of occluded targets is improved and identity confusion is reduced.
It improves tracking performance in occluded scenarios, reduces false negatives and false positives, enhances the ability to distinguish similar targets, significantly reduces the number of identity switching, and accelerates training convergence.
Smart Images
Figure CN121789031A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer modeling and systems technology, and more specifically, to a system and method for tracking and counting cotton seedlings under film. Background Technology
[0002] Cotton is one of the world's most important economic crops, widely used in textiles, medicine, food, and other fields. High-quality cotton production plays an indispensable role in economic development. Due to the large-scale application of drip irrigation technology under plastic film mulching, cotton cultivation has shifted to a new planting model. This model can conserve soil moisture, increase soil temperature, and suppress weed growth.
[0003] While this model significantly hinders the progress of automated visual inspection tasks, including multi-target tracking for cotton seedling counting, achieving accurate and effective counting remains a crucial task in precision agriculture. This is essential because it allows for quantitative assessment of seeding quality, crop emergence rates, and growth status in large-scale fields, thereby reducing reliance on labor-intensive manual methods and improving the scalability of agricultural monitoring.
[0004] With the rapid development of deep learning, existing multi-object tracking methods are mainly divided into two categories: detection-based tracking and joint detection and tracking. The former separates detection from association, and is characterized by frequent identity switching in severely occluded scenes; the latter is a method based on large language model transformer counting. Although it achieves end-to-end tracking, its object queries are usually randomly initialized, lacking prior knowledge of target features, resulting in slow training convergence speed and requiring a large number of iterations; moreover, the detection results are unstable, especially for occluded targets; and it also suffers from severe identity confusion in similar target scenes. Summary of the Invention
[0005] The technical problem to be solved by this invention is how to overcome the technical defects of existing methods based on large language model converters, such as slow training convergence speed, unstable detection results and serious identity confusion. In order to overcome these technical defects, this invention provides a cotton seedling tracking and counting system and method under film, specifically including a cotton seedling tracking and counting system under film and a cotton seedling tracking and counting method under film.
[0006] This invention provides a cotton seedling tracking and counting system under film, comprising: A data acquisition device is used to capture video data of cotton covered with film and to extract frames from the video data to obtain a continuous video frame image sequence. The tracking and counting device communicates with the data acquisition device and is used to sequentially perform convolution processing, global context-enhanced encoding processing, self-attention pooling processing, and joint detection and association processing of identity and bounding box on the continuous video frame image sequence to obtain the identity and bounding box information of cotton seedlings in each image; the self-attention pooling processing is a self-attention pooling processing guided by prior content information. The output display device communicates with the tracking and counting device to count the number of surviving cotton seedlings in each image based on the identity and bounding box information of the cotton seedlings in each image, and displays the statistical results.
[0007] The cotton seedling tracking and counting system disclosed in this invention includes a data acquisition device, a tracking and counting device, and an output display device. The data acquisition device extracts frames from video data to obtain a continuous sequence of video frame images. The tracking and counting device then sequentially performs convolution processing, global context-enhanced encoding, self-attention pooling guided by prior content information, and joint detection and association processing of identity and bounding boxes on the continuous video frame image sequence. This yields the identity and bounding box information of the cotton seedling in each image. Finally, the output display device visualizes the statistical results. The prior content information guidance enhances the detection capability of occluded targets, reduces false positives and false negatives, and avoids identity confusion, thereby improving tracking performance in occluded scenarios. Furthermore, with the assistance of prior information, the combination of convolution, encoding, and decoding processing enhances the tracking and counting device's ability to distinguish similar targets, significantly reduces the number of identity switching incidents, and improves identity preservation capabilities, overcoming the technical shortcomings of existing methods such as unstable detection results and severe identity confusion. Meanwhile, since the tracking counting device is based on convolution, encoding, pooling, decoding, and joint detection and association, parameter optimization can be focused on the pooling and joint detection and association processes, which greatly improves the training convergence speed compared to the traditional Transformer tracking method, thus achieving the technical effect of accelerating training convergence.
[0008] In one possible implementation, the data acquisition device includes: A mobile data acquisition device is used to capture video data of cotton covered with film along a specified path. The video frame extractor communicates with both the mobile acquisition unit and the tracking and counting device to extract frames from the video data at a predetermined rate and extract several images from each video frame to obtain a continuous video frame image sequence.
[0009] In one possible implementation, the mobile data collector is a drone equipped with a camera. Using a drone with a camera for real-time image acquisition can improve the efficiency of tracking and counting, enable rapid detection in large cotton fields, solve the problem of excessive labor costs in cotton yield measurement, and prepare for subsequent refined management and growth monitoring of cotton fields.
[0010] In one possible implementation, the data acquisition device further includes a data enhancement device. This data enhancement device is located on the communication line between the video frame extractor and the tracking and counting device, and communicates with both the video frame extractor and the tracking and counting device simultaneously. The data enhancement device is used to enhance the data of the continuous video frame image sequence. This can eliminate noise interference and further improve the seedling tracking and counting efficiency.
[0011] In one possible implementation, the tracking and counting device includes: The encoder architecture communicates with the video frame extractor and is used to sequentially perform convolution processing and encoding processing through the Transformer encoding algorithm on the continuous video frame image sequence to obtain the encoded output feature sequence. The decoder architecture communicates with both the encoder architecture and the output display device to sequentially perform self-attention pooling processing on the encoded output feature sequence and to obtain the identity and bounding box information through joint detection and association processing of identity and bounding box using a combination of Transformer decoding algorithm and bipartite graph matching algorithm.
[0012] In one possible implementation, the encoder architecture includes: The backbone network communicates with the video frame extractor and is used to perform convolution processing on the continuous video frame image sequence to extract the feature map of each image and obtain a series of feature maps. The first summing unit communicates with the backbone network and is used to incorporate positional encoding into the feature map series to obtain the input feature sequence. The Transformer encoder communicates with both the first summing unit and the decoder architecture to execute the Transformer encoding algorithm to perform global context-enhanced encoding on the input feature sequence to obtain the encoded output feature sequence.
[0013] In one possible implementation, the decoder architecture includes: A self-attention pooling network is used to perform self-attention pooling processing on the encoded output feature sequence to obtain an object query containing prior content information; The decoder-detector architecture communicates with the Transformer encoder, the self-attention pooling network, and the output display device to execute the Transformer decoding algorithm and the bipartite graph matching algorithm to convert the object query and the tracking query from the previous frame of video into the identity and bounding box information.
[0014] In one possible implementation, the self-attention pooling network includes an attention map projection network, a regulator, and a channel reweighting network, wherein the regulator communicates simultaneously with the attention map projection network, the channel reweighting network, and the decode-detection head architecture. The regulator is configured to operate as follows: first, the attention mapping projection network is invoked to map the encoded output feature sequence into an attention projection sequence; then, the attention projection sequence is mapped using the SoftMax function to obtain a normalized sequence; next, the normalized sequence and the encoded output feature sequence are concatenated by channel to obtain a concatenated sequence; then, the channel reweighting network is invoked to map the concatenated sequence into a channel reweighted sequence; finally, the channel reweighted sequence and the concatenated sequence are multiplied by Hada code to obtain the object query. The attention mapping projection network is configured to sequentially perform convolution, group normalization, ReLU activation, and convolution operations on the encoded output feature sequence to obtain the attention projection sequence. The channel reweighting network is configured to sequentially perform linear layer mapping, ReLU function activation, linear layer mapping, and Sigmoid function activation on the spliced sequence to obtain the channel reweighted sequence.
[0015] By setting up a self-attention pooling network with the aforementioned structure and functions to generate object queries with prior information, and by utilizing an attention mechanism to achieve feature reweighting, the model's ability to capture key features of cotton seedlings under occlusion conditions can be improved. A multi-level feature fusion strategy, combining content queries and location queries, achieves accurate target localization and identity association.
[0016] In one possible implementation, the decode-detection head architecture includes a Transformer decoder, a second summing unit, a detection head module, and a decoding controller that communicates simultaneously with the Transformer encoder, the Transformer decoder, the second summing unit, the regulator, and the output display device. The detection head module consists of two feedforward networks, and both feedforward networks communicate with the Transformer decoder to output the identity information and bounding box information of the cotton seedlings in each image, respectively. The Transformer decoder is used to execute the Transformer decoding algorithm; The decoding controller is configured to operate as follows: First, the Transformer decoder is invoked to map the combination of the object query and the encoded output feature sequence into a first embedding information sequence; then, the detection head module is invoked to convert the first embedding information sequence into a tracking query sequence consisting of identity information and bounding box information; next, the second summing unit is invoked to sum the first embedding information of the current video frame and the tracking query of the previous video frame, and the summing results are integrated into a summing result sequence according to the order of the video frames; then, the Transformer decoder is invoked to map the combination of the summing result sequence and the encoded output feature sequence into second embedding information; finally, the detection head module is invoked to convert the second embedding information sequence into an identity information sequence and a bounding box information sequence to obtain the identity and bounding box information of the cotton seedlings in each image.
[0017] The aforementioned decoder-detector architecture matches detection boxes with motion trajectories in trajectory association matching. Based on a bipartite graph matching strategy, it calculates the correlation between the predicted target and the actual target's motion state, thus eliminating detection boxes with low correlation to the average tracking results. However, simple motion association methods are prone to failure in cases of severe occlusion. Therefore, in addition to motion information, prior content information is introduced for matching. During feature extraction, a self-attention mechanism is used to reweight the channel weights, combining content features with motion features to avoid interference from highly similar seedling targets or thin film backgrounds.
[0018] The tracking and counting device with the above structure and functions adopts an end-to-end multi-target tracking framework based on Transformer. It introduces a prior content information generation mechanism on the basis of TrackFormer algorithm. While focusing on target motion, it uses a self-attention pooling network to generate object queries containing prior information, thereby improving the model's ability to identify occluded targets and improving tracking accuracy.
[0019] Another technical solution of the present invention is to provide a method for tracking and counting cotton seedlings under film, comprising the following steps: S1: Construct a video frame image sequence dataset with a continuous video frame image sequence as input and an image sequence formed by annotating each image in the continuous video frame image sequence with a cotton seedling bounding box and a unique seedling identifier as the ideal output; S2: Using the video frame image sequence dataset, the parameters of the tracking and counting device are optimized by solving the optimal value of the matching cost function between the cotton seedling bounding box and the seedling's unique identifier. S3: Acquire video data of the cotton film-coated material using a data acquisition device, and extract frames from the video data to obtain a continuous video frame image sequence; S4: The continuous video frame image sequence is sequentially processed by convolution, global context-enhanced encoding, self-attention pooling, and joint detection and association of identity and bounding box to obtain the identity and bounding box information of cotton seedlings in each image through the parameter-optimized tracking and counting device. S5: Calculate the number of surviving cotton seedlings in each image using an output display device based on the identity and bounding box information of the cotton seedlings in each image, and display the statistical results.
[0020] The cotton seedling tracking and counting method disclosed in this invention first constructs a dataset, with the ideal output being image information labeled with cotton seedling bounding boxes and unique seedling identifiers. Then, using the constructed dataset, the parameters of the tracking and counting device are optimized by solving for the optimal value of the matching cost function between the cotton seedling bounding boxes and unique seedling identifiers, thereby ensuring efficient and feasible counting. Subsequently, a data acquisition device extracts frames from video data to obtain a continuous sequence of video frame images. The tracking and counting device then sequentially performs convolution processing, global context-enhanced encoding, self-attention pooling guided by prior content information, and joint detection and association processing of identity and bounding boxes on the continuous video frame image sequence, thereby obtaining the identity and bounding box information of the cotton seedlings in each image. Finally, the statistical results are visualized through a designated output display device. The prior content information guidance enhances the detection capability of occluded targets, reduces missed detections and false detections, avoids identity confusion, and thus improves tracking performance in occluded scenarios. Furthermore, with the aid of prior information, the combination of convolution, encoding, and decoding processes enhances the ability of the tracking and timing device to distinguish similar targets, significantly reduces the number of identity switching incidents, and thus improves identity preservation capabilities, overcoming the technical shortcomings of unstable detection results and severe identity confusion in existing methods. Simultaneously, since the tracking and counting device is based on convolution, encoding, pooling, decoding, and joint detection and association, parameter optimization can be focused on the pooling and joint detection and association processes, resulting in a significant improvement in training convergence speed compared to traditional Transformer tracking methods, achieving the technical effect of accelerating training convergence. Attached Figure Description
[0021] Figure 1 This is a schematic diagram of a cotton seedling tracking and counting system under film disclosed in an embodiment of the present invention; Figure 2 This is a schematic diagram of the tracking and counting device structure disclosed in the embodiments of the present invention; Figure 3 This is a flowchart illustrating the operation of the tracking and counting device as disclosed in the embodiments of the present invention; Figure 4 This is a schematic diagram of the self-attention pooling network structure disclosed in the embodiments of the present invention; Figure 5 This is a flowchart illustrating the operation of the self-attention pooling network as disclosed in the embodiments of the present invention. Figure 6 This is a flowchart of the method disclosed in the embodiments of the present invention. Detailed Implementation
[0022] First, those skilled in the art should understand that these embodiments are merely used to explain the technical principles of the embodiments of this application and are not intended to limit the scope of protection of the embodiments of this application. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.
[0023] In the description of the embodiments of this application, it should be noted that, unless otherwise explicitly specified and limited, the term "forming a communication link structure" means that the multiple communication elements or modules involved form a network structure or network link structure through communication connection. Communication or communication connection means that there is information transmission between the first feature and the second feature. This information transmission can be unidirectional or bidirectional. The way to realize the communication connection can be electrical connection of wires, radio connection, electrical connection of electromagnetic media (such as optical fiber, semiconductor), communication realized by special channels, etc. It can be direct signal transmission or signal transmission through intermediate devices.
[0024] In the embodiments of this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0025] The present application will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0026] See Figures 1-6 This application discloses a cotton seedling tracking and counting system under plastic film. Figure 1 This is a schematic diagram of the cotton seedling tracking and counting system. The cotton seedling tracking and counting system includes a data acquisition device, a tracking and counting device, and an output display device. The tracking and counting device communicates with the data acquisition device, and the output display device communicates with the tracking and counting device.
[0027] See Figure 1 In this cotton seedling tracking and counting system, a data acquisition device is used to capture video data of the cotton seedlings covered with film and extract frames from the video data to obtain a continuous sequence of video frame images. In this embodiment, the data acquisition device includes a mobile acquisition unit, a video frame extractor, and a data enhancement device. The video frame extractor communicates with the mobile acquisition unit, and the data enhancement device is located on the communication line between the video frame extractor and the tracking and counting device, and communicates with both the video frame extractor and the tracking and counting device simultaneously.
[0028] In the data acquisition equipment, a mobile data acquisition device is used to capture video data of cotton covered with plastic film along a designated path. In this embodiment, the mobile data acquisition device is a drone equipped with a camera, with a shooting height of 30 cm, a resolution of 640×480, and a frame rate of 30fps. Since most cotton fields are vast, with relatively flat but large areas, and require real-time detection and counting, considering the limitations of wired equipment, a drone equipped with a camera is used for real-time video acquisition. Mobile data acquisition via drone can improve the efficiency of tracking and counting, enabling rapid detection in large cotton fields, solving the problem of excessively high labor costs in cotton yield measurement, and preparing for subsequent refined management and growth monitoring of cotton fields.
[0029] In a data acquisition device, a video frame extractor is used to extract frames from video data at a predetermined rate and extract several images from each video frame to obtain a continuous video frame image sequence. Specifically, in this embodiment, the video frame extractor uses the FFmpeg tool equipped on it to extract frames from each video segment at a rate of 1 frame per second, and extracts several images (in .jpg format) from each video frame.
[0030] In data acquisition equipment, data augmentation devices are used to enhance the data of consecutive video frame sequences to avoid noise interference and further improve counting accuracy and efficiency. Data augmentation methods include brightness adjustment, vertical image mirroring, horizontal image mirroring, and Gaussian blur.
[0031] See Figure 1 , Figure 2 and Figure 3 In this cotton seedling tracking and counting system, the tracking and counting device is used to sequentially perform convolution processing, global context-enhanced encoding processing, self-attention pooling processing guided by prior content information, and joint detection and association processing of identity and bounding box to obtain the identity and bounding box information of cotton seedlings in each image. For example... Figure 1 As shown, in this embodiment, the tracking and counting device includes an encoder architecture and a decoder architecture. The encoder architecture communicates with the video frame extractor, while the decoder architecture communicates with both the encoder architecture and the output display device. The arrows in the figure indicate the direction of data flow.
[0032] See Figure 2 and Figure 3 In tracking and counting devices, the encoder architecture is used to sequentially perform convolution processing on a continuous sequence of video frame images and then encode them using the Transformer coding algorithm to obtain the encoded output feature sequence, such as... Figure 2 As shown, in this embodiment, the encoder architecture includes a backbone network, a first summing unit, and a Transformer encoder, wherein the backbone network communicates with the video frame extractor (as described above and below). Figure 1 As can be seen, data enhancement devices are installed on the communication lines of the two (therefore, this is indirect communication). The first summing unit communicates with the backbone network, and the Transformer encoder communicates with both the first summing unit and the decoder architecture.
[0033] See Figure 3 In the encoder architecture, the backbone network uses a convolutional neural network to perform convolutional processing on a sequence of consecutive video frame images to extract feature maps for each image, resulting in a feature map series. Alternatively, a ResNet-50 backbone network can be used. The first summing unit incorporates positional encoding into the feature map series to obtain the input feature sequence. The Transformer encoder executes the Transformer encoding algorithm to perform global context-enhanced encoding on the input feature sequence, obtaining the encoded output feature sequence, denoted as... ,in, Represents the video frame number.
[0034] See Figure 2 , Figure 3 and Figure 4 In tracking and counting devices, the decoder architecture is used to encode the output feature sequence. The process involves sequentially performing self-attention pooling and then jointly detecting and associating identities and bounding boxes using a combination of Transformer decoding and bipartite graph matching algorithms to obtain identity and bounding box information. For example... Figure 2 As shown, in this embodiment, the decoder architecture includes a self-attention pooling network (SAPN) and a decoder-detector architecture, wherein the decoder-detector architecture communicates with the Transformer encoder, the self-attention pooling network and the output display device simultaneously.
[0035] See Figure 4 and Figure 5 In the decoder architecture, a self-attention pooling network is used to process the encoded output feature sequence. Perform self-attention pooling to obtain an object query containing prior information, denoted as... . Figure 5In this context, h represents the image height, w represents the width, and c represents the number of channels. For example... Figure 4 As shown, the self-attention pooling network includes an attention map projection network, a regulator, and a channel reweighting network. The regulator communicates simultaneously with the attention map projection network, the channel reweighting network, and the decode-detection head architecture. Prior content information is defined as: , in, This represents a self-attention pooling network, which consists of a series of convolutional layers; It is a normalization function; Indicates a channel-reweighted network; Indicates the encoded output feature sequence The feature map of the nth layer.
[0036] like Figure 5 As shown, in a self-attention pooling network, the regulator is configured to operate as follows: first, the attention map projection network is invoked to encode the output feature sequence. The sequence is mapped to an attention projection sequence; then, the attention projection sequence is mapped using the SoftMax function to obtain a normalized sequence; next, the normalized sequence and the encoded output feature sequence are concatenated by channel to obtain a concatenated sequence; then, a channel reweighting network is called to map the concatenated sequence to a channel reweighted sequence; finally, the channel reweighted sequence and the concatenated sequence are multiplied by Hada code to obtain the object query. .
[0037] like Figure 5 As shown, in a self-attention pooling network, the attention mapping projection network is configured to encode the output feature sequence. The attention projection sequence is obtained by sequentially performing convolution, group normalization, ReLU activation, group normalization, ReLU activation, and convolution. The channel reweighting network is configured to sequentially perform linear layer mapping, ReLU activation, linear layer mapping, and Sigmoid activation on the concatenated sequence to obtain the channel reweighted sequence.
[0038] See Figure 2 and Figure 3 In the decoder architecture, the decoder-detector head architecture is used to execute the Transformer decoding algorithm and the bipartite graph matching algorithm to query objects. The tracking query from the previous frame of video is transformed into identity and bounding box information. For example... Figure 2As shown, in this embodiment, the decode-detection head architecture includes a Transformer decoder, a second summing unit, a detection head module, and a decoding controller that communicates simultaneously with the Transformer encoder, Transformer decoder, second summing unit, regulator, and output display device. The detection head module consists of two feedforward networks, both of which communicate with the Transformer decoder to output the identification information and bounding box information of the cotton seedlings in each image, respectively. The Transformer decoder is used to execute the Transformer decoding algorithm.
[0039] like Figure 3 As shown, in the decoder architecture, the decoder controller is configured to operate as follows: First, the Transformer decoder is called to query the object. With the encoded output feature sequence The combined mapping is the first embedded information sequence, denoted as . Subsequently, the detection head module is invoked to transform the first embedded information sequence into a tracking query sequence composed of identity information and bounding box information, denoted as... Next, the second summation unit is called to sum the first embedded information of the current video frame and the tracking query of the previous video frame, and the summation results are integrated into a sequence of summation results according to the order of the video frames, denoted as [ , Next, the Transformer decoder is called to sum the result sequence. , ] and encoded output feature sequence The combination mapping is used to obtain the second embedded information; finally, the detection head module is called to convert the second embedded information sequence into an identity information sequence and a bounding box information sequence to obtain the identity and bounding box information of the cotton seedlings in each image.
[0040] from Figure 3 It is not difficult to see that the tracking and counting device in this embodiment is based on a novel and improved version of the TrackFormer algorithm. Since cotton planting areas are mostly located in large-scale farmland environments with complex natural conditions, and the number of cotton seedlings covered by film is large and severely affected by the film shading, manual counting is necessary, consuming a large amount of manpower and having limited accuracy. Therefore, this embodiment uses an improved trackformer algorithm combined with an edge computing platform to track and count cotton seedlings covered by film, which has advantages such as high portability and fast detection speed.
[0041] The improved trajectory tracking algorithm adopts an end-to-end multi-target tracking framework based on Transformer. On this basis, it is improved by introducing a prior content information generation mechanism. While focusing on the target motion, it uses a self-attention pooling network to generate object queries containing prior information, thereby improving the model's ability to identify occluded targets and improving tracking accuracy.
[0042] During real-time monitoring of cotton seedlings covered with plastic film in cotton fields, the seedlings can be lost due to factors such as film shading and changes in light intensity, leading to continuous changes in the identity of the tracked seedlings. Traditional query-based tracking methods, due to random initialization of object queries and lack of prior information, perform poorly in complex scenarios. Therefore, this embodiment generates object queries with prior information using a self-attention pooling network, and utilizes an attention mechanism to reweight features, improving the model's ability to capture key features of cotton seedlings under shading conditions. A multi-level feature fusion strategy is adopted, combining content queries and location queries to achieve accurate target localization and identity association.
[0043] In trajectory association matching, a bipartite graph matching strategy is used to calculate the correlation between the predicted target and the actual target's motion state, which can exclude detection boxes with low correlation to the average tracking results. However, simple motion association methods are prone to failure when encountering severe occlusion. Therefore, in addition to motion information, prior content information is introduced for matching association. During feature extraction, a self-attention mechanism is used to reweight the channel weights, combining content features with motion features, which can avoid interference from highly similar seedling targets or thin film backgrounds.
[0044] This embodiment uses an improved trajectory tracking algorithm to track, detect, and count cotton seedlings covered with film. The improved trajectory tracking algorithm detects the target and assigns the identity of the detected cotton seedlings through a trajectory query mechanism, and tracks each cotton seedling with a different identity in the video sequence.
[0045] See Figure 1 In this cotton seedling tracking and counting system, the output display device is used to count the number of surviving cotton seedlings in each image based on the identity and bounding box information of the cotton seedlings in each image, and to display the statistical results. Specifically, in this embodiment, the output display device includes a statistical unit and a display that communicate with each other. The statistical unit is used to execute a counting algorithm to count the number of surviving cotton seedlings in each image based on the identity and bounding box information of the cotton seedlings in each image, thereby obtaining the statistical results; the display is used to display the obtained statistical results for user review.
[0046] See Figure 6The cotton seedling tracking and counting method corresponding to the cotton seedling tracking and counting system under film described in this embodiment is further disclosed below. The flowchart of the method is as follows: Figure 6 As shown, it includes the following steps: S1: Construct a video frame image sequence dataset that takes a continuous sequence of video frame images as input and outputs an image sequence in which each image in the continuous video frame image sequence is labeled with a cotton seedling bounding box and a unique identifier for the seedling.
[0047] In the process of obtaining the continuous video frame image sequence in step S1, the video was captured at a height of 30 cm, a resolution of 640×480, and a frame rate of 30 fps. FFmpeg was used to extract frames from each video segment at a rate of 1 frame per second, extracting several images (in .jpg format) from each segment. Labelme software was then used to annotate the extracted cotton seedling images, marking targets in the images with rectangular boxes and adding target labels, thus obtaining the video frame image sequence dataset.
[0048] S2: Using a video frame image sequence dataset, the parameters of the tracking and counting device are optimized by solving for the optimal value of the matching cost function between the cotton seedling bounding box and the seedling's unique identifier.
[0049] In this embodiment, the matching cost function is: , in, To match the cost function, These represent the actual value and the predicted value, respectively. Indicate target The predicted probability, For the true bounding box and predicted bounding boxes The generalized intersection-union cost function, It is an L1 norm. , , It's a hyperparameter.
[0050] S3: Acquire video data of the cotton covered with film through a data acquisition device, and extract frames from the video data to obtain a continuous video frame image sequence.
[0051] Specifically, video data of cotton covered with film is captured by drones, and then frames are extracted from the video data using a video frame extractor to obtain a continuous sequence of video frame images. Data augmentation equipment can also be used for data enhancement.
[0052] S4: The continuous video frame image sequence is sequentially processed by convolution, global context-enhanced encoding, self-attention pooling, and joint detection and association of identity and bounding box to obtain the identity and bounding box information of cotton seedlings in each image through the parameter-optimized tracking and counting device.
[0053] like Figure 3 As shown, specifically, a convolutional neural network is first used to extract multi-scale features from each frame of the image to obtain a feature map sequence. Then, a first summing unit incorporates positional encoding into the feature map sequence to obtain an input feature sequence. Next, a Transformer encoder performs global context enhancement on the input feature sequence. The enhanced features are then input into an attention pooling network to generate an object query containing prior information. Finally, a decoding controller merges the object query with a tracking query from the previous frame, inputting it into the Transformer decoder for joint detection and association, thereby obtaining the identity and bounding box information of the cotton seedlings in each image.
[0054] S5: Calculate the number of surviving cotton seedlings in each image using an output display device based on the identity and bounding box information of the cotton seedlings in each image, and display the statistical results.
[0055] The cotton seedling tracking and counting system disclosed in this embodiment includes a data acquisition device, a tracking and counting device, and an output display device. The data acquisition device extracts frames from video data to obtain a continuous sequence of video frame images. The tracking and counting device then sequentially performs convolution processing, global context-enhanced encoding, self-attention pooling guided by prior content information, and joint detection and association processing of identity and bounding boxes on the continuous video frame image sequence. This yields the identity and bounding box information of the cotton seedlings in each image. Finally, the output display device visualizes the statistical results. The prior content information guidance enhances the detection capability of occluded targets, reduces missed detections and false detections, and avoids identity confusion, thereby improving tracking performance in occluded scenarios. Furthermore, with the assistance of prior information, combined with convolution, encoding, and decoding processing, the tracking and counting device enhances its ability to distinguish similar targets, significantly reduces the number of identity switching incidents, and improves identity preservation capabilities, overcoming the technical shortcomings of existing methods such as unstable detection results and severe identity confusion. Meanwhile, since the tracking counting device is based on convolution, encoding, pooling, decoding, and joint detection and association, parameter optimization can be focused on the pooling and joint detection and association processes, which greatly improves the training convergence speed compared to the traditional Transformer tracking method, thus achieving the technical effect of accelerating training convergence.
[0056] In the description of the embodiments of this application, it should be noted that the terms "inner" and "outer" and other terms indicating direction or positional relationship are based on the direction or positional relationship shown in the drawings. This is only for the convenience of description and does not indicate or imply that the device or component must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this application.
[0057] In the description of this application, the references to terms such as "an embodiment," "some embodiments," "in this embodiment," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0058] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A cotton seedling tracking and counting system under plastic film, characterized in that, include: A data acquisition device is used to capture video data of cotton covered with film and to extract frames from the video data to obtain a continuous video frame image sequence. The tracking and counting device communicates with the data acquisition device and is used to sequentially perform convolution processing, global context-enhanced encoding processing, self-attention pooling processing, and joint detection and association processing of identity and bounding box on the continuous video frame image sequence to obtain the identity and bounding box information of cotton seedlings in each image; the self-attention pooling processing is a self-attention pooling processing guided by prior content information. The output display device communicates with the tracking and counting device to count the number of surviving cotton seedlings in each image based on the identity and bounding box information of the cotton seedlings in each image, and displays the statistical results.
2. The cotton seedling tracking and counting system under film as described in claim 1, characterized in that, The data acquisition device includes: A mobile data acquisition device is used to capture video data of cotton covered with film along a specified path. The video frame extractor communicates with both the mobile acquisition unit and the tracking and counting device to extract frames from the video data at a predetermined rate and extract several images from each video frame to obtain a continuous video frame image sequence.
3. The cotton seedling tracking and counting system under film as described in claim 2, characterized in that, The mobile data collector is a drone equipped with a camera.
4. The cotton seedling tracking and counting system under film as described in claim 2 or 3, characterized in that, The data acquisition device also includes a data enhancement device, which is installed on the communication line between the video frame extractor and the tracking counting device, and communicates with both the video frame extractor and the tracking counting device. The data enhancement device is used to perform data enhancement on the continuous video frame image sequence.
5. The cotton seedling tracking and counting system under film as described in claim 4, characterized in that, The tracking and counting device includes: The encoder architecture communicates with the video frame extractor and is used to sequentially perform convolution processing and encoding processing through the Transformer encoding algorithm on the continuous video frame image sequence to obtain the encoded output feature sequence. The decoder architecture communicates with both the encoder architecture and the output display device to sequentially perform self-attention pooling processing on the encoded output feature sequence and to obtain the identity and bounding box information through joint detection and association processing of identity and bounding box using a combination of Transformer decoding algorithm and bipartite graph matching algorithm.
6. The cotton seedling tracking and counting system under film as described in claim 5, characterized in that, The encoder architecture includes: The backbone network communicates with the video frame extractor and is used to perform convolution processing on the continuous video frame image sequence to extract the feature map of each image and obtain a series of feature maps. The first summing unit communicates with the backbone network and is used to incorporate positional encoding into the feature map series to obtain the input feature sequence. The Transformer encoder communicates with both the first summing unit and the decoder architecture to execute the Transformer encoding algorithm to perform global context-enhanced encoding on the input feature sequence to obtain the encoded output feature sequence.
7. The cotton seedling tracking and counting system under film as described in claim 6, characterized in that, The decoder architecture includes: A self-attention pooling network is used to perform self-attention pooling processing on the encoded output feature sequence to obtain an object query containing prior content information; The decoder-detector architecture communicates with the Transformer encoder, the self-attention pooling network, and the output display device to execute the Transformer decoding algorithm and the bipartite graph matching algorithm to convert the object query and the tracking query from the previous frame of video into the identity and bounding box information.
8. The cotton seedling tracking and counting system under film as described in claim 7, characterized in that, The self-attention pooling network includes an attention map projection network, a regulator, and a channel reweighting network. The regulator communicates with the attention map projection network, the channel reweighting network, and the decode-detection head architecture simultaneously. The regulator is configured to operate as follows: first, the attention mapping projection network is invoked to map the encoded output feature sequence into an attention projection sequence; then, the attention projection sequence is mapped using the SoftMax function to obtain a normalized sequence; next, the normalized sequence and the encoded output feature sequence are concatenated by channel to obtain a concatenated sequence; then, the channel reweighting network is invoked to map the concatenated sequence into a channel reweighted sequence; finally, the channel reweighted sequence and the concatenated sequence are multiplied by Hada code to obtain the object query. The attention mapping projection network is configured to sequentially perform convolution, group normalization, ReLU activation, and convolution operations on the encoded output feature sequence to obtain the attention projection sequence. The channel reweighting network is configured to sequentially perform linear layer mapping, ReLU function activation, linear layer mapping, and Sigmoid function activation on the spliced sequence to obtain the channel reweighted sequence.
9. The cotton seedling tracking and counting system under film as described in claim 8, characterized in that, The decode-detection head architecture includes a Transformer decoder, a second summing unit, a detection head module, and a decoder controller that communicates with the Transformer encoder, the Transformer decoder, the second summing unit, the regulator, and the output display device. The detection head module consists of two feedforward networks, and both feedforward networks communicate with the Transformer decoder to output the identity information and bounding box information of the cotton seedlings in each image, respectively. The Transformer decoder is used to execute the Transformer decoding algorithm; The decoding controller is configured to operate as follows: First, the Transformer decoder is invoked to map the combination of the object query and the encoded output feature sequence into a first embedding information sequence; then, the detection head module is invoked to convert the first embedding information sequence into a tracking query sequence consisting of identity information and bounding box information; next, the second summing unit is invoked to sum the first embedding information of the current video frame and the tracking query of the previous video frame, and the summing results are integrated into a summing result sequence according to the order of the video frames; then, the Transformer decoder is invoked to map the combination of the summing result sequence and the encoded output feature sequence into second embedding information; finally, the detection head module is invoked to convert the second embedding information sequence into an identity information sequence and a bounding box information sequence to obtain the identity and bounding box information of the cotton seedlings in each image.
10. A method for tracking and counting cotton seedlings under plastic film, characterized in that, The cotton seedling tracking and counting system under film as described in any one of claims 1-9 comprises the following steps: S1: Construct a video frame image sequence dataset with a continuous video frame image sequence as input and an image sequence formed by annotating each image in the continuous video frame image sequence with a cotton seedling bounding box and a unique seedling identifier as the ideal output; S2: Using the video frame image sequence dataset, the parameters of the tracking and counting device are optimized by solving the optimal value of the matching cost function between the cotton seedling bounding box and the seedling's unique identifier. S3: Acquire video data of the cotton film-coated material using a data acquisition device, and extract frames from the video data to obtain a continuous video frame image sequence; S4: The continuous video frame image sequence is sequentially processed by convolution, global context-enhanced encoding, self-attention pooling, and joint detection and association of identity and bounding box to obtain the identity and bounding box information of cotton seedlings in each image through the parameter-optimized tracking and counting device. S5: Calculate the number of surviving cotton seedlings in each image using an output display device based on the identity and bounding box information of the cotton seedlings in each image, and display the statistical results.