A target counting method, device, terminal and computer readable storage medium

Abstract

The application provides a target counting method, device, terminal and computer readable storage medium. The target counting method comprises: performing target detection on a current video frame to obtain detection information of a target object contained in the current video frame; determining motion variable information between the target object and a historical target based on position information of the target object in the current video frame and position information of the historical target with the same category information as the target object in a historical video frame through a neighborhood attention module; determining whether the target object and the historical target are the same based on the motion variable information; and in response to the target object and the historical target being different, counting the target object. The neighborhood attention module of the application can obtain more rich features of the target object and the historical target, thereby improving the detection accuracy of the motion variable information, and then determining whether the target object and the historical target in different frames are the same target, so as to avoid repeated counting.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a target counting method, apparatus, terminal, and computer-readable storage medium. Background Technology

[0002] With the development of science and technology, my country's aquaculture industry is undergoing a transformation from traditional to modern aquaculture, with mechanization, automation, and intelligentization gradually being promoted in practice. Currently, there is a strong demand for automatic fish counting in aquaculture enterprises. Fish counting is a fundamental operation for biomass estimation in aquaculture, helping farmers accurately calculate reproduction rates and estimate production potential. It also provides excellent guidance for survival rate assessment, stocking density control, and transportation and sales management.

[0003] Currently, research on fish counting, both domestically and internationally, can be categorized into visual and non-visual studies based on their methodologies. Visual studies primarily involve capturing images of fish schools using underwater cameras and analyzing these images to determine the fish population. Non-visual studies mainly utilize various sensors (such as infrared optical counters and resistivity fish counters) and sonar equipment to analyze signal changes generated by fish movement to determine fish populations. The advantages of non-visual studies are relatively sensitive fish signal capture and relatively simple and mature counting algorithms. However, their disadvantages include the fact that the signals generated by these devices often affect fish movement, altering fish distribution and even habits. Furthermore, for overlapping or densely packed fish schools, signal interference can prevent accurate results. Visual studies, on the other hand, avoid the influence of non-visual methods on the fish school itself and can address the counting problem in densely packed fish schools to some extent, thus becoming a current research hotspot both domestically and internationally. Summary of the Invention

[0004] The main technical problem solved by this invention is to provide a target counting method, device, terminal and computer-readable storage medium to solve the problem of inaccurate target counting in the prior art.

[0005] To solve the above-mentioned technical problems, the first technical solution adopted by the present invention is: to provide a target counting method, the target counting method comprising:

[0006] Perform target detection on the current video frame to obtain detection information of the target objects contained in the current video frame; the detection information includes location information and category information.

[0007] The neighborhood attention module determines the motion variable information between the target object and historical targets based on the position information of the target object in the current video frame and the position information of historical targets with the same category information as the target object in the previous video frames.

[0008] Based on the motion variable information between the target object and the historical target, determine whether the target object and the historical target are the same;

[0009] If the target object is different from the historical target, then the target object is counted.

[0010] Specifically, the neighborhood attention module determines the motion variable information between the target object and historical targets based on the position information of the target object in the current video frame and the position information of historical targets with the same category information as the target object in previous video frames. This includes:

[0011] Based on the location information of each target object in the current video frame and the location information of each historical target with the same category information as the target object in the historical video frames, an attention vector is generated between the target object and the historical targets; the attention vector includes the key value K, the query Q and the value V.

[0012] Based on the attention vectors corresponding to the target object and historical targets, determine the probability value that the target object and historical targets are the same target;

[0013] Based on the attention vectors and probability values ​​corresponding to the target object and historical targets, the motion variable information between the target object and historical targets is determined.

[0014] Among them, motion variable information includes position offset and angle information;

[0015] Based on the motion variable information between the target object and historical targets, determine whether the target object and historical targets are the same, including:

[0016] If the positional offset between the target object and the historical target meets the offset threshold and the angle information meets the angle threshold, then the target object and the historical target are determined to be the same target.

[0017] The neighborhood attention module is a Transformer attention model.

[0018] Specifically, target detection is performed on the current video frame to obtain detection information of the target objects contained in the current video frame, including:

[0019] Feature extraction of video frames is performed using an object detection network to obtain multiple target feature maps at different scales;

[0020] The visual attention module introduced in the object detection network convolves each target feature map based on a matched convolution kernel to obtain the attention feature map corresponding to each target feature map.

[0021] The attention feature maps corresponding to each target feature map are fused to obtain a fused feature map;

[0022] Target detection is performed on the fused feature map to obtain the detection information of each target object contained in the video frame.

[0023] The object detection network includes M cascaded residual modules and M visual attention modules, with each residual module connected to a visual attention module, where M is a positive integer.

[0024] Feature extraction is performed on video frames using an object detection network to obtain multiple target feature maps at different scales, including:

[0025] The residual module extracts features from the current video frame, resulting in target feature maps of different scales output by each residual module.

[0026] The visual attention module introduced in the object detection network convolves each target feature map using matched convolution kernels to obtain the attention feature maps corresponding to each target feature map, including:

[0027] Each visual attention module selects a matching convolution kernel based on the scale of the target feature map and performs convolution processing on the target feature map to obtain the attention feature map corresponding to each target feature map.

[0028] Before the step of performing target detection on the current video frame to obtain the detection information of the target objects contained in the current video frame, the method further includes:

[0029] A target detection network is constructed based on the YOLOv4 network and the SKNet attention model;

[0030] Prune the target detection network to remove low-weight channels and / or remove deep residual modules.

[0031] To solve the above-mentioned technical problems, the second technical solution adopted by the present invention is: to provide a target counting device, the target counting device comprising:

[0032] The detection module is used to perform target detection on the current video frame and obtain the detection information of the target objects contained in the current video frame; the detection information includes location information and category information.

[0033] The analysis module is used to determine the motion variable information between the target object and historical targets by using the neighborhood attention module based on the position information of the target object in the current video frame and the position information of historical targets with the same category information as the target object in the previous video frames.

[0034] The processing module is used to determine whether the target object and the historical target are the same based on the motion variable information between the target object and the historical target.

[0035] The determination module is used to count the target object in response to the target object being different from the historical target.

[0036] To solve the above-mentioned technical problems, the third technical solution adopted by the present invention is to provide a terminal, the terminal including a memory, a processor and a computer program stored in the memory and running on the processor, the processor being used to execute program data to implement the steps in the target counting method as described above.

[0037] To solve the above-mentioned technical problems, the fourth technical solution adopted by the present invention is to provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps in the target counting method described above.

[0038] The beneficial effects of this invention are as follows: Unlike existing technologies, this invention provides a target counting method, apparatus, terminal, and computer-readable storage medium. The target counting method includes: performing target detection on the current video frame to obtain detection information of target objects contained in the current video frame; the detection information includes location information and category information; determining motion variable information between the target object and historical targets based on the location information of the target object in the current video frame and the location information of historical targets with the same category information as the target object in previous historical video frames using a neighborhood attention module; determining whether the target object and historical targets are the same based on the motion variable information between them; and counting the target object if it is different from the historical targets. This application uses a neighborhood attention module to determine the corresponding motion variable information based on the location information of target objects and historical targets with the same category information. The neighborhood attention module can acquire richer features of the target object and historical targets, thereby improving the detection accuracy of motion variable information and determining whether the target object and historical targets in different frames are the same target, thus avoiding duplicate counting and improving the target counting accuracy. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the target counting method provided by the present invention;

[0041] Figure 2 yes Figure 1 A flowchart illustrating a specific embodiment of step S2 in the provided target counting method;

[0042] Figure 3 This is a flowchart illustrating a specific embodiment of the target counting method provided by the present invention;

[0043] Figure 4 This is a schematic diagram of the framework of an embodiment of the target counting device provided by the present invention;

[0044] Figure 5 This is a schematic diagram of the framework of an embodiment of the terminal provided by the present invention;

[0045] Figure 6 A schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present invention. Detailed Implementation

[0046] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0047] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0048] In this article, the term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "more" in this article means two or more objects.

[0049] To enable those skilled in the art to better understand the technical solution of the present invention, the target counting method provided by the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Please see Figure 1 , Figure 1 This is a flowchart illustrating the target counting method provided by the present invention.

[0051] This embodiment provides a target counting method, which is particularly suitable for counting aquatic targets such as schools of fish under conditions of underwater imaging blur, texture distortion, or dense fish populations. The target counting method includes the following steps.

[0052] S1: Perform target detection on the current video frame to obtain the detection information of the target objects contained in the current video frame; the detection information includes location information and category information.

[0053] S2: The neighborhood attention module determines the motion variable information between the target object and historical targets based on the position information of the target object in the current video frame and the position information of historical targets with the same category information as the target object in the previous historical video frames.

[0054] S3: Based on the motion variable information between the target object and the historical target, determine whether the target object and the historical target are the same.

[0055] S4: If the target object is different from the historical target, then count the target object.

[0056] In one embodiment, the specific implementation of obtaining the detection information of the target object contained in the current video frame in step S1 includes the following steps.

[0057] Specifically, underwater cameras capture images of a predetermined water area to obtain a corresponding video stream. The video stream consists of multiple consecutive video frames. A predetermined region within each video frame is used as a statistical region. For example, the middle region of each video frame is used as the statistical region. The statistical regions within the video frames contain groups of target objects. These target objects can be aquatic organisms such as fish, crabs, and shrimp.

[0058] The object detection network, which incorporates a visual attention module, performs object detection on each video frame in the video stream, obtaining detection information for each target object contained in each video frame. This detection information includes location and category information.

[0059] In one embodiment, the object detection network includes M cascaded residual modules and M visual attention modules, with each residual module connected to a visual attention module, where M is a positive integer. The residual modules extract features from the current video frame to obtain target feature maps of different scales output by each residual module. Each visual attention module selects a matching convolution kernel based on the scale of the target feature map and performs convolution processing on the target feature map to obtain the attention feature map corresponding to each target feature map.

[0060] In one specific embodiment, the target detection network includes a head model, a neck model, and an output model cascaded in sequence; the head model includes M residual modules; the neck model includes M visual attention modules and a fusion module; the M residual modules are connected to the M visual attention modules respectively, and each visual attention module is connected to the fusion module; M is a positive integer.

[0061] By adding a visual attention module to the common object detection network, the object detection network can focus more on the fish target itself and ignore the cluttered background.

[0062] In one embodiment, a target detection network is constructed based on the YOLOv4 network and the SKNet (Squeeze-and-Excitation Networks) attention model. The target detection network is then pruned, removing low-weight channels and / or deep residual modules. Specifically, the SKNet attention model is integrated into the YOLOv4 backbone network to construct a feature extraction network that focuses more on ambiguous fish information. This enhances the YOLOv4 backbone network's ability to represent the features of the fish target itself, thereby improving the accuracy of fish target detection. In this embodiment, the YOLOv4 backbone network is specifically Darknet53.

[0063] Please see Figure 2 , Figure 2 yes Figure 1 A flowchart illustrating a specific embodiment of step S2 in the provided target counting method.

[0064] S21: Extract features from the current video frame using the residual module to obtain target feature maps of different scales output by each residual module.

[0065] Specifically, the head model primarily performs feature extraction. The head network consists of multiple stacked residual modules. Shallow residual modules extract feature maps with high resolution, containing more location and detail information, and retaining more complete information from the current video frame, making them suitable for extracting small targets. Deep residual modules extract feature maps with lower resolution, retain stronger semantic information from the current video frame, and have more abstract features, making them suitable for extracting large targets.

[0066] Prune the target detection network to remove low-weight channels and / or remove deep residual modules.

[0067] In this embodiment, a fish is used as an example to illustrate the target object. In real-world scenarios, fish are considered medium to large targets. Therefore, the number of stacked layers of residual modules in the constructed target detection network does not need to be excessive. The network depth of the target detection network needs to be pruned, removing deeper residual modules. For example, the target detection network includes three residual modules, the number of which is specifically set according to the actual situation. Here, network depth represents the number of stacked layers of residual modules. Specifically, the network depth of the target detection network is set according to the target object to be detected. Most of the information stored in the network width of the target detection network is repetitive, and features can be represented using sparser weights. Therefore, the network width is also sparsified. The network width is pruned by calculating the information entropy of each channel in each residual module, with less pruning for channels with high information content and more pruning for channels with low information content. Here, network width represents the number of channels used for feature extraction by the residual module.

[0068] After pruning, the residual modules in the target detection network perform feature extraction on the current video frame to obtain the target feature map corresponding to each residual module. In this embodiment, after processing by each residual module, the target detection network undergoes downsampling to obtain the target feature maps output by the three residual module branches.

[0069] S22: The visual attention module introduced in the object detection network is used to convolve each target feature map based on the matching convolution kernel to obtain the attention feature map corresponding to each target feature map.

[0070] Specifically, each visual attention module selects a matching convolution kernel based on the scale of the target feature map to perform convolution processing on the target feature map, thereby obtaining the attention feature map corresponding to each target feature map. Among them, the visual attention module adaptively selects the convolution kernel size according to the size of the corresponding incoming target feature map, thereby increasing the network's receptive field. By performing convolution processing on the target feature map through the visual attention module, different attention feature maps output by each visual attention module are obtained.

[0071] In this embodiment, the visual attention module is an SKNet module. The SKNet module mainly includes a Split unit, a Fuse unit, and a Select unit. Split uses convolutional kernels of different sizes to perform convolution and dilated convolution on the target feature map. The Fuse unit fuses the results of the Split unit and then performs average pooling to obtain the feature information of each channel. The Select unit normalizes the results of the Fuse unit to obtain the weights of channels at different scales. Finally, it uses these weights to weight the information obtained by the Split unit, thereby emphasizing necessary features and weakening unnecessary features, thus obtaining different attention feature maps output by each visual attention module.

[0072] S23: Perform feature fusion on the attention feature maps corresponding to each target feature map to obtain a fused feature map.

[0073] Specifically, the attention feature maps output by the M visual attention modules are fused according to their weights to obtain the fused feature map corresponding to the current video frame.

[0074] S24: Perform target detection on the fused feature map to obtain the detection information of each target object contained in the video frame.

[0075] Specifically, the output module performs target detection based on the fused feature map, and outputs the position coordinates of each target object and the corresponding category information of each target object.

[0076] When counting only target objects in the current video frame, the number of target locations corresponding to each category of information is counted to determine the count result of target objects in the current video frame. The sum of the counts of target locations corresponding to each category of information is determined as the count result of target objects in the current video frame.

[0077] This embodiment uses a vision-based approach, which has no impact on the growth and distribution of the fish population, is less harmful, and also allows managers to monitor the growth and distribution of the fish population in real time.

[0078] When it is necessary to count the target objects in a video stream, it is necessary to determine whether the target objects are the same in different video frames to avoid double counting.

[0079] In one embodiment, the specific implementation of determining the motion variable information between the target object and the historical target in step S1 includes the following steps.

[0080] Please see Figure 3 , Figure 3 This is a flowchart illustrating a specific embodiment of the target counting method provided by the present invention.

[0081] In one embodiment, the neighborhood attention module in the first neural network determines the motion variable information between the target object and the historical target based on the detection information of each target object in the current video frame and the detection information of each historical target in the historical video frames before the current video frame.

[0082] To further improve the accuracy of target counting, the decision to count a target object is made based on motion variable information of target objects with the same category information in the neighborhood and historical targets. Target objects with the same category information and historical targets belong to the same group.

[0083] This proposal analyzes target objects within the same species group because these objects are more likely to move in neighboring areas and exhibit similar motion patterns. Therefore, it employs a Transformer-based attention mechanism to allow features from identical target objects to complement each other, resulting in richer features for each object and improving the accuracy of its motion information. This step helps determine whether target objects are identical across different video frames, thus avoiding duplicate counting.

[0084] Based on the position information of each target object in the current video frame and the position information of each historical target with the same category information as the target object in the historical video frames, an attention vector is generated between the target object and the historical targets. The attention vector includes a key value K, a query Q, and a value V. Based on the attention vector between the target object and the historical targets, the probability value that the target object and the historical targets are the same target is determined. Based on the attention vector and the probability value between the target object and the historical targets, the motion variable information between the target object and the historical targets is determined.

[0085] In one specific embodiment, the first neural network includes an encoder, a first attention module, and a decoder cascaded in sequence.

[0086] The encoder generates attention vectors between target objects and historical targets in different video frames based on the detection information of each target object in the current video frame and the detection information of each historical target in the historical video frames. The attention vectors are convolved based on the first attention module to obtain the probability value that the target object and the historical target are the same target. The decoder determines the motion variable information between the target object and the historical target based on the corresponding attention vectors and probability values. The motion variable information includes position offset and angle information.

[0087] The neighborhood attention module uses a Transformer attention model. It is an attention layer specifically designed for fish swarm movement. The motion information of a single target object is weighted and combined with the motion information of historical targets in adjacent video frames to obtain the attention intensity between individual target objects. This is then decoded using a Gaussian neural network to obtain the motion information of the entire population.

[0088] The neighborhood attention module calculates the attention weights between the target object in the current video frame and historical targets in historical video frames to determine the degree of interaction between them. This amplifies information with high interaction and suppresses information with low interaction. The decoder is a Gaussian neural network used to decode the motion information of the target object and historical targets processed by the neighborhood attention module, thereby obtaining the positional offset and angle information between the two target objects. Specifically, the positional offset is compared with a preset offset.

[0089] If the positional offset between the target object and a historical target meets both the offset threshold and the angle information meets both the angle threshold, then the target object and the historical target are determined to be the same target. If the positional offset between the target object and a historical target does not meet either the offset threshold or the angle information does not meet either the angle threshold, then the target object and the historical target are determined to be different targets. The angle information is the angle between the line connecting the positional information of the same target object in different video frames and a reference direction. For example, the reference direction can be horizontal or vertical.

[0090] If the target object is different from the historical target, the target object in the current video frame is counted; if the target object is the same as the historical target, the target object in the current video frame is not counted to avoid double counting.

[0091] In this embodiment, a fish counting algorithm with a multi-dimensional attention mechanism including ontology and neighborhood is designed by focusing on two dimensions and their relationships between individual fish and the population, based on video sequences captured by an underwater camera. This enables accurate counting of fish in situations such as underwater imaging blur, texture distortion, and dense fish populations.

[0092] The target counting method provided in this embodiment includes: performing target detection on the current video frame to obtain detection information of target objects contained in the current video frame; the detection information includes location information and category information; determining motion variable information between the target object and historical targets based on the location information of the target object in the current video frame and the location information of historical targets with the same category information as the target object in previous historical video frames using a neighborhood attention module; determining whether the target object and historical targets are the same based on the motion variable information between the target object and historical targets; and counting the target object if it is not the same as the historical targets. This application uses a neighborhood attention module to determine the corresponding motion variable information based on the location information of target objects and historical targets with the same category information. The neighborhood attention module can obtain richer features of the target object and historical targets, thereby improving the detection accuracy of motion variable information, and thus determining whether the target object and historical targets in different frames are the same target, thereby avoiding duplicate counting and improving the target counting accuracy.

[0093] Please see Figure 4 , Figure 4 This is a schematic diagram of a target counting device according to an embodiment of the present invention. This embodiment provides a target counting device 60, which includes a detection module 61, an analysis module 62, a processing module 63, and a determination module 64.

[0094] The detection module 61 is used to perform target detection on the current video frame and obtain the detection information of the target objects contained in the current video frame; the detection information includes location information and category information.

[0095] The analysis module 62 is used to determine the motion variable information between the target object and historical targets by using the neighborhood attention module based on the position information of the target object in the current video frame and the position information of historical targets with the same category information as the target object in the previous historical video frames.

[0096] The processing module 63 is used to determine whether the target object and the historical target are the same based on the motion variable information between the target object and the historical target.

[0097] The determination module 64 is used to count the target object in response to the target object being different from the historical target.

[0098] The target counting device provided in this embodiment determines the corresponding motion variable information of target objects and historical targets based on the location information of target objects with the same category information and the corresponding location information of historical targets. The neighborhood attention module can obtain richer features of target objects and historical targets, thereby improving the detection accuracy of motion variable information. It can then determine whether target objects in different frames and historical targets are the same target, thereby avoiding duplicate counting and improving the target counting accuracy.

[0099] Please see Figure 5 , Figure 5 This is a schematic diagram of a terminal embodiment provided by the present invention. The terminal 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps of any of the above-described target counting method embodiments. In a specific implementation scenario, the terminal 80 may include, but is not limited to, a microcomputer, a server, etc. In addition, the terminal 80 may also include mobile devices such as laptops and tablets, which are not limited here.

[0100] Specifically, processor 82 controls itself and memory 81 to implement the steps of any of the above-described target counting method embodiments. Processor 82 may also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 82 may be implemented using integrated circuit chips.

[0101] Please see Figure 6 , Figure 6This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present invention. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps of any of the above-described target counting method embodiments.

[0102] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0103] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] The above are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A target counting method, characterized in that, The target counting method includes: Target detection is performed on the current video frame to obtain detection information of the target objects contained in the current video frame; the detection information includes location information and category information. The neighborhood attention module determines the motion variable information between the target object and the historical targets based on the position information of the target object in the current video frame and the position information of each historical target with the same category information as the target object in the historical video frames before the current video frame. Based on the motion variable information between the target object and the historical target, determine whether the target object and the historical target are the same; If the target object is different from the historical target, then the target object is counted; The step of determining motion variable information between the target object and the historical targets using a neighborhood attention module based on the position information of the target object in the current video frame and the position information of historical targets with the same category information as the target object in previous historical video frames includes: Based on the location information of each target object in the current video frame and the location information of each historical target in the historical video frames that has the same category information as the target object, an attention vector is generated between the target object and the historical target; the attention vector includes a key value K, a query Q, and a value V. Based on the attention vectors corresponding to the target object and the historical target, determine the probability value that the target object and the historical target are the same target; Based on the attention vectors corresponding to the target object and the historical target, and the probability values, motion variable information between the target object and the historical target is determined.

2. The target counting method according to claim 1, characterized in that, The motion variable information includes position offset and angle information; Determining whether the target object and the historical target are the same based on the motion variable information between the target object and the historical target includes: If the positional offset between the target object and the historical target meets an offset threshold and the angle information meets an angle threshold, then the target object and the historical target are determined to be the same target.

3. The target counting method according to claim 1, characterized in that, The neighborhood attention module is a Transformer attention model.

4. The target counting method according to claim 1, characterized in that, The step of performing target detection on the current video frame to obtain detection information of the target objects contained in the current video frame includes: Feature extraction of video frames is performed using an object detection network to obtain multiple target feature maps at different scales; The visual attention module introduced in the target detection network performs convolution on each target feature map based on a matched convolution kernel to obtain the attention feature map corresponding to each target feature map. The attention feature maps corresponding to each target feature map are fused to obtain a fused feature map; Target detection is performed on the fused feature map to obtain detection information for each target object contained in the video frame.

5. The target counting method according to claim 4, characterized in that, The target detection network includes M cascaded residual modules and M visual attention modules, with each residual module connected to a visual attention module, where M is a positive integer. The process involves extracting features from video frames using a target detection network to obtain multiple target feature maps at different scales, including: The residual module extracts features from the current video frame to obtain target feature maps of different scales output by each residual module. The step of convolving each target feature map with a matched convolution kernel using the visual attention module introduced in the target detection network to obtain the attention feature map corresponding to each target feature map includes: Each visual attention module selects a matching convolution kernel based on the scale of the target feature map and performs convolution processing on the target feature map to obtain the attention feature map corresponding to each target feature map.

6. The target counting method according to claim 5, characterized in that, Before the step of performing target detection on the current video frame to obtain detection information of the target object contained in the current video frame, the method further includes: The object detection network is constructed based on the YOLOv4 network and the SKNet attention model; The target detection network is pruned to remove channels with low weights and / or remove the residual modules deep within the network.

7. A target counting device, characterized in that, The target counting device includes: The detection module is used to perform target detection on the current video frame and obtain detection information of the target objects contained in the current video frame; the detection information includes location information and category information. The analysis module is used to determine motion variable information between the target object and the historical targets based on the position information of the target object in the current video frame and the position information of historical targets with the same category information as the target object in the previous video frames, using a neighborhood attention module; it is also used to generate an attention vector corresponding to the target object and the historical targets based on the position information of each target object in the current video frame and the position information of each historical target with the same category information as the target object in the previous video frames; the attention vector includes a key value K, a query Q, and a value V; based on the attention vector corresponding to the target object and the historical targets, it determines a probability value that the target object and the historical targets are the same target; based on the attention vector corresponding to the target object and the historical targets and the probability value, it determines the motion variable information between the target object and the historical targets. The processing module is used to determine whether the target object and the historical target are the same based on the motion variable information between the target object and the historical target; The determination module is used to count the target object in response to the target object being different from the historical target.

8. A terminal, characterized in that, The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, the processor being configured to execute program data to implement the steps in the target counting method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the target counting method as described in any one of claims 1 to 6.

Images