A method for target recognition and key area detection

Abstract

The present invention provides a method for target recognition and key area detection thereof, an electronic device, and a storage medium. The method comprises: obtaining an original image to be processed; inputting the original image into a trained first model to obtain multiple target detection frames; processing the multiple target detection frames using an intermediate module to obtain multiple first images matching the input of a second model; inputting each first image into the trained second model respectively to obtain key area detection frames corresponding to all first images; and obtaining a detection result according to the target detection frame and its corresponding key area detection frame. The method performs target detection by using a conventional target detection algorithm, and then performs key area recognition on the target image in each target detection frame, thereby directly constructing the association relationship between the target and its key area, and no subsequent matching is required. The overall algorithm complexity is reduced, the processing speed is faster, and there is no mismatching phenomenon, and the detection result is very accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for target recognition and key area detection thereof. Background Art

[0002] In the field of image processing technology, target detection is a very popular research direction. Existing target detection algorithms are mostly used to detect icons in static images and output corresponding detection boxes. For example, convolutional neural networks (CNN) and YOLO series algorithms. CNN can automatically learn and extract deep features of images, significantly improving the accuracy of detection. These network structures can be continuously optimized and adjusted through a large amount of training data to adapt to target detection in different scenarios.

[0003] In more complex scenarios, such as tracking fish in a school of fish, tracking a single individual in a herd of cattle and sheep in animal husbandry, tracking human targets in a crowd, etc., the tracking is based on the target detection algorithm. The target position in each frame of the image is detected by the target detection algorithm to achieve tracking, but in these scenarios, there is often overlap between targets, resulting in poor tracking results.

[0004] In order to improve the recognition effect of overlapping targets, the invention patent "A fish detection method based on a new target recognition algorithm" (publication number: CN118072000A) discloses a fish detection method based on a new target recognition algorithm, wherein the fish detection method includes: obtaining a picture to be detected; using a neural network model to detect the fish head, fish body and fish tail in the picture respectively, and obtaining multiple groups of detection frames; calculating the fish head-fish body association matrix and the fish body-fish tail association matrix according to the position and size of the multiple groups of detection frames; matching the fish head, fish body and fish tail; according to the matching results, merging the multiple groups of detection frames to obtain the fish detection result. The fish detection method divides the fish into fish head, fish body and fish tail, and detects the detection frame of each area respectively. For the detection of a single area, the existing algorithm is still very accurate, and then establishes an association matrix to associate and match the detection frames of these areas and merge them. In this way, the fish detection result is a combination of detection frames of multiple different areas, which can well deal with occlusion, intersection and overlap.

[0005] In this way, the key area of ​​the target can be detected, and then the target detection frame and the detection frame of the key area of ​​the target can form the detection result of the target, which can well deal with the occlusion phenomenon. The above algorithm is relatively complex, so a simpler algorithm is urgently needed to realize the detection of occluded targets. Summary of the invention

[0006] In view of the above defects of the prior art, the present invention provides a target recognition and key area detection method thereof to solve the technical problem that targets in overlapping scenes are difficult to detect.

[0007] To achieve the above-mentioned purpose and other related purposes, the present invention provides a method for target recognition and key area detection thereof, comprising the following steps: obtaining an original image to be processed; inputting the original image into a trained first model to identify the target to be detected in the original image, and obtaining multiple target detection frames; using an intermediate module to process the multiple target detection frames to obtain multiple first images matching the second model input; inputting each of the first images into the trained second model respectively to identify the key area in the first image, and obtaining the key area detection frames corresponding to all the first images; and obtaining the detection results based on the target detection frames and their corresponding key area detection frames.

[0008] In one embodiment of the present invention, the first model is a YOLOv5s model.

[0009] In one embodiment of the present invention, an intermediate module is used to process multiple target detection frames to obtain multiple first images that match the second model input, including: obtaining a number of individual fish pictures based on the multiple target detection frames; for each of the individual fish pictures, upsampling to a size of H1×W1, and then downsampling the upsampled image to a size of H2×W2, wherein H1=roundup(H0 / H2)*H2, W1=roundup(W0 / W2)*W2, H0 and W0 are the height and width of the individual fish picture, H2 and W2 are the height and width of the first image, and roundup() is a rounding-up function.

[0010] In one embodiment of the present invention, the second model includes: a backbone module, used to extract features of the first image; a target detector, which is obtained by setting the number of candidate boxes of Sparse DETR to the number of key areas, and is used to extract the key area detection box corresponding to the first image from the features of the first image; a feature processing module, used to process the features of the first image extracted by the backbone module, and input them into the target detector to match the input of the target detector.

[0011] In one embodiment of the present invention, the backbone module includes the following cascaded in sequence: a first patch embedding unit, used to perform a four-fold downsampling operation; a first DWDA unit, which is cascaded by N1 DWAs and DATs; a second patch embedding unit, used to perform a two-fold downsampling operation; a second DWDA unit, which is cascaded by N2 DWAs and DATs; a third patch embedding unit, used to perform a two-fold downsampling operation; a third DWDA unit, which is cascaded by N3 DWAs and DATs; a fourth patch embedding unit, used to perform a two-fold downsampling operation; and a fourth DWDA unit, which is cascaded by N4 DWAs and DATs.

[0012] In one embodiment of the present invention, N1, N2, N3, and N4 are equal to 2, 2, 6, and 2, respectively.

[0013] In one embodiment of the present invention, the DWA processes the input feature x by the following steps: obtaining the size of the dynamic window ks according to the channel dimension C of the input feature x; obtaining q, k, v according to the input feature x and the weight matrix; and learning the mask window q of size ks×ks of q according to q and ks. mw ; Pool v and k separately to get k w and v w ; After filling the input features, perform pooling processing to obtain a mw ; for the q and the k w Do attention calculation and get a map ; The a map , the q mw , the a mw Perform point-by-point multiplication to obtain ma mp ; The ma mp and the v w After multiplication and combining with the weight matrix, the output features of the DWA are obtained.

[0014] In one embodiment of the present invention, the size ks of the dynamic window is obtained according to the channel dimension C of the input feature x, including: calculating the square root of the channel dimension C and rounding up to obtain a first intermediate value; if the first intermediate value is an even number, adding one to the first intermediate value, otherwise it remains unchanged, and the result is recorded as the second intermediate value; the ks is equal to the smaller value of 2×size-1 and the second intermediate value, where the size, the width of the input feature x, and the height of the input feature x are equal.

[0015] In one embodiment of the present invention, the feature processing module processes the features output by the backbone module in the following steps: performing convolution processing on the features output by the fourth DWDA unit for downsampling; performing 1*1 convolution operation on the features output by the second DWDA unit, the third DWDA unit, the fourth DWDA unit, and the features after the convolution processing to make the channels equal, and then splicing them and outputting them to the target detector.

[0016] Beneficial effects of the present invention: The present invention proposes a method for target recognition and key area detection. The method uses a conventional target detection algorithm (i.e., the first model) to perform the first step of target detection, and then uses the second model to perform key area recognition on the target image in each target detection frame respectively. In this way, the association relationship between the target and its key area can be directly constructed, and no subsequent matching is required. The overall algorithm complexity is reduced, the processing speed is faster, and there will be no mismatching phenomenon, and the detection result is very accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A processing flow chart of a detection method provided by an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of an original image provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a plurality of first images provided by an embodiment of the present invention;

[0021] Figure 4 A processing flow chart of an intermediate module provided in one embodiment of the present invention;

[0022] Figure 5 A structural diagram of a second model provided by an embodiment of the present invention;

[0023] Figure 6 A structural diagram of a backbone module provided by an embodiment of the present invention;

[0024] Figure 7 A processing flow chart of DWA provided by an embodiment of the present invention;

[0025] Figure 8 for Figure 7θ mask Block diagram of the module. DETAILED DESCRIPTION

[0026] The following describes the embodiments of the present invention through specific specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. It should be noted that the following embodiments and the features in the embodiments can be combined with each other without conflict. In addition to the specific methods, equipment, and materials used in the embodiments, according to the technical personnel in this technical field's grasp of the prior art and the records of the present invention, any methods, equipment, and materials of the prior art that are similar or equivalent to the methods, equipment, and materials described in the embodiments of the present invention can also be used to implement the present invention.

[0027] It should be understood that the terms used in the embodiments of the present invention are for describing specific embodiments rather than for limiting the scope of protection of the present invention. Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those generally understood by those skilled in the art.

[0028] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In some of the embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0029] The flowchart and block diagram in the accompanying drawings illustrate the system architecture, functions and operations that may be implemented according to the methods and computer program products of various embodiments disclosed in the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, the program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0030] Before introducing the content of the present invention, let's first explain why the target and its key areas need to be detected. The patent "A fish trajectory tracking method based on Kalman filter" (CN118096838A) discloses a fish trajectory tracking method. In order to track the fish's trajectory well, the fish head and tail information is needed to estimate the next position of the fish, so that even if there is occlusion, the fish's trajectory can be tracked well. Therefore, in some cases, it is not only necessary to detect the target, but also to detect the key areas in the target.

[0031] Hereinafter, the application is described in detail by taking fish as an example. For fish, the key areas may be, for example, the head and tail of the fish. If other targets are processed, such as human targets, the key areas may be, for example, the head, limbs, torso, etc. of the human being.

[0032] See also Figure 1 , Figure 1 A method for target recognition and key area detection provided by an embodiment of the present invention includes the following steps.

[0033] The first step is to obtain the original image to be processed. The original image can be in RGB format or other formats. Figure 2 The original images shown are taken in an experimental environment, and the actual images to be processed will be more complex.

[0034] In the second step, the original image is input into the trained first model to identify the target to be detected in the original image and obtain multiple target detection frames. The first model can use a common target detection model in the prior art. For example, the YOLOv5s model is specifically used in this embodiment. In addition to this model, other models that can perform target recognition can be used as the first model.

[0035] In the third step, the intermediate module is used to process multiple target detection frames to obtain multiple first images that match the second model input. Figure 3 As shown, the second model mainly processes the single target image corresponding to each target detection frame, so it is necessary to process the output of the first model through an intermediate module to adapt to the input of the second model.

[0036] In the fourth step, each first image is input into the trained second model to identify the key area in the first image and obtain the key area detection frame corresponding to all the first images. The second model is the key part of this embodiment, through which the key area of ​​each target can be detected.

[0037] The fifth step is to obtain the detection result based on the target detection frame and its corresponding key area detection frame.

[0038] The above steps use a conventional target detection algorithm (i.e., the first model) to perform the first target detection, and then use the second model to identify the key areas of the target image in each target detection frame. This can directly construct the relationship between the target and its key areas, and no subsequent matching is required. The overall algorithm complexity is reduced, the processing speed is faster, and there will be no mismatching, and the detection results are very accurate.

[0039] It is understandable that both the first model and the second model need to be trained, and only after the training is completed can the original image to be processed be recognized. The training of the first model and the second model can be achieved through the following steps.

[0040] First, a data set needs to be constructed. In this embodiment, white wallpaper is pasted inside a glass fish tank that is 1.2 meters long, 0.5 meters wide, and 0.7 meters high. The purpose is to prevent the mirror reflection of the glass on the fish body, and a Boleth E3GMPCX15 high-definition underwater camera is placed at the corner of the fish tank, and the camera has an upward angle of 30° to record the movement of fish in the water. The collected video is framed at intervals of 1s, and the extracted pictures are annotated through the GitHub open source project anylabeling. The first step is to annotate each fish in a picture. After the annotation is completed, the first data set for training the first model is obtained. Then each fish body target is cropped in the picture, and the cropped picture continues to use anylabeling to annotate the head and tail information of each fish, thereby obtaining a second data set for training the second model.

[0041] Secondly, the first model is trained using the first data set, and the second model is trained using the second data set. During training, for example, Precision, Recall, and AP (Average Precision) indicators may be used for evaluation, and model hyperparameters may be optimized.

[0042] See also Figure 4 In a specific embodiment of the present invention, an intermediate module is used to process multiple target detection frames to obtain multiple first images matching the second model input, including: obtaining a number of single fish pictures according to the multiple target detection frames; for each single fish picture, upsampling to a size of H1×W1, and then downsampling the upsampled image to a size of H2×W2, wherein H1=roundup(H0 / H2)*H2, W1=roundup(W0 / W2)*W2, H0 and W0 are the height and width of the single fish picture, H2 and W2 are the height and width of the first image, and roundup() is a rounding up function.

[0043] Assuming that the size of the second model input is 224×224, the size of the upsampled image should be (224×m)×(224×n), where m and n are positive integers. Figure 4 FIG. 1 illustrates two sizes of single fish images, 456×813 and 519×392, respectively, and gives the specific upsampling and downsampling processes of the two images, and finally obtains a first image of size 224×224.

[0044] This variable window maximum pooling strategy fully considers the positive correlation between the local feature size and the overall size of the target object. That is, for a fish, if its size in the picture is large, the size of its corresponding head and tail in the picture will also be large, and vice versa. For larger targets, a larger window can speed up preprocessing without losing important local feature information; for smaller targets, a smaller window can retain its detailed features. This preprocessing operation gives the model stronger scale adaptability and enhances its adaptability and robustness to different input scales.

[0045] See also Figure 5 In a specific embodiment of the present invention, the second model includes a backbone module, a target detector and a feature processing module, wherein the backbone module is used to extract features of the first image; the target detector is obtained by setting the number of candidate frames of Sparse DETR to the number of key regions, and is used to extract key region detection frames corresponding to the first image from the features of the first image; the feature processing module is used to process the features of the first image extracted by the backbone module and input them into the target detector to match the input of the target detector. Through the cooperation of these three modules, the key region detection frames corresponding to the first image can be extracted quickly and accurately.

[0046] The backbone module is mainly used to extract features of the first image, and to ensure that the extracted features meet the input feature size of the target detector. The backbone module may be, for example, Swin-Transformer, ResNet50, or the following structure.

[0047] See also Figure 6In a specific embodiment of the present invention, the backbone module includes the following cascaded units: a first patch embedding unit for performing a four-fold downsampling operation; a first DWDA unit, which is formed by cascading N1 DWAs and DATs; a second patch embedding unit for performing a two-fold downsampling operation; a second DWDA unit, which is formed by cascading N2 DWAs and DATs; a third patch embedding unit for performing a two-fold downsampling operation; a third DWDA unit, which is formed by cascading N3 DWAs and DATs; a fourth patch embedding unit for performing a two-fold downsampling operation; and a fourth DWDA unit, which is formed by cascading N4 DWAs and DATs. In the backbone module, the patch embedding unit is used for downsampling operations, and the DWDA unit is used for extracting features, and the unit does not affect the size of the feature map. Assuming that the input feature map size is H×W×3, the feature map sizes output by the first DWDA unit, the second DWDA unit, the third DWDA unit, and the fourth DWDA unit are H / 4×W / 4×C, H / 8×W / 8×2C, H / 16×W / 16×4C, and H / 32×W / 32×8C, respectively. In this embodiment, through the combination of DWA (Deformable Windows Attention) and DAT (DeformableAttention Transformer), the model can not only focus on local details, but also grasp the overall features, which is especially important for common occlusion and multi-target recognition problems in underwater environments. This cascade method allows the model to have better accuracy and robustness when processing the detection of key areas (head and tail) of the target (a fish body).

[0048] Each DWDA unit is composed of a plurality of DWAs and DATs cascaded together. In a specific embodiment of the present invention, N1, N2, N3, and N4 are respectively equal to 2, 2, 6, and 2, that is, the first DWDA unit, the second DWDA unit, and the fourth DWDA unit are all of the structure of DWA+DAT+DWA+DAT; the third DWDA unit is of the structure of (DWA+DAT)×4.

[0049] DAT (Deformable Attention Transformer) is a variant Transformer architecture based on the attention mechanism, designed to improve the performance of image classification and dense prediction tasks. It focuses on solving the problem of feature modeling and relationship capture in image processing tasks. By introducing the deformable attention mechanism, it allows the model to adjust the attention weight more flexibly and finely when performing feature alignment and relationship modeling, focusing on key areas in the image, thereby improving computational efficiency and model performance. In some papers, it is also referred to as DeformableAttention Module.

[0050] The core idea of ​​DWA (Deformable Windows Attention) is that the size of the attention area that a feature vector focuses on should be determined by the amount of semantic information it contains. Specifically, when the channel dimension C of a feature vector is large, it means that it contains more underlying semantic information, so it can calculate attention with farther areas; conversely, when C is small, the information contained in the feature is still the local information of the shallow feature, so the vector should pay more attention to the attention information within the local neighborhood. Based on this, the attention mechanism of DWA (Deformable Window Attention) proposed in this embodiment aims to enhance the backbone module's perception of local structures and adapt to different feature areas by dynamically adjusting the size of the window. DWA integrates window deformation capabilities in a multi-head attention framework to process key parts of the image in an adaptive manner, thereby improving the model's processing efficiency and accuracy for complex visual tasks.

[0051] See also Figure 7 and Figure 8 In a specific embodiment of the present invention, DWA processes the input feature x through the following steps.

[0052] (1) According to the channel dimension C of the input feature x, the size ks of the dynamic window is obtained. The size of the input feature x is recorded as H×W×C, where C is its channel dimension, which is a known quantity. Then, the size ks of the dynamic window can be calculated according to C. In a specific embodiment of the present invention, the calculation steps are as follows.

[0053] a) Calculate the square root of the channel dimension C and round it up to get the first intermediate value, which can be expressed as: First intermediate value = , where That is, the round-up symbol.

[0054] b) If the first intermediate value is an even number, add one to the first intermediate value, otherwise it remains unchanged and the result is recorded as the second intermediate value. It can be expressed as:

[0055] Second median value = ;

[0056] In the formula, even represents an even number, and odd represents an odd number.

[0057] c) ks is equal to the smaller value of 2×size-1 and the second intermediate value, where size, the width of the input feature x, and the height of the input feature x are equal. It can be expressed as:

[0058] ;

[0059] This way, the size of ks does not exceed twice the size of the feature map. If it exceeds this, each feature will be calculated with a large number of padding features when sliding the window, which greatly increases the ineffective calculation overhead of the model.

[0060] (2) According to the input feature x and the weight matrix, q, k, v are obtained. Specifically, according to W q , W k , W v The three matrices are calculated according to the following formula: q=xW q , k = xW k 、V=xW V , for the input x(W×H,C), the size of the output q, k, v is also (W×H,C).

[0061] (3) Based on q and ks, learn the mask window q of size ks×ks for q mw Specifically, through θ mask Modules (such as Figure 8 As shown in the figure, it can be expressed as follows:

[0062] ,

[0063] Where W p is a fully connected layer, σ is the Sigmoid function, q mw The size is N q ×ks 2 , N q is the number of queries, that is, the number of features of q. Assuming q is 7*7*128, then q is 7*7=49.

[0064] (4) Pool v and k separately to get k w (k windows) and v w (v windows), whose size is the same as q mw are the same size.

[0065] (5) Fill the input features (set the pixels in the filled area to 0) and then perform pooling to obtain a mw (attention mask windows), its size is also N q ×ks 2 .

[0066] (6) For q and k w Do attention calculation and get a map (attention map windows). It can be expressed as:

[0067] ;

[0068] In the formula, softmax() is the normalization function, B is the relative position bias value, and d k is the dimension vector of query, assuming q is 7*7*128, d k That's 128.

[0069] (7) map ,q mw 、a mw Perform point-by-point multiplication to obtain ma mp (mask attention map windows). It can be expressed as: mp =a map ⊙q mw ⊙a mw , where ⊙ represents the point-by-point multiplication operation.

[0070] (8) will ma mp and v w After multiplication and combining with the weight matrix, we get the output features of DWA. The formula is: Output = W o (ma mp ⊙v w ).

[0071] In the above steps, the weight matrix W q , W k , W v , W o These are all things that need to be learned during model training.

[0072] As the feature pyramid progresses, the window size in the dynamic window attention (DWA) mechanism gradually increases, eventually enabling the model to achieve global self-attention capabilities. In this process, DWA effectively utilizes the semantic associations between features and greatly expands the receptive field. At the same time, since the attention calculation is limited to the dynamically determined window area, the computational complexity is greatly reduced.

[0073] The overall computational complexity of DWA is: O(N×C×(M+C+M 2 ), where: N = H × W, is the total number of elements in the input feature map, H, W, C represent the height, width, and number of channels (feature dimension) of the input feature map, respectively, and M is the number of elements in the window, usually WS 2 , where WS refers to the width and height of the window.

[0074] See also Figure 5In a specific embodiment of the present invention, the feature processing module processes the features output by the backbone module according to the following steps: convolution processing is performed on the features output by the fourth DWDA unit to perform downsampling, that is, the H / 32×W / 32×8C features are downsampled to obtain H / 64×W / 64×16C; 1*1 convolution operations are performed on the features output by the second DWDA unit, the third DWDA unit, and the fourth DWDA unit, as well as the features after the convolution processing, to make the channels equal, that is, H / 8×W / 8×2C, H / 16×W are downsampled to obtain H / 64×W / 64×16C. The four features of H / 16×4C, H / 32×W / 32×8C, and H / 64×W / 64×16C are convolved to obtain four features of sizes H / 8×W / 8×256, H / 16×W / 16×256, H / 32×W / 32×256, and H / 64×W / 64×256; they are then concatenated and output to the target detector. Taking the four features of the above sizes as an example, a feature of size (H / 8×W / 8+H / 16×W / 16+H / 32×W / 32+H / 64×W / 64)×256 is obtained after concatenation.

[0075] See also Figure 5 In a specific embodiment of the present invention, Sparse DETR is an efficient end-to-end object detector that improves computational efficiency through a dynamic sparsification mechanism (for details, see the paper SPARSEDETR: EFFICIENT END-TO-END OBJECT DETECTION WITH LEARNABLE SPARSITY published in 2022). The saliency of each marker is evaluated by a scoring network, and the top ρ% of salient markers are selected for further processing. In the encoder, only these salient markers are updated, and features are optimized through deformable self-attention and feedforward networks, while auxiliary losses are applied to stabilize training. The decoder receives the streamlined encoder output and object query, performs cross attention to focus on relevant areas, combines bipartite graph matching strategies, and finally outputs category scores and bounding boxes. The overall computational complexity of SparseDETR is significantly reduced, and the retention rate is allowed to be dynamically adjusted in the inference stage to adapt to different computational constraints, thereby improving efficiency while ensuring high detection accuracy. In this embodiment, by setting the number of candidate boxes of Sparse DETR to 2, the object detector can effectively learn the head and tail information of each fish.

[0076] After the first model and the second model are constructed through the above structure, the optimal result of the first model training and the optimal result of the second model training and the intermediate module are cascaded to obtain the overall cascade network. When a picture to be detected is input into the overall cascade network, all the fish targets in this picture are first obtained through the first model, then the picture is cropped through the intermediate module, and after the image downsampling image preprocessing operation of the variable window is performed, all the fish pictures are adjusted to a uniform size and input into the second model in parallel, and the fish head and tail features of all the fish pictures are detected at one time, and then the fish head and tail information detected by the second model is associated with the fish body information detected by the first model, and finally the entire cascade network outputs each fish and its head and tail information. At this point, the detection of the target and its key areas is achieved.

[0077] In general, the present invention extracts general features through a simple first model, and then performs refined processing with a second model. This strategy can effectively utilize limited annotation resources, which is particularly important when the cost of acquiring natural environment data is high. Fish schools in underwater environments often show high dynamics and complex interactive behaviors. Through the network structure in the present invention, the clustering and mutual occlusion problems between fish bodies can be handled more accurately. Through fine-grained feature learning, individual fish bodies and their parts can be better distinguished and tracked. Although most of the above embodiments are based on fish as an example, it is also applicable to other clustered moving targets and can be expanded to the detection of other organisms or objects, providing a powerful tool for other fields such as ecological monitoring and behavioral research.

[0078] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for target recognition and key area detection, characterized in that: The steps include: Obtain the original image to be processed; Inputting the original image into the trained first model to identify the target to be detected in the original image, and obtaining multiple target detection frames; Processing the plurality of target detection frames using an intermediate module to obtain a plurality of first images matching the second model input; Inputting each of the first images into the trained second model respectively to identify the key areas in the first images, and obtaining key area detection frames corresponding to all the first images; Obtaining a detection result according to the target detection frame and its corresponding key area detection frame; The plurality of target detection frames are processed by an intermediate module to obtain a plurality of first images matching the second model input, including: Obtaining a plurality of single fish images according to the plurality of target detection frames; For each of the single fish images, upsample to a size of H1×W1, and then downsample the upsampled image to a size of H2×W2, wherein H1=roundup(H0 / H2)*H2, W1=roundup(W0 / W2)*W2, H0 and W0 are the height and width of the single fish image, H2 and W2 are the height and width of the first image, and roundup() is a rounding up function; The second model includes: A backbone module, used for extracting features of the first image; An object detector is obtained by setting the number of candidate boxes of Sparse DETR to the number of key areas, and is used to extract the key area detection box corresponding to the first image from the features of the first image; A feature processing module, used for processing the features of the first image extracted by the backbone module, and inputting the features into the object detector to match the input of the object detector; The backbone module includes the following modules which are cascaded in sequence: A first patch embedding unit, configured to perform a four-fold downsampling operation; The first DWDA unit is composed of N1 DWAs and DATs cascaded; a second patch embedding unit, for performing a two-fold downsampling operation; The second DWDA unit is composed of N2 DWAs and DATs cascaded; a third patch embedding unit, for performing a two-fold downsampling operation; The third DWDA unit is composed of N3 DWAs and DATs cascaded; a fourth patch embedding unit, configured to perform a two-fold downsampling operation; and The fourth DWDA unit is composed of N4 DWAs and DATs cascaded together; The DWA processes the input feature x through the following steps: According to the channel dimension C of the input feature x, the size ks of the dynamic window is obtained; According to the input feature x and the weight matrix, q, k, v are obtained; According to q and ks, learn q for the mask window q of size ks×ks mw ; Pool v and k separately to get k w and v w ; The input features are filled and then pooled to obtain a mw ; For the q and k w Do attention calculation and get a map ; The a map , the q mw , the a mw Perform point-by-point multiplication to obtain ma mp ; The ma mp and the v w After multiplication and combining with the weight matrix, the output features of the DWA are obtained.

2. The target recognition and key area detection method according to claim 1, characterized in that: The N1, N2, N3, and N4 are equal to 2, 2, 6, and 2 respectively.

3. The target recognition and key area detection method according to claim 1, characterized in that: According to the channel dimension C of the input feature x, the size ks of the dynamic window is obtained, including: Calculate the square root of the channel dimension C and round it up to obtain a first intermediate value; If the first intermediate value is an even number, then increase the first intermediate value by one, otherwise, keep it unchanged, and the obtained result is recorded as the second intermediate value; The ks is equal to the smaller value of 2×size-1 and the second intermediate value, wherein the size, the width of the input feature x, and the height of the input feature x are equal.

4. The target recognition and key area detection method according to claim 1, characterized in that: The feature processing module processes the features output by the backbone module according to the following steps: Performing convolution processing on the features output by the fourth DWDA unit to perform down sampling; A 1*1 convolution operation is performed on the features output by the second DWDA unit, the third DWDA unit, the fourth DWDA unit, and the features after the convolution processing to make the channels equal, and then they are spliced ​​and output to the target detector.

Images