End-to-end target detection and orientation recognition method, apparatus and device

Abstract

The end-to-end target detection and orientation identification method, device and equipment provided by the embodiments of the present disclosure comprise: performing feature extraction on a target image to obtain first image feature information of a first down-sampling layer, second image feature information of a second down-sampling layer and third image feature information of a third down-sampling layer; performing key feature extraction on the first image feature information and the second image feature information to obtain first key feature information and second key feature information; performing self-attention calculation on the first key feature information and the second key feature information, and obtaining first target image feature information and second target image feature information according to the self-attention calculation result; and determining classification information, regression information and orientation information of the target image according to the first target image feature information, the second target image feature information and the third image feature information. The accuracy of the determined target orientation is ensured on the basis of improving the target detection efficiency.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology and related technical fields, specifically to an end-to-end target detection and orientation recognition method, apparatus, and device. Background Technology

[0002] Target orientation is the primary directional information of a target on the image plane, reflecting its true attitude and motion characteristics. In the automatic identification and situational analysis of targets such as ships, vehicles, and aircraft at sea, orientation information can not only improve the accuracy of target detection and tracking, but also be used to determine the target's movement trend, attack direction, and combat intent.

[0003] Traditional target orientation estimation often relies on manually designed features (such as target contours, edge orientation histograms, principal axis analysis, etc.), which are less robust to complex backgrounds, target occlusion, or noise interference. Furthermore, these methods are typically separated from the detection task, requiring target detection before pose estimation, resulting in a cumbersome process and a tendency for error accumulation. With the development of deep learning, researchers have begun to integrate orientation estimation into end-to-end target detection frameworks. Typical methods include orientation prediction based on angle regression, discrete angle encoding based on classification, and head-to-tail orientation inference based on keypoint detection. However, these methods still suffer from problems such as periodic ambiguity of angles, orientation fuzziness (lack of head-to-tail distinction), and decreased accuracy in low-resolution images.

[0004] Given the problems with existing technologies, there is an urgent need for an end-to-end target detection and orientation recognition method that can improve the efficiency of target detection while ensuring the accuracy of the determined target orientation. Summary of the Invention

[0005] The embodiments described herein provide an end-to-end target detection and orientation recognition method, apparatus, and device that ensures the accuracy of the determined target orientation while improving target detection efficiency.

[0006] Firstly, based on the content of this disclosure, an end-to-end target detection and orientation recognition method is provided, including: Feature extraction is performed on the target image to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer; Key features are extracted from the first image feature information and the second image feature information to obtain first key feature information and second key feature information; Self-attention calculation is performed on the first key feature information and the second key feature information, and the first target image feature information and the second target image feature information are obtained based on the self-attention calculation result; Based on the first target image feature information, the second target image feature information, and the third image feature information, the classification information, regression information, and orientation information of the target image are determined.

[0007] In some embodiments of this disclosure, the step of extracting features from the target image to obtain first image feature information of the first downsampling layer, second image feature information of the second downsampling layer, and third image feature information of the third downsampling layer includes: The target image is downsampled to obtain the first initial image feature information of the first downsampling layer, the second initial image feature information of the second downsampling layer, and the third initial image feature information of the third downsampling layer. Based on the triple attention mechanism, feature extraction is performed on the first initial image feature information, the second initial image feature information, and the third initial image feature information to obtain the first image feature information, the second image feature information, and the third image feature information.

[0008] In some embodiments of this disclosure, the triple attention mechanism includes a first branch, a second branch, and a third branch; The method of processing the first initial image feature information, the second initial image feature information, and the third initial image feature information based on the triple attention mechanism to obtain the first image feature information, the second image feature information, and the third image feature information includes: The first initial image feature information, the second initial image feature information, and the third initial image feature information are respectively input into the first branch, the second branch, and the third branch. Based on the first branch, the spatial orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. Based on the second branch, the width orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. Based on the third branch, the height orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. The first image feature information is determined based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the first initial image feature information. The second image feature information is determined based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the second initial image feature information. The third image feature information is determined based on the spatial direction attention value, width direction attention value, height direction attention value, and the third initial image feature information.

[0009] In some embodiments of this disclosure, the step of extracting key features from the first image feature information and the second image feature information to obtain first key feature information and second key feature information includes: The first image feature information and the second image feature information are respectively subjected to channel dimensionality reduction processing to obtain the first dimensionality-reduced image feature information and the second dimensionality-reduced image feature information; The first dimensionality reduction image feature information is divided into a first first sub-dimensionality reduction image feature information and a second first sub-dimensionality reduction image feature information, and the second dimensionality reduction image feature information is divided into a first second sub-dimensionality reduction image feature information and a second second sub-dimensionality reduction image feature information; Based on multiple backbone neck networks, key features are extracted from the feature information of the first sub-dimensionality reduction image. The feature information of the first sub-dimensionality reduction image after key feature extraction is then merged with the feature information of the second sub-dimensionality reduction image to obtain the first key feature information. Based on multiple backbone neck networks, key features are extracted from the feature information of the first second sub-dimensionality reduced image. The feature information of the first second sub-dimensionality reduced image after key feature extraction is then merged with the feature information of the second second sub-dimensionality reduced image to obtain the second key feature information.

[0010] In some embodiments of this disclosure, the step of performing self-attention calculation on the first key feature information and the second key feature information, and obtaining the first target image feature information and the second target image feature information based on the self-attention calculation result, includes: After performing feature alignment and channel alignment processing on the first key feature information and the second key feature information, the first target key feature information is obtained; Self-attention calculations are performed on the first target key feature information and the second target key feature information respectively, and the first target image feature information and the second target image feature information are obtained based on the self-attention calculation results.

[0011] In some embodiments of this disclosure, the step of performing feature alignment and channel alignment processing on the first key feature information and the second key feature information to obtain the first target key feature information includes: Obtain the first size information of the first image of the first downsampling layer and the second size information of the second image of the second downsampling layer; Based on the first size information, the second size information, and the first key feature information, determine the first initial key feature information under the second size information; The first initial key feature information and the second key feature information are channel aligned to obtain the first target key feature information.

[0012] In some embodiments of this disclosure, the step of performing self-attention calculations on the first target key feature information and the second key feature information, and obtaining the first target image feature information and the second target image feature information based on the self-attention calculation results, includes: The first target key feature information and the second key feature information are flattened respectively, and self-attention calculation is performed on the flattened first target key feature information and the second key feature information to obtain the first self-attention value corresponding to the first target key feature information and the second self-attention value corresponding to the second key feature information. The first target image feature information is determined based on the first target key feature information and the first self-attention value; The second target image feature information is determined based on the second key feature information and the second self-attention value.

[0013] In some embodiments of this disclosure, determining the classification information, regression information, and orientation information of the target image based on the first target image feature information, the second target image feature information, and the third image feature information includes: Based on the first target image feature information, the second target image feature information, and the third image feature information, determine the classification information and regression information of the target objects included in the target image; Based on the first target image feature information, the second target image feature information, and the third image feature information, determine the position information of the center point and head point of the target object included in the target image; Based on the position information of the center point and head point of the target object included in the target image, the orientation information of the target object included in the target image is determined.

[0014] Secondly, according to the present disclosure, an end-to-end target detection and orientation recognition device is provided, comprising: The first feature information processing module is used to extract features from the target image to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer. The second feature information processing module is used to extract key features from the first image feature information and the second image feature information to obtain first key feature information and second key feature information. The third feature information processing module is used to perform self-attention calculation on the first key feature information and the second key feature information, and obtain the first target image feature information and the second target image feature information based on the self-attention calculation result; The detection module is used to determine the classification information, regression information, and orientation information of the target image based on the first target image feature information, the second target image feature information, and the third image feature information.

[0015] Thirdly, according to this disclosure, a computer device is provided, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any of the first aspects.

[0016] The end-to-end target detection and orientation recognition method, apparatus, and device provided in this disclosure first extract features from the target image using a Triplet Attention mechanism. This mechanism consists of three branches (i.e., a first branch, a second branch, and a third branch). The three-branch structure captures the interaction relationships between the target image and the channel, height, and width dimensions of different downsampling layers and calculates attention values. Then, based on the calculated spatial, width, and height attention values, attention fusion is performed on the initial image feature information of different sampling layers to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer. This enhances the feature representation capability of the target image while reducing the large number of parameters caused by deep convolution, making the model more lightweight. Next, the C2f module of the Yolov8 baseline model extracts key features from the first and second image feature information to obtain first and second key feature information. The input of the C2f module is connected to the output of the triple attention mechanism. The C2f module is connected to the Triplet Attention mechanism. The combination of attention expands the receptive field and improves the sensitivity to details of adjacent pixels, making small targets and fine-grained features clearer. Finally, a three-headed network model is used to predict the classification, regression and orientation information of the target image.

[0017] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. It should be understood that the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure, wherein: Figure 1 This is a flowchart illustrating an end-to-end target detection and orientation recognition method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of an end-to-end target detection and orientation recognition method provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of a Triplet Attention mechanism provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of the C2f module of a Yolov8 baseline model provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of an end-to-end target detection and orientation recognition device provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure.

[0019] In the accompanying diagram, markers with the same last two digits correspond to the same elements. It should be noted that the elements in the diagram are schematic and not drawn to scale. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are also within the scope of protection of this disclosure.

[0021] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, the statement of “connecting” or “coupling” two or more parts together shall mean that these parts are directly joined together or joined through one or more intermediate components.

[0022] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of the phrase "embodiment" in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists, A and B exist simultaneously, or B exists. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0024] Furthermore, in all embodiments of this disclosure, terms such as “first” and “second” are used only to distinguish one component (or part of a component) from another component (or another part of a component).

[0025] In the description of this application, unless otherwise stated, "multiple" means two or more (including two), and similarly, "multiple groups" means two or more (including two groups).

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0027] In view of the problems existing in the prior art, the present disclosure provides an end-to-end target detection and orientation recognition method. Figure 1 This is a flowchart illustrating an end-to-end target detection and orientation recognition method provided in an embodiment of this disclosure, as shown below. Figure 1 As shown, the end-to-end target detection and orientation recognition method includes: S110. Perform feature extraction on the target image to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer.

[0028] In a specific implementation, feature extraction is performed on the target image to obtain first image feature information of the first downsampling layer, second image feature information of the second downsampling layer, and third image feature information of the third downsampling layer, including: The target image is downsampled to obtain the first initial image feature information of the first downsampling layer, the second initial image feature information of the second downsampling layer, and the third initial image feature information of the third downsampling layer. Based on the triple attention mechanism, feature extraction is performed on the first initial image feature information, the second initial image feature information, and the third initial image feature information to obtain the first image feature information, the second image feature information, and the third image feature information.

[0029] Specifically, in combination Figure 2 and Figure 3 First, the input target image is downsampled by 2, 4, 8, 16, and 32 times. After obtaining the initial image feature information of different sampling layers by downsampling the input target image, the initial image feature information of different sampling layers is extracted based on the triple attention mechanism to obtain the first image feature information, the second image feature information, and the third image feature information.

[0030] In a specific example, the image feature information acquired by the 8x downsampling layer (i.e., L / 8) is used as the first initial image feature information of the first downsampling layer, the image feature information acquired by the 16x downsampling layer (i.e., L / 16) is used as the second initial image feature information of the second downsampling layer, and the image feature information acquired by the 32x downsampling layer (i.e., L / 32) is used as the third initial image feature information of the third downsampling layer. Then, based on the triple attention mechanism, feature extraction is performed on the first initial image feature information, the second initial image feature information, and the third initial image feature information to obtain the first image feature information, the second image feature information, and the third image feature information, respectively.

[0031] In a specific implementation, the first initial image feature information, the second initial image feature information, and the third initial image feature information are processed based on a triple attention mechanism to obtain the first image feature information, the second image feature information, and the third image feature information, including: The first, second, and third initial image feature information are respectively input into the first, second, and third branches of the triple attention mechanism. Based on the first branch, the spatial orientation attention value of the first, second, and third initial image feature information is calculated; based on the second branch, the width orientation attention value of the first, second, and third initial image feature information is calculated; and based on the third branch, the height orientation attention value of the first, second, and third initial image feature information is calculated. Based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the first initial image feature information, the first image feature information is determined; based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the second initial image feature information, the second image feature information is determined; and based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the third initial image feature information, the third image feature information is determined.

[0032] The triple attention mechanism includes a first branch, a second branch, and a third branch.

[0033] Specifically, in combination Figure 3 The triple attention mechanism consists of a first branch, a second branch, and a third branch. The first branch includes a pooling layer, a convolutional layer, a normalization layer, and an activation layer. The second branch includes a first-dimensional adjustment layer, a pooling layer, a convolutional layer, a normalization layer, an activation layer, and a second-dimensional adjustment layer. The third branch includes a first-dimensional adjustment layer, a pooling layer, a convolutional layer, a normalization layer, an activation layer, and a second-dimensional adjustment layer. Finally, the spatial orientation attention value, width orientation attention value, and height orientation attention value output by the first, second, and third branches are output through an average pooling layer.

[0034] In a specific example, the feature vector of the first initial image feature information is a matrix of [C, H, W], where C is the channel, H is the height, and W is the width. After receiving the first initial image feature information, the triple attention mechanism performs channel pooling in the pooling layer of the first branch, followed by a 7x7 convolution (Conv) operation in the convolutional layer. The convolution operation primarily focuses on spatial attention. After normalization by the normalization layer, the spatial attention value is calculated by the activation layer. The second branch first rotates the feature vector of the first initial image feature information to [H, C, W] through a first dimension adjustment layer, performs width pooling in the pooling layer, and then performs a 7x7 convolution (Conv) operation in the convolutional layer. The convolution operation primarily focuses on width attention. After normalization by the normalization layer, the width attention value is calculated by the activation layer. Attention value; the third branch first rotates the feature vector of the first initial image feature information to [W, H, C] through the first dimension adjustment layer, performs height pooling in the pooling layer, and then performs a 7x7 convolution (Conv) operation in the convolution layer. The convolution operation mainly focuses on the attention in the height direction. After the attention in the spatial direction is normalized by the normalization layer, the attention value in the height direction is calculated through the activation layer. After the second and third branches have finished calculating the attention value, the rotated feature vector is rotated back to the original dimension through the second dimension adjustment layer. Finally, the average pooling layer fuses the spatial direction attention value, width direction attention value and height direction attention value of the three branches and multiplies it with the feature vector of the first initial image feature information to obtain a feature that simultaneously contains "channel-space-position" cross-dependency features. Since the rotation operation is used instead of convolution calculation in the triple attention mechanism, the number of parameters and the amount of computation are reduced.

[0035] The process of extracting features from the second and third initial image features based on the Triplet Attention mechanism is the same as the process of processing the first initial image features, and this embodiment will not be described in detail.

[0036] In this implementation, the Triplet Attention mechanism is introduced. The Triplet Attention mechanism consists of three branches (i.e., the first branch, the second branch, and the third branch). It captures the interaction between the three dimensions of channel, height, and width through an innovative three-branch structure and calculates the attention value. Then, based on the calculated spatial direction attention value, width direction attention value, and height direction attention value, attention fusion is performed on the initial image feature information of different sampling layers, which enhances the feature representation ability of the target and reduces the large number of parameters caused by deep convolution, making the model more lightweight.

[0037] S120. Extract key features from the first image feature information and the second image feature information to obtain the first key feature information and the second key feature information.

[0038] In step S110, the first image feature information is obtained by downsampling by 8 times and extracting features based on a triple attention mechanism, the second image feature information is obtained by downsampling by 16 times and extracting features based on a triple attention mechanism, and the third image feature information is obtained by downsampling by 32 times and extracting features based on a triple attention mechanism. That is, the first image feature information is high-resolution feature information, the second image feature information is medium-resolution feature information, and the third image feature information is low-resolution feature information. Since the low-resolution feature information basically only contains semantic information and does not include feature information, in step S120, the process of extracting key features only extracts them from the first image feature information and the second image feature information.

[0039] In the specific implementation, the C2f module based on the Yolov8 baseline model extracts key features from the first image feature information and the second image feature information respectively, to obtain the first key feature information and the second key feature information.

[0040] Among them, such as Figure 4 As shown, the C2f module of the YOLOv8 baseline model includes a first convolutional layer (1×1 Conv), a split layer, multiple backbone neck networks (Bottleneck), a concatenation layer (Concat), and a second convolutional layer (1×1 Conv).

[0041] Specifically, the C2f module based on the Yolov8 baseline model extracts key features from the first and second image feature information to obtain first and second key feature information, including: performing channel dimensionality reduction on the first and second image feature information to obtain first and second dimensionality-reduced image feature information; segmenting the first dimensionality-reduced image feature information into a first first sub-dimensionality-reduced image feature information and a second first sub-dimensionality-reduced image feature information, and segmenting the second dimensionality-reduced image feature information into a first second sub-dimensionality-reduced image feature information and a second second sub-dimensionality-reduced image feature information; extracting key features from the first first sub-dimensionality-reduced image feature information based on multiple backbone neck networks, and merging the extracted first first sub-dimensionality-reduced image feature information with the second first sub-dimensionality-reduced image feature information to obtain first key feature information; and extracting key features from the first second sub-dimensionality-reduced image feature information based on multiple backbone neck networks, and merging the extracted first second second sub-dimensionality-reduced image feature information with the second second second sub-dimensionality-reduced image feature information to obtain second key feature information.

[0042] Specifically, the structure of the C2f module in the Yolov8 baseline model is as follows: Figure 4 As shown, the feature vector of the first image feature information is a matrix of [C, H, W]. First, based on the first convolutional layer of 1×1, the number of channels of the feature vector of the first image feature information is reduced to C / 2. Then, the feature vector after the channel reduction is divided into two branches, namely the first sub-dimensionality reduced image feature information and the second sub-dimensionality reduced image feature information. The first sub-dimensionality reduced image feature information is processed by n backbone neck networks for key feature extraction. The second sub-dimensionality reduced image feature information is directly output to the connection layer. The connection layer concatenates the key feature extraction results of different backbone neck networks with the second sub-dimensionality reduced image feature information, so that the number of channels of the feature vector of the first image feature information is restored to C. Finally, the first key feature information is output through a 1×1 convolutional layer.

[0043] The processing procedure for the feature vector of the second image feature information is the same as that for the feature vector of the first image feature information, and this embodiment will not be described in detail.

[0044] The C2f module of the Yolov8 baseline model enhances key features such as target texture, edges, and shape, while also suppressing sea clutter, noise, and cloud interference. It introduces cross-spatial axis attention, enhancing spatial modeling capabilities and making it easier for the model to focus on details of the head and tail of the target object in the image, thus improving pose estimation. The C2f module is limited by the size of the convolutional kernels in the neck network, making it prone to missing small targets. However, by connecting the input of the C2f module to the output of the triple attention mechanism, the C2f module is combined with Triplet Attention, expanding the receptive field and improving sensitivity to details of adjacent pixels, resulting in clear detection of small targets and fine-grained features.

[0045] S130. Perform self-attention calculation on the first key feature information and the second key feature information, and obtain the first target image feature information and the second target image feature information based on the self-attention calculation result.

[0046] In step S120, key features are extracted from the first image feature information and the second image feature information to obtain the first key feature information and the second key feature information. Since the first key feature information and the second key feature information are key feature information corresponding to different sampling layers, and the image size information corresponding to different sampling layers is different, it is first necessary to perform feature alignment and channel alignment on the first key feature information corresponding to the first downsampling layer and the second key feature information corresponding to the second downsampling layer, and then perform detail compensation through self-attention calculation.

[0047] In a specific implementation, self-attention calculation is performed on the first key feature information and the second key feature information, and the first target image feature information and the second target image feature information are obtained based on the self-attention calculation results. This includes: performing feature alignment and channel alignment processing on the first key feature information and the second key feature information to obtain the first target key feature information; performing self-attention calculation on the first target key feature information and the second key feature information respectively, and obtaining the first target image feature information and the second target image feature information based on the self-attention calculation results.

[0048] Specifically, after performing feature alignment and channel alignment processing on the first key feature information and the second key feature information, the first target key feature information is obtained, including: obtaining the first size information of the first image of the first downsampling layer and the second size information of the second image of the second downsampling layer; determining the first initial key feature information of the first key feature information under the second size information based on the first size information, the second size information and the first key feature information; and performing channel alignment processing on the first initial key feature information and the second key feature information to obtain the first target key feature information.

[0049] The first size information of the first image includes height information and width information, and the second size information of the second image includes height information and width information. The feature vector of the first key feature information is [C1, H1, W1], and the feature vector of the second key feature information is [C2, H2, W2]. At this time, the height information of the first image is H1 and the width information is W1, and the height information of the second image is H2 and the width information is W2.

[0050] For example, the feature vector of the first key feature information obtained after processing the first initial image feature information corresponding to the 8x downsampling layer is [C1, H1, W1], the feature vector of the second key feature information obtained after processing the second initial image feature information corresponding to the 16x downsampling layer is [C2, H2, W2], and the feature vector of the third image feature information obtained after processing the third initial image feature information corresponding to the 32x downsampling layer is [C3, H3, W3]. The number of channels of C1, C2, and C3 are different, depending on the network width, and are preset values, such as C1 is 256, C2 is 512, C3 is 512, H1=2H2=4H3, and W1=2W2=4W3.

[0051] The specific steps for feature alignment and channel alignment of the first key feature information corresponding to the first downsampling layer and the second key feature information corresponding to the second downsampling layer are as follows: Specifically, spatial size alignment is performed based on differentiable differences, and differentiable interpolation is used to align the first key feature information. The spatial dimensions become The formula is as follows:

[0052] in, This represents the feature value at coordinates (x, y) in the first key feature information. Ω represents the feature value of the key feature information of the first target at coordinates (x', y'). The coordinate domain represents the first key feature information. (.,.) is a differentiable interpolation kernel function used to calculate the contribution weight of coordinates (x',y') to coordinates (x,y). H1 represents the height information of the first image, and W1 represents the width information of the first image. For the height information of the second image, The width information of the second image is used, and the coordinate domain of the first target key feature information is the same as that of the second key feature information.

[0053] Then, align the channel numbers, using the following main formula:

[0054] in, It maps the channel to the target dimension through linear projection.

[0055] After obtaining the first target key feature information based on feature alignment and channel alignment processing, self-attention calculation is performed on the first target key feature information and the second key feature information respectively, and the first target image feature information and the second target image feature information are obtained according to the self-attention calculation results.

[0056] It should be noted that the purpose of aligning the size and channels of the first key feature information and the second key feature information is to ensure the feasibility of subsequent attention calculation based on the first key feature information and the second key feature information.

[0057] In a specific implementation, self-attention calculations are performed on the first target key feature information and the second key feature information, respectively, and the first target image feature information and the second target image feature information are obtained based on the self-attention calculation results. This includes: flattening the first target key feature information and the second key feature information, and performing self-attention calculations on the flattened first target key feature information and the second key feature information to obtain a first self-attention value corresponding to the first target key feature information and a second self-attention value corresponding to the second key feature information; determining the first target image feature information based on the first target key feature information and the first self-attention value; and determining the second target image feature information based on the second key feature information and the second self-attention value.

[0058] Specifically, the key feature information of the first target and the key feature information of the second target are flattened into the form of [H×W, C], and local self-attention is calculated respectively to capture the spatial dependency relationship within the same feature layer, improve the expressive power of local features, and keep the amount of computation controllable.

[0059] The formula for flattening a feature image after spatial and channel alignment is as follows:

[0060] Where n is the flattened sequence index, and c is the channel index. .

[0061] After alignment and flattening, local attention is applied to the input features, and computation is performed within a window m:

[0062]

[0063]

[0064] in, Let c represent the aligned and flattened input feature, a scalar at channel index c and spatial location (u,v). This represents the value obtained after projection, a scalar at output channel index c' and spatial location (u,v). The elements represent the learnable projective weight matrix, where c is the number of input channels and c' is the number of output channels.

[0065] Local window attention is calculated as follows:

[0066] Where m is the window index, (u,v) are the spatial coordinates of the target position within the window, s and t represent the continuous coordinates within the window, and d is the dimension of each head. It is the attention result of position (u,v) within window m on output channel c'.

[0067] Convergence of attention on the bulls:

[0068] Where H is the number of heads. This represents the attention output of the h-th head at position (u, v) in window m; This represents the output projection matrix of the h-th head. This indicates a splicing operation along the head edge channel dimension. This represents the multi-head attention output at position (u,v) within window m.

[0069] After local self-attention calculation, the first target image feature information is determined based on the first target key feature information and the first self-attention value; the second target image feature information is determined based on the second key feature information and the second self-attention value.

[0070] That is, the first target key feature information after self-attention calculation is reshaped into the first size information, and then residual connection is made with the first key feature information. The second key feature information after self-attention calculation is also residually connected with the second key feature information to ensure stable gradient propagation and avoid gradient vanishing or feature degradation in the early stage of training.

[0071] S140. Based on the first target image feature information, the second target image feature information, and the third image feature information, determine the classification information, regression information, and orientation information of the target image.

[0072] In a specific implementation, the classification information, regression information, and orientation information of the target image are determined based on the first target image feature information, the second target image feature information, and the third image feature information. This includes: determining the classification information and regression information of the target objects included in the target image based on the first target image feature information, the second target image feature information, and the third image feature information; determining the position information of the center point and head point of the target objects included in the target image based on the first target image feature information, the second target image feature information, and the third image feature information; and determining the orientation information of the target objects included in the target image based on the position information of the center point and head point of the target objects included in the target image.

[0073] Existing head network models are used for regression and classification predictions, but they cannot predict the head position of a target object or further identify its orientation. Most studies focus specifically on the orientation of the target object, neglecting regression and classification predictions. To achieve predictions that integrate classification, regression, and orientation information while sharing the features of the detection head, a new detection head, head-head, is designed. It adds a new branch to the original model to determine the orientation information of the target object in the target image.

[0074] The orientation information of the target object included in the target image satisfies:

[0075] in, , These are the x and y coordinates of the head points of the target objects included in the target image. , These are the x and y coordinates of the center point of the target object included in the target image.

[0076] It should be noted that the determination of the classification, regression, and orientation information of the target image based on the first, second, and third target image feature information is implemented using a head network model. The head network model provided in this embodiment, compared to existing head network models, adds a detection head. During the training of this new head network model, in order to enable the model to clearly know which direction is the target's travel direction, i.e., the orientation information, a novel loss function—the orientation constraint loss function—is designed, and its formula is as follows:

[0077] Since the head network model makes predictions in multiple directions, the direction vectors of the predicted center point and the head point are normalized to the direction vectors of the true center point and the head point.

[0078] when When the loss value is 0, it indicates that the orientation information is reversed. A loss value of 1 indicates that the orientation information is correct.

[0079] To measure the accuracy of predicting the center point and head point positions, the SmoothL1 loss function was used, and its formula is as follows:

[0080] Among them, C and The coordinates of the predicted center point and the true center point are H and , respectively. These are the predicted head point and the actual head point coordinates, respectively. S is the normalization factor used to eliminate the impact of scale differences on the loss.

[0081] Meanwhile, considering classification information, orientation information, confidence level, and regression information, the final loss is calculated as follows:

[0082] Where L' is the loss value of the orientation information. It is the loss value of classification information. It is the loss value of regression information. , , , This represents the weighting factor. , , , The sum is 1.

[0083] Based on the final loss calculation, the parameters of the head network model are modified until the final calculated loss value meets the preset loss value, which is the target head network model. Then, the first target image feature information, the second target image feature information, and the third image feature information are input into the target head network model. Based on the target head network model, the classification information, regression information, and orientation information of the target image are determined.

[0084] The end-to-end target detection and orientation recognition method provided in this disclosure first extracts features from the target image using a Triplet Attention mechanism. This mechanism consists of three branches (a first branch, a second branch, and a third branch). The three-branch structure captures the interaction relationships between the target image and the channel, height, and width dimensions of different downsampling layers and calculates attention values. Then, based on the calculated spatial, width, and height attention values, attention fusion is performed on the initial image feature information of different sampling layers to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer. This enhances the feature representation capability of the target image while reducing the large number of parameters caused by deep convolution, making the model more lightweight. Next, the C2f module of the Yolov8 baseline model extracts key features from the first and second image feature information to obtain first and second key feature information. The input of the C2f module is connected to the output of the triple attention mechanism. The C2f module is connected to the Triplet Attention mechanism. The combination of attention expands the receptive field and improves the sensitivity to details of adjacent pixels, making small targets and fine-grained features clearer. Finally, a three-headed network model is used to predict the classification, regression and orientation information of the target image.

[0085] Based on the above embodiments, this disclosure also provides an end-to-end target detection and orientation recognition device. Figure 5 This is a schematic diagram of the structure of an end-to-end target detection and orientation recognition device provided in an embodiment of this disclosure, as shown below. Figure 5 As shown, the end-to-end target detection and orientation recognition device includes: The first feature information processing module 610 is used to extract features from the target image to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer. The second feature information processing module 620 is used to extract key features from the first image feature information and the second image feature information to obtain the first key feature information and the second key feature information. The third feature information processing module 630 is used to perform self-attention calculation on the first key feature information and the second key feature information, and to obtain the first target image feature information and the second target image feature information based on the self-attention calculation result. The detection module 640 is used to determine the classification information, regression information and orientation information of the target image based on the first target image feature information, the second target image feature information and the third image feature information.

[0086] The end-to-end target detection and orientation recognition device provided in this disclosure first extracts features from the target image using a Triplet Attention mechanism. This mechanism consists of three branches (a first branch, a second branch, and a third branch). The three-branch structure captures the interaction relationships between the target image and the channel, height, and width dimensions of different downsampling layers and calculates attention values. Then, based on the calculated spatial, width, and height attention values, attention fusion is performed on the initial image feature information of different sampling layers to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer. This enhances the feature representation capability of the target image while reducing the large number of parameters caused by deep convolution, making the model more lightweight. Next, the C2f module of the Yolov8 baseline model extracts key features from the first and second image feature information to obtain first and second key feature information. The input of the C2f module is connected to the output of the triple attention mechanism. The C2f module is connected to the Triplet Attention mechanism. The combination of attention expands the receptive field and improves the sensitivity to details of adjacent pixels, making small targets and fine-grained features clearer. Finally, a three-headed network model is used to predict the classification, regression and orientation information of the target image.

[0087] In a specific implementation, the step of extracting features from the target image to obtain first image feature information of the first downsampling layer, second image feature information of the second downsampling layer, and third image feature information of the third downsampling layer includes: The target image is downsampled to obtain the first initial image feature information of the first downsampling layer, the second initial image feature information of the second downsampling layer, and the third initial image feature information of the third downsampling layer. Based on the triple attention mechanism, feature extraction is performed on the first initial image feature information, the second initial image feature information, and the third initial image feature information to obtain the first image feature information, the second image feature information, and the third image feature information.

[0088] In a specific implementation, the triple attention mechanism includes a first branch, a second branch, and a third branch; The method of processing the first initial image feature information, the second initial image feature information, and the third initial image feature information based on the triple attention mechanism to obtain the first image feature information, the second image feature information, and the third image feature information includes: The first initial image feature information, the second initial image feature information, and the third initial image feature information are respectively input into the first branch, the second branch, and the third branch. Based on the first branch, the spatial orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. Based on the second branch, the width orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. Based on the third branch, the height orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. The first image feature information is determined based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the first initial image feature information. The second image feature information is determined based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the second initial image feature information. The third image feature information is determined based on the spatial direction attention value, width direction attention value, height direction attention value, and the third initial image feature information.

[0089] In a specific implementation, the step of extracting key features from the first image feature information and the second image feature information to obtain first key feature information and second key feature information includes: The first image feature information and the second image feature information are respectively subjected to channel dimensionality reduction processing to obtain the first dimensionality-reduced image feature information and the second dimensionality-reduced image feature information; The first dimensionality reduction image feature information is divided into a first first sub-dimensionality reduction image feature information and a second first sub-dimensionality reduction image feature information, and the second dimensionality reduction image feature information is divided into a first second sub-dimensionality reduction image feature information and a second second sub-dimensionality reduction image feature information; Based on multiple backbone neck networks, key features are extracted from the feature information of the first sub-dimensionality reduction image. The feature information of the first sub-dimensionality reduction image after key feature extraction is then merged with the feature information of the second sub-dimensionality reduction image to obtain the first key feature information. Based on multiple backbone neck networks, key features are extracted from the feature information of the first second sub-dimensionality reduced image. The feature information of the first second sub-dimensionality reduced image after key feature extraction is then merged with the feature information of the second second sub-dimensionality reduced image to obtain the second key feature information.

[0090] In a specific implementation, the step of performing self-attention calculation on the first key feature information and the second key feature information, and obtaining the first target image feature information and the second target image feature information based on the self-attention calculation result, includes: After performing feature alignment and channel alignment processing on the first key feature information and the second key feature information, the first target key feature information is obtained; Self-attention calculations are performed on the first target key feature information and the second target key feature information respectively, and the first target image feature information and the second target image feature information are obtained based on the self-attention calculation results.

[0091] In a specific implementation, the step of performing feature alignment and channel alignment processing on the first key feature information and the second key feature information to obtain the first target key feature information includes: Obtain the first size information of the first image of the first downsampling layer and the second size information of the second image of the second downsampling layer; Based on the first size information, the second size information, and the first key feature information, determine the first initial key feature information under the second size information; The first initial key feature information and the second key feature information are channel aligned to obtain the first target key feature information.

[0092] In a specific implementation, the step of performing self-attention calculations on the first target key feature information and the second key feature information, respectively, and obtaining the first target image feature information and the second target image feature information based on the self-attention calculation results, includes: The first target key feature information and the second key feature information are flattened respectively, and self-attention calculation is performed on the flattened first target key feature information and the second key feature information to obtain the first self-attention value corresponding to the first target key feature information and the second self-attention value corresponding to the second key feature information. The first target image feature information is determined based on the first target key feature information and the first self-attention value; The second target image feature information is determined based on the second key feature information and the second self-attention value.

[0093] In a specific implementation, determining the classification information, regression information, and orientation information of the target image based on the first target image feature information, the second target image feature information, and the third image feature information includes: Based on the first target image feature information, the second target image feature information, and the third image feature information, determine the classification information and regression information of the target objects included in the target image; Based on the first target image feature information, the second target image feature information, and the third image feature information, determine the position information of the center point and head point of the target object included in the target image; Based on the position information of the center point and head point of the target object included in the target image, the orientation information of the target object included in the target image is determined.

[0094] This application also provides a computer device, please refer to the following for details. Figure 6 , Figure 6 This is a basic structural block diagram of the computer device in this embodiment.

[0095] The computer device includes a memory 510 and a processor 520 that are interconnected via a system bus. It should be noted that only a computer device with components 510-520 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components may be implemented alternatively. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0096] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0097] The memory 510 includes at least one type of readable storage medium, including non-volatile memory or volatile memory, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. RAM may include static RAM or dynamic RAM. In some embodiments, the memory 510 may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory 510 may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card equipped on the computer device. Of course, the memory 510 may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory 510 is typically used to store the operating system and various application software installed on the computer device, such as the program code of the method described above. In addition, the memory 510 may also be used to temporarily store various types of data that have been output or will be output.

[0098] The processor 520 is typically used to perform the overall operation of a computer device. In this embodiment, the memory 510 is used to store program code or instructions, including computer operation instructions. The processor 520 is used to execute the program code or instructions stored in the memory 510 or to process data, such as program code that runs the methods described above.

[0099] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus system can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0100] Another embodiment of this application also provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads computer-readable program code stored in the computer-readable medium, enabling the processor to execute the functional actions specified in each step or combination of steps in the above method; and to generate means for implementing the functional actions specified in each block or combination of blocks in the block diagram.

[0101] Computer-readable media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any suitable combination thereof, wherein the memory is used to store program code or instructions, the program code including computer operation instructions, and the processor is used to execute the program code or instructions of the above-described methods stored in the memory.

[0102] The definitions of memory and processor can be found in the description of the foregoing computer device embodiments, and will not be repeated here.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0104] In the various embodiments of this application, the functional units or modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] Unless otherwise expressly indicated by the context, the singular form of words used herein and in the appended claims includes the plural form, and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Similarly, the terms “comprising” and “including” shall be interpreted as including rather than exclusively. Likewise, the terms “including” and “or” shall be interpreted as including unless such interpretation is expressly prohibited herein. Where the term “example” is used herein, particularly when it follows a set of terms, the “example” is merely exemplary and illustrative and should not be considered exclusive or extensive.

[0107] Further aspects and scope of adaptation become apparent from the description provided herein. It should be understood that various aspects of this application may be implemented individually or in combination with one or more other aspects. It should also be understood that the descriptions and specific embodiments herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0108] Several embodiments of this disclosure have been described in detail above. However, it is obvious that those skilled in the art can make various modifications and variations to the embodiments of this disclosure without departing from the spirit and scope of this disclosure. The scope of protection of this disclosure is defined by the appended claims.

Claims

1. An end-to-end target detection and orientation recognition method, characterized in that, include: Feature extraction is performed on the target image to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer; Key features are extracted from the first image feature information and the second image feature information to obtain first key feature information and second key feature information; Self-attention calculation is performed on the first key feature information and the second key feature information, and the first target image feature information and the second target image feature information are obtained based on the self-attention calculation result; Based on the first target image feature information, the second target image feature information, and the third image feature information, the classification information, regression information, and orientation information of the target image are determined.

2. The method according to claim 1, characterized in that, The step of extracting features from the target image to obtain first image feature information of the first downsampling layer, second image feature information of the second downsampling layer, and third image feature information of the third downsampling layer includes: The target image is downsampled to obtain the first initial image feature information of the first downsampling layer, the second initial image feature information of the second downsampling layer, and the third initial image feature information of the third downsampling layer. Based on the triple attention mechanism, feature extraction is performed on the first initial image feature information, the second initial image feature information, and the third initial image feature information to obtain the first image feature information, the second image feature information, and the third image feature information.

3. The method according to claim 2, characterized in that, The triple attention mechanism includes a first branch, a second branch, and a third branch; The method of processing the first initial image feature information, the second initial image feature information, and the third initial image feature information based on the triple attention mechanism to obtain the first image feature information, the second image feature information, and the third image feature information includes: The first initial image feature information, the second initial image feature information, and the third initial image feature information are respectively input into the first branch, the second branch, and the third branch. Based on the first branch, the spatial orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. Based on the second branch, the width orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. Based on the third branch, the height orientation attention value of the first initial image feature information, the second initial image feature information, and the third initial image feature information is calculated. The first image feature information is determined based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the first initial image feature information. The second image feature information is determined based on the spatial orientation attention value, width orientation attention value, height orientation attention value, and the second initial image feature information. The third image feature information is determined based on the spatial direction attention value, width direction attention value, height direction attention value, and the third initial image feature information.

4. The method according to claim 1, characterized in that, The step of extracting key features from the first image feature information and the second image feature information to obtain first key feature information and second key feature information includes: The first image feature information and the second image feature information are respectively subjected to channel dimensionality reduction processing to obtain the first dimensionality-reduced image feature information and the second dimensionality-reduced image feature information; The first dimensionality reduction image feature information is divided into a first first sub-dimensionality reduction image feature information and a second first sub-dimensionality reduction image feature information, and the second dimensionality reduction image feature information is divided into a first second sub-dimensionality reduction image feature information and a second second sub-dimensionality reduction image feature information; Based on multiple backbone neck networks, key features are extracted from the feature information of the first sub-dimensionality reduction image. The feature information of the first sub-dimensionality reduction image after key feature extraction is then merged with the feature information of the second sub-dimensionality reduction image to obtain the first key feature information. Based on multiple backbone neck networks, key features are extracted from the feature information of the first second sub-dimensionality reduced image. The feature information of the first second sub-dimensionality reduced image after key feature extraction is then merged with the feature information of the second second sub-dimensionality reduced image to obtain the second key feature information.

5. The method according to claim 1, characterized in that, The step of performing self-attention calculation on the first key feature information and the second key feature information, and obtaining the first target image feature information and the second target image feature information based on the self-attention calculation result, includes: After performing feature alignment and channel alignment processing on the first key feature information and the second key feature information, the first target key feature information is obtained; Self-attention calculations are performed on the first target key feature information and the second target key feature information respectively, and the first target image feature information and the second target image feature information are obtained based on the self-attention calculation results.

6. The method according to claim 5, characterized in that, The step of performing feature alignment and channel alignment processing on the first key feature information and the second key feature information to obtain the first target key feature information includes: Obtain the first size information of the first image of the first downsampling layer and the second size information of the second image of the second downsampling layer; Based on the first size information, the second size information, and the first key feature information, determine the first initial key feature information under the second size information; The first initial key feature information and the second key feature information are channel aligned to obtain the first target key feature information.

7. The method according to claim 5, characterized in that, The step of performing self-attention calculations on the first target key feature information and the second target key feature information respectively, and obtaining the first target image feature information and the second target image feature information based on the self-attention calculation results, includes: The first target key feature information and the second key feature information are flattened respectively, and self-attention calculation is performed on the flattened first target key feature information and the second key feature information to obtain the first self-attention value corresponding to the first target key feature information and the second self-attention value corresponding to the second key feature information. The first target image feature information is determined based on the first target key feature information and the first self-attention value; The second target image feature information is determined based on the second key feature information and the second self-attention value.

8. The method according to claim 1, characterized in that, The step of determining the classification information, regression information, and orientation information of the target image based on the first target image feature information, the second target image feature information, and the third image feature information includes: Based on the first target image feature information, the second target image feature information, and the third image feature information, determine the classification information and regression information of the target objects included in the target image; Based on the first target image feature information, the second target image feature information, and the third image feature information, determine the position information of the center point and head point of the target object included in the target image; Based on the position information of the center point and head point of the target object included in the target image, the orientation information of the target object included in the target image is determined.

9. An end-to-end target detection and orientation recognition device, characterized in that, include: The first feature information processing module is used to extract features from the target image to obtain the first image feature information of the first downsampling layer, the second image feature information of the second downsampling layer, and the third image feature information of the third downsampling layer. The second feature information processing module is used to extract key features from the first image feature information and the second image feature information to obtain first key feature information and second key feature information. The third feature information processing module is used to perform self-attention calculation on the first key feature information and the second key feature information, and obtain the first target image feature information and the second target image feature information based on the self-attention calculation result; The detection module is used to determine the classification information, regression information, and orientation information of the target image based on the first target image feature information, the second target image feature information, and the third image feature information.

10. A computer device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 8.

Images