A method and system for detecting and identifying a rotating frame of a target vehicle

Abstract

The present invention provides a method and system for detecting and identifying a rotating frame of a target vehicle, and relates to the field of vehicle detection technology. The method provided by the present invention obtains an optical image, which includes a target vehicle to be detected; inputs the optical image into a feature extraction network configured with a spatial state model, and outputs a plurality of first feature maps corresponding to the optical image; inputs the plurality of first feature maps into a feature pyramid network, and outputs a plurality of second feature maps; inputs the plurality of second feature maps into a rotating target detection network, and outputs the category of the target vehicle and a rectangular rotating frame of the target vehicle on the optical image, wherein the rectangular rotating frame is used to characterize the position and rotation angle of the target vehicle. Compared with the target detection method in the related art, the method provided by the present invention can improve the feature extraction capability as needed, thereby improving the detection accuracy, and meeting the user's usage needs in different usage scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle detection, and in particular to a method and system for detecting and identifying a rotating frame of a target vehicle. Background Art

[0002] With the rapid development of intelligent technology, the detection of small rotating targets in optical images has gained widespread attention in fields such as traffic monitoring and autonomous driving. It can also be used to detect target vehicles in images, determining their class and location within the optical image. However, existing detection methods face numerous challenges in accuracy and robustness due to factors such as the diversity of vehicles, small target size, complex fine-grained features, and varying poses.

[0003] Traditional target vehicle detection methods primarily rely on manual feature extraction techniques. This approach is typically tailored to specific detection tasks, and the feature extraction process relies heavily on manual design and experience, directly impacting detection effectiveness and algorithm performance. These traditional methods have significant limitations, including limited data volumes, difficulty generalizing manually extracted features, poor algorithm portability, high time complexity, and insufficient robustness in complex environments. Consequently, these methods are limited in scope and suffer from poor detection accuracy. This makes them particularly difficult to meet user needs in complex and changing environments, limiting their widespread practical application.

[0004] Therefore, there is an urgent need for a target vehicle rotation frame detection and recognition method and system that can quickly and accurately identify target vehicles at different angles and rotation states to meet user needs in different usage scenarios. Summary of the Invention

[0005] The embodiments of the present invention provide a method and system for detecting and identifying a target vehicle rotation frame, which can quickly and accurately identify target vehicles at different angles and rotation states, improve detection accuracy, and meet user needs in different usage scenarios.

[0006] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0007] In a first aspect, a method for detecting and recognizing a rotating frame of a target vehicle is provided, the method comprising: acquiring an optical image, the optical image including a target vehicle to be detected; inputting the optical image into a feature extraction network configured with a spatial state model, and outputting a plurality of first feature maps corresponding to the optical image, the plurality of first feature maps respectively having different numbers of channels and sizes; inputting the plurality of first feature maps into a feature pyramid network, and outputting a plurality of second feature maps, the plurality of second feature maps respectively having the same number of channels and different sizes; inputting the plurality of second feature maps into a rotating target detection network, and outputting a category of the target vehicle and a rectangular rotating frame of the target vehicle on the optical image, the rectangular rotating frame being used to characterize the position and rotation angle of the target vehicle.

[0008] In a possible implementation of the first aspect, the feature extraction network includes an image processing module, a first feature extraction module, and multiple second feature extraction modules connected in series in sequence, the image processing module is used to process the optical image to obtain multiple image blocks corresponding to the optical image, the first feature extraction module is used to determine a first feature map corresponding to the optical image based on the multiple image blocks corresponding to the optical image, the first second feature extraction module is used to determine the first feature map corresponding to the optical image based on the first feature map determined by the first feature extraction module, and the nth second feature extraction module is used to determine the first feature map corresponding to the optical image based on the first feature map determined by the n-1th second feature extraction module, wherein each second feature extraction module and the first feature map determined by the first feature extraction module have different channel numbers and sizes, respectively, and n is an integer greater than or equal to 2.

[0009] In a possible implementation of the first aspect, the first feature extraction module includes a visual state space unit and an attention fusion unit, which are connected to each other; the second feature extraction module includes a downsampling processing unit, a visual state space unit, and an attention fusion unit; the downsampling processing unit is connected to the visual state space unit, and the visual state space unit is connected to the attention fusion unit; the downsampling processing unit is configured to: perform downsampling processing on the feature map; the visual state space unit is configured to: perform cross-scan processing on the image block or feature map to obtain feature information, map the feature information based on the hidden state variable to obtain mapping information corresponding to the feature information, and perform cross-merging processing on the mapping information to obtain the feature map;

[0010] The formula for mapping feature information based on hidden state variables is:

[0011] h t =e ΔA h t-1 +Δ B x t ;

[0012] yt =Ch t-1 +Dx t ;

[0013] Among them, h t is the hidden state variable of variable t; x t is the characteristic information of variable t; y t is the mapping information corresponding to the feature information of variable t; A∈R N×N ,B∈R N×1 ,C∈R 1×N and D∈R 1 is the weight matrix;

[0014] The attention fusion unit is used to: perform weighted processing on the feature map based on the channel attention weight to obtain a channel attention weighted feature map, and perform weighted processing on the channel attention weighted feature map based on the spatial attention weight to obtain a first feature map;

[0015] The formula for determining the channel attention weight is:

[0016]

[0017] Among them, M c F is the channel attention weight, σ() is the sigmoid activation function; W0 and W1 are the shared weights of the fully connected layer; is the feature map processed by average pooling, is the feature map processed by maximum pooling;

[0018] The formula for determining the spatial attention weight is:

[0019]

[0020] Among them, M s F is the spatial attention weight; f 7×7 It is a convolution layer with a convolution kernel size of 7×7; is the channel attention weighted feature map after average pooling processing, It is the channel attention weighted feature map after maximum pooling.

[0021] In a possible implementation of the first aspect, the rotation object detection network includes a candidate box generation module and a candidate box classification and regression module;

[0022] The candidate frame generation module is used to: determine multiple horizontal anchor frames with different aspect ratios corresponding to each second feature map based on multiple second feature maps; determine a horizontal anchor frame where a target vehicle exists from the multiple horizontal anchor frames with different aspect ratios corresponding to each second feature map; determine multiple offset values ​​corresponding to each horizontal anchor frame where a target vehicle exists; and determine a tilted candidate frame corresponding to each horizontal anchor frame where a target vehicle exists based on the multiple offset values ​​corresponding to each horizontal anchor frame where a target vehicle exists.

[0023] In a possible implementation of the first aspect, the candidate box classification regression module includes a first fully connected layer and a second fully connected layer connected in parallel;

[0024] The first fully connected layer is used to determine the category of the target vehicle based on the tilted candidate box corresponding to the horizontal anchor box where the target vehicle exists;

[0025] The second fully connected layer is used to: determine a rectangular rotation box corresponding to the tilted candidate box corresponding to each horizontal anchor box containing the target vehicle; map the rectangular rotation box corresponding to the tilted candidate box corresponding to each horizontal anchor box containing the target vehicle to the second feature map based on a preset step size, and obtain coordinate values ​​corresponding to the target vehicle area included in the second feature map; grid the target vehicle area included in the second feature map to obtain a feature map of the target size, determine the grid value of the feature map of the target size at each position in each channel, and generate a rectangular rotation box of the target vehicle on the optical image based on the grid value of the feature map of the target size at each position in each channel;

[0026] Among them, the second feature map includes the coordinate values ​​(x r ,y r ,w r ,h r ,θ) is determined as follows:

[0027]

[0028] Where (x, y, w, h, θ) is the coordinate value corresponding to the rectangular rotation box; s is the preset step size;

[0029]

[0030] The grid value F of the target size feature map at the grid (i, j) of the cth channel c The formula for determining ′(i,j) is:

[0031]

[0032] F cis the feature map F of the target size in the cth channel, n is the number of samples in the grid, area(i,j) is the coordinate set in the grid (i,j), and R() is the rotation transformation.

[0033] In a possible implementation of the first aspect, before inputting multiple second feature maps into a rotating target detection network and outputting the category of the target vehicle and the rectangular rotation box of the target vehicle on the optical image, the method further includes: obtaining a training image, the training image including multiple training anchor frames and multiple real anchor frames; determining a label corresponding to each training anchor frame, the label including a positive sample, a negative sample or an invalid sample; wherein the intersection-and-union ratio of the training anchor frame labeled as a positive sample with any real anchor frame is greater than or equal to a first threshold or the intersection-and-union ratio of the training anchor frame with the target real anchor frame is greater than the intersection-and-union ratio of other training anchor frames with the target real anchor frame; the target real anchor frame is one of the multiple real anchor frames; the intersection-and-union ratio of the training anchor frame labeled as a negative sample with any real anchor frame is less than a second threshold; the intersection-and-union ratio of the training anchor frame labeled as an invalid sample with any real anchor frame is greater than or equal to the second threshold and less than the first threshold; constructing a loss function; based on the loss function, iteratively training the rotating target detection network according to the training anchor frames labeled as positive samples and negative samples to obtain a trained rotating target detection network;

[0034] The loss function L1 is:

[0035]

[0036] Where i is the index of the training anchor box; N is the total number of samples; F cls is the cross entropy loss for classification; F reg is the L1 smoothing loss for regression; represents the true label of the i-th training anchor box; p i is the foreground probability value output by the first fully connected layer; is the offset of the i-th training anchor box relative to the true anchor box under the midpoint offset representation.

[0037] The beneficial effects of the present invention are as follows: the method provided by the present invention extracts features from optical images through a feature extraction network configured with a spatial state model, thereby obtaining finer-grained visual information and effectively and accurately extracting key features from optical images. It is applicable to detection scenarios of complex textures and rotating targets, thereby improving the detection accuracy and robustness of target vehicles. In addition, the method provided by the present invention improves the feature extraction network's ability to focus on key information areas in optical images through an attention fusion unit, reducing interference from background noise and irrelevant areas. The attention fusion unit effectively improves the discriminability of feature representation by fusing the channel attention mechanism with the spatial attention mechanism, further improving the detection accuracy of target vehicles and meeting the usage requirements of rotating target positioning and boundary refinement. Finally, the method provided by the present invention implements candidate box generation and classification regression through a candidate box classification and regression module, respectively, which can effectively improve accuracy. In general, compared with target detection methods in related technologies, the method provided by the present invention can improve feature extraction capabilities and thus detection accuracy, meeting the usage requirements of users in different usage scenarios.

[0038] In a second aspect, the present invention provides a target vehicle rotation frame detection and recognition system, the system comprising: an acquisition module for acquiring an optical image, the optical image including a target vehicle to be detected; a feature extraction network for outputting a plurality of first feature maps corresponding to the optical image based on the input optical image, the plurality of first feature maps respectively having different numbers of channels and sizes; the feature extraction network is configured with a spatial state model; a feature pyramid network for outputting a plurality of second feature maps based on the input plurality of first feature maps, the plurality of second feature maps respectively having the same number of channels and different sizes; a rotating target detection network for outputting the category of the target vehicle and a rectangular rotation frame of the target vehicle on the optical image based on the input plurality of second feature maps, the rectangular rotation frame being used to characterize the position and rotation angle of the target vehicle.

[0039] In a possible implementation of the second aspect, the feature extraction network includes an image processing module, a first feature extraction module, and multiple second feature extraction modules connected in series in sequence, the image processing module is used to process the optical image to obtain multiple image blocks corresponding to the optical image, the first feature extraction module is used to determine a first feature map corresponding to the optical image based on the multiple image blocks corresponding to the optical image, the first second feature extraction module is used to determine the first feature map corresponding to the optical image based on the first feature map determined by the first feature extraction module, and the nth second feature extraction module is used to determine the first feature map corresponding to the optical image based on the first feature map determined by the n-1th second feature extraction module, wherein each second feature extraction module and the first feature map determined by the first feature extraction module have different channel numbers and sizes, respectively, and n is an integer greater than or equal to 2.

[0040] In a possible implementation of the second aspect, the first feature extraction module includes a visual state space unit and an attention fusion unit, which are connected to each other; the second feature extraction module includes a downsampling processing unit, a visual state space unit, and an attention fusion unit; the downsampling processing unit is connected to the visual state space unit, and the visual state space unit is connected to the attention fusion unit; the downsampling processing unit is used to: downsample the feature map; the visual state space unit is used to: cross-scan the image block or the feature map to obtain feature information, map the feature information based on the hidden state variable to obtain mapping information corresponding to the feature information, and cross-merge the mapping information to obtain the feature map;

[0041] The formula for mapping feature information based on hidden state variables is:

[0042]

[0043] y t =Ch t-1 +Dx t ;

[0044] Among them, h t is the hidden state variable of variable t; x t is the characteristic information of variable t; y t is the mapping information corresponding to the feature information of variable t; A∈R N×N ,B∈R N×1 ,C∈R 1×N and D∈R 1 is the weight matrix;

[0045] The attention fusion unit is used to: perform weighted processing on the feature map based on the channel attention weight to obtain a channel attention weighted feature map, and perform weighted processing on the channel attention weighted feature map based on the spatial attention weight to obtain a first feature map;

[0046] The formula for determining the channel attention weight is:

[0047]

[0048] Among them, M c F is the channel attention weight, σ() is the sigmoid activation function; W0 and W1 are the shared weights of the fully connected layer; is the feature map processed by average pooling, is the feature map processed by maximum pooling;

[0049] The formula for determining the spatial attention weight is:

[0050]

[0051] Among them, M s F is the spatial attention weight; f 7×7 It is a convolution layer with a convolution kernel size of 7×7; is the channel attention weighted feature map after average pooling processing, It is the channel attention weighted feature map after maximum pooling.

[0052] In a third aspect, an electronic device is provided, comprising a memory and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code comprises computer instructions, and when the computer instructions are executed by the processor, the electronic device executes a method as in any implementation of the first aspect.

[0053] In a fourth aspect, a computer-readable storage medium is provided, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method in any implementation of the first aspect.

[0054] In a fifth aspect, a computer program product is provided. When the computer program product is run on a computer, the computer is caused to execute the method in any implementation of the first aspect.

[0055] It can be understood that the beneficial effects that can be achieved by the system of the second aspect, the electronic device of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention;

[0057] Figure 2 This is a flow chart of a method for detecting and identifying a rotating frame of a target vehicle according to an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of the structure of a feature extraction network according to an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of the structure of a first feature extraction module and a second feature extraction module according to an embodiment of the present invention;

[0060] Figure 5 A schematic diagram of the structure of a visual state space unit according to an embodiment of the present invention;

[0061] Figure 6 This is a schematic structural diagram of a 2D selective scanning module according to an embodiment of the present invention;

[0062] Figure 7 A schematic diagram of a midpoint offset representation method for a rotating target provided by an embodiment of the present invention;

[0063] Figure 8 The figure is a schematic diagram of the hardware structure of a detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention. In the description of the present invention, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship. For example, A / B can represent A or B. The "or" in the present invention is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, in the description of the present invention, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items.

[0065] In addition, to facilitate a clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or execution order, and the words "first" and "second" do not necessarily mean different.

[0066] In the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as superior or more advantageous than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner to facilitate understanding.

[0067] With the rapid development of intelligent technology, the detection of small rotating targets in optical images has gained widespread attention in fields such as traffic monitoring and autonomous driving. It can also be used to detect target vehicles in images, determining their class and location within the optical image. However, existing detection methods face numerous challenges in accuracy and robustness due to factors such as the diversity of vehicles, small target size, complex fine-grained features, and varying poses.

[0068] Traditional target vehicle detection methods primarily rely on manual feature extraction techniques. This approach is typically tailored to specific detection tasks, and the feature extraction process relies heavily on manual design and experience, directly impacting detection effectiveness and algorithm performance. These traditional methods have significant limitations, including limited data volumes, difficulty generalizing manually extracted features, poor algorithm portability, high time complexity, and insufficient robustness in complex environments. Consequently, these methods are limited in scope and suffer from poor detection accuracy. This makes them particularly difficult to meet user needs in complex and changing environments, limiting their widespread practical application.

[0069] Therefore, there is an urgent need for a target vehicle rotation frame detection and recognition method and system that can quickly and accurately identify target vehicles at different angles and rotation states to meet user needs in different usage scenarios.

[0070] In view of this, an embodiment of the present invention provides a method for detecting and identifying a rotating frame of a target vehicle, the method comprising: acquiring an optical image, the optical image including a target vehicle to be detected; inputting the optical image into a feature extraction network configured with a spatial state model, outputting multiple first feature maps corresponding to the optical image, the multiple first feature maps respectively having different numbers of channels and sizes; inputting the multiple first feature maps into a feature pyramid network, outputting multiple second feature maps, the multiple second feature maps respectively having the same number of channels and different sizes; inputting the multiple second feature maps into a rotating target detection network, outputting the category of the target vehicle and a rectangular rotating frame of the target vehicle on the optical image, the rectangular rotating frame being used to characterize the position and rotation angle of the target vehicle.

[0071] The method provided by the present invention extracts features from optical images through a feature extraction network configured with a spatial state model, which can obtain more fine-grained visual information, and thus can effectively and accurately extract key features in optical images. It can be applied to detection scenarios of complex textures and rotating targets, thereby improving the detection accuracy and robustness of target vehicles. In addition, the method provided by the present invention improves the feature extraction network's ability to focus on key information areas in optical images through an attention fusion unit, reducing interference from background noise and irrelevant areas. The attention fusion unit can effectively improve the discriminability of feature representation by fusing the channel attention mechanism and the spatial attention mechanism, further improving the detection accuracy of target vehicles and meeting the usage requirements of rotating target positioning and boundary refinement. Finally, the method provided by the present invention realizes candidate box generation and classification regression respectively through a candidate box generation module and a candidate box classification regression module, which can effectively improve accuracy. In general, compared with target detection methods in related technologies, the method provided by the present invention can improve feature extraction capabilities and thus improve detection accuracy, meeting the usage requirements of users in different usage scenarios.

[0072] In some embodiments, the method for detecting and identifying a rotating frame on a target vehicle provided by embodiments of the present invention can be performed by a system 100 for detecting and identifying a rotating frame on a target vehicle (hereinafter referred to as detection system 100). By way of example, detection system 100 can be any electronic device 200 with data processing capabilities, such as a general-purpose computer, personal computer, laptop computer, switch, or tablet computer. The specific implementation of detection system 100 is not limited herein.

[0073] Figure 1 The electronic device 200 includes a processor 210 , a memory 220 , and a communication interface 230 .

[0074] The processor 210 may include one or more processing cores. The processor 210 uses various interfaces and lines to connect various parts of the electronic device 200, and executes various functions of the electronic device 200 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 220, and calling data stored in the memory 220. Optionally, the processor 210 can be implemented in the form of at least one hardware of a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA).

[0075] The memory 220 may include a random access memory (RAl) or a read-only memory (ROL). Optionally, the memory 220 includes a non-transitory computer-readable storage medium (NMT). The memory 220 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 220 may include a program storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as an image acquisition function, a data processing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.

[0076] The communication interface 230 is used to communicate with other devices, equipment or communication networks, such as data storage devices, image processing equipment or Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0077] In physical implementation, the aforementioned components (e.g., processor 210, memory 220, and communication interface 230) may be components within the same device (e.g., a laptop). Alternatively, at least two of the components may be provided within the same device, i.e., as different components within a single device, similar to the deployment of devices or components in a distributed system.

[0078] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 200. In other embodiments of the present invention, the electronic device 200 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0079] The following describes a method for detecting and identifying a rotating frame of a target vehicle provided by an embodiment of the present invention in conjunction with the accompanying drawings.

[0080] Figure 2 The flowchart of a method for detecting and identifying a rotating frame of a target vehicle provided by an embodiment of the present invention is as follows. Figure 1 The electronic device 200 shown is executed, that is, executed by the detection system 100. The method may include the following steps:

[0081] S1. Acquire an optical image, where the optical image includes a target vehicle to be detected.

[0082] S2. Input the optical image into a feature extraction network configured with a spatial state model, and output a plurality of first feature maps corresponding to the optical image.

[0083] The multiple first feature maps have different numbers of channels and sizes.

[0084] In some embodiments, see Figure 3 , Figure 3Schematic diagram of the structure of a feature extraction network shown in an embodiment of the present invention, the feature extraction network 120 includes an image processing module 121, a first feature extraction module 122 and multiple second feature extraction modules 123 connected in series in sequence, the image processing module 121 is used to process the optical image to obtain multiple image blocks corresponding to the optical image, the first feature extraction module 122 is used to determine the first feature map (feature map C1) corresponding to the optical image based on the multiple image blocks corresponding to the optical image, the first second feature extraction module 123 (which can also be understood as the second feature extraction module 123 connected to the first feature extraction module 122) is used to determine the first feature map (feature map C2) corresponding to the optical image based on the first feature map (feature map C1) determined by the first feature extraction module 122, the nth second feature extraction module 123 is used to determine the first feature map (feature map Cn) corresponding to the optical image based on the first feature map (feature map Cn-1) determined by the n-1th second feature extraction module 123, Figure 3 In the embodiment, the second second feature extraction module 123 is used to determine the first feature map (feature map C3) corresponding to the optical image based on the first feature map (feature map C2) determined by the first second feature extraction module 123, and the third second feature extraction module 123 is used to determine the first feature map (feature map C4) corresponding to the optical image based on the first feature map (feature map C3) determined by the second second feature extraction module 123. The first feature maps determined by each second feature extraction module 123 and the first feature extraction module 122 have different numbers of channels and sizes, respectively, and n is an integer greater than or equal to 2, that is, the feature maps C1-C4 have different numbers of channels and sizes.

[0085] In one possible implementation, see Figure 4 , Figure 4 This is a structural schematic diagram of a first feature extraction module and a second feature extraction module shown in an embodiment of the present invention, wherein the first feature extraction module 122 includes a visual state space (VSS) unit 1221 and an attention fusion unit 1222, and the visual state space unit 1221 and the attention fusion unit 1222 are connected; the second feature extraction module 123 includes a downsampling processing unit 1223, a visual state space unit 1221 and an attention fusion unit 1222; the downsampling processing unit 1223 is connected to the visual state space unit 1221, and the visual state space unit 1221 is connected to the attention fusion unit 1222; the downsampling processing unit 1223 is used to: downsample the feature map; the visual state space unit 1221 is used to: cross-scan the image block or feature map to obtain feature information, map the feature information based on the hidden state variable to obtain mapping information corresponding to the feature information, and cross-merge the mapping information to obtain the feature map.

[0086] The formula for mapping feature information based on hidden state variables is:

[0087]

[0088] y t =Ch t-1 +Dx t ;

[0089] Among them, h t is the hidden state variable of variable t; x t is the characteristic information of variable t; y t is the mapping information corresponding to the feature information of variable t; A∈R N×N ,B∈R N×1 ,C∈R 1×N and D∈R 1 is the weight matrix.

[0090] In one example, see Figure 5 , Figure 5 This is a structural diagram of a visual state space unit shown in an embodiment of the present invention. The visual state space unit (hereinafter referred to as VSS unit) includes a normalization layer, a 2D selective scanning module, a normalization layer and a feedforward neural network, wherein the normalization layer normalizes the input of each layer to make the output distribution of each layer more stable, thereby alleviating the problem of gradient disappearance or gradient explosion. The feedforward neural network is composed of a fully connected layer and a nonlinear activation function. Its main function is to transform and enhance the input features to extract higher-level feature representations. For the input data, residual connections are used to facilitate direct transmission of information and reduce the risk of gradient disappearance. For the 2D selective scanning module, it should be noted that in the fine-grained detection and recognition tasks of rotating targets, the targets usually have complex texture features, and the rotation angles are variable and the target size is small, which poses a great challenge to traditional visual feature extraction methods. In order to solve the above problems, the present invention introduces a state space model (SSM). SSM is derived from the Kalman filter and can be regarded as a linear time invariant (LTI) system. It uses hidden state variables h(t)∈R N Map the input signal u(t)∈R to the output response y(t)∈R. Specifically, the continuous-time SSM can be represented by the following linear ordinary differential equation (ODE):

[0091] h′(t)=Ah(t)+Bu(t)

[0092] y(t)=Ch(t)+Du(t)

[0093] In order to integrate the continuous-time SSM into the deep learning model, it must be discretized first. Specifically, the time interval [t a ,t b ], the hidden state variable h(t) at t=t b The analytical solution at can be expressed as:

[0094]

[0095] By sampling with a time scale parameter Δ, that is h(t b ) can be written as:

[0096]

[0097] Among them, [a, b] represents the corresponding discrete time step interval. It is worth noting that this formula is close to the result of the Zero Order Hold (ZOH) method. Although the discretization operation enables the SSM model to be applied to deep learning models of non-continuous data, its application in visual data faces great challenges. Unlike natural language data that contains time series information, visual data is inherently non-sequential and contains rich spatial information such as local texture and global structure, which is particularly prominent in the task of rotating target detection. In order to solve this problem, the method provided in an embodiment of the present invention reconstructs the SSM through a convolution operation, expands the convolution kernel from 1D to 2D, and implements it in the form of an outer product. And the input processing is performed by the Selective Scan (SS) method, and a 2D selective scanning module is used. The hardware structure of the 2D selective scanning module is as follows. Figure 6 As shown, the 2D selective scanning module includes a cross-scanning unit 610, a discretization unit 620, and a cross-merging unit 630. This enables the SSM to adapt to the feature extraction requirements of visual data and rotated targets while retaining its efficiency and structural advantages. It can also be understood that by using the 2D selective scanning module, the method provided in this embodiment of the present invention configures a spatial state model for the feature extraction network.

[0098] The attention fusion unit 1222 is configured to: perform weighted processing on the feature map based on the channel attention weight to obtain a channel attention weighted feature map, and perform weighted processing on the channel attention weighted feature map based on the spatial attention weight to obtain a first feature map;

[0099] The formula for determining the channel attention weight is:

[0100]

[0101] Among them, M cF is the channel attention weight, σ() is the sigmoid activation function; W0 and W1 are the shared weights of the fully connected layer; is the feature map processed by average pooling, is the feature map processed by maximum pooling;

[0102] The formula for determining the spatial attention weight is:

[0103]

[0104] Among them, M s F is the spatial attention weight; f 7×7 It is a convolution layer with a convolution kernel size of 7×7; is the channel attention weighted feature map after average pooling processing, It is the channel attention weighted feature map after maximum pooling.

[0105] Specifically, the following uses an example to explain the specific implementation of the attention fusion unit 1222. The attention fusion unit 1222 includes two parts: a channel attention fusion unit and a spatial attention fusion unit.

[0106] In the channel attention fusion unit, the feature map is first compressed in the spatial dimension to obtain a one-dimensional vector for subsequent processing. During the spatial compression, the channel attention fusion unit not only uses average pooling but also introduces max pooling to more comprehensively aggregate the spatial information of the feature map. The results of average and max pooling are input into a shared network, which further compresses the spatial dimension and sums them element-by-element to produce the final channel attention map. For an image, channel attention primarily focuses on the important content of the image: average pooling provides feedback to every pixel in the feature map, while max pooling mainly provides gradient feedback to the regions with the strongest response in the feature map during backpropagation. The channel attention fusion unit effectively enhances the channel features of key information while suppressing secondary features, thereby improving the discriminative power of the features.

[0107] For the spatial attention fusion unit, the spatial attention module compresses the features F′ extracted by the channel attention module in the channel dimension, and performs average pooling and maximum pooling operations respectively to generate a spatial attention map. Specifically, maximum pooling extracts the maximum value of each spatial position on the channel, and extracts it h×w times in total; average pooling averages each spatial position, and the number of extractions is also h×w. Then, the above-mentioned extracted feature maps (the number of channels is 1) are merged to generate a feature map containing 2 channels. This method enables the model to focus more on the key information areas in the image, reduce the interference of background or irrelevant areas, and thus improve the overall detection accuracy.

[0108] S3. Input the multiple first feature maps into a feature pyramid network and output multiple second feature maps.

[0109] The multiple second feature maps have the same number of channels and different sizes.

[0110] In one example, the number of first feature maps is 4, and the number of second feature maps is 5.

[0111] S4. Input the multiple second feature maps into the rotating target detection network, and output the category of the target vehicle and the rectangular rotation box of the target vehicle on the optical image, where the rectangular rotation box is used to represent the position and rotation angle of the target vehicle.

[0112] In some embodiments, the rotated target detection network includes a candidate box generation module and a candidate box classification and regression module; the candidate box generation module is used to: determine multiple horizontal anchor boxes with different aspect ratios corresponding to each second feature map based on multiple second feature maps; determine a horizontal anchor box where a target vehicle exists from the multiple horizontal anchor boxes with different aspect ratios corresponding to each second feature map; determine multiple offset values ​​corresponding to each horizontal anchor box where a target vehicle exists; and determine a tilted candidate box corresponding to each horizontal anchor box where a target vehicle exists based on the multiple offset values ​​corresponding to each horizontal anchor box where a target vehicle exists.

[0113] Optionally, the candidate box classification regression module includes a first fully connected layer and a second fully connected layer connected in parallel; the first fully connected layer is used to determine the category of the target vehicle based on the tilted candidate box corresponding to the horizontal anchor box where the target vehicle exists; the second fully connected layer is used to: determine the rectangular rotation box corresponding to the tilted candidate box corresponding to each horizontal anchor box where the target vehicle exists; based on a preset step size, the rectangular rotation box corresponding to the tilted candidate box corresponding to each horizontal anchor box where the target vehicle exists is mapped to the second feature map to obtain the coordinate value corresponding to the target vehicle area included in the second feature map; the target vehicle area included in the second feature map is gridded to obtain a feature map of the target size, the grid value of the feature map of the target size at each position in each channel is determined, and the rectangular rotation box of the target vehicle on the optical image is generated according to the grid value of the feature map of the target size at each position in each channel;

[0114] Among them, the second feature map includes the coordinate values ​​(x r ,y r ,w r ,h r ,θ) is determined as follows:

[0115]

[0116] Where (x, y, w, h, θ) is the coordinate value corresponding to the rectangular rotation box; s is the preset step size;

[0117]

[0118] The grid value F of the target size feature map at the grid (i, j) of the cth channel c The formula for determining ′(i,j) is:

[0119]

[0120] F c is the feature map F of the target size in the cth channel, n is the number of samples in the grid, area(i,j) is the coordinate set in the grid (i,j), and R() is the rotation transformation.

[0121] To facilitate understanding of this solution, the specific implementation of the rotating target detection network provided by an embodiment of the present invention is explained below through an example.

[0122] For example, for the candidate box generation module, the traditional directed candidate box generation network (such as Rotated RPN and RoI Transformer) has a high computational cost and low efficiency when generating directed candidate boxes, especially when processing tilted targets, it shows an obvious performance bottleneck. In response to these problems, the present invention proposes a tilted target representation method based on the midpoint offset representation, and adopts a simplified design of the directed candidate box generation network-Oriented RPN. Oriented RPN is a lightweight fully convolutional network that increases the output parameters of the RPN regression branch from the traditional 4 to 6 to adapt to the midpoint offset representation of tilted targets. This structure significantly improves the efficiency and speed of candidate box generation while reducing the number of parameters. For input images of any size, Oriented RPN can efficiently generate a series of tilted candidate boxes through a lightweight fully convolutional network. The specific implementation of the candidate box generation module is as follows:

[0123] The five second feature maps {P2, P3, P4, P5, P6} output by the feature pyramid network are used as input, and an identical head design (3×3 convolution layer and two 1×1 parallel convolution layers) is added to each layer. At each spatial position of all layers of each second feature map, three horizontal anchor boxes with different aspect ratios {1:2, 1:1, 2:1} are assigned. The image areas corresponding to the horizontal anchor boxes on the {P2, P3, P4, P5, P6} feature layers are {32 2 ,64 2 ,128 2 ,256 2 ,512 2 Each horizontal anchor box is represented by a 4-dimensional vector a(a x ,a y ,a w ,a h ), where (a x ,a y ) is the center coordinate of the anchor, a w and a w Represents the width and height of the anchor respectively.

[0124] One of the two 1×1 parallel convolutional layers is a classification branch, which is used to score the current horizontal anchor frame and determine whether the horizontal anchor frame contains the target vehicle to be detected. The other is a regression branch, which outputs the offset of the candidate frame relative to the horizontal anchor frame δ = (δ x ,δ y ,δ w ,δ h ,δ α ,δ βThe candidate box generation module generates 3 candidate boxes at each position of each second feature map. Therefore, the regression branch at one position has 6×3 output values. For each candidate box, the tilted candidate box can be obtained by decoding the regression output. The decoding steps are as follows:

[0125]

[0126] Among them, (x, y) is the center coordinate of the predicted candidate box, w and h are the width and height of the bounding rectangle of the predicted candidate box, Δα and Δβ are the deviation values ​​relative to the midpoint of the upper and right boundaries of the bounding rectangle, respectively. Finally, the midpoint offset representation can be used to generate the four-vertex coordinate set v = (v1, v2, v3, v4) of each candidate box according to x, y, w, h, Δα, Δβ. Among them, Δα is v1 relative to the upper midpoint of the horizontal bounding box. The deviation value of v2 relative to the upper midpoint of the horizontal bounding box is Δβ. According to symmetry, -Δα and -Δβ are the deviation values ​​of v3 and v4 about the bottom edge and the midpoint of the coordinates, respectively. In summary, the coordinates of the four vertices of the tilted bounding box can be expressed as:

[0127]

[0128] Then, the candidate box generation module predicts the parameter values ​​(x, y, w, h) of the bounding rectangle to achieve the regression of each tilted candidate box and determine the midpoint deviation parameters (Δα, Δβ), see Figure 7 , a schematic diagram of a midpoint offset representation method for a rotating target provided by an embodiment of the present invention.

[0129] For example, the input to the candidate box classification and regression module is the five second feature maps {P2, P3, P4, P5, P6} and the tilted candidate box corresponding to each horizontal anchor box containing the target vehicle. For each tilted candidate box, the Rotated Roi Align algorithm is used to extract a fixed-size feature vector from the corresponding feature map. Each feature vector is input into the first and second fully connected layers. The first fully connected layer outputs K+1 category probabilities, including background. The second fully connected layer regresses the candidate boxes of the K target categories and outputs the offset.

[0130] Specifically, the Rotated RoI Align algorithm is used to extract rotation-invariant features from each tilted candidate box. Since the candidate box generated by the directed candidate box generation network is usually a parallelogram, it is represented by four vertices v = {v1, v2, v3, v4}. Then, it is adjusted to a rectangle by extending the short diagonal to the same length as the long diagonal. After that, the (x, y, w, h, θ) of the tilted rectangle can be obtained, where is the angle between the horizontal axis and the long side of the rectangle. Then, use the step size s to map the tilted rectangle (x, y, w, h, θ) to the feature map F and obtain (x r ,y r ,w r ,h r ,θ) represents the Rotated Roi.

[0131] In some embodiments, before the above S4, the method provided by the embodiment of the present invention further includes:

[0132] Acquire a training image, where the training image includes a plurality of training anchor frames and a plurality of real anchor frames;

[0133] Determine a label corresponding to each training anchor frame, where the label includes a positive sample, a negative sample, or an invalid sample; wherein the intersection-and-union ratio of the training anchor frame with the positive sample label and any real anchor frame is greater than or equal to a first threshold, or the intersection-and-union ratio of the training anchor frame with the target real anchor frame is greater than the intersection-and-union ratio of other training anchor frames and the target real anchor frame; the target real anchor frame is one of the multiple real anchor frames; the intersection-and-union ratio of the training anchor frame with the negative sample label and any real anchor frame is less than a second threshold; the intersection-and-union ratio of the training anchor frame with the invalid sample label and any real anchor frame is greater than or equal to the second threshold and less than the first threshold;

[0134] In one example, the first threshold is 0.7 and the second threshold is 0.3.

[0135] Construct loss function;

[0136] Based on the loss function, the rotation object detection network is iteratively trained according to the training anchor frames labeled as positive samples and negative samples to obtain the trained rotation object detection network;

[0137] The loss function L1 is:

[0138]

[0139] Where i is the index of the training anchor box; N is the total number of samples; F cls is the cross entropy loss for classification; F reg is the L1 smoothing loss for regression; represents the true label of the i-th training anchor box; p i is the foreground probability value output by the first fully connected layer; is the offset of the i-th training anchor box relative to the true anchor box under the midpoint offset representation.

[0140] It should be understood that is the offset of the i-th anchor box relative to the ground truth box under the midpoint offset representation, expressed as a parameterized 6-dimensional vector This vector comes from the candidate box regression branch, and the specific calculation formula is as follows:

[0141]

[0142] As can be seen from S1-S4 above, the method provided by the embodiment of the present invention extracts features from optical images through a feature extraction network configured with a spatial state model, which can obtain finer-grained visual information, and thus can effectively and accurately extract key features in optical images. It can be applied to detection scenarios of complex textures and rotating targets, thereby improving the detection accuracy and robustness of target vehicles. In addition, the method provided by the present invention improves the ability of the feature extraction network to focus on key information areas in optical images through an attention fusion unit, reducing interference from background noise and irrelevant areas. The attention fusion unit can effectively improve the discriminability of feature representation by fusing the channel attention mechanism and the spatial attention mechanism, further improving the detection accuracy of target vehicles, and meeting the usage requirements of rotating target positioning and boundary refinement scenarios. Finally, the method provided by the present invention realizes the generation and classification regression of candidate boxes respectively through a candidate box generation module and a candidate box classification regression module, which can effectively improve the accuracy. In general, compared with the target detection method in the related art, the method provided by the present invention can improve the feature extraction capability and thus improve the detection accuracy, meeting the usage requirements of users in different usage scenarios.

[0143] In order to facilitate understanding of the beneficial effects of the present solution, the beneficial effects of the method provided by the embodiment of the present invention are explained below based on comparative experiments. In one example, the detection system 100 first obtains 331 simulated images of vehicles of 22 categories, totaling 9505 instances. Among them, in order to improve the similarity between the simulated images and the real images, the present invention comprehensively simulates different scenes and weather conditions during the simulation process. Specifically, the generation of simulated images takes into account various environmental factors, including but not limited to changes in lighting, weather effects, background complexity, etc., so that the simulated data can more realistically reflect the characteristics of rotating targets under different conditions, thereby enhancing the robustness and generalization ability of the model in practical applications. The detection device then uses the average recall rate (mRecall) and the average precision (mAP, mean Average Precision) as evaluation indicators of the method provided by the embodiment of the present invention.

[0144] Specifically, the average recall rate is a key indicator for evaluating the model's missed detection in object detection tasks. The recall rate measures the proportion of true targets detected by the model to all actual targets. Its formula is:

[0145]

[0146] Among them, True Positives refer to objects that the model correctly detected, that is, the model predicted that they were objects, and these predictions exactly matched the ground truth. False Negatives refer to objects that the model failed to detect, that is, the model missed an object. In other words, the object existed but the model did not make a correct prediction.

[0147] In multi-category detection tasks, mean recall (mRecall) is the average recall across all categories. mRecall reflects the model's object detection coverage, particularly in complex backgrounds, small objects, or rotated objects. A high mRecall indicates that the model is able to effectively detect objects across multiple categories, avoiding missed detections.

[0148] Average precision is one of the most commonly used evaluation metrics in the field of object detection, used to comprehensively assess the performance of models in multi-category tasks. For each category, precision and recall are calculated at different detection thresholds. Precision measures the proportion of boxes identified as objects by the model that are actually objects. Its calculation formula is:

[0149]

[0150] False Positives refer to locations where the model incorrectly predicts an object, but there is no object at that location. In simple terms, the model incorrectly generates a detection box in the background or non-target area.

[0151] For each category, we first calculate the corresponding precision values ​​at different recall rates. Based on these values, we calculate the area under the precision-recall curve (AP). AP is a comprehensive measure of the balance between precision and recall, taking into account the different types of errors that can occur in object detection (such as missed detections and false detections).

[0152] Based on this, mAP is the average of all AP values ​​across all categories, representing the model's detection accuracy across all categories. A higher mAP value indicates a stronger model's ability to identify objects of all categories throughout the entire detection task. For fine-grained rotated object detection, mAP effectively reflects the model's overall performance in handling various rotation angles, complex backgrounds, and varying object sizes.

[0153] This invention experimentally compares the accuracy of optically fine-grained rotating object detection with five rotating object detection schemes in related art: Technique 1, Technique 2, Technique 3, Technique 4, and Technique 5. Technique 1 transforms the problem of predicting the angle of a rotating frame into a probability distribution prediction problem. A Gaussian distribution is used to represent the angular distribution of the predicted and true frames, and the model is optimized by minimizing the KL divergence (Kullback-Leibler Divergence) between the two, thereby avoiding the problem of angular periodicity. Technique 2 is a pixel-by-pixel prediction-based target detection method for horizontal frame detection, supports rotated target detection, and uses angle encoding to avoid angular periodicity. Technique 3 represents rotated rectangular frames using vertices. Each rotated rectangle of the target frame is represented as the position coordinates of four vertices. Technique 4 is an improved candidate region generation method that can generate rotated candidate regions. Based on the conventional RPN, a rotation transformer module is added to convert horizontal candidate regions into rotated candidate regions, making the generated candidate regions more consistent with the rotated target and improving detection accuracy. Technique 5 is a rotation-equivariant network for rotated object detection. It uses rotational convolution in feature extraction. Through this convolution operation, the convolution kernel can adapt to the rotation of the object in the image, allowing the network to still extract valid features under rotated objects. Specific experimental results are shown in Tables 1, 2, and 3. Table 1 compares the optical fine-grained rotation target detection accuracy of different rotation target detection networks. Table 2 compares the recall data of different rotation target detection networks for each experiment and category. Table 3 compares the AP data of different rotation target detection networks for each experiment and category.

[0154] Table 1

[0155] technology mRecall mAP Technology 1 0.424 0.281 Technology 2 0.468 0.282 Technique 3 0.413 0.368 Technology 4 0.557 0.448 Technique 5 0.590 0.540 The present invention 0.712 0.649

[0156] Table 2

[0157]

[0158] Table 3

[0159]

[0160] As can be seen from Tables 1 to 3 above, the method provided by the embodiment of the present invention is significantly superior to the other five rotating target detection schemes in the related art in terms of accuracy indicators of optical fine-grained rotation simulation data sets compared to the five methods in the related art. This shows that the feature extraction network equipped with a spatial state model can process images more efficiently in the feature extraction stage and fully exploit the key features in the image. At the same time, the two-stage rotating target detection network can accurately generate a rotating detection box (rectangular rotating box) based on the extracted high-quality features, thereby improving the ability to accurately recognize fine-grained vehicle targets, especially in complex backgrounds and with a lot of details.

[0161] Furthermore, in order to verify the role of each step and module in the technical solution of the present invention, a set of ablation experiments was designed. By replacing the feature extraction network, adding an attention fusion module, etc., the influence of each step on the accuracy of optical fine-grained rotation target detection was analyzed. Tables 4, 5 and 6 show the overall and category accuracy indicators on the optical fine-grained rotation simulation dataset when different feature extraction networks (ResNet, Swin Transformer, VMamba and the feature extraction network configured with a spatial state model provided by the present invention) are used while keeping the rotation target detection network unchanged. Among them, Table 4 is the overall results of the optical fine-grained rotation target detection accuracy ablation experiment, Table 5 is the Recall of each category of the optical fine-grained rotation target detection accuracy experiment with different feature extraction networks, and Table 6 is the AP of each category of the optical fine-grained rotation target detection accuracy experiment with different feature extraction networks.

[0162] Table 4

[0163] Feature extraction network mRecall mAP ResNet 0.622 0.543 Swin Transformer 0.696 0.599 VMamba 0.661 0.607 The present invention 0.712 0.649

[0164] Table 5

[0165]

[0166]

[0167] Table 6

[0168]

[0169] As shown in Tables 4-6, the feature extraction network equipped with a spatial state model, as provided in embodiments of the present invention, extracts richer and more refined features compared to the ResNet and Swin Transformer commonly used in existing rotated object detection methods, thereby demonstrating superior performance in rotated object detection networks. This demonstrates that the feature extraction network equipped with a spatial state model can more effectively capture key visual information when processing fine-grained rotated objects, improving detection accuracy and robustness.

[0170] The above mainly introduces the solution of the embodiment of the present invention from the perspective of method. It can be understood that in order to realize the above functions, the detection system 100 includes at least one of the hardware structure and software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiment of the present invention.

[0171] In an embodiment of the present invention, the detection system 100 can be divided into functional units according to the above-described method example. For example, the detection system 100 can be divided into functional units corresponding to various functions, or two or more functions can be integrated into a single processing unit. The above-mentioned integrated units can be implemented in the form of hardware or software functional units. It should be noted that the division of units in the embodiment of the present invention is schematic and is merely a logical functional division. In actual implementation, other division methods may be used.

[0172] For example, Figure 8 A hardware structure diagram of a detection system provided by an embodiment of the present invention is shown. The detection system 100 includes: an acquisition module 110 for acquiring an optical image, the optical image including a target vehicle to be detected; a feature extraction network 120 for outputting, based on the input optical image, multiple first feature maps corresponding to the optical image, each having a different number of channels and sizes; a spatial state model configured in the feature extraction network 120; a feature pyramid network 130 for outputting, based on the input multiple first feature maps, multiple second feature maps, each having the same number of channels and different sizes; and a rotation target detection network 140 for outputting, based on the input multiple second feature maps, the target vehicle category and a rectangular rotation box representing the target vehicle in the optical image, the rectangular rotation box representing the target vehicle's position and rotation angle.

[0173] It should be understood that the specific description of the above optional methods can refer to the above method embodiments, which will not be repeated here. In addition, the explanation of any detection system 100 provided above and the description of the beneficial effects can refer to the above corresponding method embodiments, which will not be repeated here.

[0174] An embodiment of the present invention further provides a computer-readable storage medium storing at least one computer instruction, which is loaded and executed by a processor to implement the methods of each of the above embodiments. For explanations of the relevant contents and descriptions of the beneficial effects of any of the above-mentioned computer-readable storage media, reference can be made to the corresponding embodiments described above and will not be repeated here.

[0175] The embodiment of the present invention further provides a chip. The chip integrates a control circuit and one or more ports for implementing the functions of the above-mentioned detection system 100. Optionally, the functions supported by the chip can be referred to above and will not be repeated here.

[0176] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a random access memory, etc. The above-mentioned processing unit or processor can be a central processing unit, a general-purpose processor, a specific circuit structure (application specific integrated circuit, ASIC), a microprocessor (digital signal processor, DSP), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0177] An embodiment of the present invention further provides a computer program product comprising instructions that, when executed on a computer, cause the computer to perform any of the methods described in the above embodiments. The computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present invention are fully or partially generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium accessible by a computer or a data storage device such as a server or data center that includes one or more available media. Available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0178] It should be noted that the above-mentioned devices for storing computer instructions or computer programs provided in the embodiments of the present invention, such as but not limited to the above-mentioned memories, computer-readable storage media and communication chips, etc., all have non-transitory properties. Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention can be implemented using hardware, software, firmware or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein the communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0179] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for detecting and identifying a rotating frame of a target vehicle, characterized in that: The method comprises: Acquiring an optical image, wherein the optical image includes a target vehicle to be detected; Inputting the optical image into a feature extraction network configured with a spatial state model, and outputting a plurality of first feature maps corresponding to the optical image, wherein the plurality of first feature maps respectively have different numbers of channels and sizes; Inputting the plurality of first feature maps into a feature pyramid network, and outputting a plurality of second feature maps, wherein the plurality of second feature maps respectively have the same number of channels and different sizes; Inputting the plurality of second feature maps into a rotating target detection network, outputting a category of the target vehicle and a rectangular rotation box of the target vehicle on the optical image, wherein the rectangular rotation box is used to represent the position and rotation angle of the target vehicle; The feature extraction network includes an image processing module, a first feature extraction module and a plurality of second feature extraction modules connected in series; The first feature extraction module includes a visual state space unit and an attention fusion unit, and the visual state space unit and the attention fusion unit are connected; the second feature extraction module includes a downsampling processing unit, a visual state space unit and an attention fusion unit; the downsampling processing unit is connected to the visual state space unit, and the visual state space unit is connected to the attention fusion unit; the downsampling processing unit is used to: downsample the feature map; the visual state space unit is used to: cross-scan the image block or the feature map to obtain feature information, map the feature information based on the hidden state variable to obtain mapping information corresponding to the feature information, and cross-merge the mapping information to obtain the feature map; The formula for mapping feature information based on hidden state variables is: y t =Ch t-1 +Dx t ; Among them, h t is the hidden state variable of variable t; x t is the characteristic information of variable t; y t is the mapping information corresponding to the feature information of variable t; A∈R N×N ,B∈R N×1 ,C∈R 1×N and D∈R 1 is the weight matrix; The attention fusion unit is used to: perform weighted processing on the feature map based on the channel attention weight to obtain a channel attention weighted feature map, and perform weighted processing on the channel attention weighted feature map based on the spatial attention weight to obtain a first feature map; The formula for determining the channel attention weight is: Among them, M c F is the channel attention weight, σ() is the sigmoid activation function; W0 and W1 are the shared weights of the fully connected layer; is the feature map processed by average pooling, is the feature map processed by maximum pooling; The formula for determining the spatial attention weight is: Among them, M s F is the spatial attention weight; f 7×7 It is a convolution layer with a convolution kernel size of 7×7; is the channel attention weighted feature map after average pooling processing, It is the channel attention weighted feature map after maximum pooling.

2. The method according to claim 1, characterized in that The rotation target detection network includes a candidate box generation module and a candidate box classification regression module; The candidate frame generation module is used to: determine, according to the multiple second feature maps, multiple horizontal anchor frames with different aspect ratios corresponding to each second feature map; Determine a horizontal anchor frame where the target vehicle exists from a plurality of horizontal anchor frames with different aspect ratios corresponding to each second feature map; Determine multiple offset values ​​corresponding to each horizontal anchor frame where the target vehicle exists; and determine a tilted candidate frame corresponding to each horizontal anchor frame where the target vehicle exists according to the multiple offset values ​​corresponding to each horizontal anchor frame where the target vehicle exists.

3. The method according to claim 2, characterized in that The candidate frame classification and regression module includes a first fully connected layer and a second fully connected layer connected in parallel; The first fully connected layer is used to determine the category of the target vehicle based on the inclined candidate frame corresponding to the horizontal anchor frame where the target vehicle exists; The second fully connected layer is used to: determine a rectangular rotation box corresponding to a tilted candidate box corresponding to each horizontal anchor box containing a target vehicle; map the rectangular rotation box corresponding to the tilted candidate box corresponding to each horizontal anchor box containing a target vehicle to a second feature map based on a preset step size, and obtain coordinate values ​​corresponding to a target vehicle region included in the second feature map; grid the target vehicle region included in the second feature map to obtain a feature map of a target size, determine a grid value of the feature map of the target size at each position in each channel, and generate a rectangular rotation box of the target vehicle on the optical image according to the grid value of the feature map of the target size at each position in each channel; Among them, the second feature map includes the coordinate values ​​(x r ,y r ,w r ,h r ,θ) is determined as follows: Where (x, y, w, h, θ) is the coordinate value corresponding to the rectangular rotation box; s is the preset step size; The grid value F' of the feature map of the target size at the grid (i, j) of the c-th channel c The formula for determining (i,j) is: F c is the feature map F of the target size in the cth channel, n is the number of samples in the grid, area(i,j) is the coordinate set in the grid (i,j), and R() is the rotation transformation.

4. The method according to claim 3, characterized in that Before inputting the plurality of second feature maps into a rotating target detection network and outputting the category of the target vehicle and a rectangular rotation box of the target vehicle on the optical image, the method further includes: Acquire a training image, where the training image includes a plurality of training anchor frames and a plurality of real anchor frames; Determine a label corresponding to each training anchor frame, where the label includes a positive sample, a negative sample, or an invalid sample; wherein the intersection-and-union ratio of the training anchor frame with the positive sample label and any real anchor frame is greater than or equal to a first threshold, or the intersection-and-union ratio of the training anchor frame with the target real anchor frame is greater than the intersection-and-union ratio of other training anchor frames and the target real anchor frame; the target real anchor frame is one of the multiple real anchor frames; the intersection-and-union ratio of the training anchor frame with the negative sample label and any real anchor frame is less than a second threshold; the intersection-and-union ratio of the training anchor frame with the invalid sample label and any real anchor frame is greater than or equal to the second threshold and less than the first threshold; Construct loss function; Based on the loss function, the rotation object detection network is iteratively trained according to the training anchor frames labeled as positive samples and negative samples to obtain the trained rotation object detection network; The loss function L1 is: Where i is the index of the training anchor box; N is the total number of samples; F cls is the cross entropy loss for classification; F reg is the L1 smoothing loss for regression; represents the true label of the i-th training anchor box; p i is the foreground probability value output by the first fully connected layer; is the offset of the i-th training anchor box relative to the true anchor box under the midpoint offset representation.

5. A target vehicle rotating frame detection and recognition system, characterized in that: The system comprises: an acquisition module, configured to acquire an optical image, wherein the optical image includes a target vehicle to be detected; a feature extraction network, configured to output a plurality of first feature maps corresponding to the input optical image, wherein the plurality of first feature maps respectively have different numbers of channels and sizes; the feature extraction network is configured with a spatial state model; A feature pyramid network is configured to output a plurality of second feature maps based on the plurality of first feature maps input, wherein the plurality of second feature maps respectively have the same number of channels and different sizes; a rotating target detection network, configured to output a category of the target vehicle and a rectangular rotation box of the target vehicle on the optical image based on the input multiple second feature maps, wherein the rectangular rotation box is used to represent the position and rotation angle of the target vehicle; The feature extraction network includes an image processing module, a first feature extraction module and a plurality of second feature extraction modules connected in series; The first feature extraction module includes a visual state space unit and an attention fusion unit, and the visual state space unit and the attention fusion unit are connected; the second feature extraction module includes a downsampling processing unit, a visual state space unit and an attention fusion unit; the downsampling processing unit is connected to the visual state space unit, and the visual state space unit is connected to the attention fusion unit; the downsampling processing unit is used to: downsample the feature map; the visual state space unit is used to: cross-scan the image block or the feature map to obtain feature information, map the feature information based on the hidden state variable to obtain mapping information corresponding to the feature information, and cross-merge the mapping information to obtain the feature map; The formula for mapping feature information based on hidden state variables is: y t =Ch t-1 +Dx t ; Among them, h t is the hidden state variable of variable t; x t is the characteristic information of variable t; y t is the mapping information corresponding to the feature information of variable t; A∈R N×N ,B∈R N×1 ,C∈R 1×N and D∈R 1 is the weight matrix; The attention fusion unit is used to: perform weighted processing on the feature map based on the channel attention weight to obtain a channel attention weighted feature map, and perform weighted processing on the channel attention weighted feature map based on the spatial attention weight to obtain a first feature map; The formula for determining the channel attention weight is: Among them, M c F is the channel attention weight, σ() is the sigmoid activation function; W0 and W1 are the shared weights of the fully connected layer; is the feature map processed by average pooling, is the feature map processed by maximum pooling; The formula for determining the spatial attention weight is: Among them, M s F is the spatial attention weight; f 7×7 It is a convolution layer with a convolution kernel size of 7×7; is the channel attention weighted feature map after average pooling processing, It is the channel attention weighted feature map after maximum pooling.

6. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the target vehicle rotation frame detection and recognition method according to any one of claims 1 to 4.

Images