Image classification and recognition method, recognition device and storage medium

Abstract

The present application discloses an image classification and recognition method, a recognition device, and a storage medium. The image classification and recognition method comprises: extracting features of an image to be recognized to obtain a first feature map; performing a downsampling operation on the first feature map to obtain a second feature map; rotating the second feature map so that the boundary of the feature region matches the boundary of the target detection region to obtain a third feature map; and obtaining a classification and recognition result of the image to be recognized based on the third feature map. In the above manner, the present application can improve the efficiency and accuracy of image classification and recognition.

Description

Technical Field

[0001] The present application relates to the field of computer image classification and recognition, and in particular to an image classification and recognition method, a recognition device and a storage medium. Background Art

[0002] With the improvement of automation, computer vision can assist in monitoring in many situations, including crowd monitoring, vehicle monitoring, disaster monitoring, security monitoring, etc.

[0003] In one application scenario, the port scale and cargo throughput are huge, and there are a large number of transport ships coming and going in the offshore waters. In the supervision of ships, the traditional coastal monitoring system facilities are relatively old, set up in a specific location with a fixed viewing angle, not flexible enough, and can only monitor the coastal situation from a relatively horizontal perspective. When the ships block each other, the recognition accuracy is very low.

[0004] Compared with the shortcomings of traditional port monitoring equipment, aerial images can obtain a wider field of view, and there is no problem of mutual occlusion between ships. At the same time, the orientation distribution of ships can be seen, which provides good support for better coastal management. However, for ships in aerial images, there will be a problem that the feature map in the detection task does not fit the actual target range. Summary of the invention

[0005] The main technical problem solved by the present application is to provide an image classification and recognition method, a recognition device and a storage medium, which can improve the efficiency and accuracy of image classification and recognition.

[0006] To solve the above technical problems, a technical solution adopted in the present application is: to provide an image recognition method, the method comprising: performing feature extraction on the image to be recognized to obtain a first feature map; performing a downsampling operation on the first feature map to obtain a second feature map; rotating the second feature map so that the boundary of the feature area matches the boundary of the target detection area to obtain a third feature map; and obtaining a classification recognition result of the image to be recognized based on the third feature map.

[0007] The second feature map is rotated so that the boundary of the feature area matches the boundary of the target detection area to obtain the third feature map, including: determining the target feature area in the second feature map; rotating the target feature area based on multiple set rotation angles to obtain multiple corresponding candidate feature areas; determining a rotated feature area that matches the boundary of the target detection area among the multiple candidate feature areas; and determining the third feature map based on multiple rotated feature areas corresponding to the multiple target feature areas in the second feature map.

[0008] Among them, based on the set multiple rotation angles, the target feature area is rotated respectively to obtain the corresponding multiple candidate feature areas respectively, including: determining multiple angle channels, each angle channel corresponds to an angle interval; based on the multiple angle channels, the target feature area is rotated respectively to obtain the corresponding multiple candidate feature areas respectively; for each candidate feature area, the boundary of the corresponding angle channel is used to perform pixel interpolation processing on the candidate feature area.

[0009] Among them, the following formula is used to process the target feature area:

[0010] F On =Int(SA(Y(p)·R T (θ),C n ),θ),n=0,1,…,N-1;

[0011] Among them, C n Indicates the nth angle channel, F On Indicates that in angle channel C n The candidate feature area under, Int represents the interpolation function, SA represents the angle channel switching function, Y(p) represents the feature function corresponding to the second feature map, R(θ)=(cosθ,-sinθ;sinθ,cosθ) T It is a rotation matrix used to control the switching of the rotation angle. N is a positive integer.

[0012] Among them, the following formula is used to downsample the target detection area:

[0013]

[0014] Among them, F is the feature function corresponding to the first feature map, W(k) represents the weight of each position on the convolution kernel used, K represents the position range of the convolution calculation, and the value of k controls the traversal process of the pixel points.

[0015] Among them, among multiple candidate feature regions, determining a rotational feature region that matches the boundary of the target detection region includes: constructing a score map corresponding to multiple angle channels for each candidate feature region; determining a response value corresponding to each score map; and determining a rotational feature region that matches the boundary of the target detection region based on the score map and response value of each candidate feature region.

[0016] The score map corresponding to multiple angle channels of each candidate feature region is constructed, including: dividing each candidate feature region into d 2 sub-block areas; where d is a positive integer; perform pooling operations on each sub-block area respectively; 2 The results of the pooling operation of the sub-block areas are concatenated to form the score map of the corresponding angle channel of the candidate feature area.

[0017] Among them, the following formula is used to perform pooling operation on the sub-block area:

[0018]

[0019] in, Indicates that in angle channel C n The (i,j)th sub-block area B i,j The pooled output in Represents N×k 2 One of the score maps, w is the learning parameter, p is the sub-block area B i, The number of pixels in, (u,v) is the feature point P i, The global coordinates of .

[0020] Among them, the feature point P i,j The global coordinates (u, v) of can be defined by the following formula:

[0021]

[0022] Where (u′, v′) represents the feature point P i,j The local coordinates, (u 0 ,v 0 ) represents the coordinates of the upper left corner of the aligned feature area, when (u,v)∈B i,j When , its range is defined by the following formula:

[0023]

[0024]

[0025] Where w and h are the length and width of the regular convolution region respectively.

[0026] Among them, the following formula is used to concatenate the results of the sub-block area pooling operation to form a score map of the candidate feature area:

[0027]

[0028] The response value of the angle channel is calculated using the following formula:

[0029]

[0030] Wherein, according to the score map and the response value of each candidate feature region, determining the rotation feature region that matches the boundary of the target detection region includes: obtaining the third feature map using the following formula:

[0031]

[0032] where FA is the selected rotation feature region that best matches the boundary of the target detection region, F OM To obtain the angle channel C with the maximum product n The regional characteristics of the .

[0033] To solve the above technical problems, another technical solution adopted in the present application is: to provide an image classification and recognition device, which includes a processor and a memory, the memory stores program data, and the processor is used to execute the program data to implement the image classification and recognition method as described above.

[0034] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a computer-readable storage medium, which stores program data. When the program data is executed, it is used to implement the above image classification and recognition method.

[0035] The beneficial effects of the present application are as follows: Different from the prior art, the present application provides an image classification and recognition method applied to an image classification and recognition device, wherein the image classification and recognition device extracts features from the image to be recognized to obtain a first feature map; performs a downsampling operation based on the first feature map to obtain a second feature map; rotates the second feature map to match the boundary of the feature area with the boundary of the target detection area to obtain a third feature map; and further utilizes the third feature map to obtain a classification and recognition result of the image to be recognized. In the above manner, compared with the processing area of ​​a conventional convolution operation, the present application uses an angle channel switching method to allow the original convolution processing area to fit the target detection area through angle rotation, and constructs a feature score map to obtain the feature area that best matches the target area, thereby solving the problem of severe misalignment between the feature area and the target detection area obtained by conventional convolution, so as to more efficiently classify and recognize images. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0037] Figure 1 It is a flowchart of an embodiment of a method for image classification and recognition provided by the present application;

[0038] Figure 2 This is a schematic diagram of the misaligned rotation features in conventional convolution (left) and the aligned rotation features (right);

[0039] Figure 3This is a flowchart of step 13 of an embodiment of the method for image classification and recognition provided by the present application;

[0040] Figure 4 This is a flowchart of an embodiment of step 132 of the method for image classification and recognition provided by the present application;

[0041] Figure 5 It is a schematic diagram of a rotation feature alignment module based on angle channel switching in the image classification and recognition method provided by the present application;

[0042] Figure 6 is a flowchart of step 133 of an embodiment of the method for image classification and recognition provided by the present application;

[0043] Figure 7 This is a flowchart of an embodiment of step 1331 of the method for image classification and recognition provided by the present application;

[0044] Figure 8 It is a schematic diagram of the process of constructing a feature score map in the method for image classification and recognition provided by this application;

[0045] Fig. 9 A schematic diagram of the structure of the image classification and recognition model provided for this application;

[0046] Fig.10 It is a structural schematic diagram of an embodiment of an image classification and recognition device provided by the present application;

[0047] Fig.11 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a method for image classification and recognition provided by the present application, the method comprising:

[0050] Step 11: Extract features of the image to be identified to obtain a first feature map.

[0051] Wherein, the image to be identified includes a target detection area.

[0052] Optionally, the image to be identified is an image captured by a camera. In an embodiment of port vessel monitoring, the image to be identified may be an image captured by a camera on a drone.

[0053] Specifically, the image to be identified undergoes target detection, and the target detection result includes a corresponding target detection area (or target detection frame), which is generally a rectangle, and the rectangle is generally the circumscribed rectangle of the target to be detected, to roughly indicate the position, direction, size, etc. of the target to be detected. Taking ships as an example, the berthing positions of ships in ports vary, so the rectangles of the target detection areas formed also have different shapes. In this embodiment, the angle between the long side of the rectangle and the x-axis of the image can be defined as the rotation angle of the target detection area.

[0054] Specifically, feature extraction is to use a computer to extract image information, such as brightness, edge, texture, color and other data, to achieve image classification. In this embodiment, the image features to be identified are mainly extracted to obtain the target area to be detected. Taking the aerial photo of a ship as an example, specific data in the aerial photo is extracted to classify the objects in the image and obtain the area where the ship is located.

[0055] Step 12: Downsample the first feature map to obtain a second feature map.

[0056] Wherein, the second feature map includes a feature area.

[0057] Specifically, downsampling refers to the process of reducing an image. Downsampling can make the image fit the size of the display area and generate a thumbnail of the corresponding image.

[0058] In one embodiment, a 3x3 convolution kernel is used:

[0059] K={(-1,-1),(0,-1),...,(0,1),(1,1)}

[0060] Take the input feature map as an example to downsample it and output the reduced feature map.

[0061] Optionally, in one embodiment, the target detection area is downsampled using the following formula:

[0062]

[0063] Among them, F is the feature function corresponding to the first feature map, W(k) represents the weight of each position on the convolution kernel used, K represents the position range of the convolution calculation, and the value of k controls the traversal process of the pixel points.

[0064] Step 13: Rotate the second feature map so that the boundary of the feature area matches the boundary of the target detection area to obtain a third feature map.

[0065] Among them, the third feature map is a feature area that best matches the boundary of the target detection area.

[0066] Specifically, the traditional convolution kernel is square in shape and only performs horizontal and vertical translation during operation, which usually cannot perfectly match the boundary of the target detection area. Figure 2 As shown in the figure, the a box in the left figure is the convolution feature area extracted according to the feature extraction range, and the b box is the actual area of ​​the target object. It can be seen that there is an obvious feature misalignment problem between the two. Therefore, it is necessary to rotate the area after the convolution of the feature area to achieve alignment between the feature area and the actual area of ​​the target. The effect is as follows Figure 2 As shown in the middle right picture.

[0067] Optionally, in one embodiment, if Figure 3 As shown, Figure 3 1 is a flow chart of an embodiment of step 13, and step 13 may specifically include:

[0068] Step 131: Determine a target feature region in the second feature map.

[0069] The target feature region is a feature map that is downsampled to the size of the target detection region. For example, if the size of the second feature map is 3*3, then the target feature region can be any one of the 9 rectangular (or square) regions. Furthermore, each feature region in the second feature map can be determined in turn by traversal, and subsequent steps can be performed based on each feature region in turn.

[0070] Step 132: Based on the set rotation angles, the target feature region is rotated respectively to obtain corresponding candidate feature regions.

[0071] Specifically, the rotation is performed by rotating the original feature frame according to the result of the angle channel division by the angle classification method, wherein the rotation angle value is cyclically switched in the predetermined angle channel.

[0072] It can be understood that the number of the set multiple rotation angles (angle channels) determines the matching degree between the boundary of the subsequently determined rotated feature area and the boundary of the target detection area. Therefore, as many angle channels as possible can be set.

[0073] Optionally, in one embodiment, if Figure 4 As shown, Figure 4 1 is a flow chart of an embodiment of step 132, and step 132 may specifically include:

[0074] Step 1321: Determine a plurality of angle channels, each angle channel corresponding to an angle interval.

[0075] Optionally, in one embodiment, 1° to 90° may be divided into 10 angle channels, which may include, in order of angle size: 1° to 10°, 11° to 20°, 21° to 30°, ... 81° to 90°.

[0076] It can be understood that the center point of rotation can be determined as the rectangular center of the feature area, that is, the intersection of the two diagonals of the rectangle. The feature area rotates around this center, and each angle channel corresponds to a 10° sector area, which has a boundary.

[0077] Of course, the above is only an example, and in other embodiments, multiple angle channels based on 360° may also be determined.

[0078] Step 1322: Based on multiple angle channels, the target feature region is rotated respectively to obtain corresponding multiple candidate feature regions respectively.

[0079] Step 1323: For each candidate feature region, use the boundary of the corresponding angle channel to perform pixel interpolation processing on the candidate feature region.

[0080] Specifically, interpolation calculation is to interpolate continuous functions based on discrete data so that this continuous curve passes through all given discrete data points, which is used to fill the gaps between pixels when the image is transformed. Since the angle channel is an interval and cannot be calculated accurately, the angle classification processing will cause errors in the calculation of the feature area, so it is necessary to interpolate the area to solve the pixel offset after the feature area is rotated.

[0081] Optionally, in one embodiment, the input target feature region is rotated using the following formula:

[0082] F On =Int(SA(Y(p)·R T (θ),c n ),θ),n=0,1,....,N-1;

[0083] Among them, c n Indicates the nth angle channel, F On Indicates that in the angle channel c n The candidate feature area under, Int represents the interpolation function, SA represents the angle channel switching function, Y(p) represents the feature function corresponding to the second feature map, R(θ)=(cosθ,-sinθ;sinθ,cosθ) T It is a rotation matrix used to control the switching of the rotation angle. N is a positive integer.

[0084] Specifically, Figure 5 As shown, for the input candidate feature region, according to the divided angle channels, the boundary of its angle interval is selected to perform pixel interpolation calculation on its region to realize the rotation of the feature region.

[0085] Step 133: Determine a rotation feature region that matches the boundary of the target detection region among multiple candidate feature regions.

[0086] The purpose of this step is to determine the final required feature region among multiple candidate feature regions.

[0087] Specifically, a plurality of candidate feature regions can be evaluated by a certain method. For example, the similarity between each candidate feature region and the original image to be detected can be calculated, or the overlap between the boundary of each candidate feature region and the target region of the original image to be detected can be calculated, etc. The following is a method for scoring multiple candidate feature regions:

[0088] Optionally, in one embodiment, if Figure 6 As shown, Figure 6 1 is a flow chart of an embodiment of step 133, and step 133 may specifically include:

[0089] Step 1331: construct a score map corresponding to multiple angle channels for each candidate feature region.

[0090] Optionally, in one embodiment, if Figure 7 As shown, Figure 7 1 is a flow chart of an embodiment of step 1331, and step 1331 may specifically include:

[0091] Step 13311: Divide each candidate feature region into d 2 sub-block area.

[0092] Step 13312: Perform pooling operation on each sub-block area separately.

[0093] Specifically, pooling refers to statistically summarizing the feature values ​​of an unknown and its adjacent positions in a plane, and using the summarized result as the value of this position in the plane.

[0094] Specifically, Figure 8 As shown, taking d=3 as an example, the feature area corresponding to each angle channel is divided into 3×3 sub-block areas, and the score of each sub-block area is calculated to construct a score map.

[0095] Optionally, in one embodiment, the following formula is used to perform a pooling operation on the sub-block area:

[0096]

[0097] in, Indicates that in the angle channel c n The (i,j)th sub-block area B i,j The pooled output in N×d 2 One of the score maps, w is the learning parameter, p is the sub-block area B i,j The number of pixels in, (u,v) is the feature point P i,j The global coordinates of .

[0098] Specifically, the feature point P i,j The global coordinates (u, v) of can be defined by the following formula:

[0099]

[0100] Where (u′, v′) represents the feature point P i,j The local coordinates, (u 0 ,v 0 ) represents the coordinates of the upper left corner of the aligned feature area, when (u,v)∈B i , j When , its range is defined by the following formula:

[0101]

[0102]

[0103] Where w and h are the length and width of the regular convolution region respectively.

[0104] Step 13313: D 2 The results of the pooling operation of the sub-block areas are concatenated to form the score map of the corresponding angle channel of the candidate feature area.

[0105] Optionally, in one embodiment, the following formula is used to concatenate the results of the sub-block region pooling operation to form a score map of the candidate feature region:

[0106]

[0107] Step 1332: Determine the response value corresponding to each score map.

[0108] Specifically, the response value refers to the degree of matching between each sub-block area and the boundary of the target detection area.

[0109] Optionally, in one embodiment, the response value corresponding to each score graph is determined using the following formula:

[0110]

[0111] Step 1333: Determine a rotation feature region that matches the boundary of the target detection region based on the score map and response value of each candidate feature region.

[0112] Step 134: Determine a third feature map based on the multiple rotation feature regions corresponding to the multiple target feature regions in the second feature map.

[0113] Optionally, in one embodiment, the third characteristic map is obtained by using the following formula:

[0114]

[0115] where F A is the selected rotation feature region that best matches the boundary of the target detection region, F OM To obtain the angle channel C with the maximum product n The regional characteristics of the .

[0116] Step 14: According to the third feature map, obtain the classification recognition result of the image to be recognized.

[0117] Among them, the angle and coordinates of the target object in the image can be predicted based on the third feature map, so as to classify the target object.

[0118] Specifically, the classification result may be the type of vessel, such as a cargo ship, an engineering ship, a fishing ship, a military ship, a civilian ship, etc.

[0119] Specifically, the loss function is a way to measure the gap between the predicted value and the actual value of the output of the neural network. Among them, the cross entropy loss function is used as the classification loss function to more accurately obtain the category information of the target object. Entropy is the uncertainty that indicates whether a random variable is a certain possible situation. The cross entropy loss function is used to evaluate the distance between two sample distributions.

[0120] Understandably, we use cross entropy to evaluate the difference between the target category probability distribution obtained by the current training and the true analog distribution, that is, we use the cross entropy loss function to characterize the distance between the actual output (probability) and the expected output (probability). That is, the smaller the cross entropy value, the closer the two probability distributions are.

[0121] Specifically, in the design of the anchor frame for target detection, an adaptive anchor frame calculation method is adopted. That is, during each training, the optimal anchor frame value in different data sets is adaptively calculated to improve the quality of the preset anchor frame.

[0122] Specifically, the bounding box loss function of the target uses CIoU_Loss, which can simultaneously consider the bounding box misalignment problem, the distance information of the bounding box center point, and the aspect ratio scale information of the bounding box.

[0123] Specifically, in the process of selecting the target box, the weighted NMS method is used to output the final predicted box information. Weighted NMS is to weight the confidence of the network prediction in the process of rectangular box elimination, obtain a new rectangular box, take the rectangular box as the final predicted rectangular box, and then eliminate those boxes.

[0124] Different from the prior art, the image classification and recognition method provided in this embodiment is applied to an image classification and recognition device, which extracts features from the image to be recognized to obtain a first feature map; performs a downsampling operation based on the first feature map to obtain a second feature map; rotates the second feature map to match the boundary of the feature area with the boundary of the target detection area to obtain a third feature map; and further uses the third feature map to obtain a classification and recognition result of the image to be recognized. In the above manner, compared with the processing area of ​​a conventional convolution operation, the angle channel switching method of the present application allows the original convolution processing area to fit the target detection area through angle rotation, and constructs a feature score map to obtain the feature area that best matches the target area, thereby solving the problem of severe misalignment between the feature area and the target detection area obtained by conventional convolution, so as to more efficiently classify and recognize images.

[0125] The method of the above embodiment can be implemented using a network model.

[0126] Combine the following Fig. 9 , Fig. 9 This is a structural diagram of the image classification and recognition model provided in the present application. The image classification and recognition model 90 may include a feature extraction module 91, a rotation feature alignment module 92, a full connection module 93 and a prediction result module 94.

[0127] Among them, the feature extraction module 91 is used to extract features from the image to be identified to obtain a first feature map, and the image to be identified includes a target detection area; further, the first feature map is downsampled to obtain a second feature map, and the second feature map includes a feature area; the rotation feature alignment module 92 is used to rotate the second feature map so that the boundary of the feature area matches the boundary of the target detection area to obtain a third feature map; the full connection module 93 is used to perform a full connection operation on the third feature map, and the prediction result module 94 obtains the classification recognition result of the image to be identified based on the third feature map.

[0128] Optionally, the rotation feature alignment module 92 may specifically include an angle channel switching module 921, an angle interpolation module 922 and a feature score map module 923, wherein the angle channel switching module 921 is used to set multiple angle channels and rotate the feature area based on the angle range of the angle channel; the angle interpolation module 922 is used to interpolate the feature area obtained by the rotation based on the boundary of the angle channel; the feature score map module 923 is used to score the candidate feature areas among multiple candidate feature areas by constructing a feature score map to determine the final required feature area.

[0129] Specifically, for the initial target image transmitted from the upper network, the feature area of ​​the target area is first determined, and then the offset of the feature area is calculated by angle classification, where the angle value is cyclically switched in the established angle channel. The angle classification process will cause errors in the calculation of the feature area, so the boundary of each angle interval is used to interpolate the corresponding rotated feature area. After obtaining the rotation alignment features corresponding to different angle channels, a position-sensitive score map is constructed for each channel alignment feature, where the score map is simplified by block processing, and the response value of the corresponding angle channel is calculated. For each angle channel, the cumulative score value and response value are calculated, and the product of the two is used as the basis for selecting the angle channel. The angle channel with the largest value is the channel where the feature area that best matches the target area is selected. The score map corresponding to the channel is fully connected and inserted into the existing target detection model to optimize the extraction of rotation features, which helps to reduce the error of angle classification processing and improve the classification effect.

[0130] It can be understood that the specific implementation steps and implementation principles of this embodiment can refer to the above Figure 1 The embodiments of the present invention will not be described in detail here.

[0131] See also Fig.10 , Fig.10 1 is a schematic diagram of the structure of an embodiment of an image classification and recognition device provided by the present application. The image classification and recognition device 100 includes a memory 101 and a processor 102. The memory 101 is used to store program data, and the processor 102 is used to execute the program data to implement the following method:

[0132] A first feature map is obtained by extracting features of the image to be identified; a downsampling operation is performed based on the first feature map to obtain a second feature map; the second feature map is rotated so that the boundary of the feature area matches the boundary of the target detection area to obtain a third feature map; and the third feature map is further used to obtain a classification recognition result of the image to be identified.

[0133] See also Fig.11 , Fig.111 is a schematic diagram of a structure of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 110 stores program data 111. When the program data 111 is executed by a processor, it is used to implement the following method:

[0134] A first feature map is obtained by extracting features of the image to be identified; a downsampling operation is performed based on the first feature map to obtain a second feature map; the second feature map is rotated so that the boundary of the feature area matches the boundary of the target detection area to obtain a third feature map; and the third feature map is further used to obtain a classification recognition result of the image to be identified.

[0135] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0136] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image classification and recognition method, characterized in that: The method comprises: Extract features from the image to be identified to obtain a first feature map, wherein the image to be identified includes a target detection area; Performing a downsampling operation on the first feature map to obtain a second feature map, where the second feature map includes a feature region; Determine a target feature region in the second feature map; Based on the set multiple rotation angles, the target feature regions are rotated respectively to obtain corresponding multiple candidate feature regions respectively; Divide each candidate feature region into d 2 sub-block areas; where d is a positive integer; Performing a pooling operation on each of the sub-block areas respectively; D 2 The results of the pooling operation on the sub-block regions are concatenated to form a score map of a corresponding angle channel of the candidate feature region; Determining a response value corresponding to each of the score graphs; Determining a rotation feature region that matches a boundary of the target detection region according to the score map and the response value of each candidate feature region; Determine a third feature map based on a plurality of rotation feature regions respectively corresponding to a plurality of target feature regions in the second feature map; A classification and recognition result of the image to be recognized is obtained according to the third feature map.

2. The method according to claim 1, characterized in that The target feature region is rotated based on the set multiple rotation angles to obtain the corresponding multiple candidate feature regions, including: Determine a plurality of angle channels, each angle channel corresponding to an angle interval; Based on the multiple angle channels, the target feature regions are rotated respectively to obtain corresponding multiple candidate feature regions respectively; For each of the candidate feature regions, pixel interpolation processing is performed on the candidate feature region using the boundary of the corresponding angle channel.

3. The method according to claim 2, characterized in that The target feature area is processed using the following formula: F On =Int(SA(Y(p)·R T (θ),C n ),θ),n=0,1,…,N-1; Among them, C n represents the nth angle channel, Indicates that in angle channel C n The candidate feature area under, Int represents the interpolation function, SA represents the angle channel switching function, Y(p) represents the feature function corresponding to the second feature map, R(θ)=(cosθ,-sinθ;sinθ,cosθ) T It is a rotation matrix used to control the switching of the rotation angle. N is a positive integer.

4. The method according to claim 3, characterized in that The target detection area is downsampled using the following formula: Among them, F is the feature function corresponding to the first feature map, W(k) represents the weight of each position on the convolution kernel used, K represents the position range of the convolution calculation, and the value of k controls the traversal process of the pixel points.

5. The method according to claim 1, characterized in that: The following formula is used to perform pooling operation on the sub-block area: in, Indicates that in angle channel C n The (i,j)th sub-block area B i,j The pooled output in , N×d 2 One of the score maps, w is the learning parameter, p is the sub-block area B i,j The number of pixels in, (u,v) is the feature point P i,j The global coordinates of The feature point P i,j The global coordinates (u, v) of can be defined by the following formula: Where (u′, v′) represents the feature point P i,j The local coordinates of (u0, v0) represent the coordinates of the upper left corner of the aligned feature area. When (u, v)∈B i,j When , its range is defined by the following formula: Where w and h are the length and width of the regular convolution region respectively.

6. The method according to claim 5, characterized in that The following formula is used to concatenate the results of the sub-block area pooling operation to form a score map of the candidate feature area: The response value of the angle channel is calculated using the following formula:

7. The method according to claim 1, characterized in that The step of determining a rotation feature region that matches a boundary of the target detection region according to the score map and the response value of each candidate feature region comprises: The third characteristic map is obtained by using the following formula: where F A is the selected rotation feature region that best matches the boundary of the target detection region, F OM To obtain the angle channel C with the maximum product n The regional characteristics of the .

8. An image classification and recognition device, characterized in that: The image classification and recognition device comprises a processor and a memory, wherein the memory stores program data, and the processor is used to execute the program data to implement the image classification and recognition method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program data, and when the program data is executed, it is used to implement the image classification and recognition method according to any one of claims 1 to 7.

Images