Multi-vehicle multi-information efficient identification method and device
By constructing a multi-vehicle, multi-information recognition backbone network, acquiring multi-scale features, and fusing and clustering them to generate candidate boxes, the problem of the inability to recognize minute vehicle features in existing technologies is solved, achieving a higher recognition accuracy.
Patent Information
- Application Number
- CN202310681776.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-08
- Publication Date
- 2026-01-13
- Estimated Expiration
- 2043-06-08
AI Technical Summary
Existing technologies cannot effectively identify minute features of vehicles, affecting subsequent multi-vehicle data association and target monitoring.
By constructing a feature layer of a multi-vehicle, multi-information recognition backbone network, multi-scale features of the initial image are obtained, multi-scale feature fusion and clustering are performed, anchor boxes are determined, candidate boxes are generated, target objects are recognized, and recognition results are output.
Without changing the image size, more minute feature information was detected, thus improving the accuracy of minute feature recognition.
Smart Images
Figure CN116630927B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of vehicle multi-information recognition, and particularly relates to a multi-vehicle multi-information efficient recognition method and device. BACKGROUND
[0002] In recent years, target detection and recognition have attracted more and more attention due to their wide application and breakthrough research in various fields, and vehicle detection and recognition are particularly important for automatic driving systems. Most of the current detection and recognition algorithms using convolutional neural networks (CNN) can only recognize the trajectory and position of the vehicle in the image, and ignore the micro feature information of each vehicle, which is not conducive to subsequent multi-vehicle data association and target monitoring.
[0003] Therefore, it is urgent to provide a multi-vehicle multi-information efficient recognition method and device to solve the technical problem that the micro feature information of each vehicle cannot be recognized in the prior art. SUMMARY
[0004] Therefore, it is urgent to provide a multi-vehicle multi-information efficient recognition method and device to solve the technical problem that the micro feature information of each vehicle cannot be recognized in the prior art.
[0005] In one aspect, the present application provides a multi-vehicle multi-information efficient recognition method, comprising:
[0006] obtaining an initial picture and constructing a feature layer of a multi-vehicle multi-information recognition backbone network;
[0007] detecting the initial picture according to the feature layer, determining a preset number of first multi-scale features, and obtaining a first feature map corresponding to each first multi-scale feature;
[0008] performing multi-scale feature fusion on the first feature map corresponding to the preset number of first multi-scale features to obtain a second feature map including a preset number of preset feature scales;
[0009] clustering the second feature map to determine a preset number of first anchor boxes in each preset feature scale, detecting the image in each first anchor box, and generating a preset number of first candidate boxes;
[0010] recognizing the target object in the preset number of first candidate boxes, determining the category of the target object, and outputting a recognition result according to the category.
[0011] In some possible implementation manners, the preset number of first anchor boxes includes a large-scale anchor box.
[0012] The clustering of the second feature map, determining a preset number of first anchor boxes in each preset feature scale, detecting the image in each first anchor box, and generating a preset number of candidate boxes further comprises:
[0013] Taking the large-scale anchor box as a limited area, a to-be-detected picture in the large-scale anchor box is determined.
[0014] According to the feature layer, the to-be-detected picture is detected to determine a preset number of second multi-scale features, and a third feature map corresponding to each second multi-scale feature is obtained.
[0015] Multi-scale feature fusion is performed on the third feature map corresponding to the preset number of second multi-scale features to obtain a fourth feature map including a preset number of preset feature scales.
[0016] The fourth feature map is clustered to determine a preset number of second anchor boxes in each preset feature scale, the image in each second anchor box is detected to generate a preset number of second candidate boxes, and the preset number of second anchor boxes includes a medium-scale anchor box and a small-scale anchor box.
[0017] The target object in the preset number of first candidate boxes is identified to determine the category of the target object, and an identification result is output according to the category, which comprises:
[0018] The target object in the preset number of first candidate boxes and the preset number of second candidate boxes is identified to determine the category of the target object, and an identification result is output according to the category.
[0019] In some possible implementation manners, the feature layer comprises a CBP module and a preset number of Res modules.
[0020] The detection of the initial picture according to the feature layer to determine a preset number of first multi-scale features, and obtaining a first feature map corresponding to each first multi-scale feature comprises:
[0021] The initial picture is detected according to the CBP module to obtain a preset number of first multi-scale features.
[0022] Each Res module processes the preset number of first multi-scale features to obtain a first feature map corresponding to each first multi-scale feature output by the each Res module.
[0023] In some possible implementation manners, the multi-scale feature fusion of the first feature map corresponding to the preset number of first multi-scale features to obtain a second feature map including a preset number of preset feature scales comprises:
[0024] Multi-scale feature fusion is performed on the first feature map corresponding to the preset number of first multi-scale features to obtain a fifth feature map;
[0025] The preset number of first multi-scale features on the fifth feature map are allocated according to the preset number of preset feature scales to obtain the second feature map.
[0026] In some possible implementations, clustering the second feature map to determine a preset number of first anchor boxes at each preset feature scale includes:
[0027] The second feature map and a preset number of third anchor boxes are clustered using the multi-dimensional mean clustering method to obtain the first intersection-union ratio corresponding to each third anchor box.
[0028] The third anchor frame whose first intersection-union ratio is less than the first preset threshold is determined as the first anchor frame, and a preset number of first anchor frames are obtained.
[0029] In some possible implementations, the step of detecting the image in each first anchor box and generating a preset number of first candidate boxes includes:
[0030] Based on the size corresponding to each preset feature scale, determine the grid image corresponding to each preset feature scale;
[0031] The image in each first anchor box in the grid image is detected according to the prediction method to obtain a preset number of first candidate boxes.
[0032] In some possible implementations, the prediction method includes:
[0033] Predict the grid image in the first anchor frame to obtain a preset number of third candidate frames;
[0034] Based on the preset number of third candidate boxes and the position information of the first anchor box, determine the second intersection-union ratio of each third candidate box;
[0035] The target score for each third candidate box is calculated based on the second intersection-union ratio.
[0036] The third candidate box whose target score is greater than the second preset threshold is determined as the first candidate box.
[0037] In some possible implementations, after predicting the grid image in the first anchor frame to obtain a preset number of third candidate frames, the method further includes:
[0038] Set the coordinate system of the first anchor frame according to its width and height;
[0039] Based on the coordinates of the upper left corner of each third candidate box, the width, the height, and the distance from the center point of the preselection box to the upper left corner, the coordinates corresponding to each third candidate box are determined respectively.
[0040] In some possible implementations, the recognition result includes the category and coordinates;
[0041] The process of identifying target objects in the preset number of first candidate boxes, determining the category of the target objects, and outputting the identification result based on the category includes:
[0042] Based on the coordinates corresponding to each third candidate box, determine the coordinates corresponding to each first candidate box;
[0043] Identify the target objects in the preset number of first candidate boxes and determine the category of the target objects;
[0044] The identification result of the target object is determined based on the category and the coordinates.
[0045] On the other hand, the present invention also provides a multi-vehicle, multi-information high-efficiency identification device, comprising:
[0046] The image acquisition module is used to acquire initial images and construct the feature layer of the multi-vehicle, multi-information recognition backbone network.
[0047] The feature determination module is used to detect the initial image based on the feature layer, determine a preset number of first multi-scale features, and obtain a first feature map corresponding to each first multi-scale feature;
[0048] The feature fusion module is used to perform multi-scale feature fusion on the first feature map corresponding to the preset number of first multi-scale features to obtain a second feature map including a preset number of preset feature scales.
[0049] The feature clustering module is used to cluster the second feature map, determine a preset number of first anchor boxes in each preset feature scale, detect the image in each first anchor box, and generate a preset number of first candidate boxes;
[0050] The object recognition module is used to recognize target objects in the preset number of first candidate boxes, determine the category of the target objects, and output the recognition result according to the category.
[0051] The beneficial effects of the above embodiments are as follows: The efficient multi-vehicle, multi-information recognition method provided by the present invention obtains a first feature map of a preset number of first multi-scale features of the initial image by setting a feature layer of the multi-vehicle, multi-information recognition backbone network. Without changing the size of the initial image, more multi-scale features are obtained. These multi-scale features can include large-scale, medium-scale, and small-scale features, enabling multi-faceted detection of the initial image, thereby detecting minute feature information in the initial image. Furthermore, by fusing the multi-scale features to determine the anchor box, and then predicting the image within the anchor box, candidate boxes for each target object are obtained. This accurately identifies minute feature information in the initial image, improving the accuracy of minute feature information recognition. Attached Figure Description
[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0053] Figure 1 A schematic flowchart of an embodiment of the efficient multi-vehicle, multi-information identification method provided by the present invention;
[0054] Figure 2 A schematic diagram of an embodiment of the prediction method provided by the present invention;
[0055] Figure 3 A schematic diagram of an embodiment of the multi-vehicle, multi-information high-efficiency identification device provided by the present invention;
[0056] Figure 4 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation
[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0058] Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.
[0059] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0060] This invention provides a method and apparatus for efficient identification of multiple vehicles and multiple information, which will be described below.
[0061] Figure 1 This is a schematic flowchart of an embodiment of the efficient multi-vehicle, multi-information identification method provided by the present invention, as shown below. Figure 1 As shown, efficient methods for identifying multiple vehicles and information include:
[0062] S101. Obtain the initial image and construct the feature layer of the multi-vehicle, multi-information recognition backbone network;
[0063] S102. Detect the initial image according to the feature layer, determine a preset number of first multi-scale features, and obtain a first feature map corresponding to each first multi-scale feature;
[0064] S103. Perform multi-scale feature fusion on the first feature map corresponding to the preset number of first multi-scale features to obtain a second feature map including a preset number of preset feature scales.
[0065] S104. Cluster the second feature map to determine a preset number of first anchor boxes in each preset feature scale, detect the image in each first anchor box, and generate a preset number of first candidate boxes;
[0066] S105. Identify the target objects in the preset number of first candidate boxes, determine the category of the target objects, and output the identification result according to the category.
[0067] Compared with existing technologies, the multi-vehicle, multi-information efficient recognition method provided by this invention obtains a first feature map of a preset number of first multi-scale features from the initial image by setting a feature layer of the multi-vehicle, multi-information recognition backbone network. This obtains more multi-scale features without changing the size of the initial image. These multi-scale features can include large-scale, medium-scale, and small-scale features, enabling multi-faceted detection of the initial image and thus detecting minute feature information. Furthermore, by fusing the multi-scale features to determine anchor boxes, and then predicting the images within the anchor boxes, candidate boxes for each target object are obtained. This accurately identifies minute feature information in the initial image, improving the accuracy of minute feature information recognition.
[0068] In some embodiments of the present invention, the feature layer includes a CBP module and a preset number of Res modules; step S102 includes:
[0069] The initial image is detected by the CBP module to obtain a preset number of first multi-scale features;
[0070] Each Res module processes the preset number of first multi-scale features to obtain a first feature map corresponding to each first multi-scale feature output by each Res module.
[0071] It should be noted that the CBP module and the Res module are as follows: the CBP module consists of convolutional layers, BN layers, and the PReLU activation function; the Res module is composed of multiple CBP modules connected by connection layers. The PReLU activation function can adaptively learn and correct linear units. First, feature maps are acquired through convolutional layers. If the number of feature maps is m and the feature map size is w*h (i.e., image size), then the amount of data for BN is m*w*h. The main operation steps of the BN layer are to calculate the mean and variance of all batch data, then normalize the pixel values by dividing the difference between the pixel value and the mean by the variance, and simultaneously adding offset and scale factors to control the normalized values. The values of these factors are learned by the neural network during training. The PReLU activation function is the gradient function learned by the network. The specific PReLU activation function can be set according to actual conditions, and this embodiment of the invention does not impose any limitations on it.
[0072] In a specific embodiment of the present invention, the backbone network may contain 52 convolutional layers and 1 fully connected layer, wherein multiple residual modules are stacked, and convolutional layers are added between the residual modules for downsampling. The initial image size can be 416*416. The first 3*3 convolutional kernel in the convolutional layer is mainly used to increase the number of channels, obtain more effective feature maps without changing the image size, and expand the receptive field of the feature maps. The second 3*3 convolutional kernel with a stride of 2 is mainly used for downsampling to reduce the amount of parameter calculation in the calculation process. Multiple residual modules are then stacked, and finally, the obtained feature maps are averaged and pooled. The connection method of the residual modules is as follows: the input feature map and the output feature map are convolved twice with convolutional kernel sizes of 1*1 and 3*3, and then the two output feature maps are connected. The Res module can be 5 ResX modules. After the initial image passes through the convolutional layer, it will go through 5 ResX modules to perform 5 feature map transformations, and each transformation is 2 times larger. The more transformations, the larger the scale of the extracted features. When the image convolution stride is 1 by default, the padding width is filled by default. When the convolution stride is 2, no padding is performed.
[0073] In some embodiments of the present invention, step S103 includes:
[0074] Multi-scale feature fusion is performed on the first feature map corresponding to the preset number of first multi-scale features to obtain a fifth feature map;
[0075] The preset number of first multi-scale features on the fifth feature map are allocated according to the preset number of preset feature scales to obtain the second feature map.
[0076] It should be noted that the first feature map can be fused at multiple scales using the feature multi-scale fusion module to obtain the fifth feature map. Then, the fifth feature map can be assigned to each feature scale using the feature multi-scale fusion module to obtain the second feature map including each feature scale. The feature scale can be preset in advance according to the actual situation, such as 13*13, 26*26, 52*52.
[0077] In some embodiments of the present invention, the step of clustering the second feature map to determine a preset number of first anchor boxes in each preset feature scale includes:
[0078] The second feature map and a preset number of third anchor boxes are clustered using the multi-dimensional mean clustering method to obtain the first intersection-union ratio corresponding to each third anchor box.
[0079] The third anchor frame whose first intersection-union ratio is less than the first preset threshold is determined as the first anchor frame, and a preset number of first anchor frames are obtained.
[0080] It should be noted that multiple third anchor boxes of different sizes can be preset, and then the preset number of third anchor boxes and the second feature map can be clustered by the multi-dimensional mean clustering method to obtain a suitable first anchor box. The first preset threshold can be set according to the actual situation, and this embodiment of the invention does not impose any restrictions on it.
[0081] In a specific embodiment of the present invention, the IOU (first intersection-union ratio) can be used to participate in the distance calculation. The larger the IOU, the closer the distance to the third anchor frame, and the more suitable the third anchor frame is. The calculation is shown in Formula 1:
[0082] D(box,centroid)=1-IOU(box,centroid) (1)
[0083] In the formula, Box represents the prior box generated by clustering, centroid is the true value, and IOU represents the intersection-union ratio of the prior box and the true value of the cluster. The true value can be set according to the actual situation, and the embodiments of the present invention do not impose any restrictions on it.
[0084] In a specific embodiment of the present invention, three feature scales of feature maps can be preset, and each feature scale can raise three preset suitable anchor boxes, as shown in Table 1:
[0085] Table 1. Anchor frames of different sizes
[0086]
[0087] In some embodiments of the present invention, the step of detecting the image in each first anchor box and generating a preset number of first candidate boxes includes:
[0088] Based on the size corresponding to each preset feature scale, determine the grid image corresponding to each preset feature scale;
[0089] The image in each first anchor box in the grid image is detected according to the prediction method to obtain a preset number of first candidate boxes.
[0090] In a specific embodiment of the present invention: such as Figure 2 As shown, the grid image can be the first feature map. The anchor box located at the center of the foreground object is responsible for predicting the object. These predictions can be described as a tensor S×S×[3*(4+1+classes)], representing the object category of the target object in the initial image. The mapping relationship between the image and the feature map is as follows: the vector contained in each grid is used to predict the coordinates, confidence score, and corresponding conditional probability of the anchor box. The approximate position of the target object can be determined by sliding the window. Then, the features are fused by the feature multi-scale fusion module to obtain the fused feature layer. The region is then restricted, for example, to 12 pixel coordinate values. Then, a preset number of first candidate boxes are obtained by using the non-maximum suppression method. The non-maximum suppression method can be set according to the actual situation, and this embodiment of the invention does not impose any restrictions on it.
[0091] In some embodiments of the present invention, the prediction method includes:
[0092] Predict the grid image in the first anchor frame to obtain a preset number of third candidate frames;
[0093] Based on the preset number of third candidate boxes and the position information of the first anchor box, determine the second intersection-union ratio of each third candidate box;
[0094] The target score for each third candidate box is calculated based on the second intersection-union ratio.
[0095] The third candidate box whose target score is greater than the second preset threshold is determined as the first candidate box.
[0096] In a specific embodiment of the present invention: the second intersection-union ratio of each third candidate box can be calculated by using the position information of a preset number of third candidate boxes and the first anchor box, as shown in Formula 2:
[0097]
[0098] In the formula, It is j th Define the area in i th The target score within the grid cell. (P) i,j (Object) is just an object function. The cross-union ratio represents the intersection of the predicted bounding box and the anchor box. The higher the target score, the better. A third candidate box with a target score greater than a second preset threshold can be determined as a first candidate box. The second preset threshold can be set according to the actual situation, and this embodiment of the invention does not limit it.
[0099] In some embodiments of the present invention, after predicting the grid image in the first anchor frame to obtain a preset number of third candidate frames, the method further includes:
[0100] Set the coordinate system of the first anchor frame according to its width and height;
[0101] Based on the coordinates of the upper left corner of each third candidate box, the width, the height, and the distance from the center point of the preselection box to the upper left corner, the coordinates corresponding to each third candidate box are determined respectively.
[0102] In a specific embodiment of the present invention: the coordinates of the candidate box are calculated as shown in Formula 3:
[0103]
[0104]
[0105]
[0106]
[0107] In the formula, (c x ,c y (p) represents the coordinates of the top-left corner of the area where the center of the preselected box is located. W ,p H () represents the length and width of the anchor frame. d represents the distance from the center point of the preselected box to the top left corner. W d H This indicates the predicted width and height offsets. This represents the sigmoid activation function, which can be set according to actual conditions; this embodiment of the invention does not impose any restrictions. x ,t y ,t W ,t HThese represent the distance along the x-axis from the center point of the preselection box to the top left corner, the distance along the y-axis, and the length and width of the preselection box, respectively.
[0108] In some embodiments of the present invention, the identification result includes the category and coordinates;
[0109] Step S105 includes:
[0110] Based on the coordinates corresponding to each third candidate box, determine the coordinates corresponding to each first candidate box;
[0111] Identify the target objects in the preset number of first candidate boxes and determine the category of the target objects;
[0112] The identification result of the target object is determined based on the category and the coordinates.
[0113] In a specific embodiment of the present invention: in order to more clearly determine the recognition result of the identified target object, after determining the coordinates corresponding to each third candidate box, the coordinates corresponding to each first candidate box can be determined, and then the recognition result of the target object in each first candidate box can be determined according to the coordinates and category corresponding to each first candidate box.
[0114] In some embodiments of the present invention, the preset number of first anchor frames includes large-scale anchor frames;
[0115] After clustering the second feature map to determine a preset number of first anchor boxes in each preset feature scale, detecting the image in each first anchor box, and generating a preset number of candidate boxes, the method further includes:
[0116] Using the large-scale anchor frame as a defined region, determine the image to be detected within the large-scale anchor frame;
[0117] The image to be detected is detected according to the feature layer, a preset number of second multi-scale features are determined, and a third feature map corresponding to each second multi-scale feature is obtained.
[0118] Multi-scale feature fusion is performed on the third feature map corresponding to the preset number of second multi-scale features to obtain a fourth feature map including a preset number of preset feature scales;
[0119] The fourth feature map is clustered to determine a preset number of second anchor boxes in each preset feature scale. The image in each second anchor box is detected to generate a preset number of second candidate boxes. The preset number of second anchor boxes includes medium-scale anchor boxes and small-scale anchor boxes.
[0120] It should be noted that: most of the small feature information is within the large-scale feature. Therefore, after detecting the initial image and determining the preset number of first candidate boxes on the initial image, the large-scale anchor box can be used as the limited area to determine the image to be detected within the large-scale anchor box. Then, the detection process of the initial image is repeated on the image to be detected to obtain the small feature information in the image to be detected.
[0121] In a specific embodiment of the present invention: the specific operation steps for the image to be detected are as follows: the image to be detected is detected by the CBP module and Res module of the feature layer, which can obtain a preset number of second multi-scale features, thereby obtaining a third feature map corresponding to each second multi-scale feature. Then, the feature multi-scale fusion module performs multi-scale feature fusion on the third feature map corresponding to the preset number of second multi-scale features to obtain a fourth feature map including a preset number of preset feature scales. Then, the preset number of third anchor boxes and second feature maps are clustered according to the multi-dimensional mean clustering method to obtain the third intersection-union ratio (IU) corresponding to each third anchor box. The third IU is then set to a threshold value. The third anchor box is determined as the second anchor box, resulting in a preset number of second anchor boxes. These preset number of second anchor boxes include medium-scale anchor boxes and small-scale anchor boxes, such as anchor boxes corresponding to 26*26 and 52*52. Based on the size corresponding to each preset feature scale, a grid image corresponding to each preset feature scale is determined. The grid images in the second anchor boxes are predicted to obtain a preset number of fourth candidate boxes. The fourth intersection-union ratio (CIU) of each fourth candidate box is determined based on the position information of the preset number of fourth candidate boxes and the second anchor boxes. The target score of each fourth candidate box is calculated based on the CIU. Fourth candidate boxes with target scores greater than a second preset threshold are determined as second candidate boxes.
[0122] In a specific embodiment of the present invention: the loss function used when processing the initial image is the basic loss function. Since the detected multi-target information is part of the vehicle, most of the smaller specific information features that need to be detected are within large-scale features. To quickly obtain the location and classification information of small objects, the large-scale anchor boxes provided by the feature multi-scale fusion module are used as the limiting region to narrow the search range of the entire image. The improved loss function of the feature map is shown in Equation 4:
[0123]
[0124] In the formula, λ is a weighting function that controls the ratio between the detection box loss, the obj confidence loss, and the noobj confidence loss; i,j represent the j-th anchor box in the i-th grid. This indicates that there is a target at that point; P ij (c) represents the classification result value, C ij Represented as Indicates the confidence level of the parameter; The feature multi-scale fusion module uses a 13*13 feature map to limit the area for small object detection in the 52*52 feature map, thereby reducing the search range of image features, improving the efficiency of the algorithm, and improving the detection accuracy. The weight function and classification result value can be set according to the actual situation, and this embodiment of the invention does not impose any restrictions.
[0125] In some embodiments of the present invention, the step of identifying target objects in the preset number of first candidate boxes, determining the category of the target objects, and outputting the identification result according to the category includes:
[0126] The target objects in the preset number of first candidate boxes and the preset number of second candidate boxes are identified, the category of the target objects is determined, and the identification result is output according to the category.
[0127] In a specific embodiment of the present invention: in order to more clearly determine the recognition result of the identified target object, after determining the coordinates corresponding to each third candidate box, the coordinates corresponding to each first candidate box can be determined; after determining the coordinates corresponding to each fourth candidate box, the coordinates corresponding to each second candidate box can be determined; and then, based on the coordinates and category corresponding to each first candidate box and the coordinates and category corresponding to each second candidate box, the recognition result of the target object in each first candidate box is determined. The method for recognizing the category of the target object can be set according to the actual situation, and the present invention does not limit it here.
[0128] In this embodiment of the invention, after detecting the initial image and obtaining a preset number of first candidate boxes for the target object in the initial image, in order to more accurately identify minute feature information, the large-scale anchor box in the initial image can be used as a limited area to detect the image to be detected in the large-scale anchor box, thereby obtaining minute feature information in the image to be detected. While identifying minute feature information in the image, the accuracy of minute feature information can also be improved.
[0129] To better implement the efficient multi-vehicle, multi-information identification method in the embodiments of the present invention, based on the efficient multi-vehicle, multi-information identification method, the embodiments of the present invention also provide an efficient multi-vehicle, multi-information identification device, such as... Figure 3 As shown, the multi-vehicle, multi-information high-efficiency identification device includes:
[0130] Image acquisition module 301 is used to acquire initial images and construct the feature layer of the multi-vehicle multi-information recognition backbone network.
[0131] The feature determination module 302 is used to detect the initial image based on the feature layer, determine a preset number of first multi-scale features, and obtain a first feature map corresponding to each first multi-scale feature;
[0132] Feature fusion module 303 is used to perform multi-scale feature fusion on the first feature map corresponding to the preset number of first multi-scale features to obtain a second feature map including a preset number of preset feature scales;
[0133] The feature clustering module 304 is used to cluster the second feature map, determine a preset number of first anchor boxes in each preset feature scale, detect the image in each first anchor box, and generate a preset number of first candidate boxes.
[0134] The object recognition module 305 is used to recognize the target objects in the preset number of first candidate boxes, determine the category of the target objects, and output the recognition result according to the category.
[0135] The multi-vehicle, multi-information efficient identification device provided in the above embodiments can realize the technical solutions described in the above multi-vehicle, multi-information efficient identification method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above multi-vehicle, multi-information efficient identification method embodiments, and will not be repeated here.
[0136] like Figure 4 As shown, the present invention also provides an electronic device 400. The electronic device 400 includes a processor 401, a memory 402, and a display 403. Figure 4 Only some components of the electronic device 400 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.
[0137] In some embodiments, memory 402 may be an internal storage unit of electronic device 400, such as a hard disk or memory of electronic device 400. In other embodiments, memory 402 may also be an external storage device of electronic device 400, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 400.
[0138] Furthermore, the memory 402 may include both internal storage units of the electronic device 400 and external storage devices. The memory 402 is used to store application software and various types of data installed on the electronic device 400.
[0139] In some embodiments, processor 401 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 402 or process data, such as the efficient identification method for multiple vehicles and multiple information in this invention.
[0140] In some embodiments, display 403 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 403 is used to display information from electronic device 400 and to display a visual user interface. Components 401-403 of electronic device 400 communicate with each other via a system bus.
[0141] In some embodiments of the present invention, when the processor 401 executes the multi-vehicle, multi-information high-efficiency identification program in the memory 402, the following steps can be implemented:
[0142] Obtain the initial image and construct the feature layer of the multi-vehicle, multi-information recognition backbone network;
[0143] The initial image is detected based on the feature layer to determine a preset number of first multi-scale features, and a first feature map corresponding to each first multi-scale feature is obtained.
[0144] Multi-scale feature fusion is performed on the first feature map corresponding to the preset number of first multi-scale features to obtain a second feature map including a preset number of preset feature scales;
[0145] Cluster the second feature map to determine a preset number of first anchor boxes in each preset feature scale, detect the image in each first anchor box, and generate a preset number of first candidate boxes;
[0146] The target objects in the preset number of first candidate boxes are identified, the category of the target objects is determined, and the identification result is output according to the category.
[0147] It should be understood that when the processor 401 executes the multi-vehicle, multi-information efficient identification program in the memory 402, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.
[0148] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 400 mentioned. Electronic device 400 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 400 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).
[0149] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can realize the steps or functions of the efficient multi-vehicle, multi-information identification method provided in the above-described method embodiments.
[0150] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.
[0151] The above provides a detailed description of the efficient multi-vehicle, multi-information identification method and apparatus provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A multi-vehicle multi-information efficient identification method, characterized in that, The method comprises the following steps: acquiring an initial picture and constructing a feature layer of a multi-vehicle multi-information recognition backbone network; detecting the initial picture according to the feature layer, determining a preset number of first multi-scale features, and obtaining a first feature map corresponding to each first multi-scale feature; performing multi-scale feature fusion on the first feature maps corresponding to the preset number of first multi-scale features to obtain a second feature map including a preset number of preset feature scales; performing clustering on the second feature map, determining a preset number of first anchor boxes in each preset feature scale, detecting images in each first anchor box, and generating a preset number of first candidate boxes; recognizing target objects in the preset number of first candidate boxes, determining the categories of the target objects, and outputting recognition results according to the categories; the feature layer comprises a CBP module and a preset number of Res modules; the CBP module is composed of a convolution layer, a BN layer and a PReLu activation function, the Res module is composed of a plurality of CBP modules combined through a connection layer, the PReLu activation function adaptively learns to correct a linear unit, first, the convolution layer is used to acquire a feature map, second, if the number of obtained feature maps is m and the size of the feature map is w x h, that is, the image size, then the data volume of the BN is m x w x h; the operation steps of the BN layer are to calculate the mean and variance of all batch data, then the pixel value is subtracted from the mean and then divided by the variance to normalize, and at the same time, an offset factor and a scale change factor are added to control the normalized value, the values of the factors are learned by the neural network during training, and the PReLu activation function is a gradient function during network learning; the improved loss function when detecting images in each first anchor box is as follows: In the formula, It is a control detection loss, Confidence loss The weighting function for the proportions of confidence losses; i, j Indicates the first i The first grid j An anchor frame; This indicates that there is a target at that point; P ij ( c () represents the classification result value; C ij Represented as , representing the confidence level of the parameter; The feature multi-scale fusion module uses a 13×13 feature map to limit the area for small object detection in the 52×52 feature map.
2. The multi-vehicle multi-information efficient identification method according to claim 1, characterized in that, the preset number of first anchor boxes includes a large-scale anchor box; after the clustering of the second feature map, the determination of a preset number of first anchor boxes in each preset feature scale, the detection of images in each first anchor box, and the generation of a preset number of candidate boxes, the method further comprises the following steps: determining a to-be-detected picture in the large-scale anchor box as a limited area; detecting the to-be-detected picture according to the feature layer, determining a preset number of second multi-scale features, and obtaining a third feature map corresponding to each second multi-scale feature; performing multi-scale feature fusion on the third feature maps corresponding to the preset number of second multi-scale features to obtain a fourth feature map including a preset number of preset feature scales; performing clustering on the fourth feature map, determining a preset number of second anchor boxes in each preset feature scale, detecting images in each second anchor box, and generating a preset number of second candidate boxes; the preset number of second anchor boxes includes a medium-scale anchor box and a small-scale anchor box; the method of recognizing target objects in the preset number of first candidate boxes, determining the categories of the target objects, and outputting recognition results according to the categories comprises the following steps: recognizing target objects in the preset number of first candidate boxes and the preset number of second candidate boxes, determining the categories of the target objects, and outputting recognition results according to the categories.
3. The multi-vehicle multi-information efficient identification method according to claim 1, characterized in that, the feature layer comprises a CBP module and a preset number of Res modules; The initial picture is detected according to the feature layer, a preset number of first multi-scale features are determined, and a first feature map corresponding to each first multi-scale feature is obtained, including: The initial picture is detected according to the CBP module, and a preset number of first multi-scale features are obtained; Each Res module processes the preset number of first multi-scale features, and a first feature map corresponding to each first multi-scale feature output by the Res module is obtained.
4. The multi-vehicle multi-information efficient identification method according to claim 1, characterized in that, The first feature map corresponding to the preset number of first multi-scale features is subjected to multi-scale feature fusion, and a second feature map including a preset number of preset feature scales is obtained, including: The first feature map corresponding to the preset number of first multi-scale features is subjected to multi-scale feature fusion, and a fifth feature map is obtained; The preset number of first multi-scale features on the fifth feature map are distributed according to the preset number of preset feature scales, and the second feature map is obtained.
5. The multi-vehicle multi-information efficient identification method according to claim 1, characterized in that, The second feature map is clustered to determine a preset number of first anchor boxes in each preset feature scale, including: A multi-dimensional mean clustering method is used to cluster a preset number of third anchor boxes and the second feature map, and a first intersection over union corresponding to each third anchor box is obtained; The third anchor box with a first intersection over union less than a first preset threshold is determined as a first anchor box, and a preset number of first anchor boxes are obtained.
6. The multi-vehicle multi-information efficient identification method according to claim 1, characterized in that, Each image in the first anchor box is detected to generate a preset number of first candidate boxes, including: A grid image corresponding to each preset feature scale is determined according to the size corresponding to the preset feature scale; An image in each first anchor box in the grid image is detected according to a prediction method to obtain a preset number of first candidate boxes.
7. The multi-vehicle multi-information efficient identification method according to claim 6, characterized in that, The prediction method includes: The grid image in the first anchor box is predicted to obtain a preset number of third candidate boxes; A second intersection over union of each third candidate box is determined according to the position information of the preset number of third candidate boxes and the first anchor box, respectively; A target score of each third candidate box is calculated according to the second intersection over union; The third candidate box with a target score greater than a second preset threshold is determined as a first candidate box.
8. The multi-vehicle multi-information efficient identification method according to claim 7, characterized in that, After the grid image in the first anchor box is predicted to obtain a preset number of third candidate boxes, the following steps are further included: A coordinate system of the first anchor box is set according to the width and height of the first anchor box; The coordinates corresponding to each third candidate box are determined according to the coordinates of the upper left corner of each third candidate box, the width, the height, and the distance from the center point of each third candidate box to the upper left corner.
9. The multi-vehicle multi-information efficient identification method according to claim 8, characterized in that, The recognition result includes the category and the coordinates; The target object in the preset number of first candidate boxes is identified to determine the category of the target object, and the recognition result is output according to the category, including: The coordinates corresponding to each first candidate box are determined according to the coordinates corresponding to each third candidate box; The target object in the preset number of first candidate boxes is identified to determine the category of the target object; The recognition result of the target object is determined according to the category and the coordinates.
10. A multi-vehicle multi-information efficient recognition device, characterized by, The method comprises the following steps: An image acquisition module is configured to acquire an initial image and construct a feature layer of a multi-vehicle multi-information identification backbone network. A feature determination module is configured to detect the initial image according to the feature layer, determine a preset number of first multi-scale features, and obtain a first feature map corresponding to each first multi-scale feature. A feature fusion module is configured to perform multi-scale feature fusion on the first feature map corresponding to the preset number of first multi-scale features to obtain a second feature map including a preset number of preset feature scales. A feature clustering module is configured to cluster the second feature map, determine a preset number of first anchor boxes in each preset feature scale, detect an image in each first anchor box, and generate a preset number of first candidate boxes. An object identification module is configured to identify a target object in the preset number of first candidate boxes, determine a category of the target object, and output an identification result according to the category. The feature layer includes a CBP module and a preset number of Res modules. The CBP module is composed of a convolution layer, a BN layer and a PReLu activation function, the Res module is composed of a plurality of CBP modules combined through a connection layer, the PReLu activation function adaptively learns to correct a linear unit, first, the convolution layer is used to acquire a feature map, second, if the number of obtained feature maps is m and the size of the feature map is w x h, that is, the size of the image, then the data volume of the BN is m x w x h; the operation steps of the BN layer are to calculate the mean and variance of all batch data, then normalize the pixel value by subtracting the mean value from the pixel value and dividing the result by the variance, and add an offset factor and a scale change factor to control the normalized value, the values of the factors are learned by the neural network during training, and the PReLu activation function is a gradient function during network learning. The improved loss function when detecting the image in each first anchor box is as follows: In the formula, It is a control detection loss, Confidence loss The weighting function for the proportions of confidence losses; i, j Indicates the first i The first grid j An anchor frame; This indicates that there is a target at that point; P ij ( c () represents the classification result value; C ij Represented as , representing the confidence level of the parameter; The feature multi-scale fusion module uses a 13×13 feature map to limit the area for small object detection in the 52×52 feature map.