Vehicle information identification method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle information identification method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a plurality of to-be-identified images of a vehicle, carrying out the feature extraction of the to-be-identified images according to a plurality of multi-scale residual blocks, and obtaining a feature extraction image of the to-be-identified images, the feature extraction comprises the steps of performing multi-level convolution on the initial input image according to a plurality of preset convolution layers in each multi-scale residual block, and performing maximum pooling on the residual image to obtain a pooling output image; classifying the feature extraction images corresponding to the to-be-recognized images according to a preset recognition area to obtain a plurality of classification results and corresponding classification confidence degrees, and determining the feature extraction images with the classification results that the images contain the preset recognition area and the corresponding classification confidence degrees as target feature images; performing character recognition on the target feature map to obtain vehicle recognition information; according to the method, through feature extraction of the multiple multi-scale residual blocks, the classification accuracy is improved, and the character recognition accuracy is improved.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle information identification, and in particular to a vehicle information identification method, device, electronic device and storage medium. Background Art

[0002] With the development of intelligent transportation systems, vehicle identification has become a key technology in traffic safety management and intelligent traffic monitoring. Vehicle identification is not only crucial for traffic violation detection and accident investigation, but also plays a vital role in applications such as parking lot management and vehicle tracking. Among the various vehicle identification technologies, image processing-based vehicle contour extraction and matching methods are widely used due to their non-contact and low-cost advantages.

[0003] However, the contour-based vehicle recognition technology in related technologies has several limitations, which restrict its effectiveness and scope in practical applications. First, the accuracy of vehicle contour extraction is highly dependent on image quality, including image clarity and lighting conditions. It is easily affected by factors such as background complexity, light changes, and occlusion, resulting in reduced recognition accuracy. Second, contour-based matching methods are prone to matching errors when dealing with vehicles with similar appearances, especially when it is difficult to distinguish between vehicles of the same make and model. Moreover, relying solely on vehicle contour matching for matching cannot extract detailed vehicle parameters, and the information extracted is incomplete. Summary of the Invention

[0004] The present invention provides a vehicle information recognition method, device, electronic device and storage medium to solve the above-mentioned technical problems of low accuracy in vehicle information recognition and incomplete information extraction.

[0005] In one embodiment of the present application, the present application provides a vehicle information recognition method, including: obtaining multiple images to be recognized of a vehicle; performing feature extraction on the images to be recognized according to a multi-scale residual block to obtain a feature extraction map of the images to be recognized, the number of the multi-scale residual blocks includes multiple, and the pooled output image output by a multi-scale residual block is the initial input image of the next multi-scale residual block, the feature extraction includes performing multi-level convolution on the initial input image according to multiple preset convolution layers in each multi-scale residual block, and performing maximum pooling on the residual image to obtain a pooled output image; classifying the feature extraction map corresponding to each of the images to be recognized according to a preset recognition area to obtain multiple classification results and corresponding classification confidences, and determining the feature extraction map whose classification result is that the image contains the preset recognition area and has a high corresponding classification confidence as a target feature map; performing character recognition on the target feature map to obtain vehicle identification information; wherein, the residual image is obtained based on the initial input image and the convolution output image of the last preset convolution layer, and each of the preset convolution layers is connected in series and the convolution kernel size increases sequentially.

[0006] In one embodiment of the present application, the feature extraction also includes: assigning different channel weights to multiple channel extraction maps, wherein the channel extraction map is used to represent the image corresponding to a channel in the pooled output image of the last multi-scale residual block; and weighting is performed according to each of the channel weights and the corresponding channel extraction map to obtain a feature extraction map.

[0007] In one embodiment of the present application, different channel weights are assigned to multiple channel extraction maps, including: performing global average pooling on each of the channel extraction maps to obtain an initial feature vector, wherein the initial feature vector includes a spatial feature scalar corresponding to each channel extraction map; compressing the number of channels of the initial feature vector according to a preset first weight matrix to obtain an intermediate feature vector; and restoring the number of channels of the intermediate feature vector based on a preset second weight matrix to obtain the channel weights of each of the channel extraction maps.

[0008] In one embodiment of the present application, the residual image is obtained based on the initial input image and the convolution output image of the last preset convolution layer, and each of the preset convolution layers is connected in series and the convolution kernel size increases sequentially. It also includes: convolving the initial input image to obtain a jump image to adjust the number of channels of the initial input image to be consistent with the number of channels of the convolution output image of the last preset convolution layer; adding the jump image and the convolution output image of the last convolution layer to obtain the residual image.

[0009] In one embodiment of the present application, the number of preset convolutional layers includes three, the first preset convolutional layer is used to adjust the number of channels of the initial input image to be consistent with the number of channels output by the multi-scale residual block, the second preset convolutional layer is used to extract local features of the image, and the third preset convolutional layer is used to extract large-scale features of the image; the height and width of the convolution kernel size of the preset convolutional layer are the same; the pooled output image is used to represent the residual image with half the spatial size; the number of channels output by the next multi-scale residual block is twice the number of channels output by the previous multi-scale residual block; the initial input image of the first multi-scale residual block is the image to be identified.

[0010] In one embodiment of the present application, the feature extraction map corresponding to the image to be identified is classified according to a preset recognition area, including: performing global average pooling on the feature extraction map corresponding to the image to be identified to obtain a feature vector to be classified, and the feature vector to be classified includes the global features of each channel in the feature extraction map; classifying the feature vector to be classified according to the preset recognition area to obtain a first classification probability and a second classification probability, the first classification probability is used to characterize the probability that the image contains the preset recognition area, and the second classification probability is used to characterize the probability that the image does not contain the preset recognition area; performing probability distribution conversion on the first classification probability and the second classification probability to obtain two distribution probabilities; determining the category corresponding to the maximum distribution probability as the classification result, and determining the maximum distribution probability as the classification confidence.

[0011] In one embodiment of the present application, character recognition is performed on the target feature map to obtain vehicle identification information, including: super-resolution reconstruction of the target feature map to obtain a high-resolution feature map; identifying character features in the high-resolution feature map to obtain a character recognition result, the character recognition result including the position and content of the recognized characters and / or text blocks; regular expression matching of the character recognition result with a preset format to obtain format identification information, logically checking the parameter values of the format identification information to obtain vehicle identification information, so as to match the identity identification information of the vehicle.

[0012] In one embodiment of the present application, the present application provides a vehicle information recognition device, comprising: an image acquisition module for acquiring multiple images to be recognized of a vehicle; a feature extraction module for performing feature extraction on the image to be recognized based on a multi-scale residual block to obtain a feature extraction map of the image to be recognized, wherein the number of the multi-scale residual blocks includes multiple, the pooled output image output by a multi-scale residual block is the initial input image of the next multi-scale residual block, and the feature extraction includes performing multi-level convolution on the initial input image according to multiple preset convolution layers in each multi-scale residual block, and performing maximum pooling on the residual image. , obtaining a pooled output image; an image classification module, used to classify the feature extraction maps corresponding to each of the images to be identified according to the preset recognition area, obtain multiple classification results and corresponding classification confidences, and determine the feature extraction map whose classification result is that the image contains the preset recognition area and the corresponding classification confidence is high as the target feature map; an information recognition module, used to perform character recognition on the target feature map to obtain vehicle identification information; wherein, the residual image is obtained based on the initial input image and the convolution output image of the last preset convolution layer, and each of the preset convolution layers is connected in series and the convolution kernel size increases sequentially.

[0013] In one embodiment of the present application, the present application provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the vehicle information identification method as described in any of the above embodiments.

[0014] In one embodiment of the present application, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer executes the vehicle information identification method described in any of the above embodiments.

[0015] Beneficial effects of embodiments of the present invention: The present invention provides a vehicle information recognition method, device, electronic device and storage medium. The embodiments of the present invention use multiple serially connected multi-scale residual blocks and multiple preset convolutional layers and maximum pooling layers in each multi-scale residual block to extract features of the image to be recognized from the surface to the deep layer and from fine particles to coarse particles to obtain a feature extraction map, which can improve the classification accuracy of the feature extraction map according to the preset recognition area, thereby improving the accuracy of character recognition and improving the comprehensiveness of information extraction.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, serving to explain the principles of the present application. It is obvious that the drawings described below are merely some embodiments of the present application, and a person of ordinary skill in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0018] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied;

[0019] Figure 2 A schematic diagram showing a process of a vehicle information identification method according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of a process for extracting features from an image to be identified according to an embodiment of the present application is shown;

[0021] Figure 4 A block diagram of a vehicle information recognition device according to an embodiment of the present application is shown;

[0022] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0025] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0026] See also Figure 1 , Figure 1 Schematic diagram showing an exemplary system architecture to which the technical solution of the embodiment of the present application can be applied. Figure 1 As shown, the system architecture may include a computer device 101 and an image acquisition device 102. Computer device 101 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a general-purpose computer, a neural network computer, etc. Image acquisition device 102 is used to acquire multiple images to be recognized and transmit them to computer device 101 for vehicle information recognition.

[0027] Exemplarily, the computer device 101 obtains multiple images to be identified of a vehicle; performs feature extraction on the images to be identified according to multi-scale residual blocks to obtain a feature extraction map of the images to be identified, the number of multi-scale residual blocks includes multiple, the pooled output image output by a multi-scale residual block is the initial input image of the next multi-scale residual block, and the feature extraction includes performing multi-level convolution on the initial input image according to multiple preset convolution layers in each multi-scale residual block, and performing maximum pooling on the residual image to obtain a pooled output image; classifies the feature extraction map corresponding to each image to be identified according to the preset recognition area, obtains multiple classification results and corresponding classification confidences, and determines the feature extraction map whose classification result is that the image contains the preset recognition area and the corresponding classification confidence is high as the target feature map; performs character recognition on the target feature map to obtain vehicle identification information; wherein, the residual image is obtained based on the initial input image and the convolution output image of the last preset convolution layer, and each preset convolution layer is connected in series and the convolution kernel size increases sequentially.

[0028] In the related technology, there are problems such as low accuracy of vehicle information recognition and incomplete information extraction.

[0029] In order to solve the above technical problems, the present application provides a vehicle information identification method, device, electronic device and storage medium. The implementation details of the technical solution of the embodiment of the present application are elaborated in detail below.

[0030] See also Figure 2 , Figure 2 FIG. 1 shows a flow chart of a vehicle information identification method according to an embodiment of the present application. Figure 2 As shown, in an exemplary embodiment, the vehicle information recognition method includes at least steps S210 to S240, which are described in detail as follows:

[0031] Step S210: Acquire multiple images of the vehicle to be identified.

[0032] In one embodiment of the present application, an image capture device captures video data of a vehicle to be identified, and extracts the image data frame by frame to obtain multiple images to be identified. The video data can be obtained with user authorization, or proactively submitted by the user after reading instructions, or uploaded by the user when using the video capture function.

[0033] In one embodiment of the present application, each image to be identified is converted into a format suitable for input to a target detection model, including image resizing, normalization, etc.

[0034] Step S220: performing feature extraction on the image to be identified according to the multi-scale residual block to obtain a feature extraction map of the image to be identified. The number of multi-scale residual blocks includes multiple. The feature extraction includes performing multi-level convolution on the initial input image according to multiple preset convolution layers in each multi-scale residual block, and performing maximum pooling on the residual image to obtain a pooled output image.

[0035] The number of the multi-scale residual blocks includes multiple ones, and the pooled output image output by a multi-scale residual block is the initial input image of the next multi-scale residual block.

[0036] In one embodiment of the present application, the image to be recognized includes a Red Green Blue (RGB) format with a variable size. The tensor format of the image to be recognized is: tensor shape: [batch_size, 3, height, width], where batch_size represents the number of images to be recognized that are input simultaneously and can be one or more, 3 represents the number of channels of the image to be recognized, and RGB images have three channels, and height and width represent the height and width of the image to be recognized, respectively, and can be variable.

[0037] In one embodiment of the present application, in each multi-scale residual block, the first preset convolutional layer is used to adjust the number of channels of the initial input image, and the remaining at least one preset convolutional layer is used to extract image features.

[0038] In one embodiment of the present application, the convolution output image after convolution through a preset convolution layer for extracting image features needs to adjust the spatial size of the convolution output image to be consistent with the spatial size of the image before convolution through same-size padding (SAME padding).

[0039] In one embodiment of the present application, after the number of channels of the initial input image is adjusted by the first preset convolutional layer, the number of channels of the multi-scale residual block where the first preset convolutional layer is located no longer changes.

[0040] In one embodiment of the present application, the number of preset convolutional layers includes three, the first preset convolutional layer is used to adjust the number of channels of the initial input image to be consistent with the number of channels output by the multi-scale residual block in which it is located, the second preset convolutional layer is used to extract local features of the image, and the third preset convolutional layer is used to extract large-scale features of the image; the height and width of the convolution kernel size of the preset convolutional layer are the same; the pooled output image is used to represent the residual image with half the spatial size; the number of channels output by the next multi-scale residual block is twice the number of channels output by the previous multi-scale residual block; the initial input image of the first multi-scale residual block is the image to be recognized.

[0041] In one embodiment of the present application, the convolution kernel sizes of the preset convolution layers are 1x1, 3x3, and 5x5, respectively.

[0042] In one embodiment of the present application, each multi-scale residual block performs a convolution operation on the initial input image and uses a skip connection to directly add the input to the output. Furthermore, the number of channels and spatial size of the skip connection and the output after the multi-scale convolution are adjusted to ensure that they match.

[0043] In one embodiment of the present application, size padding is performed by using SAME padding so that the spatial size of the output is the same as the spatial size of the input, that is, the height and width of the input and output in the multi-scale residual block remain unchanged.

[0044] In one embodiment of the present application, the number of multi-scale residual blocks includes four.

[0045] In one embodiment of the present application, the residual image is obtained based on the initial input image and the convolution output image of the last preset convolution layer, and each preset convolution layer is connected in series and the convolution kernel size increases successively. It also includes: convolving the initial input image to obtain a jump image to adjust the number of channels of the initial input image to be consistent with the number of channels of the convolution output image of the last preset convolution layer; adding the jump image and the convolution output image of the last convolution layer to obtain a residual image.

[0046] In one embodiment of the present application, in the first multi-scale residual block, the input is an image to be recognized, the shape of which is [batch_size, 3, height, width]. The number of output channels of this layer is C = 64, so the number of channels of the initial input image in RGB format needs to be adjusted.

[0047] In one embodiment of the present application, 1x1 convolution: The initial input image passes through a 1x1 convolution, and the number of channels is adjusted from 3 to 64 to ensure consistency with the number of channels in subsequent convolutional layers. 3x3 convolution: The input then passes through a 3x3 convolution, using SAME padding to make the output spatial size the same as the input. The number of channels remains at 64 to extract local features. 5x5 convolution: The input then passes through a 5x5 convolution, using SAME padding to keep the output spatial size consistent with the input, while the number of channels remains at 64 to extract large-scale features.

[0048] In one embodiment of the present application, a skip connection is used: Since the input is an RGB image, the number of channels needs to be adjusted from 3 to 64 through 1x1 convolution to ensure that the number of channels is consistent with the number of channels after the multi-scale convolution. The input of the skip connection is added to the output of the multi-scale convolution to form the first residual image output.

[0049] In one embodiment of the present application, the multi-scale residual block further includes a pooling layer, which performs a maximum pooling operation on the first residual image to halve the spatial size of the first residual image to extract more global features.

[0050] In one embodiment of the present application, the output of the first multi-scale residual block is: [batch_size, 64, height / 2, width / 2].

[0051] In one embodiment of the present application, the second multi-scale residual block takes as input the pooled output image of the first multi-scale residual block, with a shape of [batch_size, 64, height / 2, width / 2]. Output channel design: The number of output channels of this layer is 2C = 128, so the number of input channels needs to be adjusted.

[0052] In one embodiment of the present application, a 1x1 convolution is performed on the initial input image, adjusting the number of channels from 64 to 128 to ensure consistency with the number of channels in subsequent convolutional layers. The image then undergoes a 3x3 convolution with same-size padding, and a 5x5 convolution with same-size padding, leaving the output channel number unchanged at 128.

[0053] In one embodiment of the present application, a skip connection is used: Because the current initial input image has 64 channels, while the output channel number after convolution is 128, it is necessary to adjust the channel number from 64 to 128 through 1x1 convolution to match the channel number of the convolution output. The input of the skip connection is added to the output of the multi-scale convolution to form a second residual image output. A maximum pooling operation is performed on the second residual image to reduce the spatial size of the second residual image by half, thereby extracting more global features.

[0054] In one embodiment of the present application, the output of the second multi-scale residual block is [batch_size, 128, height / 4, width / 4].

[0055] In one embodiment of the present application, the third multi-scale residual block takes as input the pooled output image of the second multi-scale residual block, with a shape of [batch_size, 128, height / 4, width / 4]. Output channel design: The number of output channels of this layer is 4C = 256, so the number of input channels needs to be adjusted.

[0056] In one embodiment of the present application, a 1x1 convolution is performed on the initial input image, adjusting the number of channels from 128 to 256 to ensure consistency with the number of channels in subsequent convolutional layers. The image then undergoes a 3x3 convolution with same-size padding, and a 5x5 convolution with same-size padding, leaving the output channel number unchanged at 256.

[0057] In one embodiment of the present application, a skip connection is used: Because the current initial input image has 128 channels, while the output channel number after convolution is 256, a 1x1 convolution is performed to adjust the channel number from 128 to 256 to match the channel number of the convolution output. The skip connection input is added to the output of the multi-scale convolution to form a third residual image output. A maximum pooling operation is performed on the third residual image to halve its spatial size to extract more global features.

[0058] In one embodiment of the present application, the output of the third multi-scale residual block is [batch_size, 256, height / 8, width / 8].

[0059] In one embodiment of the present application, the fourth multi-scale residual block takes as input the pooled output image of the third multi-scale residual block, with a shape of [batch_size, 256, height / 8, width / 8]. Output channel design: The number of output channels of this layer is 8C = 512, so the number of input channels needs to be adjusted.

[0060] In one embodiment of the present application, a 1x1 convolution is performed on the initial input image, adjusting the number of channels from 256 to 512 to ensure consistency with the number of channels in subsequent convolutional layers. The image then undergoes a 3x3 convolution with same-size padding, and a 5x5 convolution with same-size padding, leaving the output channel number unchanged at 512.

[0061] In one embodiment of the present application, a skip connection is used: Because the current initial input image has 256 channels, while the output channel number after convolution is 512, a 1x1 convolution is performed to adjust the channel number from 256 to 512 to match the channel number of the convolution output. The input of the skip connection is added to the output of the multi-scale convolution to form a fourth residual image output. A maximum pooling operation is performed on the fourth residual image to halve its spatial size to extract more global features.

[0062] In one embodiment of the present application, the output of the fourth multi-scale residual block is [batch_size, 512, height / 16, width / 16].

[0063] In one embodiment of the present application, feature extraction also includes: assigning different channel weights to multiple channel extraction maps, the channel extraction map is used to represent the image corresponding to a channel in the pooled output image of the last multi-scale residual block; weighting is performed according to each channel weight and the corresponding channel extraction map to obtain a feature extraction map.

[0064] In one embodiment of the present application, different channel weights are assigned to multiple channel extraction maps, including: performing global average pooling on each channel extraction map to obtain an initial feature vector, where the initial feature vector includes a spatial feature scalar corresponding to each channel extraction map; compressing the number of channels of the initial feature vector according to a preset first weight matrix to obtain an intermediate feature vector; and restoring the number of channels of the intermediate feature vector based on a preset second weight matrix to obtain the channel weights of each channel extraction map.

[0065] In one embodiment of the present application, after feature extraction by multiple multi-scale residual blocks, the final pooled output image is input into a dynamic channel attention mechanism to assign different channel weights to each channel to further enhance task-related features.

[0066] In one embodiment of the present application, a global average pooling operation is performed on the channel extraction map corresponding to each channel to compress the spatial feature information of each channel into a scalar, namely a spatial feature scalar. The result of pooling is an initial feature vector representing the global feature of each channel.

[0067] In one embodiment of the present application, channel weights for each channel are generated through two fully connected layers. The first fully connected layer compresses the number of channels to 1 / 4, generating a compressed intermediate feature vector; the second fully connected layer restores the number of channels to the original number corresponding to the pooled output image of the last multi-scale residual block, obtaining multiple weight values, and using a Sigmoid activation function to constrain each weight value to the range [0, 1] to obtain the channel weights of each channel extraction map.

[0068] In one embodiment of the present application, the channel extraction map of each channel is multiplied by the corresponding channel weight to obtain a feature extraction map. Channels with large weights are enhanced, and channels with small weights are suppressed. This feature extraction map will be used for subsequent classification tasks.

[0069] In step S230, the feature extraction maps corresponding to each image to be identified are classified according to the preset identification area to obtain multiple classification results and corresponding classification confidences, and the feature extraction map with the classification result that the image contains the preset identification area and the corresponding classification confidence is high is determined as the target feature map.

[0070] In one embodiment of the present application, the feature extraction map corresponding to the image to be identified is classified according to the preset recognition area, including: performing global average pooling on the feature extraction map corresponding to the image to be identified to obtain a feature vector to be classified, and the feature vector to be classified includes the global features of each channel in the feature extraction map; classifying the feature vector to be classified according to the preset recognition area to obtain a first classification probability and a second classification probability, the first classification probability is used to characterize the probability that the image contains the preset recognition area, and the second classification probability is used to characterize the probability that the image does not contain the preset recognition area; performing probability distribution conversion on the first classification probability and the second classification probability to obtain two distribution probabilities; determining the category corresponding to the largest distribution probability as the classification result, and determining the largest distribution probability as the classification confidence.

[0071] In one embodiment of the present application, a global average pooling operation is performed again on the feature extraction image to obtain the global features of each channel, forming a fixed-length feature vector to be classified.

[0072] In one embodiment of the present application, the preset recognition area includes a large screen of an in-vehicle infotainment system (IVI).

[0073] In one embodiment of the present application, the feature vector to be classified is input into the fully connected layer, and two numerical values are output, respectively representing the initial probabilities that the feature extraction graph belongs to "including IVI large screen" and "not including IVI large screen".

[0074] In one embodiment of the present application, the two initial probabilities are converted into probability distributions through a Softmax function, and the category corresponding to the maximum probability value is the final classification result of the feature extraction graph.

[0075] In one embodiment of the present application, a binary classification result is output, and the category with a higher probability is selected as the final classification result to determine whether the image contains an IVI large screen and the corresponding classification confidence.

[0076] In one embodiment of the present application, the output results of each frame of image are traversed, and the image frame with the highest classification confidence is selected as the target feature map.

[0077] Step S240: Perform character recognition on the target feature image to obtain vehicle identification information.

[0078] In one embodiment of the present application, character recognition is performed on a target feature map to obtain vehicle identification information, including: super-resolution reconstruction of the target feature map to obtain a high-resolution feature map; identifying character features in the high-resolution feature map to obtain a character recognition result, the character recognition result including the recognized characters, and / or the position and content of the text block; regular expression matching of the character recognition result with a preset format to obtain format identification information, logically checking the parameter values of the format identification information to obtain vehicle identification information, so as to match the vehicle's identity information.

[0079] In one embodiment of the present application, the target feature map is input into a super-resolution model, such as ESRGAN, to obtain a high-resolution output image, and the reconstructed high-resolution feature map is saved.

[0080] In one embodiment of the present application, character feature recognition is performed through optical character recognition (OCR).

[0081] In one embodiment of the present application, the preset formats include time format, battery level format, gear level format, mileage format, tire pressure format, and temperature format. The time format is represented by "HH:mm," the battery level format is represented by "XX%," the gear level format is represented by "P, N, D, or R," the mileage format is represented by "XXX km," the tire pressure format is represented by "XX bar," and the temperature format is represented by "XX°C."

[0082] In one embodiment of the present application, a matching check of a preset regular expression is performed to ensure that the extracted parameters conform to the expected format. A logical check of the parameter values is performed to ensure that the extracted reference is logically reasonable, such as the battery level does not exceed 100% and the temperature is within a reasonable range.

[0083] In one embodiment of the present application, matching vehicle identification information includes: combining various parameters in the vehicle identification information as query conditions to construct a query request. Invoking a cloud query interface, sending the query request, and obtaining query results. Outputting the final matched Vehicle Identification Number (VIN) can improve matching accuracy for similar vehicle models.

[0084] In one embodiment of the present application, see Figure 3, Figure 3 FIG. 1 shows a flow chart of feature extraction for an image to be identified according to an embodiment of the present application. Figure 3 As shown in the figure, four multi-scale residual blocks are used to extract features of the image to be identified. The dotted line includes the composition of a multi-scale residual block. The multi-scale residual block includes multiple preset convolutional layers, a maximum pooling layer, and a jump connection layer for adding the convolution input and the convolution output to obtain the residual image. The jump connection layer includes a convolution layer for aligning the spatial size of the convolution input and the convolution output, and its convolution kernel size is 1x1; the convolution kernel sizes of the preset convolution layers are 1x1, 3x3, and 5x5 respectively; the initial input image of the first multi-scale residual block is the image to be identified, and the initial input image of the next multi-scale residual block is 1x1. The input image is the output image of the previous multi-scale residual block, that is, the pooled output image after maximum pooling; the number of channels corresponding to the four multi-scale residual blocks are 64, 128, 256, and 512 respectively; the maximum pooling layer will scale the spatial size of the residual image by half; feature extraction also includes assigning different channel weights to each channel through the channel attention mechanism to further enhance task-related features; the image output by the channel attention mechanism is average pooled to perform a binary classification task, and the output includes or does not include the preset recognition area, as well as the corresponding classification confidence. This application uses multiple multi-scale residual blocks connected in series, and multiple preset convolutional layers and maximum pooling layers in each multi-scale residual block to extract features from the surface to the deep layer and from fine particles to coarse particles to obtain key image features, and assigns weights to different features through the channel attention mechanism, which can enhance channels with large weights and suppress channels with small weights, thereby improving the classification accuracy of the preset recognition area.

[0085] In one embodiment of this application, super-resolution reconstruction and heuristic algorithms are used to analyze character recognition results. For example, regular expression matching is used to check whether the format is correctly recognized, and parameter values are logically verified. This allows for more accurate identification and extraction of vehicle parameters, enabling a more precise match to the cloud-based vehicle VIN. This application can also extract detailed vehicle identification information, including time, battery level, gear position, total mileage, tire pressure, and temperature, providing more comprehensive vehicle data.

[0086] See also Figure 4 , Figure 4 A block diagram of a vehicle information recognition device according to an embodiment of the present application is shown. The device can be applied to Figure 1 The implementation environment shown in FIG. 1 is specifically configured in the computer device 101. The apparatus may also be applicable to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the apparatus is applicable.

[0087] like Figure 4As shown, a vehicle information recognition device 400 according to an embodiment of the present application includes: an image acquisition module 401 , a feature extraction module 402 , an image classification module 403 and an information recognition module 404 .

[0088] The image acquisition module 401 is used to acquire multiple images of the vehicle to be identified;

[0089] A feature extraction module 402 is configured to extract features of the image to be identified based on the multi-scale residual blocks to obtain a feature extraction map of the image to be identified, wherein the number of multi-scale residual blocks includes multiple, and the pooled output image output by one multi-scale residual block is the initial input image of the next multi-scale residual block. The feature extraction includes performing multi-level convolution on the initial input image according to multiple preset convolution layers in each multi-scale residual block, and performing maximum pooling on the residual image to obtain a pooled output image;

[0090] The image classification module 403 is configured to classify the feature extraction maps corresponding to each image to be identified according to the preset identification area, obtain multiple classification results and corresponding classification confidences, and determine the feature extraction map with a classification result indicating that the image contains the preset identification area and has a high corresponding classification confidence as the target feature map;

[0091] The information recognition module 404 is used to perform character recognition on the target feature image to obtain vehicle identification information;

[0092] Among them, the residual image is obtained based on the initial input image and the convolution output image of the last preset convolutional layer, and each preset convolutional layer is connected in series and the convolution kernel size increases successively.

[0093] It should be noted that the vehicle information identification device provided in the above embodiment and the vehicle information identification method provided in the above embodiment are based on the same concept. The specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the vehicle information identification device provided in the above embodiment can, as needed, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0094] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by one or more processors, enables the electronic device to implement the vehicle information identification method provided in the above-mentioned embodiments.

[0095] See also Figure 5 , Figure 5The following is a schematic diagram showing the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application. Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0096] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 into the random access memory (RAM) 503, such as executing the method in the above embodiment. Various programs and data required for system operation are also stored in the RAM 503. The CPU 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0097] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. Removable media 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0098] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from a removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the various functions defined in the system of the present application are executed.

[0099] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0101] The units involved in the embodiments described in the present application can be implemented by software or by hardware, and the units described can also be set in a processor. The names of these units do not constitute a limitation on the units themselves under certain circumstances. Therefore, the technical solution according to the embodiment of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present application.

[0102] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When executed by a computer processor, the computer program causes the computer to perform the vehicle information identification method provided in each of the above embodiments. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device.

[0103] In the above embodiments, unless otherwise specified, the use of serial numbers such as "first" and "second" to describe common objects only indicates that they refer to different instances of the same object, rather than indicating that the objects being described must adopt a given order, whether in time, space, sorting or any other way.

[0104] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, any equivalent modifications or alterations accomplished by a person of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A vehicle information recognition method, characterized in that: The method comprises: Acquire multiple images of vehicles to be identified; Performing feature extraction on the image to be identified according to a multi-scale residual block to obtain a feature extraction map of the image to be identified, wherein the number of the multi-scale residual blocks includes a plurality, a pooled output image output by a multi-scale residual block is an initial input image of a next multi-scale residual block, and the feature extraction includes performing multi-level convolution on the initial input image according to a plurality of preset convolution layers in each multi-scale residual block, and performing maximum pooling on the residual image to obtain a pooled output image; Classifying the feature extraction maps corresponding to the images to be identified according to the preset identification regions, respectively, to obtain a plurality of classification results and corresponding classification confidences, and determining the feature extraction map whose classification result indicates that the image contains the preset identification region and whose corresponding classification confidence is high as the target feature map; Performing character recognition on the target feature image to obtain vehicle identification information; The residual image is obtained based on the initial input image and the convolution output image of the last preset convolution layer, and the preset convolution layers are connected in series and the convolution kernel sizes increase sequentially.

2. The vehicle information recognition method according to claim 1, characterized in that: The feature extraction further includes: Assigning different channel weights to multiple channel extraction maps, where the channel extraction map is used to represent an image corresponding to a channel in the pooled output image of the last multi-scale residual block; Weighting is performed according to each of the channel weights and the corresponding channel extraction map to obtain a feature extraction map.

3. The vehicle information recognition method according to claim 2, characterized in that: Assign different channel weights to multiple channel extraction maps, including: Performing global average pooling on each of the channel extraction images to obtain an initial feature vector, wherein the initial feature vector includes a spatial feature scalar corresponding to each channel extraction image; Performing channel number compression on the initial eigenvector according to a preset first weight matrix to obtain an intermediate eigenvector; The channel number of the intermediate feature vector is restored based on a preset second weight matrix to obtain the channel weight of each channel extraction map.

4. The vehicle information recognition method according to claim 1, characterized in that: The residual image is obtained based on the initial input image and the convolution output image of the last preset convolution layer, wherein the preset convolution layers are connected in series and the convolution kernel sizes increase sequentially, further comprising: Convolving the initial input image to obtain a skip image, so as to adjust the number of channels of the initial input image to be consistent with the number of channels of the convolution output image of the last preset convolutional layer; The skip image and the convolution output image of the last convolution layer are added to obtain the residual image.

5. The vehicle information recognition method according to any one of claims 1 to 4, characterized in that: The number of the preset convolutional layers includes three, the first preset convolutional layer is used to adjust the number of channels of the initial input image to be consistent with the number of channels output by the multi-scale residual block, the second preset convolutional layer is used to extract local features of the image, and the third preset convolutional layer is used to extract large-scale features of the image; The height and width of the convolution kernel size of the preset convolution layer are the same; The pooled output image is used to represent the residual image with half the spatial size; The number of channels output by the next multi-scale residual block is twice the number of channels output by the previous multi-scale residual block; The initial input image of the first multi-scale residual block is the image to be recognized.

6. The vehicle information recognition method according to any one of claims 1 to 4, characterized in that: Classifying the feature extraction graph corresponding to the image to be identified according to the preset identification area includes: Performing global average pooling on the feature extraction map corresponding to the image to be identified to obtain a feature vector to be classified, wherein the feature vector to be classified includes the global features of each channel in the feature extraction map; Classifying the feature vector to be classified according to the preset recognition area to obtain a first classification probability and a second classification probability, wherein the first classification probability is used to represent the probability that the image contains the preset recognition area, and the second classification probability is used to represent the probability that the image does not contain the preset recognition area; Performing probability distribution conversion on the first classification probability and the second classification probability to obtain two distribution probabilities; The category corresponding to the maximum distribution probability is determined as the classification result, and the maximum distribution probability is determined as the classification confidence.

7. The vehicle information recognition method according to any one of claims 1 to 4, characterized in that: Performing character recognition on the target feature image to obtain vehicle identification information includes: Performing super-resolution reconstruction on the target feature map to obtain a high-resolution feature map; Identifying character features in the high-resolution feature map to obtain a character recognition result, wherein the character recognition result includes the position and content of the recognized characters and / or text blocks; Perform regular expression matching on the character recognition result and the preset format to obtain format recognition information, The parameter values of the format identification information are logically checked to obtain vehicle identification information to match the identity information of the vehicle.

8. A vehicle information recognition device, characterized in that: The device comprises: An image acquisition module, used to acquire multiple images of the vehicle to be identified; a feature extraction module, configured to extract features of the image to be identified based on a multi-scale residual block to obtain a feature extraction map of the image to be identified, wherein the number of the multi-scale residual blocks includes a plurality, and a pooled output image output by a multi-scale residual block is an initial input image of a next multi-scale residual block, and the feature extraction includes performing multi-level convolution on the initial input image according to a plurality of preset convolution layers in each multi-scale residual block, and performing maximum pooling on the residual image to obtain a pooled output image; An image classification module is configured to classify the feature extraction maps corresponding to the images to be identified according to the preset recognition areas, obtain multiple classification results and corresponding classification confidences, and determine the feature extraction map with the classification result that the image contains the preset recognition area and has a high corresponding classification confidence as the target feature map; An information recognition module is used to perform character recognition on the target feature image to obtain vehicle identification information; The residual image is obtained based on the initial input image and the convolution output image of the last preset convolution layer, and the preset convolution layers are connected in series and the convolution kernel sizes increase sequentially.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the vehicle information recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the vehicle information recognition method according to any one of claims 1 to 7.

Images