A container number identification method, device and computer equipment

By performing spatial transformation and feature extraction on the target area of ​​the container number, combined with the pre-trained model for serialization and encoding processing, the problem of low recognition accuracy caused by inaccurate character segmentation in the prior art is solved, and high accuracy is achieved box number identification.

CN110942057BActive Publication Date: 2024-12-06SHAANXI WANKAI INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201811113365.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2018-09-25
Publication Date
2024-12-06
Estimated Expiration
2038-09-25

AI Technical Summary

Technical Problem

The existing container box number identification method has inaccurate character segmentation in the case of light difference, defilement, tilt, etc., resulting in low recognition accuracy.

Method used

By locating the container box number from the image to be identified, performing spatial transformation, extracting feature maps, and inputting them to the pre-trained container box number identification model for serialization and encoding processing, the decoding result is directly output to determine the box number. No character splitting is required.

Benefits of technology

It improves the accuracy of container box numbers and is widely applicable, so it can identify rotating, bent and tilted box numbers without adding manual labeling.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110942057B_ABST
    Figure CN110942057B_ABST
Patent Text Reader

Abstract

The present application provides a container number recognition method, device and computer equipment. The container number recognition method provided by the present application includes: locating the target area where the container number is located from the image to be recognized containing the container number, and performing spatial transformation on the target area to obtain the transformed target area; extracting features from the transformed target area to obtain a first feature map; inputting the first feature map into a pre-trained container number recognition model, serializing the first feature map by the container number recognition model to obtain a feature sequence, encoding the feature sequence to obtain an encoding result, and decoding the encoding result and outputting a decoding result; determining the container number in the image to be recognized according to the decoding result. The container number recognition method provided by the present application can accurately identify the container number from the image to be recognized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a container number recognition method, device and computer equipment. Background Art

[0002] In gate operations, each container is usually assigned a container number to identify each container. In recent years, in order to reduce manual transcription errors and reduce labor costs, automatic identification technology is often used to identify container numbers.

[0003] The related technology discloses a container number recognition method, which includes: locating a target area where the container number is located from an image to be recognized; performing character segmentation on the target area; respectively recognizing multiple characters obtained by segmentation to obtain multiple recognition results; and combining the multiple recognition results to obtain the container number.

[0004] When the above method is used to identify the container number, it is necessary to perform character segmentation on the target area where the container number is located. This method has strong dependency and may lead to inaccurate character segmentation in conditions of poor lighting, stains, large inclinations, etc., and thus low recognition accuracy due to inaccurate character segmentation. Summary of the invention

[0005] In view of this, the present application provides a container number recognition method, device and computer equipment to provide a container number recognition method with high recognition accuracy.

[0006] The first aspect of the present application provides a method for identifying a container number, the method comprising:

[0007] Locating a target area where the container number is located from the image to be identified containing the container number, and performing spatial transformation on the target area to obtain a transformed target area;

[0008] Performing feature extraction on the transformed target area to obtain a first feature map;

[0009] Inputting the first feature graph into a pre-trained container number recognition model, serializing the first feature graph by the container number recognition model to obtain a feature sequence, encoding the feature sequence to obtain an encoding result, and decoding the encoding result to output a decoding result;

[0010] The container number in the image to be identified is determined according to the decoding result.

[0011] A second aspect of the present application provides a container number recognition device, the device comprising a detection module, an recognition module and a processing module, wherein:

[0012] The detection module is used to locate the target area where the container number is located from the image to be identified containing the container number;

[0013] The recognition module is used to perform spatial transformation on the target area to obtain a transformed target area;

[0014] The recognition module is further used to extract features from the transformed target area to obtain a first feature map;

[0015] The recognition module is further used to input the first feature graph into a pre-trained container number recognition model, serialize the first feature graph by the container number recognition model to obtain a feature sequence, encode the feature sequence to obtain an encoding result, and decode the encoding result and output a decoding result;

[0016] The processing module is used to determine the container number in the image to be identified according to the decoding result.

[0017] The third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods provided in the first aspect of the present application.

[0018] The fourth aspect of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the methods provided in the first aspect of the present application are implemented.

[0019] The container number recognition method, device and computer equipment provided by the present application locate the target area where the container number is located from the image to be recognized containing the container number, and perform spatial transformation on the target area to obtain the transformed target area, and perform feature extraction on the transformed target area to obtain a first feature map, and then input the first feature map into a pre-trained container number recognition model, and the container number recognition model serializes the first feature map to obtain a feature sequence, and encodes the feature sequence to obtain an encoding result, and decodes the encoding result and outputs a decoding result, so as to determine the container number in the image to be recognized according to the decoding result. In this way, the container number can be recognized based on the target area without character segmentation, and the recognition accuracy is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a first embodiment of a method for identifying a container number provided by the present application;

[0021] Figure 2This is a schematic diagram of a container number shown in an exemplary embodiment of the present application;

[0022] Figure 3 This is a schematic diagram showing an implementation principle of serializing a first feature graph according to an exemplary embodiment of the present application;

[0023] Figure 4 A schematic diagram of an attention model shown as an exemplary embodiment of the present application;

[0024] Figure 5 A flowchart of a second embodiment of a container number identification method provided by the present application;

[0025] Figure 6 A schematic diagram of a detection network shown as an exemplary embodiment of the present application;

[0026] Figure 7 A flowchart of a third embodiment of a container number identification method provided by the present application;

[0027] Figure 8 This is a schematic diagram showing a method for identifying a container number according to an exemplary embodiment of the present application;

[0028] Fig. 9 This is a hardware structure diagram of a computing device where a container number recognition device is located, as shown in an exemplary embodiment of the present application;

[0029] Fig.10 This is a structural schematic diagram of embodiment 1 of the container number identification device provided in this application. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0032] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0033] The present application provides a container number recognition method, device and computer equipment to provide a container number recognition method with high recognition accuracy.

[0034] The container number recognition method and device provided in the present application can be applied to computer equipment. For example, it can be applied to an image acquisition device (the image acquisition device can be a camera), and for another example, it can also be applied to a server. This is not limited in the present application.

[0035] Several specific embodiments are given below to introduce the technical solution of the present application in detail. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0036] Figure 1 This is a flow chart of the first embodiment of the container number identification method provided by this application. Please refer to Figure 1 , the method provided in this embodiment may include:

[0037] S101, locating a target area where a container number is located from an image to be recognized that contains the container number, and performing spatial transformation on the target area to obtain a transformed target area.

[0038] It should be noted that the image to be identified is a captured frame image captured by the image acquisition device, which can be an image containing the container number captured by the image acquisition device in real time, or an image containing the container number stored by the image acquisition device.

[0039] Specifically, Figure 2 This is a schematic diagram of a container number shown in an exemplary embodiment of the present application. Figure 2 Container numbers can be distributed horizontally or vertically. Regardless of the distribution, the composition structure of container numbers can be expressed in the form of XYZ. Among them, X is the 4-digit main container number (all letters), Y is a 7-digit number, and Z is a 4-digit ISO number (it should be noted that the ISO number may not appear sometimes). It should be noted that the last digit in Y is the check code, which can be calculated from the 4 letters in X and the first 6 digits in Y.

[0040] Furthermore, a relevant target detection algorithm may be used to locate the target area where the container number is located from the image to be identified that contains the container number. This is not limited in this embodiment. For example, in one embodiment, the target area where the container number is located may be located from the image to be identified based on a YOLO (You Only Look Once, referred to as YOLO) model.

[0041] For another example, in another embodiment, the detection network may be used to locate the target area where the container number is located from the image to be identified that contains the container number.

[0042] For example, a deep convolutional neural network can be constructed for container number detection. The input of the deep convolutional neural network is set as an image, and the output is the coordinate position of the container number on the image. Quadrilaterals are used to represent the regional coordinates of each row or column. When the container number consists of multiple rows or columns, multiple quadrilateral coordinates are output. The quadrilateral can be tilted, indicating that the container number has a certain direction.

[0043] The constructed deep convolutional neural network can be a network modified based on the SSD network. For example, in one embodiment, the network may include a 29-layer fully convolutional network structure, of which the first 13 layers are inherited from the VGG-16 network (the last fully connected layer in the VGG16 network is converted into a convolutional layer), followed by a 10-layer fully convolutional network structure, and then a Text-box layer connected to 6 fully convolutional network structures. The text-box layer is a key component of the constructed network modified based on SSD. The 6 fully convolutional network structures in the 6 text-box layers are respectively connected to the 6 feature map positions in the previous network. For each feature map position, a text-box layer predicts an n-dimensional vector (the n-dimensional vector can include whether it is text in 2 dimensions, horizontal boundary rectangle in 4 dimensions, rotated boundary rectangle in 5 dimensions, and quadrilateral in 8 dimensions).

[0044] It should be noted that the detailed description of the detection network will be described in detail in the following embodiments and will not be repeated here.

[0045] Furthermore, the target region can be spatially transformed by using an STN network to obtain a transformed target region. In specific implementation, the target region is input into a pre-trained STN network (Spatial Transformers, STN for short), and the STN network performs spatial transformation on the target region and outputs the transformed target region.

[0046] Specifically, the STN network can perform spatial transformation on the target area without the calibration of key points (the spatial transformation includes but is not limited to translation, scaling, rotation, etc.). In this way, the recognition accuracy can be improved by using the transformed target area for recognition.

[0047] It should be noted that the specific structure and specific implementation principle of the STN network can be found in the description of the relevant technology, which will not be repeated here.

[0048] S102: extract features from the target area after the transformation to obtain a first feature map.

[0049] Specifically, a traditional method may be used to extract features from the transformed target region. For example, a scale-invariant feature transform (SIFT) algorithm may be used to extract features from the transformed target region. Of course, a neural network may also be used to extract features from the transformed target region. For example, in one embodiment, the specific implementation process of this step may include:

[0050] The transformed target area is input into a neural network for feature extraction, and a designated layer in the neural network performs feature extraction on the transformed target area; the designated layer includes a convolutional layer, or the designated layer includes a convolutional layer and at least one of a pooling layer and a fully connected layer; and an output result of the designated layer is determined as the first feature map.

[0051] Specifically, the neural network for feature extraction may include a convolution layer, which is used to filter the input transformed target area. Further, at this time, the filtering result output by the convolution layer is the first feature map extracted. In addition, the neural network for feature extraction may also include a pooling layer and / or a fully connected layer. For example, in one embodiment, the neural network for feature extraction includes a convolution layer, a pooling layer and a fully connected layer, wherein the convolution layer is used to filter the input transformed target area; the pooling layer is used to compress the filtering result; and the fully connected layer is used to aggregate the compressed result. Further, at this time, the aggregation result output by the fully connected layer is the first feature map extracted.

[0052] S103, inputting the first feature graph into a pre-trained container number recognition model, serializing the first feature graph by the container number recognition model to obtain a feature sequence, encoding the feature sequence to obtain an encoding result, and decoding the encoding result to output a decoding result.

[0053] Specifically, Figure 3 This is a schematic diagram showing the implementation principle of serializing the first feature map according to an exemplary embodiment of the present application. Figure 3 , the process of serializing the first feature map may include:

[0054] (1) sliding a preset sliding window on the first feature map according to a preset moving step length to segment a local feature map at a location where the sliding window is located;

[0055] (2) All segmented local feature maps are determined as the above-mentioned feature sequence.

[0056] Specifically, in one embodiment, the container number recognition model may be an attention model, and the attention model may include a convolutional network, and step (1) may be implemented by the convolutional network.

[0057] In addition, the size of the preset sliding window is adapted to the above-mentioned first feature map. For example, when the dimension of the above-mentioned first feature map is A*B*C (where A and B are the height and width of the first feature map, respectively, and C is the number of channels contained in the first feature map). At this time, the size of the sliding window can be set to A*A. In addition, the preset moving step size is set according to actual needs. In this embodiment, no limitation is made to this. For example, in one embodiment, the preset moving step size is 2.

[0058] Further, refer to Figure 3 In a specific implementation, a preset sliding window can be placed at one end of the first feature map, and a local feature map at the location of the sliding window can be segmented. Then, the sliding window can be moved based on a preset moving step length, and a local feature map at the location of the moved sliding window can be segmented. In this way, this process is repeated until the sliding window moves to the other end of the first feature map. Finally, all the segmented local feature maps are determined as a feature sequence.

[0059] It should be noted that when the first feature map is segmented using the preset sliding window and the preset moving step size, if the remaining part cannot be covered by the sliding window, the first feature map can be filled. In addition, since the first feature map includes multiple channels, each segmented local feature map also includes multiple channels.

[0060] Further, Figure 4 This is a schematic diagram of an attention model shown in an exemplary embodiment of the present application. Figure 4 , the attention model may also include an input layer, a hidden layer and an output layer connected in sequence, wherein X(X1, X2, X3, X4..., X m Represents the feature sequence input to the input layer; α t,1 , α t,2 , α t,3 , α t,3 ...α t,m represents the weight parameter of each feature in the feature sequence at time t (the dimension of the weight parameter of each feature is the same as the dimension of the feature); Ct represents the encoding result at time t; S t-1 , S tRepresents the context-related hidden state at each moment (at the initial moment, the hidden state is 0); y t ,y t+1 Indicates the decoding results at each moment. In this example, the decoding results at each moment may include the confidence of each candidate character at that moment (the candidate character is the preset category in the output layer (the output layer is composed of a classifier), in this example, including 37 categories (0-9, AZ and an end character), and the recognized character at that moment (the recognized character is the candidate character with the highest confidence among the candidate characters).

[0061] Reference Figure 4 The following is a detailed introduction to the specific implementation process of encoding the feature sequence to obtain the encoding result, and decoding the encoding result and outputting the decoding result. The process may include:

[0062] (1) Calculate the weight parameter of each feature in the feature sequence at each moment.

[0063] Specifically, this step is implemented through the input layer. In addition, the weight parameter of each feature in the feature sequence at each moment can be calculated according to the first formula, and the above first formula is:

[0064]

[0065] Among them, α t,i is the weight parameter of the i-th feature in the feature sequence at time t;

[0066] X i is the i-th feature in the feature sequence;

[0067] s t-1 is the hidden layer state at time t-1;

[0068] is the activation function;

[0069] W and U are the model parameters of the attention model.

[0070] (2) Calculate the encoding result at each moment based on the weight parameter of each feature in the feature sequence at each moment and the feature sequence.

[0071] Specifically, this step is implemented by a hidden layer. In addition, the implementation process of this step may include: using the weight parameter of each feature in the feature sequence at each moment to perform weighted sum processing on the feature sequence, and determining the obtained weighted sum as the encoding result at that moment.

[0072] Referring to the previous introduction, this process can be expressed by the second formula, which is:

[0073]

[0074] Among them, Ct is the encoding result at time t.

[0075] (3) Based on the above feature sequence and the encoding results at each moment, calculate the context-related hidden layer state at each moment.

[0076] Specifically, this step is implemented by the hidden layer. In addition, the context-related hidden layer state at each moment can be calculated according to the third formula, which is:

[0077] s t =LSTM(s t-1 , C t ,y t-1 )

[0078] It should be noted that the hidden layer state at time t is related to the hidden layer state at time t-1, the decoding result at time t, and the decoding result output by the attention model at time t-1.

[0079] (4) Obtain the decoding result at each moment based on the context-related hidden layer state at each moment.

[0080] Specifically, this step is implemented by the output layer. In addition, the decoding result at each moment can be calculated by the fourth formula, which is:

[0081] y t =softmax(s t )

[0082] Specifically, the decoding result at each moment includes the confidence of each candidate character at that moment and the character recognized at that moment. It should be noted that the character recognized at that moment is the candidate character with the highest confidence among the candidate characters.

[0083] The method provided in this embodiment uses an attention model to identify the target area after spatial transformation. In this way, the container number can be identified based on the target area without character recognition, with a high accuracy rate.

[0084] S104: Determine the container number in the image to be identified according to the decoding result.

[0085] Specifically, in one embodiment, the characters recognized in the decoding result can be directly combined in sequence, and the combination result can be determined as the container number in the image to be recognized.

[0086] The method provided in this embodiment locates the target area where the container number is located from the image to be identified that contains the container number, and performs spatial transformation on the target area to obtain the transformed target area, and extracts features from the transformed target area to obtain a first feature map, and then inputs the first feature map into a pre-trained container number recognition model, and the container number recognition model serializes the first feature map to obtain a feature sequence, and encodes the feature sequence to obtain an encoding result, and decodes the encoding result and outputs a decoding result, thereby determining the container number in the image to be identified according to the decoding result. In this way, the container number can be identified based on the target area where the entire container number is located without character segmentation, and the recognition accuracy is high. In addition, the method provided in this application can identify rotated, bent, and tilted container numbers (that is, container numbers with relatively large deformations can be identified) without the need for additional manual annotation, and has a wide range of applicability.

[0087] Figure 5 This is a flow chart of the second embodiment of the container number identification method provided by this application. Please refer to Figure 5 The method provided in this embodiment is based on the above embodiment. The specific implementation process of step S101, locating the target area where the container number is located from the image to be identified containing the container number, may include:

[0088] S501. Input the above-mentioned image to be identified into a pre-trained detection network, and the above-mentioned detection network extracts multi-level features of the above-mentioned image to be identified to obtain a specified number of second feature maps, and performs classification and position regression on each of the above-mentioned second feature maps, and outputs classification results and position information of multiple candidate areas; wherein the dimensions of the above-mentioned specified number of second feature maps are different.

[0089] Specifically, the detection network can be implemented by a convolutional layer. In addition, the specified number is set according to actual needs. For example, in one embodiment, the specified number is 6.

[0090] Figure 6 This is a schematic diagram of a detection network shown in an exemplary embodiment of the present application. Figure 6 , the detection network can include a 29-layer fully convolutional network structure, in which the first 13 layers of the fully convolutional network structure are inherited from the VGG-16 network (the last fully connected layer in the VGG-16 network is converted into a convolutional layer), followed by a 10-layer fully convolutional network structure (such as Figure 6 Conv6 to Conv11_2 in the figure, where Conv8_2, Conv9_2, Conv10_2, and Conv11_2 are each preceded by a fully convolutional network structure. Figure 6It should be noted that, see Figure 6 ,This 23-layer fully convolutional network structure is used to perform multi-level feature extraction and obtain 6 second feature maps.

[0091] Further, continue to refer to Figure 6 , the 23-layer fully convolutional network structure is followed by 6 text-box layers, each of which can be implemented by a fully convolutional network structure. Figure 6 , each Text-box layer is connected to a previous full convolutional network structure, which is used to classify and regress the second feature map output by the full convolutional network structure, and output the classification results and position information of multiple candidate areas.

[0092] In addition, please continue to refer to Figure 6 ,The Text-box layer is connected to an NMS layer, which is used to perform non-maximum suppression processing on the above multiple candidate regions based on the classification results and location information of each candidate region, and obtain the target region where the container number is located.

[0093] It should be noted that, in one embodiment, the classification result can be a binary classification result of foreground and background. In addition, the position information of a candidate area can be represented by a 12-dimensional vector, which includes the coordinates of the four points of the quadrilateral containing the candidate area (8 dimensions), and the center point coordinates, width and height (4 dimensions) of the circumscribed horizontal rectangle of the quadrilateral. Of course, in one embodiment, the position information can also include the coordinates and width (5 dimensions) of the two diagonal points of the rotated boundary rectangle corresponding to the candidate area.

[0094] It should be noted that since the dimensions of the multiple second feature maps are different, that is, the receptive fields of the multiple second feature maps are different, the final target area is equivalent to that obtained by classification and position regression on feature maps with multiple different receptive fields, and has strong multi-scale detection capabilities.

[0095] The following is a brief introduction to the implementation principle of the Text-box layer for classification and position regression of the second feature map.

[0096] Specifically, the text-box layer consists of three parts: the candidate box layer, the classification layer, and the position regression layer. The candidate box layer is used to generate multiple candidate boxes of different sizes at the location of each pixel in the second feature map according to preset rules, and then provide the candidate boxes to the classification layer and the position regression layer for category judgment and position refinement.

[0097] Furthermore, the classification layer outputs the probability that each candidate box belongs to the foreground and the background. The position regression layer outputs the position information of each candidate box. It should be noted that the classification layer and the position regression layer are implemented using convolutional layers. In addition, in order to adapt to text detection, the convolutional layer can use a 3*5 convolution kernel.

[0098] For example, in one embodiment, the dimension of one of the second feature maps is 40*42*128, where 40*42 represents the height and width of the second feature map, and 128 represents the number of channels contained in the second feature map. The candidate frame layer is centered on each pixel point in the second feature map, and generates 20 candidate frames of different sizes at the location of the pixel point according to preset rules (for the specific implementation principle, please refer to the description in the relevant technology, which will not be repeated here).

[0099] In addition, the dimension of the convolution kernel in the classification layer is 40*3*5*128, where 40 represents the number of convolution kernels and 3*5 represents the size of the convolution kernel. The step size of the convolution kernel movement is 1. In this way, after the classification layer performs convolution processing on the second feature map, a first convolution processing result is obtained. The dimension of the first convolution processing result is 40*40*42. Furthermore, for the first target convolution processing result corresponding to each pixel point in the first convolution processing result (the first target convolution processing result includes 40 convolution values), the first convolution processing result represents the classification results of 20 candidate boxes corresponding to the pixel point (the classification result of each candidate box includes two classifications (two-dimensional) of foreground and background).

[0100] Furthermore, for another example, the dimension of the convolution kernel of the position regression layer is 240*3*5*128, where 240 represents the number of convolution kernels and 3*5 represents the size of the convolution kernel. The step size of the convolution kernel movement is 1. In this way, after the position regression layer performs convolution processing on the second feature map, a second convolution processing result is obtained. The dimension of the second convolution processing result is 240*40*42. For the second target convolution processing result corresponding to each pixel point in the second convolution processing result (the second target convolution processing result includes 240 convolution values), the second target convolution processing result represents the position information of 20 candidate boxes corresponding to the pixel point. Referring to the previous description, the position information of each candidate box includes 12 dimensions. For example, the first 12 convolution values ​​of these 240 convolution values ​​represent the position information of the first candidate box.

[0101] It should be noted that the candidate box mentioned above can be understood as a candidate area.

[0102] S502: Based on the classification results and location information of each candidate area, non-maximum suppression processing is performed on the above multiple candidate areas to obtain the target area where the container number is located.

[0103] Please continue to refer to Figure 6,exist Figure 6 In the example shown, this step can be implemented by detecting the NMS layer in the network.

[0104] It should be noted that the specific implementation principle and process of non-maximum suppression can be found in the description of the relevant technology and will not be repeated here.

[0105] The method provided in this embodiment is to input the image to be identified into a pre-trained detection network, and the detection network performs multi-level feature extraction on the image to be identified to obtain a specified number of second feature maps, and respectively perform classification and position regression on each of the second feature maps, output classification results and position information of multiple candidate areas, and based on the classification results and position information of each candidate area, perform non-maximum suppression processing on the multiple candidate areas to obtain the target area where the container number is located. In this way, since the dimensions of the multiple second feature maps are different, that is, the receptive fields of the multiple second feature maps are different, the target area finally obtained is equivalent to that obtained by performing classification and position regression on feature maps with multiple different receptive fields, and has a strong multi-scale detection capability. Therefore, the target area where the container number is located can be accurately located.

[0106] Optionally, in a possible implementation of the present application, the specific implementation process of step S101, locating the target area where the container number is located from the image to be identified containing the container number, may include:

[0107] (1) The image to be recognized is resized to obtain a plurality of target images of different sizes.

[0108] For example, in one embodiment, the image to be recognized may be interpolated or down-sampled to obtain the target images of different sizes.

[0109] (2) For each target image, the target image is input into a pre-trained detection network, and the detection network performs multi-level feature extraction on the target image to obtain a specified number of second feature maps, and performs classification and position regression on each of the second feature maps, and outputs classification results and position information of multiple candidate regions; wherein the dimensions of the specified number of second feature maps are different.

[0110] (3) Based on the classification results and position information of each candidate area, non-maximum suppression processing is performed on the multiple candidate areas to obtain the target area where the container number in the target image is located.

[0111] Specifically, the specific implementation process and implementation principle of steps (2) and (3) can be found in the description of the previous embodiment and will not be repeated here.

[0112] (4) Determine the target area where the container number in the image to be identified is located based on the target area where the container number in the target image is located.

[0113] Specifically, based on the target area where the container number in the target image is located, non-maximum suppression processing may be performed on the target areas where the container number in multiple target images is located to obtain the target area where the container number in the image to be identified is located.

[0114] The method provided in this embodiment adjusts the size of the image to be identified to obtain multiple target images of different sizes, and then locates the target area where the container is located in the image to be identified based on the target image. In this way, the accuracy of positioning can be further improved.

[0115] It should be noted that the networks used in this application are all pre-trained networks. The network training process may include:

[0116] (1) Build a network;

[0117] For example, for the detection network, when constructing the network, the input is set as an image, and the output is the location information of the area where the container number is located. For example, a quadrilateral is used to represent the area coordinates of each row or column. When the container number consists of multiple rows or columns, multiple quadrilateral coordinates are output. The quadrilateral can be tilted, indicating that the container number has a certain direction.

[0118] For another example, for the identification network, the input can be set as the area where the container number is located, and the output can be set as the container number string, which is represented by a line of XYZ, where X represents the 4-digit main container number, Y represents the 7-digit number, and Z represents the 4-digit ISO number.

[0119] (2) Obtain training samples;

[0120] For example, in this example, when training the detection network, the label information of the training sample is the location information of the area where the container number is located. It should be noted that a complete container number may consist of multiple rows or columns. At this time, each row or column should be marked with quadrilateral coordinates. The quadrilateral surrounds all the characters in the row or column, and it is not advisable to leave too much blank space. The quadrilateral can be tilted, indicating that the container number has a certain direction.

[0121] For another example, when training a container number recognition network, the label information of the training sample is the container number string. It should be noted that a complete container number may consist of multiple rows or columns, and the labeling results are uniformly written in the form of XYZ in one row, where X represents the 4-digit main container number, Y represents the 7-digit number, and Z represents the 4-digit ISO number. The last digit in Y is the check digit, which can be calculated from the 4 letters in X and the first 6 digits of Y, and verification should be given during labeling.

[0122] (3) Use the training set to train the above network to obtain a trained network.

[0123] Specifically, the network parameters in the network may be set to specified values ​​first, and then the network may be trained using the acquired training samples to obtain a trained network.

[0124] Specifically, the process can include two stages: forward propagation and backward propagation. Forward propagation is to input a training sample, perform forward propagation on the training sample to extract data features, and calculate the loss function. Backward propagation is to use the loss function to propagate backward from the last layer of the network in sequence, and use the gradient descent method to modify the network parameters of the network so that the loss function converges.

[0125] Figure 7 This is a flowchart of the third embodiment of the container number identification method provided by this application. Please refer to Figure 7 The method provided in this embodiment is based on the above embodiment. Step S104, the process of determining the container number in the above image to be identified according to the above decoding result, may include:

[0126] S701: Determine whether the above decoding result satisfies the specified verification rule.

[0127] Specifically, the specific implementation process of this step may include:

[0128] (1) Determine whether the composition structure of the above decoding result matches the composition structure of the container number.

[0129] Specifically, referring to the above description, the composition structure of the container number can be expressed in the form of XYZ. Among them, X is the 4-digit main container number (all letters), Y is the 7-digit number, and Z is the 4-digit ISO number (it should be noted that the ISO number may not appear sometimes). In this step, when the first 4 characters recognized in the decoding result are all letters, and the 5th to 11th characters recognized in the above decoding result are all numbers, it is determined that the composition structure of the decoding result matches the composition structure of the container number, otherwise it is determined that the composition structure of the decoding result does not match the composition structure of the container number.

[0130] (2) If not, determine that the above decoding result does not satisfy the above verification rule.

[0131] Specifically, when it is determined according to step (1) that the composition structure of the decoding result does not match the composition structure of the container number, at this time, it is determined that the decoding result does not meet the above verification rules. For example, in one embodiment, when there are numbers in the first 4 characters identified in the decoding result, it is determined that the composition result of the above decoding result does not match the composition structure of the container number, and then it is determined that the decoding result does not meet the above verification rules. For another example, when there are letters in the 5th to 11th characters identified in the decoding result, it is determined that the composition result of the above decoding result does not match the composition structure of the container number, and then it is determined that the decoding result does not meet the above verification rules.

[0132] (3) If yes, calculate the check value of the above decoding result according to the preset rules.

[0133] (4) Determine whether the above check value is equal to the check code identified in the above decoding result.

[0134] (5) If the check value is equal to the check code identified in the decoding result, it is determined that the decoding result satisfies the check rule; otherwise, it is determined that the decoding result does not satisfy the check rule.

[0135] Specifically, the check value of the decoding result can be calculated according to the following method: (1) according to the preset correspondence between letters and numbers, the first 4 characters recognized in the decoding result are converted into numbers to obtain the converted decoding result; (2) the check value of the above decoding result is calculated according to the following formula:

[0136]

[0137] S is the check value of the decoding result;

[0138] C n The nth character in the converted decoded result.

[0139] Specifically, when converting the first 4 characters identified in the decoding result into numbers, the conversion can be performed according to the preset correspondence between letters and numbers. For example, Table 1 shows the preset correspondence between letters and numbers in an exemplary embodiment of the present application:

[0140] Table 1 Correspondence between preset letters and numbers

[0141]

[0142]

[0143] In addition, referring to the previous description, the 11th digit in the container number is the check code, so the 11th character identified in the decoding result is the identified check code. In this step, it is determined whether the calculated check value is equal to the check code identified in the above decoding result, and then when the calculated check value is equal to the check code identified in the decoding result, it is determined that the decoding result meets the above check rule, otherwise it is determined that the decoding result does not meet the above check rule.

[0144] S702: If yes, determine a first combination result after sequentially combining the characters recognized in the above decoding result as the container number in the above image to be recognized.

[0145] S703. If not, correct the above decoding result to obtain a corrected decoding result, and determine a second combination result after sequentially combining the characters in the corrected decoding result as the container number in the above image to be identified; wherein the above corrected decoding result satisfies the above verification rule.

[0146] Specifically, in a possible implementation manner, the specific implementation process of this step may include:

[0147] (1) When the composition structure of the decoding result does not match the composition structure of the container number, execute step (2); when the composition structure of the decoding result matches the composition structure of the container number, execute step (5).

[0148] (2) Performing a first correction on the decoding result so that the composition structure of the decoding result after the first correction matches the composition structure of the container number.

[0149] Specifically, if there is a number among the first four characters recognized in the decoding result, in one embodiment, it can be determined whether the number is a misrecognized character recorded in a pre-established misrecognition table of different types of characters. If so, the number is replaced with a letter corresponding to the number recorded in the above-mentioned misrecognition table of different types of characters. Otherwise, the number is replaced with a letter with the highest confidence among the candidate characters at that moment. Of course, in another embodiment, the number can also be directly replaced with a letter with the highest confidence among the candidate characters at that moment.

[0150] Furthermore, if there is a letter in the 5th to 11th characters identified in the decoding result, in one embodiment, it can be determined whether the letter is a misrecognized character recorded in a pre-established misrecognition table of different types of characters. If so, the letter is replaced with a number corresponding to the letter recorded in the above-mentioned misrecognition table of different types of characters, otherwise the letter is replaced with a number with the highest confidence among the candidate characters at that moment. Of course, in one embodiment, the letter can also be directly replaced with a number with the highest confidence among the candidate characters at that moment.

[0151] For example, Table 2 of the present application is a pre-established misrecognition table of different types of characters shown in an exemplary embodiment. Referring to Table 2, during recognition, it is easy to misrecognize 0 as O, or misrecognize O as 0. For example, in one embodiment, when "0" exists in the first 4 bits identified by the decoding result, at this time, "0" is replaced with "0".

[0152] Table 2 Different types of character misrecognition table

[0153] 0 O 1 I 2 Z

[0154] (3) When the first corrected decoding result satisfies the above verification rule, a combination result of sequentially combining the characters in the first corrected decoding result is determined as the container number in the above image to be recognized.

[0155] (4) When the decoding result after the first correction does not satisfy the above verification rule, execute step (5).

[0156] (5) performing a second correction on the decoding result or the decoding result after the first correction to obtain a second corrected decoding result, and determining the combination result after sequentially combining the characters in the decoding result after the second correction as the container number in the image to be identified; wherein the decoding result after the second correction satisfies the above-mentioned verification rule.

[0157] In a possible implementation, the character with the lowest confidence among the characters in the decoding result or the first corrected decoding result may be modified to obtain the second corrected decoding result.

[0158] According to the verification rules described above, the target character corresponding to the character with the lowest confidence when the verification rules are met can be calculated, and then the character with the lowest confidence can be replaced with the target character to obtain the second corrected decoding result.

[0159] In addition, in another possible implementation, it is possible to determine whether there is a misrecognized character recorded in the misrecognition table of the same type of characters in the first 10 characters of the decoding result or the first corrected decoding result according to the pre-established misrecognition table of the same type of characters. If there is a misrecognized character recorded in the misrecognition table of the same type of characters in the 10 characters, the misrecognized character is replaced with the character corresponding to the misrecognized character to obtain the corrected decoding result, and further determine whether the corrected decoding result meets the verification rule. If it does, the combination result after the characters in the corrected decoding result are sequentially combined to determine the recognized container number. If it does not meet the requirements, the number with the lowest confidence among the characters in the decoding result or the first corrected decoding result is modified according to the aforementioned method to obtain the second corrected decoding result.

[0160] Furthermore, if there are at least two misrecognized characters recorded in the misrecognition table of the same type of characters among the 11 characters, at this time, any misrecognized character can be replaced with the character corresponding to the misrecognized character to obtain multiple corrected decoding results, and further determine whether there is a target decoding result that meets the verification rule among the multiple corrected decoding results. If there is, the target decoding result is determined as the second corrected decoding result. That is, the combined result after the characters in the target decoding result are combined in sequence is determined as the recognized container number. If not, at this time, any two misrecognized characters can be replaced with the characters corresponding to the misrecognized characters to obtain at least one corrected decoding result, and then determine whether there is a target decoding result that meets the verification rule among the corrected decoding results. If there is, the target decoding result is determined as the second corrected decoding result. That is, the combined result after the characters in the target decoding result are combined in sequence is determined as the container number in the image to be recognized. If not, the decoding result or the number with the lowest confidence among the characters in the first corrected decoding result is modified according to the aforementioned method to obtain the second corrected decoding result.

[0161] It should be noted that in this example, if there is no misrecognized character recorded in the misrecognition table of the same type of characters, then the decoding result or the number with the lowest confidence among the characters in the first corrected decoding result can be directly modified to obtain the second corrected decoding result.

[0162] For example, Table 3 is a pre-established table of misrecognized characters of the same type shown in an exemplary embodiment. Referring to Table 3, during recognition, "M" is easily misrecognized as "N". Therefore, if "M" is recognized in the decoding result, at this time, "M" can be modified to "N".

[0163] Table 3 Table of misrecognized characters of the same type

[0164] M N O D U J E F L I 6 8

[0165] The method provided in this embodiment determines whether the decoding result satisfies the specified verification rules. When the decoding result satisfies the specified verification rules, the first combination result after the characters recognized in the decoding result are sequentially combined is determined as the container number in the image to be identified. When the decoding result does not satisfy the specified verification rules, the decoding result is corrected to obtain a corrected decoding result, and the second combination result after the characters in the corrected decoding result are sequentially combined is determined as the container number in the image to be identified. The corrected decoding result satisfies the above verification rules. In this way, the accuracy rate can be further improved.

[0166] Figure 8 This is a schematic diagram showing the implementation principle of a container number recognition method according to an exemplary embodiment of the present application. Figure 8 ,exist Figure 8 In the example shown, the STN network, the network for feature extraction, and the container number recognition model are integrated into a recognition network. When the target area is input to the recognition network, the network can output a decoding result. In this way, the container number in the image to be recognized can be determined based on the decoding result.

[0167] Corresponding to the above-mentioned embodiment of the container number recognition method, the present application also provides an embodiment of a container number recognition device.

[0168] The embodiment of the container number identification device of the present application can be applied to a computer device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the computer device where it is located reading the corresponding computer program instructions in the memory into the memory and running it. From the hardware level, if Fig. 9 FIG. 1 is a hardware structure diagram of a computing device where a container number recognition device is located according to an exemplary embodiment of the present application. Fig. 9 In addition to the storage 910, processor 920, memory 930 and network interface 940 shown, the computer device where the device is located in the embodiment can also include other hardware according to the actual function of the container number recognition device, which will not be described in detail.

[0169] Fig.10 This is a structural diagram of the first embodiment of the container number recognition device provided by this application. Please refer to Fig.10 The container number recognition device provided in this embodiment may include a detection module 100, an identification module 200 and a processing module 300, wherein:

[0170] The detection module 100 is used to locate the target area where the container number is located from the image to be identified containing the container number;

[0171] The recognition module 200 is used to perform spatial transformation on the target area to obtain a transformed target area;

[0172] The recognition module 200 is further used to extract features from the transformed target area to obtain a first feature map;

[0173] The recognition module 200 is further used to input the first feature graph into a pre-trained container number recognition model, serialize the first feature graph by the container number recognition model to obtain a feature sequence, encode the feature sequence to obtain an encoding result, and decode the encoding result and output a decoding result;

[0174] The processing module is used to determine the container number in the image to be identified according to the decoding result.

[0175] The device of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0176] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of any one of the methods provided in the first aspect of the present application are implemented.

[0177] Specifically, computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (such as EPROM, EEPROM and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks.

[0178] Please continue to refer to Fig. 9 The present application also provides a computer device, including a memory 910, a processor 920, and a computer program stored in the memory 910 and executable on the processor 920, wherein the processor 920 implements the steps of any one of the methods provided in the first aspect of the present application when executing the program.

[0179] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A container number recognition method, characterized in that: The method comprises: Locating a target area where the container number is located from an image to be identified that contains the container number, and performing spatial transformation on the target area to obtain a transformed target area, wherein the spatial transformation includes: translation, scaling, and rotation; Performing feature extraction on the transformed target area to obtain a first feature map; The first feature map is input into a pre-trained container number recognition model, and a preset sliding window is slid on the first feature map according to a preset moving step length to segment a local feature map where the sliding window is located; all segmented local feature maps are determined as a feature sequence, and the feature sequence is encoded to obtain an encoding result, and the encoding result is decoded and then outputted; Determine whether the decoding result satisfies the specified verification rule; the determining whether the decoding result satisfies the specified verification rule includes: determining whether the composition structure of the decoding result matches the composition structure of the container number; if not, determining that the decoding result does not satisfy the verification rule; if matching, calculating the verification value of the decoding result according to a preset rule; determining whether the verification value is equal to the verification code identified in the decoding result; if the verification value is equal to the verification code identified in the decoding result, determining that the decoding result satisfies the verification rule, otherwise, determining that the decoding result does not satisfy the verification rule; If the specified verification rule is met, a first combination result after sequentially combining the characters identified in the decoding result is determined as the container number in the image to be identified; If the specified verification rule is not satisfied, the decoding result is corrected to obtain a corrected decoding result, wherein the corrected decoding result satisfies the verification rule; and a second combination result after sequentially combining the characters in the corrected decoding result is determined as the container number in the image to be identified; Wherein, locating the target area where the container number is located from the image to be identified containing the container number includes: using a detection network to output the coordinate position of the container number on the image to be identified, the coordinate position includes the area coordinates of each row or column of the container number represented by a quadrilateral, and the quadrilateral includes an inclined quadrilateral, indicating that the container number has a tilt angle.

2. The method according to claim 1, characterized in that The method of locating a target area where the container number is located from the image to be identified containing the container number includes: Input the image to be identified into a pre-trained detection network, and perform multi-level feature extraction on the image to be identified by the detection network to obtain a specified number of second feature maps, and perform classification and position regression on each of the second feature maps, and output classification results and position information of multiple candidate regions; wherein the dimensions of the specified number of second feature maps are different; Based on the classification results and location information of each candidate area, non-maximum suppression processing is performed on the multiple candidate areas to obtain the target area where the container number is located.

3. The method according to claim 1, characterized in that: The method of locating a target area where the container number is located from the image to be identified containing the container number includes: Resizing the image to be recognized to obtain a plurality of target images of different sizes; For each target image, the target image is input into a pre-trained detection network, and the detection network performs multi-level feature extraction on the target image to obtain a specified number of second feature maps, and performs classification and position regression on each of the second feature maps, and outputs classification results and position information of multiple candidate regions; wherein the dimensions of the specified number of second feature maps are different; Based on the classification results and position information of each candidate area, non-maximum suppression processing is performed on the multiple candidate areas to obtain the target area where the container number in the target image is located; The target area where the container number in the to-be-recognized image is located is determined according to the target area where the container number in the target image is located.

4. The method according to claim 1, characterized in that: The performing spatial transformation on the target area to obtain a transformed target area includes: The target region is input into a pre-trained STN network, and the STN network performs spatial transformation on the target region and then outputs the transformed target region.

5. The method according to claim 1, characterized in that The encoding process is performed on the feature sequence to obtain an encoding result, and the encoding result is decoded and then outputted as a decoding result, including: Calculating the weight parameter of each feature in the feature sequence at each moment; Calculating the encoding result at each moment according to the weight parameter of each feature in the feature sequence at each moment and the feature sequence; Calculating the context-related hidden layer state at each moment according to the feature sequence and the encoding result at each moment; According to the hidden layer state related to the context at each moment, the decoding result at each moment is obtained.

6. A container number recognition device, characterized in that: The device comprises a detection module, an identification module and a processing module, wherein: The detection module is used to locate the target area where the container number is located from the image to be identified containing the container number; The recognition module is used to perform spatial transformation on the target area to obtain a transformed target area, wherein the spatial transformation includes: translation, scaling, and rotation; The recognition module is further used to extract features from the transformed target area to obtain a first feature map; The recognition module is further used to input the first feature map into a pre-trained container number recognition model, slide a preset sliding window on the first feature map according to a preset moving step length to segment a local feature map where the sliding window is located; determine all the segmented local feature maps as a feature sequence, encode the feature sequence to obtain an encoding result, and decode the encoding result and output a decoding result; The processing module is used to determine whether the decoding result satisfies the specified verification rule; the determination of whether the decoding result satisfies the specified verification rule includes: determining whether the composition structure of the decoding result matches the composition structure of the container number; if not, determining that the decoding result does not satisfy the verification rule; if matching, calculating the verification value of the decoding result according to a preset rule; determining whether the verification value is equal to the verification code identified in the decoding result; if the verification value is equal to the verification code identified in the decoding result, determining that the decoding result satisfies the verification rule, otherwise, determining that the decoding result does not satisfy the verification rule; If the specified verification rule is met, a first combination result after sequentially combining the characters identified in the decoding result is determined as the container number in the image to be identified; If the specified verification rule is not satisfied, the decoding result is corrected to obtain a corrected decoding result, wherein the corrected decoding result satisfies the verification rule; and a second combination result after sequentially combining the characters in the corrected decoding result is determined as the container number in the image to be identified; Among them, the detection module is specifically used to: use the detection network to output the coordinate position of the container number on the image to be identified, and the coordinate position includes the regional coordinates of each row or column of the container number represented by a quadrilateral, and the quadrilateral includes an inclined quadrilateral, indicating that the container number has a tilt angle.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.