A deep learning-based container number correction method

Abstract

The application discloses a container number error correction method based on deep learning, and the method comprises the following steps: S1, a training decoder network and a detection network are input as a model based on a CRNN network according to a container checking rule; S2, a to-be-detected picture is input into the CRNN network, an identification result is obtained by decoding an output result through a decoder, and checking is performed according to the container number checking rule; the to-be-detected picture is input into the detection network if the to-be-detected picture does not conform to the checking rule, an error sequence of the to-be-detected picture is output, and the error sequence is corrected in characters by combining the checking rule and the category weight output by the decoder.

Description

Technical Field

[0001] This invention relates to the field of container number recognition technology, and in particular to a container number error correction method based on deep learning. Background Technology

[0002] While artificial intelligence technology is booming, it is also being actively applied in practice. For example, cargo port terminals are developing towards intelligence and automation. Among these technologies, container number recognition is the foundation and prerequisite for many automated container loading and unloading tasks, and the accuracy of this recognition process is crucial.

[0003] Unlike traditional OCR text recognition tasks, container numbers have strict and fixed verification rules, which presents both a challenge and an opportunity. The core objective and innovation of this solution lies in how to cleverly and efficiently utilize these verification rules to assist in container number recognition and thus improve accuracy.

[0004] In practical applications of container number recognition, the following phenomena frequently occur: image blurring due to camera shake or lens damage; and wear and tear on the container number characters due to age or exposure to wind and rain. All of these phenomena affect the accuracy of container number recognition due to missing local information. Currently, other solutions for container number recognition mostly do not consider using verification rules or only use them to positively verify the correctness of the sequence. If the verification rules are not met, the error location cannot be determined, thus lacking error correction capabilities.

[0005] The CRNN network model, proposed in 2015, has been widely used in industrial text recognition tasks due to its lightweight structure and high computational speed, such as license plate recognition, billboard recognition, and, as discussed in this paper, container number recognition. The CRNN network structure consists of convolutional network layers and recurrent network layers. The convolutional network is responsible for extracting image feature information, while the recurrent network is responsible for integrating the semantic information from the preceding and following layers. However, CRNN models used for text recognition tasks often employ CTC decoding, resulting in variable-length output results.

[0006] Therefore, it is necessary to redesign the decoder based on the general solution CRNN model to improve the accuracy of box number recognition tasks and solve the industry's pain points to some extent. Summary of the Invention

[0007] The purpose of this invention is to provide a container number error correction method based on deep learning, so as to at least solve one of the many defects of the prior art.

[0008] Therefore, the solution of the present invention is as follows:

[0009] A deep learning-based method for correcting container numbers includes the following steps:

[0010] S1. Based on the container inspection rules and using the CRNN network as input, train the decoder network and error detection network;

[0011] S2. Input the image to be inspected into the CRNN network. The output result is decoded by the decoder to obtain the recognition result, and then verified according to the bin number verification rules:

[0012] If the result meets the validation rules, output the result directly.

[0013] If the result does not meet the verification rules, the image to be checked is input into the error detection network, which outputs the error sequence of the image to be checked. The error sequence is then corrected by combining the verification rules with the category weights output by the decoder.

[0014] In this invention, the decoder model input dimensions include time step, batch size, and number of categories; the model output dimensions include box number length, batch size, and number of categories.

[0015] Preferably, the decoder uses a bidirectional LSTM network to extract semantic information before and after the time step, and integrates it into 11 bits corresponding to the bin number for 11 time steps.

[0016] Preferably, the number of categories in the model input includes numbers, letters, and empty characters; the number of categories in the model output dimension includes numbers and letters.

[0017] Preferably, the training set and validation set of the decoder are both 11-bit box number images, and the loss function is cross-entropy loss.

[0018] In this invention, the error correction network uses images of box numbers whose identification results do not conform to the verification rules as the input model; it uses images containing manually generated precisely blurred images and images whose CRNN model prediction results do not conform to or conform to the verification rules as the training set; and it uses images containing manually generated precisely blurred images and images whose CRNN model prediction results do not conform to or conform to the verification rules as the validation set.

[0019] Preferably, the loss function of the error correction network is cross-entropy loss.

[0020] In this invention, in step S2, the error detection network outputs the correct and incorrect sequences of the image to be inspected and counts the number of incorrect sequences. If the number of incorrect sequences is 1, the incorrect sequence is corrected by character.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0022] 1. Based on the general CRNN model, this invention designs a decoder and uses a neural network to accurately locate the error location. It then combines verification rules and class weights to design an algorithm for correction, thereby significantly improving the accuracy of box number recognition tasks and further solving the industry's pain points.

[0023] 2. This invention specifically designs an 11-bit decoder, using a bidirectional LSTM network to extract semantic information before and after each time step, integrating it into 11 bits corresponding to the bin number for each of the 11 time steps, and removing the category of null characters, thus avoiding bit mismatch and helping the error detection network to find the error location.

[0024] 3. The training set of the error detection network of the present invention consists of multiple parts. The diversity of the training set enables the network to obtain a more comprehensive and sensitive error detection capability. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is an overall flowchart of the container number correction method described in this invention.

[0027] Figure 2 This is a schematic diagram of the decoder network structure and calculation method described in this invention.

[0028] Figure 3 This is a schematic diagram of the error detection network structure and calculation method of the present invention.

[0029] Figure 4 This is a schematic diagram illustrating the error correction process using the container number error correction method in one embodiment of the present invention.

[0030] Figure 5 This is a comparison between the preliminary identification results and the correction results of the error correction process in this embodiment of the invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described in this specification are merely for explaining the invention and are not intended to limit the invention.

[0032] Existing technologies using CRNN models for text recognition often produce outputs of variable length when employing CTC decoding. To specifically address the 11-digit box number recognition task, this solution trains an 11-bit decoder network instead of the traditional CTC decoding. This avoids errors caused by bit mismatch and facilitates subsequent error detection and correction processes.

[0033] In practical applications of container number recognition, the following phenomena often occur: partial image blurring due to camera shake or lens damage; and wear and tear on container number characters due to age or weathering. All of these phenomena affect the accuracy of container number recognition due to missing local information. To simulate and reproduce these phenomena as closely as possible in practical applications of container number recognition, and to improve the generalization ability of the proposed network to specifically address these problems, the training / validation set of this proposed solution combines original erroneous images with manually blurred images.

[0034] In other text recognition scenarios, partial blurring lacks error correction capabilities. However, container numbers have well-defined verification rules, meaning there's a close relationship between preceding and following characters. Therefore, these verification rules can be used to detect and correct errors, filling in information gaps caused by blurring or incompleteness. The container number verification rules can only calculate 1-4 possible characters for the remaining digit based on the defined 10 digits. Considering the limitations of these verification rules, this solution can only correct errors in the case of a single incorrect digit, but it can still resolve 50%-60% of container number misdetection problems.

[0035] The verification rules are as follows: The container number consists of 11 digits. The first 4 digits are uppercase letters representing the company and container type, the 5th to 10th digits are numbers representing the unique identifier of the container, and the 11th digit is a number that is a check code calculated based on the first 10 digits.

[0036] The 36 categories, consisting of 10 digits and 26 letters, are uniformly coded as shown in Table 1.

[0037] Table 1:

[0038] A：10 B：12 C：13 D：14 E：15 F：16 G：17 H：18 I：19 J：20 K：21 L：23 M：24 N：25 O：26 P：27 Q：28 R：29 S：30 T：31 U：32 V：34 W：35 X：36 Y：37 Z：38

[0039] The method for calculating the 11th digit from the first 10 digits is: multiply the corresponding value of the i-th digit by 2. i-1 The sum modulo 11 is the value of the 11th digit. For example, consider the valid sequence CBHU3202732: (13 × 2) 0 +12×2 1 +18×2 2 +32×2 3 +3×2 4 +2×2 5 +0×2 6 +2×27 +7×2 8 +3×2 9 )%11=2.

[0040] The positive usage of the verification rule is as follows: After knowing the container number, calculate the value of the 11th digit based on the first 10 digits. If it is the same as the actual 11th digit, the verification is successful; otherwise, the verification is considered to have failed because the container number sequence is incorrect. However, the traditional solution does not have the ability to detect and correct errors.

[0041] like Figure 1 As shown, this invention proposes a container number error correction method based on deep learning, comprising the following steps:

[0042] Phase 1: Training the 11-bit decoder

[0043] 1. Model input: The output of CRNN (dimensions [51, 64, 37], where 51 is the time step obtained by convolution of a 200-width image, 64 is the batch size, and 37 is the number of categories including numbers, letters, and empty characters); the model output dimension is [11, 64, 36], where 11 is the box number specified length, 37 is changed to 36 because the empty character category was removed, and 64 has the same meaning as above.

[0044] Standard container numbers are strictly 11 digits. To address the potential for bit mismatches in traditional CRNN models, this solution designs an 11-bit decoder. A bidirectional LSTM network extracts semantic information from 51 time steps (obtained through convolution of a 200-width image), integrating it into 11 time steps corresponding to the 11 digits of the container number. The null character 0 is removed, ensuring the recognition result at each time step has practical meaning. Specifically, the CRNN output tensor is transposed to [37, 64, 51], then fed forward into the decoder to obtain [37, 64, 11], which is then transposed back and the 0 category removed. This ensures a strictly 11-bit output, avoiding bit mismatches and aiding the error detection network in locating errors. A schematic diagram of the 11-bit decoder network structure and calculation method is shown below. Figure 2 As shown.

[0045] 2. The training and validation sets consist entirely of 11-bit bin number images, and the loss function is cross-entropy loss. The decoder network is trained until loss convergence, and the globally optimal model is saved, achieving an accuracy of 96.59%.

[0046] Phase 2: Training the Error Detection Network

[0047] 1. The model input is an image whose recognition result does not meet the verification rules, and the model output is an 11-bit sequence similar to rrrrwrrrrrr (where w indicates that the 5th bit is incorrect).

[0048] 2. The training set consists of three parts: 1) manually generated, precisely blurred error images; 2) images whose CRNN model predictions do not conform to the validation rules; and 3) images whose CRNN model predictions conform to the validation rules. The labels are sequences of 11 'r' or 'w' characters, where 'r' indicates correct and 'w' indicates incorrect. For example, rrrrwrrrrrr indicates that the only error occurs at the 5th position.

[0049] 3. The validation set consists of three parts: 1) manually generated, precisely blurred images; 2) images whose CRNN model predictions do not conform to the validation rules; and 3) images whose CRNN model predictions conform to the validation rules. The labels are sequences of 11 r or w characters, where r represents correct and w represents incorrect.

[0050] 4. The loss function used is cross-entropy loss. The error detection network is trained until the loss convergence and the globally optimal model is saved. The accuracy of this network in identifying error locations reaches 94.67%.

[0051] Using verification rules in the forward direction only reveals the presence of errors in the sequence, but without knowing the location of the errors, correction is impossible. Therefore, the first and most crucial step in correction is to pinpoint the most likely potential error location. This network borrows the convolutional neural network part of CRNN for image feature extraction, and bidirectional LSTM has the function of extracting semantic relationships between preceding and following sequences, thus it can be used to compare features before and after the image to predict the error location. Specifically, this network performs binary classification for each time step of the image: 0 and 1, where 0 represents a normal region and 1 represents a potential error location. The loss function uses cross-entropy loss, and taking the class with the largest weight in the network output yields an 11-bit result, for example, rrrrwrrrrr indicates that the 5th bit is the error location. The training set of the error detection network consists of multiple parts, and the diversity of the training set enables the network to obtain a more comprehensive and sensitive error detection capability. A schematic diagram of the error detection network structure and calculation method is shown below. Figure 3 As shown.

[0052] Phase 3: Correcting Error Locations

[0053] 1. Input the image to be inspected into the CRNN network. The output result is decoded by the decoder obtained from the first stage training to obtain the recognition result, and then verified according to the box number verification rules. If it meets the verification rules, proceed to step 5; otherwise, proceed to step 2.

[0054] 2. Input the image to be inspected into the error detection network obtained from the second stage of training, output the 11-bit correct and incorrect sequence of the image to be inspected, and count the number of error positions.

[0055] 3. If the number of error locations is 0 or greater than 1, skip to step 6 without correction. Otherwise, skip to step 4.

[0056] 4. Combine the verification rules with the category weights output by the decoder to perform character correction at the error position i.

[0057] 5. Output recognition results that meet the verification rules (default is accurate recognition).

[0058] 6. The algorithm ends.

[0059] The forward verification rule ensures that the result of solving for the 11th bit based on the first 10 bits is unique. However, the reverse verification rule means that solving for a non-11th bit based on the other 10 bits may not yield a unique result, potentially resulting in 1-4 feasible solutions. The class weight at the corresponding time step in the decoder output represents the confidence level of classifying the character at that position into that class. The class with the highest confidence level is selected from the feasible solutions, thus achieving the goal of error correction. The specific steps are as follows:

[0060] 1) Given:

[0061] a) The model's predicted 11-bit result R;

[0062] b) Confidence P of all characters in each position;

[0063] c) The estimated error position w is only considered when w < 11;

[0064] d) The 11th check digit is V.

[0065] 2. Convert the initial recognition result R into the character's corresponding encoding value E according to encoding table 1.

[0066] 3. According to the encoding rules, the relationship between the first 10 bits of the code ei in E and the check code can be calculated. Then, using the check rule grid search, all feasible solutions should be satisfied, provided the following conditions are met:

[0067]

[0068] 4. Obtain the confidence C of all feasible solutions at the error location w through P, sort them from smallest to largest, and select the solution with the highest confidence as the repaired character.

[0069] 5. Return the repaired 11-digit recognition result.

[0070] In one embodiment of the present invention, the container number error correction method described above is used for error correction, and the process is as follows: Figure 4 As shown, after correction by the error correction network and the correction algorithm, the error position can be found on the basis of the preliminary recognition sequence obtained by the CNN+ decoder, and the character can be corrected by the correction algorithm.

[0071] In some other embodiments provided by the present invention, the improvements are as follows:

[0072] The validation set A contains 5714 images of container numbers that have been blurred by human intervention. The original CRNN model achieved an accuracy of 87.20%, while this solution achieved an accuracy of 93.54%, successfully correcting 362 images compared to the previous method.

[0073] The validation set B contains 216 images of container numbers that were incorrectly identified by the original CRNN model, meaning the accuracy of the original CRNN model was 0%. The accuracy of this solution is 59.26%, which is 128 images that were successfully corrected compared to the previous solution.

[0074] The validation set C contains 6334 images of 11-digit container numbers. The original CRNN model achieved an accuracy of 96.59%, while this solution achieved an accuracy of 98.61%, successfully correcting 128 images compared to the previous method.

[0075] The data above shows that this solution has a good correction effect on manually generated images with damaged or blurred container numbers, and also has a good correction effect on erroneous images in actual situations, but there is still room for optimization and improvement. Overall, the accuracy of the 11-digit container number recognition task has been significantly improved.

[0076] In several other embodiments of the present invention, error detection and correction are performed on images to be detected obtained in several different ways, such as... Figure 5 As shown, the sequence on the left below is the initial recognition result after CNN+ decoder, and the sequence on the right is the recognition sequence after correction. Figure 5 Taking A as an example: In the original image, motion blur was applied to the 7th character from left to right. The initial recognition result obtained by the CRNN+ decoder is BMOU6819949. Verification using validation rules reveals that this sequence does not meet the rules and requires error detection and correction. Inputting the image into the error detection network yields the correct sequence rrrrrrwrrrr, with a unique error position of 6 (starting from 0). The error position, the initial recognition result, and the category weights are input into the correction algorithm. Four characters at this position conform to the validation rules: 4, E, O, and Y. The character with the highest weight, 4, is taken as the correction result and replaced with the original character, resulting in a recognition result that conforms to the validation rules. Figure 5 B, 5C, 5D, 5E, 5F, and 5G are all examples of analysis methods and Figure 5 Example A is consistent. Where... Figure 5 Examples A and 5B show detection errors caused by manually adding blur, while examples 5C to 5G show detection errors in the original image without human intervention.

[0077] The present invention is not limited to the description in the specification and embodiments, and thus other advantages and modifications can be readily realized by those skilled in the art. Therefore, the present invention is not limited to the specific details, representative devices and illustrated examples shown and described herein without departing from the spirit and scope of the general concept as defined by the claims and their equivalents.

Claims

1. A container number error correction method based on deep learning, comprising the following steps: S1. Train the decoder network and error detection network by inputting the model with the container number verification rules and based on the CRNN network; S2. Input the image to be inspected into the CRNN network. The output result is decoded by the decoder to obtain the recognition result, and then verified according to the bin number verification rules: If the result meets the validation rules, output the result directly. If the result does not meet the validation rules, then execute the following sub-steps: S21, Input the image to be inspected into the error detection network, output the 11-bit correct and incorrect sequence of the image to be inspected, and count the number of error positions in the error sequence; S22, if the number of error positions in the error sequence is 1, then determine that the error position is the position to be corrected; otherwise, output the result directly. S23, Calculate all feasible solutions for the erroneous position according to the container number verification rules; S24, obtain the category weights corresponding to the error positions in the decoder output, and select the character with the largest category weight from all feasible solutions as the corrected character; S25: Replace the character at the corresponding error position in the original recognition result with the corrected character to obtain the corrected recognition result and output it.

2. The method for correcting container numbers according to claim 1, characterized in that, The decoder model input dimensions include time step, batch size, and number of categories; the model output dimensions include box number length, batch size, and number of categories.

3. The method for correcting container numbers according to claim 2, characterized in that, The decoder uses a bidirectional LSTM network to extract semantic information before and after the time step, and integrates it into 11 bits corresponding to the box number for 11 time steps.

4. The method for correcting container numbers according to claim 3, characterized in that, The number of categories in the model input includes numbers, letters, and empty characters; the number of categories in the model output dimension includes numbers and letters.

5. The method for correcting container numbers according to claim 2, characterized in that, The training and validation sets of the decoder are both 11-bit box number images, and the loss function is cross-entropy loss.

6. The method for correcting container numbers according to claim 1, characterized in that, The error detection network takes images of box numbers whose identification results do not conform to the verification rules as input models; and takes images containing erroneous images generated by precise blurring by humans and images whose CRNN model prediction results do not conform to or do not conform to the verification rules as training sets. The validation set consists of erroneous images generated by precise blurring by humans and images whose CRNN model predictions do not conform to or do not conform to the validation rules.

7. The method for correcting container numbers according to claim 5, characterized in that, The loss function of the error detection network is cross-entropy loss.

Images