Real-time mileage stake mark identification method and system

Abstract

The invention discloses a real-time recognition method and system for a mileage stake mark, relates to the technical field of road operation and maintenance, and solves the technical problem that a mileage stake mark sign in an image cannot be accurately recognized in the prior art. The method comprises the steps that road foreground image data is preprocessed and correspondingly labeled to obtain a target detection data set and a text recognition data set, and the target detection data set is adopted to train a target detection algorithm model to obtain a target detection model; training a text recognition algorithm model by using the text recognition data set to obtain a text recognition model, extracting a mileage pile image from the real-time road foreground image by using the target detection model, and obtaining a mileage pile number from the mileage pile image by using the text recognition model; according to the method, the position of the mileage stake mark can be quickly and accurately identified in a complex road scene, the performance is still excellent even under the conditions of insufficient illumination, stain shielding and the like, and the real-time performance of the system is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of road operation and maintenance, and in particular to a real-time identification method and system for mileage pile numbers. Background Art

[0002] Expressways are an important part of my country's transportation infrastructure. The automated identification technology of road pavement defects has become an important means of evaluating the performance of expressways. However, simply identifying pavement defects is not enough to achieve road pavement inspection. This is because pavement maintenance personnel also need to know the mileage post number location where the pavement defect is located. At present, most road inspection vehicles are equipped with global positioning navigation systems to record the route trajectory in real time. However, the use of global positioning navigation systems also requires the following considerations: (1) The global positioning system needs to be coordinated with the camera, which takes additional computing time; (2) The signal on mountainous or tunnel highways is weak, and it is impossible to ensure the real-time update of longitude and latitude information; (3) The longitude and latitude coordinates provided by the global positioning navigation system are not enough to accurately and intuitively provide the mileage location information of pavement defects. Mileage post number plates are arranged on both sides of the expressway and are natural positioning references. Even in mountainous areas or tunnels, mileage post number plates can provide intuitive and accurate positioning references for pavement maintenance personnel. In addition, mileage post number plates can also be used as an auxiliary means of the global positioning system to obtain more reliable and accurate mileage locations of pavement defects. Therefore, accurate identification of mileage pile number positions and accurate extraction of mileage pile number texts are crucial to locating pavement defects.

[0003] However, within the scope of the literature surveyed by the research team, most road inspection vehicles use the method of manually recording the mileage pile number position during the collection process. This manual recording method is time-consuming and labor-intensive and requires the recorder to stay focused at all times. The existing mileage pile number recognition algorithms based on deep learning generally have the disadvantages of low recognition accuracy, slow recognition speed, and inability to cope with various emergencies that occur in the actual environment. Therefore, there is an urgent need for an automated mileage pile number extraction technology to replace the time-consuming and labor-intensive manual operation, and to be able to cope with various situations that occur during actual application (such as missing piles, unclear pile numbers), so as to improve the accuracy and applicability of recognition. Based on this, the present invention aims to provide an automated and intelligent real-time mileage pile number information extraction method to provide reliable mileage information for the location of pavement defects.

[0004] Using a road foreground image taken by a lightweight 2D camera, the present invention uses a target detection algorithm and a text recognition algorithm to automatically extract the location and text information of the mileage pile number from the image. Since the mileage pile number plate often appears as a small target in the image, the target detection algorithm is required to accurately identify the small target object from the complex road traffic background. In addition, the mileage pile number may not be clear enough in the case of insufficient light, smudge obstruction or long distance, so it is also necessary to combine multiple frames of images for joint analysis and inference to improve the reliability of recognition. The size and text settings of the mileage pile numbers on different highways are also different, which is specifically reflected in the different fonts, font sizes, and layout angles on the mileage pile numbers. Therefore, the text recognition algorithm also needs to have a strong generalization ability to process mileage pile numbers in different states and accurately extract the mileage pile number text sequence.

[0005] Existing target detection algorithms have poor robustness in detecting small targets, resulting in the inability to accurately identify mileage post number signs in images. Summary of the invention

[0006] In order to solve the above problems in the prior art, the present invention provides a real-time recognition method and system for mileage post numbers, which solves the technical problem that the prior art cannot accurately recognize mileage post number signs in images.

[0007] A real-time recognition method for mileage pile numbers comprises: preprocessing and correspondingly labeling road foreground image data respectively to obtain a target detection data set and a text recognition data set, using the target detection data set to train a target detection algorithm model to obtain a target detection model, then using the text recognition data set to train a text recognition algorithm model to obtain a text recognition model, then using the target detection model to extract a mileage pile image from a real-time road foreground image, and then using the text recognition model to obtain the mileage pile number from the mileage pile image; the target detection algorithm model comprises two parallel branches, one branch extracts image features, and the other branch performs position encoding on the image features in a position encoding manner.

[0008] Furthermore, the extraction of image features includes: preprocessing road foreground images of consistent size, then extracting features using a residual network, and then dynamically position encoding the extracted feature layer, and adding a learnable position weight coefficient to each position on the basis of the original position encoding, so that it can automatically learn the relationship between relative positions during the back propagation process.

[0009] Furthermore, the position encoding of image features includes: using an enhanced feature encoding network with a gating mechanism to further capture effective features, and then using a feature decoding network to decode the enhanced feature layer obtained in the enhanced feature encoding network to obtain useful mileage pile number feature position information and category information, and finally using a regression prediction head and a classification prediction head to further output the feature layer obtained in the feature decoding stage as available coordinates and category information.

[0010] Furthermore, the method of further capturing effective features using an enhanced feature encoding network with a gating mechanism includes:

[0011] The feature layer with position encoding obtained in the feature extraction stage is first passed into the proposed gating mechanism, whose main function is to calculate a gating coefficient for each feature channel as a neuron parameter; when a channel has a positive effect on the final stake number positioning, it will pass the set gating mechanism, otherwise it will be filtered out;

[0012] The feature layer after the gating mechanism will be further passed into the multi-head self-attention mechanism to capture the long-distance dependency between feature information;

[0013] After the Add operation and layer normalization Normalization, it further enters the fully connected layer to obtain the effective feature layer after retrieval, and finally passes it to the feature decoding network through Add and Normalization operations to further realize the recovery of the feature layer.

[0014] Furthermore, the training to obtain the target detection model includes: training the target detection algorithm model based on the Hungarian algorithm, wherein the Hungarian algorithm matches multiple prediction results with multiple real boxes, and this process requires calculating the cost matrix, and then using the Hungarian algorithm to calculate the lowest cost according to the cost matrix, wherein the calculation process of the cost matrix is:

[0015] (a) Calculate the classification cost; (b) Calculate the L1 cost between the predicted box and the true box; (c) Calculate the IOU cost between the predicted box and the true box; Add the three calculation results according to the weights to obtain the cost matrix.

[0016] Furthermore, the text recognition algorithm model includes: using EfficientNet to extract features to generate a pile number feature sequence which is input into a GRU recurrent neural network to predict the mileage pile number.

[0017] Furthermore, the EfficientNet includes: EfficientNet-B0 as the baseline network, the first stage is a 3×3 convolution layer with a stride of 2 for feature extraction; the second to seventh stages use an inverted residual block (MBConv), and the MBConv block contains a 1×1 convolution (for expanding the number of channels), a 3×3 depth-separable convolution, and another 1×1 convolution to compress the number of channels. These blocks also include residual connections and Squeeze-and-Excitat ion (SE) modules. The last stage contains a 1×1 convolution layer for further feature fusion.

[0018] Furthermore, after the text recognition model obtains the mileage post number from the mileage post image, it uses a joint inference mechanism to confirm the content of the fuzzy post number based on the previous and next post number information. The joint inference mechanism includes: comparing and confirming by combining the recognition information of the previous and next frames, and using the credibility threshold judgment to correct the uncertain post number prediction value. If the recognition credibility of a post number is lower than the threshold, it is reconfirmed by referring to the previous and next frame data to ensure that the final output result is more accurate. Furthermore, the joint inference mechanism also includes:

[0019] Multi-frame image fusion and joint inference: By using the multiple frames of images taken continuously by the inspection vehicle and fusing the multi-frame information, the current pile number can be inferred more accurately;

[0020] Up and down logic verification and dynamic programming decoding: The logical relationship between pile numbers is represented by constructing a state diagram and a transfer cost function. Each state represents a possible pile number prediction value. The nodes in the state diagram represent the possibility of the pile number, and the edges represent the logical transfer from one pile number to another. The transfer cost function is used to judge the rationality of going from one pile number to the next. The dynamic programming algorithm is used to find the path with the minimum cost in the state diagram to ensure that the pile number sequence conforms to the logical continuity and realistic distribution.

[0021] Time series consistency analysis: By smoothing the prediction results of multiple consecutive frames of pile numbers, the fluctuations of single-frame image recognition can be eliminated.

[0022] A real-time recognition system for mileage pile numbers adopts a real-time recognition method for mileage pile numbers, including a data acquisition and processing module, a target detection module, a text recognition module and an output module. The data acquisition module is used for real-time acquisition and detection of target image preprocessing, the target detection module includes a target detection model for determining a mileage pile image based on the preprocessed detection target image, and the text recognition module includes a text recognition model for determining a mileage pile number based on the mileage pile image.

[0023] The beneficial effects of the present invention include:

[0024] In terms of algorithm innovation, the present invention proposes a target detection algorithm and a text recognition algorithm, and combines dynamic programming decoding with a joint inference mechanism, which significantly improves the detection capabilities of small targets and fuzzy pile numbers and the adaptability to complex scenes. The target detection algorithm can quickly and accurately identify the location of mileage pile numbers in complex road scenes by optimizing the feature extraction and detection process. It still performs well even in conditions such as insufficient lighting and smudges, ensuring the real-time performance of the system. The text recognition algorithm does not need to label each character one by one, which greatly reduces the time and cost of data preparation, and ensures that the identified pile number sequence conforms to the actual logical distribution through a dynamic programming decoding mechanism, further enhancing the robustness and adaptability of the algorithm under different lighting and complex backgrounds.

[0025] Another major innovation of the present invention is the introduction of a joint inference mechanism, which combines the information of multiple frames to jointly infer and confirm the fuzzy pile number. In practical applications, the pile number often becomes unclear due to changes in lighting, occlusion by stains or long distances, and a single frame image may not be able to accurately identify it. The joint inference mechanism significantly improves the success rate and accuracy of recognition by analyzing the pile number information of the previous and next frames, using time series consistency and logical verification, so that the system can still provide high-quality pile number information in real time and stably in complex environments.

[0026] In terms of practical engineering applications, the mileage pile number automatic extraction algorithm of the present invention realizes real-time and stable mileage pile number information acquisition during the inspection process of road inspection vehicles, avoiding dependence on the global navigation positioning system (GNSS), especially in areas where GNSS signals are unstable such as tunnels or mountainous areas, and can still provide accurate pile number location information. This provides reliable support for the precise positioning of road surface diseases, allowing maintenance personnel to quickly and intuitively determine the location of the disease and take efficient maintenance measures. In addition, the lightweight 2D camera solution of the present invention, combined with an efficient algorithm design, provides a lightweight, low-cost automatic extraction method that is easy to deploy on ordinary inspection vehicles, thereby expanding the scope of application of the technology and improving the overall efficiency and quality of road maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of the integrated mileage post number real-time identification method involved in an embodiment of the present application.

[0028] Figure 2 Schematic diagram of the target detection algorithm model involved in the embodiment of the present application.

[0029] Figure 3 This is a schematic diagram of the multi-head attention mechanism involved in an embodiment of the present application.

[0030] Figure 4 Schematic diagram of the gating mechanism involved in the embodiment of the present application.

[0031] Figure 5 Schematic diagram of the text recognition algorithm model involved in the embodiment of the present application.

[0032] Figure 6 This is a real-time road foreground image involved in the embodiments of the present application.

[0033] Figure 7 This is the detection result of the real-time road foreground image involved in the embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0035] A real-time identification method for mileage pile numbers, such as Figure 1 As shown, it includes: preprocessing and correspondingly labeling the road foreground image data respectively to obtain the target detection data set and the text recognition data set, using the target detection data set to train the target detection algorithm model to obtain the target detection model, and then using the text recognition data set to train the text recognition algorithm model to obtain the text recognition model, and then using the target detection model to extract the mileage post image from the real-time road foreground image, and then using the text recognition model to obtain the mileage post number from the mileage post image; the target detection algorithm model adopts two parallel branches, one branch extracts the image features, and the other branch uses the position encoding method to position encode the image features.

[0036] Specifically, the process of acquiring road foreground image data is as follows:

[0037] The road foreground image is obtained by using the front industrial 2D camera installed on the road inspection vehicle. The camera has high resolution and high dynamic range, and can capture clear images under various lighting and weather conditions. In order to ensure the accuracy and reliability of the data, the camera is precisely calibrated to ensure the quality and accuracy of the image. The collected road foreground image data is used to build a mileage pile number database. This database not only contains the images of mileage pile numbers, but also records the corresponding geographic location information and timestamps, providing important basic information for subsequent data processing and analysis. Since different highways often have different pile numbers, it is necessary to obtain the pile number data of multiple different highways to ensure data diversity in order to build a sample database for training algorithm models. The specific roads for data collection include but are not limited to various national highway kilometers, provincial highways, ring highways, urban expressways, and other sections with pile numbers, all of which are within the scope of data collection.

[0038] In the data annotation stage, a semi-automatic data annotation method is used:

[0039] The location and related information of the mileage posts are manually identified and marked in the image for preliminary annotation, and then the initial deep learning model is trained using the annotated data. Once the deep learning model reaches a certain accuracy, the data is automatically annotated through the deep learning model to improve the annotation efficiency. At the same time, in order to ensure the quality of the final data, all the data annotated by the model will be manually reviewed. This process ensures the accuracy and consistency of the data set while ensuring the efficiency of database establishment, providing high-quality input data for the training of the deep learning model.

[0040] The specific implementation steps are as follows:

[0041] (1) Obtain road foreground images of national highways, provincial highways, ring roads, urban expressways, and other road sections with stake numbers, and build an original image data platform;

[0042] (2) In view of the diversity of mileage post signs on different highways, it is necessary to select typical image samples with mileage post signs to be identified from the constructed raw data platform and try to include all the image samples with mileage post signs to be identified;

[0043] (3) Based on the selected typical samples, manually label a batch of samples. Based on this batch of labeled samples, first train a deep learning target detection model with preliminary recognition capabilities;

[0044] (4) Using this model with preliminary recognition capabilities, the remaining data can be automatically labeled and then manually calibrated. The efficiency of such semi-automatic labeling will be much higher than that of full manual labeling.

[0045] In another embodiment, the extraction of image features includes: preprocessing road foreground images of consistent size, then extracting features using a residual network, and then dynamically position encoding the extracted feature layer, and adding a learnable position weight coefficient to each position on the basis of the original position encoding, so that it can automatically learn the relationship between relative positions during the back propagation process.

[0046] Specifically, it includes: target detection image preprocessing

[0047] Image preprocessing is a key step in the mileage pile target detection stage. First, the captured road image is denoised, contrast adjusted, and color balanced to improve the visibility of the mileage pile in the image. Then, Gaussian blur and edge enhancement techniques are used to further highlight the characteristics of the mileage pile, which is particularly important for subsequent target detection. In addition, considering the influence of different lighting conditions and weather changes, adaptive histogram equalization processing is introduced to ensure that the image maintains consistent quality in various environments. Finally, the road image data with pile numbers is scaled to obtain a fixed pixel size of 640×640 and passed into the algorithm model to improve the processing speed and meet the real-time requirements.

[0048] An efficient target detection algorithm model is proposed to identify and locate mileage pile numbers. The model structure is as follows: Figure 2 As shown in the figure, the model structure is specially designed to enhance the detection capability of small targets, especially for situations such as ambiguous pile numbers, insufficient lighting and occlusion.

[0049] Feature extraction and dynamic position encoding

[0050] In order to improve the detection capability of small targets and fuzzy pile numbers, especially for insufficient lighting or occlusion in complex road scenes, this part mainly includes two parallel branches. The first branch uses the ResNet50 residual network structure to extract image features, and the second branch uses the Transformer position encoding method to position encode image features. Because in the Transformer-based algorithm adopted by the present invention, not only feature extraction but also feature query is required. Therefore, it is necessary to add position information (Pos it ion ing Embedd ing) to all features so that the network has the ability to distinguish different areas. The specific implementation steps are as follows:

[0051] (1) Input color image data of size 640×640×3 (width×height×channels) and perform image preprocessing, specifically normalizing the image data to map all pixels between 0 and 1;

[0052] (2) Use the ResNet50 feature extraction backbone network for preliminary feature extraction to provide a refined feature layer for the next step of self-attention encoding;

[0053] (3) Different from the existing target detection algorithms based on deep learning, the present invention proposes to dynamically position encode the feature layer extracted by ResNet50, so that it has the ability to distinguish different feature areas and does not lose the corresponding position information in the subsequent target query and target recovery process. The specific details are that on the basis of the original position encoding, a learnable position weight coefficient is added to each position, so that it can automatically learn the relationship between relative positions during the back propagation process, which helps to enhance the attention to the position of small target objects.

[0054] In another embodiment, the further capturing of effective features using an enhanced feature encoding network with a gating mechanism includes:

[0055] The feature layer with position encoding obtained in the feature extraction stage is first passed into the proposed gating mechanism, such as Figure 4 As shown, its main function is to calculate a gating coefficient for each feature channel as a neuron parameter; when a channel has a positive effect on the final stake number positioning, it will pass the set gating mechanism, otherwise it will be filtered out;

[0056] The feature layer after the gating mechanism will be further passed into the multi-head self-attention mechanism, such as Figure 3 As shown, it is used to capture the long-distance dependencies between feature information;

[0057] After the Add operation and layer normalization Normalization, it further enters the fully connected layer to obtain the effective feature layer after retrieval, and finally passes it to the feature decoding network through Add and Normalization operations to further realize the recovery of the feature layer.

[0058] The feature decoding network is used to decode the enhanced feature layer obtained in the enhanced feature encoding network to obtain useful pile feature position information and category information to achieve end-to-end target detection.

[0059] (1) Target query: Use a learnable query vector q to query the enhanced effective feature layer and obtain the prediction result;

[0060] (2) After passing through the multi-head self-attention mechanism structure, the refined feature layer is taken as Q, and the feature matrix extracted by the enhanced feature encoding network is taken as K, V, and then passed into the multi-head self-attention mechanism for another feature enhancement and feature retrieval.

[0061] The result prediction head (which includes the regression prediction head and the classification prediction head) is used to further output the feature layer obtained in the feature decoding stage as usable coordinate and category information.

[0062] (1) Regression prediction head: includes full connection, ReLU activation function, full connection, ReLU activation function, full connection, Sigmoid activation function. The final output contains four coefficients, namely the coordinates of the center point of the prediction box, the width and height of the prediction box. 100 prediction boxes are used.

[0063] (2) Classification prediction head: A fully connected layer is used, and the final output contains two coefficients. Because only one target is identified, the output is the number of categories + 1.

[0064] In another embodiment, the training to obtain the target detection model includes: training the target detection algorithm model based on the Hungarian algorithm, wherein the Hungarian algorithm matches multiple prediction results with multiple real boxes, and this process requires calculating a cost matrix, and then using the Hungarian algorithm to calculate the lowest cost based on the cost matrix, wherein the calculation process of the cost matrix is:

[0065] (a) Calculate the classification cost; (b) Calculate the L1 cost between the predicted box and the true box; (c) Calculate the IOU cost between the predicted box and the true box; Add the three calculation results according to the weights to obtain the cost matrix.

[0066] Specifically, in the target detection algorithm, the matching process of positive samples during training is based on the Hungarian algorithm. The matching algorithm's job is to match 100 prediction results with N real boxes. One real box only matches one prediction result, and the other prediction results are fitted as background. Therefore, the matching algorithm's job is to find the N prediction results that are most suitable for predicting N real boxes. Therefore, it is necessary to calculate a cost matrix (Cost matrix) to represent the relationship between the 100 prediction results and the N real boxes. This cost matrix consists of three parts:

[0067] (a) Calculate the classification cost. Obtain the predicted value corresponding to the category of the real box in the prediction result. If the predicted value is larger, it means that the prediction of the predicted box is more accurate, and its cost is lower; (b) Calculate the L1 cost between the predicted box and the real box. Obtain the coordinates of the predicted box in the prediction result, and make an L1 distance between the coordinates of the predicted box and the coordinates of the real box. The more accurate the prediction, the lower its cost; (c) Calculate the IOU cost between the predicted box and the real box. Obtain the coordinates of the predicted box in the prediction result, and make an IOU distance between the coordinates of the predicted box and the coordinates of the real box. The more accurate the prediction, the lower its cost. The above three parts are added according to certain weights to obtain the cost matrix, and then the Hungarian algorithm is used to calculate the lowest cost based on the cost matrix.

[0068] In this implementation, the proposed target detection model was tested on 2,000 measured road images with mileage posts against other advanced target detection algorithms, including YOLOv4-m, YOLOv5-m, FCOS, DETR, Faster R-CNN, and SSD. The test environment was python 3.8, the deep learning platform was Tensorflow 2.0, and the GPU hardware configuration was NVIDIA RTX 3070. The test results are shown in Table 1.

[0069] Table 1 Performance of different target detection algorithms

[0070]

[0071] The evaluation indexes used the three most representative indicators in the field of target detection intelligent algorithms, namely F1-Score and mAP. The larger the values ​​of these two indicators, the better the performance of the algorithm model. The indicators of all algorithms are measured at a confidence level of 0.5. Compared with the current mainstream target detection algorithms: YOLOv4-m, YOLOv5-m, FCOS, DETR, Faster R-CNN, SSD, the algorithm proposed in this invention has obvious advantages in the recognition of small targets at mileage pile numbers.

[0072] In another embodiment, the text recognition algorithm model is an efficient text recognition algorithm for extracting text information from the detected mileage post sign image. Figure 5 As shown. It includes: using EfficientNet to extract features to generate a pile number feature sequence and input it into the GRU recurrent neural network to predict the mileage pile number.

[0073] In the mileage post number text recognition stage, the cropped post number images are resized to 128×64 to be fed into the text recognition model.

[0074] GRU is a recurrent neural network (RNN) for sequence data processing, which aims to solve the gradient vanishing problem faced by traditional RNN when processing long sequence data. Through its special structural design, GRU can more effectively capture long-term dependencies in sequence modeling.

[0075] (1) Update gate: determines to what extent information needs to be passed from the past to the future. This helps the model decide how much past information to retain in the current state.

[0076] (2) Reset gate: determines how much past information needs to be forgotten. This helps the model decide how much previous information should be discarded when calculating candidates for the current state.

[0077] (3) Candidate hidden states: Use the reset gate to decide how much past information should be forgotten. Combine the current input and the past hidden state (adjusted by the reset gate) to generate the current candidate hidden state.

[0078] The present invention uses GRU to predict the pile number sequence, mainly taking advantage of its effective processing of the gradient vanishing problem. In addition, the GRU structure is simpler and has fewer parameters, so it is faster to train and more effective and faster in practical applications.

[0079] In another embodiment, the EfficientNet includes: EfficientNet-B0 as a baseline network, the first stage is a 3×3 convolution layer with a stride of 2 for feature extraction; the second to seventh stages use an inverted residual block (MBConv), and the MBConv block contains a 1×1 convolution (for expanding the number of channels), a 3×3 depth-separable convolution, and another 1×1 convolution to compress the number of channels. These blocks also include residual connections and Squeeze-and-Excitat ion (SE) modules. The last stage contains a 1×1 convolution layer for further feature fusion.

[0080] In another embodiment, after the text recognition model obtains the mileage post number from the mileage post image, it uses a joint inference mechanism to confirm the content of the ambiguous post number based on the previous and next post number information. The joint inference mechanism includes: comparing and confirming by combining the recognition information of the previous and next frames, and using the credibility threshold judgment to correct the uncertain post number prediction value. If the recognition credibility of a certain post number is lower than the threshold, it is reconfirmed by referring to the previous and next frame data to ensure that the final output result is more accurate.

[0081] The joint inference mechanism of pile numbers aims to improve the accuracy and robustness of the system's recognition of pile numbers in complex environments, especially in the face of ambiguous pile numbers, insufficient lighting, stains and occlusions, to ensure the rationality and stability of the recognition results. The mechanism combines methods such as dynamic programming decoding, joint inference of multi-frame images, and time series consistency analysis to achieve the best decoding and inference of pile number sequences.

[0082] (1) Multi-frame image fusion and joint inference: In order to improve the recognition ability of fuzzy pile numbers, this mechanism uses multiple frames of images taken continuously by the detection vehicle. These multiple frames provide pile number information at different time points. Especially when the pile number is partially blocked or the lighting is uneven, the current pile number can be inferred more accurately by fusing multiple frames of information. This method effectively improves the detection success rate of fuzzy pile numbers by combining the recognition results of multiple pile numbers in the time and space dimensions.

[0083] (2) Uplink and downlink logic verification and dynamic programming decoding: The pile numbers on the highway are arranged in a fixed logical order, usually increasing or decreasing. In the dynamic programming decoding of the pile number feature, the logical relationship between the pile numbers is represented by constructing a state diagram and a transfer cost function.

[0084] State diagram construction: Each state represents a possible pile number prediction value. The nodes in the state diagram represent the possibility of the pile number, and the edges represent the logical transfer from one pile number to another.

[0085] Define the transfer cost: The transfer cost function is used to determine the rationality of moving from one pile to the next. For example, piles should be arranged in a fixed increasing or decreasing order, and a large transfer cost indicates an illogical jump or an incorrect order.

[0086] Dynamic programming to solve the optimal path: Use the dynamic programming algorithm to find the path with the lowest cost in the state diagram to ensure that the pile number sequence conforms to the logical continuity and realistic distribution. The decoding mechanism effectively filters out unreasonable pile number jumps or outliers and outputs the logical optimal decoding sequence.

[0087] (3) Time series consistency analysis: Time series consistency analysis is also introduced in the joint inference mechanism. By smoothing the pile number prediction results of multiple consecutive frames, the fluctuation of single-frame image recognition is eliminated. Specifically, the pile number recognition results of consecutive frames are fused through the time-weighted average method to obtain a more stable and consistent pile number sequence. This time series smoothing can reduce the recognition errors caused by single-frame image quality problems.

[0088] (4) Secondary confirmation of fuzzy pile numbers: In some cases, the pile number recognition result of a single frame may be uncertain. To this end, the joint inference mechanism introduces secondary confirmation of fuzzy pile numbers. By comparing and confirming the recognition information of the previous and next frames, the uncertain pile number prediction value is corrected using the credibility threshold judgment. If the recognition credibility of a pile number is lower than the threshold, it is secondary confirmed by referring to the previous and next frame data to ensure that the final output result is more accurate.

[0089] Through the steps of multi-frame image fusion, uplink and downlink logic verification, dynamic programming decoding, time series consistency and secondary confirmation, the final output pile number recognition result not only logically conforms to the arrangement rules of highway pile numbers, but also ensures the stability and accuracy of prediction in spatial and temporal dimensions. This comprehensive mechanism greatly improves the recognition accuracy of the system in complex environments, especially when the pile number is ambiguous or information is missing, and can effectively perform joint inference and correction.

[0090] In this embodiment, the text recognition algorithm proposed in the present invention is used and tested on 2000 test images together with the currently advanced text recognition algorithms EasyOCR and Padd leOCR. The performance indicators of the test are shown in Table 2 below.

[0091] Table 2 Performance of different text recognition algorithms

[0092]

[0093]

[0094] The accuracy evaluation index is the accuracy rate of all the pile number sequences of the current predicted pile number image being predicted correctly. Compared with the current mainstream EasyOCR and Padd leOCR, the text recognition algorithm proposed in the present invention is more robust in extracting mileage pile number sequences.

[0095] Specifically, the detection result of using a real-time recognition method for mileage pile numbers involved in this embodiment to detect real-time foreground roads is as follows: Figure 6-7 shown.

[0096] A real-time recognition system for mileage post numbers includes a data acquisition and processing module, a target detection module, a text recognition module and an output module. The data acquisition module is used for real-time acquisition and detection of target image preprocessing. The target detection module includes a target detection model for determining a mileage post image based on the preprocessed detection target image. The text recognition module includes a text recognition model for determining a mileage post number based on the mileage post image.

[0097] The above-mentioned embodiments only express the specific implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A real-time identification method for mileage pile numbers, characterized in that: include: The road foreground image data are preprocessed and labeled accordingly to obtain a target detection data set and a text recognition data set, the target detection data set is used to train the target detection algorithm model to obtain the target detection model, and then the text recognition data set is used to train the text recognition algorithm model to obtain the text recognition model, and then the target detection model is used to extract the mileage post image from the real-time road foreground image, and then the text recognition model is used to obtain the mileage post number from the mileage post image; the target detection algorithm model adopts two parallel branches, one branch extracts image features, and the other branch uses position encoding to position encode the image features.

2. A real-time identification method for mileage post numbers according to claim 1, characterized in that: The extraction of image features includes: preprocessing road foreground images of consistent size, then extracting features using a residual network, and then dynamically position encoding the extracted feature layer. On the basis of the original position encoding, a learnable position weight coefficient is added to each position so that it can automatically learn the relationship between relative positions during the back propagation process.

3. A real-time identification method for mileage post numbers according to claim 1, characterized in that: The position encoding of image features includes: using an enhanced feature encoding network with a gating mechanism to further capture effective features, and then using a feature decoding network to decode the enhanced feature layer obtained in the enhanced feature encoding network to obtain useful mileage pile number feature position information and category information, and finally using a regression prediction head and a classification prediction head to further output the feature layer obtained in the feature decoding stage as available coordinates and category information.

4. A real-time identification method for mileage post numbers according to claim 3, characterized in that: The method of further capturing effective features using an enhanced feature encoding network with a gating mechanism includes: First, the extracted image features are passed into the proposed gating mechanism, and a gating coefficient is calculated for each feature channel as a neuron parameter; when a channel has a positive effect on the final stake number positioning, it will pass the set gating mechanism, otherwise it will be filtered out; The feature layer after the gating mechanism will be further passed into the multi-head self-attention mechanism to capture the long-distance dependency between feature information; After the Add operation and layer normalization, it further enters the fully connected layer to obtain the effective feature layer after retrieval, and finally passes it to the feature decoding network through the Add and Normalization operations to further realize the recovery of the feature layer.

5. A real-time identification method for mileage post numbers according to claim 1, characterized in that: The training to obtain the target detection model includes: training the target detection algorithm model based on the Hungarian algorithm, wherein the Hungarian algorithm matches multiple prediction results with multiple real frames, and this process requires calculating the cost matrix, and then using the Hungarian algorithm to calculate the lowest cost according to the cost matrix, wherein the calculation process of the cost matrix is: (a) Calculate the classification cost; (b) Calculate the L1 cost between the predicted box and the true box; (c) Calculate the IOU cost between the predicted box and the true box; Add the three calculation results according to the weights to obtain the cost matrix.

6. A real-time identification method for mileage post numbers according to claim 1, characterized in that: The text recognition algorithm model includes: using EfficientNet to extract features to generate a pile number feature sequence and inputting it into the GRU recurrent neural network to predict the mileage pile number.

7. A real-time identification method for mileage post numbers according to claim 6, characterized in that: The EfficientNet includes: EfficientNet-B0 as the baseline network, the first stage is a 3×3 convolution layer with a stride of 2 for feature extraction; the second to seventh stages use the inverted residual block MBConv, the MBConv block contains a 1×1 convolution, a 3×3 depth-separable convolution and another 1×1 convolution to compress the number of channels, and also includes residual connections and SE modules, and finally contains a 1×1 convolution layer for further feature fusion.

8. A real-time identification method for mileage post numbers according to claim 1, characterized in that: After the text recognition model obtains the mileage post number from the mileage post image, it uses a joint inference mechanism to confirm the content of the fuzzy post number based on the previous and next post number information. The joint inference mechanism includes: comparing and confirming by combining the recognition information of the previous and next frames, and using the credibility threshold judgment to correct the uncertain post number prediction value. If the recognition credibility of a certain post number is lower than the threshold, it is reconfirmed by referring to the previous and next frame data to ensure that the final output result is more accurate.

9. A real-time identification method for mileage post numbers according to claim 8, characterized in that: The joint inference mechanism also includes: Multi-frame image fusion and joint inference: By using the multiple frames of images taken continuously by the inspection vehicle and fusing the multi-frame information, the current pile number can be inferred more accurately; Up and down logic verification and dynamic programming decoding: The logical relationship between pile numbers is represented by constructing a state diagram and a transfer cost function. Each state represents a possible pile number prediction value. The nodes in the state diagram represent the possibility of the pile number, and the edges represent the logical transfer from one pile number to another. The transfer cost function is used to judge the rationality of going from one pile number to the next. The dynamic programming algorithm is used to find the path with the minimum cost in the state diagram to ensure that the pile number sequence conforms to the logical continuity and realistic distribution. Time series consistency analysis: By smoothing the prediction results of multiple consecutive frames of pile numbers, the fluctuations of single-frame image recognition can be eliminated.

10. A real-time identification system for mileage pile numbers, characterized in that: A real-time recognition method for mileage post numbers according to any one of claims 1 to 9 is adopted, comprising a data acquisition and processing module, a target detection module, a text recognition module and an output module, wherein the data acquisition module is used for real-time acquisition and detection of target image preprocessing, the target detection module comprises a target detection model for determining a mileage post image based on the preprocessed detection target image, and the text recognition module comprises a text recognition model for determining a mileage post number based on the mileage post image.

Images