Tire rubber material surface code spraying character recognition method and device

Abstract

The invention discloses a tire rubber surface code spraying character recognition method and device. The method comprises the steps that a target detection model is used for analyzing a first tire rubber material surface code spraying image, a target detection result is obtained, and the target detection result at least comprises prediction bounding box coordinate information and prediction inversion probabilities of a plurality of first characters in the first tire rubber material surface code spraying image; performing direction correction on each first character in the first tire rubber surface code spraying image according to the target detection result to obtain a second tire rubber surface code spraying image; analyzing the second tire rubber surface code spraying image by using a character recognition model to obtain a first prediction character string; and determining a target character string in the first tire rubber surface code spraying image according to the first prediction character string. The technical problem that the characters on the surface of the tire rubber material are difficult to accurately recognize by a manual visual inspection method is solved.

Description

Technical Field

[0001] This application relates to the field of tire manufacturing technology, and more specifically, to a method and apparatus for recognizing characters printed on the surface of tire rubber. Background Technology

[0002] In the modern tire manufacturing industry, with the advancement of production automation and refined quality control, the inspection of characters on the surface of tire rubber compounds has gradually become a key link in the production process. Characters on the surface of tire rubber compounds typically contain important information such as product specifications, production date, and batch number. Clear and accurate markings play an indispensable role in product traceability, quality control, and market circulation.

[0003] Currently, the detection of characters on the surface of tire rubber materials mainly relies on manual visual inspection. This method is not only inefficient and easily affected by human factors such as fatigue and subjective judgment differences, but also cannot meet the needs of large-scale, high-speed modern production.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method and apparatus for recognizing characters printed on the surface of tire rubber, so as to at least solve the technical problem that manual visual inspection methods are difficult to accurately identify characters on the surface of tire rubber.

[0006] According to one aspect of the embodiments of this application, a method for recognizing characters printed on the surface of tire rubber is provided, comprising: analyzing a first tire rubber surface printed image using a pre-trained target detection model to obtain a target detection result, wherein the target detection result includes at least: the predicted bounding box coordinate information and predicted inverted probability of each of a plurality of first characters in the first tire rubber surface printed image; performing orientation correction on each of the first characters in the first tire rubber surface printed image based on the target detection result to obtain a second tire rubber surface printed image; analyzing the second tire rubber surface printed image using a pre-trained character recognition model to obtain a first predicted string; and determining a target string in the first tire rubber surface printed image based on the first predicted string.

[0007] Optionally, the target detection model is trained using the following method: A first training sample set and a first sample label set are obtained, wherein the first training sample set includes multiple inkjet-printed images of the third tire rubber surface as first training samples, and the first sample label set includes the actual detection results of each first training sample as the corresponding first sample label, wherein the actual detection results include at least the actual bounding box coordinates of the second character within the inkjet-printed image of the third tire rubber surface and the actual inverted label; a first initial learning model is constructed, wherein the first initial learning model includes at least a target wavelet transform-based convolutional module. An improved shallow feature extraction network, an improved deep feature extraction network based on a target wavelet transform convolutional module, and an improved prediction network based on an asymmetric convolutional module are used. The target wavelet transform convolutional module is used to decompose the first training sample into multiple frequency sub-bands through a wavelet transform layer, and then use a depthwise separable convolutional layer to perform convolution operations on each frequency sub-band to obtain multiple initial feature maps. The multiple initial feature maps are then recombined using an attention mechanism to obtain the corresponding feature maps. The first initial learning model is iteratively trained using the first training sample set and the first sample label set to obtain a trained target detection model.

[0008] Optionally, the shallow feature extraction network includes at least: multiple feature fusion modules improved based on the target wavelet transform convolution module, and multiple downsampling layers; the deep feature extraction network includes at least: multiple upsampling layers, a feature pyramid module improved based on the target wavelet transform convolution module, and a path aggregation network module improved based on the target wavelet transform convolution module; the prediction network includes at least: multiple detection heads and an asymmetric convolution module at the front end of the target detection head among the multiple detection heads, wherein the asymmetric convolution module includes two cascaded one-dimensional convolutional layers, and the two one-dimensional convolutional layers are used to extract features in different directions respectively; the multiple detection heads are used to output the predicted inverted results, predicted bounding box coordinates and corresponding confidence scores of all second characters in each first training sample; and the target detection head is used to output the predicted inverted results of all second characters in each first training sample.

[0009] Optionally, the orientation of each first character in the first tire rubber surface inkjet printing image is corrected according to the target detection result to obtain the second tire rubber surface inkjet printing image, including: determining whether the predicted inverted probability of multiple first characters in the target detection result is lower than a preset threshold, and if the predicted inverted probability of the target character among the multiple first characters is not lower than the preset threshold, determining the central axis of the target character according to the predicted bounding box coordinate information of the target character in the target detection result, and rotating the target character half a circle along the central axis to obtain the orientation-corrected second tire rubber surface inkjet printing image.

[0010] Optionally, the character recognition model is trained using the following method: obtaining a second training sample set and a second sample label set, wherein the second training sample set includes: multiple inkjet-printed images of the surface of the third tire rubber material as second training samples, and the second sample label set includes: the real strings of each second training sample as the corresponding second sample label; constructing a second initial learning model, wherein the second initial learning model includes at least: a feature extraction module, a bidirectional long short-term memory network module, and a transcription module; and iteratively training the second initial learning model using the second training sample set and the second sample label set to obtain the character recognition model.

[0011] Optionally, the second initial learning model is iteratively trained using the second training sample set and the second sample label set to obtain a completed character recognition model. This includes: for each training batch during the iterative training process, inputting each second training sample of the training batch into the feature extraction module of the second initial learning model to obtain a feature sequence corresponding to each second training sample output by the feature extraction module, wherein the feature sequence includes multiple feature vectors obtained by feature extraction after segmenting the second training sample in the width direction; inputting each feature sequence into the bidirectional long short-term memory network module of the second initial learning model to obtain a fusion hidden state sequence corresponding to each feature sequence, wherein each fusion hidden state sequence includes multiple fusion hidden states obtained by feature fusion of each feature vector in the feature sequence and the corresponding forward hidden state and backward hidden state; inputting each fusion hidden state sequence into the transcription module of the second initial learning model to obtain a second predicted string corresponding to each fusion hidden state sequence; constructing a target loss function using each second predicted string and the corresponding sample label, and adjusting the model parameters of the second initial learning model according to the target loss function.

[0012] Optionally, determining the target string within the inkjet-printed image of the first tire rubber compound surface based on the first predicted string includes: verifying the first predicted string using a preset regular expression; if the verification result is successful, using the first predicted string as the target string within the inkjet-printed image of the first tire rubber compound surface; if the verification result is unsuccessful, replacing abnormal characters in the first predicted string using a preset inkjet-printed rule library for the tire rubber compound surface to obtain the target string within the inkjet-printed image of the first tire rubber compound surface, wherein the inkjet-printed rule library for the tire rubber compound surface includes at least: standard forms and corresponding error forms of various tire rubber compound surface characters, and connection order rules between various tire rubber compound surface characters.

[0013] According to another aspect of the embodiments of this application, a tire rubber surface inkjet printing character recognition device is also provided, comprising: a detection module, configured to analyze a first tire rubber surface inkjet printing image using a pre-trained target detection model to obtain a target detection result, wherein the target detection result includes at least: the predicted bounding box coordinate information and predicted inverted probability of each of a plurality of first characters in the first tire rubber surface inkjet printing image; an image processing module, configured to perform orientation correction on each of the first characters in the first tire rubber surface inkjet printing image based on the target detection result to obtain a second tire rubber surface inkjet printing image; a recognition module, configured to analyze the second tire rubber surface inkjet printing image using a pre-trained character recognition model to obtain a first predicted string; and a determination module, configured to determine a target string in the first tire rubber surface inkjet printing image based on the first predicted string.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-described method for recognizing inkjet characters on the surface of tire rubber by running the computer program.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described method for recognizing inkjet characters on the surface of tire rubber through the computer program.

[0016] In this embodiment, a method combining target detection and character recognition is used to recognize characters in the inkjet-printed image on the tire rubber surface. First, the target detection model analyzes the first inkjet-printed image on the tire rubber surface to obtain target detection results, which include at least: the predicted bounding box coordinates and predicted inverted probabilities of multiple first characters within the first inkjet-printed image. The predicted inverted probabilities provide a crucial basis for subsequent orientation correction. Then, the target detection results are used to perform orientation correction processing on the first characters in the first inkjet-printed image, effectively unifying the orientation of all characters in the first inkjet-printed image. Finally, the character recognition model is used to identify the strings contained in the corrected second inkjet-printed image on the tire rubber surface. This achieves high-precision detection and recognition of characters on the tire rubber surface in complex industrial environments, improving detection efficiency, accuracy, and adaptability. It ensures consistent detection performance under different production conditions, effectively improving quality control and production efficiency in the tire production process, and thus solving the technical problem that manual visual inspection methods are difficult to accurately identify characters on the tire rubber surface. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a flowchart illustrating an optional character recognition method for tire rubber surface according to an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the structure of an optional tire rubber surface character recognition device according to an embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the hardware structure of a computer terminal for implementing a method for recognizing characters on the surface of tire rubber, according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] To better understand the embodiments of this application, the following is a translation and explanation of some nouns or terms that appear in the description of the embodiments of this application:

[0024] Wavelet transform is a mathematical tool used to analyze the frequency components of data. It can not only analyze the frequency of data, but also provide information on time or spatial location. Therefore, it has unique advantages in processing non-stationary signals (signals whose statistical characteristics change over time). Its basic idea is to use wavelet functions (i.e., basis functions) of different scales to represent signals or images. In image processing, wavelet transform can identify features of different scales and directions, and is often used for tasks such as image compression, edge detection, texture analysis, image denoising, and feature extraction.

[0025] IoU (Intersection over Union): is an indicator that measures the degree of overlap between the predicted bounding box and the ground truth bounding box coordinates. It is calculated by using the ratio of the size of the overlapping area between the two to the size of their union area. The IoU value ranges from 0 to 1. The larger the value, the higher the degree of overlap between the predicted box and the ground truth box, and the more accurate the detection result.

[0026] LSTM (Long Short Term Memory) is a special type of Recurrent Neural Network (RNN) designed to overcome the gradient vanishing or exploding problems encountered by traditional RNNs as the number of network layers increases. It is particularly suitable for handling long-term dependencies in sequential data. LSTM introduces memory cells and three gating mechanisms (input gate, forget gate, and output gate) to allow the network to selectively remember or forget certain information, thereby transmitting effective state information in long-term sequences and achieving long-term memory of the sequence context. This makes LSTM perform well in various tasks such as natural language processing, speech recognition, and time series prediction.

[0027] CTC (Connectionist Temporal Classification) is a loss function and decoding algorithm for sequence prediction. CTC introduces a special "space" or "whitespace" category, which allows the model to output time steps even when there is no corresponding output character. It can be used to automatically handle the alignment problem between the input and output sequences when the lengths of the input and output sequences do not match.

[0028] Example 1

[0029] According to an embodiment of this application, a method for recognizing inkjet-printed characters on the surface of tire rubber is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] Figure 1 This is a flowchart illustrating a method for recognizing inkjet-printed characters on the surface of tire rubber materials according to an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0031] Step S102: Analyze the inkjet printing image on the surface of the first tire rubber using a pre-trained target detection model to obtain target detection results. The target detection results include at least the predicted bounding box coordinates and predicted inverted probabilities of each of the multiple first characters in the inkjet printing image on the surface of the first tire rubber.

[0032] Step S104: Based on the target detection results, the orientation of each first character in the inkjet printing image on the surface of the first tire rubber compound is corrected to obtain the inkjet printing image on the surface of the second tire rubber compound.

[0033] Step S106: Analyze the inkjet printing image on the surface of the second tire rubber material using a pre-trained character recognition model to obtain the first predicted string.

[0034] Step S108: Determine the target string within the inkjet printing image on the surface of the first tire rubber material based on the first predicted string.

[0035] Based on the scheme defined in steps S102 to S108 above, it can be understood that in this embodiment of the application, the method of combining the target detection model and the character recognition model is used to perform character recognition on the inkjet printing image on the tire rubber surface. This achieves the technical effect of high-precision detection and recognition of characters on the tire rubber surface in a complex industrial environment, thereby improving detection efficiency, accuracy and adaptability, ensuring consistent detection performance under different production conditions, effectively improving the quality control level and production efficiency in the tire production process, and solving the technical problem that manual visual inspection methods are difficult to accurately identify characters on the tire rubber surface.

[0036] The following describes each step of the method for recognizing characters printed on the surface of tire rubber compounds, using a specific implementation process as an example.

[0037] First, the identification system can acquire the inkjet-printed image of the first tire rubber compound surface to be analyzed. This inkjet-printed image is generated during the early stages of tire production, when the rubber compound is being formed into a tire. It is printed with product information on the surface of the rubber compound using inkjet printing technology. This information typically includes product specifications, production date, batch number, and other important parameters, and is an indispensable part of quality control and product tracking during the tire production process.

[0038] Generally, high-resolution industrial line scanners and industrial camera light sources (such as LED lighting to provide a uniform and bright lighting environment) can be pre-installed at the printing and coding station of the tire production line. The line scanner captures images of the tire rubber surface during continuous printing by scanning lines. Due to its characteristics, the line scanner can construct a complete image line by line while the tire is moving at high speed, ensuring clear image data even on a high-speed production line. Furthermore, during image acquisition, factors such as the tire's moving speed and the relative position of the camera to the tire surface can affect image stability. Therefore, image stabilization techniques, such as mechanical stabilization or software algorithm stabilization, can be employed during image acquisition to reduce image blurring.

[0039] It should be noted that the inkjet-printed images of the first tire rubber surface acquired using an industrial line scan camera may have problems such as uneven lighting, background texture interference, oil stains, and scratches. Therefore, image preprocessing operations such as image enhancement, noise reduction, normalization, image stretching, and compression can be performed on the acquired inkjet-printed images of the first tire rubber surface to improve image quality.

[0040] Furthermore, considering the random placement of rubber compound on the conveyor belt during the actual tire extrusion process, existing visual inspection solutions typically assume characters are upright or only support small-angle (±15°) tilt corrections. Once the tire is reversed (i.e., the characters are 180° upside down), the geometric topology of the characters (e.g., "6" and "9", "d" and "p") undergoes a fundamental visual reversal. In this case, traditional optical character recognition algorithms are prone to misidentification or missed detection when identifying inverted characters. Additionally, resolving inverted characters in the tire rubber compound surface inkjet printing image by traversing each character within the image and rotating the image would exponentially increase computation time, failing to meet the cycle time requirements of high-speed production lines.

[0041] Therefore, this application proposes that a pre-trained target detection model can be used to analyze the inkjet printing image on the surface of the first tire rubber compound to obtain the target detection result.

[0042] The aforementioned target detection model is a deep learning model trained on a large amount of image data of inkjet-printed characters on the surface of tire rubber. It can accurately detect the bounding box information of each character in the inkjet-printed image on the surface of tire rubber and whether it is upside down.

[0043] Alternatively, the above object detection model can be trained using the following steps:

[0044] Step 1: Obtain the first training sample set and the first sample label set.

[0045] The first training sample set includes: multiple inkjet-printed images of the surface of the third tire rubber as first training samples; the first sample label set includes: the actual detection results of each first training sample as the corresponding first sample label, and the actual detection results include at least: the actual bounding box coordinates of the second character in the inkjet-printed image of the third tire rubber surface and the actual inverted label.

[0046] Specifically, the first training sample set can be composed of a large number of inkjet-printed images of the third tire rubber surface collected by a high-resolution industrial line scan camera and an industrial camera light source during the actual production process of different tire models; while the first sample label can be manually annotated on these inkjet-printed images of the third tire rubber surface to obtain the true bounding box coordinate information of the second character in the inkjet-printed image of the third tire rubber surface and the true inverted label.

[0047] It is important to note that, in order to improve the model's generalization ability and robustness, enabling it to maintain high detection accuracy under different lighting conditions, viewpoints, scales, and various real-world variations, this application embodiment can also perform image enhancement on the aforementioned first training sample set, such as random scaling, rotation, cropping, and changes in brightness and contrast, to increase the diversity and richness of the dataset, simulate different shooting conditions and application scenarios, and make the training data more representative. This ensures that the model learns character features at different scales and improves the model's ability to handle size variations in real-world production environments.

[0048] Step 2: Construct the first initial learning model.

[0049] The model architecture of the first initial learning model can be selected from computer vision deep learning model architectures such as YOLO (You Only Look Once), SSD (Single Shot Multi Box Detector), RCNN (Region with Convolutional Neural Network), or DETR (Detection Transformer).

[0050] Optionally, the first initial learning model includes at least:

[0051] (1) A shallow feature extraction network based on the target wavelet transform convolution module is used to capture basic shallow visual features from the first training sample input and output shallow feature maps, which retain the boundary and texture information of characters in the original image.

[0052] Specifically, the aforementioned shallow feature extraction network includes at least: multiple feature fusion modules improved based on the target wavelet transform convolution module, and multiple downsampling layers. The multiple feature fusion modules can extract shallow feature maps at different scales, allowing the model to comprehensively consider both global information and local details after image weighting. The multiple downsampling layers, through max pooling, average pooling, and increasing the convolution kernel stride, reduce the spatial size (resolution) of the shallow feature maps, helping subsequent networks extract higher-level abstract features from the shallow feature maps.

[0053] (2) A deep feature extraction network based on the target wavelet transform convolution module is used to receive the shallow feature map output by the shallow feature extraction network and extract more abstract and complex deep feature maps (such as shape, font type, and relative position between characters) from the first training sample. These deep feature maps contain comprehensive information of the characters in the first training sample, which is crucial for constructing a complete representation of the characters.

[0054] Specifically, the aforementioned deep feature extraction network includes at least: multiple upsampling layers, a feature pyramid module improved based on the target wavelet transform convolution module, and a path aggregation network module improved based on the target wavelet transform convolution module. Specifically, the feature pyramid module, using a top-down path, starts from the highest layer of the shallow feature extraction network, upsamples the shallow feature map corresponding to the highest layer (e.g., through bilinear interpolation) to improve its resolution, and then aggregates it with a path aggregation network module improved by 1... The feature maps of the next layer after the convolutional layer are added together until they reach the bottom layer of the shallow feature extraction network, thereby generating a series of feature maps from high to low resolution. These feature maps have rich feature information at different scales. The path aggregation network module starts from the lowest layer of the shallow feature extraction network and passes the feature maps of the lower layer to the higher layer after upsampling and convolution operations, realizing the aggregation of features from bottom to top. At the same time, side paths are added between different layers, so that features can directly exchange information in the horizontal direction, enhancing the interdependence between features.

[0055] (3) An improved prediction network based on asymmetric convolution modules is used to receive deep feature maps from deep feature extraction networks and convert them into specific prediction outputs.

[0056] Specifically, the prediction network includes at least: multiple detection heads and an asymmetric convolutional module at the front end of the target detection head in the multiple detection heads. The multiple detection heads are used to output the predicted inverted results, predicted bounding box coordinates and corresponding confidence scores of each second character in each first training sample. The target detection head is used to output the predicted inverted results of each second character in each first training sample.

[0057] Therefore, the aforementioned detection heads can be categorized into three types based on their task: localization detection heads, classification detection heads, and orientation-aware detection heads. Among these, localization detection heads can perceive character boundaries and predict the coordinates of character bounding boxes; therefore, the expression for a localization detection head can be written as: In the formula, This represents the predicted character bounding box in the first training sample; This represents the convolutional layer in the positioning detection head; H represents the Sigmoid activation function, used to convert the predicted values ​​of the convolutional layer output into predicted probabilities of the bounding box parameters; H represents the height of the feature map; W represents the width of the feature map; and F represents the deep feature map input to the prediction network.

[0058] The classification detection head determines whether a character exists within a bounding box and assigns a confidence score to each bounding box predicted to contain a character. This score reflects the classification detection head's confidence that the bounding box contains a character; the closer the score is to 1, the greater the probability that the bounding box contains a character. Therefore, the expression for the classification detection head can be written as: In the formula, The confidence score is typically a real number between 0 and 1. This represents a classification convolution used to predict the class probability of each grid cell in an image.

[0059] The orientation-aware detector can analyze the orientation of characters within their bounding boxes in the first training sample. Its expression can be written as: In the formula, This indicates the probability of predicting an inverted position. This represents an asymmetric convolution module.

[0060] In addition, considering that the characters in the first training sample may appear in different directions, and the change of direction will directly affect the model's detection of characters, an asymmetric convolution module is introduced at the front end of the target detection head of multiple detection heads in the prediction network to enhance the sensitivity of the prediction network to the vertical gradient change, so that it can accurately capture the "top-heavy" inverted feature that the characters may appear.

[0061] Specifically, the aforementioned asymmetric convolution module can employ two cascaded one-dimensional convolutional layers (i.e., independent vertical and horizontal one-dimensional convolutional layers), such as a 1×3 convolutional layer and a 3×1 convolutional layer. This allows for more efficient capture of the features of the first training sample in both the horizontal and vertical directions. Using a vertical convolutional kernel and bias term can more specifically capture the vertical features of characters to identify tilted or inverted characters in the first training sample. Combined with a horizontal convolutional kernel, it can more comprehensively capture the multi-directional features of characters, improving the accuracy and robustness of character detection. Therefore, the expression for the asymmetric convolution module can be written as: In the formula, This represents the feature output obtained after the deep feature map input to the prediction network is subjected to a one-dimensional convolution operation in the vertical direction. This represents the feature output obtained after the deep feature map input to the prediction network is subjected to a one-dimensional convolution operation in the horizontal direction. represents the output feature map of the asymmetric convolution module, which integrates the features of the input data in two vertical and horizontal directions; X represents the feature map input to the asymmetric convolution module; and These represent the kernel weights in the vertical and horizontal directions, respectively; and These represent the vertical and horizontal biases, respectively. This allows the asymmetric convolutional module to produce non-zero outputs even when there is no input activation, forcing the network to learn continuously. ReLU represents the convolution operation; ReLU represents the activation function, used after the convolution operation to enhance the expressive power of the feature output.

[0062] Therefore, the first initial learning model decouples the traditional coupled detection head into three independent parallel branches. By selecting the most suitable convolution type for different detection heads—where the localization and classification detection heads use ordinary convolution, and the orientation-aware detection head uses asymmetric convolution—it better captures the orientation information of the first training sample. Simultaneously, through these three detection heads, the first initial learning model can simultaneously acquire the target's category information, localization information (such as the position information of the character bounding box), and orientation information (such as whether it is inverted) in a single computation flow when processing the input image. This achieves efficient and accurate acquisition of the target's comprehensive information (category, position, and orientation) in a single forward propagation, significantly improving the model's multi-task processing capability and robustness in complex scenes.

[0063] Optionally, the target wavelet transform convolutional module described above can output the feature map corresponding to the first training sample using the following method (the target wavelet transform convolutional module in the shallow feature extraction network can output a shallow feature map, and the target wavelet transform convolutional module in the deep feature extraction network can output a deep feature map):

[0064] First, the first training sample is decomposed into multiple frequency sub-bands (i.e., a segment of the signal spectrum, usually composed of a continuous range from low frequency to high frequency) through a wavelet transform layer. This can effectively distinguish the low-frequency features of character strokes and the high-frequency noise in the background, thereby achieving the purpose of distinguishing the features of different scales of the image.

[0065] Specifically, the wavelet transform layer described above uses high-pass and low-pass filters to divide the image features of the first training sample into frequency sub-bands, and the specific calculation formula is as follows: In the formula, X represents the first training sample input; This indicates a cascaded wavelet transform operation; This represents the low-frequency subband obtained by passing the filter through two low-pass filters. This represents the low-frequency to high-frequency subband obtained by passing a low-pass filter (row direction) and a high-pass filter (column direction); The high-frequency and low-frequency subbands are obtained by passing a high-pass filter (row direction) and a low-pass filter (column direction); This represents the high-frequency subband obtained by passing through two high-pass filters.

[0066] As can be seen from the above formula, the wavelet transform layer can decompose the feature map of the first training sample into low-frequency-low-frequency sub-bands (preserving the main contour and structural information of the first training sample), low-frequency-high-frequency sub-bands (capturing the texture details of the first training sample in the horizontal direction), high-frequency-low-frequency sub-bands (capturing the texture details of the first training sample in the vertical direction), and high-frequency-high-frequency sub-bands (reflecting the most complex edge and texture details of the first training sample). By dividing the frequency sub-bands, this multi-scale decomposition enables the model to simultaneously focus on the macroscopic structure and microscopic details of the image, thereby effectively distinguishing the features of the tire rubber surface texture and character strokes in the tire rubber surface inkjet printing image.

[0067] Next, depthwise separable convolutional layers are used to perform convolution operations on each frequency subband to obtain multiple initial feature maps, which efficiently enhances character features and suppresses background noise while maintaining computational efficiency.

[0068] Specifically, depthwise separable convolutional layers can independently perform convolution operations on each input channel, which means that when processing frequency subbands obtained by wavelet transform, they can specifically enhance the key features in each subband.

[0069] Generally, the depthwise separable convolutional layer mentioned above consists of two parts: depthwise convolution and pointwise convolution. Depthwise convolution convolves each feature map with an independent kernel to obtain a new feature map. This means that each input channel is associated with only one kernel, greatly reducing computation. Pointwise convolution, on the other hand, merges all the feature maps obtained from depthwise convolution using a 1×1 kernel, mixing information from different feature maps to extract more complex features. Therefore, the specific calculation formula for the depthwise separable convolutional layer is as follows: In the formula, Y represents the multi-scale feature map extracted through a depthwise separable convolutional layer. This represents the inverse wavelet transform, where n represents the number of frequency subbands obtained through the aforementioned wavelet transform layers. This represents the wavelet transform convolution of the i-th frequency sub-band. This represents the i-th frequency sub-band obtained through the wavelet transform layer.

[0070] Finally, the attention mechanism is used to recombine the multiple initial feature maps output by the depthwise separable convolutional layer to obtain the corresponding feature maps, enabling the model to focus on the most relevant frequency components and suppress unimportant tire surface texture information.

[0071] Specifically, the aforementioned attention mechanism dynamically adjusts the importance of each frequency sub-band after the convolution operation based on channel attention weights, thereby enhancing the expression of key features while suppressing channel noise to obtain the final image feature map. The specific steps are as follows:

[0072] Step S11: Generate corresponding channel attention weights to reflect the importance of each frequency sub-band feature.

[0073] The specific implementation process of step S11 above may include: first using 1 The first convolutional layer compresses multiple initial feature maps to reduce the number of channels. Then, activation functions (such as ReLU, Sigmoid, etc.) are used to process the compressed initial feature maps to highlight representative regions in the initial feature maps. Next, global average pooling (GAP) is performed to transform the compressed initial feature maps into one-dimensional vectors, which serve as the basis for attention weights. Finally, the output dimension is adjusted through a fully connected layer to make it the same as the number of frequency subbands.

[0074] Step S12: Multiply each initial feature map with the corresponding generated attention weight, and dynamically adjust the feature importance of different initial feature maps according to the attention weight, thereby strengthening the key features of the first training sample.

[0075] Step S13: After adjusting the feature weights, aggregate all initial feature maps to obtain the final feature map corresponding to the first training sample. The aggregation process can be element-wise addition, concatenation, or a single 1-bit operation. One convolutional layer is used to maintain the consistency of the number of channels.

[0076] Therefore, by adding a target wavelet transform convolution module to the shallow feature extraction network and the deep feature extraction network, the multi-scale features of the first training sample can be effectively processed, the character stroke features can be adaptively enhanced and the tire rubber surface texture features can be suppressed, so as to extract character features more accurately in the complex background of the tire rubber surface inkjet image, thus solving the problem that the tire rubber surface texture and character strokes are easily confused.

[0077] Step 3: Iteratively train the first initial learning model using the first training sample set and the first sample label set to obtain the completed target detection model.

[0078] Specifically, in the above iterative training process, the first training sample set can be divided into multiple batches, and each training batch can be processed according to the following steps to achieve iterative training:

[0079] First, each of the first training samples in the training batch is input into the first initial learning model for forward computation to obtain the coordinate information of each predicted bounding box, the predicted confidence score, and the predicted inverted probability output by the first initial learning model.

[0080] Next, the target loss function is constructed using the real bounding box coordinates associated with each first sample label, the real inverted label, the predicted bounding box coordinates, the predicted confidence score, and the predicted inverted probability.

[0081] Specifically, the expression for the above objective loss function can be written as: In the formula, Describes the target loss function. Represents the bounding box loss function. This represents the category confidence loss function. This represents the direction-aware loss function. , and These represent the weight coefficients of the bounding box loss function, the class confidence loss function, and the orientation-aware loss function, respectively.

[0082] To address the challenge of discerning the orientation of characters on tire rubber surfaces, a weighted binary cross-entropy (IoU) guided by the intersection-with-union (IoU) ratio was designed as the orientation-aware loss function. This means the IoU value is used as a dynamic weight to adjust the orientation prediction loss. Therefore, a higher IoU value indicates more accurate detection, leading to a larger weight for orientation prediction and a more focused model on learning and optimizing accuracy. Conversely, a lower IoU value indicates less accurate detection, causing the model to reduce the weight for orientation prediction to prevent background noise from inaccurate position prediction from misleading orientation recognition. Thus, the expression for the orientation-aware loss function can be written as: In the formula, Indicates the number of positive samples; This indicates an indicator function used to mark whether the i-th predicted bounding box contains a character; it is 1 if the predicted bounding box contains a character, and 0 otherwise. This represents the intersection-union ratio (IU / U) between the i-th predicted bounding box and its corresponding ground truth bounding box. This represents the focusing factor, used to adjust the impact of IoU values ​​on the perceptual loss function, suppressing the gradient of inaccurately localized training samples. A larger value indicates a more effective focusing factor. The value will increase the power of the intersection-union ratio, and the model will pay more attention to and learn the orientation perception information obtained when the localization is accurate; Indicates a true inverted label (which can be 1 or 0); This indicates the predicted probability of the object being upside down.

[0083] Then, the gradient vector of the target loss function with respect to the model parameters (including connection weights and biases) of the first initial training model is calculated using the backpropagation algorithm, and the model parameters are adjusted according to the gradient and the preset learning rate.

[0084] Repeat the above steps. The first initial detection model after adjusting the model parameters satisfies the preset iteration termination condition. The first initial detection model obtained from the iteration process when the iteration termination condition is satisfied is used as the trained target detection model for subsequent detection tasks.

[0085] In one implementation, the first initial learning model can be loaded into memory. For example, the raw data of the first initial learning model can be loaded from non-volatile memory into volatile memory so that the processor can run the first initial learning model. The raw data of the first initial learning model refers to unprocessed data, which typically includes the parameters and structural data of the first initial learning model. The structural data can be parameter-based computational relationships, such as forward propagation computational relationships between intermediate layers or between neurons. Specifically, the structural data can include structure-related code of the first initial learning model, such as code used to perform related computations between intermediate layers or between neurons.

[0086] In one implementation, a region can be partitioned in memory for loading the first initial learning model, which may include a structure data storage area and a parameter storage area. The structure data storage area stores structure-related code, and the parameters referenced by it can be pointed to by pointers to the addresses of specific parameters in the parameter storage area. During the training of the first initial learning model, it may be necessary to frequently update the parameters; in this case, updating the parameter values ​​in the parameter storage area is sufficient.

[0087] Therefore, when the recognition system performs character detection on the inkjet-printed image of the first tire rubber surface, it can call the pre-trained target detection model to analyze the inkjet-printed image of the first tire rubber surface and output three sets of prediction data: In the formula, This represents the coordinate information of the predicted bounding box output by the object detection model, where... , These represent the x-coordinate of the bottom left corner of the bounding box, the y-coordinate of the bottom left corner of the bounding box, the width of the bounding box, and the height of the bounding box, respectively. The category confidence level indicating whether the bounding box region contains characters; This indicates the predicted probability of the object being upside down.

[0088] Furthermore, the recognition system can perform orientation correction on each first character in the first tire rubber surface inkjet image based on the target detection results to obtain the second tire rubber surface inkjet image.

[0089] As an optional implementation, the recognition system can first determine whether the predicted inverted probability of multiple first characters in the target detection result is lower than a preset threshold. If the predicted inverted probability of the target character among the multiple first characters is not lower than the preset threshold, the system determines the central axis of the target character based on the predicted bounding box coordinate information of the target character in the target detection result, and rotates the target character half a circle along the central axis to obtain the second tire rubber surface inkjet image after direction correction.

[0090] In other words, if the recognition system determines that the predicted inverted probability of the target character is not lower than the preset threshold, it indicates that the target character is inverted and needs to be corrected. At this time, the central axis of the target character can be determined based on the coordinate information of the predicted bounding box (which is obtained from the geometric center of the predicted bounding box and the aspect ratio of the bounding box), and the image area of ​​the target character can be rotated 180 degrees along the central axis, thereby correcting the inverted target character to an upright position, thus avoiding a decrease in recognition rate due to incorrect orientation in subsequent character recognition processes.

[0091] After the above orientation correction process, regardless of the shooting angle or tire placement position in the actual application environment, it can be ensured that the input character recognition model is always the second tire rubber surface inkjet image containing positive characters, thus completely eliminating the impact of character inversion on character recognition accuracy.

[0092] Furthermore, the recognition system can use a pre-trained character recognition model to analyze the inkjet printing image on the surface of the second tire rubber compound to obtain the first predicted string.

[0093] Alternatively, the character recognition model described above can be trained using the following method:

[0094] Step 1: Obtain the second training sample set and the second sample label set. The second training sample set includes multiple inkjet-printed images of the surface of the third tire rubber material, which serve as the second training samples. The second sample label set includes the real strings of each second training sample as the corresponding second sample labels.

[0095] Specifically, the second training sample set can be composed of a large number of inkjet-printed images of the rubber surface of the third tire, collected using high-resolution industrial line scan cameras and industrial camera light sources during the actual production process of different tire models; while the second sample label can be manually annotated on these inkjet-printed images of the rubber surface of the third tire to obtain the real string within the inkjet-printed images of the rubber surface of the third tire.

[0096] It is important to note that, in order to improve the model's generalization ability and robustness, enabling it to maintain high detection accuracy under different lighting conditions, viewpoints, scales, and various changes in the real world, the embodiments of this application can also perform image enhancement on the aforementioned second training sample set, such as standardization (e.g., using a scaling strategy of "fixed height, proportional width") and size normalization (e.g., reducing image pixels from...). Normalization to Operations such as range selection are used to reduce image size differences caused by product specifications and batch number lengths.

[0097] Step 2: Construct the second initial learning model.

[0098] Specifically, the aforementioned second initial learning model includes at least: a feature extraction module, a bidirectional long short-term memory network module, and a transcription module. Among them:

[0099] (1) Feature extraction module, which can use a lightweight backbone network (such as ResNet18) to extract high-dimensional feature sequences from the input second training samples.

[0100] For example, if the second training sample is represented as Here, H represents the image height, W represents the image width, and C represents the image dimension, with each dimension representing a specific feature at a specific location in the image. Therefore, the second training sample can be compressed along the height direction into a feature sequence with a width of only one pixel. Then, the feature sequence is divided along its width according to the downsampling rate r to obtain a series of feature vectors. ,in, These vectors form a high-dimensional feature sequence. This is used for subsequent sequence modeling.

[0101] (2) A bidirectional long short-term memory network module is used to perform contextual modeling on the high-dimensional feature sequence output by the feature extraction module. This is because there are strong semantic relationships between the characters on the surface of tire rubber (for example, "215" is often followed by the " / " symbol, and "R" is often followed by a number).

[0102] Specifically, the bidirectional long short-term memory network module includes a forward long short-term memory network layer and a backward long short-term memory network layer. Therefore, the bidirectional long short-term memory network module can capture the forward historical information and backward future information of high-dimensional feature sequences respectively according to the following steps:

[0103] First, the high-dimensional feature sequence The input is fed into the feedforward long short-term memory (LSTM) network layer. The feedforward long short-term memory network layer starts from the high-dimensional feature sequence as its initial vector. Begin by processing each feature vector one by one until... When processing each feature vector, the feedforward long short-term memory network layer will consider the hidden state of the previous feature vector. The hidden state is updated using the current feature vector, and a new forward hidden state is calculated. This new forward hidden state contains information about all feature vectors preceding the current feature vector, i.e., forward history information. Simultaneously, the high-dimensional feature sequence... The input is fed into the feedforward long short-term memory network layer, which then processes the vector from the end of the sequence. Start by processing each feature vector in reverse order until... When processing each vector, the hidden state of the next feature vector is considered. The new backward hidden state is calculated using the current feature vector. This new backward hidden state contains information about all subsequent vectors from the current feature vector to the end of the sequence, i.e. backward future information.

[0104] Then, the new forward hidden states generated by the forward long short-term memory network layer and the backward long short-term memory network layer are respectively... and the new backward hidden state They are pieced together to form a new hidden state of fusion. This new fusion hidden state It not only contains high-dimensional feature sequences The sequence contains forward historical information from the initial feature vector to the current feature vector, as well as backward future information from the current feature vector to the feature vector at the end of the sequence, which can more comprehensively reflect each element in the sequence.

[0105] Finally, the sequence of fused hidden states is composed of all the new fused hidden states. As the output of the bidirectional long short-term memory network, it is used for subsequent character recognition tasks.

[0106] Therefore, the bidirectional long short-term memory network module provides richer and more comprehensive contextual information for each element in a high-dimensional feature sequence. In sequence recognition and analysis tasks, it can combine forward historical information and backward future information to reasonably infer ambiguous characters, effectively solving the problem of recognition difficulties when a single character is ambiguous.

[0107] (3) Transcription module, which is used to convert the feature vectors in the fused hidden state sequence output by the bidirectional long short-term memory network module into the final character sequence. Among them, the transcription module introduces a special "whitespace character" category, which allows the model to output whitespace or repeated characters without pre-segmenting the characters into single characters, greatly improving the model's flexibility in recognizing irregular or conjoined characters.

[0108] Specifically, for each fusion hidden state in the fusion hidden state sequence, the transcription module can use an activation function to convert each vector in the fusion hidden state into a value between 0 and 1, and the sum of the values ​​of all elements equals 1, forming a probability distribution vector: In the formula, This represents the probability distribution vector of the transcription module output for the t-th fusion hidden state in the fusion hidden state sequence; |L| represents the character set size, which includes numbers, letters, and whitespace, etc. This represents the weight parameters, which determine how the information in the t-th fused hidden state is mapped to each node of the output layer; Let t represent the t-th fusion hidden state; b represents the bias vector, which provides different baseline probabilities for each node of the transcription module. This represents the activation function, which is used to transform the input into a probability distribution.

[0109] The second initial learning model, comprised of the aforementioned feature extraction module, bidirectional long short-term memory network module, and transcription module, can efficiently and accurately identify and decode key information features in complex sequence data. Specifically, the extraction module is responsible for filtering and transforming valuable information from the second training samples for subsequent analysis. The bidirectional long short-term memory network module, based on this, uses bidirectional information flow to capture the correlation between elements in the feature sequence, enhancing the model's ability to understand context. Especially when dealing with tasks such as character recognition on tire rubber surfaces, it can effectively identify blurred or connected characters. The transcription module, by introducing whitespace and allowing repeated characters at time steps, can directly convert each frame output of the model into the final character or sequence result without pre-segmenting the sequence. This entire model design significantly improves the model's accuracy and robustness in recognizing indeterminate sequences composed of blurred characters.

[0110] Step 3: Iteratively train the second initial learning model using the second training sample set and the second sample label set to obtain the character recognition model.

[0111] Optionally, the above iterative training process may include:

[0112] Step S21: For each training batch in the iterative training process, input each second training sample of the training batch into the feature extraction module of the second initial learning model to obtain the feature sequence corresponding to each second training sample output by the feature extraction module. The feature sequence includes multiple feature vectors obtained by feature extraction after segmenting the second training sample in the width direction.

[0113] Step S22: Input each feature sequence into the bidirectional long short-term memory network module of the second initial learning model to obtain the fusion hidden state sequence corresponding to each feature sequence. Each fusion hidden state sequence includes multiple fusion hidden states after feature fusion of each feature vector in the feature sequence and the corresponding forward hidden state and backward hidden state.

[0114] Step S23: Input each fusion hidden state sequence into the transcription module of the second initial learning model to obtain the second predicted string corresponding to each fusion hidden state sequence.

[0115] Step S24: Construct a target loss function using each second predicted string and its corresponding sample label, and adjust the model parameters of the second initial learning model according to the target loss function.

[0116] Specifically, the CTC loss function can be used as the target loss function for the second initial learning model. The CTC loss function fuses all possible second predicted strings corresponding to the hidden state sequence and finds the second predicted string that best matches the true string through maximum likelihood estimation. Therefore, the expression for the above target loss function can be written as: In the formula, Y represents the target loss function; Y represents the true string in the second training sample. represents all possible second prediction characters, including whitespace and string repetition; B represents a many-to-one mapping function used to remove repeating characters and whitespace from the second prediction string. For example, this mapping function can be used to map the string aa-blank-bb to the string ab. This represents the second set of predicted characters mapped to the real string using a mapping function; This indicates the output string given a second training sample X. The probability; N represents the number of characters in the real string of the second training sample; This indicates that the predicted character will be output at time t. The probability value.

[0117] In addition, during training, the recognition system can dynamically adjust the learning rate using a cosine annealing strategy, as detailed below: In the formula, This represents the initial learning rate (e.g., 0.001). This represents the maximum learning rate (e.g., 0.0001). This represents the learning rate in the t-th iteration. Indicates the current batch number in training. This represents a maximum number of training batches. π represents pi (a period used to construct a cosine function, helping to calculate the trend of the learning rate over time), and cos represents the pre-function.

[0118] Step S25: Repeat steps S21-S24 until the preset iteration termination condition is met to obtain the trained character recognition model. The iteration termination condition includes, but is not limited to: convergence of the target loss function (i.e., the change in the value of the target loss function is less than a preset threshold in several consecutive iterations), and the current iteration round reaching the preset maximum number of iterations.

[0119] In one implementation, the second initial learning model can be loaded into memory. For example, the raw data of the second initial learning model can be loaded from non-volatile memory into volatile memory to enable the processor to run the character recognition model. The raw data of the second initial learning model refers to unprocessed data, which typically includes the parameters and structural data of the second initial learning model. The structural data can be parameter-based computational relationships, such as forward propagation computational relationships between intermediate layers and between neurons. Specifically, the structural data can include structure-related code of the second initial learning model, such as code used to perform related computations between intermediate layers and between neurons.

[0120] In one implementation, a region can be partitioned in memory for loading the second initial learning model, which may include a structure data storage region and a parameter storage region. The structure data storage region stores structure-related code, and the parameters referenced by it can be pointed to by pointers to the addresses of specific parameters in the parameter storage region. During the training of the second initial learning model, it may be necessary to frequently update the parameters; in this case, updating the parameter values ​​in the parameter storage region is sufficient.

[0121] Therefore, the recognition system can call the pre-trained character recognition model to analyze the inkjet-printed image on the surface of the second tire rubber compound, and form an initial prediction string by using the character with the highest probability corresponding to each fusion hidden state in the fusion hidden state sequence output by the transcription module. Then, the system removes duplicate characters and whitespace characters from the initial prediction string using the mapping function B to obtain the first prediction string. The entire recognition process does not require string segmentation, which greatly simplifies the image recognition process and improves recognition efficiency. This is particularly beneficial for recognizing conjoined characters of variable length in complex backgrounds.

[0122] Finally, the recognition system can determine the target string within the inkjet printing image on the surface of the first tire rubber compound based on the first predicted string. The target string includes, but is not limited to, key information such as the tire's specifications, production batch, manufacturer code, and quality grade.

[0123] As an optional implementation, the above-mentioned target string verification process includes:

[0124] Step 1: Validate the first predicted string using a preset regular expression.

[0125] Specifically, the recognition system can pre-set multiple regular expressions based on the standard format of the characters on the tire rubber surface (such as tire specifications usually following the format of "width / aspect ratio / inner diameter", and production batch numbers being a combination of numbers and letters). These expressions define the legal combinations, lengths, and the position and order of specific characters on the tire rubber surface.

[0126] Therefore, the first predicted string is matched against a preset regular expression to verify whether it conforms to the compliant format of the inkjet characters on the surface of the tire rubber.

[0127] Step 2: If the verification result is successful, the first predicted string is used as the target string in the inkjet printing image on the surface of the first tire rubber compound.

[0128] Step 3: If the verification result is a failure, use the preset tire rubber surface inkjet coding rule library to replace the abnormal characters in the first predicted string to obtain the target string in the first tire rubber surface inkjet coding image.

[0129] Specifically, the steps for replacing abnormal characters in the first predicted string include: First, based on the verification results, identifying abnormal characters in the first predicted string that do not match the regular expression or violate the character connection order. These abnormal characters may be recognition errors caused by image quality issues, character distortion, blurring, or other interference factors. Next, for each marked abnormal character, querying the tire rubber surface inkjet coding rule library to find possible error forms and their corresponding standard characters. The tire rubber surface inkjet coding rule library includes at least: standard forms and corresponding error forms of various tire rubber surface characters, and connection order rules between various tire rubber surface characters, such as the typical arrangement order of characters like tire specifications, production date, and batch number. Then, based on the queried rules, replacing or adjusting the abnormal characters using standard characters. If the abnormal character is blurry, it can be replaced with the standard character that most closely resembles it in appearance; if the abnormal character violates the character connection order rules, a suitable standard character is selected for replacement based on common sorting rules in the tire rubber surface inkjet coding rule library. Finally, after the replacement is complete, the generated new string is validated again using regular expressions to check whether the string now conforms to the preset standard format of the inkjet characters on the tire rubber surface. If the string passes the validation, the system confirms it as the target string.

[0130] Furthermore, after obtaining the target string within the inkjet-printed image of the first tire rubber compound surface, the recognition system can return the target string to the industrial control computer and display it on the screen. The industrial control computer can then perform subsequent processing based on the recognition results, such as data storage and anomaly alarms. For example, when an anomaly occurs in the recognition result, the industrial control computer can automatically trigger an alarm mechanism and display a warning on the display using different colors or symbols.

[0131] In summary, the tire rubber surface character recognition method provided in this application embodiment can effectively recognize characters on the tire rubber surface based on the target detection model and the character recognition model. Compared with existing character recognition schemes, it has the following technical advantages:

[0132] (1) By using the target detection model to perform depth analysis on the inkjet printing image on the surface of the tire rubber, all characters in the image can be detected and located. At the same time, the inversion probability of each character can be evaluated. Based on the target detection results, the characters in the image are corrected according to the predicted inversion probability to obtain an optimized positive character image. This correction process improves the accuracy of subsequent character recognition.

[0133] (2) The construction of the above target detection model depends on the feature extraction network and prediction network inside the model. Among them, the shallow and deep feature extraction networks based on the target wavelet transform convolution module are used to perform multi-scale feature analysis on the image, which can effectively distinguish between character strokes and tire rubber surface texture, and reduce the impact of image noise on the detection process. Meanwhile, the prediction network uses three detection heads to decouple multiple tasks. While detecting the character position, it can use the judgment of whether the character in the image is upside down to achieve direction perception without adding an additional classification network.

[0134] (3) The image after orientation correction is analyzed using a character recognition model. This model is based on a feature extraction module, a bidirectional long short-term memory network module and a transcription module, forming a deep learning recognition network for the recognition of variable-length character sequences. Combined with bidirectional context modeling and temporal transcription mechanism, it can accurately identify inkjet characters in various complex scenarios, including special fonts, blurred characters and deformed characters, effectively solving the problem of low recognition accuracy of traditional character segmentation methods under tire-adhesive characters and complex background interference.

[0135] (4) In order to further improve the accuracy of recognition, regular expression verification and inkjet coding rule library are also introduced. The format of the predicted string is verified by regular expression to see if it conforms to the preset rules. For the predicted string with abnormal format, the abnormal characters are replaced by the inkjet coding rule library on the surface of the tire rubber to ensure that the final target string is consistent with the character information on the surface of the tire rubber.

[0136] Example 2

[0137] According to an embodiment of this application, a tire rubber surface inkjet character recognition device is also provided for implementing the tire rubber surface inkjet character recognition method in Embodiment 1, such as... Figure 2 As shown, the tire rubber surface inkjet character recognition device includes at least: a detection module 22, an image processing module 24, a recognition module 26, and a determination module 28, wherein:

[0138] Detection module 22 is used to analyze the inkjet printing image on the surface of the first tire rubber using a pre-trained target detection model to obtain target detection results. The target detection results include at least the predicted bounding box coordinates and predicted inverted probabilities of multiple first characters within the inkjet printing image on the surface of the first tire rubber.

[0139] Image processing module 24 is used to perform orientation correction on each first character in the first tire rubber surface inkjet image based on the target detection result to obtain the second tire rubber surface inkjet image.

[0140] The recognition module 26 is used to analyze the inkjet printing image on the surface of the second tire rubber material using a pre-trained character recognition model to obtain the first predicted string;

[0141] The determination module 28 is used to determine the target string within the inkjet printing image on the surface of the first tire rubber material based on the first predicted string.

[0142] The following describes the functions of each module of the tire rubber surface inkjet character recognition device in conjunction with the specific implementation process.

[0143] Optionally, the above-mentioned device may further include a first model training module for training an object detection model according to the following method: acquiring a first training sample set and a first sample label set, wherein the first training sample set includes: multiple inkjet-printed images of the surface of the third tire rubber as first training samples, and the first sample label set includes: using the real detection results of each first training sample as the corresponding first sample label, wherein the real detection results include at least: the real bounding box coordinates of the second character within the inkjet-printed image of the surface of the third tire rubber and the real inverted label; constructing a first initial learning model, wherein the first initial learning model includes at least: based on the target The network comprises a shallow feature extraction network improved by a target wavelet transform convolution module, a deep feature extraction network improved by a target wavelet transform convolution module, and a prediction network improved by an asymmetric convolution module. The target wavelet transform convolution module is used to decompose the first training sample into multiple frequency sub-bands through a wavelet transform layer, and then use a depthwise separable convolution layer to perform convolution operations on each frequency sub-band to obtain multiple initial feature maps. The attention mechanism is then used to recombine the multiple initial feature maps to obtain the corresponding feature maps. The first initial learning model is iteratively trained using the first training sample set and the first sample label set to obtain a trained target detection model.

[0144] Optionally, the shallow feature extraction network includes at least: multiple feature fusion modules improved based on the target wavelet transform convolution module, and multiple downsampling layers; the deep feature extraction network includes at least: multiple upsampling layers, a feature pyramid module improved based on the target wavelet transform convolution module, and a path aggregation network module improved based on the target wavelet transform convolution module; the prediction network includes at least: multiple detection heads and an asymmetric convolution module at the front end of the target detection head among the multiple detection heads, wherein the asymmetric convolution module includes two cascaded one-dimensional convolutional layers, and the two one-dimensional convolutional layers are used to extract features in different directions, the multiple detection heads are used to output the predicted inverted results, predicted bounding box coordinates and corresponding confidence scores of all second characters in each first training sample, and the target detection head is used to output the predicted inverted results of all second characters in each first training sample.

[0145] Optionally, the image processing module is further configured to determine whether the predicted inverted probability of multiple first characters in the target detection result is lower than a preset threshold, and if the predicted inverted probability of the target character among the multiple first characters is not lower than the preset threshold, determine the central axis of the target character based on the predicted bounding box coordinate information of the target character in the target detection result, and rotate the target character half a circle along the central axis to obtain the second tire rubber surface inkjet image after direction correction.

[0146] Optionally, the above-mentioned device may further include a second model training module for training a character recognition model according to the following method: acquiring a second training sample set and a second sample label set, wherein the second training sample set includes: multiple inkjet-printed images of the surface of third tire rubber material as second training samples, and the second sample label set includes: using the real strings of each second training sample as the corresponding second sample label; constructing a second initial learning model, wherein the second initial learning model includes at least: a feature extraction module, a bidirectional long short-term memory network module, and a transcription module; and iteratively training the second initial learning model using the second training sample set and the second sample label set to obtain a character recognition model.

[0147] Optionally, the second model training module described above is further configured to, for each training batch in the iterative training process, input each second training sample of the training batch into the feature extraction module of the second initial learning model to obtain a feature sequence corresponding to each second training sample output by the feature extraction module, wherein the feature sequence includes multiple feature vectors obtained by feature extraction after segmenting the second training sample in the width direction; input each feature sequence into the bidirectional long short-term memory network module of the second initial learning model to obtain a fusion hidden state sequence corresponding to each feature sequence, wherein each fusion hidden state sequence includes multiple fusion hidden states after feature fusion of each feature vector in the feature sequence and the corresponding forward hidden state and backward hidden state; input each fusion hidden state sequence into the transcription module of the second initial learning model to obtain a second prediction string corresponding to each fusion hidden state sequence; construct a target loss function using each second prediction string and the corresponding sample label, and adjust the model parameters of the second initial learning model according to the target loss function.

[0148] Optionally, the aforementioned determining module is further configured to verify the first predicted string using a preset regular expression; if the verification result is successful, the first predicted string is used as the target string within the first tire rubber surface inkjet printing image; if the verification result is unsuccessful, the abnormal characters in the first predicted string are replaced using a preset tire rubber surface inkjet printing rule library to obtain the target string within the first tire rubber surface inkjet printing image, wherein the tire rubber surface inkjet printing rule library includes at least: standard forms and corresponding error forms of various tire rubber surface characters, and connection order rules between various tire rubber surface characters.

[0149] It should be noted that each module in the tire rubber surface inkjet character recognition device in this application embodiment corresponds one-to-one with each implementation step of the tire rubber surface inkjet character recognition method in embodiment 1. Since embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to embodiment 1, and will not be elaborated further here.

[0150] Example 3

[0151] According to an embodiment of this application, a computer program product is also provided, which includes a computer program, wherein when the computer program is executed by a processor, it implements the tire rubber surface inkjet character recognition method in Embodiment 1.

[0152] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device containing the non-volatile storage medium executes the tire rubber surface inkjet character recognition method in Embodiment 1 by running the computer program.

[0153] According to an embodiment of this application, a processor is also provided for running a computer program, wherein the computer program executes the tire rubber surface inkjet character recognition method in Embodiment 1.

[0154] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the tire rubber surface inkjet character recognition method of Embodiment 1 through the computer program.

[0155] Specifically, the computer program executes the following steps during runtime: Analyzing the inkjet-printed image of the first tire rubber surface using a pre-trained target detection model to obtain target detection results, wherein the target detection results include at least: the predicted bounding box coordinates and predicted inverted probabilities of each of the multiple first characters within the inkjet-printed image of the first tire rubber surface; performing orientation correction on each of the first characters in the inkjet-printed image of the first tire rubber surface based on the target detection results to obtain a second tire rubber surface inkjet-printed image; analyzing the second tire rubber surface inkjet-printed image using a pre-trained character recognition model to obtain a first predicted string; and determining the target string within the inkjet-printed image of the first tire rubber surface based on the first predicted string.

[0156] As an alternative implementation, the above-mentioned electronic device may exist in the form of a mobile terminal, a computer terminal, or a similar computing device. Figure 3 A hardware block diagram of a computer terminal for implementing a method for recognizing inkjet-printed characters on the surface of tire rubber is shown. Figure 3 As shown, the computer terminal 30 may include one or more processors 302 (shown as 302a, 302b, ..., 302n in the figure) 302 (processor 302 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 304 for storing data, and a transmission device 306 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus port (which may be included as one of the ports of a universal serial bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 3 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, computer terminal 30 may also include... Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown.

[0157] It should be noted that the aforementioned one or more processors 302 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 30. As involved in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0158] The memory 304 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the tire rubber surface inkjet character recognition method in this embodiment. The processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 304, thereby implementing the above-mentioned application vulnerability detection method. The memory 304 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 304 may further include memory remotely located relative to the processor 302, and these remote memories can be connected to the computer terminal 30 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0159] The transmission device 306 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 30. In one example, the transmission device 306 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 306 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0160] The display may be, for example, a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 30.

[0161] The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0162] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0164] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0165] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0166] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0167] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of tire compound surface code character recognition, characterized by, include: The target detection result is obtained by analyzing the inkjet printing image on the surface of the first tire rubber using a pre-trained target detection model. The target detection result includes at least the predicted bounding box coordinates and predicted inverted probability of each of the multiple first characters in the inkjet printing image on the surface of the first tire rubber. Based on the target detection results, the orientation of each first character in the first tire rubber surface inkjet printing image is corrected to obtain the second tire rubber surface inkjet printing image; The first predicted string is obtained by analyzing the inkjet-printed image on the surface of the second tire rubber material using a pre-trained character recognition model. The target string within the inkjet printing image on the surface of the first tire rubber compound is determined based on the first predicted string.

2. The method of claim 1, wherein, The target detection model is trained using the following method: Obtain a first training sample set and a first sample label set, wherein the first training sample set includes: multiple inkjet images of the surface of the third tire rubber as first training samples, and the first sample label set includes: the real detection results of each first training sample as the corresponding first sample label, wherein the real detection results include at least: the real bounding box coordinate information of the second character in the inkjet image of the surface of the third tire rubber and the real inverted label. A first initial learning model is constructed, wherein the first initial learning model includes at least: a shallow feature extraction network improved based on a target wavelet transform convolutional module, a deep feature extraction network improved based on the target wavelet transform convolutional module, and a prediction network improved based on an asymmetric convolutional module. The target wavelet transform convolutional module is used to decompose the first training sample into multiple frequency sub-bands through a wavelet transform layer, and then perform convolution operations on each frequency sub-band using a depth-separable convolutional layer to obtain multiple initial feature maps. Finally, an attention mechanism is used to recombine the multiple initial feature maps to obtain corresponding feature maps. The first initial learning model is iteratively trained using the first training sample set and the first sample label set to obtain the target detection model after training.

3. The method according to claim 2, characterized in that, The shallow feature extraction network includes at least: multiple feature fusion modules and multiple downsampling layers, which are improved based on the target wavelet transform convolution module; The deep feature extraction network includes at least: multiple upsampling layers, a feature pyramid module improved based on the target wavelet transform convolution module, and a path aggregation network module improved based on the target wavelet transform convolution module; The prediction network includes at least: multiple detection heads and an asymmetric convolutional module at the front end of a target detection head among the multiple detection heads, wherein the asymmetric convolutional module includes two cascaded one-dimensional convolutional layers, and the two one-dimensional convolutional layers are respectively used to extract features in different directions; the multiple detection heads are respectively used to output the predicted inverted result, predicted bounding box coordinate information and corresponding confidence score of each second character in each first training sample; and the target detection head is used to output the predicted inverted result of each second character in each first training sample.

4. The method of claim 1, wherein, Based on the target detection results, the orientation of each first character in the first tire rubber surface inkjet printing image is corrected to obtain the second tire rubber surface inkjet printing image, including: Determine whether the predicted inverted probability of multiple first characters in the target detection result is lower than a preset threshold. If the predicted inverted probability of the target character among the multiple first characters is not lower than the preset threshold, determine the central axis of the target character based on the predicted bounding box coordinate information of the target character in the target detection result, and rotate the target character half a circle along the central axis to obtain the second tire rubber surface inkjet image after direction correction.

5. The method according to claim 1, characterized in that, The character recognition model is trained using the following method: Obtain a second training sample set and a second sample label set, wherein the second training sample set includes: multiple inkjet-printed images of the surface of the third tire rubber material as second training samples, and the second sample label set includes: the real strings of each second training sample as the corresponding second sample labels; Construct a second initial learning model, wherein the second initial learning model includes at least: a feature extraction module, a bidirectional long short-term memory network module, and a transcription module; The second initial learning model is iteratively trained using the second training sample set and the second sample label set to obtain the character recognition model.

6. The method according to claim 5, characterized in that, The second initial learning model is iteratively trained using the second training sample set and the second sample label set to obtain the character recognition model after training, including: For each training batch during the iterative training process, each second training sample of the training batch is input into the feature extraction module of the second initial learning model to obtain a feature sequence corresponding to each second training sample output by the feature extraction module. The feature sequence includes multiple feature vectors obtained by feature extraction after segmenting the second training sample in the width direction. Each feature sequence is input into the bidirectional long short-term memory network module of the second initial learning model to obtain a fusion hidden state sequence corresponding to each feature sequence. Each fusion hidden state sequence includes multiple fusion hidden states obtained by feature fusion of each feature vector in the feature sequence and the corresponding forward hidden state and backward hidden state. Each fusion hidden state sequence is input into the transcription module of the second initial learning model to obtain a second prediction string corresponding to each fusion hidden state sequence. A target loss function is constructed using each second prediction string and the corresponding sample label, and the model parameters of the second initial learning model are adjusted according to the target loss function.

7. The method of claim 1, wherein, Determining the target string within the inkjet printing image on the surface of the first tire rubber compound based on the first predicted string includes: The first predicted string is verified using a preset regular expression; If the verification result is successful, the first predicted string will be used as the target string in the inkjet printing image on the surface of the first tire rubber compound. If the verification result is a verification failure, the abnormal characters in the first predicted string are replaced using a preset tire rubber surface inkjet coding rule library to obtain the target string in the first tire rubber surface inkjet coding image. The tire rubber surface inkjet coding rule library includes at least: standard forms and corresponding error forms of various tire rubber surface characters, and connection order rules between the various tire rubber surface characters.

8. A tire compound surface code character recognition apparatus characterized by comprising: include: The detection module is used to analyze the inkjet printing image on the surface of the first tire rubber using a pre-trained target detection model to obtain target detection results. The target detection results include at least the predicted bounding box coordinates and predicted inverted probabilities of each of the multiple first characters in the inkjet printing image on the surface of the first tire rubber. The image processing module is used to perform orientation correction on each first character in the first tire rubber surface inkjet image based on the target detection result to obtain the second tire rubber surface inkjet image. The recognition module is used to analyze the inkjet printing image on the surface of the second tire rubber material using a pre-trained character recognition model to obtain a first predicted string; The determination module is used to determine the target string within the inkjet printing image on the surface of the first tire rubber material based on the first predicted string.

9. A non-volatile storage medium, comprising: The non-volatile storage medium stores a computer program, wherein the device containing the non-volatile storage medium executes the tire rubber surface inkjet character recognition method according to any one of claims 1 to 7 by running the computer program.

10. An electronic device, comprising: include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the tire rubber surface inkjet character recognition method according to any one of claims 1 to 7 via the computer program.

Images