Tire Sidewall Imaging Methods

Abstract

A computer-implemented method is presented for classifying one or more embossed and / or engraved markings on a tire sidewall into one or more categories comprising digital image data of the tire sidewall. The method includes generating a first image channel from a first portion of the digital image data associated with a corresponding first portion of the tire sidewall. Generating the first image channel includes performing histogram equalization on the first portion of the digital image data to generate the first image channel. The method also includes generating a first feature map using the first image channel and applying a first classifier to the first feature map to classify the embossed and / or engraved markings into one or more first categories.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for classifying one or more embossed and / or engraved markings on a tire sidewall into one or more categories, for training a classifier to perform the task, and for generating a training data set to train the classifier. Background Art

[0002] The outer surface of a vehicle tire, i.e., the tire sidewall, carries text-based information about the tire brand, model, and some codes. The code carries, for example, the manufacturing plant, tire age, tire type, expected load, rated speed, and size, production batch, and other product information. The code may include, for example, a combination of one or more letters and / or numbers. In addition, the tire sidewall may also have other information text, logos, symbols, pictograms, and / or any other visual representation of information. For vehicle users, especially fleet operators, every detail of this information is critical because it provides a consistent and reliable method to track the status and usage of tires across the fleet, thereby greatly improving the fleet operator's ability to perform data analysis on the fleet's tire inventory.

[0003] Attempts have been made to automate the process of reading tire sidewalls, however, such systems are either 3D scanner-based systems (available from MicroEpsilon, Cognex, and Numetrix) for controlled inspection tasks indoors, or handheld laser devices for indoor and outdoor work. Such systems are expensive to manufacture (due to the laser components of the construction), difficult to calibrate, prone to damage, and / or still require operator assistance, so they are not truly automated and low-cost.

[0004] Applying optical character recognition (OCR) to images captured without structured light would significantly reduce hardware costs. However, due to outdoor use of tires, which can cause sidewall text wear (e.g., due to material corrosion, dust, dryness and / or humidity), and because the text has very low contrast (black on black) that is sometimes difficult to interpret even for human observation, let alone for automated systems, previous attempts at OCR based on color or grayscale images have not been successful.

[0005] The challenge in producing such a system is to read the text on a moving vehicle tire quickly enough as it passes by the system, and to read the text on a stationary tire quickly enough as the system's field of view moves over the stationary tire. It is also desirable to be able to compensate for variable conditions (such as varying weather conditions outside a fleet warehouse and / or dusty / dirty conditions indoors), and to be able to produce accurate and reproducible results without the need for the assistance of a human operator.

[0006] In WO2017060739 A1 an imaging system is provided for providing contrast enhanced images. In particular, lighting is important for reading embossed or engraved text, such as tire sidewall codes, because the contrast of the text and thus the legibility of the text can be improved by shadow casting. Although WO2017060739 A1 suggests performing OCR on such images using image analysis software to read the embossed marks, conventional OCR techniques described in WO2017060739 A1 perform poorly because they are too slow to read and / or have low accuracy in non-ideal, non-laboratory situations.

[0007] Recent advances in deep learning-based image classification and text recognition have pushed deep convolutional neural networks (CNNs) to the top of the performance tables for object and text detection and recognition tasks. One known type of CNN suitable for such tasks is Faster R-CNN, first proposed by Ren, S., He, K., Girshick, R., and Sun, J. in 2015: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 28 (pp. 91–99). Curran Associates, Inc.

[0008] Faster R-CNN has a deep CNN feature extractor called the backbone. There are many known deep CNN feature extractors that can be used as the backbone, including, for example, VGG16, ResNet50, ResNet100, InceptionNet, AlexNet, etc. VGG16 was first proposed by K. Simonyan and A. Zisserman in Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv technical report in 2014.

[0009] As explained in Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In C. Cortes, N.D. Lawrence, D.D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 28 (pp. 91–99). Curran Associates, Inc, the backbone feature extractor is extended to a built-in region proposal network (RPN) that proposes regions of interest (also called proposals) for further processing by the classifier. The classifier classifies the content of the RPN's proposals into preset categories based on the training data.

[0010] Other examples of CNNs known for object and text detection and recognition include Fast R-CNN, R-CNN, SPPNet, SSD, YOLO v1 / v2 / v3, and EAST text detector.

[0011] The problem with applying such a CNN to the task of reading a tire sidewall is that, as described above, images of the tire sidewall typically have low contrast between the tire sidewall background and the embossed and / or engraved markings. This is particularly true for old, worn tires, where the embossed and / or engraved markings have been partially worn away and are difficult to read even for the human eye. Increasing the camera resolution used for image capture can help ensure that image data related to the worn embossed and / or engraved markings is not completely lost relative to the background. The high resolution referred to here means that after the curvature of the tire sidewall in the image is corrected (i.e., removed during preprocessing), the image typically has a resolution of approximately 500×2000 to 500×4000 pixels.

[0012] However, as is well known in the art, using high-resolution images as input to CNNs is significantly slower and requires greater processing resources to train the CNNs as well as to infer unseen data at test time. While powerful graphics processing units (GPUs) can be applied to such tasks, they are costly compared to cheaper but less powerful central processing units (CPUs).

[0013] Therefore, there is a need for an improved computer-implemented method of classifying embossed and / or engraved indicia on a tire sidewall that can be efficiently implemented on, for example, a CPU.

[0014] The following terms have the following meanings in this document:

[0015] “Unwarping”—mapping an image of a curved tire sidewall onto an image in which the curvature has been removed or straightened;

[0016] "Image channel" - one or more predefined components of image data underlying a digital image;

[0017] “Synthetically generated / synthetic data” – data generated using an algorithm, used to increase the total amount of data available for training, when, for example, only limited data from other sources is available;

[0018] “Training duration” – the time required for the CNN to converge to a predetermined acceptable precision and recall;

[0019] “At test time” – when a trained CNN performs the task for which it was trained, e.g., when a trained CNN classifies one or more embossed and / or engraved markings on a tire sidewall that have not been seen before into one or more categories.

[0020] “Histogram equalization” - an image processing method applied to contrast adjustment of an original image by forming a histogram of the distribution of intensity values ​​of the original image over a plurality of intensity value ranges, determining a mapping between intensity values ​​of the original image and modified intensity values ​​based on the histogram, and modifying the intensity values ​​of the original image according to the mapping to produce a modified image, wherein the mapping is selected such that the distribution of pixel intensities within the intensity ranges in the modified image is more uniform than the distribution of pixel intensities within the intensity ranges in the original image according to a uniformity criterion, e.g. differences between back-tracked intensity ranges of equal width are reduced.

[0021] “Adaptive histogram equalization” – a method of adjusting contrast in image processing in which histogram equalization is performed on each pixel or pixel region of the original image by using a corresponding mapping defined by a neighborhood of the pixel or pixel region (e.g., the portion of the original image within a predetermined distance of the pixel or pixel region);

[0022] “Anchor” - a feature of an image known in prior knowledge that indicates the presence and location of other features on the image;

[0023] "Difference of Gaussians (DoG)" - a method of feature enhancement in image processing that subtracts a blurred version of an original image from another less blurred original image, where the blurred image can be obtained by convolving the original image with a Gaussian kernel with different standard deviations;

[0024] “CPU” and “GPU” - Central Processing Unit and Graphics Processing Unit. A GPU is configured to provide more processing cores running at a lower clock speed, compared to a CPU running at a higher clock speed with fewer processing cores. Summary of the invention

[0025] In summary, the present invention is directed to improving the efficiency of a trained CNN to classify embossed and / or engraved markings on a tire sidewall image, and reducing the training time required for the CNN to perform the task. Specifically, rather than using initially acquired rectified image data to generate a feature map, the rectified image data is first histogram equalized to generate a new image channel. The new image channel is used to generate a feature map for classifying the embossed and / or engraved markings. This can reduce the training duration and processing resources required to classify the embossed and / or engraved markings to an acceptable level of accuracy when testing.

[0026] A further improvement in classifier accuracy can be achieved when the feature map is not generated from a single image channel, but from three image input channels, wherein the three image input channels are composed of the above-mentioned histogram equalization channel as the first channel, a second image channel generated by performing adaptive histogram equalization on the initial rectified image data, and a third image channel including the initial rectified image data.

[0027] The inventors found that the training duration of Faster R-CNN using the above three image channels is about 8-10 times faster than that of Faster R-CNN using only the initially obtained rectified image data as a single image channel. In addition, the test accuracy of unseen data is also higher.

[0028] During testing, the inventors further found that Faster R-CNN, trained using the three image channels described above and tested on downsampled images (50%), was able to run in real time on a CPU, rather than on a more expensive GPU, to classify embossed and / or engraved marks.

[0029] The present invention thus provides a faster and cheaper way to train a CNN to classify embossed and / or engraved markings on a tire sidewall, as well as a faster and cheaper way to apply the trained CNN to new tire sidewall images with higher accuracy at test time. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of a method of classifying embossed and / or engraved indicia on a tire sidewall into one or more categories according to an embodiment.

[0031] Figure 2is a flow chart of a method of classifying embossed and / or engraved indicia on a tire sidewall into one or more categories according to an embodiment.

[0032] Figure 3 is a flow chart of a method of classifying embossed and / or engraved indicia on a tire sidewall into one or more categories according to an embodiment.

[0033] Figure 4 is a flow chart of a method of generating a training data set for training a classifier to classify embossed and / or engraved markings on a tire sidewall into one or more categories according to an embodiment.

[0034] Figure 5 is a flow chart of a method of generating a training data set for training a classifier to classify embossed and / or engraved markings on a tire sidewall into one or more categories according to an embodiment.

[0035] Figure 6 is a flow chart of a method of generating a training data set for training a classifier to classify embossed and / or engraved markings on a tire sidewall into one or more categories according to an embodiment.

[0036] Figure 7 is a flow chart of a method of training a classifier to classify embossed and / or engraved markings on a tire sidewall into one or more categories according to an embodiment.

[0037] Figure 8 is a flow chart of a method of training a classifier to classify embossed and / or engraved markings on a tire sidewall into one or more categories according to an embodiment.

[0038] Fig. 9 is a flow chart of a method of training a classifier to classify embossed and / or engraved markings on a tire sidewall into one or more categories according to an embodiment.

[0039] Fig.10 It is a block diagram showing the technical architecture of a data processing device. DETAILED DESCRIPTION

[0040] According to a first aspect, a computer-implemented method for classifying one or more embossed and / or engraved indicia on a tire sidewall into one or more categories is provided.

[0041] Now refer to Figure 1 A first embodiment is described.

[0042] In a first step, digital image data 101 defining a digital image of a tire sidewall is provided.

[0043] In a second step, a first image channel 102 is generated from the digital image data 101 by performing histogram equalization on a first portion of the digital image data 101, the first portion being associated with a corresponding first portion of the tire sidewall. The first portion may be the entire sidewall of the tire or some other portion of the sidewall of the tire.

[0044] Histogram equalization allows areas of low local contrast in an image to achieve higher contrast, and works well in situations where both the background and foreground are light or dark, such as where a tire sidewall has a black foreground and background. It can be performed using, for example, a function from any known programming function library (e.g., OpenCV), although other libraries are known to the skilled person. An example of a suitable histogram equalization function is the "equalizeHist" function of the OpenCV library. Histogram equalization is performed on the first digital image data 101 to generate a first image channel 102, thereby enhancing the contrast in the data corresponding to low contrast areas on the tire sidewall, wherein the low contrast areas include the locations where the embossed and / or engraved markings are located.

[0045] In the third step, a first feature map 103 is generated using the first image channel 102. The first feature map 103 may be, for example, the output of a convolutional layer of a CNN, where a convolution kernel slides on the input first image channel 102. It is contemplated that any suitable CNN architecture that enables the use of the first image channel 102 to generate a feature map may be used. A suitable architecture may be, for example, referenced to Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In C. Cortes, N.D. Lawrence, D.D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 28 (91–99 pages). Curran Associates, Inc.

[0046] In a fourth step, a first classifier 104 is applied to the first feature map 103 to classify the embossed and / or engraved markings into one or more first categories. Non-limiting examples of the types of categories into which some embossed and / or engraved markings may be classified include: brand logos, tire types and models, brand symbols, "DOT" codes (character sequences consisting of the letters "D", "O", "T", which are usually preceding other embossed and / or engraved sidewall markings for regulatory reasons), and character sequences "5 / " (which are usually part of the tire size engraved sidewall markings for regulatory reasons). In one embodiment, the first classifier 104 may classify the embossed and / or engraved markings into more than 370 different categories, so that different brand logos and symbols can be distinguished from each other, different tire type and model markings can be distinguished from each other, and character sequences "D", "O", "T" and / or "5 / " can be distinguished from each other and from other markings on the tire sidewall. For example, for a given input tire sidewall image, the first classifier 104 may be used to determine that the embossed and / or engraved markings on the tire sidewall include one of a plurality of tire brand logos or symbols, information indicating the tire type and model and associated tire characteristics, the character sequence “D”, “O”, “T”, and / or “5 / ”.

[0047] Reference Figure 2 , a second embodiment is provided.

[0048] In a first step which is identical to the first step of the first embodiment, digital image data 201 defining a digital image of a tire sidewall is provided.

[0049] In a second step, in the same manner as in the first embodiment, a first image channel 202a is generated from the digital image data 201 by performing histogram equalization on a first portion of the digital image data 201. As described above, the first portion of the digital image data 201 is associated with a corresponding first portion of the tire sidewall, which may be the entire tire sidewall or some other portion of the tire sidewall.

[0050] However, in addition to generating the first image channel 202a as described above, a second image channel 202b and a third image channel 202c are provided.

[0051] The second image channel 202 b is generated by performing adaptive histogram equalization on the first portion of the digital image data 201 .

[0052] Adaptive histogram equalization is similar to histogram equalization because it improves the contrast in the image. However, it is different in that it calculates several histograms, each corresponding to different parts of the image, and uses these histograms to redistribute the brightness values ​​of the image. Therefore, histogram equalization works well when the background and foreground are bright or dark, and works well when the image contains areas that are obviously brighter or darker than the rest of the image. Adaptive histogram equalization can be performed using, for example, a function from any known programming function library (such as OpenCV), although other libraries are also known to those skilled in the art. An example of a suitable adaptive histogram equalization function is the "equalizeHist" function of the OpenCV library. Performing adaptive histogram equalization on the first digital image data 201 to generate the second image channel 202b thus enhances the contrast in the data corresponding to the area that is obviously brighter or darker than the majority of the image.

[0053] In addition to generating the first image channel 202a and generating the second image channel 202b as described above, the digital image data 201 is assigned a third image channel 202c. Thus, the three channels 202a, 202b, 202c when taken together comprise image data that has been contrast enhanced in two different ways (histogram equalization and adaptive histogram equalization) in addition to the initially provided image data 201. It has been found that supplementing the digital image data 201 with two contrast-enhanced image channels in this manner is surprisingly effective in reducing training duration and allowing the trained CNN to classify embossed and / or engraved marks, which, as described above, inherently have very low contrast and are difficult to detect and recognize using OCR.

[0054] In the third step, a first feature map 203 is generated using the first, second and third image channels 202a, 202b, 202c. As described above in conjunction with the first embodiment, the first feature map 203 may be, for example, the output of a convolutional layer of a CNN, where the convolution kernel slides over the input. In this case, the input is the first, second and third image channels 202a, 202b, 202c. It is contemplated that any suitable CNN architecture may be used that generates a feature map using the first, second and third image channels 202a, 202b, 202c. For example, a suitable architecture can be found in Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In C. Cortes, N.D. Lawrence, D.D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 28 (pages 91–99). Curran Associates, Inc.

[0055] In a fourth step, which is identical to the fourth step of the first embodiment, a first classifier 204 is applied to the first feature map 203 to classify the embossed and / or engraved marks into one or more of the categories described above in connection with the first embodiment.

[0056] Reference Figure 3 , a third embodiment similar to the second embodiment is provided. In particular, as in the second embodiment, digital image data 301 of a tire sidewall having embossed and / or engraved marks is provided, and a first image channel (histogram equalization) 302a and a second image channel (adaptive histogram equalization) 302b are generated, and the digital image data 301 is assigned to a third image channel 302c. In the same manner as in the second embodiment, a first feature map 303a is generated using the three channels 302a, 302b, 302c, and a first classifier 304a is applied to the first feature map 303a to classify the embossed and / or engraved marks into one or more first categories (i.e., categories of the above-mentioned type).

[0057] One or more embossed and / or engraved marks of the first category can be used as anchors to help locate other embossed and / or engraved marks on the tire sidewall that do not belong to the first category. These other embossed and / or engraved marks may contain more detailed information related to, for example, the manufacturing plant, life, tire type or model, expected load, speed rating and size, manufacturing batch information, manufacturer details, and other detailed product information. For example, for regulatory reasons, "DOT", "5 / " or other character sequences may often be followed by alphanumeric or other characters that specify the above information type. Therefore, if these anchor character sequences can be found using the first classifier, the approximate locations of other embossed and / or engraved marks in other categories can be more effectively located and other locations discarded.

[0058] This can be achieved by providing, in addition to the three channel 302a, 302b, 302c input, the first feature map 303a and the first classifier 304a, additionally providing a second classification branch, the purpose of which is to classify these other embossed and / or engraved marks into other second categories.

[0059] The input of the second classification branch is the fourth channel 302d, which is the only channel in the second classification branch. The fourth channel 302d is generated by applying a Gaussian difference operation to the first part of the digital image data 301. Optionally, the first part is a new image created from the digital image data 301 by cropping to the estimated length of the embossed or engraved information after or before the "DOT" and / or "5 / " anchor. For example, "DOT" is usually before the remaining embossed and / or engraved marking code, which can be, for example, between 14-20 characters in length, each with a similar pixel width. By measuring the character width of one or more characters in the "DOT" anchor in pixels, an approximate image width equal to the width of 14 to 20 characters can be determined and used to crop the digital image data to generate the first part. Considering the "5 / " anchor, it usually forms a part of the tire size, so it does not appear before the rest of the characters, but appears between them. Thus, the character width of the "5" in "5 / " can be measured in pixels, and the image can be cropped to a few character widths before the "5 / " anchor (typically 3) and a few character widths after (typically 14). As defined above, the Gaussian difference operation is a feature enhancement algorithm that enhances the visibility of edges and other details in a digital image.

[0060] From this fourth channel 304d, a separate second feature map 303b is generated. Like the first feature map 303a, the second feature map 303b can be the output of a convolutional layer of, for example, a CNN, where the convolution kernel slides over the input. In this case, the input is the fourth image channel 302d.

[0061] A second classifier 304b, different from the first classifier 304a, is then applied to the second feature map 303b to classify the embossed and / or engraved markings into one or more second categories (different from the first category).

[0062] Thus, the first classifier 304a can detect and identify an anchor from which the approximate location of other embossed and / or engraved marks can be inferred, and the second classifier 303b can classify these other embossed and / or engraved marks into a second category. Depending on which category and which anchor has been selected, there may be situations where one or more of the second categories are identical to and indistinguishable from one or more of the first categories. For example, one or more alphanumeric characters may serve as anchors and at the same time contain information relating to, for example, tire characteristics, which are classified by the second classifier 304b. In this case, the alphanumeric characters would belong to both the first category and the second category. Furthermore, although symbols and logos have been described herein as generally belonging to the first category, it is conceivable that they may also belong to the second category.

[0063] Optionally, instead of performing a Gaussian difference operation on the entire first portion of the digital image data 301, a Gaussian difference operation may be performed on only a portion thereof. For example, if the first classifier has identified an anchor such as a "DOT" or "5 / " character sequence, it may be inferred that only an area of ​​a predetermined height and width of the digital image adjacent to the "DOT" or "5 / " character sequence (e.g., within a predetermined distance from the anchor) may contain embossed and / or engraved marks in the second category. Thus, the first portion of the digital image data may be cropped so that it corresponds to the predetermined area, and the Gaussian difference operation is efficiently performed only on the corresponding image block and the corresponding portion of the first portion of the digital image data, rather than on the entire first portion of the digital image data.

[0064] By cropping the image in this manner, the resolution of the image on which the Difference of Gaussian operation is performed is smaller, and the second feature map 303b is generated thereby, and the second classifier 304b is applied thereto, thereby occupying less processing resources both during training and at test time.

[0065] Thus, in one exemplary embodiment, it is contemplated that a first classifier is trained on a full, uncropped, high-resolution image to recognize more than 370 first categories, such as brand logos, tire types and models, symbols, and predetermined character sequences used as anchors, such as "DOT" and / or "5 / ". However, the first classifier is not trained to recognize and distinguish between second category character sequences that appear near the anchors, which contain more detailed tire information, such as manufacturing plant, life, tire type, expected load, speed rating and size, manufacturing batch information, manufacturer details, and other detailed product information. This task would be inefficient using the first classifier and the full, uncropped, high-resolution image, because only a small portion of the image (near the anchor) actually contains these character sequences, while the rest is empty. Instead, as described above, the task is based on the anchors, and a second classifier is trained on a smaller cropped image of a predetermined height and width. In this exemplary embodiment, the second classifier is trained to recognize more than 39 second categories, such as alphanumeric characters 0-9 and AZ, as well as additional categories, such as the " / " symbol and / or other symbols used to specify tire product information.

[0066] In addition, the inventors discovered that tire manufacturers use different techniques to produce the embossed and / or engraved markings of the logos and symbols in the first category (compared to the character sequences in the second category of designated tire product information after the "DOT" and / or "5 / " character sequence). It was surprisingly found that applying a Gaussian difference operation to the texture of the logos and symbols is not conducive to classifying the logos and symbols into the first category that is primarily distinguished by texture. Therefore, a Gaussian difference operation is not used in the first classifier branch. Instead, it was found that the Gaussian difference operation is particularly effective in enhancing the edges of the second category of embossed and / or engraved character sequences, which are embossed or engraved markings that are primarily distinguished by edges.

[0067] With respect to all of the above embodiments, although the "DOT" and "5 / " character sequences have been suggested as anchors, other character sequences and / or symbols or other embossed and / or engraved indicia may be used for this purpose.

[0068] Optionally, the methods of all the above embodiments can be performed using the Faster R-CNN network, refer to Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In C. Cortes, N.D. Lawrence, D.D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 28 (pages 91–99). Curran Associates, Inc.

[0069] Specifically, with respect to the third embodiment, the first classifier 304a may be a classifier in a first Faster R-CNN network, and the second classifier 304b may be a classifier in a second Faster R-CNN network, and the first Faster R-CNN network is different from the second Faster R-CNN network.

[0070] Thus, the first Faster R-CNN network identifies and classifies features corresponding to images of the first category, while the second Faster R-CNN identifies and classifies features corresponding to images of the second category.

[0071] Each of the first Faster R-CNN network and the second Faster R-CNN network may have its own region of interest generator to generate a plurality of regions of interest in the first feature map and the second feature map.

[0072] Each of the first and second Faster R-CNN networks may also have its own bounding box regressor to perform bounding box regression on the corresponding generated regions of interest to refine their locations. In addition, as described in Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In C. Cortes, N. D. Lawrence, D. L. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 28 (91–99 pages). Curran Associates, Inc., a region of interest pooling layer may also be provided that takes input regions of interest of non-uniform size and reshapes them to a predetermined uniform size. In the case of the third embodiment, in which the input to the fourth channel (i.e., the only input channel to the second classifier) ​​will be cropped based on the anchors detected by the first classifier, the uniform size of the rectangular region of interest output by the region of interest pooling layer can be used to calculate the average alphanumeric (or other mark) character length in pixels, and thereby determine where to crop the image of the fourth channel in the second category branch so that it captures the embossed and / or engraved marks corresponding to the second category.

[0073] As a non-limiting example, where a "DOT" anchor is detected, 15-18 alphanumeric characters are expected to follow it on the tire sidewall. For a "5 / " anchor, there should be 2-5 characters before the "5 / " and 12-15 characters after the "5 / ". In each case, the characters are typically around 100 pixels in height. Therefore, the image can be cropped and resized accordingly to ensure that all 15-18 characters and / or the 2-5 characters and 12-15 characters sequence around the anchor are retained for the fourth channel. Note that other character sequence lengths and heights can be used to determine the extent to which the image of the fourth channel should be cropped and / or resized.

[0074] Optionally, with respect to all embodiments, tire curvature in the digital image data may be corrected by mapping an image of the curved tire sidewall to an image in which the curve has been removed or straightened so that it is removed from the digital image data used to generate the input channel. This simplifies all subsequent processing.

[0075] Optionally, for all embodiments, some or all of the digital image data may be downsampled by between 50% and 80%. As described above, it is contemplated that the digital image has a high resolution (e.g., approximately 500×4000 pixels). Applying a trained CNN to such a high resolution image at test time is resource intensive and results in an unacceptable slowdown when the trained CNN is processed / inferred using only CPU-based operations. This slowdown will prevent the CNN from achieving acceptable performance when running in real time. Downsampling the input digital image data significantly reduces the required computational resources, but at the expense of reduced robustness and ability to generalise. To overcome this problem, during training, multiple training samples are randomly downsampled between 50%-80% so that the CNN is trained on both the full high resolution image and the downsampled image. Optionally, slight affine deformations may also be added to some training samples to further improve generalisation. The inventors found that a CNN trained in this manner is significantly more robust and better at generalizing images of any resolution of the input, including full high-resolution images and images downsampled between 50%-80%. Therefore, in the case where the input image is downsampled between 50%-80% at test time, the above embodiments can be executed only on the CPU at test time, thereby enabling real-time operation with acceptable performance. In the case where the embossed and / or engraved marks are large logos, the inventors found that downsampling effectively shrinks the logo to fit the receptive field of the CNN, thereby improving their detection.

[0076] Furthermore, with respect to the third embodiment, in which the digital image is cropped to a predetermined height and width in the second classification branch based on one or more anchors identified by the first classification branch, no downsampling is required. This is because cropping significantly reduces the size of the input image, thereby reducing the required computational resources. For example, the height of the cropped image may be approximately 100 pixels, compared to a height of 500 pixels for the full high-resolution input image.

[0077] According to a second aspect, a computer-implemented method for generating a training data set for training a classifier to classify one or more embossed and / or engraved marks on a tire sidewall into one or more categories is provided. The method can be used to generate training data to train the classifier used in the embodiments of the first aspect of the present invention. The advantages, technical effects and features of the embodiments of the first aspect described above are applicable to the corresponding embodiments of the second aspect and can be combined with the corresponding embodiments of the second aspect.

[0078] Reference Figure 4In one embodiment, a method of generating a training data set 400 includes providing a plurality of digital images and corresponding digital image data 401 of corresponding sidewalls of a plurality of tires, each sidewall having one or more embossed and / or engraved markings. The method also includes labeling 402 the corresponding digital image data 401 with one or more first categories, the first categories corresponding to, for example, brand logos, tire types and models, brand symbols, anchors, or other first categories as described above. For each digital image and corresponding digital image data 401, a corresponding first image channel 403 is generated from a first portion of the corresponding digital image data 401 associated with a corresponding first portion of the corresponding tire sidewall. The generating includes performing histogram equalization on the first portion of the corresponding digital image data 401 as described above.

[0079] Reference Figure 5 , describes the Figure 4 Another embodiment is similar to the embodiment shown in FIG. As in the above embodiment, the method of generating a training data set 500 includes providing a plurality of digital images of corresponding sidewalls of a plurality of tires and corresponding digital image data 501, each sidewall having one or more embossed and / or engraved marks. Using one or more first category marks 502 of the digital image data 501, and generating a first channel 503a by performing histogram equalization as described above.

[0080] However, in addition, a second image channel 503b and a third image channel 503c are also generated. The second image channel 503b is generated by performing adaptive histogram equalization on the first portion of the corresponding digital image data 501. For the third image channel 503c, the first portion of the respective digital image data 501 is allocated as the third image channel 503c. These three channels correspond to those described above in conjunction with the first aspect.

[0081] Reference Figure 6 , describes the Figure 5 Another embodiment is similar to the embodiment shown. As with the above embodiment, the method of generating a training data set 600 includes providing a plurality of digital images of corresponding sidewalls of a plurality of tires and corresponding digital image data 601, each sidewall having one or more embossed and / or engraved marks. As described above, the digital image data 601 is labeled 602a with one or more first categories, and three image channels 603a, 603b, 603c are generated by performing histogram equalization to generate a first channel 603a, generating a second channel 603b by adaptive histogram equalization, and assigning the digital image data 601 to a third image channel 603c.

[0082] However, additionally, the corresponding digital image data 601 is labeled 602b with one or more second categories, the second categories corresponding to, for example, alphanumeric characters 0-9 and AZ, and other categories, such as the " / " symbol and / or other symbols for specifying tire product information as described above in conjunction with the first aspect of the present invention. Then, a fourth image channel 603d is generated by performing a Gaussian difference operation on the first portion of the corresponding digital image data 601. As described above, the image defined by the digital image data 601 can be cropped and / or resized according to the anchor before performing the Gaussian difference operation to improve efficiency.

[0083] Alternatively, as described above, the curvature of the tire in the digital image data can be corrected by mapping the image of the curved tire sidewall to an image in which the curve has been removed or straightened so that it is removed from the digital image data used to generate the input channel. This simplifies all subsequent processing.

[0084] While it is desirable that as much of the training data as possible be real, manually labeled / annotated images, this may not always be feasible or possible. Optionally, in such a case, at least one or more of the plurality of digital images may be synthetically generated (i.e., generated using an algorithm to increase the total amount of data available for training). A non-limiting example of how the synthetic data may be generated is to take a real, fully rectified image of a tire sidewall and fill any empty spaces in that real image with text, logos, symbols, and / or other markings corresponding to the desired first and / or second categories, thereby increasing the number of (synthetic) embossed and / or engraved markings per image on which the CNN will be trained. While this technique does not offer much flexibility due to the limited amount of space available to fill in additional markings on the real image, the inventors have found that the technique produces surprisingly good results in situations where not enough real, fully manually labeled / annotated training images are available. Optionally, a slight affine transformation may also be added to some of the training data images to further improve generalization.

[0085] Optionally, as described above, some or all of the digital image data in the training data may be downsampled by between 50% and 80%. For example, where the digital images in the training data have a high resolution (approximately 500×4000 pixels), a portion of the randomly selected digital images may be downsampled by between 50% and 80%, such that the training data includes a mixture of full high resolution images and downsampled images.

[0086] The advantage of including downsampled images in the training dataset in this way is that it improves the robustness and the ability of a CNN trained on them to generalize to unseen images that are also of lower resolution and / or have been downsampled at test time, since at least some of the same low resolution images will contribute to training.

[0087] While downsampling does slightly impact detection in cases where embossed and / or engraved marks (such as “DOT” and / or “5 / ” anchors) are worn, very light, or faint, the inventors have found that downsampling to 50% provides a good tradeoff between detection of embossed and / or engraved marks (such as anchors and large logos) on the one hand and efficiency on the other.

[0088] As mentioned above about Figure 3 In the embodiment described above, in which in the fourth channel, the training data images have been cropped and / or resized to become smaller image blocks (e.g., 100 pixels in height and a predetermined number of pixels in width based on the desired character sequence length as described above), downsampling is not required because processing smaller image blocks inherently requires less computing power.

[0089] According to a third aspect, a computer-implemented method for training a classifier to classify one or more embossed and / or engraved marks on a tire sidewall into one or more categories is provided. The method can be used to train the classifier used in the embodiments of the first aspect of the invention, and can use the training data generated by the method of the second aspect of the invention. The advantages, technical effects and features of the embodiments of the first and / or second aspects described above are applicable to the corresponding embodiments of the third aspect, and can be combined with the corresponding embodiments of the third aspect.

[0090] Reference Figure 7 In one embodiment, the method includes training a first classifier 700 on first training data 701, the first training data 701 including a plurality of digital images and corresponding digital image data 702 of corresponding sidewalls of a plurality of tires, each sidewall having one or more embossed and / or engraved markings. The corresponding digital image data is labeled with one or more first categories (e.g., brand logo, tire type and model, brand symbol, anchor, or other categories as described above). Each digital image and corresponding digital image data includes: a corresponding first image channel 703 generated by performing histogram equalization on a first portion of the corresponding digital image data 702 associated with a corresponding first portion of the corresponding tire sidewall. The first classifier 700 is trained using the provided plurality of labels 704 and the first image channels 703.

[0091] Reference Figure 8 , describes the Figure 7Another embodiment similar to the embodiment shown. As with the above embodiment, the method includes training a first classifier 800 on first training data 801, the first training data 801 including a plurality of digital images and corresponding digital image data 802 of corresponding sidewalls of a plurality of tires, each sidewall having one or more embossed and / or engraved marks. The corresponding digital image data 802 are labeled with one or more first categories as described above. Each digital image and corresponding digital image data 802 includes a first image channel 803a generated by performing histogram equalization on a first portion of the corresponding digital image data 802 associated with a corresponding first portion of the corresponding tire sidewall.

[0092] However, additionally, as described above with respect to the corresponding embodiments of the first and second aspects of the present invention, each digital image and corresponding digital image data also include a second image channel 803b and a third image channel 803c. The second image channel 803b is generated by performing adaptive histogram equalization on the first portion of the corresponding digital image data 802 to generate the second image channel 803b. The corresponding digital image data 802 is assigned as the third image channel 803c. The advantages and effects of using three image channels (histogram equalization 803a, adaptive histogram equalization 803b and digital image data 803c) are described above in conjunction with the first and second aspects of the present invention. The first classifier 800 is trained using the provided multiple labels 804 and the first, second and third image channels 803a, 803b, 803c.

[0093] Reference Fig. 9 , describes the Figure 8 Another embodiment similar to the embodiment shown. As in the above embodiment, the method includes training a first classifier 900 on first training data 901, the first training data 901 including a plurality of digital images and corresponding digital image data 902 of corresponding sidewalls of a plurality of tires, each sidewall having one or more embossed and / or engraved marks. The corresponding digital image data 902 is labeled with one or more first categories as described above. Each digital image and corresponding digital image data 902 includes the same first, second and third image channels 903a, 903b, 903c as described above.

[0094] However, in addition, as described above with respect to the corresponding embodiments of the first and second aspects of the present invention, the first portion of the digital image data 902 is also marked with one or more second categories, such as, for example, alphanumeric characters 0-9 and AZ, and other categories, such as the " / " symbol and / or other symbols for specifying tire product information. In addition, a fourth image channel 903d is generated by performing a Gaussian difference operation on the first portion of the digital image data 902. This can be the entire first portion of the digital image data 902 or some other portion thereof, for example, only a cropped portion of an anchor adjacent to and / or adjacent to an anchor (i.e., multiple anchors within a predetermined distance) (e.g., a "DOT" or "5 / " character sequence). As described above in conjunction with the corresponding embodiments in the first and second aspects of the present invention, this is more efficient because the Gaussian difference only needs to be performed on a much smaller image patch, thereby reducing the processing resources required for training. The first classifier 900 is trained using the provided plurality of labels 905 corresponding to the plurality of first categories together with the first, second and third image channels 903a, 903b, 903c, and the second classifier 904 is trained using the provided plurality of labels 906 corresponding to the plurality of second categories together with the fourth image channel 903d.

[0095] With respect to all of the above embodiments, any suitable training technique may be used, such as those described in Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In C. Cortes, N.D. Lawrence, D.D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in Neural Information Processing Systems 28 (pages 91–99) Curran Associates, Inc. and references contained therein.

[0096] Unlike testing (where efficiency and capital cost are key factors and therefore it is preferred to run all method steps on a CPU), in contrast, GPUs can be used when training. In cases where the training data includes high-resolution images, a batch size of 1 is typically required to stay within the 8GB GPU RAM limit (GTX-1080). Larger batch sizes can also be used to improve results, but may require a GPU with a higher RAM limit, which is more expensive.

[0097] like Fig.10As shown in the technical architecture of , this article also provides an exemplary data processing device, which includes a device for performing the steps of the method of any of the above embodiments. In some embodiments, these steps are all performed on a CPU that is cheaper than a GPU.

[0098] The data processing device may include a processor 1000 that communicates with a memory device including an auxiliary memory 1001 (such as a disk drive), a read-only memory (ROM) 1002, and a random access memory (RAM) 1003. The processor 1000 may be implemented as one or more CPU chips that are cheaper than a GPU. The data processing device may also include an input / output (I / O) device 1004 and a network connection device 1005.

[0099] Secondary storage 1001 typically includes one or more disk drives or tape drives and is used for non-volatile storage of data and as an overflow data storage device if RAM 1003 is not large enough to hold all working data. Secondary storage 1001 can be used to store programs that are loaded into RAM 1003 when such programs are selected for execution.

[0100] In this embodiment, the secondary memory 1001 has an order processing component 1001a, which includes non-transitory instructions operated by the processor 1000 to perform various operations of the method of the present disclosure. ROM 1002 is used to store instructions and possible data read during program execution. Secondary memory 1001, RAM 1003 and / or ROM 1002 may be referred to as computer-readable storage media and / or non-transitory computer-readable media in some cases.

[0101] I / O devices 1004 may include a printer, video monitor, liquid crystal display (LCD), plasma display, touch screen display, keyboard, keypad, switches, dials, mouse, trackball, voice recognizer, card reader, paper tape reader, or other well-known input devices.

[0102] The network connection device 1005 can take the form of a modem, a modem bank, an Ethernet card, a universal serial bus (USB) interface card, a serial interface, a token ring card, a fiber distributed data interface (FDDI) card, a wireless local area network (WLAN) card, a radio transceiver card that facilitates radio communication using protocols such as code division multiple access (CDMA), global system for mobile communications (GSM), long term evolution (LTE), global microwave access interoperability (WiMAX), near field communication (NFC), radio frequency identification (RFID), and / or other air interface protocol radio transceiver cards, as well as other well-known network devices. These network connection devices 1005 can enable the processor 1000 to communicate with the Internet or one or more intranets. With this network connection, it is conceivable that the processor 1000 can receive information from the network or output information to the network during the execution of the above-mentioned method operations. Such information, which is usually represented as a sequence of instructions to be executed using the processor 1000, can be received from the network and output to the network, for example, in the form of a computer data signal contained in a carrier wave.

[0103] The processor 1000 executes instructions, codes, computer programs, scripts that it accesses from a hard disk, floppy disk, optical disk (these various disk-based systems can all be considered secondary storage 1001), flash drive, ROM 1002, RAM 1003, or network connection device 1005. Although only one processor 1000 is shown, multiple processors may be present. Thus, although instructions may be discussed as being executed by a processor, the instructions may be executed by one or more processors simultaneously, serially, or in other ways.

[0104] Although the technical architecture is described with reference to a computer, it should be understood that the technical architecture can be formed by two or more computers that communicate with each other, and these two or more computers collaborate to perform tasks. For example, but not as a limitation, the application can be partitioned in a manner that allows concurrent and / or parallel processing of the instructions of the application. Alternatively, the data processed by the application can be partitioned in a manner that allows different parts of the data set to be processed concurrently and / or in parallel by two or more computers. In one embodiment, the technical architecture can use virtualization software to provide the functions of multiple servers that are not directly bound to multiple computers in the technical architecture. In one embodiment, the functions disclosed above can be provided by executing one and / or more applications in a cloud computing environment. Cloud computing can include providing computing services via a network connection using dynamically scalable computing resources. The cloud computing environment can be established by an enterprise and / or can be rented from a third-party supplier as needed.

[0105] It is understood that by programming and / or loading executable instructions onto the technical architecture, at least one of the CPU 1000, RAM 1003, and ROM 1002 is changed, thereby partially converting the technical architecture into a special-purpose machine or device having novel functions taught by the present disclosure. The functions implemented by loading executable software into a computer can be converted into hardware implementations by well-known design rules, which are the basis of the fields of electrical engineering and software engineering.

[0106] This document also provides a computer program and a computer-readable storage medium for storing the computer program. The computer program includes instructions. When the program is executed by a computer, the instructions cause the computer to execute the steps of any method of the above embodiments.

[0107] Although the present invention has been described in accordance with the preferred embodiments described above, it should be understood that these embodiments are illustrative only and the claims are not limited to these embodiments. Those skilled in the art will be able to make modifications and substitutions based on this disclosure, and these modifications and substitutions are considered to fall within the scope of the appended claims. Each feature disclosed or shown in this specification may be incorporated into the present invention individually or in any appropriate combination with any other features disclosed or shown herein.

[0108] For example, it is conceivable that in addition to Faster R-CNN, CNNs such as Faster R-CNN, R-CNN, SPPNet, SSD, YOLO v1 / v2 / v3, and multiple text detection frameworks such as EAST (Effective and Accurate Scene Text Detector), CTPN (Connected Text Proposal Network) text detector, etc. can be used. In particular, multiple end-to-end detectors (which can be the same type or different types of CNNs) can be placed in cascade to perform the above method, thereby solving the problem of sidewall marker detection and classification in a cheap and effective manner as described above.

[0109] In addition, although the above-mentioned Faster R-CNN has the VGG16 backbone as described in K. Simonyan, A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv technical report, 2014, other backbones such as ResNet50, ResNet100, InceptionNet, AlexNet, etc. may also be used.

[0110] Furthermore, the accuracy of the above method for classifying one or more embossed and / or engraved markings on a tire sidewall into one or more categories may be further improved where the digital image data includes multiple images (optionally rectified) of a tire sidewall of a single tire. An example where the multiple images may be obtained is where the digital image data has been obtained by an imaging system that captures multiple images of the tire sidewall as the tire sidewall moves through the imaging system's field of view, the resulting multiple images providing multiple different views of the same tire sidewall and the embossed and / or engraved markings on the tire sidewall.

[0111] In general, by performing the method on multiple images of the same embossed and / or engraved indicia, as opposed to a single image, multiple classification results are obtained. Each classification result is expected to be the same because the multiple images all relate to the same tire sidewall and the same embossed and / or engraved indicia thereon. Therefore, in the event that one or more classification results are different from the other classification results, they can be determined to be false positive or false negative results.

[0112] In particular, where the digital image data comprises digital image data of a plurality of images of a tire sidewall, the steps of generating a first image channel, generating a first feature map, and applying a first classifier are performed for each image, thereby obtaining a plurality of classification results associated with embossed and / or engraved marks on the tire sidewall. Each classification result identifies into which of one or more first categories the embossed and / or engraved mark has been classified, and a corresponding plurality of respective confidence levels for each of the one or more first categories identified.

[0113] Each confidence level is a value between 0 and 1, where 1 is 100% confidence that the result is a true positive and 0 is 0% confidence that the result is a true positive.

[0114] For each of the one or more first categories in each classification result, the corresponding confidence level can be modified, for example, increased, by multiplying it by a scalar constant. Specifically, in the case where the confidence level is above a predetermined first threshold (e.g., greater than 0.95), it is multiplied by a constant (e.g., 3, 4, 5 or any other scalar) to increase it. In the case where the confidence level is equal to or lower than the first threshold, this multiplication is not performed. The multiplication has the effect of increasing the confidence level of the identified categories, which already have a high confidence level relative to those categories that do not have a high confidence level. This helps them stand out from the identified categories with lower confidence levels, thereby having a higher probability of false positives or false negatives.

[0115] After performing the multiplication where applicable, the confidence level for each identified class is accumulated (eg, by summing) to provide a confidence score for each respective class identified in the plurality of images of the tire sidewall.

[0116] Thus, where a class appears in all of the multiple images with a confidence level above the first threshold, its associated confidence level from each image will be multiplied by the constant each time, resulting in an increased confidence score indicative of a true positive.

[0117] On the contrary, in the case where a class only appears in one or a few images with a confidence level below the first threshold, or only appears in one image with a high confidence level and appears in the remaining images with low confidence levels, the resulting cumulative confidence score will be low relative to the other confidence scores and therefore indicate a false positive or true negative.

[0118] In the final step, if the corresponding confidence score is below a predetermined second threshold, it can be determined that the corresponding identified one or more first categories are false positives. The false positives can be selectively discarded and the remaining categories are retained as true positives. The retained true positives can also be used, for example, to identify the brand of the tire.

[0119] The inventors have discovered that the above method provides a fast and computationally inexpensive way to improve the accuracy of the methods described herein, where the digital image data includes multiple images of the sidewall of the same tire. An exemplary implementation of the above technique is now described. A set of labels (i.e., categories) can be set in a list having the following or other structure:

[0120]

[0121]

[0122] Each brand of tire may have multiple tire models and symbols associated with it (e.g., "BRAND1" has related tire models "Winter", "Summer", "1337Grip" and symbol "brand1_symbol"). There may also be certain non-brand names that are not uniquely associated with any brand, such as "OUTSIDE" or "ROTATION". These are marked as common tags, and while they may belong to the first category in the first classifier framework and may help identify certain common groups of tire types or inside or outside and rotation direction, their presence does not necessarily help distinguish between tire brands. Marking these types of marks as common tags means that they can be selectively discarded if they are detected at test time. Anchor categories such as "DOT" and "5 / " are also included in the list. If a given embossed and / or engraved mark has multiple different appearances (e.g., a brand's logo or text may change over time, so different versions of it may exist), multiple categories for that embossed and / or engraved mark may also be added to the list. Certain other types of embossed and / or engraved markings are also given unique labels where they are unique to a given tire brand. For example, there may be unique graphics on the sidewall of the tire that show how to store the tire inside or outside. Given the uniqueness of these graphics, they may be particularly helpful in differentiating between tire brands.

[0123] Thus, the list above shows all the categories that the first classifier can classify the embossed and / or engraved markings into. The structure of the list indicates how the categories relate to each other and / or to any associated brands, in particular how symbols, tire models and unique labels relate to their brands.

[0124] At test time, the list structure is initialized and the first classifier is applied to multiple images of tire sidewalls. For each detection of a class, the detected class is searched in the list and the associated confidence level of the detection is added to the cumulative score for the class. As described above, if the confidence level is high (e.g., greater than 0.95), it is multiplied by a constant before being added to the cumulative confidence score. A cumulative confidence score is thus generated for each detected class in the list.

[0125] Since the same marker is detected in each of the multiple images, i.e., detection redundancy is built into the system where the same marker is detected multiple times, it is expected that there will be multiple detections of the same class. Therefore, the cumulative confidence score associated with the multiple detected classes will also be high, and a consensus will be formed that these multiple detections are true positives. Conversely, false positives are unlikely to be detected multiple times, so their cumulative confidence scores will be lower and no consensus will be formed.

[0126] When all of the multiple images have been processed in this manner, the accumulated confidence scores indicate which categories have been correctly detected. Since the list structure also indicates whether a detected category is associated with a given tire brand, the confidence scores for all detected categories associated with a given brand can be aggregated. The brand with the highest aggregate score can be used to determine the brand of the imaged tire.

[0127] Additionally, once the tire brand is determined, the tire model name can be determined by searching for confidence scores associated with categories that indicate the tire model to obtain the highest score. Typically, this will exclude detection of brand symbols and generic labels that do not indicate a unique tire model.

Claims

1. A computer-implemented method for classifying one or more embossed and / or engraved markings on a sidewall of a tire into one or more categories, the method comprising: (i) providing digital image data of the sidewall of the tire; (ii) generating a first image channel from a first portion of the digital image data associated with a corresponding first portion of the sidewall of the tire, wherein generating the first image channel comprises: performing histogram equalization on the first portion of the digital image data to generate the first image channel; (iii) generating a first feature map using the first image channel; and (iv) applying a first classifier to the first feature map to classify the embossed and / or engraved markings into one or more first categories, The method further comprises: Generating a second image channel and a third image channel from the first portion of the digital image data, wherein the generating the second image channel and the third image channel comprises: performing adaptive histogram equalization on the first portion of the digital image data to generate the second image channel; and allocating the first portion of the digital image data as the third image channel, and The first feature map is generated using the first image channel, the second image channel and the third image channel.

2. The method according to claim 1, comprising: performing a difference of Gaussian operation on the first portion of the digital image data to produce a fourth image channel; generating a second feature map using the fourth image channel; as well as A second classifier is applied to the second feature map to classify the embossed and / or engraved markings into one or more second categories.

3. The method of claim 2, wherein the first classifier is a classifier in a first Faster R-CNN network and the second classifier is a classifier in a second Faster R-CNN network, the first Faster R-CNN network being different from the second Faster R-CNN network.

4. The method of claim 3, wherein the first Faster R-CNN network comprises a first region of interest generator, and the method comprises: Using the first region of interest generator, a plurality of first regions of interest are generated on the first feature map, and the first classifier is applied to the generated plurality of first regions of interest.

5. The method of claim 4, wherein the second Faster R-CNN network comprises a second region of interest generator, and the method comprises: Using the second region of interest generator, a plurality of second regions of interest are generated on the second feature map, and the second classifier is applied to the generated plurality of second regions of interest.

6. The method of claim 5, wherein each of the first Faster R-CNN network and the second Faster R-CNN network includes a corresponding bounding box regressor, and the method comprises: Before executing the step of applying the first classifier and / or the second classifier to the generated multiple first regions of interest and / or second regions of interest, using a corresponding bounding box regressor, bounding box regression is performed on the corresponding multiple first regions of interest and / or second regions of interest to refine the position of one or more of the multiple first regions of interest and / or second regions of interest.

7. The method of claim 1, wherein the sidewall of the tire has a curvature, and wherein the providing digital image data comprises correcting the curvature.

8. The method of claim 1, comprising downsampling the digital image data by between 50% and 80%.

9. The method of claim 1, wherein the method is executed on a CPU.

10. The method according to claim 1, wherein the digital image data comprises digital image data of a plurality of images of the sidewall of the tire; in, performing said generating said first image channel, said generating said first feature map, and said applying said first classifier for each respective image of said plurality of images to provide a plurality of classification results identifying into which of said one or more first categories said embossed and / or engraved indicia has been classified, and a corresponding plurality of respective confidence levels for each of said one or more first categories identified; as well as One or more of the identified first categories are determined to be false positives based on the confidence level.

11. The method according to claim 10, further comprising: For each of the one or more identified first categories in each of the classification results, if the confidence level is above a predetermined first threshold, the corresponding confidence level is modified by multiplying by a constant.

12. The method according to claim 11, further comprising: for each of the one or more identified first categories, adding the confidence level to generate a corresponding confidence score; and One or more of the identified first categories are determined to be false positives based on the confidence scores if the corresponding confidence scores are below a predetermined second threshold.

13. A computer-implemented method for generating a training data set for training a classifier to classify one or more embossed and / or engraved markings on a sidewall of a tire into one or more categories, the method comprising: providing a plurality of digital images and corresponding digital image data of respective sidewalls of a plurality of tires, each sidewall having one or more embossed and / or engraved indicia; labeling the corresponding digital image data with one or more first categories; Generating a respective first image channel from a first portion of the respective digital image data associated with a respective first portion of a respective one of the tire sidewalls, wherein the generating comprises: performing histogram equalization on said first portion of said corresponding digital image data to generate said corresponding first image channel, The method further comprises: generating respective second and third image channels from said first portion of said respective digital image data, Wherein generating the second image channel and the third image channel comprises: performing adaptive histogram equalization on the first portion of the corresponding digital image data to generate the second image channel; and The first portion of the corresponding digital image data is assigned as the third image channel.

14. The method according to claim 13, comprising: The corresponding digital image data is labeled with one or more second categories, and wherein the generating comprises: A difference-of-Gaussian operation is performed on the first portion of the corresponding digital image data to produce a fourth image channel.

15. The method of claim 13, wherein respective sidewalls of the plurality of tires have a curvature, and wherein providing the plurality of digital images and respective digital image data comprises correcting the curvature.

16. The method of claim 13, wherein at least one of the plurality of digital images is synthetically generated.

17. The method of claim 13, wherein the corresponding digital image data is downsampled between 50% and 80%.

18. A computer-implemented method for training a classifier to classify one or more embossed and / or engraved markings on a sidewall of a tire into one or more categories, the method comprising: training a first classifier on first training data, the first training data comprising, for each of a plurality of tire sidewalls, corresponding digital image data derived from a digital image of a first portion of the tire sidewall, each sidewall having one or more embossed and / or engraved indicia; wherein the digital image data is labeled with one or more labels, each label indicating that the tire includes a marking in one of a plurality of first categories, The digital image data has a first image channel. wherein the first image channel is generated by performing histogram equalization on the first portion of the corresponding digital image, wherein the digital image data has a second image channel and a third image channel, wherein the second image channel is generated by performing adaptive histogram equalization on the first portion of the digital image to generate the second image channel, and The third image channel is generated by assigning the first portion of the digital image as the third image channel.

19. The method according to claim 18, comprising: training a second classifier on second training data, the second training data comprising, for each of the plurality of tire sidewalls, corresponding second digital image data derived from the digital image of a first portion of the tire sidewall; wherein the digital image data is labeled with one or more second labels, each second label indicating that the tire includes a label in one of a plurality of second categories, Wherein, the digital image data has a fourth image channel, and The fourth image channel is generated by performing a Gaussian difference operation on the first portion of the corresponding digital image.

20. A data processing apparatus comprising means for performing the steps of the method as claimed in claim 1.

21. A computer program comprising a plurality of instructions which, when executed by a computer, cause the computer to perform the steps of the method of claim 1.

22. A computer-readable storage medium having stored thereon the computer program according to claim 21.

Images