A digital instrument panel reading recognition method and system

Abstract

The present invention discloses a method and system for recognizing readings of a digital instrument panel. The method comprises: acquiring a first image of the digital instrument panel, determining whether a model of the corresponding digital instrument panel exists in a model library; if the model of the corresponding digital instrument panel does not exist in the model library, determining whether a first sample image set exists in an image database; determining whether the number of images in the first sample image set is less than a preset threshold; if so, extracting feature information of the digital instrument panel based on the first image; generating a second image set containing numbers based on the feature information of the digital instrument panel; generating a second sample image set based on the first image and the second image set; training a model based on the first sample image set and the second sample image set; and recognizing the first image based on the trained model. The present invention can quickly complete the recognition of the digital instrument panel and improve the recognition accuracy of the digital instrument panel.

Description

Technical Field

[0001] The present invention belongs to the field of big data analysis and data mining, and in particular relates to a reading recognition method and system for a digital instrument panel. Background Art

[0002] The task of identifying numbers on various instruments is different from the traditional optical character recognition (OCR) problem. It is closer to the text recognition problem in natural scenes. Research in this area can be generally divided into two parts: detection and recognition. Unlike document-related OCR problems, detecting numbers in numbers on various instruments is itself a very difficult task, and many scholars have made many attempts.

[0003] Before the emergence of deep learning methods, methods based on traditional handcrafted features, including connected region-based and HOG-based detection box descriptions, were relatively mainstream. For example, character candidates were obtained through Maximum Stable Extremal Regions (MSERs), and these character candidates were considered vertices of a connected graph. In this case, the process of finding text lines can be regarded as a clustering process, because text from the same text line usually has the same direction, color, font, and shape. The MSER scene text detection and recognition algorithm is implemented in OpenCV.

[0004] Existing digital instrument panel recognition requires collecting a large amount of image data and then training the model to achieve subsequent recognition purposes. This requires a large amount of image data, which leads to the need to collect a large amount of training data in advance. However, in reality, it is relatively difficult to capture and collect a large number of useful images. In many cases, it is impossible to obtain sufficient image data, resulting in an inability to train a model or insufficient accuracy of the trained model. In addition, when switching to a different digital instrument panel, the model needs to be retrained due to the different types, which requires collecting training data again to complete the model training. This method results in a low recognition response rate for the digital instrument panel, and it is impossible to achieve fast recognition.

[0005] Therefore, how to provide a method for quickly completing digital instrument panel recognition with high recognition accuracy has become an urgent problem to be solved. Summary of the Invention

[0006] In view of the above-mentioned defects in the prior art, the present invention provides a method for reading and identifying a digital instrument panel, which can quickly complete the identification of the digital instrument panel and improve the identification accuracy of the digital instrument panel.

[0007] In a first aspect, the present invention provides a method for identifying readings of a digital instrument panel, comprising the following steps:

[0008] Step S101: collecting a first image of the digital instrument panel and determining whether a model of the corresponding digital instrument panel exists in a model library;

[0009] Step S103: If the model of the corresponding digital instrument panel does not exist in the model library, determine whether a first sample picture set exists in the picture database, where the first sample picture set includes all pictures of the corresponding digital instrument panel;

[0010] Step S105: determining whether the number of pictures in the first sample picture set is less than a preset threshold;

[0011] Step S106: If yes, extracting feature information of the digital instrument panel based on the first image;

[0012] Step S107: generating a second picture set containing numbers based on the characteristic information of the digital instrument panel;

[0013] Step S109: generating a second sample picture set based on the first picture and the second picture set;

[0014] Step S111: training the model based on the first sample picture set and the second sample picture set;

[0015] Step S113: Recognize the first image based on the trained model.

[0016] The model library stores trained models, and the trained models directly recognize the collected images.

[0017] If the number of pictures in the first sample picture set is greater than or equal to a preset threshold, the first sample picture set is used to train the corresponding model, and step S113 is executed.

[0018] The preset threshold is defined by the user or based on training accuracy.

[0019] Wherein, the step S106 includes:

[0020] The image size, the shape of the digital instrument panel, the digital font format, the digital font color, and the display angle of the first image are extracted.

[0021] Wherein, the step S107 includes:

[0022] Generate a digital font image with a transparent background based on the digital font format and digital font color;

[0023] generating a second image based on the digital instrument panel appearance and the digital font image;

[0024] The second pictures are enhanced to generate a second picture set containing numbers.

[0025] Wherein, the number is randomly generated.

[0026] The enhancement processing includes: stretching or shrinking the second image to change the size of the image and the shape of the number.

[0027] The enhancement process also includes adding noise processing, specifically including:

[0028] Use the Gaussian distribution function to generate a 3x3 weight matrix;

[0029] Use a bilinear interpolation method to obtain the pixel value of the pixel point P to be processed in the changed image and the pixel values ​​of the eight adjacent pixels around the pixel point P;

[0030] Based on the pixel value of the pixel point P, the pixel values ​​of the eight adjacent pixels around the pixel point P and the weight matrix, the processed pixel value of the pixel point P is obtained.

[0031] In a second aspect, the present invention further provides a system for realizing the above-mentioned method of digital instrument panel reading recognition, comprising:

[0032] A picture acquisition module, which is used to acquire a first picture of the digital instrument panel;

[0033] a feature extraction module, configured to extract feature information of the digital instrument panel based on the first image;

[0034] An image processing module is configured to determine whether a model of a corresponding digital instrument panel exists in a model library, determine whether a first sample image set exists in an image database, and determine whether the number of images in the first sample image set is less than a preset threshold; wherein the first sample image set includes all images of the corresponding digital instrument panel;

[0035] An atlas generation module is configured to generate a second picture set containing numbers based on feature information of the digital instrument panel, and to generate a second sample picture set based on the first picture and the second picture set;

[0036] A model training module, configured to train a model based on the first sample image set and the second sample image set;

[0037] The image recognition model is used to recognize the first image based on the trained model.

[0038] Compared with the existing technology, the present invention can further improve the recognition accuracy of the trained model by generating an additional second image set when the number of the first sample set is insufficient during model training, and by storing the trained model in a model library, the method can quickly and accurately complete the recognition of the digital instrument panel. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an illustrative and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0040] Figure 1 is a flow chart illustrating a method for recognizing readings of a digital instrument panel according to an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram illustrating pixel values ​​within a certain area according to an embodiment of the present invention;

[0042] Figure 3 FIG. 1 is a schematic diagram illustrating a system for recognizing readings of a digital instrument panel according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0044] The terms used in the embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The singular forms "a," "an," "the," and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, and unless the context clearly indicates otherwise, "a plurality" generally includes at least two.

[0045] It should be understood that although the terms "first," "second," "third," etc. may be used to describe "...," these "..." should not be limited to these terms. These terms are merely used to distinguish "...." For example, "first..." could also be referred to as "second...", and similarly, "second..." could also be referred to as "first..." without departing from the scope of the present invention.

[0046] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or device. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or device comprising the element.

[0047] The optional embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0048] Example 1

[0049] like Figure 1 As shown, the present invention provides a method for identifying readings of a digital instrument panel, comprising the following steps:

[0050] Step S101: Capture a first image of a digital instrument panel and determine whether a model of the corresponding digital instrument panel exists in a model library;

[0051] Step S103: If the model of the corresponding digital instrument panel does not exist in the model library, determine whether a first sample picture set exists in the picture database, where the first sample picture set includes all pictures of the corresponding digital instrument panel;

[0052] Step S105: determining whether the number of pictures in the first sample picture set is less than a preset threshold;

[0053] Step S106: If yes, extract feature information of the digital instrument panel based on the first image;

[0054] Step S107: generating a second picture set containing numbers based on the characteristic information of the digital instrument panel;

[0055] Step S109: generating a second sample picture set based on the first picture and the second picture set;

[0056] Step S111: training the model based on the first sample picture set and the second sample picture set;

[0057] Step S113: Recognize the first image based on the trained model.

[0058] Implementation column 2

[0059] Based on the first embodiment, this embodiment may further include the following contents:

[0060] The model library of this embodiment stores trained models. When it is determined in step S101 that the trained model corresponds to the first collected image, the collected image is directly recognized by calling the trained model.

[0061] The embodiment of the present invention can gradually improve the recognition of different digital instrument panels through a continuous model training process, thereby improving the recognition rate and comprehensiveness of this embodiment. In addition, during the process of training the model, the embodiment of the present invention also extracts feature information of the digital instrument panel and generates a second picture set based on the feature information, thereby expanding the sample. The expanded samples can improve the recognition accuracy after model training.

[0062] In step S105 of this embodiment of the present invention, if the number of images in the first sample image set is greater than or equal to a preset threshold, the corresponding model is trained using the first sample image set, and step S113 is executed. Specifically, when the number of images in the first sample image set is greater than or equal to the preset threshold, the model trained using the first sample image set can meet the current accuracy requirement. The preset threshold is user-defined or defined based on training accuracy. Accordingly, this application achieves the desired recognition accuracy by controlling the threshold.

[0063] Example 3:

[0064] Based on the above embodiment, this embodiment may further include the following contents:

[0065] Step S106 of the embodiment of the present invention may include:

[0066] Extract the image size, digital instrument panel shape, digital font format, and digital font color of the first image.

[0067] The method for extracting digital font formats according to the embodiment of the present invention includes the following steps: 1. Searching for liquid crystal fonts on the market and finding fonts similar to the extracted font. 2. Using an image editing tool, drawing a similar font file.

[0068] After extracting feature information, the embodiment of the present invention generates a second picture set through step S107. Step S107 may include:

[0069] Generate digital font images with transparent background based on digital font format and digital font color;

[0070] generating a second image based on the digital instrument panel shape and the digital font image;

[0071] The second image is enhanced to generate a second image set containing numbers, wherein the numbers may be randomly generated.

[0072] The enhancement process of this embodiment may include: performing a stretching or shrinking operation on the generated second image to change the size of the image and the shape of the number.

[0073] In actual application scenarios, changing the size of an image in this embodiment may include the following:

[0074] Get the image size (width, height)

[0075] Use a random number generator to generate two random integers between -7 and 7, named randomX and randomY, and use these two random numbers to calculate the size of the target image.

[0076]

[0077]

[0078] The new image size is (new width, new height)

[0079] In practical application scenarios, after obtaining the new image size, this embodiment knows the pixel values ​​Q11, Q12, Q21, and Q22 of four points in a certain area, and then recalculates the pixel value size of each pixel point in the new image (such as Figure 2 shown).

[0080] Specifically, first interpolate and calculate the pixel value f(x, y1) between Q11 and Q21, and similarly calculate Q12 and Q22. Then interpolate and calculate the value between f(x, y1) and f(x, y2). Specifically, the following interpolation formula is used to calculate the pixel value of the new image pixel:

[0081]

[0082]

[0083]

[0084] Where f(x, y) is the pixel value of the pixel.

[0085] In addition, when processing the generated second image, its display angle may be adjusted, for example, the second image may be rotated at multiple random angles.

[0086] Example 4:

[0087] Based on the above embodiment, this embodiment may further include the following contents:

[0088] In addition to the above, the enhancement processing of this embodiment may also include additional noise processing, specifically including:

[0089] Use the Gaussian distribution function to generate a 3x3 weight matrix;

[0090] Obtain the pixel value of the pixel point P to be processed in the changed image and the pixel values ​​of the eight adjacent pixels around the pixel point P;

[0091] Based on the pixel value of the pixel point P, the pixel values ​​of the eight adjacent pixels around the pixel point P and the weight matrix, the processed pixel value of the pixel point P is obtained.

[0092] Among them, when obtaining the pixel value of the pixel point P to be processed in the changed image and the pixel values ​​of the eight adjacent pixel points around the pixel point P, since a picture is equivalent to a data table, when the position point (i.e., pixel point) of the data table is known, its corresponding data (i.e., pixel value) can be indexed and obtained. The pixel value of any pixel point can be obtained through this method.

[0093] In actual application scenarios, when adding noise processing, the embodiment of the present invention generates a 3x3 weight matrix through the Gaussian distribution function, as shown in the following Table 1:

[0094]

[0095]

[0096] Table 1 Weight matrix

[0097] Assume that the pixel coordinates to be processed are (0,0), and the coordinates of the eight pixels above, below, left, and right are shown in Table 2 below:

[0098] (-1,1) (0,1) (1,1) (-1,0) (0,0) (1,0) (-1,-1) (0,-1) (1,-1)

[0099] Table 2 Pixel coordinate table

[0100] The value of the processed pixel point P is:

[0101] X (0,0) =w (-1，1) *X (-1,1) +w (0,1) *X (0,1) +w (1，1) *X (1,1) +w (-1,0) *X (-1,0) +w (0,0) *X (0,0) +w (1,0) *X (1,0) +w (-1，-1) *X (-1,-1) +w (0，-1) *X (0，-1) +w (1，-1) *X (1，-1)

[0102] Where w is the weight value of the pixel, and X is the pixel value of the pixel.

[0103] After processing each image, the image is added with noise.

[0104] Example 5

[0105] Based on the above embodiment, this embodiment may further include the following contents:

[0106] In actual application scenarios, when this embodiment performs digital instrument panel reading recognition, it mainly adopts the following steps:

[0107] Collect an image of the digital instrument panel to be identified, and select a trained model corresponding to the digital instrument panel in the model library based on the image. If a model corresponding to the digital instrument panel already exists in the model library, the digital instrument panel reading is identified based on the trained model;

[0108] When the model library does not have a trained model, determine whether there are sufficient images corresponding to the digital instrument panel in the database. If so, perform model training on all images of the digital instrument panel and perform digital instrument panel reading recognition based on the trained model.

[0109] When the number of existing digital instrument panel images in the database is insufficient, features such as image size, digital instrument panel shape, digital font format, and digital font color are extracted from the collected digital instrument panel images, and new images are generated based on the features such as image size, digital instrument panel shape, digital font format, and digital font color. The new images are then enhanced to generate a sufficient number of images; when performing the enhancement process, the size and pixel values ​​are mainly processed; wherein the generated images include the shape of the digital instrument panel and the numbers located within the shape;

[0110] The model is trained based on the enhanced images and the existing digital instrument panel images in the original database, and the digital instrument panel reading recognition is performed based on the trained model.

[0111] Example 6

[0112] See also Figure 3 As shown, an embodiment of the present invention further provides a system for digital instrument panel reading recognition implementing the above method, which includes:

[0113] A picture acquisition module, which is used to acquire a first picture of the digital instrument panel;

[0114] a feature extraction module, configured to extract feature information of the digital instrument panel based on the first image;

[0115] An image processing module is configured to determine whether a model of a corresponding digital instrument panel exists in a model library, determine whether a first sample image set exists in an image database, and determine whether the number of images in the first sample image set is less than a preset threshold; wherein the first sample image set includes all images of the corresponding digital instrument panel;

[0116] An atlas generation module is configured to generate a second picture set containing numbers based on feature information of the digital instrument panel, and to generate a second sample picture set based on the first picture and the second picture set;

[0117] A model training module, configured to train a model based on the first sample image set and the second sample image set;

[0118] The image recognition model is used to recognize the first image based on the trained model.

[0119] Example 7

[0120] An embodiment of the present disclosure provides a non-volatile computer storage medium, wherein the computer storage medium stores computer-executable instructions, and the computer-executable instructions can execute the method steps described in the above embodiment.

[0121] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0122] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0123] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0125] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0126] The above introduces the preferred embodiments of the present invention, which is intended to make the spirit of the present invention clearer and easier to understand, and is not intended to limit the present invention. Any modifications, replacements, and improvements made within the spirit and principles of the present invention should be included in the scope of protection outlined by the claims attached to the present invention.

Claims

1. A method for identifying readings of a digital instrument panel, characterized in that: The following steps are involved: Step S101: collecting a first image of the digital instrument panel and determining whether a model of the corresponding digital instrument panel exists in a model library; Step S103: If the model of the corresponding digital instrument panel does not exist in the model library, determine whether a first sample picture set exists in the picture database, where the first sample picture set includes all pictures of the corresponding digital instrument panel; Step S105: determining whether the number of pictures in the first sample picture set is less than a preset threshold; Step S106: If yes, extracting feature information of the digital instrument panel based on the first image, including: Extracting the image size, the shape of the digital instrument panel, the digital font format, and the digital font color of the first image; Step S107: generating a second picture set containing numbers based on the characteristic information of the digital instrument panel, including: Generate a digital font image with a transparent background based on the digital font format and digital font color; generating a second image based on the digital instrument panel appearance and the digital font image; performing enhancement processing on the second picture to generate a second picture set containing numbers; Step S109: generating a second sample picture set based on the first picture and the second picture set; Step S111: training the model based on the first sample picture set and the second sample picture set; Step S113: Recognize the first image based on the trained model.

2. The method according to claim 1, wherein: The model library stores trained models, and the trained models directly recognize the collected images.

3. The method according to claim 1, wherein: If the number of pictures in the first sample picture set is greater than or equal to a preset threshold, the first sample picture set is used to train the corresponding model and step S113 is executed.

4. The method according to claim 1, wherein: The preset threshold is user-defined or defined based on training accuracy.

5. The method according to claim 1, wherein: The numbers are randomly generated.

6. The method according to claim 5, wherein: The enhancement process includes: performing a stretching or shrinking operation on the second image to change the size of the image and the shape of the number.

7. The method according to claim 6, wherein: The enhancement process also includes adding noise processing, specifically including: Use the Gaussian distribution function to generate a 3x3 weight matrix; Obtain the pixel value of the pixel point P to be processed in the changed image and the pixel values ​​of the eight adjacent pixels around the pixel point P; Based on the pixel value of the pixel point P, the pixel values ​​of the eight adjacent pixels around the pixel point P and the weight matrix, the processed pixel value of the pixel point P is obtained.

8. A system for realizing digital instrument panel reading recognition according to any one of claims 1 to 7, characterized in that: include: A picture acquisition module, which is used to acquire a first picture of the digital instrument panel; a feature extraction module, configured to extract feature information of the digital instrument panel based on the first image; An image processing module is configured to determine whether a model of a corresponding digital instrument panel exists in a model library, determine whether a first sample image set exists in an image database, and determine whether the number of images in the first sample image set is less than a preset threshold; wherein the first sample image set includes all images of the corresponding digital instrument panel; An atlas generation module is configured to generate a second picture set containing numbers based on feature information of the digital instrument panel, and to generate a second sample picture set based on the first picture and the second picture set; A model training module, configured to train a model based on the first sample image set and the second sample image set; The image recognition model is used to recognize the first image based on the trained model, wherein Generating a second picture set containing numbers based on the characteristic information of the digital instrument panel includes: Generate a digital font image with a transparent background based on the digital font format and digital font color; generating a second image based on the digital instrument panel appearance and the digital font image; The second pictures are enhanced to generate a second picture set containing numbers.

Images