Character information identification method

Abstract

To prevent a person from taking much time to find a position in which letter information is described from an image showing a component.SOLUTION: The method for recognizing letter information causes a computer to execute the steps of: specifying a component by analyzing image data of an image showing the component (S13); searching for the position where letter information is described from the image according to a rule for the specified component (S17); and analyzing the letter information of the obtained description position (S19).SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a character information identification method for identifying character information marked on a part. [Background technology]

[0002] Patent Document 1 discloses an example of a method for converting a text image into a text format using optical character recognition technology. Specifically, in this method, a bounding box of a text region is determined from a picture to be identified, and a text region picture is extracted from the picture to be identified based on the bounding box. Next, a bounding box of a text line is determined from the text region picture, and a text line picture is extracted from the text region picture based on the bounding box. Then, text sequence recognition is performed on the text line picture to obtain a recognition result.

[0003] The classification results are obtained using a lightweight text sequence classification model, which is a trained model that has undergone machine learning. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-197190 Summary of the Invention [Problem to be solved by the invention]

[0005] In the above method, it may take a long time to find the text line picture, which is the part where character information is marked, from the picture to be identified. [Means for solving the problem]

[0006] A text information identification method for solving the above problem is a method for identifying text information marked on a component. The computer has a storage unit that stores rules regarding the position of text information marked on the component in association with the component. The text information identification method causes the computer to perform the following steps: identify the component by analyzing image data of an image containing the component; read the rule corresponding to the identified component from the storage unit; search for the position of the text information in the image according to the rule; and analyze the text information at the searched position.

[0007] In the text information identification method, once a part in an image can be identified, the position of the label can be searched for in the image according to the rule corresponding to the part, thereby reducing the time required to search for the position of the text information label in an image containing the part.

[0008] When having a computer identify parts, a trained model that has undergone machine learning may be used. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram showing a schematic configuration of a recognition device for implementing a character information recognition method according to an embodiment. [Figure 2] FIG. 2 is a schematic diagram showing a spark plug, which is an example of a vehicle part. [Figure 3] In FIG. 3, (a) is a graph showing the relationship between the similarity and the length of the vehicle part, and (b) is a schematic diagram showing how the search position moves in the image. [Figure 4] FIG. 4 is a flowchart showing a character information identification method according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of a character information identification method will be described below with reference to FIGS. The character information identification method of this embodiment is a method for identifying character information marked on a component mounted on a vehicle. Specifically, the character information identification method is a method for identifying the manufacturer or part number of a component that constitutes a vehicle powertrain by identifying character information marked on the component. Examples of target components include spark plugs, starters, and injectors. Hereinafter, such components will be referred to as "vehicle components." Furthermore, the manufacturers of vehicle components identified by the character information identification method are manufacturers and distributors of vehicle components.

[0011] <Identification device> Referring to FIG. 1, a description will be given of a recognition device 10 for implementing the character information recognition method of this embodiment.

[0012] The identification device 10 includes a mounting base 12 , an imaging device 14 , a user interface 16 , and a computer 20 . The installation stand 12 has an installation surface 12a on which the vehicle component 50 is installed. A plurality of vehicle components 50 can be installed on the installation surface 12a.

[0013] An example of the imaging device 14 is a camera. The imaging device 14 captures an image of the vehicle part 50 installed on the installation surface 12a. If multiple vehicle parts 50 are installed on the installation surface 12a, the imaging device 14 can capture images of the multiple vehicle parts 50. The imaging device 14 captures an image of the vehicle part 50, thereby generating an image that shows the vehicle part 50. The imaging device 14 then transmits image data, which is data of the image, to the computer 20.

[0014] Note that if the shadow of the vehicle part 50 is reflected in the image generated by the imaging device 14 capturing an image of the vehicle part 50, the accuracy of identifying the above-described character information may be reduced. Therefore, the identification device 10 may be configured to be able to change the relative position of the imaging device 14 with respect to the installation surface 12a so that the shadow of the vehicle part 50 does not appear in the image.

[0015] The user interface 16 has an operation unit 16a operated by the worker and a notification unit 16b that notifies the worker of various information. The operation unit 16a has at least one of physical buttons or switches and buttons displayed on a display screen having a touch panel. The user interface 16 outputs requests to the computer 20 in response to the worker's operation of the operation unit 16a.

[0016] The notification unit 16b has a display screen. The display screen of the notification unit 16b displays information received from the computer 20. Therefore, when the computer 20 identifies the manufacturer (i.e., the manufacturer or the distributor) of the vehicle part 50, the display screen displays the manufacturer (i.e., the manufacturer or the distributor). Furthermore, when the computer 20 identifies the product number of the vehicle part 50, the display screen displays the product number.

[0017] The computer 20 operates the imaging device 14 in accordance with a request from the user interface 16. The computer 20 also analyzes the image data received from the imaging device 14 to identify the text information marked on the vehicle component 50 on the installation stand 12. The computer 20 then displays the identification result of the text information, such as the manufacturer of the vehicle component 50 (i.e., the manufacturer or distributor) or the product number of the vehicle component 50, on the display screen of the notification unit 16b.

[0018] <Computer> The computer 20 includes a CPU 21, a first storage device 22, a second storage device 23, and a third storage device 24. The first storage device 22 stores a control program CP to be executed by the CPU 21. The CPU 21 executes the control program CP to perform the series of processes shown in FIG.

[0019] <Second storage device> The second storage device 23 stores the correspondence table TL shown in FIG. 1. The correspondence table TL is a table showing the correspondence relationship between the names of vehicle parts and rules regarding the positions where character information is marked on the vehicle parts. Therefore, the second storage device 23 that stores the correspondence table TL corresponds to the "first storage unit." In the correspondence table TL shown in FIG. 1, it is indicated that the rule corresponding to a spark plug is rule R1. It is indicated that the rule corresponding to a starter is rule R2. It is indicated that the rule corresponding to an injector is rule R3. Note that the rules include information that specifies the positions where character information is marked on the corresponding vehicle parts and a method for finding the positions in the image.

[0020] Here, the structure of a spark plug 60, which is an example of a vehicle part, will be described with reference to FIG. 2. The spark plug 60 includes a housing 61 extending in a direction along the axis of the spark plug 60 and a cylindrical portion 62 extending in one direction from the housing 61. An external electrode 63 is provided at the tip of the cylindrical portion 62. A center electrode 64 and a ground electrode 65 are provided on the opposite side of the cylindrical portion 62 across the housing 61. As shown in FIG. 2, in the spark plug 60, text information is marked on a base end 62a of the cylindrical portion 62. The text information includes a logo indicating the manufacturer of the spark plug 60 (i.e., the manufacturer or distributor) and the product number of the spark plug 60. The text information is marked in a portion PS surrounded by a dashed line in FIG. 2.

[0021] Next, with reference to Fig. 3(b), a description will be given of rule R1 corresponding to the spark plug 60. Rule R1 includes the following two points. Text information is marked on the base end 62a of the cylindrical portion 62 of the spark plug 60.

[0022] By moving the search position SPS from the external electrode 63 toward the housing 61 in the image showing the spark plug 60, the base end 62a of the cylindrical portion 62 can be found.

[0023] The diameter of the cylindrical portion 62 is substantially the same at any position along the axis of the spark plug 60. On the other hand, the diameter of the housing 61 is larger than the diameter of the cylindrical portion 62. Therefore, by moving the search position SPS of the cylindrical portion 62 from the external electrode 63 side toward the housing 61 side and searching for the part where the diameter changes, the base end 62a of the cylindrical portion 62 can be found. As a result, even if there are multiple types of spark plugs with cylindrical portions 62 of different lengths, the base end 62a of the cylindrical portion 62 can be found for any spark plug.

[0024] Returning to FIG. 1 , the second storage device 23 stores the manufacturer of the vehicle part 50 (i.e., the manufacturer or distributor) and the product number of the vehicle part 50 manufactured by that manufacturer. For example, suppose that there are two manufacturers of the spark plug 60: ABC Company and DEF Company. Also, suppose that ABC Company produces one type of spark plug, and the product number is "A111." On the other hand, suppose that DEF Company produces multiple types of spark plugs, and the product number of a first spark plug manufactured by DEF Company is "SP2a" and the product number of a second spark plug manufactured by DEF Company is "SP3b."

[0025] The second storage device 23 also stores the manufacturers (that is, the manufacturers or distributors) and product numbers of vehicle parts other than the spark plugs 60. <Third storage device> The third storage device 24 stores a trained model LM, a plurality of templates TP, and a plurality of texts TX. In other words, the third storage device 24 that stores the trained model LM corresponds to the "second storage unit."

[0026] The trained model LM is a learning model that has undergone machine learning to estimate the names of vehicle parts that appear in images captured by the imaging device 14. For example, one example of the trained model LM is an approximator of a multidimensional polynomial. For example, the trained model LM is configured by a fully connected forward propagation type neural network with one hidden layer.

[0027] The trained model LM is a learning model that, when image data is input as an input variable, outputs a first output variable and a second output variable as output variables corresponding to the input variable. The first output variable is a value indicating information for identifying vehicle parts shown in the image. The second output variable is a value corresponding to the number of vehicle parts shown in the image.

[0028] For example, the trained model LM was generated by applying the following machine learning to the training model. Machine learning was applied to the learning model so that when image data of an image showing a spark plug was used as an input variable of the learning model, a value indicating the spark plug could be output as the first output variable.

[0029] When image data of an image containing a starter was used as an input variable of the learning model, machine learning was applied to the learning model so that a value indicating the starter could be output as a second output variable.

[0030] Machine learning was applied to the learning model so that when image data of an image showing two vehicle parts is used as an input variable of the learning model, a value indicating that there are two vehicle parts can be output as the second output variable.

[0031] Machine learning was applied to the learning model so that when image data of an image showing one vehicle part is used as an input variable of the learning model, a value indicating that the number of vehicle parts is one can be output as the second output variable.

[0032] The template TP is used to search for the marking position MPS from an image showing the vehicle part according to rules corresponding to the vehicle part. The marking position MPS is an area in the image where text information about the vehicle part is shown. In this embodiment, the computer 20 searches for the marking position MPS from the image by performing well-known template matching. Therefore, the multiple templates TP include a template for a spark plug, a template for a starter, and a template for an injector.

[0033] The text TX is used to identify the character information at the marking position MPS. For example, as the text TX for a spark plug, there is a logo text for ABC Company and a logo text for DEF Company. Similarly, there is also a text for the spark plug's part number. There are also texts for starters and injectors, for example.

[0034] <Text information identification method> 4 is a flowchart showing the execution procedure of a plurality of processes constituting the character information identification method. When the computer 20 receives image data of an image generated by the imaging device 14 capturing an image of the vehicle component 50, the computer 20 starts the series of processes shown in FIG.

[0035] In step S11, the computer 20 inputs image data of the image generated by the imaging device 14 as an input variable to the trained model LM. In the next step S13, the computer 20 identifies the vehicle parts and the quantity of the vehicle parts shown in the image based on the first output variable and the second output variable of the trained model LM. In other words, the execution of step S13 by the computer 20 corresponds to "having the computer 20 identify the vehicle parts by analyzing the image represented by the image data."

[0036] In step S15, the computer 20 reads out a rule corresponding to the identified vehicle part from the correspondence table TL in the second storage device 23. Then, in step S17, the computer 20 searches for the marking position MPS in the image represented by the image data in accordance with the read-out rule. At this time, the computer 20 searches for the marking position MPS in the image by well-known template matching. In other words, the execution of steps S15 and S17 by the computer 20 corresponds to "reading out a rule corresponding to the identified vehicle part from the second storage device 23, and causing the computer 20 to search for the marking position MPS in the image in accordance with the rule."

[0037] An example of a method for searching for the marking position MPS by template matching will be described with reference to Fig. 3. Here, searching for the marking position MPS of the spark plug 60 will be described as an example. The computer 20 retrieves the spark plug template from the third storage device 24. Then, the computer 20 compares the spark plug 60 shown in the image with the spark plug template. Specifically, as indicated by the dashed arrow in FIG. 3(b), the computer 20 searches for the base end 62a of the tubular portion 62 by moving the search position SPS in the image from the external electrode 63 toward the housing 61. Specifically, the computer 20 searches for the tubular portion 62 of the spark plug 60 based on the similarity X between the diameter of the spark plug 60 at the search position SPS and the diameter of the spark plug's tubular portion in the template. As shown in FIG. 3(a), if the search position SPS is located in a portion other than the tubular portion 62, the similarity X is low. For example, if the search position SPS includes the external electrode 63 of the spark plug 60, the similarity X is less than the similarity judgment value Xth. The similarity judgment value Xth is a criterion for determining whether the search position SPS is the tubular portion 62.

[0038] As shown in FIG. 3(b), when the search position SPS is moved from the external electrode 63 side toward the housing 61, the similarity X changes as shown in FIG. 3(a). That is, when the search position SPS has not reached the cylindrical portion 62, the similarity X is low. When the search position SPS reaches the cylindrical portion 62, the similarity X continues to be equal to or greater than the similarity judgment value Xth. Then, when the search position SPS reaches the housing 61, the similarity X decreases. That is, the similarity X becomes less than the similarity judgment value Xth.

[0039] When the similarity X transitions from a state in which it is equal to or greater than the similarity determination value Xth to a state in which it is less than the similarity determination value Xth, the computer 20 determines that the search position SPS has passed the base end 62a of the cylindrical portion 62. That is, the computer 20 finds the base end 62a of the cylindrical portion 62. In the spark plug 60, text information is marked on the base end 62a of the cylindrical portion 62. This allows the computer 20 to find the marking position MPS in the image.

[0040] Returning to FIG. 4, once the computer 20 has completed searching for the marking position MPS, the process proceeds to step S19. In step S19, the computer 20 reads the character information at the marking position MPS using an OCR, and identifies the manufacturer (i.e., the manufacturer or distributor) and product number of the vehicle part based on the characters read by the OCR. In other words, the execution of step S19 by the computer 20 corresponds to "causing the computer 20 to analyze the character information at the marking position MPS." Note that "OCR" stands for optical character recognition.

[0041] An example of a method for identifying the manufacturer (i.e., manufacturer or distributor) and part number of a vehicle part will be described below. Here, the identification of the manufacturer and part number of a spark plug 60 will be described as an example.

[0042] The computer 20 reads the text for the spark plug from the third storage device 24. The computer 20 reads the character information at the marking position MPS using OCR. The computer 20 then compares the characters read by OCR with the text to identify the manufacturer. As described above, the spark plug manufacturers are ABC Company and DEF Company. For example, if the characters read by OCR are A and C, the computer 20 identifies the manufacturer of the target spark plug as ABC Company. In this case, if the characters read by OCR do not include either the characters indicating ABC Company or the characters indicating DEF Company, the computer 20 cannot identify the manufacturer.

[0043] The computer 20 also identifies the product number of the target spark plug 60 in the same manner as above. When the computer 20 completes the process for identifying the manufacturer (i.e., the manufacturing manufacturer or the selling manufacturer) and the product number, the computer 20 proceeds to step S21. In step S21, the computer 20 determines whether or not the manufacturer (i.e., the manufacturing manufacturer or the selling manufacturer) and the product number have been identified. If the manufacturer has been identified (S21: YES), the computer 20 proceeds to step S23. If the manufacturer has not been identified (S21: NO), the computer 20 proceeds to step S25.

[0044] In step S23, the computer 20 notifies the worker of the identified manufacturer (i.e., the manufacturing manufacturer or the selling manufacturer) and the product number. For example, the computer 20 displays the manufacturer and the product number on the display screen of the notification unit 16b of the user interface 16. Thereafter, the computer 20 ends the series of processes.

[0045] In step S25, computer 20 prompts the worker to visually identify the manufacturer (i.e., the manufacturing manufacturer or the selling manufacturer) and the product number. For example, computer 20 displays a message prompting the worker to visually identify the manufacturer (i.e., the manufacturing manufacturer or the selling manufacturer) and the product number on the display screen of notification unit 16b of user interface 16. Thereafter, computer 20 ends the series of processes.

[0046] <Action and effect> (1) A vehicle is made up of many vehicle parts, and the location where the text information is marked varies for each vehicle part.

[0047] In this embodiment, a correspondence table TL that associates rules corresponding to vehicle parts with vehicle parts is prepared in advance. Then, the vehicle part is identified by analyzing image data of an image generated by capturing an image of the target vehicle part. Once the vehicle part is identified, the rule corresponding to the vehicle part is read from the correspondence table TL. Then, the marking position MPS is searched for in the image indicated by the image data according to the rule. By searching for the marking position MPS in this way according to the rule corresponding to the vehicle part, it is possible to reduce the time required to search for the marking position MPS in the image.

[0048] In this embodiment, once the marking position MPS is found, the character information of the marking position MPS is analyzed, thereby making it possible to identify the manufacturer (i.e., the manufacturer or distributor) and product number of the target vehicle part.

[0049] (2) In this embodiment, by inputting image data of an image showing a vehicle part as an input variable of the trained model LM, the trained model LM outputs information for identifying the vehicle part as a first output variable. Based on this first output variable, the vehicle part installed on the installation stand 12 can be identified.

[0050] (3) In this embodiment, once a vehicle part is identified, a search for the marking position MPS is performed in accordance with the above rules by template matching using a template TP for the vehicle part. This reduces the time required to search for the marking position MPS from an image showing the vehicle part.

[0051] (4) In this embodiment, text TX is prepared for each vehicle part. Therefore, by comparing the characters read by OCR from the character information at the marking position MPS with the text for the vehicle part, the manufacturer (i.e., the manufacturer or distributor) and product number of the vehicle part can be identified.

[0052] <Example of change> This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility.

[0053] By analyzing the character information, it is possible to identify only either the manufacturer (i.e., the manufacturer or distributor) or the product number of the vehicle part. The process for reading the characters that make up the character information at the marking position may be other than OCR.

[0054] When searching for the marking position MPS from an image, methods other than template matching may be used. It is not necessary to identify the number of vehicle parts shown in the image, provided that the vehicle parts shown in the image can be identified.

[0055] If the vehicle parts shown in the images can be identified, the computer 20 may analyze image data of images in which the shadows of vehicle parts are reflected. Neural networks are not limited to feedforward networks with one hidden layer. For example, neural networks can have two or more hidden layers, or they can be convolutional neural networks or recurrent neural networks.

[0056] The trained model using machine learning does not have to be a neural network. For example, a support vector machine can be used as the trained model. The method for identifying vehicle parts from image data may be a method other than using the trained model LM.

[0057] The computer 20 is not limited to a computer having a CPU and ROM and executing software processing. In other words, the computer 20 may have any of the following configurations (a) to (c):

[0058] (a) The computer 20 has one or more processors that execute various processes according to a computer program. The processor includes a CPU and memory such as RAM and ROM. The memory stores program code or instructions configured to cause the CPU to execute processes. Memory, i.e., computer-readable media, includes any available media that can be accessed by a general-purpose or special-purpose computer.

[0059] (b) The computer 20 includes one or more dedicated hardware circuits for performing various processes. Examples of dedicated hardware circuits include application-specific integrated circuits (ASIC) and FPGA. ASIC is an abbreviation for "Application Specific Integrated Circuit," and FPGA is an abbreviation for "Field Programmable Gate Array."

[0060] (c) The computer 20 includes a processor that executes some of the various processes in accordance with a computer program, and dedicated hardware circuits that execute the remaining processes among the various processes.

[0061] The expression "at least one" used herein means "one or more" of the desired options. As an example, the expression "at least one" used herein means "only one option" or "both of two options" if the number of options is two. As another example, the expression "at least one" used herein means "only one option" or "any combination of two or more options" if the number of options is three or more. [Explanation of symbols]

[0062] 10...Identification device, 12...Installation stand, 14...Imaging device, 16...User interface, 20...Computer, 21...CPU, 22...First storage device, 23...Second storage device, 24...Third storage device, 50...Vehicle part, 60...Spark plug, LM...Trained model.

Claims

1. A character information identification method for identifying character information marked on a part, comprising: the computer has a storage unit that stores rules regarding positions at which character information is marked on the components in association with the components, and Identifying the component by analyzing image data of an image in which the component is captured; reading out the rule corresponding to the identified part from the storage unit, and searching for a marking position of the character information from the image in accordance with the rule; and analyzing the character information of the searched mark position; When the storage unit is a first storage unit, the computer has a second storage unit that stores a trained model that has been subjected to machine learning, and the trained model uses the image data as an input variable and outputs information for identifying the part as an output variable; When the part is to be identified, the computer is caused to identify the part based on the output variable of the trained model when the image data is input as an input variable to the trained model. Text information identification method.

2. The part is marked with a logo indicating the manufacturer of the part or the part number as text information, When the character information is analyzed, the character information at the marking position is analyzed to allow the computer to identify the manufacturer of the part or the part number of the part. The character information identification method according to claim 1 .

3. When the character information is analyzed, the computer is caused to identify the manufacturer or product number of the part based on the characters read by OCR from the character information at the marking position. The character information identification method according to claim 2 .

4. A character information identification method for identifying character information marked on a part, comprising: the computer has a storage unit that stores rules regarding positions at which character information is marked on the components in association with the components, and Identifying the component by analyzing image data of an image in which the component is captured; reading out the rule corresponding to the identified part from the storage unit, and searching for a marking position of the character information from the image in accordance with the rule; and analyzing the character information of the searched mark position; The computer is caused to analyze image data of an image in which the shadow of the component is not captured, thereby allowing the computer to identify the component. Text information identification method.

Images