Recognition device, recognition method, and recognition program

Abstract

To provide a recognition device, a recognition method, and a recognition program that can improve recognition accuracy for a carved character arranged on a material surface on which flaws or polished marks may occur.SOLUTION: A method comprises: acquiring, through an input unit, an image that captures a surface of an object including a carved character that is associated with a controller connected to the input unit and is carved in the object (S101); generating an object image on the basis of the acquired image (S103); extracting an area including the carved character from the object image on the basis of output from a first learning model to which the object image generated on the basis of the image has been input (S105); recognizing the carved character included in the area on the basis of output from a second learning model to which the area has been input (S107); and outputting a result of recognition (s109).SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a recognition device, a recognition method, and a recognition program. [Background technology]

[0002] Patent Document 1 discloses a technology for determining the normality of engraved characters based on a plurality of different images obtained by illuminating the characters engraved on an object from different directions. This technology performs a filtering process using a spatial filter that calculates the degree of variation in pixel values ​​around each pixel on the plurality of images to generate a plurality of filtered images, and determines the normality of the engraved characters based on the plurality of filtered images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6985771 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology described in Patent Document 1, engraved characters are recognized by applying matching techniques such as template matching to a composite image generated from a filtered image. However, there is a problem in that the surface of the material on which the engraved characters are placed may be scratched or polished, which can reduce the accuracy of the engraved character recognition. In particular, when processing metal surfaces, which are inherently difficult to process, using thermal or cutting methods, the depth of the engraved characters may vary. In addition, when a person handwrites an engraved character, personal habits may be apparent. As a result, there are cases in which engraved character recognition does not work well.

[0005] The present disclosure has been made in view of the above-mentioned problems, and an object of the present disclosure is to provide a recognition device, a recognition method, and a recognition program that can improve the recognition accuracy for characters engraved on material surfaces that are prone to scratches or polishing marks. [Means for solving the problem]

[0006] The recognition device, recognition method, and recognition program according to the present disclosure relate to a controller connected to an input unit. An image of the surface of an object, including the engraved characters engraved on the object, is acquired via the input unit. A target image generated based on the image is inputted from a first learning model, and an area including the engraved characters is extracted from the target image based on the output from the first learning model, and the engraved characters included in the area are recognized based on the output from the second learning model, and the area is inputted.

[0007] The controller may acquire a plurality of images via the input unit and generate a target image based on the plurality of images.

[0008] The multiple images may be obtained by irradiating the surface with light from different directions and capturing the surface.

[0009] The controller may perform photometric stereo based on the multiple images to generate the target image.

[0010] The target image may be a normal vector image that indicates the inclination of the surface, or a curvature image that indicates the curvature of the surface.

[0011] The multiple images may be obtained by imaging the surface under light sources having different peak wavelengths.

[0012] The controller may decompose the image into a plurality of different wavelength components of light and generate the target image based on the plurality of wavelength components.

[0013] The device may further include an output unit connected to the controller, and the controller may output the result of recognizing the engraved characters via the output unit. [Effects of the Invention]

[0014] According to the present disclosure, it is possible to provide a recognition device, a recognition method, and a recognition program that can improve the recognition accuracy for engraved characters placed on the surface of a material where scratches or polishing marks may occur. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram of a recognition system including a recognition device according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram showing the configuration of a recognition device. [Figure 3] 10 is a flowchart showing a processing procedure of the recognition device. [Figure 4] FIG. 10 is a diagram illustrating an example of region extraction. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, several exemplary embodiments will be described with reference to the drawings. Note that common parts in the drawings are given the same reference numerals, and duplicated explanations will be omitted.

[0017] [Configuration of the recognition system] Fig. 1 is a conceptual diagram of a recognition system including a recognition device according to an embodiment of the present disclosure. As shown in Fig. 1, the recognition system includes a light source 5, an imaging device 10, and a recognition device 20. The imaging device 10 is connected to the recognition device 20 via a wireless or wired network.

[0018] The light source 5 irradiates the surface of the object TG with light from one or more directions. The engraved characters engraved on the object TG are located in the area irradiated with light by the light source 5. In Fig. 1, an area LR including the engraved characters is present on the surface of the object TG, and irradiation light R1 is irradiated onto the area including the area LR.

[0019] Here, engraved characters refer to, for example, embossed or engraved characters such as serial numbers. For example, if the object TG is made of metal, there are a wide variety of methods for engraving engraved characters, such as dot marking, which involves stamping dots on the metal surface, and vibro marking, which involves thermally carving the metal surface into the shape of characters. The material of the object TG may be various, such as wood, concrete, or metal, and is not limited to the examples given here.

[0020] The light source 5 may be one or more. The light source 5 may be movable to multiple positions so that light can be irradiated onto the surface of the target object TG from multiple directions. Furthermore, the light source 5 may be installed at multiple positions.

[0021] The light source 5 may be a plurality of light sources having different peak wavelengths. Alternatively, the light source 5 may be a single light source having a plurality of wavelength components. For example, the light source 5 may be a light source capable of emitting white light. The light source 5 is not limited to the examples given here.

[0022] The imaging device 10 captures an image of the surface of the object TG, including the characters engraved on the object TG. Fig. 1 shows how scattered light R2 generated when the object TG scatters irradiated light R1 is incident on the imaging device 10.

[0023] Based on the scattered light R2, the imaging device 10 may acquire a monochromatic multi-tone image (monochrome image) for a single predetermined wavelength, or may acquire a multi-color multi-tone image (color image) for a plurality of predetermined wavelengths. Alternatively, the recognition device 20 may acquire a binarized image instead of a multi-tone image.

[0024] For example, the imaging device 10 has an imaging element such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The color image may be an RGB image that expresses various colors by combining the three primary colors of light, red, green, and blue.

[0025] The imaging device 10 may acquire a plurality of images. Here, the plurality of images may be acquired by irradiating the surface of the object TG with light from different directions and capturing the surface. In other words, when the imaging device 10 captures an image of the surface of the object TG, the direction of irradiating light from the light source 5 may be switched to acquire an image corresponding to each direction.

[0026] The multiple images may also be obtained by capturing images of the surface of the object TG under light sources each having a different peak wavelength. That is, when capturing an image of the surface of the object TG using the imaging device 10, the type of light emitted by the light source 5 may be switched to obtain an image corresponding to each type of light.

[0027] Alternatively, the image captured by the imaging device 10 may include a plurality of different wavelength components of light. For example, when the imaging device 10 captures a multicolor image relating to a plurality of predetermined wavelengths, such as a color image, the resulting image includes a plurality of different wavelength components of light.

[0028] The recognition device 20 processes the image acquired by the imaging device 10 and recognizes the characters engraved on the surface of the target TG.

[0029] [Configuration of recognition device] Fig. 2 is a block diagram showing the configuration of a recognition device. As shown in Fig. 2, the recognition device 20 includes an input unit 21 and a controller 25. The recognition device 20 may also include an output unit 23 and an operation unit 27. The controller 25 is connected to the input unit 21, the output unit 23, and the operation unit 27 so as to be able to communicate with them.

[0030] In addition, the input unit 21, the output unit 23, and the operation unit 27 may be provided in the recognition device 20 itself, or may be installed outside the recognition device 20 and connected to the recognition device 20.

[0031] The input unit 21 acquires an image captured by the imaging device 10. Although Fig. 2 shows the input unit 21 connected to the imaging device 10, it does not necessarily have to be connected to the imaging device 10. For example, the input unit 21 may acquire an image that has been captured in advance by the imaging device 10 and stored in a database (not shown) or the like.

[0032] The output unit 23 outputs the result of recognizing the engraved characters (recognition result) obtained by the controller 25, which will be described later. The output unit 23 may output information determined based on the recognition result together with or instead of the recognition result.

[0033] For example, the output unit 23 may output information related to the object TG that is associated with the recognition result. If the engraved characters are the serial number of the object TG, the output unit 23 may output information such as the name of the object TG and the manufacturing date that are associated with the serial number.

[0034] The operation unit 27 is an input device that allows a user of the recognition device 20 to perform operations. For example, the operation unit 27 is a keyboard, a mouse, a trackball, a touch panel, or the like. The operation unit 27 is not limited to the examples given here. The user's operation content input via the operation unit 27 is transmitted to the controller 25.

[0035] For example, the operation unit 27 may acquire an instruction to specify the area where the engraved characters are engraved based on a user's operation. Also, the operation unit 27 may acquire an instruction to modify the area where the engraved characters are engraved based on a user's operation.

[0036] The controller 25 is a general-purpose computer equipped with a CPU (Central Processing Unit), a memory, and an input / output unit. A computer program (recognition program) for functioning as the recognition device 20 is installed in the controller 25. By executing the computer program, the controller 25 functions as multiple information processing circuits (251, 253, 255) equipped in the recognition device 20.

[0037] In this disclosure, an example is shown in which multiple information processing circuits (251, 253, 255) are realized by software. However, it is also possible to configure the information processing circuits (251, 253, 255) by preparing dedicated hardware for executing each of the information processes described below. Also, the multiple information processing circuits (251, 253, 255) may be configured by individual hardware.

[0038] As shown in FIG. 2, the controller 25 includes a preprocessing unit 251, a region extraction unit 253, and a character recognition unit 255 as a plurality of information processing circuits (251, 253, 255).

[0039] The preprocessing unit 251 acquires an image of the surface of the object TG, including the characters engraved on the object TG, via the input unit 21. Then, a target image is generated based on the acquired image.

[0040] For example, the preprocessing unit 251 may acquire a plurality of images via the input unit 21 and generate a target image based on the plurality of images. More specifically, the preprocessing unit 251 may execute a photometric stereo method based on a plurality of images obtained by irradiating light onto the surface of the target TG from different directions and capturing images of the surface.

[0041] Here, photometric stereo analysis uses known combinations of illumination and reflectance in multiple images to analyze surface curvature and highlight changes in surface normals. Changes in surface normals represent the shape of engraved characters. Changes in surface normals may also represent defects such as cracks, scratches, and dents. However, changes in surface normals caused by defects such as cracks, scratches, and dents are often smaller than changes in surface normals caused by the shape of engraved characters. The target image may be a normal vector image showing the slope of the surface of the object TG, or a curvature image showing the curvature of the surface of the object TG.

[0042] The preprocessing unit 251 may also decompose the acquired image into a plurality of different wavelength components of light and generate an image of the target object based on the plurality of wavelength components. For example, the imaging device 10 may simultaneously irradiate the surface of the target object TG with light from a plurality of light sources having different peak wavelengths, and acquire an image consisting of the plurality of different wavelength components of light via the input unit 21.

[0043] The plurality of different wavelength components of light may be associated with a plurality of images obtained by irradiating the surface of the object TG with light from different directions and capturing the surface. Instead of acquiring a plurality of images, the pre-processing unit 251 may acquire an image having a plurality of different wavelength components of light to generate an object image.

[0044] Alternatively, the preprocessing unit 251 may binarize the acquired image or the target image generated by the above-described processing to generate a new target image. More specifically, the preprocessing unit 251 may generate a target image by performing binarization processing, in which the intensity of the processed pixel is 1 if the light intensity of the pixel included in the image is equal to or greater than a predetermined threshold, and the intensity of the processed pixel is 0 if the light intensity is less than the predetermined threshold. The binarization processing performed by the preprocessing unit 251 is not limited to the examples given here. For example, the preprocessing unit 251 may perform image processing such as binarization to improve visibility and generate an image that emphasizes text portions. Alternatively, the preprocessing unit 251 may generate an image by performing expansion / contraction processing, brightness change, etc. Furthermore, the preprocessing unit 251 may generate a new image using a neural network.

[0045] The area extraction unit 253 extracts an area including the inscription characters from the target image based on the output from the first learning model to which the target image is input. Here, the "first learning model" is a model generated by machine learning based on a data set consisting of a target image for training and an area for training (an area for training including the inscription characters) associated with the target image for training as a correct answer. The structure and generation method of the first learning model will be described later.

[0046] The region extraction unit 253 inputs the target image generated by the preprocessing unit 251 to the first learning model. Note that the region extraction unit 253 may input the image itself acquired via the input unit 21 to the first learning model instead of the target image. Then, the region extraction unit 253 calculates an output from the first learning model.

[0047] The first learning model is a model that has learned the relationship between a teacher's target image and a teacher's area associated with the target image. Therefore, when a target image generated by the preprocessing unit 251 or an acquired image is input to the first learning model, the first learning model outputs the area included in the input image. The area extraction unit 253 extracts the area output from the first learning model as an area including the engraved character.

[0048] The area extraction unit 253 extracts an area that is a part of the target image. The area extracted by the area extraction unit 253 includes one or more engraved characters.

[0049] The area extraction unit 253 may extract areas of various shapes. For example, the area extraction unit 253 may extract a rectangular area corresponding to the arrangement of the engraved characters. The width direction (or height direction) of the rectangular area may be parallel to the width direction (or height direction) of the target image, or may be tilted instead of parallel.

[0050] The character recognition unit 255 recognizes the marking character included in the area based on the output from the second learning model to which the area extracted by the area extraction unit 253 is input. Here, the "second learning model" is a model generated by machine learning based on a data set consisting of the teacher area and the teacher marking character associated with the teacher area as the correct answer. The structure and generation method of the second learning model will be described later.

[0051] Character recognition unit 255 inputs the region extracted by region extraction unit 253 to the second learning model. Then, character recognition unit 255 calculates an output from the second learning model.

[0052] The second learning model is a model that has learned the relationship between the teacher's area and the teacher's marking character associated with the area. Therefore, when the area extracted by the area extraction unit 253 is input to the second learning model, the second learning model outputs the marking character included in the input area. The character recognition unit 255 outputs the marking character output from the second learning model as the recognized marking character. The character recognition unit 255 outputs one or more marking characters included in the area.

[0053] The teacher engraved characters may be characters handwritten by a person or characters engraved by a machine, etc. Teacher engraved characters may be set in accordance with engraved characters that may appear in the image acquired via the input unit 21, and a data set to be used for generating the second learning model may be prepared.

[0054] The character recognition unit 255 may output the results of recognizing the engraved characters via the output unit 23.

[0055] Additionally, the controller 25 may generate information about the object TG that is associated with the recognized engraved characters. The generated information about the object TG may be output via the output unit 23.

[0056] Next, we will note the structure and generation method of the learning models (first learning model and second learning model). In the following, we assume that the learning model is a neural network.

[0057] A neural network includes an input layer (or kernel), an output layer where output values ​​are output, and at least one hidden layer between the input and output layers, with signals propagating through the input layer, hidden layer, and output layer in that order. Each of the input, hidden, and output layers consists of one or more units.

[0058] Units between layers are connected to each other, and each unit has an activation function (e.g., sigmoid function, normalized linear function, softmax function, etc.). A weighted sum is calculated based on multiple inputs to the unit, and the value of the activation function, which uses the sum as a variable, becomes the output of the unit. For example, in machine learning, the weights used when calculating the sum in each unit of a neural network are adjusted as parameters related to the learning model.

[0059] The controller 25 may store, in a memory (not shown), the connection relationships between units in the neural network and the weights used to calculate the sum for each unit of the neural network. Hereinafter, the "connection relationships" and "weights" between units will be referred to as "parameters."

[0060] When performing machine learning to generate a neural network related to a learning model, training data is input to the input layer of the neural network.

[0061] For example, when generating a first learning model, a target image for training is input to the input layer, and when generating a second learning model, a region for training is input to the input layer.

[0062] The parameters of the neural network are adjusted so that the error between the value output from the output layer of the neural network when training data is input to the input layer of the neural network and the correct answer contained in the training data is small.

[0063] For example, when generating the first learning model, the parameters of the neural network are adjusted so that the error between the value output from the output layer and the correct teacher's area (the teacher's area including the engraved character) is reduced. When generating the second learning model, the parameters of the neural network are adjusted so that the error between the value output from the output layer and the correct teacher's engraved character is reduced.

[0064] Alternatively, gradient descent, stochastic gradient descent, etc. may be used to minimize errors related to the output of the neural network. Here, backpropagation may be used to calculate the gradient in gradient descent or stochastic gradient descent.

[0065] Other issues that can arise with machine learning using neural networks are generalization performance (the ability to discriminate against unknown data) and overfitting (a phenomenon in which the learning model adapts to the data used to create it, but generalization performance does not improve).

[0066] Therefore, to mitigate overfitting, techniques such as regularization, which restricts the degrees of freedom of weights during learning, may be used. Other techniques, such as dropout, which probabilistically selects units in a neural network and disables other units, may also be used. Furthermore, to improve generalization performance, techniques such as data regularization, data standardization, and data augmentation, which eliminate bias in the data, may also be used.

[0067] The learning model may be expressed by a support vector machine instead of a neural network. Machine learning using a support vector machine does not have the problem of local solution convergence and tends to improve generalization performance. The learning model is not limited to the examples given here, as long as it is generated from training data by machine learning.

[0068] [Processing procedure of recognition device] FIG. 3 is a flowchart showing the processing procedure of the recognition device.

[0069] In step S101, the pre-processing unit 251 acquires, via the input unit 21, an image of the surface of the object TG, including the characters engraved on the object TG.

[0070] In step S103, the preprocessing unit 251 generates a target image based on the acquired image.

[0071] In step S105, the area extraction unit 253 extracts an area including the stamp character from the target image based on the output from the first learning model to which the target image has been input.

[0072] In step S107, the character recognition unit 255 recognizes the engraved character included in the area based on the output from the second learning model to which the area extracted by the area extraction unit 253 has been input.

[0073] In step S109, the character recognition unit 255 outputs the result of recognizing the engraved characters via the output unit 23.

[0074] [Effects of the embodiment] As described above in detail, the recognition device, recognition method, and recognition program according to the present disclosure relate to a controller connected to an input unit. An image of the surface of an object, including the inscribed characters inscribed on the object, is acquired via the input unit. A target image generated based on the image is inputted from a first learning model, and an area including the inscribed characters is extracted from the target image based on an output from the first learning model, and the inscribed characters included in the area are recognized based on an output from the second learning model, and the inscribed characters included in the area are recognized.

[0075] This improves the recognition accuracy for engraved characters placed on material surfaces where scratches or polishing marks may occur. In particular, by extracting an area that is a part of the target image and recognizing the engraved characters only in the extracted area, it is possible to prevent scratches or polishing marks from being mistakenly recognized as engraved characters.

[0076] FIG. 4 is a diagram showing an example of region extraction. FIG. 4 shows an image PT obtained by capturing an image of the surface of an object TG. In addition to an area LR containing the engraved characters, the image PT may also include scratches ER1, polishing marks ER2, and the like. If the entire image PT is used as the target for recognition of the engraved characters, the scratches ER1 and polishing marks ER2 may be recognized as the engraved characters, resulting in a decrease in the recognition accuracy of the engraved characters. However, by extracting an area LR, which is a portion of the target image, and recognizing the engraved characters using the area LR as the target, the scratches ER1 and polishing marks ER2 are prevented from being recognized as the engraved characters. As a result, the recognition accuracy of the engraved characters can be improved.

[0077] The controller may acquire multiple images via the input unit and generate a target image based on the multiple images. This reduces the influence of noise that occurs during imaging, etc., compared to when a single image is used. Therefore, the target image can more reliably capture the characteristics of the engraved characters. As a result, the recognition accuracy of the engraved characters can be improved.

[0078] The multiple images may be obtained by irradiating the surface with light from different directions and capturing the surface image, thereby enabling the captured images to accurately capture the characteristics of the unevenness of the surface of the object caused by the engraved characters.

[0079] The controller may perform a photometric stereo method based on multiple images to generate an image of the object, which can more reliably capture the characteristics of the engraved characters.

[0080] The target image may be a normal vector image showing the inclination of the surface, or a curvature image showing the curvature of the surface. This makes it possible to more emphasize the unevenness of the target surface caused by the engraved characters compared to the unevenness of the target surface caused by scratches or polishing marks. As a result, it is possible to improve the recognition accuracy for engraved characters placed on material surfaces where scratches or polishing marks are likely to occur. In particular, it is possible to prevent scratches or polishing marks from being mistakenly recognized as engraved characters.

[0081] The multiple images may be obtained by capturing an image of the surface under light sources each having a different peak wavelength. This allows different scattering depending on the wavelength, making it possible to accurately capture the uneven features of the surface of the object caused by the engraved characters. In particular, since multiple images are captured by switching the type of light source, the influence of noise generated during imaging can be reduced. Therefore, the features of the engraved characters can be captured more reliably from the target image. As a result, the recognition accuracy of the engraved characters can be improved.

[0082] The controller may decompose the image into a plurality of different wavelength components of light and generate a target image based on the plurality of wavelength components. This reduces the number of times images are captured, thereby reducing the burden on the user and the work time required to recognize the engraved characters. Furthermore, compared to using a single wavelength component, using a plurality of wavelength components can reduce the influence of noise that occurs during imaging. Therefore, the characteristics of the engraved characters can be more reliably captured from the target image. As a result, the recognition accuracy of the engraved characters can be improved.

[0083] The device may further include an output unit connected to the controller, and the controller may output the result of the recognition of the engraved characters via the output unit, thereby enabling subsequent processing to be performed based on the result of the recognition of the engraved characters.

[0084] Each function described in the above embodiments may be implemented by one or more processing circuits, including programmed processors, electrical circuits, and even devices such as application specific integrated circuits (ASICs), or circuit components arranged to perform the described functions.

[0085] According to the present disclosure, it is possible to automate the recognition of engraved characters, thereby reducing the time and effort required for workers to visually check the engraved characters. As a result, it is possible to improve the labor productivity of workers. Therefore, it is possible to contribute to, for example, Goal 8 of the Sustainable Development Goals (SDGs) led by the United Nations, "Promote inclusive and sustainable economic growth, full and productive employment and decent work for all."

[0086] Although several embodiments have been described, the embodiments can be modified or varied based on the above disclosure. All components of the above embodiments and all features described in the claims may be individually extracted and combined, unless they contradict each other. [Explanation of symbols]

[0087] 5 light source 10. Imaging device 20 Recognition device 21 Input section 23 Output section 25 Controller 27 Control section 251 Pretreatment section 253 Area extraction part 255 Character recognition section

Claims

1. A recognition device comprising an input unit and a controller connected to the input unit, The controller Acquiring an image of a surface of the object, including characters engraved on the object, via the input unit; extracting an area including the inscribed character from the target image based on an output from a first learning model to which the target image generated based on the image is input; Recognizing the inscribed character included in the area based on an output from a second learning model to which the area is input. recognition device.

2. The controller acquiring a plurality of the images via the input unit; generating the target image based on the plurality of images; The recognition device according to claim 1 .

3. The recognition device according to claim 2 , wherein the plurality of images are obtained by capturing images of the surface while irradiating the surface with light from different directions.

4. The recognition device of claim 3 , wherein the controller performs a photometric stereo method based on a plurality of the images to generate the target image.

5. The recognition device according to claim 3 , wherein the target image is a normal vector image indicating a slope of the surface or a curvature image indicating a curvature of the surface.

6. The recognition device according to claim 2 , wherein the plurality of images are obtained by imaging the surface under light sources each having a different peak wavelength.

7. The controller Decomposing the image into a plurality of different wavelength components of light; generating the target image based on the plurality of wavelength components; The recognition device according to claim 1 .

8. further comprising an output unit connected to the controller; The controller outputs the result of recognizing the engraved characters via the output unit. The recognition device according to any one of claims 1 to 7.

9. A recognition method for controlling a controller connected to an input unit, comprising: The controller Acquiring an image of a surface of the object, including characters engraved on the object, via the input unit; extracting an area including the inscribed character from the target image based on an output from a first learning model to which the target image generated based on the image is input; Recognizing the inscribed character included in the area based on an output from a second learning model to which the area is input. Recognition method.

10. A recognition program executed by a controller connected to an input unit, The controller acquiring, via the input unit, an image of a surface of the object, including characters engraved on the object; extracting an area including the inscribed character from the target image based on an output from a first learning model to which the target image generated based on the image is input; A step of recognizing the inscribed character included in the area based on an output from a second learning model to which the area is input; A recognition program that causes the

Images