Image filter generation system, image filter generation device, inference device, inference method, and program

Abstract

The present invention relates to an image filter generation system, an image filter generation device, an inference device, an inference method, and a program, which can reduce erroneous text recognition. An object image data acquisition unit (320) acquires object image data for optical character recognition (OCR). An inference result data generation unit (330) inputs the object image data for OCR into a trained model to generate first inference result data and second inference result data. An inference result data output unit (340) outputs the first inference result data and the second inference result data. An image filter generation unit (130) generates a first image filter based on the first inference result data and a second image filter based on the second inference result data. An image filter output unit (140) outputs the first image filter and the second image filter.

Description

Technical Field

[0001] The present invention relates to an image filter generation system, an image filter generation device, an inference device, an inference method, and a program. Background Art

[0002] Conventionally, it is known that in the so-called OCR (Optical Character Recognition / Reader) for recognizing text written on an object captured by a camera component, a device that performs learning through machine learning when performing image processing on the captured image of the object is used. Patent document 1 discloses an image processing device that uses a neural network for a sequential planning unit, which outputs a sequential plan of an image transformation filter used in image processing. In Patent document 1, a learning control unit causes the neural network to learn using a group of learning data pairs, which are pairs of learning images and sequential patterns (patterns) that can be combined to form a group of image transformation filters. Specifically, the learning control unit causes the neural network to learn by feeding back to the neural network an error, i.e., a loss, obtained based on the following information: a sequential plan output by inputting a learning image contained in a learning data pair into the neural network;

[0003] And the sequential pattern contained in the learning data pair.

[0004] Patent Document 2 discloses an image correction device that generates an appropriate image for an input captured image. In Patent Document 2, the parameters of a spatial filter that sets a small region image segmented from a sample image as the appropriate image are learned as taught values, thereby constructing a statistical learning rule. Furthermore, Patent Document 2 corrects the small region image, which is obtained by segmenting the input captured image, using a spatial filter created based on the parameters output by the statistical learning rule by inputting the pixel values of the pixels contained in the small region image.

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2020-154600

[0006] Patent Document 2: Japanese Patent Application Laid-Open No. 2009-10853 Summary of the Invention

[0007] The devices described in Patent Documents 1 and 2 only infer the most appropriate image filter combination and parameters for processing an input image. Consequently, these devices cannot account for environmental fluctuations that occur during actual OCR operations, such as light entering through factory windows during the morning and evening hours, variations in the position, orientation, and rotation angle of the workpiece being imaged, and individual differences in the workpiece, potentially leading to erroneous text recognition.

[0008] The present invention is proposed in view of the above-mentioned actual situation, and its purpose is to reduce the erroneous recognition of characters.

[0009] To achieve the above-mentioned object, the image filter generation system according to the present invention generates an image filter for use in image processing of article image data prior to OCR, the article image data being image data of an article captured by an imaging device. The image filter generation system comprises: an image filter generation device that generates an image filter; a learning device that learns the relationship between previously acquired article image data and the image filter used for image processing of the article image data; and an inference device that infers an image filter suitable for image processing of the article image data for OCR. The learning device includes a learning data acquisition unit that acquires learning data including article image data and image filter-related data indicating a combination of image filters used for image processing of the article image data and parameter values for each image filter; a trained model generation unit that generates a trained model indicating the relationship between the article image data and the image filter-related data through machine learning using the learning data; and a trained model output unit that outputs the trained model. The inference device includes: an article image data acquisition unit that acquires article image data to be subjected to OCR; an inference result data generation unit that inputs the article image data to be subjected to OCR into a trained model and generates first inference result data and second inference result data different from the first inference result data as inference result data, the inference result data being data indicating a combination of image filters suitable for image processing of the article image data to be subjected to OCR and parameter values for each image filter; and an inference result data output unit that outputs the first inference result data and the second inference result data. The image filter generation device includes: an image filter generation unit that generates a first image filter based on the first inference result data and a second image filter based on the second inference result data; and an image filter output unit that outputs the first image filter and the second image filter.

[0010] Effects of the Invention

[0011] According to the present invention, the inference device generates and outputs first and second inference result data, and the image filter generation device generates and outputs first and second image filters. Therefore, the user can use either the first or second image filter to process object image data, and can select the image filter that yields the highest text recognition score during OCR. As a result, the image filter generation system of the present invention can reduce text misrecognition compared to an image filter generation system that does not generate first and second inference result data. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is an overall explanatory diagram of the image filter generation system according to the first embodiment.

[0013] Figure 2 This is a diagram showing the functional configuration of the image filter generation system according to the first embodiment.

[0014] Figure 3 This is a block diagram showing the hardware configuration of each device according to the first embodiment.

[0015] Figure 4 This is an explanatory diagram of the learning data according to the first embodiment.

[0016] Figure 5 This is a diagram for explaining an outline of a process of outputting estimation result data according to the first embodiment.

[0017] Figure 6 This is a flowchart of the trained model generation process involved in implementation mode 1.

[0018] Figure 7 This is a flowchart of the estimation result data generation process according to the first embodiment.

[0019] Figure 8 This is a flowchart of the image filter generation process according to the first embodiment.

[0020] Figure 9 This is a diagram for explaining the operation of the image filter generation system according to the first embodiment.

[0021] Figure 10 This is a flowchart of the estimation result data generation process according to the second embodiment.

[0022] Figure 11 This is a flowchart of the image filter generation process according to the second embodiment.

[0023] Figure 12 This is a diagram showing the functional structure of a visual sensor according to a third embodiment. DETAILED DESCRIPTION

[0024] Hereinafter, an image filter generation system, an image filter generation device, an estimation device, an estimation method, and a program according to embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, identical or corresponding parts are denoted by the same reference numerals.

[0025] [Implementation Method 1]

[0026] (Regarding Image Filter Generation System 1 According to Embodiment 1)

[0027] An image filter generation system 1 according to the first embodiment of the present invention generates an image filter used for image processing before optical character recognition / reading (OCR) of image data obtained by photographing workpieces such as products and parts produced in a factory.

[0028] like Figure 1 As shown, image filter generation system 1 includes a visual sensor 100, which is an example of an imaging device and an example of an image filter generation device. This image filter generation device generates an image filter used for image processing of captured image data. Furthermore, image filter generation system 1 includes a learning device 200 that learns the relationship between previously acquired object image data, i.e., object image data, and the image filter used for image processing prior to performing OCR on the object image data. Furthermore, image filter generation system 1 includes an inference device 300 that infers an image filter suitable for image processing of the object image data undergoing OCR. Furthermore, image filter generation system 1 includes a storage device 400 that stores data. Visual sensor 100, learning device 200, inference device 300, and storage device 400 can transmit and receive data via a LAN (Local Area Network) (not shown).

[0029] In image filter generation system 1, visual sensor 100 first processes previously captured object image data using a pre-generated image filter to attempt optical character recognition (OCR). Visual sensor 100 then outputs learning data based on the attempted OCR results of the object image data to learning device 200. Learning device 200 then generates a trained model through machine learning using the acquired learning data. Learning device 200 then outputs the generated trained model to storage device 400, storing it there. Inference device 300 then retrieves the training model stored in storage device 400.

[0030] When the visual sensor 100 actually captures an article to be processed for OCR, it outputs the article image data to the inference device 300. Furthermore, the inference device 300 inputs the acquired article image data into a training model, generates inference result data representing the inference result of an image filter adapted for image processing of the article image data, and outputs the inference result data to the visual sensor 100. The visual sensor 100 then generates an image filter based on the acquired inference result data and performs OCR on the article image data using the generated image filter after performing image processing on the object image data.

[0031] (Regarding the Vision Sensor 100 According to Embodiment 1)

[0032] like Figure 2 As shown, the visual sensor 100 includes a camera 110, which is an example of an imaging unit that captures an image of an object. Furthermore, the visual sensor 100 includes an image filter-related data generator 120, which generates image filter-related data representing a combination of multiple types of image filters and the parameters of each image filter. Furthermore, the visual sensor 100 includes an image filter generator 130, which generates image filters; an image filter output unit 140, which outputs image filters; an image processing unit 150, which performs image processing; and an OCR unit 160, which performs optical character recognition (OCR). Furthermore, the visual sensor 100 includes an item image data output unit 170, which outputs item image data; a learning data output unit 180, which outputs learning data; and an inference result data acquisition unit 190, which acquires inference result data.

[0033] (Regarding the Learning Device 200 According to Embodiment 1)

[0034] The learning device 200 is a computer device such as a personal computer, a server computer, or a supercomputer. The learning device 200 includes a learning data acquisition unit 210 that acquires learning data, a trained model generation unit 220 that generates a trained model, and a trained model output unit 230 that outputs the trained model. The trained model generation unit 220 includes a reward calculation unit 221 that calculates a reward (described later) and a value function update unit 222 that updates a value function (described later).

[0035] (About Estimation Device 300 According to Embodiment 1)

[0036] The inference device 300 is a computer device similar to the learning device 200. The inference device 300 includes a trained model acquisition unit 310 for acquiring a trained model, an item image data acquisition unit 320 for acquiring item image data, an inference result data generation unit 330 for generating inference result data, and an inference result data output unit 340 for outputting the inference result data.

[0037] (Regarding the Storage Device 400 According to Embodiment 1)

[0038] The storage device 400 is, for example, a hard disk drive (HDD) or a so-called network attached storage (NAS) connected to a communication network via a LAN. The storage device 400 includes a trained model storage unit 410 that stores the trained model.

[0039] (Regarding the Hardware Configuration of the Learning Device 200 According to Embodiment 1)

[0040] like Figure 3 As shown, the learning device 200 includes a control unit 51 that executes processing according to a control program 59. The control unit 51 includes a CPU (Central Processing Unit). The control unit 51 operates according to the control program 59. Figure 2 The trained model generation unit 220, reward calculation unit 221, and value function update unit 222 shown in the figure function.

[0041] return Figure 3 The learning device 200 includes a main storage unit 52 which is used as a work area for the control unit 51 and which loads the control program 59. The main storage unit 52 includes a RAM (Random Access Memory).

[0042] The learning device 200 also includes an external storage unit 53 that pre-stores a control program 59. The external storage unit 53 supplies the data stored in the program to the control unit 51 in accordance with instructions from the control unit 51, and stores the data supplied from the control unit 51. The external storage unit 53 includes non-volatile memory such as a flash memory, an HDD (Hard Disk Drive), or an SSD (Solid State Drive).

[0043] The learning device 200 also includes an operation unit 54 that is operated by a user. Input information is supplied to the control unit 51 via the operation unit 54. The operation unit 54 includes information input means such as a keyboard, a mouse, and a touch panel.

[0044] The learning device 200 also includes a display unit 55 that displays information input via the operation unit 54 and information output from the control unit 51. The display unit 55 includes a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0045] return Figure 3The learning device 200 has a transceiver 56 for sending and receiving information. The transceiver 56 has information communication components such as a communication network terminal device and a wireless communication device connected to the network. Figure 2 The learning data acquisition unit 210 and the trained model output unit 230 shown in FIG.

[0046] return Figure 3 In the learning device 200 , the main storage unit 52 , the external storage unit 53 , the operation unit 54 , the display unit 55 , and the transceiver unit 56 are all connected to the control unit 51 via the internal bus 50 .

[0047] The learning device 200 realizes the following by using the main storage unit 52, the external storage unit 53, the operation unit 54, the display unit 55 and the transceiver unit 56 as resources. Figure 2 The functions of the above-mentioned units 210, 220 to 222, and 230 are shown. For example, the learning device 200 executes the learning data acquisition step performed by the learning data acquisition unit 210. Furthermore, for example, the learning device 200 executes the trained model generation step performed by the trained model generation unit 220, the reward calculation step performed by the reward calculation unit 221, and the value function update step performed by the value function update unit 222. Furthermore, for example, the learning device 200 executes the trained model output step performed by the trained model output unit 230.

[0048] (Regarding the Hardware Configuration of the Estimation Device 300 According to Embodiment 1)

[0049] In addition, if Figure 3 As shown, the inference device 300 also includes a control unit 51, a main storage unit 52, an external storage unit 53, an operation unit 54, a display unit 55, and a transceiver unit 56, similar to the learning device 200. The control unit 51 operates according to the control program 59. Figure 2 The inference result data generating unit 330 shown in FIG. Figure 2 The trained model acquisition unit 310, the item image data acquisition unit 320, and the inference result data output unit 340 shown in the figure function.

[0050] return Figure 3 The inference device 300 realizes the following by using the main storage unit 52, the external storage unit 53, the operation unit 54, the display unit 55 and the transceiver unit 56 as resources through the control unit 51. Figure 2The functions of the above-mentioned units 310 to 330 are shown. For example, the inference device 300 executes a trained model acquisition step performed by the trained model acquisition unit 310, an item image data acquisition step performed by the item image data acquisition unit 320, an inference result data generation step performed by the inference result data generation unit 330, and an inference result data output step performed by the inference result data output unit 340.

[0051] (Regarding the Hardware Configuration of the Vision Sensor 100 According to Embodiment 1)

[0052] Although not shown in the figure, the visual sensor 100 includes a control unit 51, a main storage unit 52, an external storage unit 53, an operation unit 54, and a transceiver unit 56. The control unit 51 operates according to a control program 59. Figure 2 The image filter associated data generating unit 120, the image filter generating unit 130, the image filter output unit 140, the image processing unit 150, and the OCR unit 160 shown in FIG. Figure 2 The illustrated article image data output unit 170 , learning data output unit 180 , and estimation result data acquisition unit 190 function.

[0053] return Figure 3 The visual sensor 100 uses the main storage unit 52, the external storage unit 53, the operation unit 54 and the transceiver unit 56 as resources by the control unit 51, thereby achieving Figure 2 The functions of the aforementioned units 120 to 190 are shown. For example, the visual sensor 100 executes an image filter-related data generation step by the image filter-related data generation unit 120, an image filter generation step by the image filter generation unit 130, and an image filter output step by the image filter output unit 140. Furthermore, for example, the visual sensor 100 executes an image processing step by the image processing unit 150 and an OCR step by the OCR unit 160. Furthermore, for example, the visual sensor 100 executes an article image data output step by the article image data output unit 170, a learning data output step by the learning data output unit 180, and an inference result data acquisition step by the inference result data acquisition unit 190.

[0054] (Details of the Functional Configuration of the Vision Sensor 100 According to Embodiment 1)

[0055] return Figure 2Camera 110 captures an image of a workpiece, an example of an article, that is, a qualified workpiece within a predetermined allowable range of design values, to generate article image data. Camera 110 can capture images of workpieces conveyed through an actual manufacturing line during manufacturing, through a production line similar to the actual production line, or in an environment simulating the manufacturing environment. Furthermore, when capturing images of workpieces in a simulated environment, camera 110 can capture images of the workpiece multiple times, simulating environmental fluctuations assumed during manufacturing, specifically time periods such as morning, noon, and evening, and the orientation and rotation angle of the conveyed workpiece, thereby generating multiple types of article image data.

[0056] The image filter-related data generation unit 120 generates image filter-related data for image filters used in image processing when attempting to perform OCR on object image data previously acquired from the camera 110. The object image data for which OCR is attempted includes the aforementioned object image data of workpieces actually captured during past manufacturing and object image data captured in a simulated environment. The combinations of multiple types of image filters represented by the image filter-related data are, for example, combinations of multiple types of image filters selected from among known image filters such as binarization filters, dilation filters, contraction filters, smoothing filters, noise removal filters, contour extraction filters, high-pass filters, low-pass filters, shearing filters, and edge enhancement filters. The parameters of each image filter represented by the image filter-related data are, for example, combinations of multiple types of parameter values selected from among known parameters such as threshold values, kernel sizes, gains, maximum values, and minimum values.

[0057] Furthermore, the image filter-related data generation unit 120 may generate image filter-related data based on image filters actually used in past manufacturing, for example. Furthermore, the image filter-related data generation unit 120 may generate image filter-related data by selecting a combination of image filters and parameters of each image filter using, for example, random numbers.

[0058] The image filter generation unit 130 generates an image filter based on the image filter-related data. For example, the following case is discussed: the image filter combination indicated by the image filter-related data is a combination of a noise removal filter and a contour extraction filter, where the parameters of the noise removal filter are the first parameters and the parameters of the contour extraction filter are the second parameters. In this case, the image filter generation unit 130 generates an image filter that combines the noise removal filter with the first parameters set and the contour extraction filter with the second parameters set.

[0059] When the image filter generation unit 130 generates an image filter, the image filter output unit 140 outputs the generated image filter to the image processing unit 150 .

[0060] The image processing unit 150 performs image processing on the article image data using the image filter obtained from the image filter output unit 140 .

[0061] The OCR unit 160 performs OCR on the image-processed article image data and outputs a score which is a value indicating the reliability of character recognition.

[0062] The article image data output unit 170 outputs the article image data obtained from the camera 110 and subjected to OCR to the estimation device 300 .

[0063] The learning data output unit 180 outputs the learning data to the learning device 200. The learning data includes the object image data on which OCR was attempted, and image filter-related data that identifies the image filter used for image processing of the object image data. Furthermore, the learning data includes OCR score data, which represents the score outputted after the OCR attempt was performed on the object image data using an image filter based on the image filter-related data. Therefore, the learning data includes object image data related to workpieces manufactured in the past, image filter-related data, and OCR score data.

[0064] Here, the processing of generating learning data by the visual sensor 100 in order to output the learning data to the learning device 200 by the learning data output unit 180 will be described. Figure 4 As shown, when m is a natural number, the m types of item image data generated by the camera 110 are denoted as IMG-1, IMG-2, ..., and IMG-m. Furthermore, when n is a natural number, the n types of image filter-related data generated by the image filter-related data generation unit 120 are denoted as F / P-001, F / P-002, ..., and F / P-00n. The image filter generation unit 130 generates n types of image filters based on the n types of image filter-related data F / P-001, F / P-002, ..., and F / P-00n. Furthermore, the image processing unit 150 performs image processing on each item image data IMG-1, IMG-2, ..., and IMG-m using n types of image filters. The OCR unit 160 performs OCR on the m×n types of processed item image data, outputting m×n types of OCR score data.

[0065] Here, the OCR score data for the first item image data IMG-1, which has been image-processed and OCR-ed using image filters based on the image filter-related data F / P-001, F / P-002, ..., and F / P-00n, are referred to as IMG-1_F / P-001, IMG-1_F / P-002, ..., and IMG-1_F / P-00n. Furthermore, the OCR score data for the second item image data IMG-2, which has been image-processed and OCR-ed using image filters based on the image filter-related data F / P-001, F / P-002, ..., and F / P-00n, are referred to as IMG-2_F / P-001, IMG-2_F / P-002, ..., and IMG-2_F / P-00n. In addition, the OCR score data of the mth item image data IMG-m, which has been image processed and OCRed using image filters based on the image filter associated data F / P-001, F / P-002, ..., F / P-00n, is set to IMG-m_F / P-001, IMG-m_F / P-002, ..., IMG-m_F / P-00n.

[0066] As a result, the learning data output unit 180 outputs data including m types of object image data IMG-1, IMG-2, ..., IMG-m, n types of image filter associated data F / P-001, F / P-002, ..., F / P-00n, and m×n types of OCR score data IMG-1_F / P-001, IMG-1_F / P-002, ..., IMG-1_F / P-00n, IMG-2_F / P-001, IMG-2_F / P-002, ..., IMG-2_F / P-00n, ..., IMG-m_F / P-001, IMG-m_F / P-002, ..., IMG-m_F / P-00n as learning data.

[0067] return Figure 2 The inference result data acquisition unit 190 acquires the inference result data output from the inference device 300. Furthermore, at this time, the image filter generation unit 130 generates an image filter based on the inference result data, and the image filter output unit 140 outputs the image filter to the image processing unit 150. Furthermore, the image processing unit 150 uses the acquired image filter to perform image processing on the item image data, and the OCR unit 160 performs OCR on the processed item image data.

[0068] (Details of the Functional Configuration of the Learning Device 200 According to Embodiment 1)

[0069] The learning data acquisition unit 210 acquires learning data output from the vision sensor 100. For example, the learning data acquisition unit 210 acquires data including m types of object image data IMG-1, IMG-2, ..., IMG-m, n types of image filter association data F / P-001, F / P-002, ..., F / P-00n, and m×n types of OCR score data IMG-1_F / P-001, IMG-1_F / P-002, ..., IMG-1_F / P-00n, IMG-2_F / P-001, IMG-2_F / P-002, ..., IMG-2_F / P-00n, ..., IMG-m_F / P-001, IMG-m_F / P-002, ..., IMG-m_F / P-00n as learning data.

[0070] The trained model generation unit 220 generates a trained model representing the correlation between the object image data, the image filter association data, and the OCR score data by machine learning using the acquired multiple types of learning data. The trained model generation unit 220 generates a trained model using Q-learning, which is an example of a well-known reinforcement learning algorithm, as machine learning. Here, reinforcement learning refers to machine learning in which an agent, an agent, an acting subject in a certain environment observes the parameters of the environment, i.e., the current state, and determines the action to be taken. In reinforcement learning, the environment changes dynamically due to the agent's actions, and rewards are given to the agent based on the changes in the environment. Furthermore, in reinforcement learning, the agent repeats the above and learns the action plan that obtains the most rewards through a series of actions.

[0071] In addition, in Q learning, the action value based on the action value function as an example of the value function is calculated as the action plan that obtains the most reward. Here, the state of the environment at time t is s t , set the action at time t to a t , will be due to action a t The changed state is set to s t+1 , will be due to the state from s t Change to s t+1 The reward obtained is set to r t+1 , let the discount rate be γ, let the learning coefficient be α, 0<γ≤1, 0<α≤1 holds. And, let the action value function be Q(s t , a t ), the action value function Q(s t , a t ) is expressed by the following formula 1.

[0072] [Formula 1]

[0073]

[0074] In Q-learning, when the action value is Q, if the action a with the highest value at time t+1 is t+1 The action value Q is greater than the action a performed at time t t The action value Q is increased, if action a t+1 The action value Q is less than action a t In other words, in Q learning, in order to make the action a at time t t The action value Q is close to the optimal action value at time t+1, and the action value function Q(s t , a t ) is updated. As a result, the optimal action value Q in a certain environment is propagated to the action value Q in the previous environment in sequence.

[0075] The trained model generation unit 220 substitutes the value based on the object image data included in the learning data into the state s t , and substitute the value of the image filter related data included in the learning data into action a t , thus using Q learning to generate a trained model. In addition, regarding the value of the state s based on the object image data t The substitution can be arbitrary. For example, the numerical value representing the item image data can be set to x, and the predetermined constant can be set to u. In this case, regarding the state s t , s t =u×x holds.

[0076] In addition, regarding the value of the image filter related data to the action a t As long as we can substitute the action value function Q(s t , a t ) and state s t For action a t performs calculations and is able to act based on a t The combination of image filters and the parameters of each image filter can be determined and any substitution can be made. For example, the numerical value representing the image filter related data can be set to y and the predetermined constant can be set to v. In this case, regarding action a t , a t =v×y holds.

[0077] The reward calculation unit 221 calculates the reward r based on the numerical value representing the item image data, the numerical value representing the image filter related data, and the score value based on the OCR score data included in the training data. t+1 For example, when comparing two types of learning data, if the value of the score based on the OCR score data changes due to a change in at least one of the values representing the object image data and the values representing the image filter related data, the reward calculation unit 221 also increases the reward r given. t+1 Specifically, if the score value increases, the reward r increases. t+1 For example, the reward calculation unit 221 gives a reward of +1. On the other hand, if the score value decreases, the reward r is reduced. t+1 , for example, the reward calculation unit 221 assigns a reward of -1.

[0078] For example, let's consider first item image data IMG-1 and two types of image filter-related data F / P-001 and F / P-002. In this case, since the numerical values representing each type of image filter-related data F / P-001 and F / P-002 differ, the scores based on the OCR score data IMG-1_F / P-001 and IMG-1_F / P-002 also differ. Therefore, the reward calculation unit 221 assigns the scores based on the OCR score data IMG-1_F / P-001 and IMG-1_F / P-002 as SC1 and SC2, respectively. If the image filter-related data changes from F / P-001 to F / P-002, a reward of +1 is assigned if (SC2 - SC1) > 0. On the other hand, a reward of -1 is assigned if (SC2 - SC1) ≤ 0.

[0079] The value function update unit 222 updates the reward r calculated by the reward calculation unit 221 based on the reward r t+1 , for the action value function Q(s t , a t ) is updated. In addition, the value function update unit 222 generates an action value function Q(s t , a t ) data as a trained model.

[0080] Each time the learning data acquisition unit 210 acquires learning data from the visual sensor 100, the trained model generation unit 220 repeatedly reports r t+1 Calculation and action value function Q(s t , a t ) is updated. In addition, the trained model generation unit 220 updates the action value function Q(s) by the update formula shown in the above formula 1 each time. t , a t) is updated, generating the updated action value function Q(s t , a t ) data as a trained model.

[0081] The trained model output unit 230 generates the trained model, which represents the action value function Q(s t , a t ) data is output and stored in the storage device 400.

[0082] (Details of the Functional Configuration of the Estimation Device 300 According to Embodiment 1)

[0083] The trained model acquisition unit 310 acquires the trained model stored in the storage device 400 .

[0084] The object image data acquisition unit 320 acquires object image data for OCR output from the vision sensor 100. In this embodiment, the object image data for OCR acquired by the object image data acquisition unit 320 is object image data of a workpiece captured on an actual manufacturing line before OCR is performed by the vision sensor 100. Specifically, the object image data for OCR includes various types of object image data requiring image processing, such as object image data with blurred text on the workpiece, object image data captured in a brightly lit room, and object image data captured in a dark room. Furthermore, the object image data requiring image processing may also include data indicating the probability of the image being captured on an actual manufacturing line.

[0085] The inference result data generating unit 330 inputs the article image data subjected to OCR into the trained model, and generates first inference result data and second inference result data different from the first inference result data as inference result data.

[0086] Here, an overview of the process of outputting first inference result data and second inference result data from a trained model that has received OCR-performed object image data as input will be described. First, the learning data used by the learning device 200 for machine learning includes 4 types of object image data IMG-1, IMG-2, IMG-3, IMG-4, 5 types of image filter associated data F / P-001, F / P-002, F / P-003, F / P-004, F / P-005, and 20 types of OCR score data IMG-1_F / P-001, IMG-1_F / P-002,…, IMG-1_F / P-005, IMG-2_F / P-001, IMG-2_F / P-002,…, IMG-2_F / P-005,…, IMG-4_F / P-001, IMG-4_F / P-002,…, IMG-4_F / P-005.

[0087] In addition, if Figure 5 As shown, the score values based on 20 types of OCR score data IMG-1_F / P-001, IMG-1_F / P-002,…, IMG-1_F / P-005, IMG-2_F / P-001, IMG-2_F / P-002,…, IMG-2_F / P-005,…, IMG-4_F / P-001, IMG-4_F / P-002,…, IMG-4_F / P-005 are 99, 60,…, 0, 70, 10,…, 11,…, 20, 91,…, 91.

[0088] In addition, if Figure 5 As shown, the probability that the item image data subjected to OCR is identical to the first item image data IMG-1 is 9%, the probability that it is identical to the second item image data IMG-2 is 60%, the probability that it is identical to the third item image data IMG-3 is 30%, and the probability that it is identical to the fourth item image data IMG-4 is 1%.

[0089] In this case, if Figure 5As shown, the first item image data IMG-1 achieves the highest score of 99 points when image processing and OCR are performed using an image filter based on the first image filter-related data F / P-001. Furthermore, the second item image data IMG-2 achieves the highest score of 98 points when image processing and OCR are performed using an image filter based on the third image filter-related data F / P-003. Furthermore, the third item image data IMG-3 achieves the highest score of 100 points when image processing and OCR are performed using an image filter based on the second image filter-related data F / P-002. Furthermore, the fourth item image data IMG-4 achieves the highest score of 91 points when image processing and OCR are performed using an image filter based on either the second image filter-related data F / P-002 or the fifth image filter-related data F / P-005.

[0090] Here, for example, we will discuss a case where the trained model is given the conditions that the inference result data is of two types and has a score of 90 or greater from the inference result data generation unit 330. In this case, the trained model first determines which of the two types of image filter-related data sets IMG-1 to IMG-4, each of which has a score of 90 or greater after image processing and OCR, represents the type of candidate combinations of image filter-related data sets with the highest coverage.

[0091] Specifically, the trained model did not have a 100% coverage combination for the two types of image filter-related data. Therefore, the first image filter-related data F / P-001 and the second image filter-related data F / P-002, each with a 75% coverage, were calculated as the first candidate, and the first image filter-related data F / P-001 and the third image filter-related data F / P-003 were calculated as the second candidate. Furthermore, the maximum scores for image processing and OCR using image filters based on the first candidate image filter-related data F / P-001 and F / P-002 were 99 for the first item image data IMG-1, 70 for the second item image data IMG-2, 100 for the third item image data IMG-3, and 91 for the fourth item image data IMG-4. In addition, the maximum values of the scores obtained when performing image processing and OCR using image filters based on the image filter association data F / P-001 and F / P-003 as the second candidate are 99 points for the first item image data IMG-1, 98 points for the second item image data IMG-2, 91 points for the third item image data IMG-3, and 80 points for the fourth item image data IMG-4.

[0092] The trained model then calculates expected scores for the first and second candidates based on the probability of obtaining each item's image data IMG-1 to IMG-4 on an actual manufacturing line, and outputs the candidate with the higher expected score as the inference result data. Specifically, the expected score for the first candidate is 81.82 (99 × 0.09 + 70 × 0.60 + 100 × 0.30 + 91 × 0.01 = 81.82). Meanwhile, the expected score for the second candidate is 95.81 (99 × 0.09 + 98 × 0.60 + 91 × 0.30 + 80 × 0.01 = 95.81). Therefore, the trained model outputs the second candidate, namely the first image filter-related data F / P-001 and the third image filter-related data F / P-003, as the first and second inference result data. As a result, the estimation result data generating unit 330 generates the first image filter related data F / P-001 and the third image filter related data F / P-003 as the first estimation result data and the second estimation result data.

[0093] Furthermore, in this embodiment, the inference result data generation unit 330 generates two types of inference result data: first inference result data and second inference result data. However, the inference result data generation unit 330 may generate three or more types of inference result data. For example, the inference result data generation unit 330 may generate three types of inference result data: first inference result data, second inference result data, and third inference result data.

[0094] In this case, the trained model uses the first image filter-related data F / P-001, the second image filter-related data F / P-002, and the third image filter-related data F / P-003, which have a 100% coverage rate, as the first candidate, and calculates the first image filter-related data F / P-001, the third image filter-related data F / P-003, and the fifth image filter-related data F / P-005 as the second candidate. Furthermore, the maximum scores obtained when image processing and OCR are performed using image filters based on the image filter-related data F / P-001, F / P-002, and F / P-003 as the first candidate are: 99 for the first item image data IMG-1, 98 for the second item image data IMG-2, 100 for the third item image data IMG-3, and 91 for the fourth item image data IMG-4. In addition, for the maximum values of scores when performing image processing and OCR using image filters based on the image filter association data F / P-001, F / P-003, and F / P-005 as the second candidate, the first item image data IMG-1 is 99 points, the second item image data IMG-2 is 98 points, and the third item image data IMG-3 and the fourth item image data IMG-4 are 91 points.

[0095] Therefore, the expected score for the first candidate is 98.62 (99 × 0.09 + 98 × 0.60 + 100 × 0.30 + 91 × 0.01 = 98.62). On the other hand, the expected score for the second candidate is 95.92 (99 × 0.09 + 98 × 0.60 + 91 × 0.30 + 91 × 0.01 = 95.92). Furthermore, in this case, even if the probability of obtaining each item image data IMG-1 to IMG-4 changes, the expected score for the first candidate is higher than the expected score for the second candidate. For example, if the probability of obtaining each item image data IMG-1 to IMG-4 is 25%, the expected score value of the first candidate, 97 points ((99+98+100+91) / 4=97), is higher than the expected score value of the second candidate, 94.75 points ((99+98+91+91) / 4=94.75). Therefore, the trained model outputs the first candidate, namely, the first image filter-related data F / P-001, the second image filter-related data F / P-002, and the third image filter-related data F / P-003, as the first inference result data, the second inference result data, and the third inference result data. As a result, the inference result data generation unit 330 generates the first image filter-related data F / P-001, the second image filter-related data F / P-002, and the third image filter-related data F / P-003 as the first inference result data, the second inference result data, and the third inference result data.

[0096] Furthermore, the inference result data generation unit 330 may not impose the condition of a score of 90 or greater on the trained model. Even in this case, the trained model can output the first and second inference result data by identifying the combination of image filter-related data F / P-001 to F / P-005 that has the highest expected score.

[0097] Furthermore, the item image data acquired by the item image data acquisition unit 320 for OCR may not include data indicating the probability of the image being captured on an actual manufacturing line. In this case, the estimation result data generation unit 330 may determine the combination of image filter-related data F / P-001 to F / P-005 that has the highest expected score, assuming the probability of obtaining the acquired item image data is exactly the same.

[0098] return Figure 2The inference result data output unit 340 outputs the first and second inference result data as the generated inference result data to the visual sensor 100. Thus, in the visual sensor 100, the inference result data acquisition unit 190 acquires the first and second inference result data. Furthermore, the image filter generation unit 130 generates a first image filter based on the first inference result data and a second image filter based on the second inference result data. The image filter output unit 140 outputs the first and second image filters to the image processing unit 150. The image processing unit 150 then performs image processing on the object image data using the image filters, and the OCR unit 160 performs OCR on the object image data after undergoing the image processing.

[0099] (Regarding the trained model generation process involved in Implementation 1)

[0100] Next, the flow chart will be used to explain the operation of generating and outputting the trained model by the learning device 200. Figure 6 First, the learning data acquisition unit 210 acquires new learning data from the visual sensor 100 (step S101). For example, the learning data acquisition unit 210 acquires the training data including Figure 4 The data shown are m types of item image data IMG-1, IMG-2,…, IMG-m, n types of image filter associated data F / P-001, F / P-002,…, F / P-00n, and m×n types of OCR score data IMG-1_F / P-001, IMG-1_F / P-002,…, IMG-1_F / P-00n, IMG-2_F / P-001, IMG-2_F / P-002,…, IMG-2_F / P-00n,…, IMG-m_F / P-001, IMG-m_F / P-002,…, IMG-m_F / P-00n.

[0101] Next, the trained model generation unit 220 generates a trained model through machine learning using the acquired multiple types of learning data. Specifically, the reward calculation unit 221 calculates the reward r based on the object image data, image filter related data, and OCR score data included in the acquired learning data. t+1 Calculation is performed (step S102). For example, for the first item image data IMG-1, when the image filter-related data changes from F / P-001 to F / P-002, if (SC2-SC1)>0, the reward calculation unit 221 assigns a reward of +1. On the other hand, if (SC2-SC1)≤0, the reward calculation unit 221 assigns a reward of -1.

[0102] Next, the value function updating unit 222 calculates the reward r based on the t+1 For the action value function Q(s t , a t ) is updated (step S103). For example, the value function update unit 222 updates the state s based on the value x representing the item image data. t Calculate and adjust the action a based on the value y representing the image filter related data t Then, the value function updating unit 222 updates the action value function Q(s) using the update formula shown in the above formula 1. t , a t ) is updated. Then, the trained model generation unit 220 represents the updated action value function Q(s t , a t ) data, i.e., the trained model, is output to the storage device 400, and the trained model storage unit 410 stores it (step S104), and the processing ends.

[0103] (Regarding the Estimation Result Data Generation Processing According to Embodiment 1)

[0104] Next, the operation of generating and outputting the inference result data by the inference device 300 will be described using a flowchart. Figure 7 The inference result data generation process shown in FIG2 is as follows. First, the trained model acquisition unit 310 acquires the trained model stored in the storage device 400 (step S201). Next, the object image data acquisition unit 320 acquires newly OCR-ed object image data from the vision sensor 100 (step S202). Next, the inference result data generation unit 330 inputs the newly OCR-ed object image data into the trained model to generate first and second inference result data (step S203). Then, the inference result data output unit 340 outputs the generated first and second inference result data to the vision sensor 100 (step S204), concluding the process.

[0105] (Regarding the Image Filter Generation Process According to Embodiment 1)

[0106] Next, the operation of generating and outputting an image filter by the visual sensor 100 will be described using a flowchart. When the power is turned on, the visual sensor 100 starts executing Figure 8The image filter generation process shown in FIG. First, the item image data output unit 170 outputs the OCR-processed item image data to the inference device 300 (step S301). Next, the inference result data acquisition unit 190 acquires the first inference result data and the second inference result data output from the inference device 300 (step S302). Next, the image filter generation unit 130 generates a first image filter based on the first inference result data and a second image filter based on the second inference result data (step S303). Then, the image filter output unit 140 outputs the first image filter and the second image filter to the image processing unit 150 (step S304), terminating the process.

[0107] As described above, according to the image filter generation system 1 according to this embodiment, the visual sensor 100 generates an image filter used in image processing before performing OCR on article image data obtained by imaging an article with the camera 110 .

[0108] Here, for example, we will discuss the case of performing OCR on image data of a workpiece captured by a visual sensor in a factory. In this case, sometimes, due to the workpiece not being placed in the normal position, or the factory being too bright or too dark, or other environmental factors when the workpiece is captured, it is impossible to obtain image data of the object that is easy to recognize text, and text may be misrecognized. Therefore, technicians in the past have attempted to combine various image filters and set the parameters of each image filter manually based on their know-how, preparing multiple types of image filters with high OCR reliability in a specific environment in advance, and using the image filter with the highest OCR score in the current environment. However, the prepared image filters need to take into account the type of workpiece, such as the material, color, and shape of the workpiece, as well as the type and parameters of the combined image filters. Therefore, there is a problem that manual preparation and use require a lot of time.

[0109] In contrast, in the image filter generation system 1 according to this embodiment, the visual sensor 100 automatically generates an image filter taking into account the type of workpiece indicated by the article image data and the type and parameters of the combined image filter indicated by the image filter-related data.

[0110] By adopting the above configuration, the image filter generation system 1 according to the present embodiment can shorten the time from obtaining the object image data to be subjected to OCR to performing image processing using the image filter, compared to manually preparing and applying the image filter.

[0111] Furthermore, according to the image filter generation system 1 according to this embodiment, in the learning device 200, the learning data acquisition unit 210 acquires learning data including object image data and image filter-related data from the visual sensor 100. Furthermore, the trained model generation unit 220 generates a trained model representing the correlation between the object image data and the image filter-related data through machine learning using the learning data, and the trained model output unit 230 outputs the trained model and stores it in the storage device 400.

[0112] In the inference device 300, the item image data acquisition unit 320 acquires the item image data to be subjected to OCR. Furthermore, the inference result data generation unit 330 inputs the trained model, which the trained model acquisition unit 310 has acquired from the storage device 400, into the trained model, generating first and second inference result data. Furthermore, the inference result data output unit 340 outputs the first and second inference result data to the visual sensor 100. Furthermore, in the visual sensor 100, the image filter generation unit 130 generates a first image filter based on the first inference result data and a second image filter based on the second inference result data, and the image filter output unit 140 outputs the first and second image filters.

[0113] Therefore, in the vision sensor 100, the image processing unit 150 can perform image processing on the object image data using the first image filter and the second image filter. Furthermore, the OCR unit 160 can perform OCR on the object image data processed using the first image filter and the second image filter.

[0114] Here, for example, Figure 9As shown, the object image data undergoing new OCR is designated as IMG-0, the first inference result data generated and output by the inference device 300 is designated as F / P-001, and the second inference result data is designated as F / P-002. In this case, the vision sensor 100 generates a first image filter based on the first inference result data F / P-001 and a second image filter based on the second inference result data F / P-002. Furthermore, the vision sensor 100 performs image processing and OCR on the object image data IMG-0 using each image filter. The output OCR score data is designated as IMG-0_F / P-001 and IMG-0_F / P-002, respectively, and the score values based on the OCR score data IMG-0_F / P-001 and IMG-0_F / P-002 are designated as SCA and SCB, respectively. In this case, if (SCA-SCB)>0, the visual sensor 100 adopts the result of text recognition obtained by performing image processing and OCR through the first image filter. On the other hand, if (SCA-SCB)≤0, the visual sensor 100 adopts the result of text recognition obtained by performing image processing and OCR through the second image filter.

[0115] By adopting the above configuration, the vision sensor 100 can select the image filter with the highest text recognition score from the first and second image filters during OCR. Therefore, within the so-called takt time between the time camera 110 captures an image of a workpiece and the time OCR is performed, each time camera 110 captures an image of the workpiece, the vision sensor 100 can select the image filter most appropriate for the object image data undergoing OCR and perform image processing and OCR using that image filter. As a result, the image filter generation system 1 according to this embodiment can reduce text misrecognition compared to an image filter generation system in which the inference device does not generate first and second inference result data.

[0116] Furthermore, in this embodiment, the inference device 300 generates and outputs two types of inference result data, but it may also generate and output three or more types of inference result data. For example, when the inference device 300 generates and outputs three types of inference result data, the visual sensor 100 can select the image filter with the highest score for text recognition when performing OCR from among the first, second, and third image filters. Furthermore, in this case, it is necessary to attempt image processing and OCR using image filters based on the three types of inference result data within the aforementioned takt time. Therefore, when the inference device 300 generates and outputs three or more types of inference result data, the number of types of inference result data must be determined in consideration of the takt time.

[0117] Furthermore, in the image filter generation system 1 according to this embodiment, the learning data acquired by the learning data acquisition unit 210 includes object image data, image filter-related data, and OCR score data. Furthermore, the trained model generation unit 220 generates a trained model representing the correlation between the object image data, image filter-related data, and OCR score data through machine learning using the learning data.

[0118] By adopting the above configuration, inference device 300 can generate and output inference result data with the highest score, inferred as the text recognition output during OCR, by inputting OCR-processed object image data to a trained model. As a result, image filter generation system 1 according to this embodiment can reduce misrecognition of text, compared to an image filter generation system in which the learning device does not generate a trained model representing the relationship between object image data, image filter-related data, and OCR score data.

[0119] Furthermore, according to the image filter generation system 1 of this embodiment, in the learning device 200, the machine learning performed by the trained model generation unit 220 uses the action value function Q(s t , a t ) reinforcement learning. In addition, the trained model generation unit 220 increases the reward r when at least one of the two types of learning data, the object image data and the image filter associated data, changes and the score represented by the OCR score data increases. t+1 Increasing, on the other hand, makes the reward r t+1 Reduce, thus the action value function Q(s t , a t ) is updated. And the trained model generation unit 220 generates an action value function Q(s t , a t ) data as a trained model.

[0120] By adopting the above configuration, the inference device 300 can generate and output inference result data with the highest score, inferred as the text recognition output during OCR, by inputting OCR-processed object image data to a trained model obtained through reinforcement learning based on the scores represented by the OCR score data. As a result, the image filter generation system 1 according to this embodiment can reduce misrecognition of text, compared to an image filter generation system in which the learning device does not generate a trained model obtained through reinforcement learning based on the scores represented by the OCR score data.

[0121] Furthermore, according to the image filter generation system 1 according to the present embodiment, the article image data included in the learning data includes article image data of a workpiece actually photographed during past manufacturing.

[0122] By setting it as described above, compared with an image filter generation system that does not generate a trained model through machine learning using learning data including object image data of workpieces actually photographed during past manufacturing, the image filter generation system 1 involved in this embodiment can reduce the erroneous recognition of text when OCR is performed on object image data of workpieces during actual manufacturing.

[0123] Furthermore, according to the image filter generation system 1 according to the present embodiment, the article image data included in the learning data includes article image data of a workpiece captured in a simulated environment assuming an actual environment.

[0124] By setting it as described above, compared with an image filter generation system that does not generate a trained model through machine learning using learning data including object image data of workpieces photographed in a simulated environment, the image filter generation system 1 involved in this embodiment can reduce the erroneous recognition of text when OCR is performed on object image data of workpieces during actual manufacturing.

[0125] [Implementation Method 2]

[0126] In the first embodiment, the inference device 300 generates and outputs multiple types of inference result data. However, the inference device 300 may not generate and output multiple types of inference result data. In the image filter generation system 1 according to the second embodiment, the inference device 300 generates and outputs only one type of inference result data. Figure 2 、 Figure 5 、 Figure 10 、 Figure 11 The image filter generation system 1 according to Embodiment 2 will be described in detail. In Embodiment 2, the configurations different from those in Embodiment 1 will be described, and the configurations identical to those in Embodiment 1 will be omitted due to redundancy.

[0127] (Details of the Functional Configuration of the Estimation Device 300 According to Embodiment 2)

[0128] return Figure 2 The article image data acquisition unit 320 according to the second embodiment acquires article image data for OCR output from the vision sensor 100. In this embodiment, the article image data for OCR acquired by the article image data acquisition unit 320 is article image data of a workpiece captured on an actual manufacturing production line when the vision sensor 100 performs OCR.

[0129] The inference result data generating unit 330 according to the second embodiment generates inference result data by inputting the OCR-processed article image data into the trained model.

[0130] The estimation result data output unit 340 according to the second embodiment outputs the generated estimation result data to the visual sensor 100 .

[0131] Here, the outline of the process of outputting inference result data from a trained model that inputs object image data for OCR is described. Figure 5 As shown, the first item image data IMG-1 achieves the highest score of 99 points when image processing and OCR are performed using an image filter based on the first image filter-related data F / P-001. Furthermore, the second item image data IMG-2 achieves the highest score of 98 points when image processing and OCR are performed using an image filter based on the third image filter-related data F / P-003. Furthermore, the third item image data IMG-3 achieves the highest score of 100 points when image processing and OCR are performed using an image filter based on the second image filter-related data F / P-002. Furthermore, the fourth item image data IMG-4 achieves the highest score of 91 points when image processing and OCR are performed using an image filter based on either the second image filter-related data F / P-002 or the fifth image filter-related data F / P-005.

[0132] Therefore, for example, if the OCR-ed item image data IMG-0 is most similar to the first item image data IMG-1, the trained model outputs the first image filter-related data F / P-001 as the inference result data. Furthermore, if the OCR-ed item image data IMG-0 is most similar to the second item image data IMG-2, the trained model outputs the third image filter-related data F / P-003 as the inference result data. Furthermore, if the OCR-ed item image data IMG-0 is most similar to the third item image data IMG-3, the trained model outputs the second image filter-related data F / P-002 as the inference result data. Furthermore, if the OCR-ed item image data IMG-0 is most similar to the second item image data IMG-2, the trained model outputs the second image filter-related data F / P-002 or the fifth image filter-related data F / P-005 as the inference result data. As a result, the estimation result data generating unit 330 generates any one of the above-mentioned image filter related data F / P- 001 , F / P- 002 , F / P- 003 , and F / P- 005 as the estimation result data.

[0133] (Regarding the Estimation Result Data Generation Processing According to the Second Embodiment)

[0134] Next, the operation of generating and outputting the inference result data by the inference device 300 will be described using a flowchart. Figure 10 As shown, after executing steps S201 and S202, the inference result data generation unit 330 inputs the newly OCR-processed object image data into the trained model to generate inference result data (step S213). Then, the inference result data output unit 340 outputs the generated inference result data (step S214), and the process ends.

[0135] (Regarding the Image Filter Generation Process According to Embodiment 2)

[0136] Next, the operation of generating and outputting an image filter by the visual sensor 100 will be described using a flowchart. Figure 11 As shown, after executing step S301, the inference result data acquisition unit 190 acquires the inference result data output from the inference device 300 (step S312). Next, the image filter generation unit 130 generates an image filter based on the acquired inference result data (step S313). The image filter output unit 140 then outputs the generated image filter to the image processing unit 150 (step S314), terminating the process.

[0137] As described above, according to the image filter generation system 1 of this embodiment, in the inference device 300, the inference result data generation unit 330 inputs the object image data to be subjected to OCR into the trained model to generate inference result data. Furthermore, in the vision sensor 100, the image filter generation unit 130 generates an image filter based on the inference result data, and the image filter output unit 140 outputs the image filter. Furthermore, the image processing unit 150 uses the image filter to perform image processing on the object image data, and the OCR unit 160 performs OCR on the object image data processed using the image filter.

[0138] By adopting this approach, the vision sensor 100 can use the image filter inferred by the trained model as having the highest score for text recognition during OCR to process the object image data. For example, the vision sensor 100 generates an image filter based on the inference result data F / P-001 and uses this image filter to perform image processing and OCR on the object image data IMG-0 of a workpiece captured on an actual manufacturing production line during OCR operation. Therefore, rather than using two types of image filters for image processing and OCR as in the first embodiment, the result of text recognition with the highest score can be used. As a result, each time the camera 110 captures an image of the workpiece, the vision sensor 100 can obtain the most appropriate inference result data for the object image data undergoing OCR from the inference device 300 and perform image processing and OCR using the image filter based on this inference result data.

[0139] Furthermore, the image filter generation system 1 according to the present embodiment achieves the same operational effects as those of the image filter generation system 1 according to the first embodiment.

[0140] [Implementation Method 3]

[0141] In embodiments 1 and 2, the visual sensor 100, the learning device 200, the inference device 300, and the storage device 400 are separate devices, but this is not limited to this and they may be an integrated device. For example, the image filter generating device, i.e., the visual sensor 100, may also have the functions of the other devices 200, 300, and 400. The visual sensor 100 involved in embodiment 3 may also have the functions of all these learning devices 200, inference devices 300, and storage devices 400. Figure 12 A detailed description will be given of the vision sensor 100 according to Embodiment 3. In Embodiment 3, configurations different from those in Embodiments 1 and 2 will be described, and descriptions of configurations identical to those in Embodiments 1 and 2 will be omitted due to redundancy.

[0142] (Regarding the Vision Sensor 100 According to Embodiment 3)

[0143] like Figure 12As shown, the visual sensor 100 omits the item image data output unit 170, the learning data output unit 180, and the inference result data acquisition unit 190. Furthermore, the visual sensor 100 includes a learning data acquisition unit 210, a trained model generation unit 220, a reward calculation unit 221, a value function update unit 222, an item image data acquisition unit 320, an inference result data generation unit 330, and a trained model storage unit 410. The trained model acquisition unit 310 acquires a trained model stored in the trained model storage unit 410, and the item image data acquisition unit 320 acquires item image data for OCR from the camera 110.

[0144] As described above, the visual sensor 100 according to this embodiment can function as the learning device 200 , the estimation device 300 , and the storage device 400 according to the first and second embodiments.

[0145] By adopting the above configuration, the visual sensor 100 according to this embodiment can achieve the same operational effects as the image filter generation system 1 according to the first and second embodiments.

[0146] [Change Example]

[0147] Furthermore, in the aforementioned embodiment 3, the devices 100, 200, 300, and 400 according to embodiments 1 and 2 are configured as an integrated device. However, the combination of integrated devices is not limited to this. For example, the learning device 200 and storage device 400 according to embodiments 1 and 2 may be configured as an integrated device, with the remaining devices 100 and 300 being separate devices. Alternatively, the inference device 300 and storage device 400 according to embodiments 1 and 2 may be configured as an integrated device, with the remaining devices 100 and 200 being separate devices. Furthermore, for example, the learning device 200, inference device 300, and storage device 400 according to embodiments 1 and 2 may be configured as an integrated device, with only the visual sensor 100 being a separate device.

[0148] Furthermore, in the above-described first and second embodiments, the visual sensor 100, learning device 200, inference device 300, and storage device 400 can transmit and receive data via a LAN, but the data transmission and reception structure is not limited to this. For example, data can be transmitted and received via a communication cable connecting the visual sensor 100, learning device 200, inference device 300, and storage device 400, or data can be transmitted and received via the Internet. In this case, for example, the learning device 200, inference device 300, and storage device 400 can also function as a so-called cloud server. In this case, the cloud server can also generate and store a trained model through machine learning using learning data obtained from the visual sensor 100. Furthermore, in this case, the cloud server can also input newly acquired OCR-processed object image data from the visual sensor 100 into the trained model to generate inference result data and output it to the visual sensor 100.

[0149] Furthermore, in the above-described first to third embodiments, the trained model generation unit 220 generates a trained model using Q-learning, an example of a reinforcement learning algorithm. However, the present invention is not limited thereto and other reinforcement learning algorithms may be used to generate a trained model. For example, the trained model generation unit 220 may also generate a trained model using TD-learning.

[0150] Furthermore, in the above-described first to third embodiments, the trained model generation unit 220 generates a trained model using a reinforcement learning algorithm. However, this is not a limitation and other well-known learning algorithms, such as deep learning, neural networks, genetic programming, functional logic programming, and support vector machines, may also be used to generate a trained model. Furthermore, the learning method is not limited to reinforcement learning and, for example, a trained model may be generated using well-known algorithms for various learning methods, such as supervised learning, unsupervised learning, and semi-unsupervised learning.

[0151] When the trained model generation unit 220 generates a trained model through supervised learning, the learning data must include, for example, correct answer data representing the correct answer to the text that should be recognized in the object image data subjected to OCR. Furthermore, the correct answer data may be manually input in advance or automatically input based on the results of comparing text strings recognized through OCR on multiple object image data.

[0152] Furthermore, when the trained model generation unit 220 generates a trained model through unsupervised learning, the training data must include classification data that allows for classification of each type of item image data requiring image processing, such as item image data with blurred text on the item, item image data captured in a brightly lit room, and item image data captured in a dark room. Furthermore, the training data must include image filter-related data, such as image filter-related data that contains appropriate image filters for image processing of each category of item image data. This image filter-related data must be preselected.

[0153] In addition, when the trained model generation unit 220 generates a trained model through semi-unsupervised learning, the learning data needs to include, for example, the above-mentioned classification data and correct answer data.

[0154] In addition, in the above-mentioned embodiments 1 and 2, the learning device 200 obtains learning data from the visual sensor 100 provided in the image filter generation system 1, and in the above-mentioned embodiment 3, the visual sensor 100 obtains learning data generated by itself, but it is not limited to this. For example, the learning device 200 and the visual sensor 100 may also obtain learning data from other devices or systems that perform OCR. The learning device 200 and the visual sensor 100 may, for example, obtain learning data from multiple image filter generation systems operating in the same area, or may obtain learning data from image filter generation systems operating independently in different areas. In this case, the learning device 200 and the visual sensor 100 may also add or remove other image filter generation systems that obtain learning data at any timing.

[0155] Furthermore, in the above-described first and second embodiments, the learning device 200 pre-installed in the image filter generation system 1 performs machine learning only on the learning data obtained from the visual sensor 100 to generate and output a trained model, but the present invention is not limited to this. For example, a learning device installed in another image filter generation system and performing machine learning by obtaining learning data from the visual sensor may be used as the learning device 200 of the image filter generation system 1, and learning data may be obtained from the visual sensor 100 to perform re-learning, update, and output the trained model.

[0156] Furthermore, in the above-described first and second embodiments, the inference device 300 obtains a trained model generated and output by the learning device 200 provided in the image filter generation system 1 and stored in the storage device 400. However, this is not limiting. For example, the inference device 300 may also obtain a trained model generated and output by another image filter generation device or another image filter generation system.

[0157] Furthermore, the core processing components of the visual sensor 100, learning device 200, and inference device 300, including the control unit 51, main storage unit 52, external storage unit 53, operation unit 54, transceiver unit 56, and internal bus 50, can be implemented using a conventional computer system, rather than relying on a dedicated system. For example, a computer program for executing the aforementioned operations can be distributed by storing it on a computer-readable recording medium, such as a floppy disk or DVD-ROM (Read-Only Memory), and installing this computer program on a computer, thereby configuring the visual sensor 100, learning device 200, and inference device 300 to execute the aforementioned processing. Alternatively, the computer program can be stored on a storage device of a server device on a communication network and downloaded by a conventional computer system, thereby configuring the visual sensor 100, learning device 200, and inference device 300.

[0158] In addition, when the functions of the visual sensor 100, the learning device 200, and the inference device 300 are implemented by sharing the OS (Operating System) and the application, or by collaborative action between the OS and the application, only the application part may be stored in the recording medium or storage device.

[0159] Alternatively, the computer program can be superimposed on a carrier wave and provided via a communication network. For example, the computer program can be published on a bulletin board (BBS) on a communication network and provided via the network. Furthermore, the computer program can be activated and executed under the control of the OS in the same manner as other application programs, thereby performing the aforementioned processing.

[0160] The present invention is capable of various embodiments and variations without departing from the broad spirit and scope of the present invention. Furthermore, the aforementioned embodiments are intended to illustrate the present invention and are not intended to limit the scope of the present invention. That is, the scope of the present invention is not defined by the embodiments but by the claims. Furthermore, any variations implemented within the scope of the claims and their equivalents are considered to be within the scope of the present invention.

[0161] Description of the label

[0162] 1…Image filter generation system, 50…Internal bus, 51…Control unit, 52…Main storage unit, 53…External storage unit, 54…Operation unit, 55…Display unit, 56…Transmitter / receiver unit, 59…Control program, 100…Visual sensor, 110…Camera, 120…Image filter associated data generation unit, 130…Image filter generation unit, 140…Image filter output unit, 150…Image processing unit, 160…OCR unit, 170…Object image data output unit, 180…Learning data According to the output unit, 190...inference result data acquisition unit, 200...learning device, 210...learning data acquisition unit, 220...trained model generation unit, 221...reward calculation unit, 222...value function updating unit, 230...trained model output unit, 300...inference device, 310...trained model acquisition unit, 320...object image data acquisition unit, 330...inference result data generation unit, 340...inference result data output unit, 400...storage device, 410...trained model storage unit.

Claims

1. An inference device for inferring an image filter used for image processing before performing OCR on object image data, the object image data being image data of an object captured by an imaging device. The inference device has: an article image data acquisition unit that acquires the article image data for performing the OCR; an inference result data generating unit that inputs the article image data subjected to the OCR into a trained model generated by machine learning using learning data, and generates first inference result data and second inference result data different from the first inference result data as inference result data, the learning data including image filter association data, which is data indicating a combination of image filters used for image processing of the article image data and parameter values for each image filter, and the article image data acquired in advance; the image filter association data is data indicating a score of text recognition output by the OCR when image processing of the article image data is performed using the image filters based on the image filter association data; the inference result data is data indicating a combination of image filters suitable for image processing of the article image data subjected to the OCR and parameter values for each image filter; and An estimation result data output unit outputs the first estimation result data and the second estimation result data.

2. An inference method for inferring an image filter used for image processing before performing OCR on object image data, the object image data being image data of an object captured by an imaging component, This inference method has: an article image data acquisition step, wherein a computer acquires the article image data for performing the OCR; an inference result data generating step in which the computer inputs the article image data subjected to the OCR into a trained model generated by machine learning using learning data, thereby generating first inference result data and second inference result data different from the first inference result data as inference result data, the learning data including image filter association data, OCR score data, and the previously acquired article image data, the image filter association data being data indicating a combination of image filters used for image processing of the article image data and parameter values for each image filter, the OCR score data being data indicating a score of text recognition output by the OCR when image processing of the article image data is performed using the image filters based on the image filter association data, and the inference result data being data indicating a combination of image filters suitable for image processing of the article image data subjected to the OCR and parameter values for each image filter; and The estimation result data outputting step outputs the first estimation result data and the second estimation result data.

3. A computer-readable recording medium storing a program for causing a computer to function as: an article image data acquisition unit that acquires article image data, which is image data of an article captured by the imaging unit for OCR; an inference result data generating unit that inputs the article image data subjected to the OCR into a trained model generated by machine learning using learning data, and generates first inference result data and second inference result data different from the first inference result data as inference result data, the learning data including image filter association data, OCR score data, and the previously acquired article image data, the image filter association data being data indicating a combination of image filters used for image processing before the OCR of the article image data and parameter values for each of the image filters, the OCR score data being data indicating a score of text recognition output by the OCR when image processing of the article image data is performed using the image filters based on the image filter association data, the inference result data being data indicating a combination of image filters suitable for image processing of the article image data subjected to the OCR and parameter values for each of the image filters; and An estimation result data output unit outputs the first estimation result data and the second estimation result data.

Images