OPTICAL INFORMATION READING METHOD, OPTICAL INFORMATION READING DEVICE, AND PROGRAM

Abstract

To appropriately adjust imaging conditions regardless of the environment in which information to be read is placed when analyzing a captured image and reading optical information contained in the image. [Solution] When an image is periodically captured while irradiating an aimer light that serves as a reference for directing an imaging unit to optical information to be read, and the image is analyzed to read the optical information contained in the image, the amount of movement of the imaging unit within a certain time range is estimated based on an image captured within the time range, and imaging conditions are adjusted based on pixel values ​​of pixels in the captured image by imaging adjustment process 212. When it is determined that the imaging aim is set based on the estimated amount of movement (Yes in S101), imaging conditions are adjusted based on pixel values ​​of pixels within a predetermined range near the position of the aimer light in the captured image (S102 to S106, S113).

Description

[Technical field]

[0001] The present invention relates to an optical information reading method for reading optical information such as code symbols contained in a captured image, an optical information reading device for reading such optical information, and a program for causing a computer to execute the optical information reading method. [Background technology]

[0002] 2. Description of the Related Art Optical information reading devices are known that capture an image of an object to be read using an imaging unit such as a camera, and read optical information such as code symbols and characters contained in the resulting image. In general, such optical information reading devices start capturing an image when a trigger to start reading is given, and then perform various processes for reading information on the obtained image data. However, if the operator is unable to align the image properly and the captured image does not contain the optical information that should be read, or if the operator moves the device and the captured image is blurred, the information may not be able to be read or may be misread.

[0003] As a reading control technology that takes such a situation into consideration, Patent Document 1 discloses an optical information reading device that outputs the decoding result by a decoding unit when the number of times that the amount of change in a numerical value for a specific part of an area occupied by optical information in an captured image falls below a predetermined value exceeds a predetermined number. Patent Document 2 discloses a barcode reader device that automatically decodes a barcode only when the barcode stops at a proper position.

[0004] Patent Document 3 discloses an optical information reader that does not perform a decode process on image data of an image that has been captured after a capture instruction has been given and before a waiting period has elapsed. Patent Document 4 discloses an optical information reading device that compares the total number of both light and dark patterns that make up a light and dark pattern string in a binary signal captured within a specified period, between two or more consecutive specified periods, and performs decoding if the result satisfies specified stability conditions.

[0005] As prior art from a different perspective, Patent Document 5 discloses an optical information reading device that changes control conditions related to reading by a reading means when it is determined that the reading of an information code by the reading means has failed and the optical information reading device has remained within a predetermined range for a predetermined period of time. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6638614 [Patent Document 2] Japanese Patent Application Publication No. 5-324898 [Patent Document 3] Patent No. 3918713 [Patent Document 4] Patent No. 3944997 [Patent Document 5] Patent No. 6065719 [Patent Document 6] Patent No. 5381928 [Patent Document 7] Patent No. 4175223 [Patent Document 8] Patent No. 3632578 Summary of the Invention [Problem to be solved by the invention]

[0007] The techniques described in Patent Documents 1 to 4 are generally considered to decode when the reading device judges that it is expected to obtain input of a decodable image, etc. However, there is room for improvement in the criteria for this judgment.

[0008] In addition, when the target to be read is a variety of things such as paper wrapping, the LCD screen of a device, or a metal surface, prior to reading the code symbol or character, an analysis process is performed to determine what the target is, and filtering is performed according to the analysis results, so it may take some time from capturing the image to reading it. Considering such cases, for example, if the optical information reading device were to start decoding only after determining that it was in a state where it could expect to obtain input such as an image from which code symbols, characters, etc. can be read, it would take a long time to obtain the decoded results.

[0009] In order to output the reading result in a short time, it is also important to adjust the imaging conditions so that an image of appropriate brightness is obtained so that reading does not fail due to poor image quality. This adjustment can be performed based on the image obtained in the most recent imaging so that an image with conditions that are thought to make reading more likely to be successful is obtained. However, depending on the environment in which optical information is read, the brightness of the area in which the optical information to be read exists may be completely different from that of the surrounding area. For example, there may be a case where the information to be read is printed on a piece of paper, and behind the paper is a window through which bright light shines. In this case, if both the paper and the window are within the imaging range, the window area is expected to be imaged extremely brightly and have a large area, while the area of ​​the information to be read is expected to be imaged darkly in shadow. In such cases, adjusting the imaging conditions based on the brightness of the entire image would not produce an image suitable for reading the information. However, if the information to be read is not at an appropriate brightness, it is not easy to determine where the information is located even if the image is analyzed, and it is also difficult to adjust the imaging conditions based on the brightness of the area near the information to be read. The present invention has been made in consideration of the above circumstances, and has an object to make it possible to appropriately adjust the imaging conditions when analyzing a captured image and reading the optical information contained in the image, regardless of the environment in which the information to be read is located. [Means for solving the problem]

[0010] A first aspect of the present invention provides the following optical information reading method, which aims to shorten the time from successfully capturing an image from which the optical information to be read can be read to outputting the reading result when analyzing a captured image and reading optical information contained in the image. This optical information reading method includes an imaging procedure for periodically capturing an image using an imaging unit, a reading procedure for analyzing the image captured in the imaging procedure and reading optical information contained in the image, and an estimation procedure for estimating the amount of movement of the imaging unit during a certain time range based on the image captured in the imaging procedure during the time range. It is preferable to further include a reading control procedure, and to determine whether the amount of movement estimated in the estimation procedure is equal to or less than a predetermined first standard. Then, if it is determined that the amount of movement is equal to or less than the first standard, the reading procedure is continued as is to decode optical information if the image being analyzed in the reading procedure is an image captured within the certain time range, and if the image being analyzed in the reading procedure is an image captured before the certain time range, the reading procedure is newly executed to analyze an image captured in the imaging procedure within or after the certain time range and read optical information contained in the image.

[0011] In such an optical information reading method, the estimation of the movement amount of the imaging unit in the estimation procedure may include estimating a movement range of the imaging unit in the certain time range based on three or more images captured in the imaging procedure in the certain time range. Also, the first criterion may be that the movement range estimated in the estimation procedure falls within a predetermined convergence range. Alternatively or in addition, the estimation of the movement amount of the imaging unit in the estimation procedure may include estimation of an average movement amount of the imaging unit per frame in the certain time range based on three or more images captured in the imaging procedure in the certain time range. Also, the first criterion may be that the average movement amount estimated in the estimation procedure is equal to or less than a predetermined threshold.

[0012] In addition, each of the optical information reading methods may include an irradiation step of irradiating an aimer light as a reference for directing the imaging unit to the optical information to be read, and an aimer light detection step of detecting the position of the aimer light in the image captured in the imaging step.Furthermore, the method may include an imaging adjustment step of adjusting the imaging conditions of the imaging unit based on the pixel values ​​of pixels in the image captured in the imaging step. The imaging adjustment procedure may be a procedure of adjusting imaging conditions of the imaging unit based on pixel values ​​of pixels within a predetermined range near the position of the detected aimer light in the image captured in the imaging procedure when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than the first reference value.

[0013] Alternatively, the imaging adjustment procedure may be a procedure of adjusting imaging conditions of the imaging unit based on pixel values ​​of pixels around the position of the aimer light in an image captured in the imaging procedure, and when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than the first reference, the imaging conditions of the imaging unit may be adjusted based on pixel values ​​of pixels in a narrower range around the position of the aimer light compared to when it is determined that the amount of movement estimated in the estimation procedure is not equal to or less than the first reference. Alternatively, the method may include an imaging adjustment procedure for adjusting imaging conditions of the imaging unit based on pixel values ​​of pixels at a predetermined reference position in an image captured in the imaging procedure, and the imaging adjustment procedure may be a procedure for adjusting the imaging conditions of the imaging unit based on pixel values ​​of pixels in a narrower range of the image when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than the first reference, compared to when it is determined that the amount of movement estimated in the estimation procedure is not equal to or less than the first reference.

[0014] In addition, each of the optical information reading methods may include an imaging adjustment procedure for adjusting imaging conditions of the imaging unit, including at least one of exposure time and illumination time. The estimation of the movement amount of the imaging unit in the estimation procedure may include estimation of the movement amount of the imaging unit per frame based on images captured in the imaging procedure in all or part of the certain time range. The upper limit of the value of the at least one item set in the imaging adjustment procedure may be determined based on the estimated movement amount of the imaging unit per frame. Furthermore, it is preferable to include a step of acquiring an allowable amount of image blur when reading the optical information, and to determine an upper limit of the value of at least one of the items set in the imaging adjustment step based on the estimated amount of movement of the imaging unit per frame and the acquired amount of blur.

[0015] The present invention also provides an optical information reading method including an imaging procedure for periodically capturing images by an imaging unit, an estimation procedure for estimating a movement range of the imaging unit in a certain time range based on three or more images captured in the imaging procedure within the time range, and a reading procedure for analyzing images captured in the imaging procedure within or after the time range to read optical information contained in the images when it is determined that the movement range estimated in the estimation procedure falls within a predetermined convergence range.

[0016] In order to output the reading result in a short time, it is also important to adjust the imaging conditions so that an image of appropriate brightness is obtained so that reading does not fail due to poor image quality. This adjustment can be performed based on the image obtained in the most recent imaging so that an image with conditions that are thought to make reading more likely to be successful is obtained.

[0017] However, depending on the environment in which optical information is read, the brightness of the area in which the optical information to be read exists may be completely different from that of the surrounding area. For example, there may be a case where the information to be read is printed on a piece of paper, and behind the paper is a window through which bright light shines. In this case, if both the paper and the window are within the imaging range, the window area is expected to be imaged extremely brightly and have a large area, while the area of ​​the information to be read is expected to be imaged darkly in shadow.

[0018] In such cases, adjusting the imaging conditions based on the brightness of the entire image would not produce an image suitable for reading the information. However, if the information to be read is not at an appropriate brightness, it is not easy to determine where the information is located even if the image is analyzed, and it is also difficult to adjust the imaging conditions based on the brightness of the area near the information to be read. In view of the above circumstances, a second aspect of the present invention provides the following optical information reading method, which aims to enable appropriate adjustment of imaging conditions when analyzing a captured image and reading optical information contained in the image, regardless of the environment in which the information to be read is located.

[0019] This optical information reading method includes an imaging procedure for periodically capturing an image using an imaging unit, a reading procedure for analyzing the image captured in the imaging procedure and reading optical information contained in the image, and an estimation procedure for estimating the amount of movement of the imaging unit during a certain time range based on the image captured in the imaging procedure during the time range. Furthermore, it is preferable to include an irradiation procedure for irradiating aimer light that serves as a reference for directing the imaging unit to the optical information to be read, an aimer light detection procedure for detecting the position of the aimer light in the image captured in the imaging procedure, and an imaging adjustment procedure for adjusting the imaging conditions of the imaging unit based on the pixel values ​​of pixels in the image captured in the imaging procedure. In addition, the imaging adjustment procedure may be a procedure of adjusting imaging conditions of the imaging unit based on pixel values ​​of pixels within a predetermined range near the position of the detected aimer light in the image captured in the imaging procedure when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than a predetermined first standard.

[0020] Alternatively, the imaging adjustment procedure may be a procedure of adjusting imaging conditions of the imaging unit based on pixel values ​​of pixels around the position of the aimer light in an image captured in the imaging procedure, and when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than a predetermined first standard, the imaging conditions of the imaging unit may be adjusted based on pixel values ​​of pixels in a narrower range around the position of the aimer light compared to when it is determined that the amount of movement estimated in the estimation procedure is not equal to or less than the first standard.

[0021] Alternatively, in addition to the above imaging procedure, reading procedure, and estimation procedure, an imaging adjustment procedure may be provided for adjusting imaging conditions of the imaging unit based on pixel values ​​of pixels at a predetermined reference position in an image captured in the imaging procedure, and the imaging adjustment procedure may be a procedure for adjusting the imaging conditions of the imaging unit based on pixel values ​​of pixels in a narrower range of the image when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than a predetermined first standard, compared to when it is determined that the amount of movement estimated in the estimation procedure is not equal to or less than the first standard.

[0022] Furthermore, when adjusting the brightness of an image, if reading fails under certain conditions, it is not easy to distinguish whether the failure is due to inadequate brightness, blurring of the image, or the absence of optical information to be read within the imaging range in the first place. Therefore, even if the imaging conditions are adjusted just because reading has failed, it does not necessarily mean that reading will be successful. For example, if one of the typical causes of a reading failure is an image that is too dark, and if adjustments were made to obtain a brighter image by increasing the exposure time or the lighting time whenever a reading failure occurs, then in an environment that is already bright, the image captured would suffer from halation, which could actually hinder reading.

[0023] In view of the above circumstances, a third aspect of the present invention provides the following optical information reading method, which aims to enable appropriate adjustment of imaging conditions with a small number of trials when analyzing a captured image and reading optical information contained in the image, regardless of the environment in which the information to be read is located.

[0024] This optical information reading method includes an imaging procedure for periodically capturing an image using an imaging unit, a reading procedure for analyzing the image captured in the imaging procedure and reading optical information contained in the image, and an estimation procedure for estimating the amount of movement of the imaging unit during a certain time range based on the image captured in the imaging procedure during the time range. Furthermore, it is preferable to provide an imaging adjustment procedure for adjusting the imaging conditions of the imaging unit based on pixel values ​​of pixels in an image captured in the imaging procedure so that the brightness of an image obtained in a subsequent imaging operation will be a predetermined target level, and a target adjustment procedure for changing the target level in the imaging adjustment procedure to a higher level when reading of the optical information in the reading procedure has failed while the amount of movement estimated in the estimation procedure is equal to or less than a predetermined first standard.

[0025] In such an optical information reading method, the imaging adjustment procedure may be a procedure of calculating a brightness index value of the image based on pixel values ​​of pixels in the image captured in the imaging procedure, and adjusting the imaging conditions of the imaging unit so that the brightness index values ​​of images obtained in subsequent imaging become the target level. Furthermore, the imaging adjustment step may be a step of adjusting at least one of an exposure time and an illumination lighting time among the imaging conditions.

[0026] Furthermore, at least when the amount of movement estimated in the estimation procedure is equal to or less than the first criterion, the brightness index value may be determined based on a threshold value when pixels sampled from an image captured in the imaging procedure are classified into a first class of dark pixels and a second class of bright pixels based on the variance of pixel values ​​of pixels within each class. Furthermore, the classification may be performed so as to minimize the degree of variance of pixel values ​​of pixels within each class.

[0027] In addition, if the amount of movement estimated in the estimation procedure is equal to or less than the first reference, the brightness index value may be calculated in a first procedure, and if the amount of movement estimated in the estimation procedure exceeds the first reference, the brightness index value may be calculated in a second procedure different from the first procedure. At this time, it is preferable that the brightness index value obtained in the first step for an image from which optical information can be read in the reading step is smaller than the brightness index value obtained in the second step for the same image.

[0028] In addition, in each of the optical information reading methods, the target adjustment procedure may include a procedure of setting the target level to a predetermined initial value when it is determined that the amount of movement estimated in the estimation procedure is not equal to or less than a predetermined first standard. In addition, in each of the optical information reading methods, the target adjustment procedure may be a procedure of returning the target level to a predetermined initial value when the reading of the optical information in the reading procedure fails when the target level is at a predetermined upper limit value.

[0029] Moreover, each of the optical information reading methods may include an irradiation step of irradiating an aimer light as a reference for directing the imaging unit to the optical information to be read, and an aimer light detection step of detecting the position of the aimer light in the image captured in the imaging step.The imaging adjustment step may be a step of determining whether the amount of movement estimated in the estimation step is equal to or less than a predetermined first reference, and adjusting the imaging conditions of the imaging unit based on pixel values ​​of pixels within a predetermined range near the position of the detected aimer light in the image captured in the imaging step if it is determined to be equal to or less than the first reference.

[0030] Alternatively, the imaging adjustment procedure may be a procedure of adjusting imaging conditions of the imaging unit based on pixel values ​​of pixels around the position of the aimer light in an image captured in the imaging procedure, and a procedure of adjusting the imaging conditions of the imaging unit based on pixel values ​​of pixels in a narrower range around the position of the aimer light when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than the first reference, compared to a case where it is determined that the amount of movement estimated in the estimation procedure is not equal to or less than the first reference.

[0031] Alternatively, instead of providing the illumination procedure and the aimer light detection procedure, the imaging adjustment procedure may be a procedure of adjusting the imaging conditions of the imaging unit based on the pixel value of a pixel at a predetermined reference position in an image captured in the imaging procedure, and when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than the first reference, the imaging conditions of the imaging unit may be adjusted based on the pixel values ​​of pixels in a narrower range of the image, compared to when it is determined that the amount of movement estimated in the estimation procedure is not equal to or less than the first reference.

[0032] In addition, when adjusting the brightness of an image, if the image is too dark and reading fails, it is not enough to just make it brighter. If the image is too bright and even parts of the optical information to be read that have low light reflectance, such as black objects, are captured brightly, sufficient contrast cannot be obtained and reading will fail in the same way as if the image were too dark. In view of the above circumstances, a fourth aspect of the present invention provides the following optical information reading method, which aims to enable accurate adjustment of imaging conditions to obtain an image of suitable brightness for reading when analyzing a captured image and reading optical information contained in the image.

[0033] This optical information reading method includes an imaging procedure for periodically capturing an image by an imaging unit, and a reading procedure for analyzing the image captured in the imaging procedure and reading optical information contained in the image. Furthermore, the apparatus may include an imaging adjustment procedure for calculating a brightness index value of an image captured in the imaging procedure based on pixel values ​​of pixels in the image, and adjusting the imaging conditions of the imaging unit so that the brightness index value of images obtained in subsequent imaging will be at a predetermined target level, and the brightness index value may be determined based on a threshold value when pixels sampled from the image captured in the imaging procedure are classified into a first class of dark pixels and a second class of bright pixels based on the variance of pixel values ​​of pixels within each class.

[0034] Furthermore, the classification may be performed so as to minimize the degree of variance of pixel values ​​of pixels within each class. Alternatively, in such an optical information reading method, the classification may be performed so as to minimize the intra-class variance, which is a weighted average of the variance of pixel values ​​of pixels within each class, taking into account the number of pixels belonging to each class. In each of the optical information reading methods, the brightness index value may be a value indicating a pixel value brighter than the threshold value, or the brightness index value may be a value indicating a pixel value in the vicinity of the threshold value and brighter than the threshold value. In each of the optical information reading methods described above, the imaging adjustment step may be a step of adjusting at least one of an exposure time and an illumination lighting time among the imaging conditions.

[0035] Also, in order to output the reading results in a short time, it is important to adjust the imaging conditions so that an image with as little blur as possible is obtained, so that reading does not fail due to poor image quality. To achieve this, it is simple to set a short exposure time. However, there are also cases where you may want to set a relatively long exposure time, such as when you want to secure a large amount of exposure light because the surroundings are dark, but trying to achieve this by only increasing the gain of the light receiving element results in large noise. In view of the above circumstances, a fifth aspect of the present invention provides the following optical information reading method, which aims to enable appropriate adjustment of imaging conditions while providing a wide range of exposure time options when analyzing a captured image and reading optical information contained in the image.

[0036] This optical information reading method includes an imaging procedure for periodically capturing an image by an imaging unit, a reading procedure for analyzing the image captured in the imaging procedure to read optical information contained in the image, and an estimation procedure for estimating the amount of movement of the imaging unit per frame based on the images captured in the imaging procedure within a certain time range. It is preferable to further include an imaging adjustment procedure for adjusting imaging conditions of the imaging unit including at least one of exposure time and illumination time. And, it is preferable to determine an upper limit of the value of the at least one of the items set in the imaging adjustment procedure based on the estimated amount of movement of the imaging unit per frame.

[0037] Such an optical information reading method preferably includes a step of acquiring an amount of image blur that is tolerable when reading the optical information, and an upper limit of the value of at least one of the items set in the imaging adjustment step is determined based on the estimated amount of movement of the imaging unit per frame and the acquired amount of blur.

[0038] A sixth aspect of the present invention provides the following optical information reading method, which aims to shorten the time from successfully capturing an image from which the optical information to be read can be read to outputting the reading result when analyzing a captured image and reading optical information contained in the image. This optical information reading method includes an imaging procedure for periodically capturing an image using an imaging unit, a reading procedure for analyzing the image captured in the imaging procedure and reading optical information contained in the image, and an estimation procedure for estimating the amount of movement of the imaging unit during a certain time range based on the image captured in the imaging procedure during the time range.

[0039] Furthermore, it is preferable to provide a first reading control procedure for newly executing the reading procedure at a point in time when a time corresponding to the certain time range has elapsed since the capture of the image being analyzed in the reading procedure, in which an image captured in the imaging procedure after the image being analyzed is analyzed and optical information contained in the image is analyzed, and a second reading control procedure for determining whether the amount of movement estimated in the estimation procedure is equal to or less than a predetermined first standard, and not executing the first reading control procedure if it is determined that the amount of movement is equal to or less than the first standard. Alternatively, the method may include a first reading control procedure for newly executing the reading procedure at a point in time when a time corresponding to the certain time range has elapsed since the start of the reading procedure, in which the most recent image captured in the imaging procedure is analyzed and optical information contained in the image is read, and a second reading control procedure for determining whether the amount of movement estimated in the estimation procedure is equal to or less than a predetermined first standard, and not executing the first reading control procedure if it is determined that the amount of movement is equal to or less than the first standard.

[0040] Furthermore, each aspect of the present invention described above can be embodied in any form, such as an apparatus, a system, a program, or a recording medium having a program recorded thereon, in addition to being embodied in the form of the method described above. Effect of the Invention

[0041] According to the configuration of the present invention, when analyzing a captured image and reading optical information contained in the image, The imaging conditions are adjusted appropriately regardless of the environment in which the information to be read is placed. It is possible. [Brief description of the drawings]

[0042] [Figure 1] FIG. 1 is a block diagram showing the hardware configuration of a reader 100 which is an embodiment of an optical information reader according to the present invention. [Diagram 2] FIG. 2 is a functional block diagram showing a functional configuration of the reading device 100 shown in FIG. [Diagram 3] FIG. 3 is a diagram showing a basic execution timing of a reading process for reading optical information, which is executed by the reading device 100 shown in FIG. 1, from the time when a reading start trigger is detected. [Figure 4] 4A and 4B are diagrams each showing a different example of execution timing of a reading process for reading optical information, which is executed by the reading device 100, from a point in time when a certain time has elapsed since detection of a reading start trigger. [Diagram 5] FIG. 5 is a flowchart of a process executed by the CPU 121 of the reading device 100 when a reading start trigger is detected. [Figure 6] 6A and 6B are diagrams showing examples of images captured in two consecutive frames, while Fig. 6C is a diagram for explaining estimation of the amount of movement of the image sensor 111 based on the positions of feature points in those images. [Figure 7] Fig. 7A is a diagram showing an example of a range used as a template image in an image captured in a certain frame, and Fig. 7B is a diagram showing an example of a search range to be compared with the template image in the image of the next frame. [Figure 8] 8A and 8B are diagrams showing examples of movement paths obtained by connecting movement vectors of each frame within a predetermined time range T. FIG. [Figure 9] FIG. 9 is a flowchart of the aim determination process 220 executed by the CPU 121 of the reading device 100. [Figure 10] FIG. 10 is a continuation of the flowchart in FIG. [Figure 11] FIG. 11 is a flowchart of the movement amount estimation process in the process of FIG. [Figure 12] FIG. 12 is a flowchart of a reading routine started by the process of FIG. [Figure 13] 13A and 13B are diagrams showing different examples of the dimming range determined in the imaging adjustment process 212. FIG. [Figure 14] FIG. 14 is a diagram showing an example of a frequency distribution of pixels within a dimming range, and is also a diagram for explaining classification into black pixel classes and white pixel classes by the discriminant analysis method. [Figure 15] FIG. 15 is a flowchart of the imaging adjustment process 212 during the process of FIG. [Figure 16] 16A to 16C are diagrams showing examples of images to be processed in each reading process 210 when the reading process 210 is retried multiple times while performing the imaging adjustment process 212. Fig. 16D is a diagram showing an enlarged view of the image in Fig. 16C. [Figure 17] 17A to 17D correspond to FIGS. 16A to 16D, respectively, and are diagrams showing other examples of images to be processed. [Figure 18] 18A and 18B are diagrams corresponding to FIGS. 4A and 4B, respectively, showing different examples of execution timing of the reading process in a comparative example of the present invention. [Figure 19] FIG. 19 is a flowchart of a process corresponding to FIG. 5, which is executed by the CPU 121 of the reading device 100 when a reading start trigger is detected in a modification of the present invention. [Figure 20] FIG. 20 is a flowchart of a process corresponding to FIG. 5, which is executed by the CPU 121 of the reading device 100 when a reading start trigger is detected in another modified example of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the hardware configuration of a reader 100 which is an embodiment of an optical information reader according to the present invention. A reading device 100 shown in FIG. 1 is a device for optically reading optical information such as a code symbol 102a or a character string 102b, which is represented by parts having a light reflectance different from the surroundings on a reading object 101. The object 101 to be read may be a record carrier such as paper that statically carries a code symbol 102a or a character string 102b, or it may be a display that dynamically displays them.

[0044] As shown in FIG. 1, the reading device 100 includes an optical unit 110, a control unit 120, an operation unit 131, a notification unit 132, and a display unit 133. Of these, the optical unit 110 is an imaging unit that includes an image sensor 111, a lens 112, an LD (laser diode) 113, and a pulsed LED (light emitting diode) 114, and optically captures an image of the object 101 to be read.

[0045] The imaging sensor 111 is a light receiving element for capturing an image of an imaging target such as the read target 101, and can be configured, for example, by a CMOS (complementary metal oxide semiconductor) image sensor. The imaging sensor 111 can generate image data indicating the gradation value of each pixel based on the charge accumulated in each pixel of the image sensor by imaging, and output the image data to the control unit 120. In the imaging sensor 111, the pixels are arranged two-dimensionally.

[0046] The lens 112 is an optical system for forming an image of reflected light from an object to be imaged on the imaging sensor 111 . The LD 113 is an irradiation unit for irradiating the imaging target with aimer light (marker light) indicating the center position of the imaging range of the imaging sensor 111 so that the operator can use it as a reference for directing the optical unit 110 (particularly the imaging range of the imaging sensor 111) to the optical information on the reading target 101. The procedure for irradiating this aimer light is the irradiation procedure. The shape of the aimer light may be any shape, such as a circle, a rectangle, or a cross. The aimer light may indicate an appropriate target position or target range other than the central position. The pulsed LED 114 is a light projecting unit for projecting illumination light onto an object to be imaged.

[0047] Next, the control unit 120 includes a CPU 121, a ROM 122 that stores programs executed by the CPU 121 and data such as various tables, a RAM 123 that the CPU 121 uses as a working area when executing various processes, and a communication I / F 124 for communicating with external devices.

[0048] The CPU 121 is a processor, and by executing a program stored in the ROM 122 using the RAM 123 as a working area, it controls the operation of the entire reading device 100 including the optical unit 110, the operation unit 131, the notification unit 132, and the display unit 133, and realizes various functions including those described below with reference to Fig. 2. These functions may include processes such as reading, displaying, externally outputting or storing optical information included in image data of an image captured by the imaging sensor 111, estimating the movement status of the imaging sensor 111, controlling processing related to reading of optical information based on the results of the estimation, and adjusting imaging conditions in the optical unit 110. The communication I / F 124 is an interface for communicating with various external devices, such as a data processing device that uses the identification result of the character string 102b.

[0049] The operation unit 131 is an operation means such as a button or a trigger for accepting an operation by an operator. The notification unit 132 is a notification means for issuing various notifications to the operator. Specific notification methods include, but are not limited to, displaying a message or data on a display, turning on or blinking a lamp, outputting a sound from a speaker, and the like. The display unit 133 is a display means for displaying the contents of the optical information read by the reading device 100, information on the operating state of the reading device 100, and the like, and can be configured with a liquid crystal display or the like. The notification unit 132 and the display unit 133 may be configured as common hardware.

[0050] When the reading device 100 is operated automatically by control from an external device or autonomous control, the operation unit 131, the notification unit 132, and the display unit 133 do not need to be provided. The above-mentioned reading device 100 can be configured as, for example, a handheld or stationary type code symbol reading device with a character reading function, but is not limited thereto. A general-purpose computer such as a smartphone or a personal computer may be used as all or part of the hardware.

[0051] Characteristic features of the above-described reading device 100 include a method for estimating the movement status of the image sensor 111, a method for controlling the process related to reading optical information based on the result of the estimation, and a method for adjusting the image capturing conditions in the optical unit 110. These points will now be described. First, the functions of the reader 100 related to reading optical information will be described. FIG. 2 is a functional block diagram showing the functional configuration.

[0052] 2, the reading device 100 includes the functions of an imaging unit 141, an image acquisition unit 142, an information reading unit 143, an output unit 144, an aim determination unit 145, a reading control unit 146, an imaging condition setting unit 147, and an exposure time upper limit setting unit 148. In the example described here, the functions of these units are realized by the CPU 121 executing software to control the optical unit 110 and other units of the reading device 100, but some or all of them may be realized by a dedicated control circuit.

[0053] 2 has a function of controlling the optical unit 110 to periodically capture images, acquiring image data obtained by capturing images of each frame (imaging cycle period), and transferring the image data to the image acquisition unit 142. That is, the imaging unit 141 also functions as an imaging control unit that executes an imaging control procedure. The imaging conditions are determined by the imaging condition setting unit 147. The image acquisition unit 142 has a function of holding the image data acquired by the imaging unit 141 so that the information reading unit 143, the aim determination unit 145, and the imaging condition setting unit 147 can refer to the image data.

[0054] The information reading unit 143 is a reading unit having a function of acquiring image data from the image acquiring unit 142, extracting objects, decoding, and the like, and reading optical information in an image represented by the image data. The actual processing differs depending on the type of optical information to be read and the assumed environment in which the optical information exists.

[0055] That is, for example, when characters are assumed as optical information, it is considered that the information reading unit 143 performs character recognition processing. Also, when it is assumed that the optical information is in an environment where it is difficult to read due to blurring or background patterns, or when it is assumed that the background conditions are various, such as paper, metal surface, display, etc., it is considered to perform image analysis processing to identify what the captured object is, and filter processing to process the image so that it is easier to decode according to the analysis results, prior to processing such as object extraction and decoding.

[0056] The order in which the processes are actually performed may be selected according to an operator's operation or according to settings that are automatically made according to some conditions. The setting of the imaging conditions performed by the imaging condition setting unit 147 may be performed in conjunction with the processing performed by the information reading unit 143.

[0057] The output unit 144 has a function of outputting the reading result such as a character string obtained by the information reading unit 143 to an external device such as a data processing device that processes the data, using the communication I / F 124 and the notification unit 132, and notifying the operator of the success of the reading. The method of notifying the operator can be any method such as a buzzer or vibration, and may not be performed if the notification is not necessary. The output unit 144 may also have a function of displaying the reading result on the display unit 133.

[0058] The aim determination unit 145 has a function of an estimation unit that estimates the amount of movement of the image sensor 111 in a specific time range by acquiring image data of each frame from the image acquisition unit 142 and analyzing the image data in chronological order. The estimated amount of movement may include the amount of movement per frame and the amount of movement within a range of multiple frames. The estimated amount of movement may also include the amount of movement as a scalar amount that does not take into account the trajectory along the way, which may be expressed as the amount of translational movement or the amount of rotational movement, and the movement range that takes into account the trajectory along the way. If the entire optical unit 110 is a rigid body, the amount of movement of the optical unit 110 may be recognized as being the same as the amount of movement of the image sensor 111. If the entire reading device 100 is a rigid body, the amount of movement of the reading device 100 may be recognized as being the same as the amount of movement of the image sensor 111.

[0059] The reading control unit 146 has a function of restarting the process performed by the information reading unit 143 as necessary when the estimated result of the movement amount estimated by the aim determination unit 145 is equal to or smaller than a predetermined standard (first standard). The conditions for restarting will be described later with reference to Figs. 4A and 4B. The imaging condition setting unit 147 has a function of an imaging adjustment unit that adjusts the imaging conditions of the imaging unit 141 based on the contents of image data obtained by imaging and the imaging conditions used for imaging, and supplies the adjusted conditions to the imaging unit 141. The imaging conditions to be adjusted may include, for example, the exposure time and gain of the imaging sensor 111. They may also include the intensity and lighting time of the illumination by the pulse LED 114. The exposure time upper limit setting unit 148 has a function of setting an upper limit of the exposure time set by the imaging condition setting unit 147. Alternatively or in addition to this, the exposure time upper limit setting unit 148 may have a function of setting an upper limit of the intensity of illumination by the pulse LED 114 or the lighting time.

[0060] Next, the basic execution timing of the reading process for reading optical information executed by the reading device 100 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the execution timing of this reading process from the time when a reading start trigger is detected. Note that in Fig. 3, the process executed by the aim determination unit 145 is omitted.

[0061] In FIG. 3, the horizontal direction indicates the passage of time, and each of the squares lined up in the upper row indicates the imaging time for one frame. When the reading device 100 detects a read start trigger that instructs the reading of optical information to start, such as an operation by an operator or receipt of a command from outside, the imaging unit 141 starts imaging. Then, when imaging of one frame is completed, the image data obtained by imaging is stored in the image acquisition unit 142.

[0062] Furthermore, when the reading device 100 detects that image data captured in a new frame 201 is held in the image acquisition unit 142 while the reading process 210 is not being executed, the reading device 100 starts the reading process 210 for analyzing the image and reading optical information contained in the image, with the image data as the processing target. In this embodiment, the reading process 210 includes, in this order, a target analysis process 211, an image capture adjustment process 212, a filter process 213, and an object extraction / decoding process 214. Of these, the image capture adjustment process 212 corresponds to the function of the image capture condition setting unit 147, and the rest correspond to the functions of the information reading unit 143.

[0063] The object analysis process 211 is a process for performing image analysis to identify the object that has been photographed. The imaging adjustment process 212 is a process for adjusting the imaging conditions of the optical unit 110 based on the contents of the image data and the imaging conditions used for capturing the image. If it is necessary to change the imaging conditions, the necessary settings are made to the optical unit 110 during the imaging adjustment process 212, and imaging is performed according to the changed settings from the next frame. However, the image captured according to the changed settings becomes the target of the reading process 210 the next time the reading process 210 is started.

[0064] The filter process 213 is a process for processing image data so as to increase the success rate of the object extraction / decoding process 214 according to the identification result in the object analysis process 211. It is also possible to use AI (artificial intelligence) in the object analysis process 211 and determine the algorithm and parameters of the filter process 213 by the AI. The imaging adjustment process 212 and the filtering process 213 may be performed in any order.

[0065] The object extraction / decoding process 214 is a process for extracting objects indicating information to be read, such as code symbols and characters, from the image represented by the image data after the filter process 213, and acquiring the information to be read by performing decoding, character recognition, etc. The object extraction / decoding process 214 does not need to be separated into an object extraction process and a decoding process, and may be, for example, a process in which an AI model is trained in advance to determine what an image of a specific size (e.g., 100 x 100 pixels) is, and the image is input to the trained model to determine what optical information in the image is.

[0066] When the reading device 100 succeeds in reading information through the object extraction and decoding process 214, it outputs the reading result. On the other hand, when the reading fails, it retries the reading process 210. In this case, the image data of the latest frame 201a (when referring to a specific frame 201, a code with an alphabet is used) whose image has been captured at the time of the retry is processed. The reading device 100 thereafter repeats the reading process 210 in the same manner until a predetermined number of retries is reached or until the reading of the information is successful. When the predetermined number of retries is reached, the reading device 100 notifies the operator of a reading failure and stops the reading process 210.

[0067] Next, with reference to FIG. 4A and FIG. 4B, the execution control of the reading process 210, which is one of the features of this embodiment, will be described, taking into consideration the process performed by the aim determination unit 145. 4A and 4B show schematic examples of different execution timings of the reading process for reading optical information executed by the reading device 100, starting from a point in time when a certain time has elapsed since detection of a reading start trigger.

[0068] Although not shown in Fig. 3, the reading device 100 executes aim determination process 220 for estimating the movement status of the image sensor 111 at the timing indicated by the dashed line each time imaging of one frame is completed. This process corresponds to the function of the aim determination unit 145. Details of the process will be described later using Fig. 9 etc., but the aim determination process 220 in this embodiment includes a process of estimating the amount of movement of the image sensor 111 in a certain time range based on images captured in that time range, and a process of estimating the amount of movement of the image sensor 111 per frame. The aim determination process 220 can be executed in a time of about 1 ms (millisecond), which is extremely short compared with the image capture frame 201 and the reading process 210. The aim determination process 220 may be executed in parallel with the reading process 210 in the CPU 121, or may be executed by interrupting the reading process 210.

[0069] Here, a possible usage situation of the reading device 100 is that when an operator performs a reading start operation on the reading device 100 and a reading start trigger is given, the imaging range of the image sensor 111 is not yet directly facing the optical information to be read, and then the operator moves the reading device 100 to directly face the optical information to be read, and then stops the reading device 100.

[0070] This stationary state is usually a state in which the operator subjectively believes that the reading device 100 can read the optical information of the reading target in the current positional relationship, and is considered to be suitable for reading in many cases in terms of the positional relationship between the actual imaging range and the optical information. In other words, this stationary state is a state in which the reading device 100 is aimed at the optical information of the reading target. However, when the operator is operating the reading device 100 by hand, the reading device 100 is not completely stationary.

[0071] In such a usage situation, the reading device 100 will be (almost) stationary at some timing while the reading device 100 is repeatedly executing the reading process 210 as described in Fig. 3. This timing is indicated by arrow A in Fig. 4A and Fig. 4B. Then, in the aim determination process 220 performed according to the image capture several frames later, the estimated result of the movement range of the image sensor 111 in the most recent predetermined time range T (a period of a predetermined number of frames) falls within a predetermined convergence range. In other words, the movement amount calculated as the movement range is equal to or less than a predetermined standard. When this condition is satisfied, the reading device 100 determines that the reading device 100 has come to a (substantially) standstill and that the aim on the optical information has been determined. This timing is indicated by arrow B in Figures 4A and 4B. At the timing of arrow B, it is considered that the reading process 210 is being executed, but the stage of the reading process 210 varies depending on the case. The object analysis process 211 may have just started, or the object extraction and decoding process 214 may be about to finish.

[0072] Here, if the processing target of the reading process 210 being executed at the timing of the arrow B is image data captured at least in a frame after the timing of the arrow A, the image is captured with a well-aimed target, so it is considered that there is a low possibility that the object extraction / decoding process 214 will fail due to the image being significantly blurred or the optical information of the reading target not being within the imaging range. Therefore, by continuing the reading process 210 being executed, it can be expected that there is a relatively high possibility that the reading will be successful.

[0073] However, it is not easy to accurately identify the timing of arrow A. On the other hand, it is considered that the first time that the estimated result of the movement range in the aim determination process 220 falls within the predetermined convergence range is the process in which the predetermined time range T starts near the timing of arrow A. Therefore, in this embodiment, a criterion (called "criterion R") is set that the image data captured within the predetermined time range T in the aim determination process 220 performed at the timing of arrow B is the processing target, and if criterion R is satisfied, the reading process 210 being executed at the timing of arrow B is continued as is.

[0074] 4A shows an example in which a reading process 210a (a reference symbol with an alphabet is used to indicate a specific reading process 210) within a predetermined time range T has ended, and the next reading process 210b is being executed at the timing of arrow B, which processes image data captured in a frame 201b within the predetermined time range T. In this case, the reading device 100 continues the reading process 210b as is.

[0075] On the other hand, if the processing target of the reading process 210 being executed at the timing of arrow B is image data captured in a frame prior to the timing of arrow A, the image was captured without a clear aim, so there is a high possibility that the object extraction and decoding process 214 will fail due to the image being significantly blurred or the optical information of the reading target not being within the imaging range. Therefore, even if the reading process 210 is continued as is, it will result in a reading failure and the processing time will be wasted.

[0076] In this case, the reading device 100 stops the reading process 210 being executed at the timing of the arrow B and starts a new reading process 210. That is, the reading process 210 is started anew if the above-mentioned criterion R is used and criterion R is not satisfied. The image data to be processed in the new reading process 210 may be any image captured within the predetermined time range T, but considering the commonality of the process with the repetition of the reading process 210 as assumed in FIG. 3, it is preferable to process the image data of the latest frame. Alternatively, it is also possible to process the image data captured in the frame with the smallest estimated value of the movement amount of the image sensor 111 per frame.

[0077] 4B shows an example in which the reading process 210a being executed at the timing of arrow B was started before the predetermined time range T. In this case, the image data to be processed was naturally captured before the predetermined time range T and does not satisfy the criterion R. Therefore, the reading device 100 stops the reading process 210a at the timing of arrow B and newly starts the reading process 210b with the image data captured in the latest frame 201c as the processing target. In this case, the reading process 210a is wasted, but the time spent waiting for the reading process 210a, which is likely to end in failure, can be saved, and the reading process 210b, which is expected to have a relatively high chance of succeeding, can be started early. Therefore, it can be said that the time until the reading result can be output can be shortened by the amount of the saved waiting time.

[0078] Here, it is also possible to consider not making a judgment on the reference R and always starting a new reading process 210b at the timing of arrow B even in the case of FIG. 4A. Even in this case, the newly started reading process 210b can be expected to have the same possibility of successful reading as in the case of FIG. 4B. However, if this is done, as can be seen from a comparison between FIG. 4A and FIG. 4B, the start timing of the new reading process 210b will be later than in the case of FIG. 4A. This is because the time of the reading process 210 performed within the specified time range T will be wasted. Conversely, by making a judgment on criterion R and continuing the ongoing reading process 210 if criterion R is met, it can be said that the time until the reading results can be output can be further shortened compared to the case where a new reading process 210b is always started at the timing of arrow B.

[0079] In both the cases of Figures 4A and 4B, the reading process 210b using image data captured in a targeted state is not necessarily successful. This is because the brightness and contrast of the image may not be suitable for decoding. In this case, a retry is required as in the case described in Figure 3. However, by advancing the start time of the reading process 210b, the start time of the reading process 210 after the retry can be advanced by the same amount, so even if a retry is required, the time until the reading result can be output can still be shortened by the amount of the saved waiting time. In the example described here, once it is determined that the aim is set, the aim determination process 220 is not performed, and therefore a new reading process 210 is not started during the reading process 210. However, as will be described later in a modified example, this is not essential.

[0080] Next, the process executed by the reading device 100 to realize the function described with reference to Fig. 2 and the operation described with reference to Fig. 3 to Fig. 4B will be described in more detail with reference to a flowchart. The process described here is a process according to an embodiment of the optical information reading method of the present invention.

[0081] First, FIG. 5 shows a flowchart of the process executed by the CPU 121 when a reading start trigger is detected. In this process, the CPU 121 first instructs the optical unit 110 to capture images under default imaging conditions registered in advance (S11). After this, the optical unit 110 continues the operation of capturing images according to the imaging conditions set at that time for each frame period until it is instructed to stop capturing images. The imaging conditions may be changed midway. The default imaging conditions may always be the same, or may be automatically changed according to settings made by the user or surrounding conditions detected by any sensor.

[0082] Next, the CPU 121 starts the reading routine shown in Fig. 12 (S12). The reading routine in Fig. 12 is the reading process 210 shown in Fig. 3 to Fig. 4B and the process related to the retry. The specific contents will be described later. Thereafter, the CPU 121 waits until the imaging sensor 111 completes imaging of the next frame (S13), and upon completion, executes the aim determination process 220 shown in Figures 9 and 10 (S14). The specific contents of the aim determination process 220 will be described later.

[0083] When the CPU 121 determines that the aim is not determined in the aim determination process 220 (No in S15), and the execution of the reading routine is continuing (Yes in S16), the CPU 121 returns to step S13 and repeats the process. That is, the aim determination process 220 is executed every time imaging of one frame is completed. When the reading routine processing is completed in step S16, the CPU 121 instructs the optical unit 110 to stop capturing images (S21) and ends the processing. This route is taken in cases where the target was not determined until reading was successful or reading failure was confirmed after a specified number of retries, and a new reading process 210b as shown in FIG. 4B is not started until the end.

[0084] On the other hand, if the target is determined in step S15, the CPU 121 checks which frame of image data is to be processed in the reading routine (S17). Then, if the image data of a frame within the period (predetermined time range T) counted in the most recent target determination process 220 is not to be processed (No in S18), the reading routine in FIG. 12 is restarted (S19). This starts a new reading process 210. The reading routine being executed may be stopped. Also, the count of the number of retries referred to in step S88 in FIG. 12 may be handed over from the reading routine being executed to the restarted reading routine. This route corresponds to the case in FIG. 4B.

[0085] If the answer is Yes in step S18, step S19 is skipped and the currently executed reading routine continues as is. This route corresponds to the case in FIG. 4A. In either case, the CPU 121 then waits until the reading routine is completed while monitoring the execution status of the reading routine (S20), and when the reading routine is completed, it instructs the optical unit 110 to stop image capture (S21) and ends the process. 4A and 4B, the time required to output the read result can be reduced. The processes in steps S15 to S19 are part of the read control procedure, and correspond to the function of the read control unit 146.

[0086] Next, the estimation of the amount of movement of the image sensor 111 and the determination of whether or not the aim has been determined, which are performed in the aim determination process 220, will be outlined with reference to FIGS. 6A to 8B. First, Fig. 6A and Fig. 6B show examples of images 20a and 20b captured in two consecutive frames, respectively. Image 20a is the previous frame, and image 20b is the next frame. Both images contain a code symbol 21, which is the optical information to be read. Reference numerals 22a and 22b indicate the positions in each of the images 20a and 20b of virtual reference points used to estimate the travel distance.

[0087] If the imaging sensor 111 is moved (relative to the code symbol 21) between the imaging timing of one frame and the imaging timing of the next frame, the position of the code symbol 21 in the images 20a, 20b captured in these two frames will differ. Since it is considered that the operator rarely rotates the imaging sensor 111 (including the optical unit 110) or the code symbol 21 (the object to be read that carries the image sensor 111) near the point where the aim is determined, if only translational movement is considered, it can be said that the amount of movement of the code symbol 21 in the image corresponds to the amount of movement of the imaging sensor 111.

[0088] Therefore, by detecting where the reference point in image 20a has moved to in image 20b and plotting their positions as shown in FIG. 6C, the amount of movement of image sensor 111 during the one-frame period between the two frames can be estimated. A vector (called a "movement vector") pointing from the reference point position 22b in the image 20b of the later frame to the reference point position 22a in the image 20a of the previous frame indicates an estimate of the direction and magnitude of movement of the image sensor 111 between these two frames. However, if the scale of the image (distance in real space per pixel) is unknown, the distance moved in real space cannot be determined. The magnitude of the movement vector can be determined in pixel units, but only the relative magnitude of the distance moved can be estimated from this. If the scale of the image can be estimated separately, the distance moved can also be estimated.

[0089] It is considered that near the point at which the aim is set, the operator will rarely move the image sensor 111 or the code symbol 21 in a direction perpendicular to the surface appearing in the image. Therefore, when determining whether the aim is set or not, it is safe to assume that the direction of relative movement between the image sensor 111 and the code symbol 21 is essentially only parallel to the surface appearing in the captured image. Then, all points on image 20a are considered to move in the same direction and by the same amount in image 20b. Therefore, no matter which point on image 20a is used as the reference point, the movement vector obtained is the same. However, since it is necessary to prevent the reference point from moving out of the frame of image 20b, it is preferable to take the reference point near the center of the image.

[0090] In actual processing, it is difficult to precisely estimate where a specific point on image 20a is located on image 20b. Therefore, a predetermined range near the center of image 20a is used as a template image, which is sequentially compared with images of the same size at various positions in image 20b to search for a position that is most similar to the template image (has the smallest degree of difference). Then, the template image is assumed to have moved to the position that is determined to be most similar, and a movement vector is calculated.

[0091] FIG. 7A shows an example of a range in the image 20a that is used as a template image. Here, when determining the destination, it is preferable to sample many pixels in the optical information portion of the object to be read, since it is more accurate to detect changes in various black and white patterns in the image. However, if an image with a wide range is used as the template image, points within the range must be sparsely sampled in order to increase the processing speed, which reduces the accuracy of the determination. Also, when the optical information to be read is small and surrounded by a monochromatic background, using an image with a wide range reduces the contribution of the optical information portion, and the accuracy of the determination decreases.

[0092] On the other hand, if an image with a narrow range is used, sampling can be performed densely, but if the image is locally bright or dark and solid white or black areas are generated, the similarity score will be high even at positions different from the actual movement. Also, in the case of an image containing an object with parallel contours such as a barcode, there is a problem that the similarity score will be high at a position moved parallel to the contour, regardless of the actual movement.

[0093] Therefore, in this embodiment, as shown in Fig. 7A, a template image 23 having a relatively wide range and a template image 24 having an even narrower range are used, and each is compared with image 20b. For example, template image 23 may be centered to coincide with image 20a and have a range of 1 / 2 the number of pixels of the entire image vertically and horizontally, and template image 24 may have a range of 1 / 4 the number of pixels.

[0094] 7B shows an example of a search range 25 in image 20b for comparing with template image 23, using a wide template image 23 as an example. Search range 25 may be an area obtained by expanding the periphery of the position of template image 23 in image 20a by a width of about the maximum amount of movement to be obtained as an estimation result. If search range 25 is made too wide, the amount of calculation increases and the time required for aim determination process 220 becomes longer, so the width of search range 25 may be determined taking this into consideration.

[0095] By cutting out images of the same size as the template image 23 from various positions within this search range 25 and successively determining the degree of difference with the template image, it is possible to identify the position with the smallest degree of difference. By calculating a motion vector for each frame within a predetermined time range T as described above and connecting the calculated motion vectors, it is possible to estimate the movement path and movement range of the image sensor 111 within the predetermined time range T. Strictly speaking, it is considered that the scale of the image may differ in each frame, but considering the direction of movement performed near the time when the aim is actually set, it is considered that an estimation accuracy sufficient for determining whether or not the aim is set can be obtained even if the difference in scale is ignored. This point has also been confirmed in experiments conducted by the inventors.

[0096] 8A and 8B show examples of movement paths obtained by connecting the movement vectors of each frame within the predetermined time range T thus obtained. Each arrow in the figure indicates one movement vector, and reference numeral 41 indicates a virtual movement end position within a predetermined time range T. The movement vector 42a of the last frame is positioned so that its tip is at the movement end position 41, the movement vector 42b of the previous frame is positioned so that its tip is at the root of the movement vector 42a, and so on. In this manner, the root position of the movement vector 42n of the first frame becomes the virtual movement start position 43 within the predetermined time range T. For example, when all of the movement vectors 42a to 42n arranged in this manner are within a predetermined convergence range 44 centered on the movement end position 41 as shown in FIG. 8A, it can be determined that the amount of movement of the image sensor 111 is equal to or less than a first reference and the aim is set.

[0097] On the other hand, even if the movement end position 41 and the movement start position 43 are the same as those in FIG. 8A, if at least a portion of the movement vectors 42a to 42n falls outside the convergence range 44 as shown in FIG. 8B, it can be determined that the aim is not fixed. When the aim is set, even if there is some movement of the image sensor 111 due to the movement of the operator's hand, it is considered that the movement will only occur near a specific point, and from this perspective, the standard is that the movement falls within the convergence range 44. Even if the amount of movement per frame is somewhat large, it can be interpreted that the aim is set as long as the movement is only near a specific point, so it is considered that using the convergence range 44 as a standard will enable more accurate detection of the aim being set than using only the amount of movement as a standard.

[0098] Even if the aim is fixed, if the amount of movement per frame is relatively large, the image blur will be large, which may cause the reading process 210 to fail. Therefore, in this embodiment, if the amount of movement per frame (average of the magnitude of the movement vector) exceeds a predetermined threshold, it is determined that the aim is not fixed. However, it is not essential to set this criterion.

[0099] 9 and 10 show a flowchart of the aim determination process 220 for performing the aim determination described above. Also, FIG. 11 shows a flowchart of the movement amount estimation process during the process of FIG. The aim determination process 220 is a process of the estimation procedure, and corresponds to the function of the aim determination unit 145 . In the aim determination process 220 of Fig. 9, the CPU 121 first acquires image data captured in the latest frame and the frame immediately preceding it (S31). Then, a template image of a wide first range is cut out from the image data of the previous frame (S32), and the movement amount estimation process of Fig. 11 is executed using the template image (S33). The template image is as described with reference to Fig. 7A.

[0100] 11, the CPU 121 first sets a primary search range by expanding an area corresponding to the template image in the image of the latest frame by p1 pixels vertically and horizontally (S61). Then, images of the same size as the template image at positions every d1 pixels vertically and horizontally within the primary search range are set as comparison images, and an SSDA (Sequential Similarity Detection Algorithm) calculation is performed on each comparison image (S62). SSDA is calculated for each position by SAD (Sum of Absolute Difference) = R SAD While calculating, if the SAD exceeds a certain threshold during the addition of Σ, the calculation is terminated.

[0101]

number

[0102] For the first contrast image, the SAD is calculated without a threshold, and then the SAD at that time is used as the threshold to calculate the SAD for the next contrast image. If the threshold is exceeded during the Σ addition, the calculation is stopped and the calculation for the next contrast image is started. If the SAD is obtained without exceeding the threshold until the end, this value is used as the next threshold. The position of the contrast image at this time is also stored. The above is performed for contrast images in all positions, and the final threshold value and the position of the contrast image when that threshold was obtained are obtained as the calculation results of SSDA. If the processing load of the SSDA is large and the calculation takes too much time, the calculation may be performed only on pixels sampled at appropriate intervals in the template image and the comparison image, such as every other pixel or every third pixel.

[0103] In the SSDA of step S62, it is possible to determine which position of the comparison image has the smallest SAD within the primary search range, that is, the comparison image with the smallest difference from the template image. This position can be used as a candidate for the destination of the template image movement. Also, since the calculation can be interrupted during the Σ addition when it is found that the SAD does not become the minimum, the total amount of calculation can be suppressed. Note that the SAD can be used as an index of the degree of difference between two images, indicating the degree of difference, but it is also possible to use other indices.

[0104] Next, the CPU 121 sets, as the secondary search range, the range obtained by expanding the position of the comparison image with the smallest SAD obtained in the SSDA by p2 pixels vertically and horizontally (S63). However, p2 < p1. Then, for each position every d2 pixels vertically and horizontally within the secondary search range, an image of the same size as the template image is used as the comparison image, and the SSDA operation is performed on each comparison image (S64). However, d2 < d1. The processes of steps S63 and S64 are to more precisely search the periphery of the movement destination candidate obtained in steps S61 and S62.

[0105] The CPU 121 estimates that the position where the smallest SAD is obtained in the SSDA of step S64 is the movement destination of the template image in the image of the latest frame, and generates a vector indicating the movement from this position to the position of the template image as the movement vector (S65). Also, for the comparison image at the position where the smallest SAD is obtained, ZNCC (Zero - mean Normalized Cross - Correlation) = R ZNCC is obtained, and this value is stored as the similarity (S66), and then the process returns to the original process.

[0106] The ZNCC can be obtained according to Equation 2 and can be used as an index of similarity, indicating the degree of similarity between two images. However, it is also possible to use other indices instead. When the similarity is small, the movement destination obtained by the SSDA is considered less reliable.

[0107]

number

[0108] Returning to the explanation of FIG. 9, after step S33, CPU 121 cuts out a template image of a second range narrower than the first range from the image data of the previous frame (S34), and executes the movement amount estimation process of FIG. 11 again using the template image (S35). Thereafter, the CPU 121 selects one of the first and second ranges that has the greater similarity calculated in the movement amount estimation process as the processing target (S36), and determines whether the similarity calculated for the processing target is equal to or greater than a predetermined threshold (S37). This threshold may be determined based on the degree of similarity at which the SSDA result can be trusted.

[0109] If the answer is Yes in step S37, CPU 121 stores the motion vector calculated for the processing target as the motion vector corresponding to the latest frame (S38). If the answer is No, the motion vector calculated this time is not used, and the fact that there is no motion vector corresponding to the latest frame is stored (S39). In either case, the CPU 121 then judges whether or not the processing for a predetermined number of frames has been completed (S40). That is, it judges whether or not the aim determination process 220 has been performed for a sufficient number of frames to determine whether or not the aim is set. The predetermined number of frames corresponds to the predetermined time range T shown in Figures 4A and 4B. Although six frames are shown in Figures 4A and 4B, this is not limited to this.

[0110] If the answer is Yes in step S40, the process proceeds to step S41 in FIG. Then, CPU 121 sequentially determines whether movement vectors have been stored for a predetermined percentage or more of the nearest neighboring frames, i.e., whether it has been determined in step S37 that the similarity has exceeded a threshold (S41), whether the average magnitude of the movement vectors stored for the nearest neighboring frames is below a threshold (S42), and whether the trajectory (movement range: see Figures 8A and 8B) connecting the movement vectors stored for the nearest neighboring frames falls within a predetermined convergence range 44 (S43).

[0111] If all of these are Yes, CPU 121 determines that the aim on the optical information to be read is set (S44). If even one is No, it determines that the aim is not set (S45). If No in step S40, it also determines that the aim is not set because there is not enough information available to determine that the aim is set (S45). After step S44 or S45, the original process is returned to.

[0112] The determination in step S41 is provided because if the number of reliable movement vectors is too small, the movement range cannot be appropriately determined in step S43. The predetermined ratio may be set to, for example, 25%. The judgment in step S42 is provided because, even if the overall movement range falls within the predetermined convergence range 44, if the movement per frame is too large, it is difficult to consider that the aim is fixed and the condition is not suitable for reading.

[0113] The judgment in step S43 is the same as that described with reference to Figures 8A and 8B. However, when connecting the motion vectors to obtain the estimated result of the motion range, the motion vectors that were not adopted in step S39 may be ignored. For example, if the motion vector of the Nth frame is not adopted, the tip of the motion vector of the N-1th frame may be connected to the root of the motion vector of the N+1th frame.

[0114] Through the above process, the reading device 100 estimates the amount of movement of the image sensor 111, and based on the result, it is possible to determine whether the aim on the optical information to be read has been set. When making a determination taking into account the movement path as in step S43, at least two movement vectors are required. This is because using only one vector is no different from simply taking into account the distance or magnitude of the movement. Therefore, in order to determine that the aim has been set, it is necessary to perform the aim determination process 220 at least twice, and at least three frames of image data are used for this purpose.

[0115] FIG. 12 shows a flowchart of a reading routine started by the process of FIG. The reading routine is executed in parallel with the processing of Fig. 5, steps S81 to S85 are the reading processing 210 described with reference to Fig. 3, and steps S86 and after are processing related to the retry. The reading routine is processing of the reading procedure, and in this processing, the CPU 121 functions as a reading unit.

[0116] In the process of the reading routine, the CPU 121 first obtains image data of an image captured in the latest frame at this point in time and sets it as the processing target (S81). The CPU 121 then sequentially executes a target analysis process 211, an image capture adjustment process 212, a filter process 213, and an object extraction / decoding process 214 on the image data to be processed (S82-S85). Each of these processes has been described using FIG. 3, and the image capture adjustment process 212 will be described in detail later.

[0117] After the above, the CPU 121 judges whether the reading of the optical information by the object extraction and decoding process 214 was successful (S86), and if successful, outputs the data obtained by the reading as the reading result (S87) and ends the process. If it fails in step S86, the process returns to step S81 and is repeated until the number of retries exceeds a specified value (No in S88), and when the specified value is exceeded (Yes in S88), a read error is output (S89) and the process ends. In this manner, the reading device 100 executes the operation described with reference to FIG.

[0118] Next, the basic concept of the imaging adjustment process 212 will be described with reference to FIGS. 13A to 14. FIG. As described above, the imaging adjustment process 212 is a process that adjusts the imaging conditions of the optical unit 110 based on the content of the image data and the imaging conditions used for capturing the image, and more specifically, it adjusts the integrated light amount given to the imaging sensor 111 during imaging and the amplification factor (gain) of the imaging sensor 111.

[0119] More specifically, a frequency distribution of pixel values ​​in a specific region (called a "dimming range") in an image captured under certain imaging conditions is obtained, and an index value of image brightness (called a "brightness index value") that serves as a reference for adjustment is obtained from this frequency distribution. The integrated light amount and amplification rate are then adjusted so that a similarly obtained brightness index value for an image captured in the next imaging is expected to be a predetermined target brightness index value, that is, the brightness of the image captured in the next imaging is expected to be a predetermined target level determined by the target brightness index value. Note that the brightness index value and target brightness index value described below are not numbers that directly indicate the brightness of the image itself, but can indicate the level of brightness of the image.

[0120] 13A and 13B show examples of the dimming range determined in the imaging adjustment process 212. FIG. Fig. 13A shows an example of a relatively wide dimming range 51 that is determined when the aim is not set (the aim is not determined to be set in the aim determination process 220). Fig. 13B shows an example of a relatively narrow dimming range 52 that is determined when the aim is set. These dimming ranges 51 and 52 are reference positions that are ranges for acquiring pixel values ​​used as a reference for adjusting the imaging conditions.

[0121] The reason why the image capture adjustment process 212 performs adjustment based on the image within the dimming range rather than the entire image is to adjust the image capture conditions based on the state of the image near the optical information to be read so that the optical information appears in the image in a manner that makes it easy to read. Ideally, therefore, the dimming range should be exactly the same as the area where the optical information is located.

[0122] However, the more adjustment of the imaging conditions is required, the more difficult it is to specify the position of the optical information in the image. Therefore, when the aim at the optical information is not determined, a relatively wide range in the image 50a is set as the dimming range 51 as shown in Fig. 13A, in the hope that the optical information is somewhere inside. If the aimer light 53 can be detected, the dimming range 51 may be set around the aimer light 53, or may be set around the center position of the image 50a. Other suitable positions may also be used as the reference.

[0123] On the other hand, as shown in image 50a of Fig. 13A, for example, in a case where a barcode, which is optical information, is carried on a non-luminous body and there is a strong light source such as a window that shines light into the surroundings, it is considered that the image characteristics such as brightness are significantly different between the optical information and its surroundings in the image. In such a case, if a wide range is set as the dimming range, there is a possibility that the imaging conditions will be adjusted based on a part whose characteristics are significantly different from the optical information part. In the example of Fig. 13A, the optical information part is dark, but because the surroundings are bright, the image will be adjusted to be darker. The reason why the imaging conditions are not adjusted based on the entire image even when the aim is not determined, but the dimming range 51 is set to a part of the image 50a, is to alleviate this adverse effect to some extent.

[0124] On the other hand, if the aim at the optical information is fixed, it is assumed that the operator will align the aimer light emitted by the LD 113 with the position of the optical information to be read, so it is expected that there is optical information around the aimer light. Therefore, as shown in Fig. 13B, by setting the dimming range 52 to a relatively narrow range (at least a range narrower than the dimming range 51) around and near the position of the aimer light 53 in the image 50b, it is considered that adjustments that contribute to successful reading of the optical information will be possible.

[0125] The size of the dimming range 52 should be determined in consideration of the standard usage of the reading device 100 and the standard carrying mode of optical information, so that it is expected that the image of the background other than the carrier surrounding the optical information to be read will not be captured at a non-negligible rate if the target is set. A number of candidates may be prepared and switched according to the reading mode setting.

[0126] When the aimer light 53 is not used or cannot be detected, it is assumed that the operator aligns the vicinity of the center of the imaging range with the position of the optical information, so a similar effect can be expected by setting the dimming range 52 in a relatively narrow range near the center of the image 50b. However, the dimming range 52 may be set based on another appropriate position in the image where the operator is assumed to align with the optical information. By defining a narrow dimming range 52 as described above, even if image characteristics such as brightness differ significantly between the optical information and its surroundings, the imaging conditions can be adjusted based on the state of the image near the optical information so that the optical information appears in the image in an easily readable manner.

[0127] The adjustment of the integrated light amount and the amplification factor based on the image of the light control area can be performed as follows, for example. First, several pixels are sampled from all over the dimming area, the pixel values ​​are tallied, and the frequency distribution is calculated. For example, sampling can be performed on 20 vertical x 20 horizontal (400 points).

[0128] An example of this frequency distribution is shown in Figure 14. Reference numeral 60 denotes a graph of the relative frequency distribution. From this frequency distribution, the above-mentioned brightness index value can be obtained. For example, when a relatively wide dimming range 51 is used, in the case where the brighter the pixel, the larger the pixel value, the pixel value of an appropriate quantile close to the upper limit of the cumulative relative frequency is set as the brightness index value D c The inventors' experiments have revealed that a relatively good adjustment can be achieved by setting the target brightness index value to a value of about 600 in the case of 1024 gradations. c The reason why is set to a quantile close to the upper limit rather than the upper limit is to exclude overexposed areas due to specular reflection.

[0129] Then, the black level of the image sensor 111 is set to D b , brightness index value D c The integrated light amount in the image obtained is I c , the amplification factor of the image sensor 111 is g c If the proportionality constant determined by the surrounding environment at the time of image capture is k, the following relationship is considered to hold: D c -D b = k × I c ×g c ... (1) In addition, the target brightness index value D T The integrated light amount and amplification factor that should be set to achieve I n , g n Then, the following relationship can be considered to hold: D T -D b = k × I n ×g n (2)

[0130] From equations (1) and (2), the following relationship can be derived. I n ×g n =(D T -D b ) / (D c -D b )×I c ×g c ... (3) Therefore, when adjusting the imaging conditions, D c , D T , D b , I c、 g c Based on the value of I, the appropriate I is calculated according to Eq. (3). n and g n It is then sufficient to determine the value and set it in the optical section 110. In addition, the cumulative light intensity I c and I n is determined by the exposure time of the image sensor 111 when the surroundings are bright and the lighting is not turned on and when the surroundings are dark and the lighting is turned on, it can be determined by the integrated light amount of the lighting (lighting time when the light amount is constant). Therefore, when adjusting the imaging conditions, it is considered to adjust at least one of the items of the exposure time and the lighting time.

[0131] In addition, the equation (3) is I n ×g n This is the target value, and I n and g n There are countless combinations of values. Among them, we will choose a specific I n and g n The basic idea is to select the value of the amplification factor g n If you make I too large, the noise in the image will increase, and if you extend the exposure time or the lighting time, the image will become blurred. n and g n It is best to adjust both in a balanced manner, rather than making one of them too large.

[0132] On the other hand, when a relatively narrow dimming range 52 is used, the brightness index value D is calculated based on the threshold between the classes when the pixels are classified into two classes, black pixels (small pixel value, first class of dark pixels) and white pixels (large pixel value, second class of bright pixels), based on the pixel value of each sampled pixel. c It has been found through experiments by the inventors that a relatively good adjustment can be performed by determining the brightness index value D cThe inventors' experiments have shown that particularly good adjustments can be made by setting to a value indicating a pixel value brighter (in this example, larger) than the threshold value, and in particular to a value indicating a pixel value near the threshold value that is brighter (in this example, larger) than the threshold value.

[0133] Classification into two classes can be performed, for example, using discriminant analysis as follows. First, if we define the value of the threshold n, we can calculate the number of pixels belonging to each class, and the average and variance of the pixel values ​​of the pixels belonging to that class, as shown in Figure 14. We can also calculate the average and variance of the pixel values ​​of all pixels. Based on these values, we can calculate the intra-class variance σ w 2 The intra-class variance is a weighted average of the variance of pixel values ​​of pixels in each class, taking into account the number of pixels belonging to each class, and is small when pixel values ​​are concentrated within a class.

[0134]

number

[0135] On the other hand, the between-class variance σ b 2 is defined as in Equation 4. The inter-class variance is the variance σ t 2 This is the remainder of the variance that is not represented by the within-class variance.

number

[0136] Then, the degree of separation defined by Equation 5 is calculated for various n, and the threshold n that maximizes this is found, thereby determining the threshold n that can most appropriately classify the two classes. This essentially corresponds to determining the threshold n that minimizes the within-class variance.

number

[0137] 14 shows a frequency distribution including a portion with a low frequency near the center of the gradation value in order to make the boundary between the expected classes easier to understand, but even if the frequency distribution does not include such a portion, classification by the discriminant analysis method is possible. Also, the threshold value n obtained here may be different from the threshold value used in the binarization performed in the object extraction / decoding process 214.

[0138] Brightness index value D when using a relatively narrow dimming range 52 c It is preferable that the brightness index value D is a value that indicates a pixel value brighter than the threshold value n. c It is more preferable that is close to the threshold n as in the above example, rather than being a value far from the threshold n. Here, the neighborhood of the threshold n is the brightness index value D c The purpose of this is to adjust the brightness of the white part to the lowest brightness as the target brightness. In other words, the white part buried in the blackest part is adjusted to the target brightness index value D T This is to adjust the brightness (pixel value) to the value indicated by

[0139] Brightness index value D c The standard for how close to the threshold value n should be can be determined from the area ratio containing thin bars, which is determined by the barcode standard, when the object to be read is a barcode, for example. Pixels that capture white bars in a barcode are generally considered to be classified as a white pixel class, but thinner white bars tend to be buried in the black bars on either side and have smaller pixel values ​​(darker values) than thicker white bars. Therefore, it is considered that pixels that capture thin white bars tend to be concentrated in the white pixel class with smaller pixel values ​​(closer to the threshold n). Therefore, if the area ratio of the thin:thick white bars is 30:70, and the target is achieved using a particularly narrow dimming range 52, and it is considered that the barcode exists throughout the dimming range 52, the brightness index value D c By placing cIt can be said that:

[0140] Also, for example, in the Code128 standard, the character code "d" is represented by a black-white-black-white-black-white sequence of bars with thicknesses of 1:4:1:2:2:1. Among these, there are three types of white bars sandwiched between black bars, with a width ratio of 4:2:1. The area of ​​the thinnest white bar among these is 1 / (4+2+1)=14.2% of the white portion. Therefore, it is believed that the pixels that capture the thinnest white bar are concentrated in the bottom 14.2% of the white pixel class in terms of pixel count. Brightness index value D c By setting the pixel value in this range, the pixel value of the thinnest white bar is the brightness index value D c It can be said that:

[0141] The correspondence between the number of pixels and the pixel value varies depending on the captured image. However, based on images captured under various conditions of the expected reading target, the pixel value range of the thinnest white bar is estimated as described above, and the brightness index value D is calculated within that range. c So, the brightness index value D c It is advisable to determine in advance how much higher the pixel value should be than the threshold value n. As an example, the inventors have found through experiments that when reading a barcode, a pixel value that is about 5% larger than the maximum pixel value than the obtained threshold value n can be used to adjust the imaging conditions. In addition, for example, the pixel values ​​are divided into about 20 classes, and the class value one class above the class to which the threshold value n belongs is set as the brightness index value D c It may also be used as a. Furthermore, each time brightness adjustment is performed, a pixel value that is included in a predetermined lower ratio may be obtained based on the distribution of pixel values ​​in the white pixel class.

[0142] In the same way, when reading two-dimensional codes or performing OCR, the brightness index value D is calculated based on the area ratio of the most white-coloured part of the white part that appears in the code symbol or character set to be read. c It is possible to determine whether the above is acceptable. However, if the ratio of the most white-crushed areas is extremely small, the brightness index value D is set based on this ratio. c However, it is not always useful to set the brightness index value D c As described later, the target brightness index value D T By gradually increasing the brightness, it is possible to find the right brightness.

[0143] The brightness index value D obtained as above c When using I n ×g n The target value of can be calculated according to equation (3) as in the above case. Also, the target brightness index value D T In the case of a narrow dimming range 52 using the discriminant analysis method (when the target is set), a pixel value close to the upper limit is set as the brightness index value D c Compared with the wide dimming range of 51, the brightness index value D c Since the value of tends to be small, the target brightness index value D T If the same value is used as the brightness index value D, the image captured after the adjustment will be brighter than the image captured in the wide dimming range 51. c The above brightness index value D c The magnitude relationship does not necessarily hold for all images, but it is considered to hold almost certainly for any image from which optical information can be read.

[0144] In the case of a narrow dimming range 52, the imaging conditions can be adjusted by focusing on the image near the optical information. By making such adjustments, only the parts of the optical information represented in black (parts with low light reflectance) will still have low pixel values, while the other parts, including the background and the parts represented in white (parts with high light reflectance), will all have high pixel values, which is expected to result in increased contrast of the optical information.

[0145] The brightness index value D is calculated by focusing on the threshold when classifying the pixels into dark and bright classes by focusing on the intra-class variance. c By determining and adjusting based on this value, it is possible to adjust so that the pixel values ​​of the black parts of the optical information do not rise too high. Therefore, it is possible to avoid making the image too bright, causing the black parts to become bright pixels and become indistinguishable from the white parts. This allows accurate adjustment of the imaging conditions to obtain an image with a brightness suitable for reading. In particular, when the light reflectance of the background is somewhat low or the printing of code symbols or the like has bleed and the white parts are somewhat crushed, obtaining such a bright image is expected to enable the black parts of the optical information to be extracted from the image with high accuracy.

[0146] The intra-class variance described here is an example of an evaluation function that represents the degree of variance of pixel values ​​of pixels in each class. It is not essential to take a weighted average in the evaluation function, and the variance of pixel values ​​of pixels belonging to each class may be reflected in the evaluation function in a different form. In addition, the brightness index value D c It is not essential that the brightness index value D be a value that indicates a pixel value brighter than the threshold value between classes. c It is preferable that the pixel value is in the vicinity of the threshold value between the classes, but the user may be allowed to set how much brighter or darker than the threshold the pixel value should be.

[0147] In addition, the target brightness index value D is about the same as that in the case of a wide dimming range of 51. T If the object extraction and decoding process 214 fails even after adjusting the imaging conditions using T It is useful to make adjustments and retry using In this reading device 100, as shown in Tables 1 and 2, a plurality of target luminance index values ​​D T When using a narrow dimming range 52 (when the target is set), first, the target brightness index value D TThen, the imaging conditions are adjusted using the above to perform the object extraction and decoding process 214. If this fails, that is, if the optical information cannot be read even when the target is set, the target brightness index value D T The target brightness index value D T The higher the level of brightness, the higher the level of brightness of the target image. The reading mode may be set by an operator or automatically by the reading device 100 according to some criteria. Table 1 is for reading code symbols and Table 2 is for OCR (Optical Character Recognition).

[0148] [Table 1]

[0149] [Table 2]

[0150] If the object extraction and decoding process 214 fails even at the maximum level, the image may have become too bright, so the level is returned to the minimum and a retry is performed. By performing such an operation, it is possible to successfully read optical information with a small number of retries, and it is possible to shorten the time required to output the read results.

[0151] Furthermore, by selectively using the wide dimming range 51 and the narrow dimming range 52 as described above, particularly when the target is set, the narrow dimming range 52 can be used, so that when analyzing the captured image and reading the optical information contained in the image, the imaging conditions can be appropriately adjusted regardless of the environment in which the information to be read is placed. If the target is set, and the reading fails in the object extraction and decoding process 214, the target brightness index value D T Retrying the imaging by readjusting the imaging conditions using also contributes to this effect.

[0152] When the aim is set, it is considered that there is a high probability that the optical information to be read is included within the dimming range, so by changing the target level of image brightness after that state is reached, it is possible to adjust the image brightness while reliably reflecting the contrast of the optical information, and it is considered that an image with a brightness suitable for reading can be obtained at any target level. At this time, if the image is too bright, contrast information is lost and this can cause misreading, so after the aim is set, it is preferable to start a trial of the object extraction and decoding process 214 from an image that is not too bright.

[0153] In addition, if the target level of image brightness is changed before the aim is decided, the image may become too bright, causing halation, making it difficult to judge whether the aim is decided or not. From this point of view, it is useful not to make the image too bright when the aim is not decided. The above-mentioned effects can be increased by changing the dimming range to the narrow dimming range 52 when the target is determined, but it is not essential to do this at the same time.

[0154] In addition, in the reading device 100, the luminance index value D obtained by any of the above methods c When using I n ×g n In determining specific imaging conditions based on the target values, an upper limit is set for the exposure time when the illumination is not turned on. This upper limit can be determined based on the amount of movement of the imaging sensor 111 per frame calculated in the aim determination process 220. Specifically, it can be determined as follows.

[0155] First, the amount of movement of the image sensor 111 per frame can be calculated from the magnitude of the movement vector of each frame. At this time, data for a predetermined number of frames used as the reference in step S40 in FIG. 9 may be collected, or data for a different number of frames may be collected. It is also possible to use the average value of the data in the collected range, or the maximum value, median value, most recent value, etc. The amount of movement calculated in this way is defined as x [pixel].

[0156] Here, the length of one frame is t f , exposure time t exp Then, the amount of movement during the exposure time is p x [pixel] can be expressed as follows: p x =t exp / t f ×x ··· (4) On the other hand, if the amount of image blur in an image that is permissible for reading optical information is defined as permissible blur p [pixels], p is required to satisfy the following formula. p>p x =t exp / t f ×x ··· (5) This can be transformed into p / x×t f >t exp ... (6) and the upper limit of the allowable exposure time is p / x×t f It can be seen that it is determined by

[0157] For example, the allowable blur p = 1 [pixel], the length of one frame t f = 10 [ms], the upper limit of the permissible exposure time according to the movement amount x [pixels] of the image sensor 111 per frame can be determined as shown in Table 3 below. Of course, the upper limit of the exposure time does not have to be such a discrete value, and may be a continuous value obtained by substituting the numerical value of the movement amount into the left side of equation (6). An upper limit of the permissible exposure time may be determined when the movement amount is 10 pixels or more.

[0158] [Table 3]

[0159] The amount of light to be used for the next image capture I n and amplification factor g n In determining the amount of movement x, the exposure time is determined so as not to exceed an upper limit corresponding to the amount of movement x. In this way, the imaging conditions can be appropriately adjusted to prevent a situation in which the optical information cannot be read due to blurring even if the brightness is appropriate.

[0160] That is, a wide range of exposure time options are available, especially when the amount of movement of the image sensor 111 is small. In particular, in cases where the target to be read is far away in a dark environment, it may be difficult to capture a bright image suitable for reading with an exposure time of about 1000 μs, and setting a longer exposure time will greatly contribute to increasing the success rate of reading. The above upper exposure time limits are equally applicable to both targeted and untargeted optical information.

[0161] The magnitude of the allowable blur p depends on the size of the elements constituting the optical information in the image, the complexity of the optical information, the contrast and noise of the image, etc. This value should be set in advance assuming a standard usage environment of the reading device 100. For example, it is possible to consider that the smallest element of the code symbol to be read (such as a bar in a barcode or a dot in a two-dimensional code) fits within one pixel, and to predetermine the allowable blur p to be 0.5 [pixels]. It is also possible to prepare individual values ​​according to combinations of the distance to the read target and the type of the read target. When performing super-resolution below 1 pixel / element, it is also possible to set the allowable blur p to be less than 0.5 [pixels].

[0162] The lighting time when the light is turned on can be considered in the same way. In this case, even if there is a period of exposure without lighting, the effect of the exposure on the image during that period is relatively small. Also, the exposure time is usually set to a value close to the lighting time. Therefore, the upper limit of the lighting time can be determined in the same way as above.

[0163] Next, Fig. 15 shows a flowchart of the above-described imaging adjustment process 212. This process is executed in S83 of Fig. 12 as described above, and is an imaging adjustment procedure process. In this process, the CPU 121 functions as an imaging adjustment unit. In the imaging adjustment process 212, the CPU 121 first determines whether or not it has been determined in the aim determination process 220 of FIGS. 9 and 10 that the aim has been determined (S101).

[0164] When the aim is set, the CPU 121 detects the position of the aimer light 53 in the image to be processed (S102), and sets a dimming range 52 narrower than that set in step S107, centered on the position of the aimer light 53 (S103). In addition, the target brightness index value D is set according to the set reading mode, starting from level 0, gradually increasing the level, and returning to level 0 after the maximum level. T (See S104, Tables 1 and 2).

[0165] The position of the aimer light 53 may be detected by appropriately adopting a known technique. For example, the techniques of Patent Documents 6 to 8 can be adopted. The process of step S102 is an aimer light detection procedure, in which the CPU 121 functions as an aimer light detection unit. The process of step S104 is a target adjustment procedure, in which the CPU 121 functions as a target adjustment unit.

[0166] Next, the CPU 121 applies the above-mentioned discriminant analysis method to the image of the dimming range 52 set in step S103, and obtains the pixel value of the threshold n that is the boundary between the black pixel class and the white pixel class (S105). Then, the pixel value of the threshold n+α (α>0) is used as the brightness index value Dc (S106).

[0167] On the other hand, if the target is not determined in step S101, the CPU 121 sets a dimming range 51 that is wider than the one set in step S103 and is centered on the center of the image to be processed (S107). T is set to a value of level 0 according to the set reading mode (S108). Next, the CPU 121 obtains a frequency distribution of pixel values ​​in the image of the dimming range 51 set in step S108, and determines the pixel value that is a predetermined upper quantile as the brightness index value D c It is stipulated (S109).

[0168] After step S106 or S109, the CPU 121 acquires the magnitude of the movement vector calculated in the most recent aim determination process (S110). As described above, an appropriate counting method such as the average value, the maximum value, or the like may be used. When capturing an image without turning on the illumination (No in S111), the CPU 121 sets an upper limit of the exposure time based on the magnitude of the movement vector acquired in step S110 (S112). For this setting, for example, the above-mentioned formula (6) may be used. The CPU 121 then determines whether the brightness index value in the next image capture will be the target brightness index value D T The amplification factor and exposure time of the image sensor 111 are adjusted so as to satisfy the upper limit of the exposure time and are set in the optical unit 110 (S113), and the process returns to the original process.

[0169] On the other hand, when capturing an image with the illumination turned on (Yes in S111), the CPU 121 sets an upper limit for the illumination illumination time based on the magnitude of the movement vector acquired in step S110 (S114). This setting may also be performed using, for example, the above-mentioned formula (6). The CPU 121 then calculates whether the brightness index value in the next image capture will be the target brightness index value D T The gain and exposure time of the image sensor 111 and the illumination lighting time of the pulse LED 114 are adjusted so as to satisfy the upper limit of the illumination lighting time and are set in the optical unit 110 (S115), and the original process is resumed.

[0170] Through the above processing, the reading device 100 can adjust the imaging conditions as described with reference to FIGS. 13A, 13B, and 14. In this adjustment, the target brightness index value D T is set to an initial value of level 0 when the aim is not determined, and when the aim is determined and reading fails and the imaging adjustment process 212 is repeated, the level is gradually set to a higher value.

[0171] When capturing an image with the illumination turned on, it is also possible to set an upper limit on the exposure time at the same time as or instead of the illumination time. In cases where the distance to the reading target 101 is too far for the light to reach even with the illumination turned on, or where the illumination light intensity or illumination time is separately limited to reduce glare, it may be necessary to adjust the exposure time to capture an image with a more desirable brightness even when the illumination is turned on. When making such an adjustment, it is preferable to set an upper limit on the exposure time to prevent blurring.

[0172] 16A to 16D and 17A to 17D show examples of images to be processed in each reading process 210 when the reading process 210 is retried multiple times while the imaging adjustment process 212 shown in FIG. 15 is being executed. Fig. 16A shows an example of an image to be processed in the second reading process 210 executed in the reading routine restarted in step S19 of Fig. 5 after the aim is set. In the first reading process 210, the image to be processed is captured under imaging conditions adjusted before the aim is set, and is a case of a retry after reading has failed in the first reading process 210.

[0173] In this case, in the imaging adjustment process 212 in the first reading process 210, a brightness index value and a target brightness index value are determined in steps S102 to S106 to adjust the imaging conditions. Since a target brightness index value of level 0 is used initially, the image shown in Fig. 16A that is processed in the second reading process 210 is an image captured under imaging conditions adjusted using the target brightness index value of level 0.

[0174] In the example of Figure 16A, we are trying to read a two-dimensional code, and the image only shows the image around the code symbol. As can be seen from the figure, the code symbol we are trying to read is printed in a slightly blurred state, and because the image in Figure 16A is quite dark, the distribution of black and white elements in the code symbol is not clear. Figure 16B shows an example of an image processed in the third reading process 210 after Figure 16A. This image was captured under imaging conditions adjusted using a target brightness index value of level 1 in the imaging adjustment process 212 during the second reading process 210. As can be seen from the figure, the image is brighter overall than in Figure 16A, the background is almost white, and the white elements of the code symbol also have a high contrast with the black elements, but the white elements are still somewhat crushed, making it difficult to successfully read the image.

[0175] FIG. 16C shows an example of an image processed in the fourth reading process 210 after FIG. 16B. This image was captured under imaging conditions adjusted using a target brightness index value of level 2 in the imaging adjustment process 212 during the third reading process 210. The image shown in FIG. 16D is an enlarged version of the image in FIG. 16C. As can be seen from the figure, the image in FIG. 16C is brighter overall than FIG. 16B, and as can be seen from FIG. 16D, the contours of the white elements are relatively clear. In this state, a successful reading can be expected, and in an experiment in which this image was actually captured, the fourth reading process 210 was successful in reading.

[0176] 17A to 17D show images corresponding to those of FIGS. 16A to 16D when attempting to read a two-dimensional code printed on a carrier with a dark background color. In this case, in the image of Fig. 17A captured after adjustment using the target brightness index value of level 0, the black elements of the code symbol are almost indistinguishable from the background color. However, as the level of the target brightness index value is increased, the contrast increases, and as shown in Fig. 17C and Fig. 17D, in the image captured under the imaging conditions adjusted using the target brightness index value of level 2, the black elements and the white elements can be distinguished to a level that is sufficient for decoding. In an experiment in which this image was actually captured, the image was successfully read in the fourth reading process 210, which processed the image shown in Fig. 17C and Fig. 17D.

[0177] In this way, by performing the imaging adjustment process 212 described using Figures 13A to 15, it is possible to adjust the imaging conditions so that even optical information that is difficult to read due to print loss or contrast issues can be read to some extent. This type of adjustment is possible because it detects a state in which the aim at the optical information is set, and then, in that state, adjusts the imaging conditions so as to obtain a bright image based on the characteristics of the image in the vicinity of the optical information to be read, thereby making it possible to set imaging conditions with a much brighter brightness than when the imaging conditions are adjusted based on the characteristics of a wide area of ​​the image.

[0178] When setting such conditions, if the amplification factor of the image sensor 111 is set to a low value to avoid noise, the exposure time (or the illumination time) will be too long, which may result in a reading failure due to shaking; however, since an upper limit on the exposure time is set separately based on the amount of movement of the image sensor 111, reading failure due to shaking can also be prevented.

[0179] Next, Fig. 18A and Fig. 18B show the execution timing of the reading process in a comparative example of the present invention in a format corresponding to Fig. 4A and Fig. 4B. In the explanation of the comparative example, the same reference numerals as those of the reading device 100 in the above-mentioned embodiment will be used to explain the parts that are common to or correspond to the reading device 100 in the above-mentioned embodiment.

[0180] The comparative example described here differs from the above-described embodiment only in that the aim determination process 220 is not performed, and therefore the read process 210 is not restarted when the aim is determined. Even in this case, if reading process 210 is executed using an image captured in a frame after the timing indicated by arrow A at which reading device 100 described using Figures 4A and 4B comes to a (near) standstill, it is expected that reading will be successful to the same extent as in the case of Figures 4A and 4B.

[0181] For example, as shown in Fig. 18A, if a certain reading process 210c ends with a reading failure immediately after capturing frame 201e immediately after the timing of arrow A, reading process 210d can be started immediately with the image captured in frame 201e as the processing target. In this reading process 210d, the reading success rate can be expected to be the same as in the case of reading process 210b in Fig. 4A. In this case, the time required from the timing of arrow A to obtaining the reading result can be expected to be the same as in the case of Fig. 4A.

[0182] On the other hand, as shown in Fig. 18B, if a certain reading process 210c ends with a reading failure immediately after capturing frame 201d including the timing of arrow A, reading process 210d is started with the image captured at frame 201d at this timing as the processing target. In this case, even if capturing of frame 201b immediately after the timing of arrow A is completed, the next reading process 210e cannot be started until reading process 210d is completed. Also, even if frame 201d is the processing target, successful reading is not expected because the reading device 100 has not yet come to a standstill. Ultimately, after completion of the reading process 210d, a reading process 210e is started, the image captured in the latest frame 201f at that time being processed, and it is expected that the reading process 210e will be as successful as the reading process 210b in FIG. 4A.

[0183] In this case, the time required from the timing of arrow A until the reading result is obtained is considered to be longer than in the case of Fig. 4A by approximately the time required for reading process 210d. In a case where the processing load of reading process 210d is relatively large and one reading process 210d takes a certain amount of time, this time is considered to be longer than the predetermined time range T, which is the difference in the required time between the case of Fig. 4B and the case of Fig. 4A.

[0184] Here, we estimate the average time required from the timing of arrow A to obtaining the reading result for three cases: this comparative example, the embodiment described above, and a first variant of the embodiment described above, in which the reading process 210b is always started anew at the timing of arrow B in Figures 4A and 4B when it is determined that the target has been set.

[0185] In this case, the following (a) to (d) are assumed. (a) At the timing of arrow A, the stage to which the ongoing reading process 210 has progressed is random. (b) If reading process 210 is executed on an image captured in a frame after the timing of arrow A, the reading will be successful, but if reading process 210 is executed on an image captured in an earlier frame, the reading will fail. (c) The time required for the read process 210 in the case of a failed read is f, and the time required for the read process 210 in the case of a successful read is s. In general, when a read fails, the entire range of the image is searched and decoded before it is determined to be a failure, so it takes longer than when the read succeeds, and the time required for the read process 210 in the case of a successful read is s. <fである。 (d) The length of the predetermined time range T (substantially the same as the time from the timing of the arrow A until it can be determined in the aim determination process 220 that the aim has been established) is set to a.

[0186] First, in the comparative example, the required time is the shortest in the case of FIG. 18A (only the time for the read process 210d when the read is successful). Strictly speaking, one frame period is added to this, but this will be ignored here for simplicity. The same applies to the case of FIG. 4B. The required time is the longest in the case of FIG. 18B (f+s) (the sum of the read process 210d when the read is unsuccessful and the read process 210e when the read is successful). In this case, the average required time T1 is (f+2s) / 2.

[0187] Next, in the first modified example, the required time is always the same as that in Fig. 4B. This required time is the average required time as it is, and the average required time T2 is a+s (the sum of the predetermined time range T and the read process 210b for which reading was successful). Here, since T2-T1=f / 2-a, if it can be determined by the aim determination process 220 that the aim is set in less than half the time of f, it is believed that the time required for reading can be shortened compared to the comparative example, even if the reading process 210b is necessarily started anew at the timing of arrow B.

[0188] Next, in the above-described embodiment, a restart does not occur as in FIG. 4A with a probability of a / f, and a restart occurs as in FIG. 4B with a probability of 1-a / f. The time required when no restart occurs is the shortest in the case of Figure 4A, which is s, and the longest in the case where the reading process 210b starts just before the timing of the arrow B, which is a+s (almost the same as the case of Figure 4B). In this case, the average time required when no restart occurs is a / 2+s. The average time required in the case where a restart occurs is a+s, which is the same as in the first modified example.

[0189] Then, the average required time T3 in the above embodiment is T3=(a / f)(a / 2+s)+(1-a / f)(a+s) =a+sa 2 / 2f It is.

[0190] Here, T3-T1=(fa) 2Since it is 1 / 2f, according to the above-described embodiment, if it can be determined by the aiming determination process 220 that the aim has been determined in a time shorter than f, it is considered that the required reading time can be shortened compared to the case of the comparative example. This condition is looser than that in the case of the first modification. Also, it can be seen that the greater the value of f, that is, the more time-consuming the reading process 210 is, and further, the shorter the time (with fewer frames) it takes to determine that the aim has been determined, the greater the effect of shortening the required time.

[0191] Also, if a exceeds f, the reading process 210 has ended once from the timing of arrow A until it can be determined that the aim has been determined, and at the timing of arrow B, it is considered that the reading process 210 for processing the image captured after the timing of arrow A is in progress. Then, a restart of the reading process 210 is not necessary, and it will operate substantially the same as the comparative example.

[0192] Therefore, for example, even when the reading process 210 is a light process and f < a, no inconvenience occurs compared to the case of the comparative example, except for the additional light load on the aiming determination process 220. When it is known that the reading process 210 is a light process due to the selection of the reading mode or the like and f < a is expected, the aiming determination process 220 may not be performed.

[0193] [Modification] The description of the embodiment ends here. However, in this invention, the specific configuration of the device, the specific procedure of the process, the values of various parameters, the type of optical information to be read, the required time for each process, etc. are not limited to those described in the embodiment. Of the functions provided in the reading device 100 of the above-mentioned embodiment, the function of restarting the reading process 210 according to the result of the aim determination process 220, the function of changing the width of the dimming range or changing the method of calculating the brightness index value and the target brightness index value based on whether the aim is set or not, the function of raising the target level of image brightness adjustment when reading fails when the aim is set, and the function of setting the upper limit of the exposure time and the illumination lighting time according to the movement amount of the image sensor 111 can be implemented independently or in any combination. Also, the target analysis process 211 and the filter process 213 in the reading process 210 are not essential.

[0194] In the above embodiment, an example has been described in which an image captured in the most recent frame is processed when restarting the reading process 210. However, this is not essential, and an image captured in any frame within a predetermined time range T after the arrow A in Fig. 4B may be processed. Although this is not preferable because it would delay the start of the reading process 210, it is also possible to use an image captured in a frame after the predetermined time range T.

[0195] In the above-described embodiment, once it is determined that the aim is set, the aim determination process 220 is not performed thereafter. However, after the aim is set, the operator may move the reading device 100 significantly, causing the aim to be missed, and then the aim may be set again. Taking this into consideration, the aim determination process 220 may be continued even after the aim is set once.

[0196] Fig. 19 shows a flowchart of the process in this case, which corresponds to Fig. 5. The differences between the process in Fig. 19 and the process in Fig. 5 are as follows. First, instead of step S15, step SA is performed to determine whether the target has changed from an untargeted state to a targeted state. This is because the restart-related processes of steps S17 to S19 are performed at the timing of this change. Note that when step S19 is executed for the second or subsequent times, the number of retries counted in step S88 of the reading routine may be set to the value previously counted. Also, the process differs from FIG. 5 in that after the answer is Yes in step S18 or after step S19 is completed, the process proceeds to step S16 instead of step S20.

[0197] In the above modification, it is possible that the target may change to an untargeted state after the target has been determined. In this case, in the imaging adjustment process of FIG. 15, the target brightness index value D T In some cases, step S108 is performed after the level of the target brightness index value D T At this point, it is advisable to temporarily return to level 0, and start from level 0 again the next time step S104 is executed.

[0198] As another modification, if the target has not been determined when a time period equivalent to the predetermined time range T has elapsed since the start of the reading routine, the reading routine (reading process 210) may be always restarted. Fig. 20 shows a flowchart of the process in this case, which corresponds to Fig. 5. The differences between the process in Fig. 20 and the process in Fig. 5 are as follows.

[0199] First, if the target is not set and the process proceeds from step S15 to S16 and it is determined that the reading routine is still being executed, then in step SB it is determined whether or not a time equivalent to a predetermined time range T has elapsed since the start of the most recent reading routine. This time may be a time slightly shorter than T. Then, if the time has elapsed, the process restarts the reading routine in step SC, and the process returns to step S13 and is repeated. If the time has not elapsed in step SB, the process returns to step S13 and is repeated.

[0200] 20, when the target is set, the reading process 210 is necessarily within a time period equivalent to the predetermined time range T from the start, so at this point, the image data of the frames within the period (predetermined time range T) counted in the most recent aim determination process 220 must be the object of processing (the judgment criteria of step SB are determined so that this holds true, taking into consideration the length of the frames). Therefore, it is considered that there is no case where the reading routine needs to be restarted in step S19, so steps S17 to S19 do not need to be executed. However, they may be executed just to be sure. In the above process, the processes in steps SB and SC correspond to a first read control procedure, and step S15 corresponds to a second read control procedure, which correspond to the functions of the first read control unit and the second read control unit, respectively.

[0201] In the above modification, there is no need to restart the reading process 210 when the target is determined, and when the target is determined, the operation shown in FIG. 4A will be performed with a probability of 1. Therefore, if the average time T4 required from the timing of arrow A to obtaining the reading result in this modification is estimated in the same way as in the above-mentioned cases of T1 to T3, then T4=a / 2+s It is. Then, since T3-T4=a(fa) / 2f, if it can be determined by the aim determination process 220 that the aim is set in less than f (if f>a), it is believed that the time required for reading can be further shortened compared to the above-described embodiment.

[0202] However, in this modification, before the target is determined, the reading process 210 is restarted without waiting for its completion. Therefore, except for special cases where the reading is successful within a predetermined time range T, the reading will not be successful before the target is determined. In many cases, it is unlikely that a successful read will be achieved before the target is determined, so this is not a major inconvenience, but it is important to keep this in mind when adopting this modified example.

[0203] The criterion in step SB may be the timing of capturing an image to be processed in the reading routine being executed. In other words, if a time period equivalent to a predetermined time range T has elapsed since the capturing timing, the reading routine may be restarted in step SC. Even in this case, once the aim is determined, the image data of the frames within the period tallied in the most recent aim determination process 220 is always the subject of the reading process 210.

[0204] When the image capturing timing is used as the criterion, the image data to be processed in the restarted reading routine does not have to be that of the latest frame. If image data captured before the latest frame is used, the time from the start of the reading routine to the next time step SB becomes Yes and the reading routine is restarted will be shorter, but if image data captured within a predetermined time range T going back from the restart point is used, the same effect of shortening the required reading time can be obtained as in the case of the latest frame.

[0205] As another modified example, the brightness index value D c It is also possible to classify pixels to obtain θ by a method different from that of the embodiment described above. For example, under the assumption that the distribution of pixel values ​​of pixels capturing the black parts of a code symbol or character and pixels capturing the white (or background) parts both follow a normal distribution, classification may be performed by using a statistical method to determine a threshold value that minimizes the average misclassification rate (the probability that a black part is classified into the light pixel class, or vice versa) when all pixels are classified into black pixel class and white pixel class using a single threshold value. Specifically, this classification can be performed by determining a threshold value n that minimizes the value of J in the following equation 6.

[0206]

number

[0207] Based on the threshold value n obtained using this method, the brightness index value D c Even if the brightness index value D is calculated, it is possible to obtain substantially the same effect as in the above embodiment with respect to the adjustment of the imaging conditions. By calculating the variance of the pixel values ​​of the pixels classified into each class by the threshold value for various threshold candidate values ​​and classifying the classes based on the calculated variance, the brightness index value D c A meaningful threshold n can be obtained to find

[0208] It is useful to perform classification so that the degree of variance of pixel values ​​of pixels in each class is minimized as in the above-mentioned embodiment, and it is also useful to perform classification so that the average misclassification rate is minimized as in this modified example. Other criteria can also be used. As long as classification is performed so that pixels capturing black parts are classified into a class of pixels that are as dark as possible, and pixels capturing white parts or the background are classified into a class of pixels that are as bright as possible, a specific algorithm is not important. The accuracy of classification does not need to be strict. In order to perform such classification, it is useful to perform classification at least based on the variance of pixels classified into each class.

[0209] As yet another variant, the functions of the reading device 100 in each of the above-mentioned embodiments may be distributed across multiple devices, and for example, some of the functions shown in Figure 2, etc. may be provided in the connected data processing device. Furthermore, the reading device 100 may be a device that is held by an operator, or a stationary device that mainly moves the reading target 101 to align it with the imaging range of the image sensor 111. The above-described embodiment is applicable even when either the reading device 100 or the reading target 101, or both, are moved to determine the target. The movement of the image sensor 111 may be considered as a relative movement with respect to the reading target 101 and the optical information carried thereon.

[0210] Furthermore, in determining whether or not the aim is set in the aim determination process 220, it is not essential to take into account the movement range or movement trajectory of the image sensor 111. It is also possible to omit step S43 in Fig. 10 and make the determination mainly based on the amount of movement in step S42. Even when making the determination based on this criterion, the effect of shortening the required time can be obtained by performing the processes of steps S17 to S19 in Fig. 5. However, whether or not the aim is set can be determined more accurately by also performing step S43.

[0211] In the above embodiment, the reading object by the reading device 100 is a code symbol or a character, but the present invention can also be applied to a case where face recognition is performed to distinguish a face. In this case, an image of a person's face is captured, and instead of a decoding process, a process of comparing the image with a face pattern is performed. Even in this case, the function of restarting the reading process 210 according to the result of the aim determination process 220, the function of changing the width of the dimming range and the method of calculating the brightness index value and the target brightness index value based on whether the aim is set, the function of raising the target level of the image brightness adjustment when the reading fails when the aim is set, and the function of determining the upper limit of the exposure time and the illumination lighting time according to the movement amount of the image sensor 111 are generally effective in the same manner as in the above embodiment.

[0212] Furthermore, an embodiment of the program of the present invention is a program for causing one computer, or multiple computers working in cooperation, to control the required hardware and realize the functions of the reading device 100 in the above-mentioned embodiment, or to execute the processing described in the above-mentioned embodiment.

[0213] Such a program may be stored in a ROM or other non-volatile storage medium (flash memory, EEPROM, etc.) that is included in the computer from the beginning. It may also be provided by recording it on any non-volatile recording medium such as a memory card, CD, DVD, or Blu-ray disc. Furthermore, it may be downloaded from an external device connected to a network and installed on a computer to be executed.

[0214] Furthermore, it goes without saying that the configurations of the embodiments and modified examples described above can be implemented in any combination as long as they are not mutually inconsistent, and that only some of them can be extracted and implemented. [Explanation of symbols]

[0215] 20a, 20b...image, 21...code symbol, 22a, 22b...reference point, 23, 24...template image, 25...search range, 41...movement end position, 42a to 42n...movement vector, 43...movement start position, 44...convergence range, 50a, 50b...image, 51, 52...dimming range, 53...aimer light, 60...graph of relative frequency distribution, 100...reading device, 101...reading object, 102a...code symbol, 102b...character string, 110...optical unit, 111...imaging sensor, 112...lens, 113...LD, 114...pulse LED, 120...control unit, 131...operation unit, 132...notification unit, 133...display unit, 141...imaging unit, 142...image acquisition unit, 143...information reading unit, 144...output unit, 145...aimaging determination unit, 146...reading control unit, 147...imaging condition setting unit, 148...exposure time upper limit setting unit, 201...frame (imaging frame), 210...reading process, 211...object analysis process, 212...imaging adjustment process, 213...filter process, 214...object extraction / decoding process, 220...aimaging determination process

Claims

1. an imaging procedure for periodically capturing images by an imaging unit; a reading step of analyzing the image captured in the imaging step and reading optical information contained in the image; an estimation step of estimating a movement amount of the imaging unit in a certain time range based on an image captured in the imaging step in the certain time range; an irradiation step of irradiating an aimer light which serves as a reference for directing the imaging unit to optical information to be read; an aimer light detection step of detecting a position of an aimer light in an image captured in the imaging step; an imaging adjustment step of adjusting an imaging condition of the imaging unit based on a pixel value of a pixel in an image captured in the imaging step; the imaging adjustment step is a step of adjusting an imaging condition of the imaging unit based on pixel values ​​of pixels within a predetermined range near the position of the detected aimer light in the image captured in the imaging step when it is determined that the amount of movement estimated in the estimation step is equal to or less than a predetermined first standard.

2. an imaging procedure for periodically capturing images by an imaging unit; a reading step of analyzing the image captured in the imaging step and reading optical information contained in the image; an estimation step of estimating a movement amount of the imaging unit in a certain time range based on an image captured in the imaging step in the certain time range; an irradiation step of irradiating an aimer light which serves as a reference for directing the imaging unit to optical information to be read; an aimer light detection step of detecting a position of the aimer light in an image captured in the imaging step; an imaging adjustment step of adjusting an imaging condition of the imaging unit based on pixel values ​​of pixels around the position of the aimer light in an image captured in the imaging step; the imaging adjustment step is a step of adjusting imaging conditions of the imaging unit based on pixel values ​​of pixels in a narrower range around the position of the aimer light when it is determined that the amount of movement estimated in the estimation step is equal to or less than a predetermined first standard, compared to when it is determined that the amount of movement estimated in the estimation step is not equal to or less than the first standard.

3. an imaging procedure for periodically capturing images by an imaging unit; a reading step of analyzing the image captured in the imaging step and reading optical information contained in the image; an estimation step of estimating a movement amount of the imaging unit in a certain time range based on an image captured in the imaging step in the certain time range; an imaging adjustment step of adjusting an imaging condition of the imaging unit based on a pixel value of a pixel at a predetermined reference position in an image captured in the imaging step; the imaging adjustment procedure is a procedure of adjusting the imaging conditions of the imaging unit based on pixel values ​​of pixels in a narrower range of the image when it is determined that the amount of movement estimated in the estimation procedure is equal to or less than a predetermined first standard, compared to when it is determined that the amount of movement estimated in the estimation procedure is not equal to or less than the first standard.

4. An imaging unit; an imaging control unit that periodically causes the imaging unit to capture images; a reading unit that analyzes the image captured by the imaging unit and reads optical information contained in the image; an estimation unit that estimates a movement amount of the imaging unit within a certain time range based on an image captured by the imaging unit within the time range; an irradiation unit that irradiates an aimer light that serves as a reference for directing the imaging unit to optical information to be read; an aimer light detection unit that detects a position of the aimer light in an image captured by the imaging unit; an imaging adjustment unit that adjusts an imaging condition of the imaging unit based on a pixel value of a pixel in an image captured by the imaging unit; When the imaging adjustment unit determines that the amount of movement estimated by the estimation unit is equal to or less than a predetermined first standard, the imaging adjustment unit adjusts the imaging conditions of the imaging unit based on pixel values ​​of pixels within a predetermined range near the position of the detected aimer light in the image captured by the imaging unit.

5. An imaging unit; an imaging control unit that periodically causes the imaging unit to capture images; a reading unit that analyzes the image captured by the imaging unit and reads optical information contained in the image; an estimation unit that estimates a movement amount of the imaging unit within a certain time range based on an image captured by the imaging unit within the time range; an irradiation unit that irradiates an aimer light that serves as a reference for directing the imaging unit to optical information to be read; an aimer light detection unit that detects a position of the aimer light in an image captured by the imaging unit; an image capturing adjustment unit that adjusts an image capturing condition of the image capturing unit based on pixel values ​​of pixels around the position of the aimer light in an image captured by the image capturing unit; an imaging adjustment unit that adjusts the imaging conditions of the imaging unit based on pixel values ​​of pixels in a narrower range around the position of the aimer light when the estimation unit determines that the estimated movement amount is equal to or less than a predetermined first standard, compared to when the estimation unit determines that the estimated movement amount is not equal to or less than the first standard.

6. An imaging unit; an imaging control unit that periodically causes the imaging unit to capture images; a reading unit that analyzes the image captured by the imaging unit and reads optical information contained in the image; an estimation unit that estimates a movement amount of the imaging unit within a certain time range based on an image captured by the imaging unit within the time range; an imaging adjustment unit that adjusts imaging conditions of the imaging unit based on a pixel value of a pixel at a predetermined reference position in an image captured by the imaging unit; The imaging adjustment unit adjusts the imaging conditions of the imaging unit based on pixel values ​​of pixels in a narrower range of the image when the estimation unit determines that the estimated movement amount is equal to or less than a predetermined first standard, compared to when the estimation unit determines that the estimated movement amount is not equal to or less than the first standard.

7. A program for causing a processor that controls the imaging unit to execute the optical information reading method according to claim 1 .

Images