Two-dimensional code reading system and two-dimensional code

Abstract

To provide a two-dimensional code reading system configured to improve reading speed and the accuracy of acquiring color information of an ink which changes depending on environmental changes.SOLUTION: A two-dimensional code has a positioning pattern colored in a reference color which is used for determining the amount of change in color according to an imaging environment. A reading system includes an image acquisition apparatus which acquires an image of the two-dimensional code, a data processing apparatus, and a storage apparatus which stores color information of the reference color, and position information of the reference color and the ink. The data processing apparatus detects coordinates of four corners of the two-dimensional code from the image of the two-dimensional code, to identify the position of a region where the color information is to be detected, detects the color information in the region, the region including a reference color region and an ink region, determines the reference color region and the ink region based on the position information of the reference color stored in the storge apparatus and the color information and the position of the region, and corrects color information in the ink region using a relationship between the color information of the reference color stored in the storage apparatus and the detected color information of the reference color region.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a two-dimensional code reading system and a two-dimensional code. [Background technology]

[0002] Indicators that estimate the environment to which a cord has been exposed and management systems that utilize these indicators are known. Patent Document 1 describes a temperature evaluation system that includes "a processing device that includes: an input device that acquires image data of an indicator that uses a thermochromic material; a storage device (database) that stores the relationship between color density and temperature for each thermochromic material; a color density estimation unit that estimates the color density of the thermochromic material from the image data; a material identification unit that identifies the thermochromic material used in the indicator; and a temperature estimation unit that selects the relationship between color density and temperature of the thermochromic material identified by the material identification unit from the relationship between color density and temperature for each thermochromic material, and estimates the maximum or minimum temperature to be reached from the relationship between color density and density of the identified thermochromic material and the color density estimated by the color density estimation unit."

[0003] This temperature evaluation system is configured with an indicator consisting of a thermochromic material that indicates the environment to which it is exposed by color density, and a one-dimensional or two-dimensional barcode. Furthermore, color information of the thermochromic material is obtained from a photographed image of the indicator attached to the item, and the maximum or minimum temperature reached by the item is estimated. This allows for centralized management of the thermochromic data acquired at each location during distribution by having a management device (e.g., a management server) that manages the environment in which the item is placed, and management terminals (input devices) installed at each location.

[0004] However, in the system described in Patent Document 1, the shooting environment and shooting equipment have a strong influence when capturing an image of an indicator, making it difficult to accurately obtain color information of the thermochromic material from the captured image. Examples of shooting environments that affect images include the type and brightness of the light source. The spectrum of light hitting the subject varies depending on the type of light source (lighting model)—direct sunlight, cloudy daylight, fluorescent light, incandescent light—and the degree of light attenuation from the light source. In many cases, there are multiple light sources, and their light may be mixed. It is not uncommon for the location and orientation of the sensor, the photographer's position, and other factors to vary depending on the shooting conditions. Furthermore, the time of shooting (sun position) and weather are constantly changing. These factors cause the brightness and spectrum of the light hitting the sensor of the subject to change in various ways.

[0005] To address this issue, Patent Document 2 describes a measurement device that includes an analysis unit that calculates a correction value for the color sample from color image data obtained by simultaneously capturing an image of a color-changing color sensor and three color samples, and a correction unit that corrects the color of the color sensor using the correction value.As a result, even when the color sensor is captured under various light sources, the color of the color sensor can be corrected based on the color samples captured in the same image.

[0006] Patent Document 3 also describes a color evaluation system that obtains highly accurate measurement values ​​from color sensors photographed in various shooting environments. The color information of four or more reference colors photographed in a first shooting environment is registered in a storage device, and the sensor color of a color sensor that changes depending on the physical quantity being measured and four or more reference colors that do not change depending on the physical quantity being measured are acquired from the shooting data during measurement. The sensor color and the four or more reference colors are acquired from the shooting data during measurement. The color conversion coefficient between the first shooting environment and the second shooting environment is calculated based on the amount of change from the color information of the four or more reference colors in the first shooting environment read from the storage device to the color information of the four or more reference colors in the second shooting environment acquired from the shooting data. The color evaluation system includes a color evaluation unit that corrects the sensor color to make it look like it was photographed in the first shooting environment by calculating a conversion equation including a term indicating the parallel translation of the affine transformation based on the sensor color acquired from the shooting data during measurement and the color conversion coefficient. This color evaluation system obtains highly accurate measurement values ​​from color sensors photographed in various shooting environments. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 2018-205222 [Patent Document 2] Japanese Patent Application Laid-Open No. 2012-093277 [Patent Document 3] Japanese Patent Publication No. 2020-038073 Summary of the Invention [Problem to be solved by the invention]

[0008] While the technologies described in Patent Documents 2 and 3 can obtain with high precision the color tone of the temperature-indicating material part, which indicates the environment to which it is exposed by color density, they are subject to limitations in the system required to obtain the code information in the indicator, the color information of the reference color part, and the color information of the temperature-indicating material part, making it difficult to miniaturize the code and limiting the products that can be used. Also, obtaining both code information and high-precision color information requires slow processing, and it takes time to read multiple indicators, resulting in issues that make them unsuitable for practical use.

[0009] The present invention is a two-dimensional code reading system that has an environmental detection area in the data area that changes color in response to environmental changes in order to solve the above-mentioned problems, and aims to improve the accuracy of obtaining color information and the reading speed of the environmental detection area that changes color in response to environmental changes. [Means for solving the problem]

[0010] To achieve the above object, the present invention provides a two-dimensional code reading system having an environment detection area in its data area that changes color in response to environmental changes, comprising an image capture device for capturing an image of the two-dimensional code, a data processing device, and a storage device for storing color information for a reference color and position information for the reference color and the environment detection area, wherein the two-dimensional code has a positioning pattern colored with a reference color used to determine the amount of color change depending on the shooting environment, and the data processing device recognizes a reference position of the two-dimensional code from the image of the two-dimensional code, determines the area colored with the reference color and the environment detection area in the acquired image of the two-dimensional code based on the detected reference position and the position information for the reference color and the environment detection area stored in the storage device, detects color information for the reference color and the environment detection area, and corrects the color information of the environment detection area using the relationship between the color information for the reference color stored in the storage device and the color information for the detected area of ​​the reference color. Other aspects of the present invention will be described in the embodiments below. [Effects of the Invention]

[0011] According to the present invention, a two-dimensional code reading system can be provided that has an environmental detection area in the data area that changes color in response to environmental changes, improving the accuracy of obtaining color information and reading speed of the environmental detection area, which changes color in response to environmental changes. [Brief explanation of the drawings]

[0012] [Figure 1A] 2 is a schematic diagram of a label item and a first two-dimensional code according to the present embodiment. FIG. [Figure 1B]3 is a schematic diagram of a finder pattern of a two-dimensional code according to the present embodiment. FIG. [Figure 1C] 3A and 3B are schematic diagrams of alignment patterns of a two-dimensional code according to the present embodiment. [Figure 1D] FIG. 4 is a schematic diagram of a second two-dimensional code according to the present embodiment. [Figure 1E] FIG. 10 is a schematic diagram of a third two-dimensional code according to the present embodiment. [Figure 1F] FIG. 10 is a schematic diagram of a fourth two-dimensional code according to the present embodiment. [Figure 1G] FIG. 10 is a schematic diagram of a fifth two-dimensional code according to the present embodiment. [Figure 1H] FIG. 10 is a schematic diagram of a sixth two-dimensional code according to the present embodiment. [Figure 1I] FIG. 10 is a schematic diagram of a seventh two-dimensional code according to the present embodiment. [Figure 1J] FIG. 10 is a schematic diagram of an eighth two-dimensional code according to the present embodiment. [Figure 1K] FIG. 10 is a schematic diagram of a ninth two-dimensional code according to the present embodiment. [Figure 1L] FIG. 12 is a schematic diagram of a tenth two-dimensional code according to the present embodiment. [Figure 2] FIG. 1 is a diagram illustrating a configuration of a reading system according to an embodiment of the present invention. [Figure 3] 10A to 10C are diagrams illustrating a color correction method according to the present embodiment. [Figure 4A] 10A and 10B are diagrams illustrating a color determination method according to the present embodiment. [Figure 4B] 10A and 10B are diagrams illustrating a color determination method according to the present embodiment. [Figure 5] FIG. 2 is a diagram illustrating a processing flow of the reading system according to the embodiment. [Figure 6] 10A and 10B are diagrams illustrating an example of a display of an output result of the reading system according to the present embodiment. [Figure 7] 1 is a diagram illustrating an outline of an information processing system that utilizes a reading system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] The present invention will be described below, but the present invention is not limited to the following content and can be implemented by any modifications within the scope that does not significantly impair the effects of the present invention. The present invention can be implemented by combining different embodiments. In the following description, the same components in different embodiments are given the same reference numerals, and redundant explanations will be omitted.

[0014] <Two-dimensional code> First, a two-dimensional code applied to a reading system according to an embodiment of the present invention will be described. There are multiple possible two-dimensional codes that can be applied to this reading system, but in the following embodiment, a two-dimensional code using QR Code (registered trademark) will be described as an example. QR Code is standardized by ISO / IEC 18004 and the like. It should be noted that PDF417, DataMatrix, Maxicode, AztecCode, and the like can also be used as the two-dimensional code.

[0015] FIG. 1A is a schematic diagram of a label item 100 on which a two-dimensional code 200 (first two-dimensional code) according to this embodiment is arranged. The label item 100 is composed of the two-dimensional code 200 and a serial ID 101. The two-dimensional code 200 includes three finder patterns 300 (positioning patterns) for position detection arranged at three corners of the two-dimensional code 200, a first reference color 301, a second reference color 302, and a third reference color 303 arranged within the three finder patterns 300, cells 500 constituting a data area, and an alignment pattern 400 for correcting misalignment of the cells 500 caused by distortion. The two-dimensional code 200 further includes an environment detection area 600 arranged to overlap the data area composed of the cells 500. The environment detection area 600 is an area whose color changes depending on environmental changes. The finder patterns 300 for position detection are colored with a reference color. 1A, different reference colors (first reference color 301 to third reference color 303) are assigned to three finder patterns 300. Here, the reference colors are used to find the amount of color change depending on the shooting environment.

[0016] By arranging the first reference color 301, the second reference color 302, the third reference color 303 and the environment detection area 600 all within the two-dimensional code 200, distortion of the two-dimensional code can be reduced when the two-dimensional code is read by the reading system 700 described below compared to when they are arranged outside the two-dimensional code, and therefore the two-dimensional code itself can be made smaller.

[0017] Furthermore, by arranging the first reference color 301, second reference color 302, third reference color 303, and environment detection area 600 all within the two-dimensional code 200, the two-dimensional code, reference colors, and environment detection area are positioned close to each other (because they are located inside), which makes it possible to speed up reading of the two-dimensional code using the reading system 700 described below, compared to when they are arranged outside the two-dimensional code. For example, the reference color can be arranged in the start pattern or stop pattern when PDF417 is used instead of the QR code, in the alignment pattern or clock pattern when DataMatrix is ​​used, or in the central finder pattern when Maxicode or AztecCode is used.

[0018] The two-dimensional code 200 can be arranged in the same form as the label 100 and used in the form of a sticker. A sticker-type label is preferable in terms of ease of handling and installation and cost, but it can be used in any form, such as a tag or card, depending on the product to be managed.

[0019] The serial ID 101 attached to the label display item 100 is an ID number linked to the two-dimensional code 200, and can be written anywhere on the label display item 100, making it possible to visually manage the two-dimensional code 200. It is desirable to place the serial ID 101 with a space (quiet zone) of four cells away from the two-dimensional code 200.

[0020] 1A is a version 2 (25 × 25 cells) QR code. The cell configuration (version) of the QR code can be changed depending on the content of the information to be input, but version 2 or higher is preferred because it includes an alignment pattern 400 for distortion correction.

[0021] The two-dimensional code 200 in Fig. 1A is a QR code with an error correction level of "H." The error correction level of a QR code can be lowered depending on the amount of information and the application, but it is desirable to set the error correction level to "H" in order to prevent reading errors when the QR code is intentionally missing or when it becomes stained, dirty, or damaged during the logistics process by arranging an environment detection area 600 within the QR code.

[0022] The environment detection area 600 disposed within the two-dimensional code 200 is not limited to a medium such as ink that changes color in response to environmental changes, but from a manufacturing process perspective, it is preferable that the medium be in the form of ink that can be printed on the two-dimensional code 200. Examples of the environment detection area 600 include those that reflect, as color information, the results of detecting environmental conditions such as temperature, temperature history, humidity, light, gas concentration, and vibration, as well as the pH of a liquid, the concentrations of various ions in a liquid, the concentrations of various pharmaceuticals, the concentrations of various amino acids and proteins, and the presence of viruses and bacteria.

[0023] In this embodiment, the RGB (Red, Green, Blue) model is used as the color model for explanation. In the RGB model, the colors that make up an image are expressed as a combination of values ​​ranging from 0 (the darkest) to 255 (the brightest) for each of the three colors R (red), G (green), and B (blue). Hereinafter, RGB components are expressed in square brackets as [R, G, B], e.g., red [255, 0, 0], darkest black [0, 0, 0], and brightest white [255, 255, 255]. Note that black or a color equivalent to black is used for the color of untouched colored portions of the two-dimensional code 200 in this embodiment.

[0024] The first reference color 301, second reference color 302, and third reference color 303 provided inside the three finder patterns 300 arranged at the three corners of the two-dimensional code 200 are used for color correction of the environment detection area 600 (correction of color information of the environment detection area 600) according to the shooting environment, and are fixed colors (unlike the color of the environment detection area 600, do not change) colored with ordinary color inks, etc. Examples of the three reference colors are shown below. The first reference color 301 is painted red [255,0,0]. The second reference color 302 is painted green [0,255,0]. The third reference color 303 is painted blue [0,0,255].

[0025] When printing a two-dimensional code 200 in color, if full color is expressed using the four colors CMYK (cyan, magenta, yellow, and black), there are areas that can be reproduced in RGB but cannot be reproduced in CMYK. Therefore, there is no particular problem if red is used as the first reference color 301, green as the second reference color 302, and blue as the third reference color 303 in the area that can be reproduced in CMYK. Furthermore, possible combinations of reference colors are not limited to red, green, and blue, but it is preferable to select three colors that are located at different positions in the RGB color space. Specifically, it is preferable that each of the three finder patterns is colored with a reference color that has a different vector direction connecting it to white (255,255,255) in the RGB space.

[0026] FIG. 1B is an enlarged view of finder pattern 300 within two-dimensional code 200. Finder pattern 300 is composed of first finder pattern region 310, second finder pattern region 320, and third finder pattern region 330. This finder pattern allows the position of the two-dimensional code to be recognized, enabling high-speed reading. In this embodiment, a reference color is assigned within finder pattern 300, but the finder pattern must retain the function of recognizing the position of the two-dimensional code. From any position in the A, B, or C directions of finder pattern 300, the ratio of white (light color) cells to black (dark color) cells must be arranged at a ratio of 1:1:3:1:1. Colors can be assigned to any portion within this rule. The reference colors to be placed in the first finder pattern area 310, the second finder pattern area 320, and the third finder pattern area 330 are not particularly limited as long as they are colors that can be read by a two-dimensional code reader, but the colors used in the first finder pattern area 310 and the third finder pattern area 330 are preferably colors that are recognized as being similar to dark colors or low-brightness black, and the colors used in the second finder pattern area 320 are preferably colors that are recognized as being similar to light colors or high-brightness white cells.

[0027] FIG. 1C is an enlarged view of an alignment pattern 400 in a two-dimensional code 200. The alignment pattern has the function of correcting misalignment of each cell caused by distortion. The alignment pattern is composed of a first alignment pattern region 410, a second alignment pattern region 420, and a third alignment pattern region 430. Each of these alignment pattern regions can also have the first reference color 301, the second reference color 302, and the third reference color 303, as well as an environment detection region 600. However, the function of correcting misalignment of cells caused by distortion must be maintained. The reference colors to be placed in the first finder pattern area 310, the second finder pattern area 320, and the third finder pattern area 330 are not particularly limited as long as they are colors that can be read by a two-dimensional code reader, but the colors used in the first alignment pattern area 410 and the third alignment pattern area 430 are preferably colors that are recognized as dark colors or low-brightness black, and the colors used in the second alignment pattern area 420 are preferably colors that are recognized as light colors or high-brightness white cells.

[0028] Specific embodiments of two-dimensional codes that can be applied to the reading system of the present invention are shown in Figures 1D to 1L below. Figures 1D to 1L are schematic diagrams of two-dimensional codes (second to tenth two-dimensional codes) that are different from the two-dimensional code (first two-dimensional code) used in the display object of Figure 1A.

[0029] Two-dimensional code 201 (second two-dimensional code) shown in Fig. 1D is a QR code in which a first reference color 301 and a second reference color 302 are arranged in a first finder pattern area 310 (see Fig. 1B) of each of two of the three finder patterns 300 arranged at the three corners of the QR code. The two finder patterns to be colored with the reference colors can be selected arbitrarily, and the reference color to be colored in first finder pattern area 310 can be set arbitrarily.

[0030] Two-dimensional code 202 (third two-dimensional code) shown in FIG. 1E is a QR code in which a first reference color 301 is placed in a first finder pattern area 310 (see FIG. 1B) of one of three finder patterns 300 arranged at the three corners of the QR code. The one finder pattern to be assigned the reference color can be selected arbitrarily, and the reference color to be assigned to first finder pattern area 310 can be set arbitrarily. As described above, it is sufficient that the reference color is assigned to at least one of the three finder patterns.

[0031] The two-dimensional code 203 (fourth two-dimensional code) shown in FIG. 1F is a QR code in which a first reference color 301 is placed in the first finder pattern areas 310 (see FIG. 1B) of all three finder patterns 300 located at the three corners of the QR code. The reference color to be placed in the first finder pattern area 310 can be set arbitrarily. As described above, the reference colors of the finder patterns may be the same color. As mentioned above, it is desirable to select three colors that are located at distant positions in the RGB color space.

[0032] 1G (fifth two-dimensional code) is a QR code in which a first reference color 301, a second reference color 302, and a third reference color 303 are arranged in third finder pattern areas 330 (see FIG. 1B) of three finder patterns 300 arranged at three corners of the QR code. The reference colors arranged in third finder pattern area 330 can be set arbitrarily.

[0033] Two-dimensional code 205 (sixth two-dimensional code) shown in Fig. 1H is a QR code in which first reference color 301, second reference color 302, and third reference color 303 are arranged in first finder pattern area 310 (see Fig. 1B) and third finder pattern area 330 (see Fig. 1B) of three finder patterns 300 arranged at the three corners of the QR code. The reference colors to be arranged in first finder pattern area 310 and third finder pattern area 330 can be set arbitrarily.

[0034] Two-dimensional code 206 (seventh two-dimensional code) shown in FIG. 1I is a QR code in which first reference color 301, fourth reference color 304, and third reference color 303 are arranged in first finder pattern region 310 (see FIG. 1B), second finder pattern region 320 (see FIG. 1B), and third finder pattern region 330 (see FIG. 1B) of three finder patterns 300 arranged at three corners of the QR code, respectively. The reference colors to be arranged in first finder pattern region 310, second finder pattern region 320, and third finder pattern region 330 can be set arbitrarily, but fourth reference color 304 arranged in second finder pattern region 320 is preferably a light color such as gray [200, 200, 200] that is close to white.

[0035] The two-dimensional code 207 (eighth two-dimensional code) shown in Figure 1J is a QR code in which a fourth reference color 304 is arranged in the second alignment pattern region 420 (see Figure 1C) of the QR code alignment pattern 400. This color arrangement allows the fourth reference color 304 to be arranged to surround four environment detection regions 600 that are located at different positions in the RGB color space, thereby improving the color accuracy of the environment detection region. The reference colors to be arranged in the first finder pattern region 310 (see Figure 1B), the second finder pattern region 320 (see Figure 1B), the third finder pattern region 330 (see Figure 1B), and the second alignment pattern region 420 (see Figure 1C) can be set arbitrarily.

[0036] The two-dimensional code 208 (ninth two-dimensional code) shown in FIG. 1K is a QR code in which an environment detection area 600 is placed in a different position from the QR code. The environment detection area 600 can be placed in any position within the two-dimensional code as long as it does not overlap with the finder pattern 300 or the alignment pattern 400. The environment detection area 600 can be expanded up to 30% of the area occupied by the cell 500, and can be placed in any size and position as long as the two-dimensional code can be read using a commercially available two-dimensional code reader or smartphone camera.

[0037] The two-dimensional code 209 (tenth two-dimensional code) shown in FIG. 1L is a two-dimensional code in which the environment detection region 600 of the two-dimensional code 200 (first two-dimensional code) is colored in the first alignment pattern region 410 of the alignment pattern 400. By utilizing the alignment pattern 400 as the environment detection region 600, it is possible to prevent loss of cells 500 in the two-dimensional code, making it easier to obtain information from the two-dimensional code. Using a similar concept, the environment detection region 600 can also be provided in the finder pattern 300. When providing the environment detection region 600 in either the alignment pattern 400 or the finder pattern 300, it is necessary to provide an environment detection region that can maintain the functionality of the alignment pattern 400 or the finder pattern 300 before and after the color of the environment detection region 600 changes.

[0038] <Reading system> Next, a reading system according to an embodiment of the present invention will be described. 2 is a configuration diagram of a two-dimensional code reading system 700. The reading system 700 is used to read, for example, the two-dimensional codes shown in FIGS. 1A and 1C to 1L. The reading system 700 includes an image capture device 710 that captures an image of the two-dimensional code, an input device 720, an output device 730, a data processing device 740, and a storage device 760.

[0039] The image capture device 710 is an imaging device such as a camera, and captures an image of the label marking. In addition to the label marking, it is also possible to capture images of the memorized product, information related to the product, the surrounding environment, etc., as needed. Once the image of the two-dimensional code is captured, it is stored in the image data storage unit 761 in the storage device 760.

[0040] The input device 720 is a part that receives instructions from an operator, and is composed of buttons, a touch panel, and the like.

[0041] The output device 730 is a device that outputs instruction information, scanned images, scanning results, etc. to the operator, and is configured with a display and a communication device. This configuration is standard, and any or all of the image acquisition device 710, input device 720, and output device 730 may be configured to be connected to the outside of the scanning system 700.

[0042] (Storage device) The storage device 760 is a part that stores various data and is composed of the following storage units: There are.

[0043] The image data storage unit 761 is a part that stores the image of the two-dimensional code input from the image acquisition device 710.

[0044] The two-dimensional code position data storage unit 762 is a part that stores data representing the reference position of the code recognized from the image stored in the image data storage unit 761 by the two-dimensional code position recognition unit 742, which will be described later.

[0045] The two-dimensional code data storage unit 763 is a part that stores data of a character string represented by a code recognized from an image stored in the image data storage unit 761 by a two-dimensional code recognition unit 743 (to be described later).

[0046] The reference color position data storage unit 764 is a part that stores in advance the position information of the reference color set in the two-dimensional code.

[0047] The environment detection area position data storage unit 765 is a part that stores in advance the position information of the environment detection area set within the two-dimensional code.

[0048] The reference color measured color data storage unit 766 is a part that stores measured color information (RGB values, etc.) of the reference color extracted from the reference color area specified by the reference color position data storage unit 764.

[0049] The environment detection area measured color data storage unit 767 is a part that stores measured color information (RGB values, etc.) of the environment detection area extracted from the area of ​​the environment detection area identified by the environment detection area position data storage unit 765.

[0050] The reference color base data storage unit 768 is a portion that stores color information (RGB values, etc.) of the reference color that serves as a standard used to correct the actually measured color information (RGB values, etc.) of the reference color.

[0051] The corrected color data storage unit 769 of the environment detection area is a part that stores color information (RGB values, etc.) obtained by correcting the actually measured color information (RGB values, etc.) of the environment detection area stored in the image data storage unit 761.

[0052] The environment detection area judgment color data storage unit 770 is a unit that stores, for example, color information (RGB values, etc.) of a threshold value that serves as a judgment standard for determining whether or not there is a temperature deviation, and the relationship between the amount of environmental change and the color information of the environment detection area, based on the corrected color of the environment detection area stored in the environment detection area correction color data storage unit 769. The relationship between the amount of environmental change and the color information of the environment detection area indicates, for example, the temperature of the surrounding environment that corresponds to a certain color in the environment detection area.

[0053] The judgment result memory unit 771 is a part that stores results such as "NG" / "OK" or "■" / "■" determined by the judgment unit 749 described below based on the magnitude of the RGB values ​​stored in the correction color data memory unit 769 of the environment detection area relative to the RGB values ​​stored in the judgment color data memory unit 770 of the environment detection area.

[0054] The read data storage unit 772 is a unit that stores data other than that processed by the data processing device 740, which will be described later. In general, when grasping changes in the product environment, it is desirable to have all the information about when, where, and what happened. Therefore, in addition to the product-related information stored in the two-dimensional code data storage unit 763 and data such as temperature deviation from the environment detection area stored in the determination result storage unit 771, it is preferable to store the date and time of the read process, location information, the number of the reading device (to identify the worker), weather information linked to web information, and the like. Note that as long as the same reading device is used, the reading device number does not change, so it may be added when outputting data, which will be described later.

[0055] (Data Processing Device) The data processing device 740 processes data input from the image acquisition device 710 and the input device 720 and data stored in the memory device 760, and outputs the results to the output device 730 or stores them in the memory device 760, and is composed of the following processing units.

[0056] The input control unit 741 is a part that classifies data input from the image acquisition device 710 or the input device 720 into commands, data, etc., and transfers them to the storage device 760 and each part of the data processing device 740. In particular, as main data, it transfers image data of a two-dimensional code including a reference color and an environment detection area to the image data storage unit 761.

[0057] The two-dimensional code position recognition unit 742 recognizes the position of a code from the image data stored in the image data storage unit 761 and stores the recognition results in the two-dimensional code position data storage unit 762. Image data is typically composed of hundreds to thousands of dots both vertically and horizontally, and the image itself does not contain data indicating which parts are codes. Therefore, the color data of each dot is analyzed to recognize which parts of the image data correspond to the reference position of the code. The display form and number of reference positions vary depending on the code standard, but are not limited to these standards. Furthermore, even if the image stored in the image data storage unit 761 contains multiple two-dimensional codes, the positions of multiple two-dimensional codes can be recognized.

[0058] The two-dimensional code recognition unit 743 is a part that uses the position data stored in the two-dimensional code position data storage unit 762 to recognize character string data represented by a code from the image stored in the image data storage unit 761 and stores it in the two-dimensional code data storage unit 763.

[0059] The reference color position determination unit 744 is a unit that identifies the type of two-dimensional code using character string data from the position data stored in the two-dimensional code position data storage unit 762 and the information stored in the two-dimensional code data storage unit 763, detects the coordinates (X, Y) of the four corners of the two-dimensional code from an acquired image of the two-dimensional code, and determines the positions of the reference colors (first reference color 301 to fourth reference color 304) based on the coordinates of the detected four corners and the position information previously stored in the reference color position data storage unit 764. Here, identifying the type of two-dimensional code means, for example, identifying which type of two-dimensional code it is, such as those shown in Figures 1A, 1D to 1L.

[0060] The environment detection area position determination unit 745 is a part that identifies the type of two-dimensional code using character string data from the position data stored in the two-dimensional code position data memory unit 762 and the information stored in the two-dimensional code data memory unit 763, detects the coordinates (X, Y) of the four corners of the two-dimensional code from the captured image of the two-dimensional code, and determines the position of the environment detection area 600 based on the coordinates of the detected four corners and the position information previously stored in the environment detection area position data memory unit 765.

[0061] The reference color measured color determination unit 746 is a part that determines the RGB values ​​of the reference colors in the acquired image from the image stored in the image data storage unit 761 and the position information of the reference colors (first reference color 301 to fourth reference color 304) determined by the reference color position determination unit 744.

[0062] The environment detection area measured color determination unit 747 is a part that determines the RGB values ​​of the environment detection area 600 in the acquired image from the image stored in the image data storage unit 761 and the position information of the environment detection area 600 determined by the environment detection area position determination unit 745.

[0063] The correction color determination unit 748 of the environment detection area is a part that determines RGB values ​​by correcting the color tones that have changed due to the lighting environment to the original colors from the measured RGB values ​​in the acquired image determined by the measured color determination unit 747 of the environment detection area.

[0064] The method for determining the correction color of the environment detection area is described below. FIG. 3 is a diagram illustrating a color correction method according to this embodiment. FIG. 3 illustrates a method for correcting the measured RGB values ​​of the environment detection area. The environment detection area correction color determination unit 748 calculates the difference between the RGB values ​​stored in the reference color base data storage unit 768 and the RGB values ​​of the reference color in the acquired image stored in the reference color measured color data storage unit 766, and applies the resulting difference in RGB values ​​to the RGB values ​​of the environment detection area in the acquired image stored in the environment detection area measured color data storage unit 767, thereby correcting the measured colors of the environment detection area in the acquired image. The corrected RGB values ​​of the environment detection area are stored in the environment detection area correction color data storage unit 769.

[0065] The specific color correction method is shown below. Here, we will show a method of correcting the color of the environment detection area 600 using three reference colors: a first reference color 301, a second reference color 302, and a third reference color 303, which are provided inside three finder patterns 300 in the two-dimensional code 200 shown in Fig. 1A. There is no limitation on the number of reference colors, and the following formula can be converted depending on the number of reference colors.

[0066] Regarding the reference color, the relationship between the original color and the color in the scanned image is as shown in equation (1).

number

[0067] The relationship between the reference colors can be written in matrix form as shown in equation (2). Here, R di and R si In the above, i indicates the number of each of the three reference colors provided.

number

[0068] In addition, the relationship of formula (3) holds in the environment detection region.

number

[0069] The relationship between these environment detection regions can be written in matrix form as shown in equation (4).

number

[0070] Therefore, if C is obtained from the relationship of the reference color, the color T of the environment detection area in the acquired image s By calculating the product of these, the original color of the environment detection area, T c You can see that.

[0071] To find C, use equation (5).

number

[0072] Returning to FIG. 2, the determination unit 749 compares the RGB values ​​stored in the correction color data storage unit 769 of the environment detection area with the RGB values ​​stored in the judgment color data storage unit 770 of the environment detection area, and determines, for example, whether or not there is a temperature deviation based on the magnitude of the vector values ​​of the RGB values ​​in the RGB space of FIGS. 4A and 4B, which will be described later.

[0073] FIG. 4A is a diagram for explaining a color determination method according to this embodiment. A first color determination method for determining the color of the environment detection area 600 will be explained with reference to FIG. 4A. The equation (6) is a function of the actually measured color (r m ,g m ,b m ) The measured color here is the color corrected by the reference color using the above method.

[0074]

number

[0075] The first color determination method for determining the color of the environment detection area 600 shown in FIG. 4A can return a color determination by calculating the simple formula (6), but it does not take into account the vector direction in the RGB color space, and will determine that the colors are the same even if the color tones are different if the vector lengths are the same.

[0076] Therefore, when the color tones of the environment detection area 600 are different but the values ​​of formula (6) are close, the second determination method shown in FIG. 4B can be applied.

[0077] In equations (7) and (8), the starting color (R i ,G i ,B i ) as the origin of the final color (R f ,G f ,B f ) and measured color (r m ,g m ,b m ) vector. The measured color here is the color corrected by the reference color using the above method. Final color (R,G,B)=(R f ?R i ,G f ?G i ,B f ?B i )...Equation (7) Measured color (r,g,b)=(r m ?R i ,g m ?G i ,b m ?B i )...Equation (8)

[0078] From equations (7) and (8), the lengths of the vectors from the starting color to the final color and the measured color (absolute values ​​of R and r, respectively) are expressed by equations (9) and (10).

[0079]

number

[0080] Actual measured color (r m ,g m ,bm ) to the vector of equation (7) M ,g M ,b M ) is expressed by equation (11).

[0081]

number

[0082] The inner product of the vectors of equations (9) and (10) is expressed by equation (12).

[0083]

number

[0084] The ratio of the length of the vector from the starting color to the correction color (the above equation (10)) to the length of the vector from the starting color to the final color (the above equation (9)) is calculated by substituting equations (11) and (12) into equations (9) and (10) to obtain equation (13).

[0085]

number

[0086] Returning to FIG. 2 , the output control unit 750 controls the output of information processed by the data processing device 740 to the output device 730. Specifically, this is a part that outputs to the output device 730 product information linked to the character string stored in the two-dimensional code data storage unit 763, information such as the presence or absence of a temperature deviation determined from the color information of the environment detection area stored in the determination result storage unit 771, or information stored in the read data storage unit 772. When the output destination is a screen or the like, it is preferable to output the result each time a reading operation is performed. In this case, it is also preferable to output the determination result stored in the determination result storage unit 771 for the environment detection area. When the output destination is a communication destination or the like, the output process may be performed each time a reading operation is performed, or may be performed by aggregating data from several readings or at predetermined intervals.

[0087] <Reading system processing flow> 5 is a diagram illustrating a process flow T800 of the reading system 700 according to this embodiment. The process flow T800 will be described below in conjunction with FIG.

[0088] First, based on a command from the input control unit 741, image data of a two-dimensional code including a reference color and an environment detection area is input from the image acquisition device 710. The input data is stored in the image data storage unit 761. The two-dimensional code position recognition unit 742 recognizes the reference position of the two-dimensional code from the image recorded in the image data storage unit 761, and the recognized reference position is recorded in the two-dimensional code position data storage unit 762. The two-dimensional code recognition unit 743 identifies a data area using the image recorded in the image data storage unit 761 and the position data recorded in the two-dimensional code position data storage unit 762, and decodes the code represented by the data area. The two-dimensional code recognition unit 743 identifies the type of two-dimensional code using the character string data obtained by decoding. (Step T801)

[0089] In step T802, based on the identified type of two-dimensional code, the position information of the reference color and environment detection area of ​​the two-dimensional code corresponding to the identified type of two-dimensional code is called from the storage device 760, thereby identifying the position information and color information of the reference color and environment detection area arranged in the two-dimensional code. The identification result is recorded in the two-dimensional code data storage unit 763.

[0090] Using the position data stored in the two-dimensional code position data storage unit 762 and the information stored in the two-dimensional code data storage unit 763, the coordinates (X, Y) of the four corners of the two-dimensional code are detected from the acquired image of the two-dimensional code. The reference color position determination unit 744 and the environment detection area position determination unit 745 use the coordinates of the detected four corners and the position information of the reference color and the environment detection area stored in advance in the reference color position data storage unit 764 and the environment detection area position data storage unit 765 to determine the positions of the colors to be used in the analysis for the reference color and the environment detection area, respectively (Step T802).

[0091] The reference color measurement color determiner 746 and the environment detection area measurement color determiner 747 determine color information for the reference color area and the environment detection area from the acquired image based on the position information determined by the reference color position determiner 744 and the environment detection area position determiner 745. Here, color information is, for example, RGB values. The determined color information is transferred to the reference color measurement color data memory 766 and the environment detection area measurement color data memory 767, which store the color information for the reference color area and the environment detection area, respectively. If strong lighting causes "overexposure" (where the reference color or the environment detection area is obscured by reflected light), if there is an "extreme illuminance difference" between multiple reference colors or between the reference color and the environment detection area, if the 2D code image is too small or too large, or if the shooting angle during image capture is too acute and the 2D code image is excessively distorted, return to step T801 and perform 2D code recognition and image reading again (step T803).

[0092] If the measured color data of the reference color and environment detection area can be acquired without any problems in step T803, color correction of the environment detection area (correction of the color information of the environment detection area) is then performed. Details are as explained above. Using the color information stored in the reference color base data storage unit 768, the reference color measured color data storage unit 766, and the environment detection area measured color data storage unit 767, the environment detection area corrected color determination unit 748 determines the corrected color of the environment detection area, and stores it in the environment detection area corrected color data storage unit 769 (step T804).

[0093] Thereafter, as explained above, the determination unit 749 compares the color information stored in the correction color data storage unit 769 for the environment detection area with the color information in the judgment color data storage unit 770 for the environment detection area to determine whether the amount of environmental change has exceeded a preset threshold value. That is, the data processing device 740 uses the color information of the corrected environment detection area to determine whether the surrounding environment of the two-dimensional code has exceeded a preset threshold value. The determination unit 749 may calculate the surrounding environment information of the two-dimensional code from the color information of the corrected environment detection area and the relationship between the amount of environmental change stored in the judgment color data storage unit 770 for the environment detection area and the color information of the environment detection area. (Step T805)

[0094] The result of the determination made by the determination unit 749 is stored in the determination result storage unit 771. At the same time, the date and time of the reading process, location information, the number of the reading device (to identify the worker), weather information linked to the web information, etc. are stored in the read data storage unit 772 (step T806).

[0095] Finally, the product information linked to the character string stored in the two-dimensional code data storage unit 763, information such as the presence or absence of temperature deviation determined from the color information of the environment detection area stored in the determination result storage unit 771, or information stored in the read data storage unit 772 are output to the output device 730, and if a smartphone is used as the hardware of the reading system, the output device 730 outputs the reading result to the smartphone screen or the like (Step T807).

[0096] FIG. 6 is a diagram showing an example of the display of the output result of the reading system 700 according to this embodiment. Referring to FIG. 6, an example of the output display of the reading result on the display 901 of the smartphone 900 is shown. When multiple label-marked objects are lined up on the display 901, it is possible to simultaneously acquire images of multiple items and display the reading results. Specifically, considering on-site workability and processing speed, it is efficient and preferable to recognize up to eight two-dimensional codes, process them in parallel using four threads in the order they were recognized, and then, once processing is complete, process the remaining two-dimensional codes in parallel. The number of parallel processing threads is not particularly limited, as the optimal number varies depending on the on-site work and the processing speed of the smartphone 900.

[0097] Furthermore, the result determined in step T805 is displayed as "■" 902a, which indicates NG when there is a temperature deviation in AR (Augmented Reality), or "■" 902b, which indicates OK when there is no temperature deviation in AR, making it easier for the worker to recognize the management status. After all the two-dimensional codes have been read, by touching the end processing button 903, all the acquired data output in step 802 can be viewed on the smartphone screen or sent to a specified address.

[0098] In the process flow T800 of this embodiment, it is preferable to execute a program on a general-purpose smartphone or the like that has a camera, a screen, and a communication device, but the present invention is not limited to this form.

[0099] <Information Processing System> Fig. 7 is a diagram illustrating an outline of an information processing system 1000 that utilizes a reading system 700 according to this embodiment. Fig. 7 shows a block diagram of the information processing system 1000 that includes an information processing device 1100. Although not shown, the information processing device 1100 is configured to include, for example, a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), I / F (Interface), and the like. The information processing device 1100 is realized by the CPU executing a predetermined control program stored in the ROM.

[0100] The information processing device 1100 is composed of an input unit 1110, a storage unit 1120, an information processing unit 1130, and an output unit 1140. The input unit 1110 is a unit that inputs information output from the smartphone 900. The storage unit 1120 is a unit that accumulates and stores the information output from the smartphone 900.

[0101] The information processing unit 1130 is a unit that organizes and analyzes the information accumulated in the storage unit 1120. For example, by statistically analyzing the information from the smartphone 900 including the reading system 700, it is possible to know when, what kind of worker, and in what place a temperature deviation occurs, which can lead to business improvements.

[0102] The output unit 1140 is a part that transmits the information processed by the information processing unit 1130 to a location that requires it. If the information processing device 1100 is a device that has a display, it can output the processing results to the display.

[0103] The following is an example in which the information processing system 1000 is applied to distribution from production to consumers. The information processing system 1000 enables consumers who receive a product to understand the physical quantity of a product, such as wine, during the distribution process. For example, if the product is wine, the two-dimensional code 200 is displayed on a label attached to the wine, a box containing the wine, or the like. While FIG. 7 is explained using the two-dimensional code 200 as an example, the other two-dimensional codes 201 to 209 can be used in a similar manner.

[0104] The information processing system 1000 includes a smartphone 900 having a reading system 700 for a two-dimensional code 200, and an information processing device 1100. The smartphone 900 (input / output device), typically a smartphone, is a terminal with an imaging function, such as an information communication terminal. The smartphone 900 (input / output device) is used to upload information acquired and analyzed by the reading system 700 for the two-dimensional code 200 to the information processing device 1100 during product distribution. While FIG. 7 shows an example of a smartphone 900 (input / output device) capable of both input and output on a single terminal, dedicated input and output devices can also be used.

[0105] When collecting a product, a person in charge of transporting the product (e.g., a driver) uses a smartphone 900 (input / output device) to capture an image of the two-dimensional code 200 displayed on the product. As a result, input information including an image of the two-dimensional code 200 is uploaded to the information processing device 1100. The two-dimensional code 200 and the reading system 700 determine whether or not there is a temperature deviation at the time of reading. At this time, in addition to product identification information (item identification information), the image capture time, image capture location, etc. are also uploaded as input information. Examples of item identification information include the GTIN (Global Trade Item Number), EAN (European Article Number) code, and UPC (Universal Product Code). It is preferable that this item identification information be linked to information about the item, such as the manufacturer, seller, production date, and expiration date.

[0106] Similarly, each person in charge at the export warehouse, customs warehouse, and import warehouse uses a smartphone 900 (input / output device) to capture an image of the two-dimensional code 200 displayed on the product. This causes each piece of input information to be uploaded to the information processing device 1100. Then, based on the uploaded input information, the information processing device 1100 determines whether or not there is a temperature deviation at the time of reading.

[0107] On the other hand, a consumer can access the URL (website) stored in the two-dimensional code 200 by reading the two-dimensional code 200 of the product using a smartphone 900 (input / output device). Reading can be performed, for example, using a dedicated application running on the smartphone 900 (input / output device).

[0108] It should be noted that the smartphone 900 on the manager's side, such as the person in charge of product transportation, includes the reading system 700 of this embodiment, but the smartphone 900 on the consumer's side may not include the reading system 700 of this embodiment. In Fig. 7, the smartphone 900 is shown as a smartphone or a tablet terminal, but is not limited to this, and the smartphone 900 may be a dedicated reading reader as an input device and a personal computer as an output device.

[0109] The two-dimensional code reading system 700 of this embodiment is a two-dimensional code reading system having an ink in its data area that changes color in response to environmental changes. The two-dimensional code has a positioning pattern colored with a reference color that is used to determine the amount of color change depending on the shooting environment. The system is equipped with an image acquisition device 710 that acquires an image of the two-dimensional code, a data processing device 740, and a storage device 760 that stores color information of the reference color and position information of the reference color and the ink. The data processing device 740 detects the coordinates (X, Y) of the four corners of the two-dimensional code from the image of the two-dimensional code, identifies the position of the area where color information is to be detected based on the detected coordinates of the four corners, and detects the color information of the area. The area includes a reference color area and an ink area. The reference color area and the ink area are determined (identified) from the position information of the reference color stored in the storage device 760 and the color information and position of the area, and the color information of the ink area can be corrected using the relationship between the color information of the reference color stored in the storage device 760 and the color information of the detected reference color area. Note that the ink region that changes color in response to environmental changes corresponds to the environment detection region 600. This makes it possible to improve the accuracy of obtaining color information about ink that changes color in response to environmental changes and the reading speed. [Explanation of symbols]

[0110] 100 Labeled items 101 Serial ID 200 2D code (1st 2D code) 201 2D code (2nd 2D code) 202 2D code (3rd 2D code) 203 2D code (4th 2D code) 204 2D code (5th 2D code) 205 2D code (6th 2D code) 206 2D code (7th 2D code) 207 2D code (8th 2D code) 208 2D code (9th 2D code) 209 2D code (10th 2D code) 300 Finder pattern (positioning pattern) 301 1st reference color 302 2nd reference color 303 3rd reference color 304 4th reference color 310 First finder pattern area 320 Second finder pattern area 330 Third finder pattern area 400 Alignment Pattern 410 First alignment pattern area 420 Second alignment pattern area 430 Third alignment pattern area 500 cells (data area) 600 environmental sensing areas 700 Reading System (2D code reading system) 710 Image Acquisition Device 720 Input Device 730 Output Device 740 Data Processing Device 741 Input control section 742 Two-dimensional code position recognition unit 743 Two-dimensional code recognition unit 744 Reference color position determination unit 745 Environmental sensing area positioning unit 746 Reference Color Measurement Unit 747 Environmental detection area actual color determination part 748 Correction color determination part of environment detection area 749 Judgment section 750 Output control section 760 storage device 761 Image data storage unit 762 Two-dimensional code position data storage unit 763 Two-dimensional code data storage unit 764 Reference color position data storage unit 765 Environment detection area position data storage unit 766 Reference color measurement data storage section 767 Environmental detection area actual color data storage section 768 Reference color base data storage section 769 Correction color data storage unit for environment detection area 770 Environmental detection area judgment color data storage unit 771 Judgment result storage unit 772 Read data storage unit 900 smartphones (mobile devices) 901 Display 902a AR display “NG” 902b AR display “OK” 903 Processing end button 1000 Information Processing Systems 1100 Information processing equipment 1110 Input section 1120 Storage section 1130 Information Processing Department 1140 Output section T800 reading system processing flow T801 Procedure "Code recognition and image reading" T802 Procedure "Getting Reference Colors and Ink Colors" T803 Procedure "Burst highlights, extreme illumination differences, image size, image angle" T804 Procedure "Color Correction" T805 Procedure "Color Analysis" T806 Procedure "Data Storage" T807 Procedure "Data Output / Send"

Claims

1. A two-dimensional code reading system having an environment detection area in a data area that changes color in response to environmental changes, The method includes an image capture device that captures an image of the two-dimensional code, a data processing device, and a storage device that stores color information of a reference color and position information of the reference color and an environment detection area, the two-dimensional code has a positioning pattern colored with a reference color used to determine the amount of color change depending on the shooting environment, The data processing device includes: Recognizing a reference position of the two-dimensional code from an image of the two-dimensional code; determining an area colored with the reference color and the environment detection area in an image of the two-dimensional code acquired from the detected reference position and the position information of the reference color and the environment detection area stored in the storage device, and detecting color information of the reference color and the environment detection area; The color information of the environment detection area is corrected using the relationship between the color information of the reference color stored in the storage device and the color information of the detected reference color area. A two-dimensional code reading system characterized by the above.

2. The two-dimensional code reading system according to claim 1, The data processing device identifies the type of the two-dimensional code based on the information stored in the data area of ​​the two-dimensional code, and retrieves the reference color of the two-dimensional code and the position information of the environment detection area corresponding to the identified type of two-dimensional code from the storage device. A two-dimensional code reading system characterized by the above.

3. The two-dimensional code reading system according to claim 1, the storage device stores a relationship between an amount of change in the environment and color information of the environment detection area; The data processing device calculates ambient environment information of the two-dimensional code from the corrected color information of the environment detection area and the relationship between the amount of environmental change and the color information of the environment detection area stored in the storage device. A two-dimensional code reading system characterized by the above.

4. The two-dimensional code reading system according to claim 1, The data processing device uses the corrected color information of the environment detection area to determine whether the surrounding environment of the two-dimensional code exceeds a preset threshold value. A two-dimensional code reading system characterized by the above.

5. The two-dimensional code reading system according to claim 1, the image acquisition device acquires an image including a plurality of the two-dimensional codes; The data processing device performs a process of recognizing a plurality of the two-dimensional codes in parallel. A two-dimensional code reading system characterized by the above.

6. The two-dimensional code reading system according to claim 1, The environment sensing area is formed of ink that changes color in response to changes in the environment. A two-dimensional code reading system characterized by the above.

7. The two-dimensional code reading system according to claim 1, the two-dimensional code is a QR code and has three of the positioning patterns; Each of the three positioning patterns is colored with a reference color that has a different direction of the vector connecting with white (255, 255, 255) in the RGB space. A two-dimensional code reading system characterized by the above.

8. A two-dimensional code having a data area made up of a plurality of cells and a positioning pattern, an environment detection area that changes color in response to environmental changes, the environment detection area being superimposed on the data area; the positioning pattern has a portion colored with a reference color used to calculate a color change amount according to a shooting environment, the two-dimensional code has three of the positioning patterns, Each of the three positioning patterns is colored with a reference color that has a different direction of the vector connecting with white (255, 255, 255) in the RGB space. A two-dimensional code characterized by:

9. The two-dimensional code according to claim 8, The environment sensing area is formed of ink that changes color in response to changes in the environment. A two-dimensional code characterized by:

10. The two-dimensional code according to claim 8, the reference colors include a first reference color and a second reference color that are different from each other; A two-dimensional code, wherein the positioning pattern has a portion marked with a first reference color and a portion marked with a second reference color.

Images