Printed matter reading system, printed matter reading method, and portable information device

Abstract

To provide a printed matter reading system, a printed matter reading method, and a portable information device that are capable of detecting color information with high accuracy even if positional misalignment occurs between an environment detection area and an information recording area.SOLUTION: A data processing unit includes: analysis area setting means for recognizing a reference area (200) to be a predetermined reference position, and setting, in the recognized reference area, an analysis area (400) including an environment detection area (300) where printing with environment detection ink is performed and a non-printing area (500) where printing with the environment ink is not performed; and analysis means (747, 748) for acquiring color information of the analysis area set by the analysis area means, comparing an actual relationship (ΔG) of the color information between the environment detection area (300) and the non-printing area (500) with a reference relationship (ΔGref) of color information to be a reference, and determining that printed matter has experienced an environmental change if the reference relationship and the actual relationship differ.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a reading system for reading a printed matter drawn on the surface of a print medium, and more particularly to a reading system for reading a printed matter whose color changes depending on an environmental change, a method for reading a printed matter, and a portable information device. [Background technology]

[0002] Recently, inks that store the history of environmental changes such as temperature have been developed, and these inks are being applied to the surface of printing media as prints that detect temperature. This ink is known as temperature-sensing ink, and has the property that its color does not change within a certain set controlled temperature, but changes color when the temperature exceeds or falls below the controlled temperature. Furthermore, it is irreversible, meaning that once the color changes, it does not return to its original state even if the temperature returns to the original controlled temperature, so evidence of deviation from the controlled temperature remains as a color change.

[0003] The following documents are known as documents related to a system for reading printed matter using ink whose color changes depending on the environment.

[0004] JP 2022-83761 A (Patent Document 1) describes a code that includes an environment detection area that changes color in response to changes in the external environment, and an information recording area that records item identification information that identifies an item.

[0005] Furthermore, JP 2018-517199 A (Patent Document 2) describes providing an environmental sensing area on a substrate that changes its color state in response to changes in environmental, physical, or biological conditions, and a permanent color information recording area.

[0006] In addition, JP 2019-175033 A (Patent Document 3) describes that the ink used to print the cells that make up some of the multiple code areas has the property of being more susceptible to color change than the ink used to print the cells that make up the remaining code areas.

[0007] The above-mentioned Patent Documents 1 to 3 are mainly intended for printed matter or labels in which the positions of the environment detection area and the information recording area are always fixed at fixed positions. Then, by estimating the position information of the environment detection area from the position information of the information recording area, changes in the environment detection area at the estimated position (for example, color change due to temperature) are detected. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2022-83761 A [Patent Document 2] Special Publication No. 2018-517199 [Patent Document 3] JP 2019-175033 A Summary of the Invention [Problem to be solved by the invention]

[0009] Incidentally, in Patent Documents 1 to 3, the inks used in the environment detection area, which changes color in response to changes in the external environment, and the information recording area, which records various information and does not change color, are inks with different properties, making it difficult to print them all at once. On the other hand, the techniques of Patent Documents 1 to 3 are effective for printed matter such as labels, in which the environment detection area and information recording area can be positioned in advance.

[0010] However, when direct marking a product that moves at high speed on a production line, such as with an industrial inkjet printer, it is difficult to accurately align the positions of the environment detection area and the information recording area. Therefore, even if one of the areas has been printed in advance, it is difficult to perform direct marking on the other area of ​​a product that moves at high speed.

[0011] For example, in the case of a product having a label affixed with an information recording area printed with various information such as a QR code, when an environmental detection area that changes color in response to changes in the external environment is formed by direct marking on the label of a product moving at high speed on a production line, it is difficult to accurately position the environmental detection area and the information recording area relative to each other.

[0012] Therefore, in order to perform positioning accurately, it is necessary to slow down the movement speed of the product on the production line or to intermittently stop the product moving along the production line, which requires modifications to the production line.

[0013] Ultimately, if the environment detection area, which changes color in response to changes in the external environment, and the information recording area, which records various information and does not change color, cannot be accurately positioned, the position of the environment detection area cannot be accurately estimated using the information recording area as a reference, and color information of the environment detection area cannot be accurately detected.

[0014] For this reason, there is a demand for a reading system that can accurately detect color information in the environment detection area even when a positional deviation occurs between the environment detection area and the information recording area.

[0015] An object of the present invention is to provide a print reading system, a print reading method, and a portable information device that can detect color information with high accuracy even if there is a misalignment between the environment detection area and the information recording area. [Means for solving the problem]

[0016] The present invention is a printed material reading system equipped with a data processing unit that reads printed material printed on the surface of a printing medium with environmental detection ink that changes color in response to environmental changes, and is characterized in that the data processing unit is equipped with an analysis area setting means that recognizes a reference area that serves as a predetermined reference position, and sets an analysis area within the recognized reference area that includes an environmental detection area printed with environmental detection ink and a non-printed area not printed with environmental ink, and an analysis means that acquires color information of the analysis area set by the analysis area means, compares the actual relationship of color information between the environmental detection area and the non-printed area with a standard relationship of color information between the reference environment detection area and the non-printed area, and determines that the printed material has experienced an environmental change if the standard relationship differs from the actual relationship.

[0017] The present invention is a printed material reading system equipped with a data processing unit that reads printed material printed on the surface of a printing medium with environmental detection ink that changes color in response to environmental changes, and is characterized in that the data processing unit executes the steps of: recognizing a reference area that serves as a predetermined reference position; setting an analysis area within the recognized reference area that includes an environmental detection area printed with environmental detection ink and a non-printed area not printed with environmental detection ink; acquiring color information of the set analysis area; comparing the actual relationship of color information between the environmental detection area and the non-printed area with a reference relationship of color information between the reference environment detection area and the non-printed area; and determining that the printed material has experienced an environmental change if the reference relationship and the actual relationship differ. Effect of the Invention

[0018] According to the present invention, a reading system can be provided that can detect color information of an environmental detection area with high accuracy, even if a printed material printed with environmental detection ink that changes color in response to environmental changes is misaligned from a reference area that is a predetermined reference position. [Brief description of the drawings]

[0019] [Figure 1]11 is an explanatory diagram illustrating a reading screen of a printed matter consisting of a plurality of dots printed with environment-sensing ink; [Diagram 2] 2 is an explanatory diagram illustrating a reading screen of a printed matter consisting of a plurality of dots printed with the environment-sensing ink and misaligned with respect to FIG. 1; FIG. [Diagram 3] FIG. 2 is an enlarged view of a first analysis region set on the reading screen of FIG. 1. [Figure 4] FIG. 11 is an explanatory diagram showing a grayscale image of an actual printed matter read before experiencing a temperature change. [Diagram 5] FIG. 5 is an enlarged view of the CLP portion of FIG. 4. [Figure 6] FIG. 11 is an explanatory diagram showing a grayscale image of an actual printed matter that has been read after experiencing a temperature change. [Figure 7] FIG. 7 is an enlarged view of the CLP portion of FIG. 6. [Figure 8] FIG. 6 is an explanatory diagram illustrating the analysis result (histogram) of the grayscaled color information shown in FIG. 5. [Figure 9] FIG. 8 is an explanatory diagram illustrating the analysis result (histogram) of the grayscaled color information shown in FIG. 7. [Figure 10] 11 is an explanatory diagram illustrating a reading screen of a printed matter consisting of a single dot printed with environment-sensing ink; [Figure 11] 11 is an enlarged view of a first analysis region set on the reading screen of FIG. 10. FIG. [Figure 12] FIG. 11 is an explanatory diagram illustrating a reading screen in which a printed material consisting of a single dot is placed inside a two-dimensional code. [Figure 13] FIG. 11 is an explanatory diagram illustrating a reading screen in which a printed material consisting of a single dot is placed outside a two-dimensional code. [Figure 14] 11 is an explanatory diagram illustrating a reading screen of a printed material consisting of characters printed with environment-sensing ink. FIG. [Figure 15] FIG. 15 is an enlarged view of a first analysis region set on the reading screen of FIG. 14. [Figure 16]1 is a configuration diagram showing a configuration of a print reading system according to an embodiment of the present invention; [Figure 17] FIG. 17 is a flowchart illustrating a control flow of a reading process in the reading system of FIG. 16. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiment, and various modifications and application examples within the technical concept of the present invention are also included within its scope.

[0021] <Explanation of how color is analyzed> First, the form of the printed matter in the embodiment of the present invention will be described. There are various possible forms of the printed matter read by the reading system, but in the following embodiment, the form of the printed matter formed by an industrial inkjet printer will be described as an example. In this case, the printed matter is a symbol, character, or other shape formed by a plurality of dots made of ink droplets. Note that the following describes an example in which temperature sensing ink is used as the environment sensing ink. 1 shows a reading screen 100 in which a label (corresponding to the surface of a printing medium) attached to a product package or the like is printed with a plurality of dots using temperature sensing ink. In this example, the environment detection area and the information recording area are formed separately on the label at a distance from each other, and the information recording area is not shown. In addition, the position information (coordinate information) of the information recording area and the position information of the reference area and the first analysis area described later are managed by a computer.

[0022] A rectangular area (hereinafter referred to as a reference area 200) is set on the label as a space for printing the temperature sensing ink, which is a predetermined reference position. This reference area 200 is linked to, for example, position information of the information recording area, and the position of the reference area 200 is determined from the position information of the information recording area. It is also possible to set a reference position separate from the information recording area.

[0023] Therefore, the inkjet printer prints a plurality of dots of a printed matter made of temperature sensitive ink, whose color changes in response to environmental changes, for example, temperature, within the frame of the reference area 200. In this embodiment, the dots become the environment detection area 300. Note that hereinafter, the printed matter may also be referred to as the environment detection area 300.

[0024] The shape of the printed matter can be any shape, such as letters, symbols, or other shapes, as long as it is possible to detect changes in color information due to temperature changes. When reading multiple dots made of temperature-sensitive ink printed on a label, the following method is used.

[0025] On the reading screen 100, a first analysis area 400 for acquiring color information of the environment detection area 300 is set within the frame of the reference area 200. The first analysis area 400 is a rectangular area formed in approximately the center of the reference area 200, and image information within this first analysis area 400 is to be read.

[0026] In the drawings, the reference area 200 and the first analysis area 400 are shown as rectangular frames for convenience of explanation, but in reality they are not shown in this way, and the reference area 200 and the first analysis area 400 are set by a computer reading and processing the image, as described below. Therefore, the position coordinates of the reference area 200 and the first analysis area 400 are set by the reading and processing of the image, and the image reading and analysis are performed based on these.

[0027] Even if the environment detection area 300 is located anywhere within the frame of the reference area 200, in other words, even if the environment detection area 300 is misaligned, the first analysis area 400 is set to include at least the environment detection area 300 and a non-printed area 500 that serves as a background for product packaging, etc. The non-printed area 500 is an area where no printed matter is formed by dots.

[0028] Here, the method of recognizing the reference area 200 is not particularly specified, but can be performed by a method such as pattern matching with a reference area registered in advance in the data processing unit. Also, the method of recognizing the first analysis area 400 is not particularly specified, but can be performed by a method of recognition based on position information of a reference area registered in advance in the data processing unit.

[0029] Fig. 2 shows a label reading screen 101 in which the environment detection area 300 is located in a different place from that in Fig. 1 due to misalignment. Even in this case, the first analysis area 400 includes at least the environment detection area 300 and the non-printed area 500 that serves as a background for the product package or the like.

[0030] In this way, by setting the reference area 200 and the first analysis area 400 and positioning the environment detection area 300 within the frame of the reference area 200, even if the environment detection area 300 is slightly misaligned, the image information (color information) of the environment detection area 300 can be read by the first analysis area 400.

[0031] In this embodiment, when detecting a change in color information as described later, the first analysis area 400 needs to be set so as to detect image information of the environment detection area 300 and the non-print area 500.

[0032] Fig. 3 is an enlarged view of the first analysis area 400 of the reading screen 100 in Fig. 1, showing the concept of acquiring color information. The first analysis area 400 is an image reading area, and is composed of a second analysis area 410 having an area smaller than the minimum area of ​​the environment detection area 300. The second analysis area 410 is a color information analysis area.

[0033] That is, in this embodiment, the first analysis area 400 is divided into a lattice pattern to set a plurality of analysis areas (hereinafter, referred to as second analysis areas) 410. The second analysis area 410 is rectangular, and its area is set to be smaller than the area of ​​the environment detection area 300 (here, dots). Therefore, the environment setting area 300 includes a plurality of second analysis areas 410.

[0034] The color analysis patterns in this second analysis area 410 are divided into three types: first, a second analysis area 411 that includes only the non-printed area 500; second, a second analysis area 412 that includes only the environmental detection area 300; and third, a second analysis area 413 that includes both the non-printed area 500 and the environmental detection area 300.

[0035] If the multiple second analysis regions 410 include at least the second analysis region 411 consisting only of the non-printed region 500 and the second analysis region 412 consisting only of the environment detection region 300, it is possible to acquire color information of the environment detection region 300, as will be described later. Note that the second analysis region 413 including both the non-printed region 500 and the environment detection region 300 can also be regarded as the second analysis region 411 consisting only of the non-printed region 500 or the second analysis region 412 consisting only of the environment detection region 300, and image processing can be performed on it.

[0036] Next, a method for reading the color of the environment detection area 300 will be described using an actual read image. This embodiment is characterized in that the read image is converted into a grayscale image and a change in color information is detected from a change in density distribution.

[0037] 4 shows a scanned image 600 of the first analysis area 400 of the printed matter before the environment detection area 300 changes color, and this scanned image 600 is a grayscale image. The dark dots are the environment detection area 300, and the light-colored areas in the background are the non-printed areas 500 that serve as the base.

[0038] Fig. 5 shows the environment detection area 300 and its surroundings, which is an enlarged view of the dot portion CLP in Fig. 4. In this embodiment, one pixel in the image is the second analysis area 410, and there are also a second analysis area 411 which is only the non-printed area 500, a second analysis area 412 which is only the environment detection area 300, and a second analysis area 413 which includes both the non-printed area 500 and the environment detection area 300.

[0039] Here, in FIG. 3, the second analysis area 413, which includes both the non-printed area 500 and the environment detection area 300, is shown with the non-printed area 500 and the environment detection area 300 clearly separated, but in the actual image they are displayed as averaged information.

[0040] 5, it can be seen that the plurality of grid-shaped second analysis regions 410 at least include a second analysis region 411 consisting only of the non-printed region 500 and a second analysis region 412 consisting only of the environment detection region 300.

[0041] 6 is a scanned image 601 of the print after the environment detection area 300 in FIG. 4 has changed color, and this scanned image is a grayscale image. Furthermore, FIG. 7 shows the environment detection area 300 obtained by cutting out and enlarging the dot portion of the environment detection area 300 in FIG. 6. This example also shows the environment detection area 300 obtained by cutting out and enlarging the dot portion CLP in FIG. 6, as in FIG. 5, and its surroundings. In this embodiment, one pixel in the image is the second analysis area 410, and there are a second analysis area 411 including only the non-printed area 500, a second analysis area 412 including only the environment detection area 300, and a second analysis area 413 including both the non-printed area 500 and the environment detection area 300.

[0042] 7, it can be seen that the plurality of grid-shaped second analysis regions 410 at least include a second analysis region 411 consisting only of the non-print region 500 and a second analysis region 412 consisting only of the environment detection region 300.

[0043] Figure 8 shows the analysis results of the color information before the color change in Figure 4. As an example, it shows a histogram of the values ​​(G values) obtained by converting a color image consisting of RGB (Red, Green, Blue) into a grayscale image using the following formula (1). Here, the values ​​(G values) obtained by converting into a grayscale image are treated as luminance values. G=Red×0.30+Green×0.59+Blue×0.11……Formula (1) 8, the G value is "255" for white and "0" for black, and the darker the color, the closer the G value is to "0." By extracting the G values ​​of two frequently occurring points from the histogram in FIG. 8, it is possible to extract peak 411pb originating from non-printed area 500 and peak 412pb originating from environment detection area 300.

[0044] 5 can be separated from a peak 411pb originating from the second analysis region 411 which is only the non-printed region 500, and a peak 412pb originating from the second analysis region 412 which is only the environment detection region 300. The peak 412pb originating from the second analysis region 412 in the environment detection region 300 appears toward the white side because the color of the environment detection region 300 is light gray.

[0045] Here, the second analysis region 413 including both the non-printed region 500 and the environment detection region 300 is included in a region with large numerical values ​​of the histogram including the peak 412pb derived from the second analysis region 412 of the environment detection region 300 only.

[0046] On the other hand, Fig. 9 shows the analysis result of the color information after the color change in Fig. 6. By extracting the G values ​​of two frequently occurring points from the histogram in Fig. 9, it is possible to extract a peak 411pa originating from the non-printed region 500 and a peak 412pa originating from the environment detection region 300. As a result, it is possible to separate the peak 411pa originating from the second analysis region 411 only in the non-printed region 500 in Fig. 7 from the peak 412pa originating from the second analysis region 412 only in the environment detection region 300. The peak 412pa originating from the second analysis region 412 in the environment detection region 300 appears toward the black side because the color of the environment detection region 300 is dark gray.

[0047] 8 and 9, the peaks 411pb and 411pa in the second analysis region 411, which includes only the non-printed region 500, acquire color information from the same background, and have almost the same G value. On the other hand, the peaks 412pa and 412pa in the second analysis region, which includes only the environment detection region 300, change color due to temperature, and therefore have significantly different G values.

[0048] Therefore, by calculating the difference between the G value of peaks 411pb, 411pa originating from the second analysis region 411 containing only the non-printed region 500 and the G value of peaks 412pb, 412pa originating from the second analysis region 412 containing only the environment detection region 300, it is possible to determine whether or not the color has changed.

[0049] In this way, by determining the "change in the relationship" between the G value and the base before and after the color change of the temperature sensing ink, it is possible to determine the color change of the temperature sensing ink.

[0050] 8, the difference ΔG between the G value of the peaks 411pb, 411pa originating from the second analysis region 411 consisting only of the non-printed region 500, which is the background, and the G value of the peaks 412pb, 412pa originating from the second analysis region 412 consisting only of the environment detection region 300, is more than twice as large. Therefore, by detecting the difference ΔG, which is the change in the relationship between the non-printed region 500 and the environment detection region 300, it is possible to detect a change in the color of the environment detection region 300.

[0051] Therefore, for example, by determining in advance the difference threshold value ΔGref when the product has not experienced any temperature change, such as when it is shipped or stored in a warehouse, and comparing this with the difference ΔG when the product experiences a temperature change and discolors when it is distributed to a retail store or convenience store, it becomes possible to reliably determine whether the product has experienced a temperature change.

[0052] Here, the environment detection area 300 may be formed with ink or the like that changes color in response to environmental changes, and is not limited to the above-mentioned temperature detection ink. From the viewpoint of the manufacturing process, it is preferable that the ink be an ink that can be printed on the upper surface of the base material, which is the non-printed area 500.

[0053] Furthermore, the printing ink forming the environment detection area 300 should have the ability to detect environmental conditions other than temperature, such as temperature history, humidity, light, gas concentration, and vibration, as well as the pH of the liquid, the concentrations of various ions in the liquid, the concentrations of various pharmaceutical drugs, the protein concentrations of various amino acids, the presence of viruses and bacteria, and the like, and the detection results should be reflected as a change in color information.

[0054] <Explanation of printed matter> The printed matter described above is a printed matter consisting of a plurality of dots formed by an industrial inkjet printer, but the present invention can also be applied to printed matter of different shapes as described below.

[0055] Figure 10 shows a reading screen 102 for a single-dot printout having a larger area than the dots depicted in Figure 1, drawn with temperature sensing ink, and unlike Figure 1, an environment sensing area 300, which is a single dot, is arranged within the frame of a reference area 200. In this case as well, a non-printed area 500 exists.

[0056] Fig. 11 is an enlarged view of the first analysis area 400 of the reading screen 102 in Fig. 10, showing the concept of acquiring color information. In this case, the first analysis area 400 is also formed in a lattice pattern, and is made up of a second analysis area 411 that includes only the non-printed area 500, a second analysis area 412 that includes only the environment detection area, and a second analysis area 413 that includes both the non-printed area 500 and the environment detection area 300.

[0057] Since the second analysis region 411 includes only the non-printed region 500, and the second analysis region 412 includes only the environment detection region 300, it is possible to acquire color information of the environment detection region 300. The histograms of the peaks of the second analysis region for the color information before and after discoloration are obtained in the same manner as in Figures 8 and 9. In this way, the present invention can be applied to a simple single printed material.

[0058] The positions and sizes of the reference area 200 and the first analysis area 400 can be changed as desired depending on the range of misalignment of the printed matter, and the reference area 200 can also be used as the first analysis area 400. The larger the frame length of the first analysis area 400, the more analysis can be performed even if the printed matter is significantly misaligned. However, this increases the amount of image information required for analyzing and processing the color information, which increases the analysis processing time, so it is preferable to set an appropriate frame length taking into account the misalignment and analysis processing time.

[0059] Next, an example will be described in which the print material shown in FIG. 10 is applied to a two-dimensional code. In this two-dimensional code, a QR code (registered trademark) may be used for the reference area 200. The QR code is standardized by ISO / IEC18004 and the like. In addition, PDF417, DataMatrix, Maxicode, AztecCode, and the like can also be used as the two-dimensional code. By using a two-dimensional code such as a QR code, it is possible to record item identification information for identifying an item, and to link it to environmental change information obtained from the environment detection area 300.

[0060] FIG. 12 shows a QR code reading screen 104 having a single dot printed with temperature sensitive ink, and unlike FIG. 1, the QR code is used as the reference area 200.

[0061] In this example, the environment detection area 300 and the QR code (information recording area) are formed so as to overlap each other. In addition, the position information (coordinate information) of the QR code (information recording area) and the position information of the first analysis area 400 are managed by a computer.

[0062] A predetermined reference area 200 is set in the QR code as a space for printing temperature sensing ink. This reference area 200 is a QR code, and the position information of the QR code is set as the position of the reference area 200. The QR code has the functions of both an information recording area and the reference area 200.

[0063] Therefore, an inkjet printer prints a single dot of temperature sensitive ink, whose color changes in response to environmental changes, for example temperature, within the frame of a QR code, which is the reference area 200. When a QR code is used for the reference area 200, it is possible to detect the position information of the first analysis area 400 from the position information of the QR code.

[0064] 12 is a QR code of version 2 (25×25 cells). The cell configuration (version) of the QR code can be changed depending on the content of the information to be input, but version 2 or higher is preferable because an alignment pattern for distortion correction is arranged.

[0065] The QR code in the reference area 200 in Fig. 12 is a QR code with an error correction level of "H." The error correction level of the QR code can be lowered depending on the amount of information and the purpose, but it is desirable to set the error correction level to "H" in order to prevent reading errors when a non-printed area 500 is formed within the QR code and the environment detection area 300 is further formed so as to overlap the non-printed area 500, and when the QR code is lost due to positional deviation in the environment detection area 300, or when stains, dirt, or damage occurs during logistics.

[0066] In Fig. 12, the reference area 200 in Fig. 10 is changed to a QR code, and after acquiring the position information of the first analysis area 400 from the QR code of the reference area 200, the method of acquiring color information of the environment detection area 300 from the first analysis area 400 is the same as in the example shown in Fig. 10. Therefore, even if the reference area 200 is changed to a QR code, the above-mentioned actions and effects can be achieved.

[0067] FIG. 13 is a modified example of FIG. 12, and shows an example in which the first analysis area 400 and the environment detection area 300 are arranged outside the QR code that is the reference area 200. In FIG.

[0068] As shown in Fig. 13, even if the first analysis area 400 is located outside the QR code that is the reference area 200, the position information of the first analysis area 400 can be recognized from the QR code, and if the reference area 200 and the environment detection area 300 are contained within the read image, the color information of the environment detection area 300 can be obtained in the same manner as in Fig. 12. Therefore, even if the first analysis area 400 is located outside the QR code of the reference area 200, the above-mentioned actions and effects can be achieved.

[0069] The example in Fig. 14 shown next is a reading screen 103 for a printout of characters (here, numbers) drawn with temperature sensitive ink, and unlike Fig. 1, multiple numbers are arranged as an environment detection area 300 within the frame of a reference area 200. In this example, the numbers themselves also correspond to various information and have the function of an information recording area.

[0070] Fig. 15 is an enlarged view of the first analysis area 400 of the reading screen 103 in Fig. 14, showing the concept of acquiring color information. In this case, the first analysis area 400 is also formed in a lattice shape, and is made up of a second analysis area 411 that includes only the non-printed area 500, a second analysis area 412 that includes only the environment detection area, and a second analysis area 413 that includes both the non-printed area 500 and the environment detection area 300.

[0071] In this example as well, since the second analysis region 411 includes only the non-printed region 500, and the second analysis region 412 includes only the environment detection region 300, it is possible to acquire color information of the environment detection region 300. Note that the histograms of the peaks of the second analysis region for the color information before and after discoloration are obtained in the same manner as in Figures 8 and 9. In this way, the present invention can be applied not only to simple figures, but also to complex shapes such as characters.

[0072] <Reading system description> Next, a print reading system according to an embodiment of the present invention will be described. Note that this reading system is intended for use with portable information devices such as tablet terminals and smartphones.

[0073] This portable information device is also configured to communicate with an externally located information processing device using wireless communication and transmit read information. The external information processing device is, for example, a management server located in a management center. The management center can then manage the temperature history of the target products. For example, it becomes possible to trace the dates and locations where deviations from the temperature control range were found in the entire product.

[0074] The reading system described below is characterized in that the data processing recognizes a reference area that serves as a predetermined reference position, and within the recognized reference area, an analysis area setting means sets an analysis area that includes an environmental detection area where environmental detection ink is printed and a non-printed area where environmental ink is not printed, and an analysis means acquires color information of the analysis area set by the analysis area means, compares the actual relationship of the color information between the environmental detection area and the non-printed area with a standard relationship of the reference color information, and if the standard relationship and the actual relationship differ, determines that the printed matter has experienced an environmental change.

[0075] 16 shows the configuration of a reading system 700 for reading a printed matter on a label. The reading system 700 includes an image capture device 710 for capturing an image of a reference area, an input device 720, an output device 730, a data processing device 740, and a storage device 760.

[0076] The image capture device 710 is an imaging device such as a camera, and captures an image of the label. If necessary, it is also possible to capture images of memorized products, information related to the products, the surrounding environment, etc., in addition to the label. When an image of the reference area 200 is captured, it is stored in the image data storage unit 761 of the storage device 760.

[0077] The input device 720 is a section that accepts instructions from an operator, and is composed of buttons, a touch panel, and the like.

[0078] The output device 730 is a device that outputs instruction information, read images, read results, etc. to an operator, and is configured with a display and a communication device. This configuration is standard, and any or all of the image acquisition device 710, the input device 720, and the output device 730 may be configured to be connected to the outside of the reading system 700.

[0079] <Storage device description> The storage device 760 is a section for storing various data, and is composed of the following storage units:

[0080] The image data storage unit 761 has a function of storing the image of the reference area 200 input from the image acquisition device 710.

[0081] The reference area shape data storage unit 762 has a function of storing shape data representing the reference area 200 recognized by a reference area shape data recognition unit 742 (to be described later) from the image stored in the image data storage unit 761 .

[0082] The reference area storage unit 763 has a function of storing the shape pattern of the reference area 200 recognized by the reference area recognition unit 743 (to be described later) from the image stored in the image data storage unit 761 .

[0083] The first analytical region position data storage unit 764 has a function of storing the position information of the first analytical region 400 in advance.

[0084] The first analysis region actual measured color data memory unit 765 has the function of storing all actual measured color information (RGB values) of the second analysis region 410 extracted from the first analysis region 400 identified by the position data memory unit 764 of the first analysis region 400.

[0085] The G value data storage unit 766 of the first analysis region has a function of storing G values ​​obtained by converting the actual measured color information (RGB values) into grayscale by the color information conversion unit 745 described later. In this case, all color information of the first analysis region is converted into grayscale.

[0086] The environment detection area G value data storage section 767 has a function of storing the G values ​​412pb, 412pa derived from the environment detection area 300, extracted by a color information histogram generating section 746, which will be described later.

[0087] The non-print area G value data storage section 768 has a function of storing the G values ​​411pb, 411pa derived from the non-print area 500 extracted by the color information histogram generating section 746, which will be described later.

[0088] The color data memory unit 769 of the environment detection area has the function of storing the value of the difference ΔG in G values ​​calculated by the environment detection area discoloration calculation unit 747 described below based on the G values ​​412pb, 412pa of the environment detection area recorded in the G value data memory unit 767 of the environment detection area and the G values ​​411pb, 411pa of the non-printed area recorded in the G value data memory unit 768 of the non-printed area.

[0089] The environment detection area judgment color data storage unit 770 has a function of storing, for example, a value of a difference threshold value ΔGref of color information that serves as a judgment criterion for judging whether or not a temperature change has been experienced.

[0090] The judgment result storage unit 771 has a function of storing a symbol representing the result, such as "NG" / "OK" or "X" / "Check", judged by the judgment unit 749 described later based on the magnitude of the difference ΔG value stored in the environment detection area color data storage unit 769 relative to the difference threshold value stored in the environment detection area judgment color data storage unit 770. The result may be a symbol or data.

[0091] <Description of Data Processing Device> The data processing device 740 processes data input from the image acquisition device 710 and the input device 720, and data stored in the memory device 760, and outputs the results to the output device 730 or stores them in the memory device 760, and is composed of the following processing units.

[0092] The input control unit 741 has a function of classifying data input from the image acquisition device 710 or the input device 720 into commands, data, etc., and transferring the data to the storage device 760 and each unit of the data processing device 740. In particular, the main data is image data of the reference area 200, the first analysis area 400, and the environment detection area 300, and the input control unit 741 transfers these image data to the image data storage unit 761.

[0093] The reference area shape data recognition unit 742 has a function of recognizing the shape of the reference area 200 from the image data stored in the image data storage unit 761 and storing the recognition result in the reference area shape data storage unit 762 .

[0094] Image data is usually made up of hundreds to thousands of dots in both the vertical and horizontal directions, and the image itself does not contain any data indicating which part is the reference area 200. Therefore, the color data of each dot is analyzed to determine which part of the image data corresponds to the reference position of the reference area 200.

[0095] The reference area recognition unit 743 has the function of using the shape data stored in the reference area shape data memory unit 762 to recognize the shape pattern of the reference area 200 from the image stored in the image data memory unit 761, setting specific coordinates, and storing them in the reference area memory unit 763.

[0096] The position determination unit 744 of the first analysis area has the function of determining the position of the first analysis area 400 based on the shape data stored in the reference area shape data memory unit 762 and the coordinate information of the reference area 200 stored in the reference area memory unit 763, based on the position information previously stored in the position data memory unit 765 of the first analysis area 400.

[0097] The color information conversion unit 745 has the function of converting all measured color information (RGB values) of the second analysis area, which is extracted from the identified first environment area 400 and recorded in the measured color data memory unit 765 of the first analysis area, into grayscale and calculating the G value.

[0098] The color information histogram generating section 746 has a function of generating a histogram of the G values ​​calculated by the color information converting section 745 .

[0099] The discoloration calculation unit 747 of the environment detection area has a function of calculating the difference ΔG between the G values ​​412pb, 412pa of the environment detection area recorded in the G value data memory unit 767 of the environment detection area and the G values ​​411pb, 411pa of the non-printed area recorded in the G value data memory unit 768 of the non-printed area.

[0100] The judgment unit 748 compares the value of the difference ΔG stored in the color data memory unit 769 of the environment detection area with the value of the difference threshold value ΔGref stored in the judgment color data memory unit 770 of the environment detection area, and judges from the comparison result whether or not a temperature change has been experienced.

[0101] For example, by determining in advance the difference threshold value ΔGref when the product has not experienced any temperature changes, such as when it is shipped or stored in a warehouse, and comparing this with the difference ΔG when the product experiences a temperature change and discolors when it is distributed to a retail store or convenience store, it is possible to reliably determine whether the product has experienced a temperature change.

[0102] The output control unit 749 controls the output of information processed by the data processing device 740 to the storage device 760. Specifically, it has a function of outputting information stored in the determination result storage unit 771 to the output device 730.

[0103] When the output destination is a screen or the like, it is preferable that the read result is output each time a read operation is performed. When the output destination is located in a remote location, the output process may be performed each time a read operation is performed, or the data read multiple times may be compiled and transmitted at predetermined intervals.

[0104] <Control flow explanation> Next, the control flow of the above-mentioned processing will be briefly explained with reference to FIG. 17 and FIGS.

[0105] The following control flow is characterized by executing the steps of recognizing a reference area that serves as a predetermined reference position, setting an analysis area within the recognized reference area that includes an environmental detection area where environmental detection ink is printed and a non-printed area where environmental ink is not printed, acquiring color information of the set analysis area, comparing the actual relationship of the color information between the environmental detection area and the non-printed area with the standard relationship of the reference color information, and determining that the printed matter has experienced an environmental change if the standard relationship and the actual relationship differ.

[0106] The following process is based on the assumption that temperature sensing ink has been printed on the label using an inkjet printer.

[0107] <Step S10> In step S10, as shown in Fig. 1, a reference area 200 in which a printed matter made of temperature sensing ink printed by an inkjet printer is present is recognized. The position of this reference area 200 can be obtained based on the position information of the information recording area as described above. At the same time, the shape (rectangular area) of the reference area 200 is also obtained, and the image area to be read on the label is determined. Once the reference area 200 has been recognized, the process proceeds to step S11.

[0108] <Step S11> In step S11, the entire image in the reference area 200 is read and stored in a predetermined RAM area. Note that the image is a color image, and color information is assigned to each pixel. After the image in the reference area 200 is read, the process proceeds to step S12.

[0109] <Step S12> In step S12, as shown in Fig. 3, an image corresponding to the first analysis region 400 is read from the RAM area and converted to grayscale as shown in Figs. 4 to 7. By converting to grayscale, a histogram of color information can be obtained. Once the image of the first analysis region 400 has been converted to grayscale, the process proceeds to step S13.

[0110] <Step S13> In step S13, from the image grayscaled in step S12, peaks originating from the non-printed area 500 and peaks originating from the environment detection area 300 are obtained. As shown in Figures 8 and 9, the peaks originating from the non-printed area 500 have G values ​​that lean towards the white side, and the peaks originating from the environment detection area 300 have G values ​​that lean towards the black side.

[0111] Here, the color of the environment detection area 300 changes before and after the temperature change, so the G value of the peak originating from the environment detection area 300 differs. After the peak originating from the non-printed area 300 and the peak originating from the environment detection area 300 are obtained, the process proceeds to step S14.

[0112] <Step S14> In step S, the actual difference in G value between the peak originating from the non-printed area 300 after discoloration and the peak originating from the environment detection area 300, which was obtained in step S13, is compared with a predetermined difference threshold to determine whether the actual difference is greater than the difference threshold. If it is determined that the actual difference is greater, it is determined that a temperature change has been experienced, and if it is determined that the actual difference is smaller, it is determined that a temperature change has not been experienced. After these determinations have been made, the process proceeds to step S15. The difference threshold may also be the difference value before the temperature change was experienced.

[0113] <Step S15> and <Step S16> In step S15, the result of the determination in step S14 is stored in a predetermined storage area, and the process proceeds to step S16 to output the result of the determination, or to transmit the read information to an external information processing device such as a management server in a management center by the wireless communication device 780. When this is completed, the process exits to the end and waits for the next start timing.

[0114] The transmitted read information is managed by a management server, and for example, a management center can manage the temperature history of the target product. In addition, the date and location when it was discovered that the temperature was out of the controlled range can be managed, making it possible to trace the overall temperature management of the product.

[0115] As described above, the present invention is a print reading system equipped with a data processing unit that reads a print printed on a surface of a print medium with environment sensing ink that changes color in response to environmental changes, the data processing unit is characterized by comprising: an analysis area setting means that recognizes a reference area that serves as a predetermined reference position, and sets an analysis area within the recognized reference area that includes an environment detection area printed with the environment sensing ink and a non-printed area not printed with the environmental ink, and an analysis means that acquires color information of the analysis area set by the analysis area means, compares an actual relationship of color information between the environment detection area and the non-printed area with a reference relationship of color information between the reference environment detection area and the non-printed area, and determines that the print has experienced an environmental change if the reference relationship and the actual relationship differ.

[0116] This makes it possible to provide a reading system that can detect color information of the environmental detection area with high accuracy, even if a printed material printed with environmental detection ink, which changes color in response to changes in the environment, is misaligned from a reference area that serves as a predetermined reference position.

[0117] The present invention is not limited to the above-described embodiments, but includes various modified examples. The above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace other configurations with respect to the configuration of each embodiment. [Explanation of symbols]

[0118] 100...reading screen, 200...reference area, 300...environment detection area, 400...first analysis area, 410...second analysis area, 411...second analysis area consisting of only non-printed area, 412...second analysis area consisting of only environment detection area, 413...second analysis area including both non-printed area and environment detection area, 500...non-printed area, 600...environment detection area, 700...reading system, 710...image acquisition device, 720...input device, 730...output device, 740...data processing device, 741...input control unit, 742...reference area shape data recognition unit, 743...reference area recognition unit, 744...first analysis area position determination unit, 745...color information Conversion unit, 746...color information histogram generation unit, 747...color discoloration calculation unit for environment detection area, 748...judgment unit, 749...output control unit, 760...storage device, 761...image data storage unit, 762...reference area shape data storage unit, 763...reference area storage unit, 764...position data storage unit for first analysis area, 765...actual measured color data storage unit for first analysis area, 766...G value data storage unit for first analysis area, 767...G value data storage unit for environment detection area, 768...G value data storage unit for non-printed area, 769...color data storage unit for environment detection area, 770...judged color data storage unit for environment detection area, 771...judgment result storage unit.

Claims

1. A print reading system including a data processing unit for reading a print that is printed on a surface of a print medium using an environment-sensing ink that changes color in response to an environmental change, The data processing unit includes: an analysis area setting means for recognizing a reference area that is a predetermined reference position, and setting an analysis area including an environment detection area in which the environment detection ink is printed and a non-printed area in which the environment detection ink is not printed within the recognized reference area; an analysis means for acquiring color information of the analysis area set by the analysis area setting means, comparing an actual relationship of the color information between the environment detection area and the non-printed area with a reference relationship of the color information between the environment detection area and the non-printed area serving as a reference, and determining that the printed matter has experienced an environmental change if the reference relationship and the actual relationship differ; A printed material reading system comprising:

2. The print reading system according to claim 1, The analysis means reads the color information in a color information analysis area having an area smaller than the minimum area of ​​the environment detection area. A printed matter reading system comprising:

3. The print reading system according to claim 2, The analyzing means converts the actually measured color information of the color information analysis area into a gray scale. A printed matter reading system comprising:

4. The print reading system according to claim 3, The analysis means generates a histogram of the grayscaled image. A printed matter reading system comprising:

5. The print reading system according to claim 4, The analysis means extracts luminance values ​​of grayscale images of the environment detection area and the non-printed area in the histogram. A printed matter reading system comprising:

6. The print reading system according to claim 5, The analysis means obtains a difference in luminance value between the grayscale images of the environment detection area and the non-print area. A printed matter reading system comprising:

7. The print reading system according to claim 6, The analysis means compares the difference with a preset difference threshold and determines whether the difference exceeds the difference threshold. A printed matter reading system comprising:

8. A system for reading a printed matter according to any one of claims 1 to 7, The environment-sensing ink is a temperature-sensing ink, and the color information of the temperature-sensing ink changes when it experiences a temperature change, and the changed color information is irreversible. A printed matter reading system comprising:

9. A system for reading a printed matter according to any one of claims 1 to 7, The reference area is a two-dimensional code. A printed matter reading system comprising:

10. A portable information device including an imaging device and a data processing unit that processes an image captured by the imaging device, The data processing unit is a data processing unit according to any one of claims 1 to 9. A portable information device comprising:

11. The portable information device according to claim 10, The portable information device includes a wireless communication device, and is wirelessly connected to an external information processing device to transmit read information. A portable information device comprising:

12. A reading method for a print reading system having a data processing unit that reads a print printed with temperature sensitive ink that changes color in response to a change in temperature on the surface of a print medium, comprising: The data processing unit includes: A first step of recognizing a reference area serving as a predetermined reference position where a printout using the temperature sensing ink is printed; a second step of setting an analysis area within the recognized reference area, the analysis area including an environment detection area in which the temperature sensing ink is printed and a non-printed area in which the temperature sensing ink is not printed; A third step of acquiring color information of the set analysis area; a fourth step of comparing an actual relationship of the color information between the environment sensing area and the non-printing area with a reference relationship of the reference color information; If the reference relationship and the actual relationship are different, the fifth step is to determine that the print has experienced an environmental change. A method for reading a printed matter, comprising:

13. A method for reading a printed matter according to claim 12, comprising the steps of: The third step is A step of reading the color information in a color information analysis area having an area less than the minimum area of ​​the environment detection area, and a step of converting the actually measured color information of the color information analysis area into a grayscale image and generating a histogram of the grayscale image, The fourth step is a step of extracting luminance values ​​of the grayscaled images of the environment detection area and the non-printed area in the histogram; a step of calculating a difference between the luminance values ​​of the grayscaled images of the environment detection area and the non-printed area, and comparing the difference with a preset difference threshold; The fifth step is determining whether the difference exceeds the difference threshold, and determining that the temperature sensing ink has experienced a temperature change if the difference exceeds the difference threshold. A method for reading printed matter comprising the steps of:

Images