Information processing device and information processing program

Abstract

To provide an information processing device and an information processing program that can present information for confirming the authenticity of an object to be inspected even when there are abnormal values at some locations.SOLUTION: In an information processing system, an information processing device includes a processor. The processor collates images of a plurality of locations on a surface of an object to be inspected, which is an inspection target, with images of a plurality of locations corresponding to the plurality of locations on the surface of the object to be inspected, on a surface of at least one reference object, which serves as reference (S112), and presents a successfully collated number among the plurality of locations and identification information on the reference object that matches the object to be inspected at successfully collated locations (S117).SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Patent document 1 discloses a system for determining the authenticity of a product by matching a random pattern in a preset area on the product surface, the system including: a memory for storing features of the random pattern of the area of ​​a genuine product in a reference state of a mark that is in a preset positional relationship with the area; an image acquisition means for acquiring a photographed image including the mark and the area of ​​the target product; an image processing means for acquiring parameters for converting the mark in the photographed image so that it is identical to the mark in the reference state and correcting the area portion in the photographed image using the parameters to an image showing the area when the mark is in the reference state; an extraction means for extracting features from the image of the corrected area portion obtained by the image processing means; a comparison processing means for comparing the features of the target product obtained by the extraction means with the features of the genuine product stored in the memory to calculate a similarity; and a judgment means for judging whether the similarity exceeds a predetermined threshold. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2014-29678 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a technology that can be used to determine the authenticity of an object being inspected by photographing multiple points on the surface of an object such as a product and registering the images in advance as information unique to the object being inspected, and then comparing the registered images with images of similar points on the surface of the object being inspected. In determining the authenticity of an object to be inspected, the accuracy of identification at a given location may be reduced due to surface changes caused by disturbances such as the adhesion of dirt when the object to be inspected is photographed, external stress, or environmental changes. If, due to a reduction in identification accuracy, some of the multiple locations checked have abnormal values, the inspection object as a whole may be immediately judged to have failed the matching, and even a genuine product may be judged to be a counterfeit. An object of the present invention is to provide an information processing device and an information processing program that can present information for confirming the authenticity of an object to be inspected even if abnormal values ​​are present in some locations. [Means for solving the problem]

[0005] An information processing device according to a first aspect includes a processor that compares images of multiple locations on a surface of an inspected object to be inspected with images of multiple locations on the surface of at least one reference object that serves as a reference and that correspond to multiple locations on the surface of the inspected object, and presents the number of locations among the multiple locations that are successfully matched and identification information of the reference object that matches the inspected object at the locations where matching was successful.

[0006] An information processing device according to a second aspect is the information processing device according to the first aspect, wherein the processor presents the identification information and also presents an index indicating the degree of match between the inspected object and the reference object.

[0007] An information processing device according to a third aspect is the information processing device according to the second aspect, wherein, when presenting a plurality of pieces of identification information, the processor presents the identification information arranged in order of the size of the index.

[0008] An information processing device according to a fourth aspect is an information processing device according to any one of the first to third aspects, wherein the multiple locations are each classified by location on the inspected object, and the processor uses a threshold value set for each location to determine the success of matching with the reference object for the location classified into that location.

[0009] An information processing device according to a fifth aspect is an information processing device according to any one of the first to third aspects, in which the multiple locations are each classified according to location on the object to be inspected, and when two or more locations are classified into one location and the reference objects that match the object to be inspected are different in the two or more locations classified into the one location, the processor gives priority to one of the two or more locations and determines the reference object that matches the object to be inspected in the one location.

[0010] An information processing device according to a sixth aspect is an information processing device according to any one of the first to fifth aspects, wherein images of two or more of the multiple locations on the surface of the reference object are positioned within a field of view of an imaging means when imaging the reference object, and the two or more locations are positioned within an area located inside the field of view range, but narrower than the field of view range.

[0011] An information processing device according to a seventh aspect is an information processing device according to the sixth aspect, in which the two or more locations arranged within the narrow area that are furthest from each other are arranged along the outer edge of the narrow area, sandwiching the center of the narrow area.

[0012] The information processing program according to the eighth aspect causes the information processing device according to any one of the first to seventh aspects to function. Effect of the Invention

[0013] According to the first and eighth aspects, the number of successful matches and the identification information of the reference object that matches the test object at the successful matched portion can be presented as information for confirming the authenticity of the test object. As a result, it is possible to provide an information processing device that is more practical than a device that immediately determines that matching has failed when there is an abnormal value in a portion.

[0014] According to the second aspect, by also presenting an index showing the degree of match with a reference object, it becomes possible to infer the accuracy of the matching results of the identification information, thereby enabling more stable and comprehensive individual identification.

[0015] According to the third aspect, when multiple pieces of identification information are presented, the identification information is presented in order of the size of the index indicating the degree of match, making it possible to grasp the order of the certainty of the identification information.

[0016] According to the fourth aspect, by using a threshold value set for each location, it becomes possible to change the threshold value depending on the location, and it becomes possible to give priority depending on the location.

[0017] According to the fifth aspect, it is possible to give priority to any one of two or more locations.

[0018] According to the sixth aspect, the two or more locations are arranged within the field of view, but within an area narrower than the field of view. This makes it possible to deal with the occurrence of some degree of shifting due to lateral shifting, rotation, etc. of the photographing location during observation, and makes it possible to place the two or more locations within the field of view.

[0019] According to the seventh aspect, by arranging the two furthest points out of two or more points along the outer edge of the narrow area, with the center of the narrow area in between, rust, dirt, etc. are less likely to affect both of the two points compared to arranging the points not along the outer edge but inside the outer edge. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing device. [Diagram 2] FIG. 2 is a block diagram showing a hardware configuration of the information processing device. [Diagram 3] FIG. 1 is a conceptual diagram showing an example of multiple locations on a bottle as a reference object. [Figure 4]13 is a diagram showing an example of a settable range of registration locations within a field of view and the registration locations; FIG. [Diagram 5] (A) and (B) are diagrams showing the case where the registration location settable range is not set, where (A) is a schematic diagram of the photographed location of the reference object, and (B) is a schematic diagram of the photographed location where lateral shift has occurred on the object being inspected, and (C) and (D) are diagrams showing the case where the registration location settable range is set, where (C) is a schematic diagram of the photographed location of the reference object, and (D) is a schematic diagram of the photographed location where lateral shift has occurred on the object being inspected. [Figure 6] (A) and (B) are diagrams showing the case where the registration location settable range is not set, where (A) is a schematic diagram of the photographed location of the reference object, and (B) is a schematic diagram of the photographed location where rotation has occurred on the object being inspected, and (C) and (D) are diagrams showing the case where the registration location settable range is set, where (C) is a schematic diagram of the photographed location of the reference object, and (D) is a schematic diagram of the photographed location where rotation has occurred on the object being inspected. [Figure 7] 4 is a flowchart showing a flow of information processing by the information processing device. [Figure 8] 13 is a flowchart showing a flow of a comprehensive judgment process. [Figure 9] 13 is a flowchart showing the flow of a process for selecting candidates for a reference object (identification ID) that matches each shooting location ID. [Figure 10] 13 is a flowchart showing a flow of a process for implementing a final determination. [Figure 11] 13 is an example of a screen presented to a user when matching is successful. [Figure 12] 13 is another example of a screen presented to the user when matching is successful. [Figure 13] 13 is yet another example of a screen presented to the user when matching is successful. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, the present embodiment will be described with reference to the drawings. Note that the same components and processes are given the same reference numerals throughout the drawings, and duplicated explanations will be omitted.

[0022] FIG. 1 is a diagram showing a schematic configuration of an information processing system according to this embodiment.

[0023] FIG. 1 is a diagram showing a schematic configuration of an information processing system 10 including an information processing device 30 according to this embodiment. As shown in Fig. 1, the information processing system 10 is configured by connecting a user terminal device 20 used by a user 100 as a user and an information processing device 30 as a server device to a communication means such as a network. As the communication means, various wireless or wired networks such as the Internet, Ethernet (registered trademark) or wifi (registered trademark) can be applied as described later. Also, while Fig. 1 shows an example of two users 100, two user terminal devices 20, and one information processing device 30, the number is not limited to this.

[0024] In the information processing system 10, each user 100 inputs captured images of multiple locations on the surface of an object to be inspected to a user terminal device 20. The information processing device 30 receives the captured images from the user terminal and compares the captured images with registered images of multiple locations of a reference object that are captured and registered in advance, thereby determining whether the object is authentic. In this embodiment, the authenticity of the object is determined at multiple locations on the object to be inspected, each of which includes multiple locations. In FIG. 1, the information processing device 30 is shown as a device separate from the user terminal device 20. However, the information processing device 30 can be realized by any computer, and may be realized in any of the user terminal devices 20.

[0025] FIG. 2 is a block diagram showing the hardware configuration of the information processing device 30. As shown in FIG. 2, the information processing device 30 has a function as a computer, and includes a CPU (Central Processing Unit) 31 which is an example of a processor, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a storage 34, an input unit 35 as a user interface, a display unit 36 ​​which is a liquid crystal display device, and a communication I / F (communication interface) 37. The various units of the information processing device 30 are connected to each other via a bus 39 so as to be able to communicate with each other.

[0026] The CPU 31, which is an example of a processor, is a central processing unit that executes various programs and controls each part. That is, the CPU 31 reads a program from the ROM 32 or the storage 34, and executes the program using the RAM 33 as a working area. The CPU 31 controls each part of the information processing device 30 and performs various calculation processes according to the program recorded in the ROM 32 or the storage 34.

[0027] The ROM 32 stores various programs and various data. The RAM 33 temporarily stores programs or data as a working area. The storage 34 is configured with a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs including an operating system, and various data.

[0028] The input unit 35 as a user interface is an interface used when the server administrator uses the information processing device 30, and is, for example, an input unit 35 such as a button or a touch panel. The display unit 36 ​​has a display screen such as a liquid crystal display.

[0029] The communication interface 37 is an interface for communicating with a user terminal device 20 such as a personal computer. A wired or wireless communication method is used for the communication interface 37. For example, Ethernet (registered trademark), FDDI, Wi-Fi (registered trademark), etc. are used as the communication standard for the communication interface 37.

[0030] When executing the above programs, the information processing device 30 realizes various functions using the above hardware resources.

[0031] In this embodiment, an information processing program for causing a computer to function as the information processing device 30 described above is stored in the RAM 33 or the storage 34. The CPU 31 can read the above-mentioned information processing program from the ROM 32 or the storage 34, and execute the program using the RAM 33 as a working area. 1 and 2, the user side has an imaging device such as a camera for imaging the surface of the object to be inspected, and a reading device such as a scanner for reading the image captured by the imaging device as data, etc. The administrator side having the information processing device 30 may also have a similar reading device, etc.

[0032] FIG. 3 is a conceptual diagram showing an example of multiple locations on a bottle as a reference object.

[0033] A bottle 110 will be taken as an example of a reference object that serves as a reference for comparison with the object to be inspected. In the bottle 110 serving as the reference object, two locations, a cap 120 and a paper label 130 of the product, are registered as a set in association with the same identification information. This identification information, also known as an individual ID, is a number, symbol, letter, etc. that can identify a bottle 110 of a pre-registered product, and is assigned to the bottle 110 when it is registered.

[0034] In this embodiment, a shooting location ID is assigned to each of the two locations, the lid 120 and the label 130. For example, a number is assigned to each location, such as the shooting location ID (X1) for the lid 120 and the shooting location ID (Y1) for the label 130. Furthermore, at each photographing location, a plurality of registered locations (three locations in this example) are set to be registered as images in advance, and each location is assigned a registered location number. For example, in a field of view 150 that is a photographed image of a rectangular frame obtained by photographing a lid, a registered location 141 at the upper left corner of the rectangular frame is set as registered location number R1, a registered location 142 at the center of the rectangular frame is set as registered location number R2, and a registered location 143 at the lower right corner of the rectangular frame is set as registered location number R3. Of course, these numbers are assigned for convenience, and are not limited to the above-mentioned numbers.

[0035] Here, the detailed setting positions of the above-mentioned registration points 141, 142, and 143 will be further explained with reference to Figs. Fig. 4 is a diagram showing an example of a registration location settable range and registration locations within the field of view. Fig. 5 is a diagram for explaining a comparison between a case where registration locations are provided within the entire field of view and a case where a narrower registration location settable range is provided within the field of view and then registration locations are provided within the registration location settable range. Fig. 6 is a diagram for explaining a comparison between a case where registration locations are provided within the entire field of view and a case where a narrower registration location settable range is provided within the field of view and then registration locations are provided within the registration location settable range.

[0036] In this embodiment, as shown in FIG. 4, all of the multiple registration points 141, 142, and 143 on the surface of the reference object are located within a region inside the field of view range 150, within a registration point settable range 160 (the diagonal line region in FIG. 4), which is an area narrower than the field of view range 150.

[0037] On the other hand, as shown in Fig. 5(A) and Fig. 6(A), it is assumed that the registration points 141 and 143 are set within the above-mentioned field of view 150 but outside the registration point settable range 160, which is an area narrower than the field of view 150. In this case, if the field of view of an imaging device such as a camera shifts sideways when the location is photographed, the registration point 143 will not be within the field of view, as shown in Fig. 5(B). In this case, the registration point 143 will not be used for authenticity determination. Also, as shown in Fig. 6(B), if the field of view of the camera is rotated, the registration points 141 and 143 will fall outside the field of view.

[0038] Therefore, in this embodiment, as shown in Figures 4 and 5(C), a registration location settable range 160 is provided, and the positions of the registration locations are determined so that all registration locations 141, 142, and 143 fall within that range. Therefore, as shown in Figure 5(D), even if the camera's field of view range 150 shifts slightly to the side, the registration locations can be kept within field of view range 150. Also, as shown in Figure 6(D), even if the camera's field of view range 150 rotates slightly, the registration locations can be kept within field of view range 150.

[0039] Furthermore, in this embodiment, of the two or more registration locations 141, 142, 143 located within the registration location settable range 160, which is an area narrower than the field of view range 150 described above, the two registration locations 141 and 143 that are furthest from each other are located on the diagonal of the registration location settable range 160, sandwiched between the center of the registration location settable range 160, along the outer edge of the registration location settable range 160 (the diagonal line area in Figure 4), which is the narrow area described above.

[0040] Here, the center is, for example, the intersection of the diagonals of the rectangular field of view range 150 or the registration location settable range 160. Also, the center is, for example, the center of gravity of the field of view range 150 or the registration location settable range 160 when the field of view range 150 or the registration location settable range 160 is not rectangular.

[0041] According to this embodiment, the two registration points 141, 143 that are furthest from each other among the multiple registration points 141, 142, 143 are positioned along the outer edge of the registration point settable range 160, sandwiching the center of the registration point settable range 160. This ensures a distance between the two registration points 141, 143, making it easier to prevent partial surface changes due to, for example, rust, dirt, etc. from affecting both of the registration points 141, 143.

[0042] The images of the above-mentioned registered locations 141, 142, and 143 include, for example, a random pattern that is an inherent feature on the surface of an article, or a random pattern that is generated on the surface of a product by a mold or the like during the process of forming the product even if the surface of the product does not originally have such a random pattern. The random pattern means an inherent feature on the surface of the article, or a surface feature that is generated on the surface of the product by a mold or the like during the process of forming the product (for example, generated by a difference in mold, temperature, material, etc.). Therefore, products that are continuously produced at the same time, that is, products of the same lot, have the same features, while products produced in different lots even if they are produced on the same line may have different features. The shooting location and the registered location are determined in advance so that such features are included. The information processing device 30 identifies the lot of the inspected object or determines that it is a counterfeit by comparing features such as a random pattern included in the location corresponding to the registered location in the photographed image obtained by photographing the photographed location of the inspected object with the features of the genuine product that have been registered in advance.

[0043] As a more specific example, the unique feature of an object includes a random pattern having randomness, such as the transparency of the paper of the label 130 of the bottle 110, which varies randomly. Such a random pattern is a unique feature of the object distributed along the surface of the object, and is useful for determining the authenticity (matching) of the object. For example, since it is impossible to control the degree of entanglement of the fibrous material that forms the paper of the label 130 during manufacturing, the degree of entanglement of the fibrous material that forms the paper can be considered to be random. The degree of entanglement of the fibrous material that forms the paper can be observed using transmitted light.

[0044] Furthermore, as described above, the unique features of an object are not limited to random patterns, but regular, characteristic geometric patterns, symbols, or unevenness can also be used to determine whether the object being inspected is genuine (matches) with a reference object. Furthermore, it is possible to determine the authenticity of the inspected object and the reference object by the above-mentioned method even if a special mark, minute metal flakes mixed in paint, or unevenness is formed at a predetermined position. Furthermore, it is also possible to perform determination by combining multiple random patterns, regular patterns, special marks, minute metal flakes mixed in paint, etc., as described above.

[0045] Next, the operation of the information processing device 30 will be described. Fig. 7 is a flowchart showing the flow of information processing by the information processing device. Fig. 8 is a flowchart showing the flow of the comprehensive judgment process in step S108 in Fig. 7. Fig. 9 is a flowchart showing the flow of the selection process of identification information candidates for each shooting location ID in step S114 in Fig. 8. Fig. 10 is a flowchart showing the flow of the implementation process of the final judgment in step S116 in Fig. 8.

[0046] In step S101, the CPU 31 acquires an image from the user terminal device 20. At this time, the user 100 designates, in the user terminal device 20, whether the image is an image of a reference object or an image of an object to be inspected. The designation is transmitted to the information processing device 30 as an attribute of the image, etc. Then, the process proceeds to the next step S102. In step S102, the CPU 31 sets, in the acquired image, the registration location settable range 160, which is an area narrower than the visual field range 150 as described above, and then proceeds to the next step S103. In step S103, the CPU 31 performs a so-called cut-out process to set the above-mentioned registration locations 141, 142, and 143 within the above-mentioned registration location settable range 160. Then, the process proceeds to the next step S104.

[0047] In step S104, the CPU 31 performs the process of extracting data from the above-mentioned registration locations 141, 142, and 143. Then, the process proceeds to the next step S105. In step S105, CPU 31 determines whether or not to perform registration processing on the image. Whether or not to perform registration processing is determined based on whether the attribute of the image is a reference object or an object to be inspected. If the attribute of the image is a reference object, CPU 31 determines to perform registration processing. If it is determined to perform registration processing (step S105: YES), the process proceeds to step S106, and if it is determined not to perform registration processing (step S105: NO), i.e., if the attribute of the image is an object to be inspected, the process proceeds to step S107.

[0048] In step S106, the CPU 31 stores the image including the registration points 141, 142, and 143 as an image of the reference object in the information processing device 30. Then, the process ends.

[0049] In step S107, CPU 31 performs a comparison process between registered locations 141, 142, and 143 of the image of the object to be inspected and the registered image of the reference object. Here, the reference object is selected from among pre-registered reference objects, which corresponds to the object to be inspected. Specifically, for example, if the object to be inspected is a bottle as described above, a reference object registered as a bottle is selected. A plurality of reference objects may be selected. When there are a plurality of corresponding reference objects, the object to be inspected is compared to all of the reference objects. In the following, a description will be given assuming that there are a plurality of corresponding reference objects. Furthermore, when different locations such as the lid 120 and the label 130 are set on the same object, as in the case of the bottle described above, a comparison process is performed between the registered locations 141, 142, and 143 having the same photography location ID. When comparing corresponding registered locations, for example, the distribution of brightness values ​​of pixels included in the images of the corresponding registered locations of the reference object and the inspected object is compared, and a correlation value is calculated using a normalized cross-correlation method in template matching of an image processing technique. Here, an example of the correlation value is the matching rate of the images of the corresponding registered locations of the reference object and the inspected object. By comparing the correlation value with a preset threshold, a comparison process is performed for each registered location to determine whether the inspected object matches the reference object. The comparison of the images of the corresponding registered locations of the reference object and the inspected object is not limited to the above example, and may be performed by any method. For example, the image of the registered location of the inspected object may be compared with the image of the corresponding registered location of the reference object by pattern matching. Then, the process proceeds to the next step S108. In step S108, the CPU 31 performs a comprehensive determination process, and then the process ends.

[0050] The above-mentioned comprehensive judgment process will be described with reference to the flowchart of FIG. In step S110, CPU 31 counts the number of registered locations of the inspected object that are determined to match the reference object (identification ID) by the judgment based on the correlation value in the above-mentioned comparison process. Then, the process proceeds to the next step S111. There are cases where a plurality of reference objects are determined to match one registered location, but even in such cases, the number of registered locations that match the reference object is counted as one.

[0051] In step S111, the CPU 31 determines whether or not the number of registered locations determined to match the reference object (identification ID) is not 0 as a result of the counting. If the CPU 31 determines that the number of registered locations that match the reference object is not 0 (step S111: YES), it proceeds to step S112, and if the CPU 31 determines that the number of registered locations that match the reference object is 0 (step S111: NO), it proceeds to step S118.

[0052] In step S112, the CPU 31 tallies up the scores of the registered portions of the reference object that have been determined to match the reference object (identification ID), and then proceeds to the next step S113. In this embodiment, the "score" refers to the number of registered points determined to match the reference object, and is given, for example, for each shooting location. For example, assume that two registered points out of a plurality of registered points on the object under test are determined to match the reference object A. In this case, the score is given the value 2. Also, assume that one of the multiple registered points on the object under test is determined to match reference object A, and the other two registered points are determined to match reference object B. In this case, in this step, the number of registered points that is greater is given as the score. That is, in this example, 2, which is the number of registered points that match reference object B, is given as the score.

[0053] The score does not have to be an integer. For example, when one of the three registered points 141, 142, and 143 included in one shooting location matches the corresponding registered point of any of the reference objects, the score may be 1 / 3. In step S113, CPU 31 determines whether the score of the shooting location with the highest score among the multiple shooting locations is greater than a threshold. If CPU 31 determines that the highest score is greater than the threshold (step S113: YES), it proceeds to step S114, and if it determines that the highest score is equal to or less than the threshold (step S113: NO), it proceeds to step S118. The threshold is, for example, 0. If the threshold is set to 0, the number of registered locations determined to match in step S111 is not 0, so in principle, it does not proceed to step S118. The larger the threshold, the stricter the match determination will be.

[0054] Also, in this embodiment, as described above, a shooting location ID (for example, shooting location ID (X1) for lid 120 and shooting location ID (Y1) for label 130) is assigned to each different location of lid 120 and label 130. A different threshold such as the above-mentioned thresholds "0", "1", and "2" may be set for each shooting location ID, that is, for each lid 120 and label 130 as described above. In other words, the threshold set for each shooting location ID may be used to determine whether or not a part classified into that location matches a reference object, that is, whether or not matching with the reference object was successful. In this way, by using the threshold set for each shooting location ID, it becomes possible to change the threshold depending on the location, and priority can be given based on location.

[0055] In step S114, the CPU 31 identifies which reference object matches for each shooting location ID, and selects the identified reference object as a candidate for the reference object matching the object to be inspected. Specifically, the CPU 31 associates the identification ID of the reference object as a candidate with the shooting location. This process will be described in detail later. Then, the CPU 31 proceeds to the next step S115. In step S115, the CPU 31 determines whether or not there is a reference object identified as matching for any of all the shooting locations. If the CPU 31 determines that there is a matching reference object (step S115: YES), the CPU 31 proceeds to the next step S116, and if the CPU 31 determines that there is no identification information candidate (step S115: NO), the CPU 31 proceeds to step S118.

[0056] In step S116, the CPU 31 performs a process of implementing a final judgment. In the final judgment, for example, for each photographing location of the inspected object, it is judged whether or not a matching reference object has been specified, in other words, whether or not matching with any reference object has been successful. Note that this process will be described in detail later. Then, the process proceeds to the next step S117. In step S117, CPU 31 presents to the user, as a result of the judgment, for example, the identification IDs of the reference objects judged to match the entire inspected object, and the number of successful matches for each shooting location. Then, the process returns to the process shown in FIG.

[0057] In step S118, the CPU 31 finally determines that the matching has failed, and proceeds to the next step S117. In this case, in step S117, the result of the matching failure is notified to the user, and the process returns to the process shown in FIG.

[0058] Next, the process of selecting candidates for a reference object (identification ID) that matches for each shooting location in step S114 in FIG. 8 will be described with reference to FIG. In step S151, as long as there is an unprocessed shooting location ID, the CPU 31 proceeds to the next step S152 and repeats the processes from step S151 to step S155. For example, a shooting location ID is assigned to each shooting location, and the process is performed for each shooting location in the order of the shooting location ID. In step S152, the CPU 31 sums up the scores of the registered locations included in the photography location to be processed for each matching reference object (identification ID) and compares them, and then proceeds to the next step S153. In step S153, the CPU 31 determines whether or not there are a plurality of reference objects (identification IDs) that have the same maximum combined score among the matched reference objects (identification IDs) in the same shooting location being processed. In this specification, the presence of a tie is also expressed as the presence of a tie when there are multiple reference objects (identification IDs) that match the test object in the same shooting location and have the same and maximum combined score. If the CPU 31 determines that there is a tie for a shooting location ID (step S153: YES), it proceeds to the next step S156, and if it determines that there is no tie for a shooting location ID (step S153: NO), it proceeds to step S154.

[0059] In step S154, CPU 31 associates the identification ID of the reference object with the maximum score described above with the photography location being processed, and selects the identification ID as a candidate for the reference object that matches the object to be inspected, and then proceeds to the next step S155. In step S155, as long as there is an unprocessed shooting location, the CPU 31 returns to step S151 and repeats the processes from step S151 to step S155. If there is no unprocessed shooting location, the process ends and returns to the process in FIG. In step S156, the CPU 31 determines whether or not it is possible to assign priorities to the registration locations according to the priority rule.

[0060] Here, the priority rule is, for example, a rule that determines which registered location is to be prioritized among registered locations in the photography location being processed. A specific example will be described using the example shown in Fig. 3. It is assumed that, among registered locations 141, 142, and 143 in the lid 120 that is the photography location, the registered location 142 has a high priority and the registered locations 141 and 143 have a low priority. Here, if the registered location 141 and the registered location 142 have the same and highest scores, the CPU 31 determines that the reference object (identification ID) that matches the registered location 142 is prioritized according to the priority rule, that is, the registered location 142 can be prioritized.

[0061] In addition, in the case where a plurality of registration locations are provided in this way, when a plurality of reference objects matching the registration locations are selected, the reference object with the most matches may be prioritized by majority vote. Also, it is possible to determine which photography location is to be prioritized not among the registration locations but among the photography locations. In this case, when the matching reference object differs for each photography location, the reference object that matches the photography location with the higher priority is prioritized. In step S156, if CPU 31 determines that prioritization based on the priority rule is possible (step S156: YES), the process proceeds to step S154, and if it determines that prioritization based on the priority rule is not possible (step S156: NO), the process proceeds to step S157. When proceeding to step S154, a reference object that matches the registered portion for which prioritization is possible is selected as a candidate for a reference object that matches the object to be inspected. In step S157, the CPU 31 determines that there is no matching reference object candidate based on the photography location ID currently being processed, and proceeds to step S155.

[0062] Next, the process of implementing the final determination in step S116 in FIG. 8 will be described with reference to FIG. In step S171, the CPU 31 determines whether or not there is one reference object candidate selected by the processing in step S114 and Fig. 9. If the CPU 31 determines that there is not one reference object candidate to be identified (step S171: NO), the CPU 31 proceeds to step S172, and if the CPU 31 determines that there is one reference object candidate (step S171: YES), the CPU 31 proceeds to step S178. In step S172, the CPU 31 determines whether or not the mode is the strict mode. If the CPU 31 determines that the mode is not the strict mode (step S172: NO), the process proceeds to step S173, and if the CPU 31 determines that the mode is the strict mode (step S172: YES), the process proceeds to step S177. The strict mode is a mode in which, for example, when there are two or more candidates for the reference object, the matching is determined to be inaccurate and the matching is strictly determined to be a failure. The strict mode can be set in advance by a user or an administrator of the information processing device 30. If the strict mode is not set, the processes of steps S172 and S177 are omitted, and the process of step S173 is executed. In step S173, the CPU 31 executes a majority vote among the reference object candidates, and then proceeds to the next step S174.

[0063] In step S174, CPU 31 determines whether or not there is a tie for the most votes as a result of the majority vote for the reference object candidates. If there is no tie for the most votes (step S174: YES), CPU 31 proceeds to step S175, and if there is a tie for the most votes (step S174: NO), CPU 31 proceeds to step S176. In step S175, CPU 31 determines that the reference object with the most votes is the reference object that matches the test object, and ultimately determines that matching has been successful, and then returns to the processing of FIG. In step S176, CPU 31 determines the reference object candidate with the highest score calculated in step S152 when the selected candidate is selected from among the reference object candidates with the most votes as the reference object matching the inspected object, and finally determines that matching has been successful. Then, the process returns to FIG. 8.

[0064] In step S177, the CPU 31 finally determines that the comparison has failed, and returns to the process of FIG. In step S178, CPU 31 determines that one reference object candidate selected in step S114 and the process of Fig. 9 is a reference object that matches the inspected object, and finally determines that matching has been successful. Then, the process returns to the process of Fig. 8. Next, an example will be given of what information is notified to the user when it is finally determined in steps S175, S176, and S178 that the matching is successful.

[0065] FIG. 11 shows an example of a screen presented to the user when the matching is successful. When the reference object candidate with the most votes is determined to be a reference object matching the object to be inspected in step S175, a screen such as that shown in FIG. 11 is presented to the user. The screen is provided with a details display section 300. The details display section 300 shows how many of the registered locations have been successfully matched. In addition, below the number of registered locations that have been successfully matched, it shows which reference object each registered location has been determined to match. In the example shown in FIG. 11, it can be seen that, of registered locations R1 to R6, all registered locations other than registered locations R2 and R5 have been determined to match product A, and the number of matches is four. In this way, even if there is a registration location R2 that is determined to have no corresponding product, the number of successful matches and the identification information (name or identification ID of the reference object) of the reference object that matches the test object in the successful matched registration location can be presented as information for confirming the authenticity of the test object. As a result, it is possible to provide an information processing device that is highly practical compared to a case where a match is immediately deemed to have failed when there is an abnormal value such as "no corresponding product" in the registration location R2.

[0066] FIG. 12 shows another example of the screen presented to the user when the matching is successful. When the candidate for the reference object with the highest score among the candidates for the reference object with the highest number of votes is determined to be the reference object matching the object to be inspected by the above-mentioned step S176, for example, a screen as shown in FIG. 12 is presented to the user. The screen is provided with a detail display section 310. The detail display section 310 presents the same information as the detail display section 300 of FIG. 11, and further presents a score indicating the degree of matching with the reference object for each shooting location. For example, the lid with the shooting location ID of X1 is matched as matching with the product A at the registered locations R1 and R3, so that a score of 2 is given as the product A. Also, the label with the shooting location ID of Y1 is matched as matching with the product B at the registered location R5, and there is no other matching with a product with a high score, so that a score of 1 is given as the product B. As a result of comparing these scores, it is displayed on the screen that it is matched with the product A with a high score. In this way, which reference object and which shooting location were judged to match, as well as the score at that time, are displayed together with the reference object that was judged to match the entire test object. This makes it clear the basis for the matched reference object.

[0067] In the example shown in Figure 12, a score is also presented as an index showing the degree of match with the reference object, allowing the user to estimate the reliability of the matching results of the identification information, enabling more stable and comprehensive individual identification. Furthermore, in this embodiment, as described above, product A, which is the reference object with a score of 2, is displayed at the top, and product B with a score of 1 is displayed below it. When the number of matches with the registered parts of the reference object and multiple reference objects with the matching parts are presented, the reference objects (product A, product B) are presented in order of the magnitude of the score numerical value as an index representing the degree of match described above. In this way, when multiple matching reference objects are presented, the user can grasp the order of the certainty of the identification information by presenting the reference objects arranged in order of the size of the index representing the degree of match.

[0068] FIG. 13 shows yet another example of a screen presented to the user when matching is successful. When the selected one reference object candidate is determined to be a matching reference object by the above-mentioned step S178, and the matching is finally determined to be successful, for example, a screen as shown in FIG. 13 is presented to the user. The screen is provided with a details display section 320. The details display section 320 shows how many of the registered locations have been successfully matched. In addition, under the number of registered locations that have been successfully matched, it shows which reference object each registered location has been determined to match. In the example shown in FIG. 11, it can be seen that, of the registered locations R1 to R6, the registered locations other than the registered location R2 have been determined to match the product A, and the number of matches was five.

[0069] Next, other variations will be described. As a variation of the image capture, in order to increase the accuracy of matching, in this embodiment, multiple (three) registration locations are cut out from one captured image, but this is not particularly limited. Fewer than this number may be cut out, and more than this number may be cut out.

[0070] In order to improve the accuracy of the comparison, multiple images may be taken and multiple registration locations may be extracted from the multiple images instead of a single image. This makes it possible to reduce the influence of differences in shooting conditions.

[0071] Also, images of different surfaces may be registered as objects with a common identification ID. Specifically, for example, in this embodiment, images of the surfaces of the lid 120 and the label 130 of the bottle 110 are registered, but an image of the bottom surface of the bottle 110, which is a different surface, may also be registered.

[0072] As a variation of the reference object registration, rather than photographing the object once and registering that one image, it is also possible to photograph the object multiple times, select the best image, and extract and register multiple registration locations. Furthermore, when cutting out the registration portion, images of the cut out registration portion that have poor image quality may not be registered. Although the above is a variation of registration, similar variations can be obtained in matching. For example, one optimal captured image may be selected, and multiple registration locations may be cut out from that one captured image and compared, or images with poor image quality may not be compared.

[0073] As a variation of the notification of the matching result, only when one identification ID is determined to be true, that identification ID may be notified, and if two or more identification IDs are true, the matching may be treated as a failure.

[0074] The present invention is not limited to the above-described embodiment, and various modifications, changes, and improvements are possible without departing from the spirit and scope of the present invention. For example, the above-described embodiment may be appropriately combined to form a configuration.

[0075] In this embodiment, the embodiment has been described as an example in which each process is realized by software, but the same process as the flowchart of each process may be implemented in, for example, an ASIC (Application Specific Integrated Circuit) and processed by hardware. In this case, the processing speed can be increased compared to when each process is realized by software.

[0076] In the above-mentioned embodiment, the information processing program is installed in a ROM or the like, but the present invention is not limited to this. The information processing program according to the present invention may be provided in a form recorded in a computer-readable storage medium. For example, the information processing program according to the present invention may be provided in a form recorded in an optical disc such as a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM. The information processing program according to the present invention may be provided in a form recorded in a semiconductor memory such as a USB (Universal Serial Bus) memory or a flash memory. Furthermore, the information processing device 30 may acquire the program through a communication line from an external device connected to the communication line.

[0077] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.). In addition, the operations of the processor in each of the above embodiments may be performed not only by one processor, but also by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate. [Explanation of symbols]

[0078] 10 Information processing system, 20 User terminal device, 30 Information processing device, 31 CPU, 32 ROM, 33 RAM, 34 Storage, 35 Input unit, 36 Display unit, 37 Communication I / F, 100 User, 110 Bottle, 120 Lid, 130 Label, 141, 142, 143 Registration location, 150 Field of view range, 160 Registration location settable range, 300, 310, 320 Detail display unit

Claims

1. A processor is provided. The processor, comparing images of a plurality of locations on a surface of an object to be inspected with images of a plurality of locations on a surface of at least one reference object serving as a reference, the plurality of locations corresponding to the plurality of locations on the surface of the object to be inspected; An information processing device that displays the number of successful matches among a plurality of locations, and also displays identification information of a reference object that matches the inspected object for each successful match.

2. The processor, 2. The information processing apparatus according to claim 1, further comprising: a display that displays, in addition to the identification information, an index that indicates a degree of match between the inspection object and the reference object.

3. The processor, The information processing apparatus according to claim 2 , wherein, when a plurality of pieces of identification information are presented, the pieces of identification information are presented in order of the size of the index.

4. The plurality of locations are classified according to locations on the object to be inspected, The processor, Using a threshold value set for each of the locations, a match between the location and the reference object is determined for each of the locations.

4. The information processing device according to claim 1.

5. The plurality of locations are classified according to locations on the object to be inspected, The processor, When two or more locations are classified into one location, and the reference object that matches the object to be inspected is different in the two or more locations classified into the one location, one of the two or more locations is given priority to determine the reference object that matches the object to be inspected in the one location.

4. The information processing device according to claim 1.

6. images of two or more of the plurality of points on the surface of the reference object are arranged within a field of view of an image capturing means when capturing an image of the reference object; The two or more locations are located within an area inside the field of view and narrower than the field of view.

6. The information processing device according to claim 1.

7. Among the two or more points disposed within the narrow region, the two points furthest from each other are disposed along the outer edge of the narrow region and sandwiching the center of the narrow region. The information processing device according to claim 6.

8. An information processing program for causing a computer to function as the information processing device according to any one of claims 1 to 7.

Images