Information processing apparatus, information processing method, and program

Abstract

To reduce time and effort for preparation of information necessary for identification and classification of negotiable securities and non-negotiable securities.SOLUTION: An information processing apparatus 10 comprises: an image information acquisition unit 122 which acquires first image information indicating a first image being an image of a gift certificate GT to be identified; a feature extraction unit 124 which generates a feature vector of the first image by executing, on the first image, processing of generating a feature vector indicating a feature quantity of the image; a distance calculation unit 125 which calculates inter-vector distances which are distances between the feature vector of the first image and feature vectors of second images being respective images of a plurality of reference gift certificates GT; and an identification unit 126 which identifies the gift certificate GT to be identified on the basis of the inter-vector distances.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In retail stores such as supermarkets, in addition to currency, valuable securities such as gift certificates may be handled. For example, in retail stores that handle gift certificates, after business hours, tasks such as sorting used gift certificates by type are performed. For this reason, a gift certificate deposit machine has been proposed that sorts multiple gift certificates into gift certificates of the same type and amount as a first type of gift certificate and gift certificates of a different type or amount from the first type of gift certificate (for example, see Patent Document 1).

[0003] It has also been proposed that by connecting a gift certificate deposit machine to a POS (Point Of Sales) register and change dispenser, store clerks at retail stores and other outlets would be able to identify gift certificates received from customers and process the payment for purchased items (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-044663 A [Patent Document 2] JP 2015-200938 A Summary of the Invention [Problem to be solved by the invention]

[0005] In recent years, research into identifying gift certificates using AI (Artificial Intelligence) has been progressing. For example, a common method of allocating gift certificates is to identify images of gift certificates using a model (hereinafter also referred to as an image recognition model) that has learned the types of gift certificates using images of various gift certificates circulating in the market as training data. This type of identification method requires learning with a lot of training data to create an image recognition model. In recent years, the types of securities such as gift certificates and regional promotion coupons circulating in the market have been increasing, and not only securities but also non-securities such as service coupons and coupons have been increasing. For this reason, in the conventional identification method using AI that uses an image recognition model, there was a problem that learning with training data requires a lot of time and effort, and it takes a huge amount of effort to generate information necessary to identify the gift certificates to be allocated. Hereinafter, securities and non-securities may be collectively referred to as securities.

[0006] In recent years, the number of cases where customers use semi-self-service registers or self-service registers when paying for purchased items at retail stores and other stores has been increasing. For this reason, there is a demand for more accurate recognition of securities and service coupons such as gift certificates in a shorter time. However, with conventional recognition methods, it can take more than 10 seconds to recognize a gift certificate, and there is also the problem that the recognition accuracy decreases when the processing time is shortened. [Means for solving the problem]

[0007] An information processing device according to a preferred embodiment of the present invention includes an acquisition unit that acquires first image information indicating a first image, which is an image of a security to be identified; a generation unit that generates a feature vector of the first image by executing a process on the first image to generate a feature vector indicating image features; a calculation unit that calculates an inter-vector distance, which is the distance between the feature vector of the first image and the feature vector of a second image, which is an image of each of a plurality of reference securities registered in advance; and an identification unit that identifies the security to be identified based on the inter-vector distance.

[0008] In addition, an information processing method according to a preferred embodiment of the present invention acquires first image information indicating a first image, which is an image of a security to be identified, generates a feature vector of the first image by performing a process on the first image to generate a feature vector indicating image features, calculates a vector distance between the feature vector of a second image, which is an image of a reference security, and the feature vector of the first image, and identifies the security to be identified based on the vector distance.

[0009] In addition, an information processing method according to a preferred embodiment of the present invention obtains first image information indicating a first image, which is an image of a security to be registered, and generates a feature vector of the first image by executing a process on the first image to generate a feature vector indicating image features, and registers the first image information, the generated feature vector, and the name and face value of the security to be registered in correspondence with each other.

[0010] In addition, a program related to a preferred embodiment of the present invention causes a processor to function as an acquisition unit that acquires first image information indicating a first image, which is an image of the security to be identified; a generation unit that generates a feature vector of the first image by executing a process on the first image to generate a feature vector indicating the features of the image; a calculation unit that calculates an inter-vector distance, which is the distance between the feature vector of a second image, which is an image of a reference security, and the feature vector of the first image; and an identification unit that identifies the security to be identified based on the inter-vector distance. Effect of the Invention

[0011] According to the present invention, securities can be easily identified. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is an explanatory diagram for explaining an overview of an information processing system according to a first embodiment. [Diagram 2]FIG. 2 is an explanatory diagram for explaining an example of the bill type table shown in FIG. [Diagram 3] 13 is a flowchart showing an example of the operation of the information processing device when a new gift certificate is registered in the certificate type table. [Figure 4] 11 is an explanatory diagram for explaining an example of a confirmation screen displayed on a display device when a gift certificate to be identified is read by an image input device. FIG. [Diagram 5] 4 is a flowchart showing an example of an operation of the information processing device shown in FIG. [Figure 6] 6 is a flowchart showing an example of the surface identification process shown in FIG. 5. [Figure 7] FIG. 11 is an explanatory diagram for explaining an overview of an information processing system according to a first modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. In each drawing, the dimensions and scale of each part are appropriately different from the actual ones. In addition, since the embodiment described below is a preferred specific example of the present invention, various technically preferable limitations are attached, but the scope of the present invention is not limited to these embodiments unless otherwise stated in the following description to the effect that the present invention is particularly limited. In addition, in the present invention, a case is exemplified in which the information processing system 1 is a securities processing system that identifies and classifies securities such as gift certificates GT by type, but the present invention is not limited to such an embodiment. For example, the information processing system 1 may be a system that identifies non-securities other than securities (for example, service vouchers, coupons, discount vouchers, etc.). In the present invention, securities and non-securities are referred to as securities.

[0014] [1. First embodiment] First, an example of an overview of an information processing system 1 according to an embodiment will be described with reference to FIG.

[0015] FIG. 1 is an explanatory diagram for explaining an overview of an information processing system 1 according to the first embodiment.

[0016] In this embodiment, it is assumed that the information processing system 1 is a securities processing system that identifies and classifies securities such as gift certificates GT by type. Also, in this embodiment, it is assumed that the securities are gift certificates GT.

[0017] The information processing system 1 includes, for example, an information processing device 10, an image input device 20, and a display device 30. In the example shown in FIG. 1, the information processing device 10 is connected to each of the image input device 20 and the display device 30 so as to be able to communicate with each other. The connection between the information processing device 10 and the image input device 20 and the connection between the information processing device 10 and the display device 30 may be a connection using either a wired or wireless connection, or a connection using both a wired and wireless connection. The connection between a plurality of elements including the information processing device 10 may be a connection that allows the plurality of elements to communicate with each other, and may be a connection via a network. The network may include the Internet and a local area network.

[0018] In this embodiment, for example, it is assumed that the information processing device 10, the image input device 20, and the display device 30 are installed in a retail store that handles gift certificates GT. In this case, one or both of the image input device 20 and the display device 30 may be included in the information processing device 10.

[0019] The image input device 20 is, for example, an optical device such as a camera or an image scanner. For example, the image input device 20 generates image information showing an image of the gift certificate GT by capturing an image of the gift certificate GT. The image information generated by the image input device 20 is transmitted to the information processing device 10. For example, when a user such as an employee of a retail store where the image input device 20 is installed identifies and classifies gift certificates GT by type, the user causes the image input device 20 to read the gift certificate GT to be identified. The image input device 20 generates image information showing, for example, an image of the read gift certificate GT (image of the gift certificate GT to be identified) and transmits the generated image information to the information processing device 10. The image input device 20 may store the image information in a shared memory (not shown). In this case, the information processing device 10 may read the image information from the shared memory.

[0020] Any information processing device can be adopted as the information processing device 10. For example, the information processing device 10 includes a processing device 120 that controls each unit of the information processing device 10, a storage device 140 that stores various information, and a communication device 160.

[0021] The communication device 160 is hardware for communicating with an external device that exists outside the information processing device 10.

[0022] The storage device 140 includes, for example, one or both of a volatile memory such as a random access memory (RAM) that functions as a working area for the processing device 120 and a non-volatile memory such as an electrically erasable programmable read-only memory (EEPROM) that stores various information such as the control program PG1. The storage device 140 may be detachable from the information processing device 10. Specifically, the storage device 140 may be a storage medium such as a memory card that is detachable from the information processing device 10. The storage device 140 may also be, for example, a storage device (e.g., online storage) that is communicatively connected to the information processing device 10 via a network or the like.

[0023] In this embodiment, the storage device 140 stores a denomination table TBL in addition to the control program PG1. The denomination table TBL will be described later with reference to FIG.

[0024] The processing device 120 is a processor that controls the entire information processing device 10, and is configured to include, for example, one or more CPUs (Central Processing Units). For example, the processing device 120 executes a control program PG1 stored in the storage device 140 to function as an image information acquisition unit 122, a surface identification unit 123, a feature extraction unit 124, a distance calculation unit 125, a recognition unit 126, and a display control unit 128. As a result, the information processing device 10 functions as, for example, a securities processing device that recognizes gift certificates GT and tallys them by type. The control program PG1 may be transmitted from another device via a network.

[0025] Furthermore, for example, when the processing device 120 is configured to include multiple CPUs, some or all of the functions of the processing device 120 may be realized by the multiple CPUs operating in cooperation with each other according to a program such as the control program PG1. Furthermore, the processing device 120 may be configured to include hardware such as a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array) in addition to one or more CPUs, or instead of some or all of the one or more CPUs. In this case, some or all of the functions of the processing device 120 may be realized by hardware such as a DSP.

[0026] Thus, in this embodiment, the control program PG1 includes application programs for causing the processing device 120 to function as the image information acquisition unit 122, the surface identification unit 123, the feature extraction unit 124, the distance calculation unit 125, the identification unit 126, and the display control unit 128. Note that the control program PG1 may include, for example, an operating system program for the processing device 120 to control each unit of the information processing device 10. The control program PG1 is an example of a "program." Also, the image information acquisition unit 122 is an example of an "acquisition unit," the feature extraction unit 124 is an example of a "generation unit," and the distance calculation unit 125 is an example of a "calculation unit."

[0027] The image information acquisition unit 122 acquires, for example, first image information indicating a first image, which is an image of the gift certificate GT to be identified, from the image input device 20 via the communication device 160. Note that, when the image input device 20 stores the first image information in a shared memory (not shown), the image information acquisition unit 122 may acquire the first image information from the shared memory. Also, the image information acquisition unit 122 may acquire first side image information indicating an image of the first side of the gift certificate GT to be identified, and second side image information indicating an image of the second side opposite to the first side of the gift certificate GT to be identified. Note that one of the first side and the second side of the gift certificate GT corresponds to the front side of the gift certificate GT, and the other of the first side and the second side corresponds to the back side of the gift certificate GT. In this embodiment, it is assumed that the image information acquisition unit 122 acquires the first side image information and the second side image information.

[0028] The front surface specifying unit 123 specifies the front and back surfaces of the gift certificate GT to be identified based on the number of contours included in the image of the first surface shown by the first surface image information and the number of contours included in the image of the second surface shown by the second surface image information. For example, the front surface specifying unit 123 specifies the side of the first surface or the second surface with the fewer contours as the front surface of the gift certificate GT, and specifies the side with the greater number of contours as the back surface of the gift certificate GT. The front surface specifying unit 123 then specifies the image information showing the image of the front surface of the first surface image information or the second surface image information as the first image information. Note that, as a method for calculating the number of contours included in an image, for example, a known method for detecting edges included in an image (for example, the Canny method) can be adopted.

[0029] The feature extraction unit 124 executes a process of generating a feature vector indicating the feature amount of an image on the first image (an image of the front side of the gift certificate GT to be recognized) indicated by the first image information acquired by the image information acquisition unit 122, thereby generating a feature vector of the first image. Then, the feature extraction unit 124 generates feature information indicating the feature vector of the first image. The process of generating a feature vector includes, for example, a plurality of convolution layers and pooling layers in a convolutional neural network. In this embodiment, for example, the feature extraction unit 124 generates the feature vector of the first image using a model that has been trained (metric learning) to reduce the distance between the feature vectors of a plurality of images of the same type of certificate and to increase the distance between the feature vectors of a plurality of images of different types of certificates. In this way, in this embodiment, a model that has been trained on metric learning specifically for gift certificates is used.

[0030] The distance calculation unit 125 calculates the inter-vector distance, which is the distance between the feature vector of the second image, which is an image of the reference gift certificate GT, and the feature vector of the first image. Here, the reference gift certificate GT is, for example, a gift certificate GT registered in the certificate type table TBL, with the feature vector of the image (second image) of the front side of the gift certificate GT and the type (certificate type) of the gift certificate GT corresponding to each other. For example, the reference gift certificate GT is associated with certificate type information indicating the type of the gift certificate GT and feature information indicating the feature vector of the image of the front side of the gift certificate GT.

[0031] The recognition unit 126 recognizes and classifies the gift certificate GT to be recognized based on the inter-vector distance. For example, the recognition unit 126 determines whether the inter-vector distance is equal to or less than a first distance. If the inter-vector distance is equal to or less than the first distance, the recognition unit 126 determines that the first image and the second image are identical or similar (highly related), and if the inter-vector distance is greater than the first distance, the recognition unit 126 determines that the first image and the second image are different (lowly related).

[0032] Here, for example, the first distance is set to a value that allows the types of the two gift certificates GT shown by the first image and the second image to be considered to be the same as each other if the distance (vector distance) between the feature vector of the first image and the feature vector of the second image is equal to or less than the first distance. Therefore, in this embodiment, when a second image in which the vector distance is equal to or less than the first distance is specified, the recognition unit 126 determines that the type of the gift certificate GT to be identified is the same as the type of the reference gift certificate GT shown by the second image in which the vector distance is equal to or less than the first distance. In other words, when a second image in which the vector distance is equal to or less than the first distance is specified, the recognition unit 126 specifies the type information of the reference gift certificate GT shown by the specified second image as the type information of the gift certificate GT to be identified.

[0033] The first distance can be set arbitrarily, but if the first distance is set large, the proportion of identifiable gift certificates GT will increase, but there is a risk of an increase in misrecognition. On the other hand, if the first distance is set small, there is a risk of a decrease in the proportion of identifiable gift certificates GT, but it is possible to reduce misrecognition. Therefore, it is preferable to set the first distance appropriately according to the required standards. In addition, it is also possible to make the judgment criteria easier to understand by setting the first distance to "1.0", "10", or "100" for convenience.

[0034] Furthermore, when a second image in which the inter-vector distance is equal to or less than the first distance is not identified, the recognition unit 126 can also register the feature information indicating the feature vector of the first image and the first image information in the note type table TBL in a manner that corresponds to each other. In this embodiment, registering the feature information and the first image information in the note type table TBL in a manner that corresponds to each other corresponds to storing the feature information and the first image information in the memory device 140 in a manner that corresponds to each other. In this embodiment, storing the feature information and the first image information in a memory such as the memory device 140 in a manner that corresponds to each other may be referred to as storing the feature information and the first image information in the note type table TBL in a manner that corresponds to each other.

[0035] When a second image in which the inter-vector distance is equal to or less than the first distance is specified, the display control unit 128 causes the display device 30 to display the specified second image and the first image. For example, the display control unit 128 causes the display device 30 to display a confirmation screen SC shown in FIG. 4, which will be described later.

[0036] The display device 30 is an output device such as a display that performs output to the outside. For example, the display device 30 displays an image such as a confirmation screen SC shown in FIG. 4 described later under the control of the processing device 120 (more specifically, the display control unit 128) of the information processing device 10. Note that the display device 30 may be configured to be integrated with an operation device (for example, a touch panel) that accepts operations by an administrator of the information processing device 10. In this embodiment, it is assumed that the display device 30 is configured to be integrated with the operation device.

[0037] The configuration of the information processing system 1 is not limited to the example shown in Fig. 1. For example, the information processing device 10 may have an operation device that accepts operations by an administrator of the information processing device 10 or the like.

[0038] Furthermore, the information processing device 10 may be communicably connected to the image input device 20 and the display device 30 via a network such as a LAN (Local Area Network). In this case, the information processing device 10 may function as a server for identifying and classifying the gift certificates GT by type. For example, an application executed by a terminal device or the like functioning as the display device 30 may call APIs related to the processing of each of the image information acquisition unit 122, the surface identification unit 123, the feature extraction unit 124, the distance calculation unit 125, the identification unit 126, and the display control unit 128, thereby executing the processing of identifying and classifying the gift certificates GT by type.

[0039] Next, an example of the bill type table TBL will be described with reference to FIG.

[0040] Fig. 2 is an explanatory diagram for explaining an example of the certificate type table TBL shown in Fig. 1. The certificate type table TBL has, for example, one or more records. In the example shown in Fig. 2, the certificate type table TBL has a plurality of records that correspond one-to-one to the plurality of types of gift certificates GT registered in the certificate type table TBL.

[0041] Each record in the denomination table TBL has, for example, denomination information, information indicating a storage destination of image information, and information indicating a storage destination of characteristic information. The denomination information also includes, for example, identification information, information indicating the denomination name, and information indicating the face value. The identification information is information for identifying each type of gift certificate GT from among multiple types. The denomination name is the name of the type of gift certificate GT corresponding to the identification information. For example, the denomination name is displayed on a confirmation screen SC shown in FIG. 4 described below so that employees of a retail store where the display device 30 is installed can easily recognize the type of gift certificate GT used at the retail store. The face value is the amount written on the gift certificate GT.

[0042] Here, the types of gift certificates GT are classified by, for example, a combination of the issuer and face value of the gift certificate GT. For this reason, for example, three gift certificates GT corresponding to three types of certificates with identification information of "001", "002", and "003" have the same issuer of the gift certificate GT but different face values, and therefore are different types from each other. Also, for example, three gift certificates GT corresponding to three types of certificates with identification information of "003", "006", and "022" have the same face value but different issuers of the gift certificate GT, and therefore are different types from each other.

[0043] The types of gift certificates GT are not limited to a combination of the issuer and face value of the gift certificate GT. For example, the types of gift certificates GT may be classified according to the issuer of the gift certificate GT. Alternatively, for example, when a single company that issues the gift certificate GT issues multiple different brands of gift certificates GT, the types of gift certificates GT may be classified according to a combination of the issuer, face value, and brand of the gift certificate GT. Brands of gift certificates GT may include nationwide gift certificates GT and regional gift certificates GT.

[0044] The storage destination of the image information is, for example, an address indicating a memory area in which image information indicating an image of the front side and an image of the back side of the gift certificate GT corresponding to the type of certificate indicated by the identification information is stored. The storage destination of the feature information is, for example, an address indicating a memory area in which feature information indicating a feature vector of the image of the front side of the gift certificate GT corresponding to the type of certificate indicated by the identification information is stored. One or both of the image information and the feature information may be stored in the memory device 140 in which the type of certificate table TBL is stored, or may be stored in a memory device different from the memory device 140.

[0045] In this way, in the type table TBL, the type of gift certificate GT, the image, and the feature vector are associated with each other.

[0046] Next, a method for registering a new gift certificate TG in the certificate type table TBL will be described with reference to the flowchart of FIG.

[0047] FIG. 3 is a flow chart showing an example of the operation of the information processing device 10 when a new gift certificate TG is registered in the certificate type table TBL. The first gift certificate GT to be registered will be described as a XX 1000 yen gift card with identification information "001" in the certificate type table TBL in FIG. 2. In the following, the nth gift certificate GT may be referred to as a gift certificate GTn. For example, in FIG. 3, the XX 1000 yen gift card with identification information "001" shown in FIG. 2, i.e., the first gift certificate GT to be registered, is also referred to as a gift certificate GT1. The gift certificate TG registered in the certificate type table TBL by the operation shown in FIG. 3 corresponds to a "reference security."

[0048] First, in step S51, the processing device 120 functions as the image information acquisition unit 122 and acquires from the image input device 20 first side image information and second side image information indicating images of the first and second sides, respectively, of the gift certificate GT to be registered.

[0049] Next, in step S52, the processing device 120 functions as the front side identification unit 123 and executes a front side identification process to identify which of the first side and the second side is the front side of the gift certificate GT1. Details will be described in Fig. 6, but for example, the front side identification unit 123 identifies the image of the front side of the gift certificate GT1 based on the number of contours in the image of the first side and the number of contours in the image of the second side.

[0050] If the front side of the gift certificate is identified in step S51, step S52 can be omitted by acquiring an image of only the front side.

[0051] Next, in step S53, the processing device 120 functions as the feature extraction unit 124 and calculates a feature vector indicating the feature amount of the image (second image) of the front side of the gift certificate GT1 identified in step S52. That is, the feature vector of the second image is calculated by the processing of step S53.

[0052] The feature vector is calculated using a model trained by the metric learning method described above. The feature vector of the first registered gift certificate GT1 is located in a certain region (first region) in a specific space.

[0053] Next, in step S54, the processing device 120 stores the image information of the front side of the gift certificate GT1 identified in step S52 in the image information storage destination "ab123" shown in Figure 2. In addition, the processing device 120 stores the feature vector calculated in step S53 in the feature information storage destination "tke98" shown in Figure 2. The processing device 120 may store the image information of the front side of the gift certificate GT1 immediately after step S52.

[0054] Next, in step S55, the processor 120 stores the items in the denomination table TBL, namely, the identification information, denomination name, face value, storage destination of the image information, and storage destination of the characteristic information, of the gift certificate GT1 in association with each other in the denomination table TBL. This completes the registration of the gift certificate GT1.

[0055] Next, in step S56, the processor 120 checks whether there is a gift certificate GT to be registered next, and if there is no gift certificate to be registered next, the registration is completed. If there is a gift certificate to be registered next, the processor 120 registers the second gift certificate GT2.

[0056] The second gift certificate GT2 to be registered is a XX gift card of 5,000 yen, whose identification information is "002" in the certificate type table TBL in Fig. 2. The processing device 120 executes a series of processes from step S51 to step S55, similar to the gift certificate GT1. Image information of the front side of the gift certificate GT2 is stored in the image information storage destination "kh296" shown in Fig. 2. In addition, the feature vector calculated in step S53 is stored in the feature information storage destination "raw47" shown in Fig. 2.

[0057] The calculation of the feature vector of the front image of the gift certificate GT2 in step S53 is performed in a region (second region) that is distant from the first region that indicates the feature vector of the already registered gift certificate GT1.

[0058] Similarly, in step S53, the feature vectors of the nth gift certificate GTn from the third gift certificate onwards are positioned in an area (the nth area) that is distant from each of the n-1 areas of the n-1 gift certificates that have already been registered.

[0059] In this way, the identification information, denomination name, face value, storage destination of image information, and storage destination of characteristic information of each gift certificate TGn are stored in the denomination table TBL.

[0060] In this manner, a new gift certificate TG is registered in the certificate type table TBL by reading in one gift certificate TG to be registered, and then, using metric learning, it is positioned in a spatial domain of a feature vector of an already registered gift certificate GT at a predetermined distance from the spatial domain of the feature vector.

[0061] The operations shown in Fig. 3 may be executed by a server communicatively connected to the information processing device 10. The configuration of the note type table TBL is not limited to the example shown in Fig. 2. For example, each record of the note type table TBL may include the characteristic information itself instead of information indicating the storage destination of the characteristic information. For example, each record of the note type table TBL may include the image information itself instead of information indicating the storage destination of the image information.

[0062] Next, with reference to FIG. 4, an example of a confirmation screen SC that is displayed on the display device 30 when identifying the gift certificate TG to be identified will be described.

[0063] Fig. 4 is an explanatory diagram for explaining an example of a confirmation screen SC displayed on the display device 30 when the gift certificate TG to be identified is read by the image input device 20. For example, when the information processing device 10 executes the operation shown in Fig. 5 described below, the display device 30 displays the confirmation screen SC under the control of the processing device 120 (more specifically, the display control unit 128) of the information processing device 10. For example, the gift certificate GT to be identified is classified as a type of certificate having an inter-vector distance equal to or less than a first distance among multiple types of certificates registered in the certificate type table TBL.

[0064] The confirmation screen SC displays, for example, the reading order, the denomination, the face value, the reference image RIMG, the read image, and a plurality of buttons BT for a GUI (Graphical User Interface).

[0065] The reading order is, for example, the reading order when the image input device 20 reads a plurality of gift certificates GT to be identified. The denomination name is the name of the type of the gift certificate GT to be identified. For example, of the plurality of denominations registered in the denomination table TBL, the denomination name of the denomination whose inter-vector distance is equal to or less than the first distance is specified as the denomination name of the gift certificate GT to be identified. The face value is the face value of the gift certificate GT to be identified. For example, of the face values ​​of the plurality of denominations registered in the denomination table TBL, the face value of the same denomination as the gift certificate GT to be identified is specified as the face value of the gift certificate GT to be identified. In addition, the reference image RIMG is an image of the front of the gift certificate GT corresponding to the same denomination as the gift certificate GT to be identified among the plurality of denominations registered in the denomination table TBL. For example, the reference image RIMG is read from the storage location of the image information of the same denomination as the gift certificate GT to be identified among the storage locations of the plurality of image information registered in the denomination table TBL.

[0066] The read image is an image of the gift certificate GT to be identified (an image of the gift certificate GT read by the image input device 20). In the example shown in FIG. 4, an image IIMGf of the front side and an image IIMGb of the back side of the gift certificate GT to be identified are displayed on the confirmation screen SC as the image of the gift certificate GT to be identified. Note that the number area NAR in the image IIMGb of the back side of the gift certificate GT is an area in which a gift certificate number is written so that the card company that issued the gift certificate GT can manage the gift certificate GT. The number area NAR in which the gift certificate number is written varies depending on the card company or the type of the gift certificate. There are also gift certificates GT in which the gift certificate number is not written. In this embodiment, the operation of the information processing device 10 is described mainly with respect to the classification of the gift certificate GT, so that a case in which the gift certificate number is not read is assumed, but the gift certificate number may be read by the information processing device 10 as shown in a first modified example described later.

[0067] The gift certificate GT with an unknown denomination name shown in Fig. 4 (gift certificate GT with a reading order of "4") is a gift certificate GT that has been determined not to correspond to any of the multiple denominations registered in the denomination table TBL. In this case, a denomination corresponding to the unknown gift certificate GT may be newly registered in the denomination table TBL. New registration of a denomination may be initiated by, for example, pressing a "New Registration" button BT or the like, in accordance with the flow shown in Fig. 3 (for example, a series of processes in steps S54 and S55 in Fig. 3).

[0068] A user identifying a gift certificate GT can easily check whether the gift certificate GT to be identified has been correctly identified by checking the confirmation screen SC. For example, the user may select a correctly classified gift certificate GT on the confirmation screen SC and confirm the classification of the selected gift certificate GT by pressing the "Confirm" button BT. The classification of the gift certificate GT may be confirmed without pressing the "Confirm" button BT. The user may also select an incorrectly identified gift certificate GT on the confirmation screen SC and cancel the identification (classification) of the selected gift certificate GT by pressing the "Delete" button BT.

[0069] The confirmation screen SC is not limited to the example shown in Fig. 4. For example, buttons BT such as "Delete", "Confirm", and "New Registration" do not have to be displayed on the confirmation screen SC. Also, an operation screen for editing the note type table TBL, such as registering a new note type to the note type table TBL and deleting a note type from the note type table TBL, may be displayed on the display device 30 separately from the confirmation screen SC.

[0070] Next, an overview of the operation of the information processing device 10 will be described with reference to FIGS.

[0071] Fig. 5 is a flow chart showing an example of the operation of the information processing device 10 shown in Fig. 1. The operation shown in Fig. 5 is executed when identifying the gift certificates GT and counting them by type.

[0072] First, in step S100, the processing device 120 functions as the image information acquisition unit 122 and acquires from the image input device 20 first side image information and second side image information indicating images of the first and second sides, respectively, of the gift certificate GT to be recognized.

[0073] Next, in step S200, the processing device 120 functions as the front side identification unit 123 and executes a front side identification process to identify which of the first side and the second side is the front side of the gift certificate GT. Details will be described in Fig. 6, but for example, the front side identification unit 123 identifies the image of the front side of the gift certificate GT based on the number of contours in the image of the first side and the number of contours in the image of the second side.

[0074] Next, in step S300, the processing device 120 functions as the feature extraction unit 124 and calculates a feature vector indicating the feature amount of the image (first image) of the front side of the gift certificate GT identified in step S200. That is, the feature vector of the first image is calculated by the processing of step S300.

[0075] Then, in step S400, the processing device 120 functions as the distance calculation unit 125 and calculates an inter-vector distance, which is the distance between the feature vector calculated in step S300 (the feature vector of the first image) and a reference feature vector of a reference gift certificate GT registered in advance in the certificate type table TBL. The reference feature vector is, for example, the feature vector of the second image, which is an image of the reference gift certificate GT (the feature vector of the certificate type registered in the certificate type table TBL). For example, when multiple certificate types are registered in the certificate type table TBL, the distance calculation unit 125 calculates the inter-vector distance between the feature vector of each of the multiple certificate types (i.e., the feature vector of each of the multiple second images) and the feature vector of the first image. Therefore, when multiple certificate types are registered in the certificate type table TBL, multiple inter-vector distances corresponding to the multiple certificate types are calculated.

[0076] Next, in step S500, the processing device 120 functions as the identification unit 126 and searches for a reference feature vector whose inter-vector distance is equal to or less than the first distance. For example, the processing device 120 determines whether the inter-vector distance calculated in step S400 is equal to or less than the first distance. Note that, for example, when a plurality of inter-vector distances are calculated in step S400, the processing device 120 determines whether the inter-vector distance is equal to or less than the first distance for each of the plurality of inter-vector distances calculated in step S400.

[0077] Then, in step S520, the processing device 120 functions as the discrimination unit 126 and determines whether or not there is a reference feature vector whose inter-vector distance is equal to or less than the first distance.

[0078] If the result of the determination in step S520 is positive, the processing device 120 proceeds to step S600. On the other hand, if the result of the determination in step S520 is negative, it is determined that the gift certificate does not correspond to any of the multiple types of certificates registered in the certificate type table TBL, such as the gift certificate GT in the reading order "4" shown in Figure 4, and is registered as "unknown."

[0079] If the result of the determination in step S520 is negative, the processing device 120 can proceed to step S700 in Fig. 5. If the result of the determination in step S520 is negative and the processing device 120 does not execute the processing in step S700, the processing device 120 can end the operation shown in Fig. 5.

[0080] In step S600, the processing device 120 functions as the identification unit 126 and identifies the type of the gift certificate GT to be identified. For example, the identification unit 126 identifies the type of the gift certificate GT corresponding to the reference feature vector whose inter-vector distance is equal to or less than the first distance as the type of the gift certificate GT to be identified. If there are multiple reference feature vectors whose inter-vector distance is equal to or less than the first distance, the identification unit 126 may identify the type of the gift certificate GT corresponding to the reference feature vector whose inter-vector distance is the smallest as the type of the gift certificate GT to be identified. Alternatively, the processing device 120 may function as the display control unit 128 and display multiple images of the gift certificate GT corresponding to multiple reference feature vectors whose inter-vector distance is equal to or less than the first distance and the image of the gift certificate GT to be identified on the confirmation screen SC to allow the user to select one. Then, the identification unit 126 may identify the type of the gift certificate GT selected by the user via the confirmation screen SC as the type of the gift certificate GT to be identified.

[0081] In this manner, the gift certificate GT to be identified is classified into the type specified in step S600.

[0082] Next, in step S620, the processing device 120 functions as the display control unit 128, and causes the display device 30 to display an image of the gift certificate GT to be identified, etc. For example, the display control unit 128 causes the display device 30 to display a confirmation screen SC. As a result, for example, an image of a reference gift certificate GT (e.g., reference image RIMG in FIG. 4) whose inter-vector distance is equal to or less than the first distance, and an image of the gift certificate GT to be identified (e.g., images IIMGf and IIMGb of gift certificate GT in FIG. 4) are displayed on the display device 30.

[0083] Also, in step S700, the processing device 120 functions as the recognition unit 126, and registers the image and feature vector of the gift certificate GT to be recognized in the certificate type table TBL in association with each other according to the flow shown in Fig. 3 (for example, a series of processes in steps S54 and S55 in Fig. 3). As a result, for example, first image information indicating a first image, which is the image of the gift certificate GT to be recognized, and feature information indicating the feature vector of the first image are stored in the storage device 140 in association with each other.

[0084] For example, the user may register the denomination information of the gift certificate GT, the type of which has not been specified, in the denomination table TBL via the confirmation screen SC shown in Fig. 4. Note that the identification information included in the denomination information may be automatically classified by the processing device 120 (e.g., the identification unit 126).

[0085] Also, for example, when the process of step S700 is executed, the type of the gift certificate GT to be identified is not specified and is unknown. For this reason, the processing device 120 may notify the user that the type of the gift certificate GT to be identified is unknown. In this case, the user may cause the image input device 20 to read the gift certificate GT to be identified again, thereby causing the information processing device 10 to execute the operation shown in FIG. 5 again. For example, depending on the accuracy of the reading of the gift certificate GT by the image input device 20, even if the type of the gift certificate GT has not been specified, the type of the gift certificate GT may be specified by using the image read again by the image input device 20.

[0086] When the process of step S700 is executed, the operation shown in Fig. 5 is terminated by ending step S700. When the process of steps S600 and S620 is executed, the operation shown in Fig. 5 is terminated by ending step S620.

[0087] Thus, in this embodiment, when a second image in which the inter-vector distance is equal to or less than the first distance is identified, the recognition unit 126 determines that the type of the gift certificate GT to be recognized is the same as the type of the reference gift certificate GT indicated by the second image in which the inter-vector distance is equal to or less than the first distance. Then, the display control unit 128 causes the display device 30 to display the second image in which the inter-vector distance is equal to or less than the first distance and the first image. Furthermore, when a second image in which the inter-vector distance is equal to or less than the first distance is not identified, the recognition unit 126 stores in the storage device 140 the feature information indicating the feature vector of the first image and the first image information in association with each other.

[0088] Next, the surface identification process of step S200 will be described with reference to FIG.

[0089] Fig. 6 is a flowchart showing an example of the surface identification process shown in Fig. 5. For example, the processing device 120 functioning as the surface identification unit 123 executes a series of processes from step S220 to step S280 shown in Fig. 6 as the surface identification process of step S200 shown in Fig. 5. Therefore, the process of step S220 is executed after the process of step S100 shown in Fig. 5 is executed. In addition, after the process of step S280 is executed, the process of step S300 shown in Fig. 5 is executed.

[0090] First, in step S220, the surface specifying unit 123 calculates the number of contours (hereinafter also referred to as the number of contours N1) included in the image of the first surface (the image indicated by the image information of the first surface).

[0091] Next, in step S222, surface specifying unit 123 calculates the number of contours (hereinafter also referred to as contour number N2) included in the image of the second surface (the image indicated by the second surface image information).

[0092] Then, in step S240, the surface specifying unit 123 determines whether the number of contours N1 is equal to or smaller than the number of contours N2.

[0093] If the result of the determination in step S240 is positive, in step S260, the face identification unit 123 identifies the first face, which has the fewer number of contours, as the face of the gift certificate GT, and proceeds to step S280.

[0094] On the other hand, if the result of the determination in step S240 is negative, in step S262, the face identification unit 123 identifies the second face, which has the fewer number of contours, as the face of the gift certificate GT, and proceeds to the process in step S280.

[0095] In step S280, the front surface identification unit 123 identifies the image of the front surface of the gift certificate GT as an image (first image) for which a feature vector is calculated.

[0096] In a typical gift certificate GT, the number of outlines on the front side is smaller than that on the back side, since the stores that sell the gift certificate GT and notes, etc. are written on the back side. Therefore, in this embodiment, as described above, of the first and second sides, the side with the fewer outlines is identified as the front side of the gift certificate GT.

[0097] For example, a method may be adopted in which the number of characters on the first side and the number of characters on the second side are calculated by character recognition processing that recognizes characters contained in an image using optical character recognition (OCR) technology. In this case, of the first and second sides, the side with the fewer number of characters is identified as the front side of the gift certificate GT. However, the process of calculating the number of characters contained in an image by character recognition processing tends to take more time than the process of calculating the number of contours contained in an image. In other words, the time required for the process of calculating the number of contours contained in an image is shorter than the time required for the process of calculating the number of characters contained in an image by character recognition processing. For this reason, in this embodiment, the time required for the front side identification process to identify the front side of the gift certificate GT can be shortened compared to the mode in which the number of characters contained in the images of each of the first and second sides is calculated by character recognition processing.

[0098] The operation of the information processing device 10 is not limited to the examples shown in Figs. 3, 5, and 6. For example, if the front and back sides are specified when the image input device 20 reads the gift certificate GT, the front side identification process of step S52 may be omitted from the operation shown in Fig. 3, and the front side identification process of step S200 may be omitted from the operation shown in Fig. 5. For example, in a mode in which information (e.g., gift certificate number, etc.) written on the back side of the gift certificate GT is not referenced, the image information acquisition unit 122 does not need to acquire image information showing an image of the back side of the gift certificate GT to be identified. For example, the image information acquisition unit 122 acquires image information showing an image of the front side of the gift certificate GT to be identified as first image information from the image input device 20. In this case, too, the front side identification process of step S52 may be omitted from the operation shown in Fig. 3, and the front side identification process of step S200 may be omitted from the operation shown in Fig. 5. For example, the process of step S222 in Fig. 6 may be executed before the process of step S220, or may be executed in parallel with the process of step S220.

[0099] 5 shows the operation of the information processing device 10 when identifying and classifying one gift certificate GT to be identified, in order to make the operation of the information processing device 10 easier to understand. Therefore, when multiple gift certificates GT are to be classified, the operation shown in FIG. 5 is repeatedly executed. Note that, when multiple gift certificates GT are to be classified, after all image information of the multiple gift certificates GT to be classified is obtained, the series of processes from steps S200 to S620 may be repeatedly executed.

[0100] Also, for example, the operation of registering the type information of the gift certificate GT, whose type was not specified in the operation shown in Fig. 5, in the type table TBL may be performed each time the processing of step S700 is performed. Alternatively, the operation of registering the type information of the gift certificate GT, whose type was not specified in the operation shown in Fig. 5, in the type table TBL may be performed after the series of processing from step S100 to step S700 has been performed for all of the multiple gift certificates GT.

[0101] Here, in this embodiment, as described above, a model that has undergone distance learning specifically for gift certificates is used to identify gift certificates GT. In a general method for identifying gift certificates GT (hereinafter also referred to as the comparative example), for example, an image recognition model that has been trained to identify gift certificates GT by type is used. In the comparative example using an image recognition model, in order to create an image recognition model, it is necessary to perform learning for identifying the type of gift certificate GT for each type using a large amount of training data. Also, in the comparative example, when training the image recognition model on a new type of certificate that has not been used to create the image recognition model, a large amount of training data for the new type of certificate is required. For this reason, in the comparative example, the time and effort required for learning increases.

[0102] In contrast, in the metric learning dedicated to gift certificates in this embodiment, when a new denomination is added to the denomination table TBL, learning is completed with one image. Therefore, in this embodiment, the user can easily add a new denomination (add a reference denomination). In other words, since a model generated by metric learning dedicated to gift certificates is used, there is no need to use a large amount of training data for each denomination for learning. Therefore, as shown in the flowchart of FIG. 3, when registering a new gift certificate GT, registration is completed simply by reading one reference gift certificate GT, thereby reducing the time and effort required for learning. In other words, in this embodiment, it is possible to reduce the effort required for generating information required for identifying the gift certificate GT.

[0103] In addition, in this embodiment, a lightweight model is used that generates a feature vector indicating the feature amount of an image and calculates the vector distance, which is the distance from a reference feature vector registered in advance, compared to the image recognition model. Therefore, when identifying a new gift certificate GT to be identified as shown in the flowchart of Fig. 5, it is possible to process it faster than before. In addition, since a general personal computer can be adopted as the information processing device 10, it is possible to realize it at low cost.

[0104] In addition, in this embodiment, a feature vector is extracted from the image of the entire gift certificate GT. Then, based on the distance between the two feature vectors (vector distance), it is determined whether the two gift certificates GT corresponding to the two feature vectors are the same type. Therefore, in this embodiment, it is possible to prevent a decrease in accuracy in identifying the type of gift certificate GT due to noise such as dirt, notes, and seal impressions on the gift certificate GT.

[0105] As described above, in this embodiment, the information processing device 10 includes an image information acquisition unit 122 that acquires first image information indicating a first image, which is an image of the gift certificate GT to be identified, a feature extraction unit 124 that generates a feature vector of the first image by executing a process on the first image to generate a feature vector indicating the feature amounts of the image, a distance calculation unit 125 that calculates the inter-vector distance, which is the distance between the feature vector of a second image, which is an image of each of a plurality of reference gift certificates GTs registered in advance, and the feature vector of the first image, and a recognition unit 126 that identifies the gift certificate GT to be identified based on the inter-vector distance.

[0106] In this manner, in this embodiment, the gift certificate GT to be identified is identified and classified based on the inter-vector distance, which is the distance between the feature vector of the first image, which is an image of the gift certificate GT to be identified, and the feature vector of the second image, which is an image of the reference gift certificate GT. For this reason, in this embodiment, it is not necessary to perform learning for identifying the type of gift certificate GT for each type of certificate using a large amount of training data. As a result, in this embodiment, it is possible to reduce the time and effort required for learning. In other words, in this embodiment, it is possible to reduce the effort required for generating information (e.g., a learned model) required for identifying the gift certificate GT.

[0107] Furthermore, in this embodiment, the feature vector of each of the second images of the multiple reference gift certificates GT is calculated from the second image of one gift certificate GT in each reference by using a model trained by distance learning in the feature extraction unit 124. This allows the user to easily add a reference gift certificate GT in this embodiment.

[0108] Furthermore, in this embodiment, when a second image in which the inter-vector distance is equal to or less than the first distance is identified, the identification unit 126 determines that the type of the gift certificate GT to be identified is the same as the type of the reference gift certificate GT indicated by the second image in which the inter-vector distance is equal to or less than the first distance. Thus, in this embodiment, the type of the reference gift certificate GT indicated by the second image in which the inter-vector distance is equal to or less than the first distance is identified as the type of the gift certificate GT to be identified. As a result, in this embodiment, the gift certificate GT to be identified can be easily identified and classified by type.

[0109] Furthermore, in this embodiment, the information processing device 10 further includes a display control unit 128 that, when a second image in which the inter-vector distance is equal to or less than the first distance is identified, causes the display device 30 to display the second image in which the inter-vector distance is equal to or less than the first distance and the first image. In this manner, in this embodiment, an image (first image) of the gift certificate GT to be identified and an image (second image) of a reference gift certificate GT of the same type as the gift certificate GT to be identified are displayed on the display device 30. This allows the user to easily check whether the gift certificate GT to be identified has been correctly sorted, etc. As a result, in this embodiment, it is possible to prevent the sorting work of the gift certificate GT from being ended in a state in which the gift certificate GT to be identified has been incorrectly sorted.

[0110] Furthermore, in this embodiment, when a second image in which the inter-vector distance is equal to or less than the first distance is not identified, the recognition unit 126 associates feature information indicating the feature vector of the first image with the first image information, and stores them in the storage device 140. In this case, for example, the first image information and feature information stored in the storage device 140 are associated with each other, and the type information indicating the type of gift certificate GT indicated by the first image information is stored in the storage device 140, thereby newly registering a reference gift certificate GT. In this way, in this embodiment, when a second image in which the inter-vector distance is equal to or less than the first distance is not identified, the gift certificate GT to be recognized can be easily registered as a reference gift certificate GT.

[0111] Moreover, in this embodiment, the information processing device 10 further includes a front side identification unit 123. The image information acquisition unit 122 acquires first side image information showing an image of the first side of the gift certificate GT to be identified, and second side image information showing an image of the second side opposite to the first side of the gift certificate GT to be identified. The front side identification unit 123 identifies the front and back sides of the gift certificate GT to be identified based on the number of contours included in the image of the first side shown by the first side image information and the number of contours included in the image of the second side shown by the second side image information, and identifies the image information showing the image of the front side from the first side image information and the second side image information as first image information.

[0112] In this manner, in this embodiment, the front and back of the gift certificate GT to be identified are identified based on the number of contours included in the images of the first and second sides of the gift certificate GT to be identified. As a result, in this embodiment, it is possible to prevent the feature vector of the first image used to calculate the inter-vector distance from being extracted from the image of the back side of the gift certificate GT. As a result, in this embodiment, the type of the gift certificate GT to be identified can be efficiently identified. Also, for example, the user can have the image input device 20 read the gift certificate GT without being aware of the front and back sides of the gift certificate GT. Therefore, in this embodiment, it is possible to improve usability when classifying the gift certificate GT to be identified by the information processing device 10. Also, in this embodiment, since the front side of the gift certificate GT is identified based on the number of contours included in the image, it is possible to reduce the time required for the front side identification process to identify the front side of the gift certificate GT compared to the case where the front side of the gift certificate GT is identified based on the number of characters included in the image.

[0113] [2. Modifications] The present invention is not limited to the above-described embodiments. Specific modified embodiments are exemplified below. Two or more embodiments selected from the following examples may be combined.

[0114] [First Modification] In the above-described embodiment, the information processing device 10 may read, from the gift certificate GT to be identified, a gift certificate number that is used by the card company to manage the gift certificate GT.

[0115] Fig. 7 is an explanatory diagram for explaining an overview of an information processing system 1 according to a first modified example. Elements similar to those explained in Fig. 1 to Fig. 6 are given the same reference numerals and detailed explanations are omitted.

[0116] The information processing system 1 shown in FIG. 7 has the same configuration as the information processing system 1 shown in FIG. 1. For example, the information processing system 1 has an information processing device 10 including a processing device 120, a storage device 140, and a communication device 160, an image input device 20, and a display device 30. However, a control program PG2 stored in the storage device 140 of the information processing device 10 is different from the control program PG1 shown in FIG. 1. The control program PG2 is an example of a "program". In this embodiment, the control program PG2 includes application programs for causing the processing device 120 to function as the image information acquisition unit 122, the surface specification unit 123, the feature extraction unit 124, the distance calculation unit 125, the identification unit 126, the character recognition unit 127, and the display control unit 128. The control program PG2 may include, for example, an operating system program for the processing device 120 to control each unit of the information processing device 10. The control program PG2 may be transmitted from another device via a network.

[0117] The processing device 120 shown in Fig. 7 is similar to the processing device 120 shown in Fig. 1, except that the processing device 120 executes a control program PG2 instead of the control program PG1 shown in Fig. 1. For example, the processing device 120 executes the control program PG2 stored in the storage device 140 to function as an image information acquisition unit 122, a surface specification unit 123, a feature extraction unit 124, a distance calculation unit 125, an identification unit 126, a character recognition unit 127, and a display control unit 128. That is, this modification differs from the above-mentioned embodiment in that the processing device 120 functions as the character recognition unit 127.

[0118] The image information acquisition unit 122, the surface identification unit 123, the feature extraction unit 124, the distance calculation unit 125, the identification unit 126, and the display control unit 128 are the same as those in the above-described embodiment. The following description will focus on the character recognition unit 127, which is different from the above-described embodiment.

[0119] The character recognition unit 127 recognizes the gift certificate number written in the number area NAR in the image IIMGb by performing character recognition processing on the number area NAR in the image IIMGb of the back side of the gift certificate GT identified by the recognition unit 126. The number area NAR in which the gift certificate number is written varies depending on the card company or type of gift certificate, as described in FIG. 4. Therefore, the character recognition unit 127 specifies the position of the number area NAR based on the type of gift certificate GT from which the gift certificate number is to be read. Then, the character recognition unit 127 performs character recognition processing on the number area NAR in the image IIMGb of the back side of the gift certificate GT.

[0120] Character recognition processing is performed on the number area NAR, thereby recognizing the gift certificate number of the gift certificate GT to be identified. For example, the character recognition unit 127 generates character information indicating the gift certificate number recognized by character recognition processing on the number area NAR in the image IIMGb on the back side of the gift certificate GT. Note that the character recognition unit 127 may generate character information by, for example, converting a character image indicating the gift certificate number written in the number area NAR into a character code by character recognition processing. Furthermore, when the image input device 20 stores the first image information in a shared memory (not shown), the character recognition unit 127 may store the character information indicating the gift certificate number recognized by character recognition processing in the shared memory in association with the first image information.

[0121] The character recognition process by the character recognition unit 127 to recognize the gift certificate number of the gift certificate GT to be identified is executed, for example, after the process of step S600 shown in Fig. 5 is executed. Therefore, in this modified example, if the type of the gift certificate GT to be identified is unknown, the character recognition process to recognize the gift certificate number of the gift certificate GT is not executed. As a result, in this modified example, the character recognition process to recognize the gift certificate number of the gift certificate GT can be executed efficiently.

[0122] Furthermore, in step S620 shown in FIG. 5, the display control unit 128 may cause the display device 30 to display the gift certificate number recognized by the character recognition unit 127 in addition to the image of the gift certificate GT to be identified.

[0123] As described above, the present modified example can also provide the same effects as those of the above-mentioned embodiment. Furthermore, in the present modified example, since the gift certificate number is read from the gift certificate GT whose type has been specified, the read gift certificate GT can be specified as a used gift certificate GT and processed (deleted), and tasks such as the management of the gift certificate GT can be efficiently performed.

[0124] [Second modified example] In the above-described embodiment and modified example, a plurality of images of gift certificates GT may be associated with one type of certificate information. In this case, for example, a plurality of feature vectors extracted from a plurality of images of gift certificates GT are associated with one type of certificate information. In this modified example, even if a plurality of designs of gift certificates GT exist for the same type of gift certificate GT due to a change in design, for example, it is possible to identify the gift certificates GT by type while preventing the management of the reference gift certificates GT from becoming complicated.

[0125] As described above, in this modified example as well, the same effects as those of the second embodiment described above can be obtained.

[0126] [Third Modification] In the above-described embodiment and modified example, the information processing system 1 is a securities processing system that identifies securities such as gift certificates GT by type, but the present invention is not limited to such an embodiment. For example, the information processing system 1 may be a system that identifies non-securities other than securities (e.g., service vouchers, coupons, discount vouchers, etc.). As described above, the same effects as those of the above-described embodiment and modified example can be obtained in this modified example as well.

[0127] [Fourth Modification] In the above-described embodiment and modified example, the information processing system 1 is a securities processing system that identifies and classifies securities such as gift certificates GT by type, but the present invention is not limited to such an embodiment. For example, the information processing system 1 may be incorporated into an existing system such as a POS register. In this case, an existing application may call APIs (Application Programming Interfaces) related to the processing of each of the image information acquisition unit 122, the surface identification unit 123, the feature extraction unit 124, the distance calculation unit 125, the identification unit 126, and the display control unit 128, thereby executing the processing of identifying gift certificates GT and classifying them by type. [Explanation of symbols]

[0128] 1...information processing system, 10...information processing device, 20...image input device, 30...display device, 120...processing device, 122...image information acquisition unit, 123...surface identification unit, 124...feature extraction unit, 125...distance calculation unit, 126...recognition unit, 127...character recognition unit, 128...display control unit, 140...storage device, 160...communication device, GT...gift certificate.

Claims

1. an acquisition unit that acquires first image information indicating a first image which is an image of the security to be identified; a generation unit that generates a feature vector of the first image by executing a process of generating the feature vector indicating a feature amount of the image on the first image; a calculation unit that calculates an inter-vector distance between the feature vector of a second image, which is an image of each of a plurality of reference securities registered in advance, and the feature vector of the first image; an identification unit that identifies the securities to be identified based on the inter-vector distance; An information processing device comprising:

2. The information processing device described in claim 1, characterized in that the feature vector of the second image of each of the multiple reference securities is calculated from the second image of one security in each reference by using a model trained by distance learning in the generation unit.

3. The identification unit is when the second image in which the inter-vector distance is equal to or less than the first distance is identified, it is determined that the type of the security to be identified is the same as the type of the reference security indicated by the second image in which the inter-vector distance is equal to or less than the first distance; 3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

4. a display control unit that, when the second image in which the inter-vector distance is equal to or less than the first distance is identified, causes a display device to display the second image in which the inter-vector distance is equal to or less than the first distance and the first image.

4. The information processing apparatus according to claim 3.

5. The identification unit is if the second image in which the inter-vector distance is equal to or less than the first distance is not specified, feature information indicating the feature vector of the first image and the first image information are stored in a storage device in association with each other.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. Further comprising a surface specification part, The acquisition unit is Acquire first side image information showing an image of a first side of the security to be identified, and second side image information showing an image of a second side of the security to be identified that is opposite to the first side; The surface specifying unit is identifying the front and back of the security to be identified based on the number of contours included in the image of the first side represented by the first side image information and the number of contours included in the image of the second side represented by the second side image information; Identifying image information showing an image of the front side from among the first side image information and the second side image information as the first image information.

6. The information processing apparatus according to claim 5,

7. obtaining first image information indicating a first image which is an image of a security to be identified; generating a feature vector of the first image by executing a process for generating a feature vector indicating a feature amount of an image on the first image; Calculating an inter-vector distance between the feature vector of a second image, which is an image of a reference security, and the feature vector of the first image; identifying the securities to be identified based on the vector distances; 23. An information processing method comprising:

8. Obtaining first image information indicating a first image which is an image of a security to be registered; generating a feature vector of the first image by executing a process for generating a feature vector indicating a feature amount of an image on the first image; registering the first image information, the generated feature vector, and the name and face value of the security to be registered in association with each other; 23. An information processing method comprising:

9. The processor, an acquisition unit that acquires first image information indicating a first image which is an image of the security to be identified; a generation unit that generates a feature vector of the first image by executing a process of generating a feature vector indicating a feature amount of an image on the first image; a calculation unit that calculates an inter-vector distance between the feature vector of a second image, which is an image of a reference security, and the feature vector of the first image; an identification unit that identifies the securities to be identified based on the inter-vector distance; To function as program.

Images