Object management device

Abstract

This object management device comprises: an acquisition means for acquiring a registered image obtained by imaging a surface of an object to be registered, on which a handwritten character is written; a character recognition means for outputting a registered character recognition result obtained by character-recognizing the handwritten character in the registered image; and a registration means for generating, as data for determining the identity of the object to be registered, registration data in which the registered character recognition result and the registered image, or registered object information which is a feature amount extracted from the registered image, and are associated with each other.

Description

object management device

[0001] The present disclosure relates to an object management device, an object management method, an object management system, and a recording medium.

[0002] A device has been proposed that acquires an image of an object, uses that image as a match image, performs image matching with each of multiple registered images that have been registered in advance, and detects the registered image that matches the match image (for example, Patent Document 1).

[0003] WO2020 / 255394

[0004] However, in order to perform image matching between a match image and each of the registered images that have been registered in advance and to detect a registered image that matches the match image, many image matches are required, which requires a lot of calculation time.

[0005] An object of the present disclosure is to provide an object management device that solves the problem that a large number of image matches are required to detect a registered image that matches a match image, which takes a long time for calculation.

[0006] The object management device of the present disclosure is configured to include: a first acquisition means for acquiring a registration image obtained by photographing the surface of an object to be registered on which handwritten characters are written; a first character recognition means for outputting a registration character recognition result obtained by character recognition of the handwritten characters in the registration image; and a registration means for generating registration data including registration object information, which is the registration image or features extracted from the registration image, and the registration character recognition result, as data for determining the identity of the object to be registered.

[0007] Another object management device in the present disclosure is configured to include: a first acquisition means for acquiring a match image obtained by photographing the surface of an object to be matched, the surface having handwritten characters written thereon; a first character recognition means for outputting a match character recognition result obtained by character recognizing the handwritten characters in the match image; an extraction means for extracting registration data including a registration character recognition result similar to the match character recognition result from a plurality of stored registration data including a registration image relating to the object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result; and a determination means for determining the identity of the object to be matched and the object to be registered by a matching process using the match image or the matching object information which is a feature extracted from the match image, and the registration object information in the extracted registration data.

[0008] Furthermore, the object management method of the present disclosure is configured so that a computer acquires a registration image obtained by photographing the surface of an object to be registered on which handwritten characters are written, outputs a registration character recognition result obtained by character recognition of the handwritten characters in the registration image, and generates registration data including the registration image or registration object information, which is a feature extracted from the registration image, and the registration character recognition result as data for determining the identity of the object to be registered.

[0009] Furthermore, another object management method in the present disclosure is configured so that a computer acquires a match image obtained by photographing the surface of an object to be matched, the surface having handwritten characters written thereon; outputs a match character recognition result obtained by character recognition of the handwritten characters in the match image; extracts registration data including a registration character recognition result similar to the match character recognition result from a plurality of stored registration data including a registration image relating to the object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result; and determines the identity of the object to be matched and the object to be registered by a matching process using the match image or the matching object information which is the feature extracted from the match image and the registration object information in the extracted registration data.

[0010] Furthermore, the computer-readable recording medium of the present disclosure is configured to record a program for causing a computer to perform the following processes: acquiring a registration image obtained by photographing the surface of an object to be registered on which handwritten characters are written; outputting a registration character recognition result obtained by character recognition of the handwritten characters in the registration image; and generating registration data including the registration image or registration object information, which is a feature extracted from the registration image, and the registration character recognition result as data for determining the identity of the object to be registered.

[0011] Furthermore, another computer-readable recording medium in the present disclosure is configured to record a program for causing a computer to perform the following processes: acquiring a match image obtained by photographing the surface of an object to be matched, on which handwritten characters are written; outputting a match character recognition result obtained by character-recognizing the handwritten characters in the match image; extracting registration data including a registration character recognition result similar to the match character recognition result from a plurality of stored registration data including a registration image relating to the object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result; and determining the identity of the object to be matched and the object to be registered by a matching process using the match image or the matching object information which is a feature extracted from the match image, and the registration object information in the extracted registration data.

[0012] In addition, the object management system in the present disclosure is configured to include: an acquisition means for acquiring a registration image obtained by photographing the surface of an object to be registered on which handwritten characters are written; a character recognition means for outputting a registration character recognition result obtained by character recognizing the handwritten characters in the registration image; and a registration means for generating registration data that associates the registration image or registration object information, which is a feature extracted from the registration image, with the registration character recognition result as data for determining the identity of the object to be registered.

[0013] Another object management system in the present disclosure is configured to include: an acquisition means for acquiring a match image obtained by capturing a surface of an object to be matched that has handwritten characters written on it; a character recognition means for outputting a match character recognition result obtained by character recognizing the handwritten characters in the match image; an extraction means for extracting registration data including a registration character recognition result similar to the match character recognition result from a plurality of stored registration data including a registration image related to the object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result; and a determination means for determining the identity of the object to be matched and the object to be registered by a matching process using the match image or the matching object information which is a feature extracted from the match image, and the registration object information in the extracted registration data.

[0014] By having the configuration described above, the present disclosure can reduce the number of registered images that must be compared with a match image, and as a result, can shorten the calculation time required to detect registered images that match the match image.

[0015] 1 is a block diagram of a first object management system according to the present disclosure. FIG. 2 is a block diagram showing an example of the configuration of a first mobile terminal according to the present disclosure. FIG. 3 is a block diagram showing an example of the configuration of a first server according to the present disclosure. FIG. 4 is a diagram showing an example of the contents of a registration data DB of the first server according to the present disclosure. FIG. 5 is a flowchart showing an example of the procedure for a registration operation of the first object management system according to the present disclosure. FIG. 6 is a diagram showing an example of a registration image including handwritten characters in the first object management system according to the present disclosure. FIG. 7 is a flowchart showing an example of the procedure for a matching operation of the first object management system according to the present disclosure. FIG. 8 is a block diagram showing an example of the configuration of a second mobile terminal according to the present disclosure. FIG. 9 is a block diagram showing an example of the configuration of a second server according to the present disclosure. FIG. 10 is a flowchart showing an example of the procedure for a registration operation of the second object management system according to the present disclosure. FIG. 11 is a block diagram of a first object management device according to the present disclosure. FIG. 12 is a block diagram of a second object management device according to the present disclosure. FIG. 13 is a block diagram of a third object management system according to the present disclosure. FIG. 14 is a block diagram of a fourth object management system according to the present disclosure.

[0016] First Embodiment First, an overview of an object management system according to a first embodiment of the present disclosure will be described.

[0017] Handwritten characters will appear visually different if the written string is different, and even the same string of characters can appear visually different if the writer or writing implement is different. Therefore, an image obtained by photographing the surface of an object on which handwritten characters are written can serve as an identifier for identifying the individual object. Furthermore, if the surface of an object on which handwritten characters are written has surface features such as patterns or unevenness due to the material, the surface features are not necessarily the same for each object, and the relative positional relationship between the surface features and the handwritten characters is not necessarily the same for each object, so the image of the surface of the object on which handwritten characters are written can become even more different for each individual object. However, if only the image is used as an individual identifier, a large number of image matches would be required to detect a registered image that matches the match image.

[0018] Therefore, the object management system of this embodiment uses a combination of an image obtained by photographing the surface of an object on which handwritten characters are written and the results of character recognition of the handwritten characters as data for determining the identity of the object.

[0019] During registration, the object management system generates registration data in a registration data DB (database) for each object to be registered, the registration data including a registration image obtained by photographing the surface of the object to be registered on which handwritten characters are written, or registration object information which is feature values ​​extracted from the registration image, and a registration character recognition result obtained by character recognition of the handwritten characters in the registration image.

[0020] During matching, the object management system matches matching data including a matching image obtained by photographing the surface of the object to be matched on which handwritten characters are written, or matching object information which is feature information extracted from the matching image, and a matching character recognition result obtained by character recognition of the handwritten characters in the matching image, with multiple registration data stored in a registration data DB. In this case, the object management system first extracts registration data including a registration character recognition result similar to the matching character recognition result in the matching data from the multiple registration data stored in the registration data DB. Next, the object management system determines the identity of the object to be matched and the object to be registered by a matching process using the matching object information in the matching data and the registered object information in the extracted registration data.

[0021] By configuring and operating the object management system as described above, the number of registered images that must be matched with the match image can be reduced, thereby shortening the calculation time required to find a registered image that matches the match image.

[0022] The above is an overview of the object management system according to this embodiment. Details of the object management system according to this embodiment will be described below.

[0023] Referring to FIG. 1, an object management system 1 according to a first embodiment of the present disclosure is a system for managing individual objects 5 such as products, merchandise, or parcels for purposes such as origin indication, quality control, or distribution management.

[0024] The object management system 1 includes a mobile terminal 2 and a server 3. The mobile terminal 2 and the server 3 are connected to each other so that they can communicate with each other via a network 4 such as the Internet. Although only one mobile terminal 2 is shown in FIG. 1, multiple mobile terminals 2 may be present.

[0025] The mobile terminal 2 is a mobile terminal such as a smartphone, tablet terminal, or communication terminal used for registering and matching the managed object 5. The server 3 is an information processing device such as a personal computer that cooperates with the mobile terminal 2 and is used for registering and matching the managed object 5.

[0026] Referring to FIG. 2, the mobile terminal 2 is composed of a camera 21, a communication interface (I / F) unit 22, an operation input unit 23, a screen display unit 24, a memory unit 25, and an arithmetic processing unit 26.

[0027] The camera 21 is an imaging device that captures images of the object 5 to be managed. The camera 21 is, for example, a visible light and color camera equipped with a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor having a pixel capacity of several million pixels. The communication I / F unit 22 is composed of a data communication circuit and is configured to wirelessly communicate data with external devices such as the server 3. The operation input unit 23 is composed of devices such as a keyboard and a mouse and is configured to detect operator operations and output them to the arithmetic processing unit 26. The screen display unit 24 is composed of devices such as an LCD (Liquid Crystal Display) and is configured to display various information on the screen in response to instructions from the arithmetic processing unit 26.

[0028] The storage unit 25 is made up of a storage device such as a hard disk or memory, and is configured to store processing information and a program 251 required for various processes in the arithmetic processing unit 26. The program 251 is a program that is read into the arithmetic processing unit 26 and executed to realize various processing units, and is read in advance from an external device or recording medium (not shown) via a data input / output function such as the communication I / F unit 22 and stored in the storage unit 25. The main processing information stored in the storage unit 25 includes a registered image 252, auxiliary information 258, a character recognition result 253, registered data 254, a match image 255, a character recognition result 256, and a determination result 257.

[0029] The registered image 252 is an image generated from an image captured by the camera 21 of the surface of the object to be registered, on which handwritten characters are written. The attached information 258 is any information related to the object to be registered. The attached information 258 may be, for example, the management number of the object to be registered, the registration date and time, or information indicating the contents of the object. The character recognition result 253 is text resulting from character recognition of the handwritten characters in the registered image 252, and is also referred to as the registered character recognition result. The registered data 254 is data in which the registered object information, which is the registered image 252 or feature amounts extracted from the registered image, the character recognition result 253, and the attached information 258 are associated with each other. The registered data 254 is registered in a registered data DB 352 provided in the server 3 as data for determining the identity of the object to be registered.

[0030] Matching image 255 is an image generated from an image of the surface of a matching object on which handwritten characters are written, captured by camera 21. Character recognition result 256 is text resulting from character recognition of the handwritten characters in matching image 255, and is also called a matching character recognition result. Determination result 257 is data indicating whether the matching object identified by matching image 255 is identical to any of the registration target objects registered in registration data DB 352.

[0031] The arithmetic processing unit 26 has a processor such as a CPU (Central Processing Unit) and its peripheral circuits, and is configured to read and execute a program 251 from the storage unit 25, thereby realizing various processing functions through cooperation between the above hardware and the program 251. The main processing units realized by the arithmetic processing unit 26 are a registration means 261 and a matching means 262.

[0032] The registration means 261 is configured to generate registration data 254 for determining the identity of an object to be registered, and to request the server 3 to register the data. The registration means 261 is configured to include an acquisition unit 2611, a character recognition unit 2612, and a registration unit 2113.

[0033] The acquisition unit 2611 is configured to acquire an image obtained by photographing the surface of the object to be registered on which handwritten characters are written using the camera 21, cut out a partial image of a predetermined size including the handwritten characters from the acquired image, and store it in the storage unit 25 as a registered image 252. The acquisition unit 2611 is also configured to acquire auxiliary information 258 related to the object to be registered from the operator of the portable terminal 2 via the operation input unit 23 or the like, and store it in the storage unit 25.

[0034] The character recognition unit 2612 is configured to read the registered image 252 from the storage unit 25, perform character recognition on handwritten characters in this image, and store the character recognition result 253 in the storage unit 25. The character recognition unit 2612 may be realized by, for example, an OCR (Optical Character Reader) application program.

[0035] The registration unit 2113 is configured to read the registered image 252, the character recognition result 253, and the auxiliary information 258 from the storage unit 25. The registration unit 2113 is also configured to generate registered object information, which is the read registered image 252 or feature values ​​extracted from the registered image. The feature values ​​may be any feature values ​​that can be used to determine the identity of an individual. For example, the registration unit 2113 may extract the feature values ​​by sequentially performing a frequency transform, a polar coordinate transform or a log-polar coordinate transform, and a frequency transform on the registered image 252. An image obtained by performing a frequency transform such as a Fourier transform on an image is called a frequency spectrum image. Feature values ​​obtained by performing a polar coordinate transform or a log-polar coordinate transform on the frequency spectrum image are called Fourier-Mellin features. Feature values ​​obtained by performing a frequency transform such as a Fourier transform on the Fourier-Mellin features are called a Fourier-Mellin frequency spectrum image. This Fourier-Mellin frequency spectrum image is invariant to the magnification, rotation, and translation of the captured image. Hereinafter, the feature amount extracted from the registered image 252 will be referred to as a registered feature amount.

[0036] The registration unit 2113 is also configured to display the read character recognition result 253 on the screen display unit 24 and prompt the operator of the portable terminal 2 to confirm or correct it. Hereinafter, when simply referred to as the character recognition result 253, it means the character recognition result after confirmation or correction by the operator. Note that confirmation or correction of the character recognition result 253 by the operator is not essential and may be omitted. However, if the content of the character string to be handwritten is determined by a higher-level system rather than by the operator's judgment, it is desirable to correct it to the correct content.

[0037] The registration unit 2113 is also configured to create registration data 254 including the registration image 252 or registration object information, which is a registration feature extracted from the registration image, the character recognition result 253, and auxiliary information 258, and store the data in the storage unit 25. The registration unit 2113 is also configured to read out the registration data 254 from the storage unit 25, and transmit a registration request including the registration data 254 to the server 3 via the communication I / F unit 22.

[0038] The matching means 262 is configured to determine whether the object to be matched is the same as a registered object registered in the registration data DB 352 of the server 3. The matching means 262 is configured to include an acquisition unit 2621, a character recognition unit 2622, an extraction unit 2623, and a determination unit 2624.

[0039] The acquisition unit 2621 is configured to acquire an image obtained by photographing the surface of the object 5 to be matched, on which handwritten characters are written, with the camera 21, cut out a partial image of a predetermined size containing the handwritten characters from the image, and store it in the memory unit 25 as a match image 255.

[0040] The character recognition unit 2622 is configured to read the match image 255 from the storage unit 25 , perform character recognition on the handwritten characters in this match image, and store the character recognition result 256 in the storage unit 25 .

[0041] The extraction unit 2623 is configured to extract registration data including a registration character recognition result similar to the character recognition result 256 from among the multiple registration data registered in the registration data DB 352 of the server 3. For example, the extraction unit 2623 first transmits a registration data acquisition request to the server 3 via the communication I / F unit 22. In response to this acquisition request, the server 3 transmits all registration data stored in the registration data DB 352 to the portable device 2. The extraction unit 2623 extracts registration data including a registration character recognition result similar to the character recognition result 256 from among the registration data received by the portable device 2. For example, the extraction unit 2623 calculates the distance between the character recognition results (such as the Levenshtein distance or the Jaro-Winkler distance) and determines that the two are similar if the distance is within a threshold, and otherwise determines that the two are not similar. The extraction unit 2623 transmits the extracted registration data to the determination unit 2624.

[0042] The determination unit 2624 is configured to read the match image 255 from the storage unit 25 and generate match object information, which is the match image or features extracted from the match image. The extracted features are the same as the registered features. The features extracted from the match image 255 are referred to as match features. The determination unit 2624 is also configured to determine the identity of the object to be matched and the object to be registered by a matching process using the match object information and the registered object information included in the registered data transmitted from the extraction unit 2623.

[0043] When the matching object information and the registered object information are matching features and registered features, the determination unit 2624 determines the identity of the object to be matched and the object to be registered, for example, using the following method. First, the determination unit 2624 calculates a cross power spectrum between the matching features and each registered feature. Next, the determination unit 2624 performs an inverse frequency transform on each cross power spectrum to calculate a correlation value map. Next, the determination unit 2624 calculates a matching score representing the similarity between the matching image and each registered image from each correlation value map. Next, the determination unit 2624 determines the identity of the objects based on these matching scores. For example, if the maximum value of the matching scores is equal to or greater than a threshold, the determination unit 2624 determines that the object related to the matching image is identical to the object related to the registered data including the registered image with the maximum matching score. Furthermore, if the maximum value of the matching scores is not equal to or greater than a threshold, the determination unit 2624 determines that an object identical to the object related to the matching image is not registered.

[0044] Furthermore, when the matching object information and the registered object information are a matching image and a registered image, the determination unit 2624 extracts the above-described matching features and registered features from the matching image and the registered image, respectively, and then determines the identity of the object to be matched and the object to be registered by the same method as above. Alternatively, the determination unit 2624 determines the identity of the object to be matched and the object to be registered by template matching using a comparison of the shading between the matching image and the registered image.

[0045] Furthermore, when the determination unit 2624 determines that the object to be matched is identical to any of the registered object, it is configured to create a determination result 257 including auxiliary information included in the registration data of the registered object determined to be identical, and store the result in the storage unit 25. When the determination unit 2624 determines that the object to be matched is not identical to any of the registered object, it is configured to create a determination result 257 indicating that no identical registered object exists, and store the result in the storage unit 25. Furthermore, the determination unit 2624 is configured to read out the determination result 257 from the storage unit 25, display it on the screen of the screen display unit 24, and / or transmit it to an external terminal via the communication I / F unit 22.

[0046] Referring to FIG. 3, an example of the server 3 is composed of a camera 31, a communication I / F unit 32, an operation input unit 33, a screen display unit 34, a memory unit 35, and an arithmetic processing unit 36.

[0047] The camera 31 may be a camera with the same specifications as the camera 21 of the portable terminal 2. The communication I / F unit 32 is composed of a data communication circuit and is configured to perform data communication wirelessly with an external device such as the portable terminal 2. The operation input unit 33 is composed of devices such as a keyboard and a mouse and is configured to detect operations by the operator and output them to the arithmetic processing unit 36. The screen display unit 34 is composed of devices such as an LCD and is configured to display various information on the screen in response to instructions from the arithmetic processing unit 36.

[0048] The storage unit 35 is made up of storage devices such as a hard disk and memory, and is configured to store processing information and a program 351 required for various processes in the arithmetic processing unit 36. The program 351 is a program that is read into the arithmetic processing unit 36 ​​and executed to realize various processing units, and is read in advance from an external device or recording medium (not shown) via a data input / output function such as the communication I / F unit 32 and stored in the storage unit 35. The main processing information stored in the storage unit 35 is a registration data DB 352.

[0049] The registration data DB 352 is configured to store one or more registration data. FIG. 4 is a diagram showing an example of the contents of the registration data DB 352. The registration data DB 352 in this example is composed of multiple entries, with one registration data stored in each entry. An entry that stores registration data is composed of a registration number column, a registration object information column, a character recognition result column, and an additional information column. The registration object information column, character recognition result column, and additional information column of the same entry store registration object information (registered image or registered feature amount), character recognition result, and additional information related to a certain object to be registered. The registration number column stores a serial number or the like that uniquely identifies the registration data.

[0050] The arithmetic processing unit 36 ​​has a processor such as a CPU and its peripheral circuits, and is configured to read and execute a program 351 from the storage unit 35, thereby realizing various processing functions through cooperation between the above hardware and the program 351. The main processing unit realized by the arithmetic processing unit 36 ​​is a DB management means 361.

[0051] When the DB management means 361 receives a registration request from the mobile terminal 2 via the communication I / F unit 32, it reserves one free entry in the registration data DB 352 and stores the newly assigned registration number, the registered object information in the registration data included in the received registration request, the character recognition result, and the ancillary information in the registration number column, registered object information column, character recognition result column, and ancillary information column of the free entry.

[0052] In addition, when the DB management means 361 receives an acquisition request from the mobile terminal 2 through the communication I / F unit 32, it is configured to read out the registration data registered in each entry of the registration data DB 352 and transmit it to the mobile terminal 2 through the communication I / F unit 32.

[0053] Next, the operation of the object management system 1 according to this embodiment will be described. The operation of the object management system 1 is broadly divided into a registration operation and a matching operation. The registration operation is an operation of registering registration data related to an object to be registered in the registration data DB 352. The matching operation is an operation of matching the matching data of the object to be matched with the registration data of the object to be registered stored in the registration data DB 352.

[0054] First, the registration operation will be described. FIG. 5 is a flowchart showing an example of the registration operation procedure. First, the operator of the portable terminal 2 handwrites characters on any surface of the object 5 to be registered using any writing instrument (step S1). The type, number, and orientation of the handwritten characters are arbitrary. For example, the type of characters may be Japanese hiragana, katakana, kanji, or numbers, or may be a foreign language other than Japanese (e.g., English, Chinese, Korean, French, or Spanish). The number of characters may be one or more. The characters may be written vertically or horizontally. The writing instrument may be a fountain pen, mechanical pencil, ballpoint pen, felt-tip pen, magic marker, highlighter, black pencil, colored pencil, or the like. The material of the surface of the object 5 may be arbitrary, such as cardboard, metal, glass, wood, or plastic. The color of the surface of the object 5 may be arbitrary, such as a single color, a mixture of multiple colors, or a pattern. In addition to writing characters by hand directly on the surface of the object 5, a label with handwritten characters may be attached to the surface of the object 5.

[0055] Next, the operator of the portable device 2 operates the portable device 2 to capture an image of the surface of the object 5 to be registered, including the handwritten characters, from approximately the front using the camera 21 (step S2). At this time, the acquisition unit 2611 of the portable device 2 cuts out a partial image including the handwritten characters from the image captured by the camera 21, and stores it in the storage unit 25 as a registered image 252. Figure 6 shows an example of the registered image 252. The registered image in this example shows a handwritten eight-digit number.

[0056] Next, the character recognition unit 2612 of the portable terminal 2 recognizes the handwritten characters in the registered image 252, and displays the character recognition result 253 on the screen display unit 24 and stores it in the storage unit 25 (step S3). In this case, it is assumed that the character recognition result 253 is "20220424." Next, the operator of the portable terminal 2 checks the displayed character recognition result and inputs OK if there is no problem, or the correct content if it is incorrect, from the operation input unit 23 (step S4). In this case, it is assumed that the operator corrects the result to "20220429." At this time, the character recognition unit 2612 rewrites the character recognition result 253 with the corrected content.

[0057] Next, the operator of the portable terminal 2 inputs auxiliary information through the operation input unit 23 (step S5). At this time, the registration unit 2113 receives the input auxiliary information 258 from the operation input unit 23 and stores it in the storage unit 25. Next, the registration unit 2113 generates registered object information, which is the registered image 252 or registered features extracted from the image (step S6). Next, the registration unit 2113 creates registration data including the registered object information, the character recognition result 253, and the auxiliary information 258 (step S7). Next, the registration unit 2113 transmits a registration request including the created registration data to the server 3 via the communication I / F unit 22 (step S8).

[0058] The DB management means 361 of the server 3 receives the registration data included in the registration request sent from the mobile terminal 2 (step S9). Next, the DB management means 361 adds a registration number to the registration data and stores it in the registration data DB 352 (step S10).

[0059] With the above, registration data related to one registration target object has been registered in registration data DB 352. An operator of the same portable terminal 2 and operators of other portable terminals 2 can register registration data related to other registration target objects in registration data DB 352 in the same manner as above.

[0060] Next, the matching operation will be described. Fig. 7 is a flowchart showing an example of the procedure for the matching operation. First, the operator of the portable terminal 2 operates the portable terminal 2 to photograph the surface of the object 5 to be matched, including handwritten characters, with the camera 111 (step S11). At this time, the acquisition unit 2621 of the matching means 262 of the portable terminal 2 acquires the photographed image from the camera 21, cuts out a partial image including the handwritten characters, and saves it in the storage unit 25 as a matching image 255. It is assumed that the saved matching image 255 is the same as the registered image shown in Fig. 6.

[0061] Next, the character recognition unit 2622 of the portable terminal 2 recognizes the handwritten characters in the match image 255 and stores the character recognition result 256 in the storage unit 25 (step S12). Here, it is assumed that the character recognition result 256 is "20220424".

[0062] Next, the extraction unit 2623 of the portable terminal 2 acquires all of the registration data stored in the registration data DB 352 from the server 3 (step S13). Specifically, the extraction unit 2623 transmits an acquisition request to the server 3 via the communication I / F unit 22. When the DB management means 361 of the server 3 receives this acquisition request via the communication I / F unit 32, it transmits a registration data group including all of the registration data stored in the registration data DB 352 to the portable terminal 2 via the communication I / F unit 32. The extraction unit 2623 of the portable terminal 2 receives this via the communication I / F unit 22.

[0063] Next, the extraction unit 2623 of the portable terminal 2 extracts, from the enrollment data group, enrollment data including a registered character recognition result similar to the character recognition result 256 (step S14). The extraction unit 2623 excludes, at this point, enrollment data including a registered character recognition result that is not similar to the character recognition result 256 from the target of image matching. For example, if the character recognition result 256 is "20220424," enrollment data including a registered character recognition result similar to it, such as "20220429," is extracted, but enrollment data including a registered character recognition result that is not similar to it, such as "14284485" or "78462179," is not extracted. The enrollment data group extracted by the extraction unit 2623 is passed to the determination unit 2624.

[0064] Next, the determination unit 2624 of the portable terminal 2 generates matching object information, which is the matching image 255 or matching features extracted from the matching image (step S15). Next, the determination unit 2624 determines the identity of the matching target object and the registration target object by a matching process using the registered object information and matching object information for each registration data in the registration data group received from the extraction unit 2623, and stores the determination result 257 in the storage unit 25 (step S16). Next, the determination unit 2624 displays the determination result 257 on the screen display unit 24 and / or outputs it to an external device via the communication I / F unit 22 (step S17).

[0065] As described above, during registration, the portable device 2 registers, for each object to be registered, registration data in the registration data DB 352 of the server 3, including registration images obtained by photographing the surface of the object to be registered on which handwritten characters are written or registration object information, which is feature values ​​extracted from the registration images, registration character recognition results obtained by character recognition of the handwritten characters in the registration images, and associated information. During matching, the portable device 2 matches matching data, which associates matching images obtained by photographing the surface of the object to be matched on which handwritten characters are written or matching object information, which is feature values ​​extracted from the matching images, with matching character recognition results obtained by character recognition of the handwritten characters in the matching images, against a plurality of registration data stored in the registration data DB 352. In this case, the portable device 2 first extracts, from the plurality of registration data stored in the registration data DB 352, registration data including registration character recognition results similar to the matching character recognition results in the matching data. Because the calculation of the similarity between the matching character recognition results and the registration character recognition results is text matching, it can be calculated with a much smaller amount of calculation than image matching. Next, the portable device 2 performs a matching process using the matching object information in the matching data and the registered object information in the extracted registered data to determine the identity of the object to be matched and the object to be registered. As a result, the portable device 2 can reduce the number of registered images that must be matched with the matching image, and can shorten the calculation time required to detect a registered image that matches the matching image.

[0066] Next, an example of how the object management system 1 according to this embodiment is used will be described.

[0067] <Equipment Management> When multiple cardboard boxes containing equipment are stored in a warehouse or the like, the operator of the mobile terminal 2 handwrites a control number or the like on the surface of each cardboard box, takes a photo of the surface including the handwritten characters with the mobile terminal 2 such as a smartphone, and registers the registration data for each cardboard box in the registration data DB 352. At this time, the additional information for each registration data includes, for example, the type and quantity of equipment stored in each cardboard box, the storage date, etc. When checking the equipment at a later date, the operator can check the contents without opening the cardboard box by taking a photo of the handwritten characters on the cardboard box whose contents he wants to check with the mobile terminal 2 and comparing the photo.

[0068] <Determining the Authenticity of Mail> When sending a package, the sender signs the cardboard box by hand, takes a photo of the side including the handwritten signature using a mobile terminal 2 such as a smartphone, and registers the registration data related to the cardboard box in the registration data DB 352. The sender then leaves the cardboard box in a predetermined location, such as the front door of their home. The sender also writes, for example, information about the delivery destination and the contents of the package in the attached information of the registration data. The postal delivery company photographs the handwritten portion of the cardboard box placed in the predetermined location using a mobile terminal 2 such as a smartphone, and compares the photograph with the handwritten portion to confirm the authenticity of the delivery and the delivery destination, etc., before delivering the item.

[0069] [Second embodiment] The object management system 1A of this embodiment differs from the first embodiment in that it includes a mobile terminal 2A instead of the mobile terminal 2 and a server 3A instead of the server 3, but is otherwise the same as the first embodiment.

[0070] 8, portable terminal 2A differs from portable terminal 2 in that it has program 259 instead of program 251, and has registration means 263 and matching means 264 instead of registration means 261 and matching means 262. Processing unit 26 realizes registration means 263 and matching means 264 by reading and executing program 259.

[0071] The registration means 263 is configured to acquire an image of the surface of the object to be registered, on which handwritten characters are written, captured by the camera 21. The registration means 263 is also configured to acquire additional information regarding the object to be registered from the operator of the portable terminal 2A via the operation input unit 23. The registration means 263 is also configured to transmit a registration request including the image and the additional information to the server 3A via the communication I / F unit 22.

[0072] The matching means 264 is configured to acquire an image of the surface of the object to be matched, on which handwritten characters are written, photographed by the camera 21. The matching means 264 is also configured to transmit a matching request including the image to the server 3A via the communication I / F unit 22. The matching means 264 is also configured to receive a determination result, which is a response to the matching request, from the server 3A, and display it on the screen display unit 24 and / or output it to an external device via the communication I / F unit 22.

[0073] 9, server 3A differs from server 3 in that it has registered image 353, attached information 35A, character recognition result 354, match image 356, character recognition result 357, determination result 358, and program 359 instead of program 351, and has registration means 362 and matching means 363 instead of DB management means 361, but is otherwise the same as server 3. Processing unit 36 ​​implements registration means 362 and matching means 363 by reading and executing program 359.

[0074] The registration means 362 is configured to generate registration data for determining the identity of an object to be registered and register the data in the registration data DB 352. The registration means 362 is configured to include an acquisition unit 3621, a character recognition unit 3622, and a registration unit 3623.

[0075] The acquisition unit 3621 is configured to acquire an image obtained by capturing an image of the surface of the object to be registered, on which handwritten characters are written, and associated information from the portable terminal 2A. The acquisition unit 3621 is also configured to cut out a partial image of a predetermined size including the handwritten characters from the acquired image, and store the cutout partial image as a registered image 353 in the storage unit 35. The acquisition unit 3621 is also configured to store the acquired associated information in the storage unit 35 as associated information 35A.

[0076] The character recognition unit 3622 is configured to read the registered image 353 from the storage unit 35 , perform character recognition on the handwritten characters in the registered image 353 , and store the character recognition result 354 in the storage unit 35 .

[0077] The registration unit 3623 is configured to read out the registered image 353, the auxiliary information 35A, and the character recognition result 354 from the storage unit 35. The registration unit 3623 is also configured to generate registered object information, which is the read-out registered image 353 or registered feature amounts extracted from the registered image. The registered feature amounts may be the same as those in the first embodiment. The registration unit 3623 is also configured to create registered data including the registered object information, the character recognition result 253, and the auxiliary information 35A, and register the data in the registered data DB 352.

[0078] The matching means 363 is configured to determine whether or not the matching data related to the object to be matched received from the portable terminal 2A is identical to any of the registered data registered in the registered data DB 352. The matching means 363 is configured to include an acquisition unit 3631, a character recognition unit 3632, an extraction unit 3633, and a determination unit 3634.

[0079] The acquisition unit 3631 is configured to acquire an image obtained by capturing an image of the surface of the object to be matched, on which handwritten characters are written, from the portable terminal 2A. The acquisition unit 3621 is also configured to cut out a partial image of a predetermined size including the handwritten characters from the acquired image, and store the partial image in the storage unit 35 as the match image 356.

[0080] The character recognition unit 3632 is configured to read the match image 356 from the storage unit 35 , perform character recognition on the handwritten characters in the match image 356 , and store the character recognition result 357 in the storage unit 35 .

[0081] The extraction unit 3633 is configured to extract registration data including a registration character recognition result similar to the character recognition result 357 from among the multiple registration data registered in the registration data DB 352. The extraction unit 3633 transmits the extracted registration data to the determination unit 3634.

[0082] The determination unit 3634 is configured to read the match image 356 from the storage unit 35 and generate match object information, which is the match image 356 or a match feature extracted from the match image 356. The match feature may be the same as the registered feature. The determination unit 3634 is also configured to determine the identity of the object to be matched and the object to be registered by a matching process using the match object information and the registered object information included in the registered data transmitted from the extraction unit 3633. If the determination unit 3634 determines that the object to be matched is identical to any of the registered objects, the determination unit 3634 is configured to create a determination result 358 including auxiliary information included in the registered data of the registered object determined to be identical, and store the result in the storage unit 35. If the determination unit 3634 determines that the object to be matched is not identical to any of the registered objects, the determination unit 3634 is configured to create a determination result 358 indicating that no identical registered object exists, and store the result in the storage unit 35. The determination unit 3634 is also configured to read out the determination result 358 from the storage unit 35 and transmit it via the communication I / F unit 32 to the portable terminal 2A that has transmitted the image of the object to be matched.

[0083] Next, the operation of the object management system 1A according to this embodiment will be described. First, the registration operation will be described.

[0084] 10 is a flowchart showing an example of the registration operation procedure. First, the operator of the portable terminal 2A handwrites characters on any surface of the object 5 to be registered using any writing implement (step S21). Next, the operator operates the portable terminal 2A to photograph the surface of the object 5 to be registered, including the handwritten characters, from approximately directly in front with the camera 21 (step S22). Next, the operator inputs additional information from the operation input unit 23 (step S23). Next, the registration means 263 of the portable terminal 2A transmits a registration request including the image and additional information to the server 3A via the communication I / F unit 32 (step S24).

[0085] The acquisition unit 3621 of the registration means 362 of the server 3A receives the registration request transmitted from the portable terminal 2A and stores the registration image 353 generated from the image of the object to be registered contained therein and the attached information 35A in the storage unit 35 (step S25). Next, the character recognition unit 3622 performs character recognition on the handwritten characters in the registration image 353 and transmits the character recognition result 354 via the communication I / F unit 32 to display it on the screen display unit 24 of the portable terminal 2A and store it in the storage unit 35 (step S26). Next, the character recognition unit 3622 corrects the character recognition result as necessary through dialogue with the operator of the portable terminal 2A (step S27).

[0086] Next, the registration unit 3623 of the server 3A generates registered object information, which is the registered image 353 or registered features extracted from the registered image, creates registered data including the registered object information, the character recognition result 354, and the ancillary information 35A, and stores the data in the registered data DB 352 (step S28).

[0087] With the above, registration data related to one registration target object has been registered in registration data DB 352. An operator of the same portable terminal 2 and operators of other portable terminals 2 can register registration data related to other registration target objects in registration data DB 352 in the same manner as above.

[0088] Next, the matching operation will be described. Fig. 11 is a flowchart showing an example of the procedure for the matching operation. First, the operator of the portable terminal 2A operates the portable terminal 2A to capture an image of the surface of the object 5 to be matched, including handwritten characters, preferably from directly in front, using the camera 21 (step S31). Next, the matching means 264 of the portable terminal 2A acquires the captured image from the camera 21 and transmits a matching request including the image to the server 3A via the communication I / F unit 22 (step S32).

[0089] The acquisition unit 3631 of the matching means 363 of the server 3A receives the matching request from the portable terminal 2A via the communication I / F unit 32, extracts a partial image including handwritten characters from the image included in the matching request, and stores the partial image as a matching image 356 in the storage unit 35 (step S33). Next, the character recognition unit 3632 performs character recognition on the handwritten characters in the matching image 356 and stores the character recognition result 357 in the storage unit 35 (step S34). Next, the extraction unit 3633 extracts registration data including a registration character recognition result similar to the character recognition result 357 from the registration data group stored in the registration data DB 352 (step S35). The registration data group extracted by the extraction unit 3633 is passed to the determination unit 3634.

[0090] Next, the determination unit 3634 generates matching object information, which is the matching image 356 or matching features extracted from the match image 356 (step S36). Next, the determination unit 3634 determines the identity of the matching target object and the registration target object by a matching process using the registered object information and the matching object information for each registration data in the registration data group received from the extraction unit 3633, and transmits the determination result 358 to the portable device 2A via the communication I / F unit 32 (step S38).

[0091] The checking means 264 of the portable terminal 2A receives the determination result 358 through the communication I / F unit 22 and displays it on the screen display unit 24 (step S39).

[0092] As described above, during registration, the server 3A acquires, for each object to be registered, from the portable device 2A, an image of the surface of the object to be registered on which handwritten characters are written and associated information, and registers registration data in the registration data DB 352 that associates registration character recognition results obtained by character recognition of the handwritten characters in the registration image generated from the image with registration object information, which is registration features extracted from the registration image, or the registration image. During matching, the server 3A acquires, from the portable device 2A, an image of the surface of the object to be matched on which handwritten characters are written, and extracts from the registration data DB 352 registration data that includes registration character recognition results similar to matching character recognition results obtained by character recognition of the handwritten characters in a match image generated from the image. Next, the server 3A performs a matching process using the matching object information, which is the match image or the match features extracted from the match image, and the registered object information in the extracted registration data, to determine the identity of the object to be matched and the object to be registered, and transmits the determination result to the portable device 2A. As a result, the server 3A can reduce the number of registered images that must be compared with the match image, and can shorten the calculation time required to detect a registered image that matches the match image.

[0093] Third Embodiment Next, an object management device according to a third embodiment of the present disclosure will be described. In this embodiment, an object management device having an image matching function according to the present disclosure will be described.

[0094] Referring to FIG. 12, an object management device 10 according to this embodiment is configured as a mobile terminal or a server, and is composed of an acquisition unit 11, a character recognition unit 12, an extraction unit 13, and a determination unit 14.

[0095] The acquisition unit 11 is configured to acquire a match image obtained by capturing an image of a surface of an object to be matched on which handwritten characters are written. The acquisition unit 11 can be configured, for example, by the acquisition unit 2621 of the portable terminal 2 or the acquisition unit 3631 of the server 3A, but is not limited thereto.

[0096] The character recognition means 12 is configured to output a collation character recognition result obtained by character recognition of handwritten characters in the collation image acquired by the acquisition means 11. The character recognition means 12 can be configured by, for example, the character recognition unit 2622 of the portable terminal 2 or the character recognition unit 3632 of the server 3A, but is not limited to this.

[0097] The extraction means 13 is configured to extract registration data including a registration character recognition result similar to the comparison character recognition result output from the character recognition means 12 from a plurality of stored registration data including a registration image related to an object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result. The extraction means 13 can be configured, for example, by the extraction unit 2623 of the portable terminal 2 or the extraction unit 3633 of the server 3A, but is not limited to this.

[0098] The determination means 14 is configured to determine the identity of the object to be matched and the object to be registered by a matching process using the match image acquired by the acquisition means 11 or match object information, which is a feature extracted from the match image, and the registered object information in the extracted registration data. The determination means 14 can be configured by, for example, the determination unit 2624 of the portable terminal 2 or the determination unit 3634 of the server 3A, but is not limited to this.

[0099] The object management device 10 configured as described above operates as follows. First, the acquisition means 11 acquires a match image obtained by capturing a photograph of the surface of an object to be matched, on which handwritten characters are written. Next, the character recognition means 12 outputs a match character recognition result obtained by character recognition of the handwritten characters in the match image acquired by the acquisition means 11. Next, the extraction means 13 extracts registration data including a registration character recognition result similar to the match character recognition result output from the character recognition means 12 from a plurality of stored registration data in which registration images of the object to be registered or registration object information, which is registration feature amounts extracted from the registration images, are associated with registration character recognition results. Next, the determination means 14 determines the identity of the object to be matched and the object to be registered by a matching process using the match image acquired by the acquisition means 11 or the registration object information, which is feature amounts extracted from the match image, and the registration object information in the extracted registration data.

[0100] According to the object management device 10 configured and operating as described above, the number of registered images that must be compared with the match image can be reduced, and as a result, the calculation time required to detect a registered image that matches the match image can be shortened.

[0101] Fourth Embodiment Next, an object management device according to a fourth embodiment of the present disclosure will be described. In this embodiment, an object management device having an image registration function according to the present disclosure will be described.

[0102] Referring to FIG. 13, an object management device 15 according to this embodiment is configured as a mobile terminal or a server, and is composed of an acquisition unit 16, a character recognition unit 17, and a registration unit 18.

[0103] The acquisition unit 16 is configured to acquire a registration image obtained by capturing a surface of an object to be registered on which handwritten characters are written. The acquisition unit 16 can be configured, for example, by the acquisition unit 2611 of the portable terminal 2 or the acquisition unit 3621 of the server 3A, but is not limited thereto.

[0104] The character recognition means 17 is configured to output a registered character recognition result obtained by character recognition of handwritten characters in the registered image acquired by the acquisition means 16. The character recognition means 17 can be configured by, for example, the character recognition unit 2612 of the portable terminal 2 or the character recognition unit 3622 of the server 3A, but is not limited to this.

[0105] The registration means 18 is configured to generate registration data, which associates the registration image acquired by the acquisition means 16 or the registration object information, which is the registration feature extracted from the registration image, with the registration character recognition result output from the character recognition means 17, as data for determining the identity of the object to be registered. The registration means 18 can be configured, for example, by the registration unit 2113 of the portable terminal 2 or the registration unit 3623 of the server 3A, but is not limited to this.

[0106] The object management device 15 configured as described above operates as follows: First, the acquisition means 16 acquires a registration image obtained by capturing a photograph of the surface of an object to be registered, on which handwritten characters are written. Next, the character recognition means 17 outputs a registration character recognition result obtained by character recognition of the handwritten characters in the registration image acquired by the acquisition means 16. Next, the registration means 18 generates registration data that associates the registration image acquired by the acquisition means 16 or registration object information, which is registration feature amounts extracted from the registration image, with the registration character recognition result output from the character recognition means 17, as data for determining the identity of the object to be registered.

[0107] According to the object management device 15 configured and operating as described above, the number of registered images that must be compared with the match image can be reduced, and as a result, the calculation time required to detect a registered image that matches the match image can be shortened.

[0108] Fifth Embodiment Next, an object management system according to a fifth embodiment of the present disclosure will be described. In this embodiment, an object management system having an image matching function according to the present disclosure will be described.

[0109] Referring to Figure 14, the object management system 10A according to this embodiment is configured as a system including a mobile terminal and a server, and is composed of an acquisition means 11A, a character recognition means 12A, an extraction means 13A, and a determination means 14A.

[0110] The acquisition unit 11A is configured to acquire a match image obtained by capturing an image of a surface of an object to be matched on which handwritten characters are written. The acquisition unit 11A can be configured, for example, by the acquisition unit 2621 of the portable terminal 2 or the acquisition unit 3631 of the server 3A, but is not limited thereto.

[0111] The character recognition means 12A is configured to output a comparison character recognition result obtained by character recognition of handwritten characters in the comparison image acquired by the acquisition means 11 A. The character recognition means 12A can be configured by, for example, the character recognition unit 2622 of the portable terminal 2 or the character recognition unit 3632 of the server 3A, but is not limited to this.

[0112] The extraction means 13A is configured to extract registration data including a registration character recognition result similar to the comparison character recognition result output from the character recognition means 12A from a plurality of stored registration data including a registration image related to an object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result. The extraction means 13A can be configured, for example, by the extraction unit 2623 of the portable terminal 2 or the extraction unit 3633 of the server 3A, but is not limited to this.

[0113] The determination means 14A is configured to determine the identity of an object to be matched and an object to be registered by a matching process using the match image acquired by the acquisition means 11A or match object information, which is a feature extracted from the match image, and the registered object information in the extracted registration data. The determination means 14A can be configured by, for example, the determination unit 2624 of the portable terminal 2 or the determination unit 3634 of the server 3A, but is not limited to this.

[0114] The object management device 10A configured as described above operates as follows. First, the acquisition unit 11A acquires a match image obtained by capturing a photograph of the surface of a target object on which handwritten characters are written. Next, the character recognition unit 12A outputs a match character recognition result obtained by character recognition of the handwritten characters in the match image acquired by the acquisition unit 11A. Next, the extraction unit 13A extracts registration data including a registration character recognition result similar to the match character recognition result output from the character recognition unit 12A from a plurality of stored registration data in which the registration image of the target object to be registered or the registration object information, which is registration feature values ​​extracted from the registration image, is associated with the registration character recognition result. Next, the determination unit 14A determines the identity of the target object to be registered and the target object to be registered by a matching process using the match image acquired by the acquisition unit 11A or the registration object information, which is feature values ​​extracted from the match image, and the registration object information in the extracted registration data.

[0115] According to the object management device 10A configured and operating as described above, the number of registered images that must be compared with the match image can be reduced, and as a result, the calculation time required to detect a registered image that matches the match image can be shortened.

[0116] Sixth Embodiment Next, an object management system according to a sixth embodiment of the present disclosure will be described. In this embodiment, an object management system having an image registration function according to the present disclosure will be described.

[0117] Referring to FIG. 15, an object management system 15A according to this embodiment is configured as a system including a mobile terminal and a server, and is configured from an acquisition unit 16A, a character recognition unit 17A, and a registration unit 18A.

[0118] The acquisition unit 16A is configured to acquire a registration image obtained by capturing a surface of an object to be registered on which handwritten characters are written. The acquisition unit 16A can be configured, for example, by the acquisition unit 2611 of the mobile terminal 2 or the acquisition unit 3621 of the server 3A, but is not limited thereto.

[0119] The character recognition means 17A is configured to output a registered character recognition result obtained by character recognition of handwritten characters in the registered image acquired by the acquisition means 16A. The character recognition means 17A can be configured by, for example, the character recognition unit 2612 of the portable terminal 2 or the character recognition unit 3622 of the server 3A, but is not limited to this.

[0120] The registration means 18A is configured to generate registration data that associates the registration image acquired by the acquisition means 16A or the registration object information, which is the registration feature extracted from the registration image, with the registration character recognition result output from the character recognition means 17A, as data for determining the identity of the object to be registered. The registration means 18A can be configured, for example, by the registration unit 2113 of the mobile terminal 2 or the registration unit 3623 of the server 3A, but is not limited to this.

[0121] The object management system 15A configured as described above operates as follows. First, the acquisition means 16A acquires a registration image obtained by capturing a photograph of the surface of an object to be registered, on which handwritten characters are written. Next, the character recognition means 17A outputs a registration character recognition result obtained by character recognition of the handwritten characters in the registration image acquired by the acquisition means 16A. Next, the registration means 18A generates registration data that associates the registration image acquired by the acquisition means 16A or registration object information, which is registration features extracted from the registration image, with the registration character recognition result output from the character recognition means 17A, as data for determining the identity of the object to be registered.

[0122] According to the object management device 15A configured and operating as described above, the number of registered images that must be compared with the match image can be reduced, and as a result, the calculation time required to detect a registered image that matches the match image can be shortened.

[0123] [Other Embodiments] The present disclosure has been described above with reference to the above-described embodiments, but the present disclosure is not limited to the above-described embodiments. Each embodiment can be combined with other embodiments as appropriate. Furthermore, various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0124] For example, in the above embodiment, a part of the processing executed by the mobile terminal 2 may be configured to be executed by the server 3. For example, a part or all of the processing of steps S3, S4, S6, and S7 in Fig. 5 and steps S12, S13, S14, S15, S16, and S17 in Fig. 7 may be configured to be executed by the server 3.

[0125] In addition, in the above embodiment, a part or all of the processing executed by the server 3A may be configured to be executed by the mobile terminal 2A. For example, a part or all of the processing of steps S26, S27, and S28 in Fig. 10 and steps S34, S35, S36, and S37 in Fig. 11 may be configured to be executed by the mobile terminal 2A.

[0126] Furthermore, in the above embodiment, the registered object information and the registered character recognition results in the registration data related to the object to be registered were extracted and recognized from the same image of the object, but they may also be extracted and recognized from multiple different images of the object. In this case, the image from which the registered object information is extracted does not necessarily have to contain all of the handwritten characters, but may be an image in which a predetermined portion of the handwritten characters (for example, the first one or two characters) is captured with high image quality (high resolution). On the other hand, the image to be subjected to character recognition may be an image in which all of the handwritten characters are captured with low image quality (low resolution).

[0127] Similarly, the matching object information and the matching character recognition results in the matching data related to the object to be matched are extracted and recognized from the same image of the object, but they may also be extracted and recognized from multiple different images of the object. In this case, the image from which the matching object information is extracted does not necessarily have to be an image that contains all of the handwritten characters, and may be an image in which a predetermined portion of the handwritten characters (for example, the first one or two characters) is captured with high image quality (high resolution). On the other hand, the image to be subjected to character recognition may be an image captured with low image quality (low resolution) as long as it contains all of the handwritten characters.

[0128] Furthermore, instead of the above-mentioned CPU, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof may be used.

[0129] The present disclosure can be used in the general field of managing individual objects such as products, merchandise, or parcels for purposes such as origin identification, quality control, or distribution control.

[0130] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. [Supplementary Note 1] An object management device comprising: a first acquisition means for acquiring a registration image obtained by capturing an image of a surface of an object to be registered, the surface having handwritten characters written thereon; a first character recognition means for outputting a registration character recognition result obtained by performing character recognition on the handwritten characters in the registration image; and a registration means for generating registration data that associates the registration image or registration object information, which is a feature extracted from the registration image, with the registration character recognition result as data for determining the identity of the object to be registered. [Supplementary Note 2] The object management device according to Supplementary Note 1, wherein the first acquisition means further acquires auxiliary information related to the object to be registered, and the registration data further includes the auxiliary information. [Supplementary Note 3] The object management device according to Supplementary Note 2, wherein the registration means further writes the registration data to a storage means. [Supplementary Note 4] The object management device according to Supplementary Note 3, further comprising: a second acquisition means for acquiring a match image obtained by capturing an image of the surface of an object to be matched, the surface having handwritten characters written thereon; a second character recognition means for outputting a match character recognition result obtained by character recognition of the handwritten characters in the match image; an extraction means for extracting registration data including a registration character recognition result similar to the match character recognition result from a plurality of stored registration data; and a determination means for determining the identity of the object to be matched and the object to be registered by a matching process using the match image or match object information which is a feature extracted from the match image and the registered object information included in the extracted registration data. [Supplementary Note 5] The object management device according to Supplementary Note 4, wherein the determination means further outputs the auxiliary information included in the registration data related to the object to be registered that has been determined to be identical to the object to be matched.[Supplementary Note 6] An object management device comprising: a first acquisition means for acquiring a match image obtained by capturing an image of the surface of an object to be matched, the surface having handwritten characters written thereon; a first character recognition means for outputting a match character recognition result obtained by character recognition of the handwritten characters in the match image; an extraction means for extracting registration data including a registration character recognition result similar to the match character recognition result from a plurality of stored registration data including a registration image related to the object to be registered or registration object information which is a feature extracted from the registration image, and the registration character recognition result; and a determination means for determining the identity of the object to be matched and the object to be registered by a matching process using the match image or the matching object information which is the feature extracted from the match image and the registration object information in the extracted registration data. [Supplementary Note 7] The object management device according to Supplementary Note 6, wherein the registration data further includes auxiliary information, and the determination means further outputs the auxiliary information included in the registration data related to the object to be registered that has been determined to be identical to the object to be matched. [Supplementary Note 8] The object management device according to Supplementary Note 6 or 7, further comprising: a second acquisition means for acquiring a registration image obtained by capturing an image of the surface of the object to be registered, the surface having handwritten characters written thereon; a second character recognition means for outputting a registration character recognition result obtained by character recognition of the handwritten characters in the registration image; and a registration means for generating registration data, in which the registration image or registration object information, which is a feature extracted from the registration image, is associated with the registration character recognition result, as data for determining the identity of the object to be registered. [Supplementary Note 9] An object management method, in which a computer acquires a registration image obtained by capturing an image of the surface of the object to be registered, the surface having handwritten characters written thereon; outputs a registration character recognition result obtained by character recognition of the handwritten characters in the registration image; and generates registration data, in which the registration image or registration object information, which is a feature extracted from the registration image, is associated with the registration character recognition result, as data for determining the identity of the object to be registered. [Supplementary Note 10] The object management method according to Supplementary Note 9, in which the computer further acquires auxiliary information related to the object to be registered, and adds it to the registration data. [Supplementary Note 11] The object management method according to Supplementary Note 10, wherein the computer further writes the registration data to a storage means.[Supplementary Note 12] The object management method according to Supplementary Note 11, wherein the computer further acquires a match image obtained by photographing a surface of an object to be matched, on which handwritten characters are written, outputs a match character recognition result obtained by performing character recognition on the handwritten characters in the match image, extracts registration data including a registration character recognition result similar to the match character recognition result from a plurality of registration data, and determines the identity of the object to be matched and the object to be registered by a matching process using the match image or match object information which is a feature extracted from the match image and the registration object information included in the extracted registration data. [Supplementary Note 13] The object management method according to Supplementary Note 12, wherein the computer further outputs the accessory information included in the registration data related to the object to be registered that is determined to be identical to the object to be matched. [Supplementary Note 14] An object management method in which a computer acquires a match image obtained by photographing the surface of an object to be matched, the match image having handwritten characters written thereon, outputs a match character recognition result obtained by performing character recognition on the handwritten characters in the match image, extracts registration data including a registration character recognition result similar to the match character recognition result from a plurality of stored registration data including a registration image related to the object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result, and determines the identity of the object to be matched and the object to be registered by a matching process using the match image or the matching object information which is the feature extracted from the match image and the registration object information in the extracted registration data. [Supplementary Note 15] The object management method according to Supplementary Note 14, in which the registration data further includes auxiliary information, and the computer further outputs the auxiliary information included in the registration data related to the object to be registered that is determined to be identical to the object to be matched. [Supplementary Note 16] The object management method described in Supplementary Note 14 or 15, wherein the computer further acquires a registration image obtained by photographing the surface of the object to be registered, on which handwritten characters are written; outputs a registration character recognition result obtained by character recognition of the handwritten characters in the registration image; and generates registration data that associates the registration image or registration object information, which is a feature extracted from the registration image, with the registration character recognition result as data for determining the identity of the object to be registered.[Supplementary Note 17] A computer-readable recording medium having recorded thereon a program for causing a computer to perform the following processes: acquiring a registration image obtained by photographing the surface of an object to be registered, the surface having handwritten characters written on it; outputting a registration character recognition result obtained by character recognition of the handwritten characters in the registration image; and generating registration data that associates the registration image or registration object information, which is a feature extracted from the registration image, with the registration character recognition result, as data for determining the identity of the object to be registered. [Supplementary Note 18] The recording medium according to Supplementary Note 17, having recorded thereon a program for causing the computer to further perform the following processes: acquiring auxiliary information related to the object to be registered, and adding it to the registration data. [Supplementary Note 19] The recording medium according to Supplementary Note 18, having recorded thereon a program for causing the computer to further perform the following processes: storing the registration data in storage means. [Supplementary Note 20] The recording medium according to Supplementary Note 19, having recorded thereon a program for causing the computer to further perform the following processes: acquiring a match image obtained by capturing an image of the surface of an object to be matched, on which handwritten characters are written; outputting a match character recognition result obtained by performing character recognition on the handwritten characters in the match image; extracting, from a plurality of stored registration data, registration data including a registration character recognition result similar to the match character recognition result; and determining the identity of the object to be matched and the object to be registered by a matching process using the match image or match object information which is a feature extracted from the match image and the registration object information included in the extracted registration data. [Supplementary Note 21] The recording medium according to Supplementary Note 20, having recorded thereon a program for causing the computer to further perform the following processes: outputting the accessory information included in the registration data related to the object to be registered which has been determined to be identical to the object to be matched.[Supplementary Note 22] A computer-readable recording medium having recorded thereon a program for causing a computer to perform the following steps: acquiring a comparison image obtained by photographing the surface of an object to be compared, the surface having handwritten characters written thereon, outputting a comparison character recognition result obtained by character recognition of the handwritten characters in the comparison image, extracting registration data including a registration character recognition result similar to the comparison character recognition result from a plurality of stored registration data including a registration image related to the object to be registered or registration object information which is feature amounts extracted from the registration image, and a registration character recognition result, and determining the identity of the object to be compared and the object to be registered by a matching process using the comparison image or the matching object information which is feature amounts extracted from the comparison image and the registration object information in the extracted registration data. [Supplementary Note 23] The recording medium according to Supplementary Note 22, wherein the registration data further includes auxiliary information, and further causing the computer to perform the following steps: outputting the auxiliary information included in the registration data related to the object to be registered that has been determined to be identical to the object to be compared. [Supplementary Note 24] The recording medium according to Supplementary Note 22 or 23, having recorded thereon a program for causing the computer to further perform the following processes: acquiring a registration image obtained by photographing the surface of an object to be registered on which handwritten characters are written, a process of outputting a registration character recognition result obtained by character recognition of the handwritten characters in the registration image, and a process of generating registration data in which the registration image or registered object information which is feature amounts extracted from the registration image is associated with the registration character recognition result, as data for determining the identity of the object to be registered. [Supplementary Note 25] An object management system comprising: acquisition means for acquiring a registration image obtained by photographing the surface of the object to be registered on which handwritten characters are written, character recognition means for outputting a registration character recognition result obtained by character recognition of the handwritten characters in the registration image, and registration means for generating registration data in which the registration image or registered object information which is feature amounts extracted from the registration image is associated with the registration character recognition result, as data for determining the identity of the object to be registered.[Supplementary Note 26] An object management system comprising: an acquisition means for acquiring a match image obtained by capturing an image of a surface of an object to be matched, the surface having handwritten characters written thereon; a character recognition means for outputting a match character recognition result obtained by character recognizing the handwritten characters in the match image; an extraction means for extracting registration data including a registration character recognition result similar to the match character recognition result from a plurality of stored registration data including a registration image relating to the object to be registered or registration object information which is a feature extracted from the registration image, and the registration character recognition result; and a determination means for determining the identity of the object to be matched and the object to be registered by a matching process using the match image or the matching object information which is the feature extracted from the match image, and the registration object information in the extracted registration data.

[0131] REFERENCE SIGNS LIST 1 Object management system 2 Portable terminal 2A Portable terminal 4 Network 5 Object 10 Object management device 11 Acquisition means 12 Character recognition means 13 Extraction means 14 Determination means 15 Object management device 16 Acquisition means 17 Character recognition means 18 Registration means 10A Object management device 11A Acquisition means 12A Character recognition means 13A Extraction means 14A Determination means 15A Object management device 16A Acquisition means 17A Character recognition means 18A Registration means 21 Camera 22 Communication I / F unit 23 Operation input unit 24 Screen display unit 25 Memory unit 26 Arithmetic processing unit 31 Camera 32 Communication I / F unit 33 Operation input unit 34 Screen display unit 35 Memory unit 36 ​​Arithmetic processing unit 251 Program 252 Registered image 253 Character recognition result 254 Registration data 255 Matching image 256 Character recognition result 257 Determination result 258 Auxiliary information 259 Program 25A Auxiliary information 261 Registration means 262 Matching means 351 Program 352 Registration data DB 353 Registration image 354 Character recognition result 356 Matching image 357 Character recognition result 358 Determination result 359 Program 35A Auxiliary information 361 DB management means 362 Registration means 363 Matching means 2611 Acquisition unit 2612 Character recognition unit 2613 Registration unit 2621 Acquisition unit 2622 Character recognition unit 2623 Extraction unit 2624 Determination unit 3621 Acquisition unit 3622 Character recognition unit 3623 Registration unit 3631 Acquisition unit 3633 Extraction section 3634 Judgment section

Claims

1. a first acquisition means for acquiring a registration image obtained by capturing an image of a surface of an object to be registered on which handwritten characters are written; a first character recognition means for outputting a registered character recognition result obtained by character recognition of the handwritten characters in the registered image; a registration means for generating registration data in which the registration image or registration object information, which is a feature extracted from the registration image, is associated with the registration character recognition result as data for determining the identity of the object to be registered; An object management device comprising:

2. The first acquisition means further acquires auxiliary information related to the object to be registered, The registration data further includes the auxiliary information. The object management device according to claim 1 .

3. The registration means further writes the registration data into a storage means. The object management device according to claim 2 .

4. a second acquisition means for acquiring a collation image obtained by capturing an image of a surface of an object to be collated on which handwritten characters are written; a second character recognition means for outputting a collation character recognition result obtained by character recognition of the handwritten characters in the collation image; an extraction means for extracting registration data including a registration character recognition result similar to the collated character recognition result from a plurality of stored registration data; and a determination means for determining the identity of the object to be matched and the object to be registered by a matching process using the match image or match object information, which is a feature amount extracted from the match image, and the registered object information included in the extracted registered data. The object management device according to claim 3 .

5. The determination means further outputs the auxiliary information included in the registration data relating to the object to be registered that is determined to be identical to the object to be matched. The object management device according to claim 4 .

6. a first acquisition means for acquiring a collation image obtained by capturing an image of a surface of an object to be collated, on which handwritten characters are written; a first character recognition means for outputting a collation character recognition result obtained by character recognition of the handwritten characters in the collation image; extraction means for extracting registration data including a registration character recognition result similar to the comparison character recognition result from a plurality of stored registration data including a registration image relating to an object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result; a determination means for determining the identity of the object to be matched and the object to be registered by a matching process using the match image or match object information, which is a feature extracted from the match image, and the registered object information in the extracted registered data; An object management device comprising:

7. The computer A registration image is acquired by capturing a surface of an object to be registered on which handwritten characters are written; outputting a registration character recognition result obtained by character recognition of the handwritten character in the registration image; generating registration data in which the registration image or registration object information, which is a feature extracted from the registration image, is associated with the registration character recognition result as data for determining the identity of the object to be registered; Object management method.

8. The computer A matching image is obtained by photographing the surface of an object to be matched, on which handwritten characters are written; outputting a collation character recognition result obtained by character recognition of the handwritten characters in the collation image; extracting registration data including a registration character recognition result similar to the comparison character recognition result from a plurality of stored registration data including a registration image relating to an object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result; determining the identity of the object to be matched and the object to be registered by a matching process using the match image or match object information, which is a feature extracted from the match image, and the registered object information in the extracted registered data; Object management method.

9. On the computer, A process of acquiring a registration image by capturing an image of a surface of an object to be registered on which handwritten characters are written; a process of outputting a registration character recognition result obtained by character recognition of the handwritten characters in the registration image; generating registration data in which the registration image or registration object information, which is a feature extracted from the registration image, is associated with the registration character recognition result as data for determining the identity of the object to be registered; A program to perform the following.

10. On the computer, A process of acquiring a matching image by capturing an image of a surface of an object to be matched on which handwritten characters are written; a process of outputting a collation character recognition result obtained by character recognition of the handwritten characters in the collation image; A process of extracting registration data including a registration character recognition result similar to the comparison character recognition result from a plurality of stored registration data including a registration image relating to an object to be registered or registration object information which is a feature extracted from the registration image, and a registration character recognition result; a process of determining the identity of the object to be matched and the object to be registered by a matching process using the match image or match object information, which is a feature extracted from the match image, and the registered object information in the extracted registered data; A program to perform the following.