Object identification support apparatus, object identification support method, object identification support program, and storage medium

Abstract

To provide an object identification support apparatus, an object identification support method, an object identification support program, and a storage medium capable of easily identifying an object.SOLUTION: An object identification support apparatus 10 comprises a hardware configuration including a central processing unit, a memory, a storage apparatus, an input apparatus, an output apparatus, a communication device or the like, each of which is connected to one another via a bus through its respective interface (I / F), and comprises: an object information acquisition unit 11 for acquiring an object image by capturing the object; a voice data acquisition unit 12 for acquiring voice data including voice for vocal confirmation regarding the object; a criterion selection unit 13 for selecting identification criterion information including at least one piece of object identification image information based on the voice data; and an object identification unit 14 for identifying a type of the object based on the identification criterion information and the object image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an object identification assistance device, an object identification assistance method, an object identification assistance program, and a recording medium. [Background technology]

[0002] In recent years, the ready-meal industry, which provides so-called ready-meals such as boxed lunches and side dishes, has been attracting attention. In the ready-meal industry, various cooked dishes need to be produced by unspecified employees, so an appropriate cooking support system is required from the viewpoint of quality control. For example, Patent Document 1 describes a cooking support method in a cooking support system in which a server device that stores information related to cooking recipes and a terminal device having a display are connected via a network. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-068785 A Summary of the Invention [Problem to be solved by the invention]

[0004] The system described in Patent Document 1 is only capable of displaying recipes for food products, and is unable to confirm whether the cook has used appropriate ingredients and tools. Therefore, it is conceivable to use image recognition by AI to identify finished products. However, in order to identify products using image recognition by AI, it is necessary to prepare a large amount of training data, which is costly and time-consuming. Such issues are not limited to the manufacture of food products, but are similar in various fields of product manufacturing.

[0005] In view of the above, an object of the present disclosure is to provide an object identification assistance device, an object identification assistance method, an object identification assistance program, and a recording medium that are capable of easily identifying an object. [Means for solving the problem]

[0006] In order to achieve the above object, the object identification assistance device of the present disclosure comprises: The present invention includes an object information acquisition unit, a voice data acquisition unit, a criterion selection unit, and an object identification unit, The object information acquisition unit acquires an object image obtained by capturing an image of an object, The voice data acquisition unit acquires voice data including a voice for confirming the object, the criterion selection unit selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification unit identifies the type of the object based on the identification standard information and the object image.

[0007] The object identification assistance method of the present disclosure includes: The method includes an object information acquiring step, a voice data acquiring step, a criterion selecting step, and an object identifying step, The object information acquisition step acquires an object image by capturing an image of the object, The voice data acquisition step acquires voice data including a voice for confirming the object, The criterion selection step selects identification criterion information based on the voice data, The identification reference information includes at least one object identification image information, The object identifying step identifies a type of the object based on the identification reference information and the object image.

[0008] The object identification assistance program of the present disclosure is The method includes an object information acquisition step, a voice data acquisition step, a reference selection step, and an object identification step; The object information acquisition step acquires an object image by capturing an image of the object; The voice data acquisition step acquires voice data including a voice for confirming the object, the criterion selection step selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification step includes identifying a type of the object based on the identification reference information and the object image; The present invention is an object identification assistance program for causing a computer to execute each of the above procedures.

[0009] The recording medium of the present disclosure includes: The method includes an object information acquisition step, a voice data acquisition step, a reference selection step, and an object identification step; The object information acquisition step acquires an object image by capturing an image of the object; The voice data acquisition step acquires voice data including a voice for confirming the object, the criterion selection step selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification step includes identifying a type of the object based on the identification reference information and the object image; A computer-readable recording medium having recorded thereon an object identification assistance program for causing a computer to execute each of the above procedures. Effect of the Invention

[0010] According to the present disclosure, an object can be easily identified. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a configuration of an example of an object identification assistance device according to the present disclosure. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of an object identification assistance device according to the present disclosure. [Diagram 3] FIG. 3 is a flowchart showing an example of processing in the object identification assistance device of the present disclosure. [Figure 4]FIG. 4 is a block diagram showing a configuration of an example of an object identification assistance device according to the present disclosure. [Diagram 5] FIG. 5 is a flowchart showing an example of processing in the object identification assistance device of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In the present disclosure, the "object" is not particularly limited and includes, for example, any item. Specific examples of the object include food (dishes), lunch boxes, etc.

[0013] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. The present disclosure is not limited to the following embodiment. In each of the drawings, the same parts are given the same reference numerals. In addition, the description of each embodiment can be mutually incorporated unless otherwise specified, and the configurations of each embodiment can be combined unless otherwise specified.

[0014] [Embodiment 1] Fig. 1 is a block diagram showing an example of the configuration of an object identification support device 10 (hereinafter also referred to as "this device") according to this embodiment. As shown in Fig. 1, this device 10 includes an object information acquisition unit 11, a voice data acquisition unit 12, a criterion selection unit 13, and an object identification unit 14. In addition, this device 10 may include, for example, an input unit, an output unit, a display unit, and / or a storage unit, although these are not shown.

[0015] The device 10 may be, for example, a single device including each of the above-mentioned units, or a device to which each of the above-mentioned units can be connected via a communication line network. The device 10 may be connected to an external device described later via the communication line network. The communication line network is not particularly limited, and a known network may be used, for example, wired or wireless. Examples of the communication line network include an Internet line, a World Wide Web (WWW), a telephone line, a Local Area Network (LAN), a Storage Area Network (SAN), a Delay Tolerant Networking (DTN), a Low Power Wide Area (LPWA), and a Local 5G (L5G). Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), a Local 5G, and a LPWA. Examples of the wireless communication may include a form in which each device directly communicates with each other (Ad Hoc communication), an infrastructure (infrastructure communication), and an indirect communication via an access point. The device 10 may be incorporated into a server as a system, for example. Furthermore, the present device 10 may be, for example, a personal computer (PC, for example, desktop type or notebook type) in which the program of the present disclosure is installed, a smartphone, a tablet terminal, etc. The present device 10 may be in the form of cloud computing or edge computing, for example, in which at least one of the above-mentioned units is on a server and the other units are on a terminal.

[0016] 2 illustrates a block diagram of a hardware configuration of the present device 10. The present device 10 includes, for example, a central processing unit (CPU, GPU, etc.) 101, a memory 102, a bus 103, a storage device 104, an input device 105, an output device 106, a communication device 107, etc. Each part of the present device 10 is connected to each other via the bus 103 by each interface (I / F).

[0017] The central processing unit 101 cooperates with other components through a controller (such as a system controller or an I / O controller) and controls the entire device 10. In the device 10, the central processing unit 101 executes, for example, the program disclosed herein and other programs, and also reads and writes various information. Specifically, for example, the central processing unit 101 functions as an object information acquisition unit 11, a voice data acquisition unit 12, a criterion selection unit 13, and an object identification unit 14. The device 10 may include, as a computing device, other computing devices such as a CPU, a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), or a combination of these.

[0018] The bus 103 can also be connected to, for example, an external device. Examples of the external device include a user terminal, an external storage device (external database, etc.), a printer, an external input device, an external display device, an external imaging device, etc. The present device 10 can be connected to an external network (the above-mentioned communication line network) by, for example, a communication device 107 connected to the bus 103, and can also be connected to other devices via the external network.

[0019] An example of the memory 102 is a main memory (primary storage device). When the central processing unit 101 performs processing, the memory 102 reads various operation programs, such as the program of the present disclosure, stored in the storage device 104 described below, and the central processing unit 101 receives data from the memory 102 and executes the program. The main memory is, for example, a RAM (random access memory). The memory 102 may also be, for example, a ROM (read only memory).

[0020] The storage device 104 is also called an auxiliary storage device, for example, in contrast to the main memory (primary storage device). As described above, the storage device 104 stores an operating program including the program of the present disclosure. The storage device 104 may be, for example, a combination of a recording medium and a drive that reads and writes from the recording medium. The recording medium is not particularly limited, and may be, for example, an internal or external type, such as a hard disk (HD), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, or memory card. The storage device 104 may be, for example, a hard disk drive (HDD) in which the recording medium and the drive are integrated, or a solid state drive (SSD).

[0021] In the present device 10, the memory 102 and the storage device 104 can store various information such as log information, information acquired from an external database (not shown) or an external device, information generated by the present device 10, and information used when the present device 10 executes processing. In this case, the memory 102 and the storage device 104 may store, for example, the above-mentioned information of the user of the present device. Note that at least a part of the information may be stored, for example, in an external server other than the memory 102 and the storage device 104, or may be distributed and stored in a plurality of terminals using a block chain technology or the like. In addition, when the present device 10 includes the storage unit, for example, the memory 102 and the storage device 104 function as the storage unit. The storage unit can store, for example, an object image, voice data, identification criterion information, object attribute information, an object identification model, and the like.

[0022] The device 10 further includes, for example, an input device 105 and an output device 106. The input device 105 may be, for example, a pointing device such as a touch panel, a track pad, or a mouse; a keyboard; an imaging means such as a camera or a scanner; a card reader such as an IC card reader or a magnetic card reader; and an audio input means such as a microphone. The output device 106 may be, for example, a display device such as an LED display or a liquid crystal display; an audio output device such as a speaker; a printer; or the like. In the first embodiment, the input device 105 and the output device 106 are configured separately, but the input device 105 and the output device 106 may be configured as an integrated device, such as a touch panel display.

[0023] Next, an example of the object identification support method of this embodiment will be described with reference to the flowchart of Fig. 3. The object identification support method of this embodiment is implemented as follows, for example, using the device 10 of Fig. 1 or Fig. 2. Note that the object identification support method of this embodiment is not limited to use of the device 10 of Fig. 1 or Fig. 2.

[0024] The object information acquisition unit 11 acquires an object image obtained by capturing an image of an object (S1, object information acquisition step). The object image may be, for example, a still image or a video. When the object image is a video, the video may include, for example, audio data described later. The object information acquisition unit 11 may acquire the object image from, for example, an imaging device that captured the object (for example, a camera, a video camera, an information processing terminal with a camera function (for example, a smartphone, a tablet terminal, a notebook computer, etc.)), or may acquire the object image from a recording medium that stores the object image (for example, various storage devices, server devices, databases, etc.), or may acquire the object image captured by an input device (for example, an imaging device) 105 of the device 10. The object information acquisition unit 11 may further acquire object attribute information of the object, for example. The object attribute information is not particularly limited as long as it is associated information related to the object, and any information may be used. A specific example of the object attribute information is information including at least one selected from the group consisting of the weight, contents, size, shape, and material of the object. The object information acquisition unit 11 may acquire the object attribute information by performing image analysis of the object image, or may acquire the object attribute information from an external sensor. Specific examples of the external sensor include a weight sensor such as a weighing scale, a length measurement sensor, a shape measurement sensor, and a material discrimination sensor. The object information acquisition unit 11 may record the acquired object image and object attribute information in a storage unit of the device 10.

[0025] The voice data acquisition unit 12 acquires voice data including a voice confirmation voice related to the object (S2, voice data acquisition step). The voice data acquisition unit 12 may acquire the voice data from, for example, a microphone (including a smartphone, tablet terminal, notebook computer, etc. equipped with a microphone) set at a manufacturing site of the object, or may acquire the voice data from a recording medium (for example, various storage devices, server devices, databases, etc.) in which the voice data is saved, or may acquire the voice data acquired by an input device (for example, a microphone) 105 of the device 10. When acquiring the voice data from the recording medium, it is preferable that the voice data and the object image are information acquired simultaneously in the recording medium.

[0026] The criterion selection unit 13 selects identification criterion information based on the voice data (S3, criterion selection step). The criterion selection unit 13 can select the identification criterion information by, for example, referring to a criterion selection database in which the voice data and the identification criterion information are linked. In the criterion selection database, for example, the waveform data of the voice data or the name information of the object contained in the voice data is linked to the identification criterion information related to the voice data. The criterion selection unit 13 can select the identification criterion information linked to the voice data by, for example, analyzing the voice data to extract the waveform data of the voice data or the name information of the object contained in the voice data, and searching the extracted waveform data or name information from the criterion selection database. The identification criterion information includes at least one object identification image information. The object identification image information is, for example, an image of a reference object, and is image information that serves as a reference for comparison with an object image by the object identification unit 14 described later. In the criterion selection database, the identification criterion information may be, for example, linked to at least one object attribute information for each object identification image information. The object attribute information is not particularly limited as long as it is, for example, information associated with the object that can be used to identify the object, and may be any information. Specific examples of the object attribute information include, for example, information including at least one selected from the group consisting of the weight, contents, size, shape, and material of the object. In this case, the criterion selection unit 13 may select identification criterion information based on, for example, the voice data and the object attribute information.

[0027] The object identification unit 14 identifies the type of the object based on the identification reference information and the object image (S4, object identification step). The object identification unit 14 can, for example, use a known image analysis method to execute identification of the type of the object based on the identification reference information and the object image. The known image analysis method is not particularly limited, and may be, for example, a method using machine learning or not using machine learning. A specific example of the known image analysis method is, for example, a method of acquiring the feature amount of the object identification image information included in the identification reference information and the feature amount of the object image using a known image analysis library such as OpenCV, and calculating the similarity between images by feature amount matching based on the feature amount. In this case, the object identification unit 14 can determine that the object included in the object image is the reference object indicated by the object identification image information when the similarity between the images exceeds a predetermined threshold value. Furthermore, the object identification unit 14 may identify the type of the object by image analysis using a machine-learned model (also referred to as an object identification model, for example) that has learned the relationship between the object identification image information included in the identification reference information and the object indicated by the object image information. In this case, the object identification unit 14 can identify the type of the object by, for example, selecting a machine-learned model that has learned the object identification image information included in the identification reference information selected in S3, and inputting the object image into the selected machine-learned model. The machine-learned model may be a configuration of the present device 10, or may be a configuration outside the present device 10. In the latter case, the object identification unit 14 can identify the type of object included in the object image by, for example, transmitting the acquired object image to the machine-learned model, which is the external configuration, and receiving a determination result by the machine-learned model.

[0028] The object identification model includes, for example, an input layer for inputting an object image, an output layer for outputting the type of the object, and at least one intermediate layer provided between the input layer and the output layer. The object identification model may be a program module that is a part of artificial intelligence software. An example of the multi-layered network is a neural network. An example of the neural network is a convolution neural network (CNN), but is not limited to CNN, and may be a trained model constructed with other learning algorithms such as a neural network other than CNN, a support vector machine (SVM), a Bayesian network, or a regression tree.

[0029] The object identification model may be, for example, a trained model generated in advance. The trained model may also be a trained model (derived model) retrained using training data for object identification and a trained model that has already been generated. Furthermore, the trained model may be a trained model obtained by transfer learning using a trained model generated using training data for object identification, or a trained model generated by model compression of a trained model generated using training data for object identification.

[0030] Also, when the object attribute information is acquired in S1, the object identification unit 14 may identify the type of the object based on, for example, the identification reference information, the object image, and the object attribute information. In this case, the object identification unit 14 may verify whether the identified object is a reference object by, for example, matching the object attribute information of the identified object with the object attribute information of the reference object included in the identification reference information. In this case, the object identification unit 14 may determine that the object is a reference object when, for example, the object attribute information of the identified object matches the object attribute information of the reference object included in the identification reference information. Also, for example, when the object attribute information of the identified object does not match the object attribute information of the reference object included in the identification reference information, the object identification unit 14 may determine that the object is not a reference object. When the object identification unit 14 determines that the object is not a reference object, for example, it may repeat the process of S3 or S4.

[0031] The present device 10 may, for example, output the identification result by the object identification unit 14. The output may, for example, be output (storage) to the memory 102 or storage device 104 of the present device 10, or may be output to an external device via a communication network. The external device may, for example, be an external storage device, or a device that uses the object identification result data by the present device 10. A specific example of a device that uses the object identification result data is a printing device that prints labels for objects.

[0032] The object identification assistance device of the present disclosure can acquire an object image obtained by capturing an object by an object information acquisition unit, acquire voice data including a voice confirmation voice related to the object by an audio data acquisition unit, select identification reference information including at least one object identification image information based on the audio data by a criterion selection unit, and identify the type of the object based on the identification reference information and the object image by an object identification unit. Therefore, according to the present disclosure, for example, the types of identification reference information required for identifying an object can be reduced, and objects can be easily identified.

[0033] [Embodiment 2] The second embodiment is another example of an object identification assistance device according to the present disclosure.

[0034] FIG. 4 is a block diagram showing an example of the configuration of the object identification support device 10A. As shown in FIG. 4, the object identification support device 10A includes a label output unit 15 in addition to the configuration of the object identification support device 10 of the first embodiment. The hardware configuration of the object identification support device 10A is the same as that of the object identification support device 10 of FIG. 2, except that the central processing unit 101 has the configuration of the object identification support device 10A of FIG. 4 instead of the configuration of the object identification support device 10 of FIG. 1. The process of the label output unit 15 will be described below. The process of the label output unit 15 can be inserted at any position in the flowchart of FIG. 3 described in the first embodiment, but it is preferable to insert it after S4 as shown in FIG. 5.

[0035] First, steps S1 to S4 are executed in the same manner as steps S1 to S4 in the first embodiment.

[0036] The label output unit 15 outputs, for example, label information of the identified object (S11, label output step). The label information is, for example, information for identifying the label of the object. The label information may include, for example, information for printing the label of the object. Specifically, the label information may include, for example, information such as the product name, manufacturing date, ingredients, expiration date, best before date, and manufacturing location. The label output unit 15 can output the label information to, for example, a printer for printing a label.

[0037] The object identification support device of this embodiment outputs label information of an identified object by, for example, a label output unit. Therefore, the object identification support device of this embodiment makes it possible, for example, to print a label for an identified object. When a pre-printed label is attached to a completed product at a product manufacturing site, the attached label may differ from the label that should have been attached. The object identification support device of this embodiment makes it possible to print a label corresponding to an identified object. Therefore, the object identification support device of this embodiment makes it possible, for example, to suppress errors in attaching a label to an object.

[0038] [Embodiment 3] The program of this embodiment is a program for causing a computer to execute each step of the object identification support method described above. Specifically, the program of this embodiment is a program for causing a computer to execute an object information acquisition step, a voice data acquisition step, a criterion selection step, and an object identification step.

[0039] The object information acquisition step acquires an object image by capturing an image of the object; The voice data acquisition step acquires voice data including a voice for confirming the object, the criterion selection step selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification step identifies the type of the object based on the identification reference information and the object image.

[0040] The program of this embodiment can also be said to be a program that causes a computer to function as an object information acquisition procedure, a voice data acquisition procedure, a criterion selection procedure, and an object identification procedure.

[0041] The program of this embodiment can be implemented by using the description of the object identification support device and the object identification support method of the present disclosure. For example, the "procedure" in each of the steps can be read as "processing". The program of this embodiment may be recorded in a computer-readable recording medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples thereof include random access memory (RAM), read-only memory (ROM), hard disk (HD), flash memory (e.g., SSD (Solid State Drive), USB flash memory, SD / SDHC card, etc.), optical disk (e.g., CD-R / CD-RW, DVD-R / DVD-RW, BD-R / BD-RE, etc.), magneto-optical disk (MO), floppy (registered trademark) disk (FD), etc. Furthermore, the object identification assistance program of this embodiment (also referred to as, for example, a programming product or an object identification assistance program product) may be in a form distributed from an external computer, for example. The "distribution" may be, for example, distribution via a communication network, or distribution via a device connected by wire. The object identification assistance program of this embodiment may be installed and executed on the device to which it is distributed, or may be executed without being installed.

[0042] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be appropriately combined with other embodiments.

[0043] <Additional Notes> A part or all of the above-described embodiments can be described as follows, but is not limited to the following. (Appendix 1) The present invention includes an object information acquisition unit, a voice data acquisition unit, a criterion selection unit, and an object identification unit, The object information acquisition unit acquires an object image obtained by capturing an image of an object, The voice data acquisition unit acquires voice data including a voice for confirming the object, the criterion selection unit selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification unit identifies the type of the object based on the identification standard information and the object image. (Appendix 2) The identification standard information is linked to at least one piece of object attribute information for each of the object identification image information, the object information acquisition unit acquires object attribute information of the object in association with the object image; 2. The object identification assistance device according to claim 1, wherein the criterion selection unit selects the identification criterion information based on the voice data and the object attribute information. (Appendix 3) 3. An object identification assistance device as described in claim 2, wherein the object attribute information includes at least one selected from the group consisting of the weight, size, shape, and material of the object. (Appendix 4) A label output unit is included. 4. The object identification assistance device according to claim 1, wherein the label output unit outputs label information of the identified object. (Appendix 5) 5. An object identification assistance device according to any one of claims 1 to 4, wherein the object is a cooking item. (Appendix 6) 6. The object identification assistance device according to claim 5, wherein the food is a lunch box. (Appendix 7) The method includes an object information acquiring step, a voice data acquiring step, a criterion selecting step, and an object identifying step, The object information acquisition step acquires an object image by capturing an image of the object, The voice data acquisition step acquires voice data including a voice for confirming the object, The criterion selection step selects identification criterion information based on the voice data, The identification reference information includes at least one object identification image information, The object identification step includes identifying a type of the object based on the identification standard information and the object image. (Appendix 8) The identification standard information is linked to at least one piece of object attribute information for each of the object identification image information, The object information acquisition step acquires object attribute information of the object in association with the object image, The object identification support method according to claim 7, wherein the criterion selection step selects the identification criterion information based on the voice data and the object attribute information. (Appendix 9) 9. The object identification support method according to claim 8, wherein the object attribute information includes at least one selected from the group consisting of the weight, size, shape, and material of the object. (Appendix 10) A label output process is included. 10. The object identification support method according to any one of appendices 7 to 9, wherein the label output step outputs label information of the identified object. (Appendix 11) 11. The method for supporting object identification according to any one of appendices 7 to 10, wherein the object is a cooking item. (Appendix 12) 12. The object identification assistance method according to claim 11, wherein the food is a lunch box. (Appendix 13) The method includes an object information acquisition step, a voice data acquisition step, a reference selection step, and an object identification step; The object information acquisition step acquires an object image by capturing an image of the object; The voice data acquisition step acquires voice data including a voice for confirming the object, the criterion selection step selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification step includes identifying a type of the object based on the identification reference information and the object image; An object identification assistance program for causing a computer to execute each of the above procedures. (Appendix 14) The identification standard information is linked to at least one piece of object attribute information for each of the object identification image information, the object information acquisition step acquires object attribute information of the object in association with the object image; The object identification assistance program according to claim 13, wherein the criterion selection step selects the identification criterion information based on the voice data and the object attribute information. (Appendix 15) 15. An object identification assistance program according to claim 14, wherein the object attribute information includes at least one selected from the group consisting of the weight, size, shape, and material of the object. (Appendix 16) Includes label output procedures, 16. The object identification assistance program according to any one of appendices 13 to 15, wherein the label output step outputs label information of the identified object. (Appendix 17) 17. The object identification assistance program according to any one of appendixes 13 to 16, wherein the object is a cooked item. (Appendix 18) 18. The object identification assistance program according to claim 17, wherein the food is a lunch box. (Appendix 19) The method includes an object information acquisition step, a voice data acquisition step, a reference selection step, and an object identification step; The object information acquisition step acquires an object image by capturing an image of the object; The voice data acquisition step acquires voice data including a voice for confirming the object, the criterion selection step selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification step includes identifying a type of the object based on the identification reference information and the object image; A computer-readable recording medium having recorded thereon an object identification assistance program for causing a computer to execute each of the above procedures. (Appendix 20) The identification standard information is linked to at least one piece of object attribute information for each of the object identification image information, the object information acquisition step acquires object attribute information of the object in association with the object image; 20. The recording medium of claim 19, wherein the criterion selection step selects the identification criterion information based on the voice data and the target attribute information. (Appendix 21) 21. The recording medium of claim 20, wherein the object attribute information includes at least one selected from the group consisting of the weight, size, shape, and material of the object. (Appendix 22) Includes label output procedures, 22. The recording medium according to any one of appendices 19 to 21, wherein the label output step outputs label information of the identified object. (Appendix 23) 23. The recording medium according to any one of appendices 19 to 22, wherein the object is a cooked product. (Appendix 24) 24. The recording medium according to claim 23, wherein the food is a lunch box. [Industrial Applicability]

[0044] According to the present disclosure, an object can be easily identified. Therefore, the present disclosure is widely useful in fields that utilize object identification using images. [Explanation of symbols]

[0045] 10. Object Identification Assistance Device 11 Object information acquisition unit 12 Audio data acquisition unit 13 Criteria Selection Section 14 Object Identification Unit 15 Label output section 101 Central Processing Unit 102 Memory 103 Bus 104 Storage device 105 Input Device 106 Output Device 107 Communication Devices

Claims

1. The present invention includes an object information acquisition unit, a voice data acquisition unit, a criterion selection unit, and an object identification unit, The object information acquisition unit acquires an object image obtained by capturing an image of an object, The voice data acquisition unit acquires voice data including a voice for confirming the object, the criterion selection unit selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification unit identifies the type of the object based on the identification standard information and the object image.

2. The identification standard information is linked to at least one piece of object attribute information for each of the object identification image information, the object information acquisition unit acquires object attribute information of the object in association with the object image; The object identification assistance device according to claim 1 , wherein the criterion selection unit selects the identification criterion information based on the voice data and the object attribute information.

3. 3. The object identification support device according to claim 2, wherein the object attribute information includes at least one selected from the group consisting of weight, size, shape, and material of the object.

4. A label output unit is included. The object identification assistance device according to claim 1 , wherein the label output unit outputs label information of the identified object.

5. The object identification assistance device according to claim 1 , wherein the object is a cooking item.

6. 6. The object identification assistance device according to claim 5, wherein the food is a boxed lunch.

7. The method includes an object information acquiring step, a voice data acquiring step, a criterion selecting step, and an object identifying step, The object information acquisition step acquires an object image by capturing an image of the object, The voice data acquisition step acquires voice data including a voice for confirming the object, The criterion selection step selects identification criterion information based on the voice data, The identification reference information includes at least one object identification image information, The object identification step includes identifying a type of the object based on the identification standard information and the object image.

8. The identification standard information is linked to at least one piece of object attribute information for each of the object identification image information, The object information acquisition step acquires object attribute information of the object in association with the object image, 8. The object identification assistance method according to claim 7, wherein the criterion selection step selects the identification criterion information based on the voice data and the object attribute information.

9. The method includes an object information acquisition step, a voice data acquisition step, a reference selection step, and an object identification step; The object information acquisition step acquires an object image by capturing an image of the object; The voice data acquisition step acquires voice data including a voice for confirming the object, the criterion selection step selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification step includes identifying a type of the object based on the identification reference information and the object image; An object identification assistance program for causing a computer to execute each of the above procedures.

10. The method includes an object information acquisition step, a voice data acquisition step, a reference selection step, and an object identification step; The object information acquisition step acquires an object image by capturing an image of the object; The voice data acquisition step acquires voice data including a voice for confirming the object, the criterion selection step selects identification criterion information based on the voice data; The identification reference information includes at least one object identification image information, The object identification step includes identifying a type of the object based on the identification reference information and the object image; A computer-readable recording medium having recorded thereon an object identification assistance program for causing a computer to execute each of the above procedures.

Images