Recognition device, terminal device, recognizer construction device, recognizer modification device, construction method, and modification method

Abstract

To reduce a decrease in recognition accuracy even when the number of objects is large.SOLUTION: A recognition device includes a communication part and a control part. The communication part acquires images. The control part is functioned as an object recognizer 23. The object recognizer 23 causes a plurality of classifiers 24 to sequentially classify objects in an image to estimate the objects. The plurality of classifiers 24 include a classifier 24 in a top layer 25 and a plurality of classifiers 24 in a lower layer 26. The plurality of classifiers 24 in the lower layer 26 include classifiers 24lst in a lowest layer in which further subcategory is specific information of the objects. The numbers of layers from the classifier 24 in the top layer 25 to the classifiers 24lst in the lowest layer in at least two objects having different specific information among objects to be estimated by the object recognizer 23 are different.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a recognition device, a terminal device, a recognizer construction device, a recognizer modification device, a construction method, and a modification method. [Background technology]

[0002] In recent years, progress has been made in the development of technology that uses machine learning to recognize objects contained in images.

[0003] For example, Patent Document 1 describes a recognition method that combines general object recognition and specific object recognition. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2011 / 081192 Summary of the Invention [Problem to be solved by the invention]

[0005] There is room for improvement in conventional techniques. For example, the recognition accuracy decreases as the number of objects to be recognized increases.

[0006] In view of the above, an object of the present disclosure is to reduce the decrease in recognition accuracy even when the number of objects to be recognized is large. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, a recognition device according to a first aspect of the present invention comprises: an acquisition unit that acquires an image; a control unit that functions as an object recognizer that estimates objects in the image by sequentially classifying the objects in the image using a plurality of classifiers arranged in a layered structure; the plurality of classifiers include a top-layer classifier that classifies objects appearing in the image into one of a plurality of categories, and a plurality of lower-layer classifiers that classify each category classified by the top-layer classifier into a lower-level category, the lower layer classifiers include a bottom layer classifier in which the lower category to be classified is specific information of the object, At least two objects having different specific information among the objects estimated by the object recognizer have different numbers of layers from the top-layer classifier to the bottom-layer classifier.

[0008] Furthermore, a terminal device according to a second aspect comprises: An imaging unit; an acquisition unit that acquires an image generated by the imaging unit; and a control unit that functions as an object recognizer that estimates an object appearing in the image by having a plurality of classifiers arranged in a plurality of layers classify the object appearing in the image in order, the plurality of classifiers including a top-layer classifier that classifies the object appearing in the image into one of a plurality of categories, and a plurality of lower-layer classifiers that classify the object appearing in the image into a lower category within each of the categories classified by the classifiers in the upper layers, the lower-layer classifiers including a bottom-layer classifier that classifies the lower category into which the lower category is specific information of the object; and a communication unit that assigns the specific information of the object appearing in the image to recognition devices having different numbers of layers from the top-layer classifier to the bottom-layer classifier for at least two objects estimated by the object recognizer, and acquires the specific information of the object appearing in the image; an output device that notifies the specific information;

[0009] Furthermore, a recognizer construction device according to a third aspect comprises: An information processing device that constructs an object recognizer that estimates specific information that identifies an object appearing in an image by sequentially classifying the object using a plurality of classifiers that are layered into a plurality of layers, an acquisition unit that acquires at least an image and the specific information of each of the plurality of objects; a control unit that constructs the plurality of classifiers based on images and identification information of each of the plurality of objects; the plurality of classifiers include a top-level classifier that classifies a category of an object in an image acquired by the acquisition unit based on the image, and a bottom-level classifier that classifies an object belonging to a category classified by the classifier in the higher level into any of the specific information, The control unit determining a category to which each of the plurality of objects belongs based on an initial criterion, and constructing the top-level classifier that classifies the objects into the determined category; constructing a provisional bottom-layer classifier that classifies all objects belonging to each category into the respective specific information based on the images and the specific information of all objects belonging to each category classified by the top-layer classifier; If the provisional lowest-level classifier satisfies a predetermined condition, it is determined to be the lowest-level classifier; Until the provisional lowest-level classifier satisfies predetermined conditions, the following steps are repeated: determining an arbitrary criterion by which all objects determined to belong to any category classified by the classifier in the layer one level above the provisional lowest-level classifier can be classified; determining a lower-level category to which each of the objects belongs based on the arbitrary criterion; replacing the provisional lowest-level classifier with an intermediate-level classifier constructed based on images of all objects belonging to each of the determined lower-level categories and the lower-level categories; and constructing a provisional lowest-level classifier that classifies all objects belonging to each category based on the images and specific information of each of the objects belonging to each category classified by the intermediate-level classifier.

[0010] Furthermore, a recognizer correction device according to a forty-fourth aspect provides: an acquisition unit that acquires at least an image and specific information of a new object; the object recognizer includes an acquisition unit that acquires an image using the new object, and a control unit that functions as an object recognizer that estimates the object in the image by having a plurality of classifiers arranged in a plurality of layers classify the object in the image in order, the plurality of classifiers including a top-layer classifier that classifies the object in the image into one of a plurality of categories, and a plurality of lower-layer classifiers that classify the object in the image into a lower category within each of the categories classified by the classifier in the upper layer, the lower-layer classifiers including a bottom-layer classifier in which the lower category to be classified is specific information of the object, and a control unit that corrects the object recognizer in a recognition device having different numbers of layers from the top-layer classifier to the bottom-layer classifier for at least two objects estimated by the object recognizer, The control unit using the object recognizer to estimate specific information of the new object based on the image; Identifying the lowest-level classifier that classified the specific information; a provisional lowest-level classifier constructed based on the images of all objects classified by the lowest-level classifier and the new object and the specific information is replaced with the lowest-level classifier; If the provisional lowest-level classifier satisfies a predetermined condition, it is determined to be the lowest-level classifier; Until the provisional lowest-level classifier satisfies predetermined conditions, the following steps are repeated: determining an arbitrary criterion by which all objects determined to belong to any category classified by the classifier in the layer one level above the provisional lowest-level classifier can be classified; determining a lower-level category to which each of the objects belongs based on the arbitrary criterion; replacing the provisional lowest-level classifier with an intermediate-level classifier constructed based on images of all objects belonging to each of the determined lower-level categories and the lower-level categories; and constructing a provisional lowest-level classifier that classifies all objects belonging to each category based on the images and specific information of each of the objects belonging to each category classified by the intermediate-level classifier.

[0011] The construction method from the fifth viewpoint is as follows: A method for constructing an object recognizer that estimates specific information for identifying an object appearing in an image by sequentially classifying the object using a plurality of classifiers arranged in a layered structure, the method comprising: acquiring at least an image and specific information for each of the plurality of objects; constructing the plurality of classifiers based on images and identification information of each of the plurality of objects; the plurality of classifiers include a top-level classifier that classifies a category of an object in an image acquired by the acquisition unit based on the image, and a bottom-level classifier that classifies an object belonging to a category classified by the classifier in the higher level into any of the specific information, The step of constructing a plurality of classifiers comprises: determining a category to which each of the plurality of objects belongs based on an initial criterion; constructing a top-level classifier that classifies the objects into the determined categories; A step of constructing a provisional bottom-level classifier that classifies all objects belonging to each category based on the images and the identification information of all objects belonging to each category classified by the top-level classifier; determining the provisional lowest-level classifier as the lowest-level classifier if the provisional lowest-level classifier satisfies a predetermined condition; The method includes repeating the steps of determining an arbitrary criterion by which all objects determined to belong to any category classified by a classifier in a layer one level above the provisional lowest-level classifier can be classified, determining a lower category to which each of the objects belongs based on the arbitrary criterion, and replacing the provisional lowest-level classifier with an intermediate-level classifier constructed based on images of all objects belonging to each of the determined lower-level categories and the lower-level categories, until the provisional lowest-level classifier satisfies a predetermined condition, and constructing a provisional lowest-level classifier that classifies all objects belonging to each category based on the images and specific information of each of the objects belonging to each category classified by the intermediate-level classifier.

[0012] The modification method according to the sixth aspect is as follows: acquiring at least an image and identification information of a new object; an acquisition unit that acquires an image using the new object; and a control unit that functions as an object recognizer that estimates the object in the image by having a plurality of classifiers arranged in a plurality of layers classify the object in the image in order, the plurality of classifiers including a top-layer classifier that classifies the object in the image into one of a plurality of categories, and a plurality of lower-layer classifiers that classify the object in the image into a lower category within each of the categories classified by the classifier in the upper layer, the lower-layer classifiers including a bottom-layer classifier in which the lower category to be classified is specific information of the object; and a step of training the object recognizer in a recognition device having different numbers of layers from the top-layer classifier to the bottom-layer classifier for at least two objects estimated by the object recognizer, The step of training the object recognizer includes: using the object recognizer to estimate specific information of the new object based on the image; A step of identifying the lowest-level classifier that classified the specific information; a step of replacing the lowest-level classifier with a provisional lowest-level classifier constructed based on the images of all objects classified by the lowest-level classifier and the new object and the specific information; determining the provisional lowest-level classifier as the lowest-level classifier if the provisional lowest-level classifier satisfies a predetermined condition; The method includes repeating the steps of determining an arbitrary criterion by which all objects determined to belong to any category classified by a classifier in a layer one level above the provisional lowest-level classifier can be classified, determining a lower category to which each of the objects belongs based on the arbitrary criterion, and replacing the provisional lowest-level classifier with an intermediate-level classifier constructed based on images of all objects belonging to each of the determined lower-level categories and the lower-level categories, until the provisional lowest-level classifier satisfies a predetermined condition, and constructing a provisional lowest-level classifier that classifies all objects belonging to each category based on the images and specific information of each of the objects belonging to each category classified by the intermediate-level classifier. [Effects of the Invention]

[0013] According to the present disclosure, even if the number of objects to be recognized is large, the decrease in recognition accuracy can be reduced. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a configuration diagram showing a schematic configuration of an information processing system having a terminal device including a recognition device according to a first embodiment. [Figure 2] FIG. 2 is a perspective view showing the appearance of the terminal device of FIG. [Figure 3] FIG. 3 is a block diagram showing a schematic configuration of the recognition device of FIG. 2. [Figure 4] 2 is a configuration diagram of an object recognizer in which a control unit functions in the recognition device of FIG. 1. [Figure 5] FIG. 5 is a tree diagram of an object recognition model corresponding to the object recognizer of FIG. 4. [Figure 6] FIG. 2 is a diagram illustrating a schematic configuration of the information processing device in FIG. [Figure 7] 7 is a flowchart for explaining a recognizer construction process executed by the control unit of FIG. 6. [Figure 8] 7 is a flowchart for explaining a correction process executed by the control unit of FIG. 6. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following drawings, the same components are denoted by the same reference numerals.

[0016] 1 , an information processing system 11 having a terminal device 10 including a recognition device according to an embodiment of the present disclosure may be configured to include at least one terminal device 10, a network 12, and an information processing device (recognizer construction device, recognizer correction device) 13. In this embodiment, the information processing system 11 includes a plurality of terminal devices 10. The terminal device 10 and the information processing device 13 may communicate with each other via the network 12.

[0017] The information processing system 11 is applicable to any system that identifies a detection target based on an image of the detection target included in an image. The information processing system 11 is applicable, for example, to a checkout system that identifies a product that is a detection target based on an image. The information processing system 11 will be described below using an example in which it is applied to a checkout system.

[0018] The terminal device 10 may capture an image of a product placed on the terminal device 10 by a purchaser. The terminal device 10 may perform object recognition on the captured image and estimate which product in the store corresponds to an object contained in the image. An object in the image refers to an object depicted in the image. The information processing device 13 may calculate a billing amount based on the estimation results for all products captured on the terminal device 10. The information processing device 13 may notify the terminal device 10 of the billing amount. The terminal device 10 may present the billing amount to the purchaser and request payment of the purchase amount.

[0019] The information processing system 11 performs object recognition using an object recognizer constructed so that the product name can be identified based on an image of the product. A product whose product name can be identified by an object recognizer is also called a registered product. The information processing device 13, functioning as a recognizer construction device, constructs a new object recognizer. The information processing device 13, functioning as a recognizer correction device, corrects the object recognizer for a product other than a product whose product name can be identified by an existing object recognizer so that the product name can be identified based on an image of the product. Details of the terminal device 10 and the information processing device 13 are described below.

[0020] As shown in FIG. 2, the terminal device 10 includes an imaging unit 14, an output device 15, a mounting base 16, a support column 17, and a recognition device 18.

[0021] The imaging unit 14 is fixed, for example, so as to be able to image the entire range of the mounting table 16. The imaging unit 14 is fixed, for example, to a support column 17 extending from the side of the mounting table 16. The imaging unit 14 is fixed, for example, so as to be able to image the entire top surface us of the mounting table 16 and so that its optical axis is perpendicular to the top surface us.

[0022] The imaging unit 14 may be configured to include a visible light or infrared camera. The camera includes an imaging optical system and an imaging element. The imaging optical system includes optical components such as one or more lenses and an aperture. The lens may be of any type regardless of focal length, such as a general lens, a wide-angle lens including a fisheye lens, or a zoom lens with a variable focal length. The imaging optical system forms an image of a subject on the light-receiving surface of the imaging element. The imaging element may be, for example, a charge-coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or a far infrared ray (FIR) camera. The imaging element captures the subject image formed on the light-receiving surface to generate a captured image.

[0023] The output device 15 may be any conventionally known display that displays an image. The display may function as a touch screen, as described below. The output device 15 may be a speaker that notifies information. For example, the output device 15 may notify the recognition result of the object recognition by the recognition device 18. The output device 15 may issue various notifications when a malfunction occurs in the information processing system 11, etc. The output device 15 may notify the success or failure of the determination by the recognition device 18 and whether or not the object recognizer needs to be corrected by voice, etc.

[0024] 3, the recognition device 18 includes a communication unit 19 (acquisition unit), a storage unit 20, and a control unit 21. The recognition device 18 may further include an input unit 22. In the first embodiment, the recognition device 18 is configured as a device separate from the imaging unit 14 and the output device 15, but may be configured integrally with at least one of the imaging unit 14, the mounting base 16, the support column 17, and the output device 15, for example.

[0025] The communication unit 19 includes, for example, a communication module that communicates with the imaging unit 14 via a communication line that may be wired or wireless. The communication unit 19 receives, or in other words, acquires, an image from the imaging unit 14 as a signal. The communication unit 19 may include a communication module that communicates with the output device 15 via a communication line. The communication unit 19 may transmit an image to be displayed to the output device 15 as an image signal. The communication unit 19 may receive a position signal corresponding to a position where a contact is detected on the display surface from the output device 15, which is a display. The communication unit 19 includes a communication module that communicates with the information processing device 13 via the network 12. The communication unit 19 receives parameters for constructing an object recognizer from the information processing device 13. The communication unit 19 may transmit result information corresponding to a confirmed recognition result, which will be described later, to the information processing device 13. The communication unit 19 may receive amount information corresponding to a billing amount from the information processing device 13.

[0026] The input unit 22 is capable of detecting an operation input from a user. The input unit 22 includes at least one input interface capable of detecting an input from a user. The input interface is, for example, a physical key, a capacitance key, a pointing device, a touch screen integrated with a display, a microphone, etc. In this embodiment, the input / output interface is a touch screen using the output device 15.

[0027] The storage unit 20 includes any of semiconductor memory, magnetic memory, and optical memory. Examples of semiconductor memory include RAM (Random Access Memory) and ROM (Read Only Memory). Examples of RAM include SRAM (Static Random Access Memory) and DRAM (Dynamic Random Access Memory). Examples of ROM include EEPROM (Electrically Erasable Programmable Read Only Memory). The storage unit 20 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 20 stores data used in the operation of the recognition device 18 and data obtained by the operation of the recognition device 18. For example, the storage unit 20 stores system programs, application programs, embedded software, and the like. For example, the storage unit 20 stores parameters for constructing an object recognizer obtained from the information processing device 13.

[0028] The control unit 21 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process. The dedicated circuit may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 21 executes processes related to the operation of the recognition device 18 while controlling each part of the recognition device 18. The processes related to the operation of the recognition device 18 include, for example, object recognition.

[0029] The control unit 21 functions as an object recognizer that estimates an object included in an image acquired by the communication unit 19. The control unit 21 may output a recognition result of the object recognition to the output device 15. The control unit 21 may provide the recognition result to the information processing device 13.

[0030] 4, the object recognizer 23 is configured to include a plurality of classifiers 24 arranged in a plurality of layers. The object recognizer 23 performs object recognition by having the plurality of classifiers 24 classify objects in order from the upper layer to the lower layer. The plurality of classifiers 24 includes a classifier 24 in a top layer 25 and classifiers 24 in a plurality of lower layers 26.

[0031] The classifier 24 in the top layer 25 classifies objects appearing in images acquired by the communication unit 19 into one of a plurality of categories. The categories classified by the classifier 24 in the top layer 25 may be categories related to packaging such as bags, boxes, bottles, cans, trays, etc., categories related to product classification information, or categories related to other external features of the objects. Alternatively, the categories classified by the classifier 24 in the top layer 25 may be categories corresponding to features determined by clustering, as described below. The classifier 24 in the top layer 25 may classify objects based on, for example, a deep neural network (DNN), pattern matching, image analysis, etc. Pattern matching is based on image features such as SIFT features, SURF features, or bag of features. Note that since the classifier 24 in the top layer 25 classifies categories rather than identifying objects, it is preferable to employ a classifier suitable for general object recognition, such as a DNN.

[0032] Each classifier 24 in the lower layer 26 classifies each category classified by the classifier 24 in the next higher layer into a lower category. The category classified by the lower layer 26 may be a category other than the category classified by the classifier 24 in the higher layer. The category classified by the lower layer 26 may be a category that classifies the size of the object. The classifier 24 in the lower layer 26 may classify the object based on, for example, DNN, pattern matching, image analysis, etc. Note that the classifier 24 in the lowest layer, which will be described later, lst The classifiers 24 in the lower layers 26 other than the above are for classifying categories rather than identifying objects, so it is preferable to use classifiers suitable for general object recognition such as DNNs.

[0033] The classifier 24 in the lower layer 26 is the lowest layer classifier 24 lst Includes the bottom classifier 24 lst The lowest level category is the identification information for identifying the object. The identification information is, for example, the product name of the object in the terminal device 10 used in the retail store, an ID linked to the product name and managed on the master, etc. lstmay classify objects based on, for example, DNN, pattern matching, image analysis, etc. Note that the lowest-level classifier 24 lst Since the object to be identified is to be identified, it is preferable to employ a classifier suitable for specific object recognition such as pattern matching.

[0034] Among a plurality of objects that can be estimated by the object recognizer 23, in other words, among a plurality of objects used to construct or modify the object recognizer, for at least two objects with different specific information, the classifier 24 in the top layer 25 to the classifier 24 in the bottom layer lst For example, for an object recognized by a classifier 24 in the second layer, the classifier 24 is the lowest classifier 24 lst The number of layers corresponding to the object is two. For an object recognized by the classifier 24 in the third layer overall, the classifier 24 is the lowest classifier 24 lst and the number of layers corresponding to the object is 3. In Fig. 4, the maximum number of layers of the object recognizer 23 is 3, but it may be 4 or more.

[0035] The number of categories into which at least some of the multiple classifiers 24 classify may be equal to or less than a first threshold. The first threshold is set to an arbitrary value that causes the accuracy rate of objects identified by the object recognizer 23 to exceed a predetermined value. The degree of variation in the classified categories of the features used for classification by at least some of the multiple classifiers 24 may be equal to or less than a second threshold. The degree of variation is, for example, the variance of the features of all objects belonging to the classified category. The second threshold is set to an arbitrary value that causes the accuracy rate of objects identified by the object recognizer 23 to exceed a predetermined value.

[0036] Bottom classifier 24 lst The accuracy rate of all objects classified by at least a portion of the classification system may be equal to or greater than a third threshold.

[0037] The classifiers 24 in the top layer 25 may be models built to identify categories of objects in an image, while the classifiers 24 in the lower layers 26 may be models built to further classify the categories identified by the classifiers 24 in the higher layers.

[0038] An example of an object recognition model corresponding to the object recognizer 23 will be described below. As shown in FIG. 5 , the object recognition model 27 has a hierarchical structure including at least a first layer divided by categories of image capture objects and a second layer divided by classes belonging to each category. The first layer corresponds to categories classified by the classifier 24 in the top layer 25 of the object recognizer 23. The second layer corresponds to categories classified by the classifier 24 in the lower layer 26 that is one layer below the top layer 25 of the object recognizer 23. If further subdivision is required for a class in the second layer that belongs to a given category, a third layer of subclasses is provided immediately below the second layer, which is further grouped by the classes belonging to the second layer. The third layer corresponds to categories classified by the classifier 24 in the lower layer 26 that is two layers below the top layer 25 of the object recognizer 23. In the description of the object recognition model 27, for ease of interpretation, the categories in the second layer will be referred to as classes, and the categories in the third layer will be referred to as subclasses.

[0039] The procedure for estimating an object using the object recognizer 23 will be described below using the object recognition model 27. The control unit 21 classifies any object X, which is an object included in an image, into one of a group of categories in a preset first layer using the classifier 24 in the top layer 25. In the illustrated object recognition model 27, the group of categories in the first layer is a classification according to the type of packaging of the object, and includes a category of identical or similar box-types and a category of identical or similar bag-types.

[0040] When object X is classified as a box type, control unit 21 further classifies object X into one of the classes in the second layer belonging to the box type using classifier 24 corresponding to the box type category. In the illustrated object recognition model 27, the classes belonging to the box type may include cookie A and chocolate B, which are specific information.

[0041] When object X is classified as a bag-type object, the control unit 21 uses a classifier 24 corresponding to the bag-type category to further classify object X into one of the classes in the second layer that belong to the bag-type category. Unlike the box-type category, the bag-type category has a huge number of classes, which reduces object recognition accuracy. Therefore, the classes in the bag-type category include classes that are further subdivided by further classification based on similarity based on arbitrarily determined features within the classes, rather than specific information. Subdivision is performed by classifying the classes in the second layer into subclasses in a third layer below the second layer. For example, the classes in the second layer may be classes related to the size of the object, specifically, classes based on whether the area of ​​the bag is large or small compared to an area threshold.

[0042] When object X is classified as a bag type and the bag is large, control unit 21 uses classifier 24 corresponding to the large bag category to further classify object X into one of the subclasses in the third layer belonging to large. The subclasses belonging to large may include rice cracker C and potato chips D, which are specific information. When object X is classified as a bag type and the bag is small, control unit 21 uses classifier 24 corresponding to the small bag category to further classify object X into one of the subclasses in the third layer belonging to small. The subclasses belonging to small may include candy E, which is specific information.

[0043] 6, the information processing device 13 is configured to include a communication unit (acquisition unit) 28 and a control unit 29. The information processing device 13 may further include a storage unit 30. As described above, the information processing device 13 constructs an object recognizer 23 that estimates specific information that identifies an object appearing in an image by causing a plurality of classifiers 24, which are hierarchically organized into a plurality of layers, to perform classification in order.

[0044] The communication unit 28 may include at least one communication module connectable to the network 12. The communication module is, for example, a communication module compatible with communication standards such as a wired LAN (Local Area Network), a wireless LAN, or Wi-Fi. The communication unit 28 may be connected to the network 12 via a wired LAN or the like by the communication module.

[0045] The communication unit 28 may include a communication module capable of communicating with various external devices via, for example, a communication line. The communication module is a communication module that complies with the standards of the communication line. The communication line may be configured to include at least one of a wired and a wireless communication line.

[0046] The communication unit 28 may be directly or indirectly connected to the imaging unit 14 via the network 12 or a communication line. Being indirectly connected to the imaging unit 14 means, for example, that the communication unit 28 is connected to the terminal device 10 that incorporates the imaging unit 14. The communication unit 28 acquires images generated by the imaging unit 14.

[0047] The communication unit 28 may be connected to the input unit directly or indirectly via the network 12 or a communication line. The input unit may be the input unit 22 of the recognition device 18 or an input interface directly connected to the information processing device 13. The communication unit 28 acquires the specific information detected by the input unit.

[0048] When constructing a new object recognizer 23, communication unit 28 acquires at least an image and identification information of each of a plurality of objects to be recognized by object recognizer 23. When modifying an existing object recognizer 23 based on a new object, communication unit 28 acquires at least an image and identification information of the object to be used for the modification.

[0049] When constructing a new object recognizer 23, the communication unit 28 may acquire the feature amounts of each of multiple objects. When modifying an existing object recognizer 23 based on a new object, the communication unit 28 may acquire the feature amounts of the object to be used for the modification. The feature amounts may be generated as information by detection by an input unit or calculation by another information processing device based on an image.

[0050] The communication unit 28 may acquire an instruction for determining a classification criterion for constructing the classifier 24. The instruction for determining the classification criterion is, for example, an instruction for specifying a category to be classified by the classifier 24, as will be described later. When acquiring the instruction, a category corresponding to each of a plurality of objects is also acquired. The specified category is a category determined from the appearance of the object. The instruction may be generated by an operation input by a user detected by the input unit.

[0051] The communication unit 28 may be directly or indirectly connected to the output device 15 via the network 12 or a communication line. Being indirectly connected to the output device 15 means, for example, that the communication unit 28 is connected to a terminal device 10 that includes the output device 15.

[0052] The storage unit 30 includes any of semiconductor memory, magnetic memory, and optical memory. The semiconductor memory is, for example, RAM, ROM, etc. The RAM is, for example, SRAM, DRAM, etc. The ROM is, for example, EEPROM, etc. The storage unit 30 may function as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 30 stores data used in the operation of the information processing device 13 and data obtained by the operation of the information processing device 13. For example, the storage unit 30 stores system programs, application programs, embedded software, etc.

[0053] The control unit 29 is configured to include at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for a specific process. The dedicated circuit may be, for example, an FPGA, an ASIC, or the like. The control unit 29 executes processes related to the operation of the information processing device 13 while controlling each unit of the information processing device 13. The processes related to the operation of the information processing device 13 include, for example, building the object recognizer 23 and correcting the object recognizer 23.

[0054] When the control unit 29 acquires information indicating the recognition result of the object recognition from the terminal device 10, the control unit 29 may read the estimated selling price of the object, which is the recognition result, from the storage unit 30. The control unit 29 may calculate the billing amount by adding up the selling prices of the estimated objects. The control unit 29 may transmit monetary information corresponding to the billing amount to the terminal device 10 to which the information indicating the estimated object has been added.

[0055] The control unit 29 constructs the object recognizer 23 based at least on the image and identification information of the object. Furthermore, the control unit 29 constructs a plurality of classifiers 24 that constitute the object recognizer 23 based at least on the image and identification information of the object. A method for constructing the plurality of classifiers 24 by the control unit 29 will be described below.

[0056] The control unit 29 determines a category to which each of the plurality of objects belongs based on an initial criterion. The control unit 29 may determine the initial criterion based on an instruction for determining a classification criterion acquired by the communication unit 28. The control unit 29 may determine the initial criterion by performing clustering on feature amounts obtained by image analysis of the plurality of objects. For the clustering, at least one of feature amounts generated by image analysis of the images of the plurality of objects and feature amounts acquired by the communication unit 28 may be used.

[0057] The control unit 29 constructs a classifier 24 in the top layer 25 that classifies objects into the determined category. The control unit 29 may construct the classifier 24 in the top layer 25 by any method. The any method is, for example, a statistical method such as machine learning or non-machine learning. For example, the control unit 29 performs machine learning on an image of the object using the category determined for the object based on an initial criterion as annotation data. Alternatively, the control unit 29 performs non-machine learning using a combination of the category determined for the object based on the initial criterion and an image of the object.

[0058] The control unit 29 calculates the provisional classification data for the lowest-level classifier 24 based on the images and specific information of all objects belonging to each category classified by the classifier 24 in the top layer 25. vlst The control unit 29 constructs the provisional lowest-level classifier 24 by any method. vlst The arbitrary method may be a statistical method such as machine learning or non-machine learning. vlst classifies all objects belonging to each category classified by the classifier 24 in the top layer 25 into their respective specific information.

[0059] The control unit 29 calculates the provisional lowest-level classifier 24 vlst satisfies a predetermined condition, the provisional lowest-level classifier 24 vlst The bottom classifier 24 lst It is confirmed as follows.

[0060] The predetermined condition is the provisional bottom classifier 24 vlst This is a condition under which it can be determined that further subdivision of the categories classified by is unnecessary.

[0061] The predetermined condition is the provisional bottom classifier 24 vlst The third condition may be that the accuracy rate of the objects classified as "1" is equal to or greater than a third threshold. The accuracy rate may be calculated based on the verification image and the identification information for each object.

[0062] Alternatively, the predetermined condition may be at least one of a first condition and a second condition. The first condition is a condition for determining whether a provisional bottom-level classifier is a vlstThe second condition is that the number of specific information classified by the provisional bottom classifier is less than or equal to the first threshold. vlst The provisional lowest-level classifier of the feature used for classification by the classifier 24 in the next higher level vlst The degree of variation in the category corresponding to is equal to or less than the second threshold.

[0063] The control unit 29 selects the provisional lowest-level classifier 24 when the predetermined condition is the first condition. vlst and the accuracy rate of the provisional lowest-level classifier 24 when the predetermined condition is the second condition. vlst The correct answer rate of the object classified as lst This may be applied to determining whether a certainty exists.

[0064] The control unit 29 determines whether the first condition is satisfied by the provisional lowest-level classifier 24 vlst From the accuracy rate of the objects classified as lst If the correct answer rate of the object to be classified by the classifier 24 decreases, the control unit 29 determines whether the correct answer rate of the object to be classified by the classifier 24 decreases. ... lst The construction of the intermediate layer classifier 24 will be explained later.

[0065] The control unit 29 determines whether the provisional lowest-level classifier 24 satisfies the second condition. vlst From the accuracy rate of the objects classified as lst If the correct answer rate of the object to be classified by the classifier 24 decreases, the control unit 29 determines whether the correct answer rate of the object to be classified by the classifier 24 decreases. ... lst You may stop building the

[0066] The control unit 29 may use any one of the first condition, the second condition, and the third condition described above for each classifier 24. Therefore, the control unit 29 may use the classifiers 24 in the middle layer and the classifiers 24 in the bottom layer that are constructed to comply with the first condition. lst , the intermediate layer classifier 24 and the bottom layer classifier 24 constructed to satisfy the second condition lst , the bottom classifier 24 constructed to satisfy the third condition lst may be constructed to simultaneously include an object recognizer 23.

[0067] The control unit 29 calculates the provisional lowest-level classifier 24 vlst The intermediate layer classifier 24 is constructed and the provisional bottom layer classifier vlst to the intermediate layer classifier 24, and the temporary lowest layer classifier 24 vlst Repeat the construction of the

[0068] The control unit 29 uses the provisional bottom-layer classifier to construct the intermediate-layer classifier 24. vlst The control unit 29 determines an arbitrary criterion by which all objects determined to belong to an arbitrary category classified by the classifier 24 in the next higher layer can be classified. The control unit 29 may determine the arbitrary criterion based on an instruction for determining the classification criterion acquired by the communication unit 28. The control unit 29 may determine the arbitrary criterion by performing clustering on feature quantities of multiple objects obtained by image analysis. For the clustering, at least one of feature quantities generated by image analysis of images of the multiple objects and feature quantities acquired by the communication unit 28 may be used. For example, as described above, the arbitrary criterion may be determined as the feature quantity, which is the size of the object. The size of the object may be calculated as the area ratio of the area occupied by the object to the entire area of ​​the image. A statistical value such as the average or median of the object size may be used as the arbitrary criterion for classification.

[0069] The control unit 29 generates a provisional lowest-level classifier based on the determined arbitrary criteria. vlstThe classifier 24 in the next higher layer determines a lower category to which each of the objects determined to belong to any category classified by the classifier 24 in the next higher layer belongs. The determination of the lower category may be based on the category of each of the objects obtained together with the features or instructions used for clustering.

[0070] The control unit 29 constructs the intermediate layer classifier 24 based on the images of all objects belonging to the determined lower category and the determined lower category. The control unit 29 may construct the intermediate layer classifier 24 using any method. The any method is, for example, a statistical method such as machine learning or non-machine learning. For example, the control unit 29 performs machine learning on the image of the object using the lower category determined for the object based on any criterion as annotation data. Alternatively, the control unit 29 performs non-machine learning using a combination of the lower category determined for the object based on any criterion and the image of the object.

[0071] The control unit 29, in conjunction with the construction of the intermediate-layer classifier 24, creates a provisional lowest-layer classifier 24 that classifies all objects belonging to each category classified by the intermediate-layer classifier 24 into the respective specific information. vlst The control unit 29 constructs a provisional bottom-level classifier 24 based on the images and specific information of all objects belonging to each category classified by the intermediate-level classifier 24. vlst Build.

[0072] The control unit 29 may store the parameters for constructing the object recognizer 23 constructed as described above in the storage unit 30. The control unit 29 may provide the parameters for constructing the object recognizer 23 to the terminal device 10 via the communication unit 28.

[0073] When the control unit 29 acquires an image and identification information of a new object for the existing object recognizer 23 via the communication unit 28, the control unit 29 modifies the object recognizer 23 using the new object. Modification of the object recognizer 23 using the new object will be described below. The object recognizer 23 may be an object recognizer 23 whose parameters are stored in the storage unit 30, or may be an object recognizer 23 that corresponds to parameters generated by an external information processing device and acquired via the communication unit 28.

[0074] The control unit 29 estimates specific information from the acquired image by object recognition using the object recognizer 23. The control unit 29 estimates specific information from the lowest-level classifier 24 that classified the specific information. lst Identify.

[0075] The control unit 29 determines the lowest classifier 24 lst Based on the images and specific information of all objects classified by vlst The control unit 29 constructs the provisional lowest-level classifier 24 by any method. vlst The arbitrary method may be, for example, a statistical method such as machine learning or non-machine learning. The control unit 29 may construct the specified lowest-level classifier 24 lst The newly constructed provisional bottom-level classifier 24 vlst Replace with.

[0076] The control unit 29 calculates the provisional lowest-level classifier 24 vlst satisfies a predetermined condition, the provisional lowest-level classifier 24 vlst The bottom classifier 24 lst The predetermined condition is that when a new object recognizer 23 is constructed, the provisional lowest-level classifier 24 vlst The bottom classifier 24 lst The predetermined condition may be the same as the predetermined condition for determining the above.

[0077] The control unit 29 calculates the provisional lowest-level classifier 24 vlst The intermediate layer classifier 24 is constructed and the provisional bottom layer classifier vlstto the intermediate layer classifier 24, and the temporary lowest layer classifier 24 vlst The construction of the intermediate layer classifier 24 will be explained later.

[0078] The control unit 29 uses the provisional bottom-layer classifier to construct the intermediate-layer classifier 24. vlst The control unit 29 determines an arbitrary criterion by which all objects determined to belong to an arbitrary category classified by the classifier 24 in the next higher layer can be classified. The control unit 29 may determine the arbitrary criterion based on an instruction for determining the classification criterion acquired by the communication unit 28. The control unit 29 may determine the arbitrary criterion by performing clustering on feature quantities of the plurality of objects. For the clustering, at least one of feature quantities generated by image analysis of images of the plurality of objects and feature quantities acquired by the communication unit 28 may be used.

[0079] The control unit 29 generates a provisional lowest-level classifier based on the determined arbitrary criteria. vlst The classifier 24 in the next higher layer determines a lower category to which each of the objects determined to belong to any category classified by the classifier 24 in the next higher layer belongs. The determination of the lower category may be based on the category of each of the objects obtained together with the features or instructions used for clustering.

[0080] The control unit 29 constructs the intermediate layer classifier 24 based on the images of all objects belonging to the determined lower category and the determined lower category. The control unit 29 may construct the intermediate layer classifier 24 using any method. The any method is, for example, a statistical method such as machine learning or non-machine learning. For example, the control unit 29 performs machine learning on the image of the object using the lower category determined for the object based on any criterion as annotation data. Alternatively, the control unit 29 performs non-machine learning using a combination of the lower category determined for the object based on any criterion and the image of the object.

[0081] The control unit 29, in conjunction with the construction of the intermediate-layer classifier 24, creates a provisional lowest-layer classifier 24 that classifies all objects belonging to each category classified by the intermediate-layer classifier 24 into the respective specific information. vlst The control unit 29 constructs a provisional bottom-level classifier 24 based on the images and specific information of all objects belonging to each category classified by the intermediate-level classifier 24. vlst Build.

[0082] The control unit 29 may store the parameters for constructing the object recognizer 23 modified as described above in the storage unit 30. The control unit 29 may provide the parameters for constructing the object recognizer 23 to the terminal device 10 via the communication unit 28.

[0083] Next, the recognizer construction process executed by the control unit 29 of the information processing device 13 in this embodiment will be described with reference to the flowchart in Fig. 7. The recognizer construction process starts when an execution command for constructing a new recognizer is received via the communication unit 28 of the information processing device 13.

[0084] In step S100, the control unit 29 acquires at least an image and specific information of each of all objects to be recognized by the object recognizer to be constructed. To acquire the image and specific information, the control unit 29 may cause the output device 15 of the terminal device 10 to output an input request for the image and specific information. After acquiring at least the image and specific information, the process proceeds to step S101.

[0085] In step S101, the control unit 29 determines the category to which each of the plurality of objects belongs based on the initial criteria. After the determination, the process proceeds to step S102.

[0086] In step S102, the control unit 29 constructs the classifier 24 in the top layer 25 so as to classify the object into the category determined in step S101. After construction, the process proceeds to step S103.

[0087] In step S103, the control unit 29 controls the provisional classifier 24 in the lowest layer to classify the objects belonging to each category classified by the classifier 24 in the top layer 25 into specific information. vlst After construction, the process proceeds to step S104.

[0088] In step S104, the control unit 29 determines whether the classifier 24 in the bottom layer is a provisional classifier in the undetermined state. vlst After the selection, the process proceeds to step S105.

[0089] In step S105, the control unit 29 selects the provisional lowest-level classifier 24 selected in step S104. lst The predetermined condition is, as described above, whether the provisional lowest-level classifier 24 vlst The accuracy rate of the object classified into is equal to or greater than the third threshold, and the provisional lowest-level classifier vlst the number of specific information classified by the provisional lowest-level classifier is equal to or less than a first threshold, and vlst The provisional lowest-level classifier of the feature used for classification by the classifier 24 in the next higher level vlst The degree of variation in the category corresponding to the predetermined condition is equal to or less than a second threshold. If the predetermined condition is not met, the process proceeds to step S106. If the predetermined condition is met, the process proceeds to step S108.

[0090] In step S106, the control unit 29 determines the category to which each of the plurality of objects belongs based on an arbitrary criterion. After the determination, the process proceeds to step S107.

[0091] In step S107, the control unit 29 constructs the intermediate-layer classifiers 24 so as to classify the objects into the categories determined in step S106. Furthermore, the control unit 29 constructs the provisional lowest-layer classifiers 24 so as to classify the objects belonging to the categories classified by the intermediate-layer classifiers 24 into specific information. vlst After construction, the process returns to step S104.

[0092] In step S108, which is reached when the accuracy rate is equal to or greater than the third threshold in step S105, the control unit 29 performs the following steps: vlst The bottom classifier 24 lst After the determination, the process proceeds to step S109.

[0093] In step S109, the control unit 29 calculates all the provisional lowest-level classifiers vlst is the lowest classifier lst It is determined whether all the provisional bottom-level classifiers 24 vlst If not, the process returns to step S104. vlst If it is determined, the process proceeds to step S110.

[0094] In step S110, the control unit 29 stores the parameters for constructing the object recognizer 23 formed by all the classifiers 24 in the storage unit 30. Furthermore, the control unit 29 controls the communication unit 28 to provide the parameters for constructing the object recognizer 23 to the terminal device 10. After the parameters are stored and provided, the recognizer construction process ends.

[0095] Next, the correction process executed by the control unit 29 of the information processing device 13 of this embodiment will be described with reference to the flowchart in Fig. 8. The correction process starts when an execution command for correcting the recognizer is received via the communication unit 28 of the information processing device 13.

[0096] In step S200, the control unit 29 acquires the existing object recognizer 23 to be modified. The existing object recognizer 23 may be acquired by reading parameters from the storage unit 30. After acquisition, the process proceeds to step S201.

[0097] In step S201, the control unit 29 acquires at least an image and specific information of the new object for the existing object recognizer 23 acquired in step S200. To acquire the image and specific information, the control unit 29 may cause the output device 15 of the terminal device 10 to output an input request for the image and specific information. After acquiring at least the image and specific information, the process proceeds to step S202.

[0098] In step S202, the control unit 29 uses the object recognizer 23 acquired in step S200 to estimate each new object acquired in step S201 based on the image of the object. Furthermore, the control unit 29 uses the lowest-level classifier 24 that estimated the object, in other words, classified the object into specific information. lst After the identification, the process proceeds to step S203.

[0099] In step S203, the control unit 29 determines whether the lowest-level classifier 24 specified in step S202 lst Based on the images and specific information of all objects classified by

[0049] and each new object, a provisional bottom-level classifier 24 vlst After construction, the process proceeds to step S204.

[0100] In step S204, the control unit 29 determines whether the lowest-level classifier 24 specified in step S202 lst The tentative lowest-level classifier 24 newly constructed in step S203 vlst After the replacement, the process proceeds to step S205.

[0101] In steps S205 to S211, the control unit 29 performs the same processes as steps S104 to S110 in the recognizer construction process. After the parameters are stored and assigned in step S211, the correction process ends.

[0102] The recognition device 18 of this embodiment configured as described above includes a control unit 21 that functions as an object recognizer 23 that estimates an object appearing in an image by having a plurality of classifiers 24 arranged in a layered structure in a plurality of layers classify the object appearing in the image in order, and the plurality of classifiers 24 include a classifier 24 in a top layer 25 that classifies the object appearing in the image into one of a plurality of categories, and a plurality of classifiers 24 in a lower layer 26 that classify the object appearing in the image into a lower category within each category classified by the classifier 24 in the next higher layer, and the classifier 24 in the lower layer 26 classifies the object into a lower category by the classifier 24 in the lowest layer 24, which classifies the object into specific information of the object. lst and a classifier 24 in the top layer 25 to a classifier 24 in the bottom layer for at least two objects among the objects estimated by the object recognizer 23. lst In an object recognizer that estimates an object by classifying objects in order using hierarchical classifiers, as in Patent Document 1, when the number of categories to be classified increases, objects may be concentrated in some categories. In a classifier, the classification accuracy generally decreases as the number of categories to be classified increases. Therefore, as described above, in the object recognition described in Patent Document 1, when objects are concentrated in some categories, the classification accuracy of the classifier that classifies specific information of objects belonging to those categories may decrease. On the other hand, the recognition device 18 having the above configuration estimates objects by classifying objects in order using hierarchical classifiers, as in Patent Document 1, when objects are concentrated in some categories, the classification accuracy of the classifier that classifies specific information of objects belonging to those categories may decrease. lst Since the number of layers up to the end of the hierarchy is different, the lowest-level classifier 24 lst and a classifier 24 that classifies objects into further lower categories. Therefore, the recognition device 18 can reduce the concentration of objects in any category in each classifier 24. As a result, even if the number of objects is large, the recognition device 18 can suppress a decrease in classification accuracy in each classifier 24 and reduce a decrease in object recognition by the object recognizer 23.

[0103] In the recognition device 18 of this embodiment, the number of categories classified by at least some of the multiple classifiers 24 is equal to or less than a first threshold. As described above, generally, the more categories a classifier classifies, the lower the classification accuracy. In such a case, the recognition device 18 having the above-described configuration can suppress the decrease in classification accuracy.

[0104] In the recognition device 18 of this embodiment, the degree of variation in the classified category of the features used for classification by at least some of the multiple classifiers 24 is below a second threshold. Generally, within a classified category, variation in the features used for classification indicates room for further classification with high accuracy. In such cases, the recognition device 18 having the above configuration can function as an object recognizer 23 composed of classifiers 24 with high classification accuracy.

[0105] The information processing device 13 configured as described above and functioning as a recognizer construction device of this embodiment constructs a classifier 24 in the top layer 25 that determines the category to which each of a plurality of objects belongs based on an initial criterion and classifies the objects into the determined category, and constructs a provisional classifier 24 in the bottom layer 24 that classifies all objects belonging to each category into the respective specific information based on the images and specific information of all objects belonging to each category classified by the classifier 24 in the top layer 25. vlst and construct a provisional bottom-level classifier 24 vlst If satisfies a predetermined condition, the lowest classifier 24 lst The provisional bottom-level classifier 24 vlst The provisional lowest-level classifier 24 vlst The intermediate classifier 24 is constructed based on the images of all objects belonging to each of the determined lower categories and the determined lower categories, and the intermediate classifier 24 is constructed based on the images of all objects belonging to each of the determined lower categories and the determined lower categories. vlstand a provisional bottom-level classifier 24 that classifies all objects belonging to each category into their respective specific information based on the images and specific information of all objects belonging to each category classified by the intermediate-level classifier 24. vlst This configuration allows the information processing device 13 to improve the classification accuracy of each classifier 24 even when the number of objects becomes enormous. Therefore, the information processing device 13 is configured with classifiers 24 with high classification accuracy, and therefore can construct an object recognizer 23 with high recognition accuracy.

[0106] In the information processing device 13 functioning as the recognizer construction device of this embodiment, the predetermined condition is the provisional lowest-level classifier 24 vlst The accuracy rate of objects classified as "1" is equal to or higher than a third threshold. With this configuration, the information processing device 13 can construct an object recognizer 23 with further improved recognition accuracy.

[0107] In the information processing device 13 functioning as the recognizer construction device of this embodiment, the predetermined condition is the provisional lowest-level classifier 24 vlst a first condition that the number of specific information classified by the provisional lowest-level classifier 24 is equal to or less than a first threshold; vlst The provisional lowest-level classifier 24 of the feature used for classification by the classifier 24 in the next higher level vlst The second condition is that the degree of variation in the category corresponding to the first category is equal to or less than a second threshold. With this configuration, the information processing device 13 can construct an object recognizer 23 with further improved recognition accuracy.

[0108] The information processing device 13 functioning as the recognizer construction device of this embodiment generates a provisional lowest-level classifier 24 when the predetermined condition is the first condition. vlst and the accuracy rate of the provisional lowest-level classifier 24 when the predetermined condition is the second condition. vlst The lowest classifier 24 lst With this configuration, the information processing device 13 can construct an object recognizer 23 with further improved recognition accuracy.

[0109] The information processing device 13 functioning as the recognizer construction device of this embodiment is the lowest-level classifier 24 after satisfying the first condition. lst The accuracy rate of the object classified by the provisional lowest classifier 24 before satisfying the first condition is vlst If the accuracy rate of the object classified as 1 falls below the accuracy rate of the object classified as 2, the intermediate layer classifier 24 is constructed to satisfy the first condition, and the lowest layer classifier 24 lst With this configuration, the information processing device 13 prevents a decrease in classification accuracy, and therefore prevents a decrease in recognition accuracy in the object recognizer 23 to be constructed.

[0110] The information processing device 13 functioning as the recognizer construction device of this embodiment is the lowest-level classifier 24 after satisfying the second condition. lst The accuracy rate of the object classified by the provisional lowest classifier 24 before satisfying the second condition is vlst If the accuracy rate of the object classified as lst With this configuration, the information processing device 13 prevents a decrease in classification accuracy, and therefore prevents a decrease in recognition accuracy in the object recognizer 23 to be constructed.

[0111] The information processing device 13, which functions as a recognizer construction device of this embodiment, determines the initial criteria and at least a part of the arbitrary criteria through clustering. With this configuration, the information processing device 13 can determine appropriate criteria for the actually obtained image. Therefore, the information processing device 13 can construct an object recognizer 23 with further improved recognition accuracy.

[0112] In the information processing device 13 functioning as the recognizer construction device of this embodiment, the communication unit 28 further acquires instructions for determining the classification criteria, and the control unit 29 determines the initial criteria and at least a part of the arbitrary criteria based on the instructions acquired by the communication unit 28. With this configuration, the information processing device 13 can determine the criteria that the user determines to be applicable as the classification criteria.

[0113] The information processing device 13 functioning as the recognizer correction device of this embodiment configured as described above uses the object recognizer 23 to estimate specific information of a new object based on an image of the new object, and generates the lowest-level classifier 24 that classifies the specific information. lst and the lowest classifier 24 lst A provisional bottom-level classifier 24 is constructed based on the images and specific information of all objects classified by vlst The lowest classifier 24 lst Replace with the provisional bottom classifier 24 vlst If the condition is satisfied, the lowest classifier 24 lst The provisional bottom-level classifier 24 vlst The provisional lowest-level classifier 24 vlst The intermediate classifier 24 is constructed based on the images of all objects belonging to each of the determined lower categories and the determined lower categories, and the intermediate classifier 24 is constructed based on the images of all objects belonging to each of the determined lower categories and the determined lower categories. vlst and a provisional bottom-level classifier 24 that classifies all objects belonging to each category into their respective specific information based on the images and specific information of all objects belonging to each category classified by the intermediate-level classifier 24. vlst With this configuration, the information processing device 13 can correct the existing hierarchical object recognizer 23 so as to perform object recognition of a new target object while reducing the degradation of recognition accuracy.

[0114] The above has described embodiments of the recognition device 18 and the information processing device 13 that functions as a recognizer construction device and a recognizer correction device. However, embodiments of the present disclosure can also be embodied as a method or program for implementing the device, or as a storage medium on which a program is recorded (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card).

[0115] Furthermore, the implementation form of the program is not limited to application programs such as object code compiled by a compiler or program code executed by an interpreter, but may also be in the form of a program module incorporated into an operating system. Furthermore, the program may or may not be configured so that all processing is performed solely by the CPU on the control board. The program may also be configured so that part or all of it is executed by another processing unit mounted on an expansion board or expansion unit added to the board as needed.

[0116] The drawings illustrating the embodiments of the present disclosure are schematic, and the dimensional ratios and the like in the drawings do not necessarily correspond to the actual ones.

[0117] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art could make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications or alterations are included in the scope of the present disclosure. For example, the functions included in each component can be rearranged so as not to cause logical inconsistencies, and multiple components can be combined or divided into one.

[0118] For example, in this embodiment, the information processing device 13 that functions as a recognizer construction device and a recognizer correction device is configured to construct and correct the object recognizer 23, but the recognizer construction device and the recognizer correction device may be separate devices.

[0119] Furthermore, in this embodiment, the information processing device 13, which functions as a recognizer construction device and a recognizer correction device, constructs and corrects the object recognizer 23, and the terminal device 10 is configured to include a recognition device 18. However, the information processing device 13 may not only construct and correct the object recognizer 23, but also function as a recognition device. In such a configuration, for example, the terminal device 10 may have a communication unit that provides the information processing device 13 with an image generated by the imaging unit 14 and acquires specific information that is the recognition result by the object recognizer 23 in the information processing device 13.

[0120] In this embodiment, the information processing device 13 that functions as a recognizer construction device constructs the object recognizer 23 by using the provisional lowest-level classifier 24 vlst The intermediate layer classifier 24 is replaced with the temporary lowest layer classifier 24. vlst The provisional bottom-layer classifier 24, including the top-layer classifier 25, is not limited to vlst A higher layer classifier 24 may be substituted for a middle layer classifier 24 .

[0121] All of the features described in this disclosure and / or all steps of all of the disclosed methods or processes may be combined in any combination except combinations in which these features are mutually exclusive. Furthermore, each feature described in this disclosure may be replaced by an alternative feature serving the same, equivalent, or similar purpose, unless expressly denied. Thus, unless expressly denied, each disclosed feature is only one example of a generic series of identical or equivalent features.

[0122] Furthermore, embodiments of the present disclosure are not limited to the specific configurations of any of the above-described embodiments, but rather extend to any novel feature or combination thereof described herein, or any novel method or process step or combination thereof described herein.

[0123] In this disclosure, descriptions such as "first" and "second" are identifiers for distinguishing the configuration. In this disclosure, the configurations distinguished by descriptions such as "first" and "second" can have their numbers interchanged. For example, the first threshold and the second threshold can have their identifiers "first" and "second" interchanged. The identifiers are interchanged simultaneously. The configurations remain distinguished even after the identifiers are interchanged. The identifiers may be deleted. A configuration from which the identifiers have been deleted is distinguished by a symbol. The identifiers "first" and "second" in this disclosure should not be used solely to interpret the order of the configurations or to justify the existence of an identifier with a smaller number. [Explanation of symbols]

[0124] 10 Terminal Equipment 11 Information Processing Systems 12 Network 13 Information processing equipment 14 Imaging unit 15 Output Devices 16 Mounting table 17 Support pillar 18 Recognition device 19 Communications Department 20 Memory section 21 Control Unit 22 Input section 23 Object Recognizer 24 Classifier twenty four lst Bottom-level classifier twenty four vlst A provisional bottom-level classifier 25 Top Floor 26 Lower Class 27 Object Recognition Model 28 Communications Department 29 Control Unit 30 Storage section

Claims

1. A recognition device that estimates an object appearing in an image using an object recognizer including a top-layer classifier, at least one bottom-layer classifier, and an intermediate-layer classifier located between the top-layer classifier and the bottom-layer classifier, the top-layer classifier classifies the object into one of a plurality of first categories based on a feature amount of the object appearing in the image; at least one classifier in the intermediate layer further classifies the objects classified into each of the first categories by the classifier in the uppermost layer into any one of second categories based on the feature amount; determining whether to classify the image into the first category or the second category based on the feature amount; The lowest-level classifier includes a control unit that determines, based on the feature amount, specific information for identifying the object classified into categories by the higher-level intermediate-level classifier. recognition device.

2. The control unit causes one of the classifiers to classify the object in the first category or the second category. The recognition device according to claim 1 .

3. The first category and the second category are determined by performing clustering on feature amounts of the object obtained by performing image analysis on the image of the object. The recognition device according to claim 1 .

4. an imaging unit that captures an image and generates an image; a communication unit that transmits the image to the recognition device according to claim 1 and receives a recognition result of the image from the recognition device. electronic equipment.

5. A display unit for displaying the output result of the recognition device according to any one of claims 1 to 3 is provided. Information processing system.

6. A recognition device that estimates an object appearing in an image using an object recognizer including a top-layer classifier, at least one bottom-layer classifier, and an intermediate-layer classifier located between the top-layer classifier and the bottom-layer classifier, classifying the object captured in the image into one of a plurality of first categories based on a feature amount of the object by the top-layer classifier; further classifying the objects classified into each of the first categories by the top-layer classifier based on the feature amount into any one of the second categories by at least one classifier in the intermediate layer; determining whether to classify the object into one of the first categories or one of the second categories based on the feature amount of the object; and identifying, by the classifier in the lowest layer, specific information for identifying the object classified into categories by the classifier in the higher intermediate layer, based on the feature amount. program.

Images