Article discrimination system

Abstract

The item identification system includes: an identification information acquisition unit, an image acquisition unit for photographing the appearance of the target item, a determination unit for determining the consistency between the photographed image of the target item and the appearance image data stored in the storage unit, and determining whether there is a normal state where the consistency of the appearance image data is above a determination threshold, and a learning processing unit for performing learning processing when the determination unit determines that it is not a normal state. The learning processing is the process of storing the data of the photographed image as new appearance image data in the storage unit in association with the identification information acquired by the identification information acquisition unit. The storage unit stores two or more predetermined numbers of appearance image data in the storage unit in ascending order of the time points in the storage unit, associated with one piece of identification information.

Description

Technical Field

[0001] This invention relates to an article identification system that uses image data of the appearance of an article. Background Technology

[0002] Japanese Patent Application Publication No. 2015-43175 (Patent Document 1) discloses an example of the article identification system described above. Hereinafter, the symbols shown in parentheses in the background description are those of Patent Document 1. In Patent Document 1, an article identification system is applied in an article identification device (3) that identifies the position of an article (50) being taken out of a container (51). This article identification device (3) uses a template image including the appearance of the outer surface of the article (50) to identify the position of the article (50). Specifically, the article identification device (3) identifies the position of the article (50) by detecting an area (M) in an image of the container (51) containing the article (50) that has a consistency with the template image of a predetermined threshold. Thus, Patent Document 1 discloses an article identification system that uses a template image as appearance image data. Summary of the Invention

[0003] However, regarding the appearance design of articles, such as the shape of the outer surface, there are cases where even articles of the same type can change. For example, in the case of an article being a container for holding goods (e.g., a cardboard box for holding food), even for containers of the same type (specifically, containers for holding the same type of goods), the design of the container's outer surface (packaging design) may change depending on the season or promotional activities such as sales. In order to make appropriate identification in the article identification system, when the appearance design of an article changes, appearance image data corresponding to the changed appearance design is required. Therefore, it is considered to generate and register new appearance image data every time the appearance design of an article changes. However, in this case, since the frequency of generating and registering appearance image data is very high, it may lead to an increase in the labor and time of the operator using the article identification system and a decrease in the processing speed of the device using the article identification system (article identification device in Patent Document 1).

[0004] Therefore, it is desirable to implement an item discrimination system that can suppress the frequency of generation and registration of appearance image data.

[0005] The disclosed item identification system comprises: a storage unit for storing appearance image data of an item, i.e., appearance image data, associated with the item's identification information; an identification information acquisition unit for acquiring the identification information of the item to be identified, i.e., the target item; an imaging unit for capturing the appearance of the target item; a determination unit for deriving the consistency between the captured image of the target item captured by the imaging unit and the appearance image data stored in the storage unit associated with the identification information acquired by the identification information acquisition unit, and determining whether there is a normal state where the consistency of the appearance image data is above a determination threshold; and a learning processing unit for performing learning processing when the determination unit determines that it is not the normal state, wherein the learning processing is a process of storing the captured image data as new appearance image data associated with the identification information acquired by the identification information acquisition unit in the storage unit, and the storage unit stores two or more predetermined numbers of appearance image data associated with one piece of identification information in the storage unit in order from newest to oldest storage time.

[0006] According to this structure, when the determination unit determines that the state is not normal—that is, when the appearance image data with a similarity to the captured image of the target item exceeding the determination threshold does not exist in the storage unit—the captured image of the target item can be stored in the storage unit as new appearance image data, associated with the identification information of the target item, using learning processing. Therefore, when the appearance design of the target item changes, appearance image data corresponding to the changed appearance design can be generated and registered. This reduces the operator's labor and time compared to the operator manually generating and registering appearance image data corresponding to the changed appearance design.

[0007] However, regarding the appearance design of an object, there are cases where it is periodically changed between multiple appearance designs or where it reverts to the original design after a temporary change. For example, if the object is a container for holding goods, the appearance design of the object may be changed periodically between multiple appearance designs according to the season, or it may be temporarily changed due to sales or other promotional activities and then revert to the original design. In these cases, the changed appearance design of the object is the same as the appearance design of the object before the change or earlier. With this in mind, in this structure, a predetermined number of appearance image data are stored in the storage unit in ascending order of the time points stored in the storage unit, associated with one identification information. Thus, it is possible to ensure that when the appearance design of the object is periodically changed between multiple appearance designs or when it reverts to the original design after a temporary change, appearance image data corresponding to the changed appearance design exists in the storage unit. When appearance image data corresponding to the changed appearance design exists in the storage unit, it is not necessary to generate and register appearance image data corresponding to the changed appearance design; therefore, the frequency of generation and registration of appearance image data can be suppressed accordingly.

[0008] Furthermore, in this structure, although appearance image data stored in the storage unit prior to the aforementioned predetermined number of appearance image data is deleted from the storage unit, in cases where the changed appearance design of the object article is the same as the previous appearance design of the object article, the previous appearance design is generally the most recently used appearance design. Therefore, according to this structure, by limiting the number of appearance image data stored in association with one identification information to a predetermined number or less, it is possible to reduce the storage capacity required by the storage unit and retain appearance image data in the storage unit corresponding to appearance designs that are more likely to be reused.

[0009] Further features and advantages of the item identification system become clear from the following description of the embodiments illustrated with reference to the accompanying drawings. Attached Figure Description

[0010] Figure 1 This is a perspective view showing a portion of the logistics equipment used in the application of an item identification system;

[0011] Figure 2 This is a block diagram showing the general structure of the item identification system;

[0012] Figure 3 This is a flowchart illustrating an example of a control flow; and

[0013] Figure 4 This is a flowchart illustrating another example of a control flow. Detailed Implementation

[0014] The implementation method of the item identification system is illustrated with reference to the accompanying drawings.

[0015] The item identification system 1 is a system that identifies the appearance image data of an item 80 (the object of identification) using the appearance image data of the item 80. In this embodiment, in Figure 1 The illustrated logistics equipment 100 uses an item identification system 1. Figure 1 The image only shows the vicinity of the retrieval point P in the logistics equipment 100. A first conveying device 91 and a second conveying device 92 are provided at the retrieval point P. The first conveying device 91 moves a support 10, such as a pallet, into the retrieval point P. A group of multiple items 80, i.e., an item group 8, is placed on the support 10. At the retrieval point P, a retrieval process is performed to sequentially select an item 81 from the item group 8 placed on the support 10 and retrieve the selected item 81 from the support 10. The second conveying device 92 removes the item 81 (processed item 83) retrieved from the support 10 from the retrieval point P.

[0016] Article 80 has an appearance (design) corresponding to its type. Articles 80 of different types generally have different appearances. Elements constituting the appearance of article 80 can be exemplified by markings (text, graphics, photographs, drawings, symbols, or combinations thereof, etc.) designed on the outer surface of article 80 by printing or affixing, the shape of the outer surface of article 80, the color of the outer surface of article 80, and the shape of article 80 (e.g., the shape of the outer edge of the outer surface). Furthermore, regarding the design of article 80, there are cases where even articles 80 of the same type may have variations. Figure 1 In the example shown, the article 80 placed on the support 10 (first support 11) that is moved into the retrieval point P is the same type of article 80 as the article 80 placed on the support 10 (second support 12) that is subsequently moved into the retrieval point P; however, the appearance of the upper surface S is different. Furthermore, in Figure 1 In the text, regarding the processed item 83, the mark or appearance set on the upper surface S is omitted.

[0017] Items 80 of the same kind have the same shape and size. Among multiple items 80 of different kinds, there may also be cases where one or both of their shape and size are common. For example... Figure 1 As shown, in this embodiment, the article 80 is formed into a cuboid shape (including a cube shape). Then, the article 80 is arranged with one outer surface facing upwards, in other words, with one of the three sides extending from one vertex of the cuboid forming the shape of the article 80 along the vertical direction V. That is, the article 80 has six outer surfaces, including the upper surface S (top surface).

[0018] In this embodiment, article 80 is a container for holding goods (goods, etc.). Therefore, the type of article 80 is determined according to the type of goods held within it. That is, two articles 80 holding the same type of goods are considered to be articles 80 of the same type. On the other hand, two articles 80 holding different types of goods are considered to be articles 80 of different types, even if they are containers of the same type. One article 80 may hold one type of goods or multiple types of goods of the same type. Article 80 may be, for example, a cardboard box or a container.

[0019] Support 10 supports the item assembly 8 from below. (Example) Figure 1 As shown, in this embodiment, multiple items 80 are arranged in two horizontal directions orthogonal to each other within a single support 10. These two horizontal directions are two directions orthogonal to each other in a horizontal plane orthogonal to the vertical direction V. Furthermore, in this embodiment, multiple items 80 are stacked (in other words, overlapped in the vertical direction V) within a single support 10. In this embodiment, multiple items 80 of the same type are placed on a single support 10. That is, the group 8 of items placed on the support 10 is a group of multiple items 80 of the same type. Figure 1 In the example shown, the group of articles 8 placed on a support 10 includes multiple articles 80 arranged in different directions. That is, the articles 80 constituting the group of articles 8 placed on the support 10 are not in the same direction.

[0020] like Figure 2As shown, the item identification system 1 includes a storage unit 21, an identification information acquisition unit 22, an imaging unit 23, a judgment unit 31, and a learning processing unit 32. In this embodiment, the item identification system 1 also includes a size measurement unit 24, an operation unit 25, and a retrieval processing unit 33. In this embodiment, the item identification system 1 includes a control device 20, which includes multiple functional units including the judgment unit 31, the learning processing unit 32, and the retrieval processing unit 33. Furthermore, the item identification system 1 can also be structured without at least one of the size measurement unit 24, the operation unit 25, and the retrieval processing unit 33. The technical features of the item identification system 1 disclosed in this specification can also be applied to the item identification method in the item identification system 1, and the item identification method in the item identification system 1 is also disclosed in this specification. The item identification method includes: a process where the storage unit 21 stores appearance image data (described later); a process where the identification information acquisition unit 22 acquires identification information of the target item 81; a process where the imaging unit 23 captures the appearance of the target item 81; a process where the determination unit 31 determines whether it is in a normal state (described later); and a process where the learning processing unit 32 performs the learning processing (described later). In this embodiment, the item identification method also includes: a process where the size measurement unit 24 measures the size of the target item 81; a process where the storage unit 21 stores size data (described later); a process where the operation unit 25 moves the target item 81; and a process where the removal processing unit 33 removes the target item 81 from the support 10. In these processes, processes where the control device 20 (specifically, the functional units provided by the control device 20) is not the main body (e.g., processes performed by the storage unit 21) are controlled by the control device 20 and performed by the main body of the process.

[0021] The control device 20 is configured to read and write data to the storage unit 21. Furthermore, the control device 20 is configured to acquire identification information of the object 81 acquired by the identification information acquisition unit 22, an image of the object 81 captured by the imaging unit 23, and the dimensions of the object 81 measured by the size measurement unit 24. Additionally, the control device 20 is configured to control the operation of the operation unit 25. In this way, the control device 20 is communicatively connected to each of the storage unit 21, the identification information acquisition unit 22, the imaging unit 23, the size measurement unit 24, and the operation unit 25 via wired or wireless means. The operation of the first conveying device 91 and the second conveying device 92 is controlled by the control device 20, or by other control devices that can communicate with the control device 20. The control device 20 includes an arithmetic processing unit such as a CPU (Central Processing Unit) and peripheral circuitry such as a memory; the various functions of the control device 20 are realized through the cooperation of this hardware and the program executing on the arithmetic processing unit or other hardware.

[0022] The multiple functional units of the control device 20 are configured to exchange information with each other. Furthermore, the multiple functional units of the control device 20 are at least logically distinct, but do not necessarily need to be physically distinct. Moreover, the multiple functional units of the control device 20 do not need to be implemented in common hardware; they can be implemented as multiple hardware units capable of communicating with each other. That is, the control device 20 can be constructed using multiple hardware units capable of communicating with each other instead of a single hardware unit.

[0023] The storage unit 21 stores the appearance image data of the article 80 in association with the identification information of the article 80. Here, the identification information is information used to identify the type of the article 80 (i.e., type information). The appearance image data includes, for example, information about a mark provided on the outer surface of the article 80, information about the appearance of the outer surface of the article 80, information about the color of the outer surface of the article 80, and information about the shape of the article 80 (e.g., the outer edge shape of the outer surface). In this embodiment, the appearance image data is the image data of the upper surface S of the article 80. Therefore, the appearance image data includes, for example, information about a mark provided on the upper surface S of the article 80, information about the appearance of the upper surface S of the article 80, information about the color of the upper surface S of the article 80, and information about the shape of the upper surface S of the article 80 (e.g., the outer edge shape of the upper surface S).

[0024] The storage unit 21 stores two or more predetermined numbers of appearance image data in the storage unit 21 in ascending order of the time points in which they were stored, associated with one piece of identification information. That is, appearance image data stored before the predetermined number of appearance image data is deleted from the storage unit 21. Therefore, the number of appearance image data stored associated with one piece of identification information is limited to a predetermined number or less. The predetermined number is, for example, 5. Furthermore, for identification information where the number of appearance image data generated to date is less than the predetermined number, a state is achieved where the predetermined number of appearance image data is stored in the storage unit 21 associated with that identification information.

[0025] The appearance image data is pre-collected and stored in the storage unit 21, or generated and stored in the storage unit 21 by the learning process described later. Regarding the collection of appearance image data, for example, items 80 that can be identified as objects of the item identification system 1 (i.e., items 80 that can become object items 81) are considered as objects. For example, an item 80 placed on a support 10 that is predetermined to be moved to the removal point P can become an object item 81. When feature extraction processing is performed on the image of the object item 81 in the consistency derivation process described later, the appearance image data that has undergone the same feature extraction processing can be stored in the storage unit 21.

[0026] In this embodiment, the storage unit 21 also stores the size data of the article 80, i.e., the size data, in association with the identification information of the article 80. In the storage unit 21, the size data of the article 80, which can become the object article 81, is collected and stored in advance. In this embodiment, the size data is the three-dimensional size data of the article 80. As described above, in this embodiment, the shape of the article 80 is formed as a cuboid, and the size data includes information on the dimensions of each of the three sides extending from one vertex of the cuboid forming the shape of the article 80.

[0027] The storage unit 21 has a hardware structure comprising a storage medium capable of storing and rewriting information, such as a flash memory or a hard disk. Figure 2 The illustration shows a case where the storage unit 21 is constructed from a device different from the control device 20; however, the storage unit 21 can be constructed using a storage device already present in the control device 20. Furthermore, the storage unit 21 can be located in a server or cloud server that can communicate with the control device 20.

[0028] The identification information acquisition unit 22 acquires the identification information of the target article 81. Furthermore, "acquiring the identification information of the target article 81" refers to both acquiring the identification information of the article 80 that has already been selected as the target article 81, and acquiring the identification information of the article 80 that is intended to be selected as the target article 81. For example... Figure 1 As shown, the identification information acquisition unit 22 includes a reading device 60 that reads the identification information of the object article 81. The identification information acquisition unit 22 acquires the identification information of the object article 81 read by the reading device 60. The identification information acquired by the identification information acquisition unit 22 (in other words, the identification information read by the reading device 60) is held in the identification information holding unit 70. The identification information holding unit 70 holds the identification information of the article 80 in a state that can be read from the outside. The reading device 60 is configured to be able to read the identification information held by the identification information holding unit 70. For example, if the identification information holding unit 70 uses a barcode to represent the identification information, the reading device 60 is a barcode reader that reads barcodes. The identification information holding unit 70 may not be a one-dimensional barcode like a barcode, but a two-dimensional code like a QR code (registered trademark), or a mark such as text or symbols. In addition, the identification information holding unit 70 may also be an IC tag or the like that capable of storing identification information and reading the identification information wirelessly. Furthermore, Figure 1 The configuration of the reading device 60 shown is an example, and the configuration (position or number, etc.) of the reading device 60 can be changed appropriately.

[0029] The identification information holding part 70 is provided for each item 80 or for each group of items 8. When the identification information holding part 70 is provided for each item 80, it is provided on the outer surface of each item 80, for example, by printing or attaching. Furthermore, when the identification information holding part 70 is provided for each group of items 8, it is provided on the outer surface of one item 80 constituting the group of items, for example, by attaching, or on the outer surface of the support 10 supporting the group of items 8. For example, when the identification information holding part 70 is an arrival mark, it is provided on the object (item 80 or support 10, etc.) by attaching.

[0030] As described above, in this embodiment, the item group 8 placed on the support 10 is a group of multiple items 80 of the same type. In this way, when the item group 8 placed on the support 10 is a group of multiple items 80 of the same type, an identification information holding unit 70 can be provided for each item group 8. Furthermore, even when the item group 8 placed on the support 10 is a group of multiple items 80 of the same type, an identification information holding unit 70 can be provided for each item 80. In this case, identification information for all items 80 constituting the item group 8 can be obtained by reading identification information from the identification information holding unit 70 provided for each item 80 constituting the item group 8.

[0031] Furthermore, when the identification information of the article 80 is stored in the storage unit 21 in association with the identification information of the support body 10 supporting the article 80, the identification information holding unit 70 can also adopt a structure that holds the identification information of the support body 10. In this case, the reading device 60 is configured to read the identification information of the support body 10 from the identification information holding unit 70 provided on the support body 10, and the identification information acquisition unit 22 obtains the identification information of the article 80 associated with the identification information of the support body 10 by referring to the storage unit 21.

[0032] The appearance of the object being photographed (item 81) in photography section 23. For example... Figure 1 As shown, the imaging unit 23 uses the first camera 41 to photograph the appearance of the object 81. In this embodiment, the imaging unit 23 photographs the upper surface S of the object 81. Therefore, the first camera 41 is positioned above the object 81 in the vertical direction V. For example, a color camera capable of capturing color images can be used as the first camera 41. Furthermore, Figure 1 The configuration of the first camera 41 shown is an example, and the configuration of the first camera 41 (such as its position or number) can be changed as appropriate.

[0033] The size measuring unit 24 measures the dimensions of the object article 81. Furthermore, "measuring the dimensions of the object article 81" refers to both measuring the dimensions of the article 80 that has already been selected as the object article 81 and measuring the dimensions of the article 80 that is intended to be selected as the object article 81. For example... Figure 1 As shown, in this embodiment, the size measurement unit 24 uses a second camera 42 positioned above the object object 81 in the vertical direction V to measure the size of the object object 81. In this embodiment, the size measurement unit 24 measures the three-dimensional size of the object object 81. That is, the size measurement unit 24 measures the size of the object object 81 in a planar view (a view along the vertical direction V) and the height of the object object 81 (the size in the vertical direction V). Here, a TOF (Time of Flight) camera (i.e., a TOF-type distance image sensor) is used as the second camera 42. The size measurement unit 24 measures the size of the object object 81 based on a distance image (an image with distance information) obtained by capturing the object object 81 with the second camera 42. Furthermore, in the TOF method, the distance to the object is detected based on the time of flight of the light reflected back from the object. A stereo camera or the like can also be used as the second camera 42. The stereo camera is configured to capture the object from two viewpoints, and the distance to the object is derived based on the parallax between these two viewpoints, thereby generating a distance image. Furthermore, Figure 2 The configuration of the second camera 42 shown is an example, and the configuration of the second camera 42 (such as its position or number) can be changed as appropriate.

[0034] The dimension measuring unit 24 obtains the three-dimensional dimensions of the object article 81 as described below. The dimension measuring unit 24 extracts the outline of the object article 81 from the image (distance image) captured by the second camera 42, and obtains the dimensions of the object article 81 in a planar view (specifically, the dimensions of two sides of the quadrilateral forming the outer edge of the upper surface S of the object article 81). Furthermore, the dimension measuring unit 24 derives the difference between the height of the upper surface S and the height of the lower surface of the object article 81 from the image (distance image) captured by the second camera 42, and obtains the height of the object article 81. Moreover, the lower surface of the object article 81 is positioned at the same height as the upper surface of the support 10 or the upper surface S of the article 80 one level below the object article 81; therefore, the height of the lower surface of the object article 81 can be obtained based on the height of either of these two upper surfaces.

[0035] The operating unit 25 moves the object 81. For example... Figure 1As shown, the operation unit 25 includes a holding part 50 for holding an article 80 and a moving mechanism 51 for moving the holding part 50. In this embodiment, the holding part 50 holds the upper surface S of the article 80. Specifically, the holding part 50 adsorbs and holds the upper surface S of the article 80. The operation unit 25 moves the holding part 50, which holds the object article 81, by means of the moving mechanism 51, thereby moving the object article 81. In this embodiment, the moving mechanism 51 is configured to move the holding part 50 to the support 10 (in which the first conveying device 91 has moved into the removal point P) of the support body 10. Figure 1 In the example shown, the position is above the first support 10 and above the second conveying device 92. Thus, the operating unit 25 is configured to move the object 81 removed from the support 10 to the second conveying device 92. Although details are omitted, in... Figure 1 In the example shown, the moving mechanism 51 is configured to move the retaining part 50 connected to the front end of the multi-joint arm by rotating the turntable and extending and retracting the multi-joint arm supported on the turntable.

[0036] like Figure 1 As shown, in this embodiment, the object article 81 is an article 80 included in the article group 8 (here, a group of multiple articles 80 of the same type) placed on the support 10 and on which no other articles 80 are placed. Then, in this embodiment, the article discrimination system 1 is configured such that, when the object article 81 is moved using the operation unit 25 such that a gap is formed between it and adjacent articles 80 (adjacent articles 82), the size of the object article 81 is measured using the size measuring unit 24. Furthermore, even when the object article 81 is released from being held by the holding unit 50, it is still moved by the operation unit 25 within a range maintained by being placed on the support 10. Figure 1 The diagram shows an object article 81, placed on a support 10 (specifically, a first support 11), moved to a state where a gap is formed between it and each of two adjacent articles 82. By measuring the dimensions of the object article 81 using the dimension measuring unit 24 in this state, the accuracy of extracting the outline of the object article 81 can be improved, thereby increasing the accuracy of measuring the dimensions of the object article 81 in a plan view. Furthermore, the accuracy of detecting the height of a surface positioned at the same height as the lower surface of the object article 81 can be improved, thereby increasing the accuracy of measuring the height of the object article 81. Moreover, the process of moving the object article 81 using the operation unit 25 to form a gap with the adjacent articles 82 can be performed only when predetermined conditions are met, such as when the dimensions of the object article 81 cannot be properly measured.

[0037] The determination unit 31 is a functional unit that calculates the consistency between the captured image of the object item 81 captured by the imaging unit 23 and the appearance image data stored in the storage unit 21 in association with the recognition information acquired by the recognition information acquisition unit 22, and determines whether there is a normal state where the appearance image data with a consistency of ≥10 ...

[0038] As described above, in the storage unit 21, a predetermined number of appearance image data are stored in the storage unit 21 in order from newest to oldest time, and associated with one identification information. In this embodiment, the determination unit 31 performs consistency derivation processing in the order of appearance image data stored in the storage unit 21 from newest to oldest time. When appearance image data with a consistency of ≥ determination threshold is detected, the consistency derivation processing ends, and the state is determined to be normal. Furthermore, as in this embodiment, when a group of multiple items 80 of the same type is placed on a support 10, the following structure can be adopted: That is, for the target item 81 selected from the item group 8, consistency derivation processing is performed in the order of appearance image data stored in the storage unit 21 from newest to oldest time. Then, if an appearance image data with a consistency of more than the determination threshold is detected from the multiple appearance image data stored in the storage unit 21 other than the latest appearance image data stored in the storage unit 21 at the latest time point, the consistency export process is first performed using the detected appearance image data for the object item 81 selected next from the item group 8. If the consistency is less than the determination threshold, the consistency export process is performed on the remaining appearance image data in the order of the appearance image data stored in the storage unit 21 from the newest to the oldest time point.

[0039] The learning processing unit 32 is a functional unit that performs learning processing when the determination unit 31 determines that the state is not normal. The learning processing involves storing the data of the captured image of the object 81 taken by the imaging unit 23 as new appearance image data in the storage unit 21, associated with the identification information obtained by the identification information acquisition unit 22. Furthermore, the storage unit 21 stores a predetermined number of appearance image data associated with one piece of identification information in the storage unit 21 in ascending order of the time points stored therein. Therefore, when the learning processing unit 32 stores the data of the captured image of the object 81 taken by the imaging unit 23 as new appearance image data in the storage unit 21, and when a predetermined number of appearance image data has already been associated with the identification information and stored in the storage unit 21, it deletes the oldest appearance image data stored in the storage unit 21 from the predetermined number of appearance image data stored in the storage unit 21, and stores the new appearance image data in the storage unit 21.

[0040] In this embodiment, the learning processing unit 32 is configured to perform learning processing when the determination unit 31 determines that the state is not normal, provided that the size of the object article 81 measured by the size measurement unit 24 falls within the allowable size range. If the size of the object article 81 measured by the size measurement unit 24 does not fall within the allowable size range, it is determined to be an abnormal state, and no learning processing is performed. The allowable size range is based on size data stored in the storage unit 21 in association with the identification information obtained by the identification information acquisition unit 22. For example, the allowable size range is 95% to 105% of the size data stored in the storage unit 21 (in other words, ±5% of that size data). In this embodiment, the learning processing unit 32 performs learning processing based on the condition that the size of the object article 81 measured by the size measurement unit 24 falls within the allowable size range in all three dimensions. Specifically, the learning processing unit 32 performs learning processing based on the condition that the dimensions of the object item 81 measured by the size measurement unit 24 are within the allowable size range in each of the three extension directions extending from one vertex of the cuboid forming the shape of the item 80. If the learning processing unit 32 determines that an abnormal state is present, the item identification system 1 notifies the operator, such as the manager, of the abnormal state (by sound, warning light illumination, etc.).

[0041] The retrieval processing unit 33 is a functional unit that performs the retrieval processing of the object item 81 from the support 10. The retrieval processing unit 33 controls the operation unit 25 to perform the retrieval processing. Figure 1In the example shown, during the retrieval process, the operation unit 25 is controlled to transfer the object article 81, which will be removed from the support 10, to the second conveying device 92. In this embodiment, the retrieval processing unit 33 retrieves the object article 81 from the support 10 if the determination unit 31 determines it to be in a normal state, or if the determination unit 31 determines it to be in an abnormal state and the learning processing unit 32 performs learning processing. That is, if the learning processing unit 32 determines it to be in an abnormal state, the retrieval processing unit 33 does not perform the retrieval process.

[0042] like Figure 1 As shown, in this embodiment, the target item 81 is an item 80 included in the item group 8 (here, a group of multiple items 80 of the same type) placed on the support 10 and on which no other items 80 are placed. During the removal process, the removal processing unit 33 sequentially selects the target item 81 from the item group 8 and removes the selected target item 81 from the support 10. At this time, whenever the removal processing unit 33 selects the target item 81 to be removed, the determination unit 31 determines whether it is in a normal state based on the captured image of the newly selected target item 81.

[0043] Next, the order of the item identification processes performed by the item identification system 1 of this embodiment will be described in sequence. Figure 3 The examples shown and Figure 4 The example shown. One approach to the item identification method in item identification system 1 includes executing... Figure 3 The procedures for each processing step shown, in another embodiment of the item identification method in item identification system 1, include the execution of... Figure 4 The procedures for each processing step are shown.

[0044] First of all, Figure 3 The example shown illustrates this. Figure 3 An example of a control flow is shown where the identification information acquisition unit 22 acquires the identification information of each object item 81 for each object item 81. Furthermore, Figure 3 The control flow for item identification processing for a single object item 81 is shown. That is, whenever object item 81 is selected, the following steps are executed: Figure 3 The series of processes shown. In a scenario where the type of item 80 selected as object item 81 may change with each selection of object item 81, such as... Figure 3 As shown in the control flow, the identification information acquisition unit 22 performs the acquisition of identification information for each object item 81. For example, this is done when object items 81 are selected sequentially from the support 10 holding multiple types of items 80, or when multiple types of items 80 are moved one by one to the item identification processing location (in... Figure 1 In the example shown, if items are taken out from point P and selected as object items 81 in sequence starting from the first item 80 moved in there, the types of items 80 selected as object items 81 may change with each selection of object items 81.

[0045] like Figure 3 As shown, the control device 20 controls the identification information acquisition unit 22 to perform identification information acquisition processing (step #1) to acquire identification information of the object article 81. Furthermore, the control device 20 controls the imaging unit 23 to perform imaging processing (step #2) to capture images of the appearance of the object article 81. Also, the imaging processing (step #2) may be performed simultaneously with the identification information acquisition processing (step #1) or before the identification information acquisition processing (step #1), rather than after it. Then, the determination unit 31 performs consistency derivation processing (step #3) based on the captured image of the object article 81 taken in the imaging processing (step #2) and the appearance image data stored in the storage unit 21 in association with the identification information acquired in the identification information acquisition processing (step #1). Furthermore, if the captured image obtained in the previously selected object 81 includes the currently selected object 81, that is, if a captured image of the currently selected object 81 already exists, the captured image can be used to perform the consistency export process (step #3) instead of performing the captured image (step #2).

[0046] The determination unit 31 determines whether the state is normal based on the consistency degree derived in the consistency degree derivation process (step #3). Specifically, if appearance image data with a consistency degree of ≥100% exists in the storage unit 21, the determination unit 31 determines it to be in a normal state (step #4: Yes), and the process ends. As described above, in this embodiment, the consistency degree derivation process is performed in the order of appearance image data stored in the storage unit 21 from newest to oldest at the specified time points. When appearance image data with a consistency degree of ≥100% is detected, the consistency degree derivation process ends, and the state is determined to be normal. Furthermore, if the state is determined to be normal and the process ends, or if the learning process described later (step #7) is performed and the process ends, for example, if the process of identifying the item is performed (at the point where the item identification process was performed), the process may also be performed to determine the state to be normal. Figure 1 In the example shown, the process of removing object item 81 from point P is shown.

[0047] On the other hand, if the appearance image data with a consistency of more than the determination threshold is not present in the storage unit 21, the determination unit 31 determines that it is not a normal state (step #4: No), and the control device 20 controls the size measurement unit 24 to perform size measurement processing (step #5) to measure the size of the object article 81. Then, if the size of the object article 81 measured in the size measurement processing (step #5) is within the allowable size range based on the size data stored in the storage unit 21 in association with the identification information obtained in the identification information acquisition processing (step #1) (step #6: Yes), the learning processing unit 32 performs learning processing (step #7) to store the data of the captured image of the object article 81 as new appearance image data in association with the identification information obtained in the identification information acquisition processing (step #1). Furthermore, regarding the captured image of the object article 81 as new appearance image data, the captured image of the object article 81 obtained in the capture processing (step #2) or the captured image of the object article 81 captured by the capture unit 23 after determining that the size of the object article 81 measured in the size measurement processing (step #5) is within the allowable size range. On the other hand, if the size of the object article 81 measured in the size measurement processing (step #5) is not within the allowable size range (step #6: no), the learning processing unit 32 determines it to be an abnormal state (step #8), and the processing ends.

[0048] Next, regarding Figure 4 The example shown illustrates this. Figure 4 An example of a control flow is shown where the identification information acquisition unit 22 acquires the identification information of the target item 81 for each item group 8 (in other words, each support 10). For example, when selecting the target item 81 sequentially from a group of multiple items 80 of the same type placed on the support 10, according to... Figure 4 The control flow shown is used to perform item identification processing.

[0049] like Figure 4 As shown, when the support body 10 (in Figure 1In the example shown, when the first support 11 is moved into the removal point P (step #10: Yes), the control device 20 controls the identification information acquisition unit 22 to perform identification information acquisition processing (step #11) to acquire identification information of the article 80 (the article 80 pre-selected as the target article 81) placed on the support 10. Then, if the article 80 corresponding to the identification information acquired in the identification information acquisition processing (step #11) is not the article 80 that is the target of the removal process (step #12: No), the control device 20 determines an abnormal state (step #22), and the process ends. For example, if the support 10, which is carrying an article 80 that is not the target of the removal process, is mistakenly moved into the removal point P, an abnormal state is determined in this way.

[0050] On the other hand, if the item 80 corresponding to the identification information obtained in the identification information acquisition process (step #11) is the item 80 to be removed (step #12: Yes), the removal process unit 33 performs an item selection process (step #13) to select the item 81 from the item group 8 placed on the support 10. The item 81 is selected from the items 80 on which no other items 80 are placed. Next, the control device 20 controls the imaging unit 23 to perform an image capture process (step #14) to capture the appearance of the item 81 selected in the item selection process (step #13). Then, the determination unit 31 performs a consistency derivation process (step #15) based on the image of the item 81 captured in the image capture process (step #14) and the appearance image data stored in the storage unit 21 in association with the identification information obtained in the identification information acquisition process (step #11). Furthermore, if the captured image obtained in the previously selected object 81 includes the currently selected object 81, that is, if a captured image of the currently selected object 81 already exists, the captured image can be used to perform the consistency export process (step #15) without performing the captured image process (step #14).

[0051] The determination unit 31 determines whether the state is normal based on the consistency degree derived in the consistency degree derivation process (step #15). Specifically, if the appearance image data with a consistency degree of ≥15 exists in the storage unit 21, the determination unit 31 determines it to be in a normal state (step #16: Yes), and the retrieval processing unit 33 performs the retrieval processing (step #17) to retrieve the object item 81 from the support 10. Figure 1In the example shown, during the removal process (step #17), the object item 81 removed from the support 10 is transferred to the second conveying device 92 and then moved out from the removal point P by the second conveying device 92. Then, if there is an item 80 remaining in the support 10 (step #18: Yes), the process returns to step #13. Therefore, whenever an object item 81 (here, the object item 81 that becomes the object of the removal process) is selected in the object item selection process (step #13), it is determined whether it is in a normal state based on the image captured by the newly selected object item 81 in the consistency export process (step #15). On the other hand, if there is no item 80 remaining in the support 10 (step #18: No), the process ends.

[0052] Like this, in Figure 4 In the control flow shown, whenever object item 81 is selected in the object item selection process (step #13), the consistency export process (step #15) is executed. Figure 4 The control flow shown is used, for example, when selecting an object item 81 sequentially from a group of multiple items 80 of the same type placed on the support 10. In this case, if, during the consistency export process (step #15), an appearance image data with a consistency threshold or higher is detected other than the latest appearance image data stored in the storage unit 21 from among the multiple appearance image data stored in the storage unit 21, then the consistency of the detected appearance image data is highly likely to be high during the consistency export process (step #15) performed on the next object item 81.

[0053] With this in mind, a structure like the following can be adopted, for example. Basically, for the object item 81 selected from item group 8, consistency export processing is performed in the order of appearance image data stored in storage unit 21 from newest to oldest at each point in time. Then, if appearance image data with a consistency threshold or higher is detected other than the latest appearance image data stored in storage unit 21 from among the multiple appearance image data stored in storage unit 21, consistency export processing is first performed on the next object item 81 selected from that item group 8 using the detected appearance image data. If the consistency threshold is insufficient, consistency export processing is performed on the remaining appearance image data in the order of appearance image data stored in storage unit 21 from newest to oldest at each point in time.

[0054] If the appearance image data with a consistency of more than the determination threshold is not present in the storage unit 21, the determination unit 31 determines that it is not a normal state (step #16: No), and the control device 20 controls the size measurement unit 24 to perform size measurement processing (step #19) to measure the size of the object article 81. Then, if the size of the object article 81 measured in the size measurement processing (step #19) is not included in the allowable size range based on the size data stored in the storage unit 21 in association with the identification information obtained in the identification information acquisition processing (step #11) (step #20: No), the learning processing unit 32 determines that it is an abnormal state (step #22), and the processing ends.

[0055] On the other hand, if the size of the object article 81 measured in the size measurement process (step #19) falls within the allowable size range (step #20: Yes), the learning processing unit 32 performs a learning process (step #21) that associates the data of the captured image of the object article 81 as new appearance image data with the identification information obtained in the identification information acquisition process (step #11) and stores it in the storage unit 21. The retrieval processing unit 33 then performs a retrieval process (step #17) to remove the object article 81 from the support 10. In this way, the retrieval processing unit 33 performs the retrieval process when the learning process is determined to be in a normal state or when the learning process is determined to be in an abnormal state. Furthermore, the captured image of the object article 81 that serves as new appearance image data in the learning process (step #21) is either the captured image of the object article 81 obtained in the shooting process (step #14) or the captured image of the object article 81 taken by the shooting unit 23 after determining that the size of the object article 81 measured in the size measurement process (step #19) falls within the allowable size range.

[0056] Furthermore, during the removal process (step #17), the orientation of the object item 81 is determined by image analysis using the appearance image and appearance image data of the object item 81. This allows the object item 81, removed from the support 10, to be transferred to its destination in the same orientation. Figure 1 In the example shown, the second conveying device 92). Therefore, even with... Figure 1 Unlike the example shown, when the upper surface S of the item 80 is formed into a square shape, for example, the object item 81 (processed item 83) removed from the support 10 can be moved out from the removal point P in such a way that the opening and closing direction of the cover provided on the upper surface S is consistent.

[0057] [Other Implementation Methods]

[0058] Next, other implementations of the item identification system will be described.

[0059] (1) In the above embodiment, the following structure was described as an example: when the determination unit 31 determines that the state is not normal, the learning processing unit 32 performs learning processing on the condition that the size of the object article 81 measured by the size measuring unit 24 is within the allowable size range. However, this disclosure is not limited to such a structure. In addition to the size of the object article 81 measured by the size measuring unit 24 being within the allowable size range, other conditions may be included in the conditions for the learning processing unit 32 to perform learning processing. Furthermore, the condition for the learning processing unit 32 to perform learning processing may not include the size of the object article 81 measured by the size measuring unit 24 being within the allowable size range. In this case, the following structure can be adopted: for example, if the article discrimination system 1 does not have a size measuring unit 24, when the determination unit 31 determines that the state is not normal, the learning processing unit 32 performs learning processing without considering other conditions (i.e., unconditionally). In cases where the item identification system 1 does not have a size measurement unit 24, the storage unit 21 may be configured to store the size data of the item 80 without associating it with the identification information of the item 80.

[0060] (2) In the above embodiments, a structure in which multiple articles 80 are arranged in two orthogonal horizontal directions on a single support 10 has been described as an example. However, this disclosure is not limited to such a structure, and a structure in which multiple articles 80 are arranged in only one horizontal direction on a single support 10, or a structure in which only one article 80 is arranged at the same height in a single support 10, can also be used. Furthermore, in the above embodiments, a structure in which multiple articles 80 are stacked on a support 10 has been described as an example. However, this disclosure is not limited to such a structure, and a structure in which the articles 80 are not stacked on the support 10 can also be used.

[0061] (3) In the above embodiment, the structure of multiple articles 80 of the same type placed on one support 10 was described as an example. However, this disclosure is not limited to such a structure, and a structure of multiple articles 80 of different types placed on one support 10 is also possible. In addition, a structure in which only one article 80 is supported on one support 10 or a structure in which the article 80 is directly supported on the conveying surface of the first conveying device 91 is also possible.

[0062] (4) In the above embodiments, the structure in which the article 80 is formed into a cuboid shape has been described as an example. However, this disclosure is not limited to such a structure, and the article 80 may also be in a shape other than a cuboid. Furthermore, in the above embodiments, the structure in which the article 80 is a container for containing goods has been described as an example. However, this disclosure is not limited to such a structure, and the article 80 may also be an article other than a container.

[0063] (5) In the above embodiment, the take-out point P in the logistics equipment 100 (in Figure 1 The example shown illustrates the application of item identification system 1 at the loading and unloading area. However, this disclosure is not limited to this structure, and the item identification system of this disclosure can also be applied to places where the type and quantity of items 80 are checked against the contents of the delivery order (e.g., the contents recorded in the delivery document) (item inspection operation), or to places where the type and quantity of items 80 are database-based.

[0064] (6) Furthermore, the structures disclosed in the above embodiments can be combined with those disclosed in other embodiments (including combinations of embodiments described as other embodiments) as long as no contradiction arises. Regarding other structures, the embodiments disclosed in this specification are merely illustrative in all respects. Therefore, various changes can be appropriately made without departing from the spirit of this disclosure.

[0065] [Summary of the above embodiments]

[0066] The following is a summary of the item identification system described above.

[0067] The item identification system comprises: a storage unit that stores appearance image data of an item in association with the item's identification information; an identification information acquisition unit that acquires the identification information of the item to be identified; an imaging unit that captures an image of the object item; a determination unit that calculates the consistency between the image captured by the imaging unit and the appearance image data stored in the storage unit in association with the identification information acquired by the identification information acquisition unit, and determines whether the appearance image data with a consistency of more than a determination threshold is in a normal state; and a learning processing unit that performs learning processing if the determination unit determines that the state is not normal. The learning processing is a process of storing the data of the captured image as new appearance image data in association with the identification information acquired by the identification information acquisition unit in the storage unit. The storage unit stores two or more predetermined numbers of appearance image data in association with one piece of identification information in order from newest to oldest storage time.

[0068] According to this structure, when the determination unit determines that the state is not normal—that is, when the appearance image data with a similarity to the captured image of the target item exceeding the determination threshold does not exist in the storage unit—the captured image of the target item can be stored in the storage unit as new appearance image data, associated with the identification information of the target item, using learning processing. Therefore, when the appearance design of the target item changes, appearance image data corresponding to the changed appearance design can be generated and registered. This reduces the operator's labor and time compared to the operator manually generating and registering appearance image data corresponding to the changed appearance design.

[0069] However, regarding the appearance design of an object, there are cases where it is periodically changed between multiple appearance designs or where it reverts to the original design after a temporary change. For example, if the object is a container for holding goods, the appearance design of the object may be changed periodically between multiple appearance designs according to the season, or it may be temporarily changed due to sales or other promotional activities and then revert to the original design. In these cases, the changed appearance design of the object is the same as the appearance design of the object before the change or earlier. With this in mind, in this structure, a predetermined number of appearance image data are stored in the storage unit in ascending order of the time points stored in the storage unit, associated with one identification information. Thus, it is possible to ensure that when the appearance design of the object is periodically changed between multiple appearance designs or when it reverts to the original design after a temporary change, appearance image data corresponding to the changed appearance design exists in the storage unit. When appearance image data corresponding to the changed appearance design exists in the storage unit, it is not necessary to generate and register appearance image data corresponding to the changed appearance design; therefore, the frequency of generation and registration of appearance image data can be suppressed accordingly.

[0070] Furthermore, in this structure, although appearance image data stored in the storage unit prior to the aforementioned predetermined number of appearance image data is deleted from the storage unit, in cases where the changed appearance design of the object article is the same as the previous appearance design of the object article, the previous appearance design is generally the most recently used appearance design. Therefore, according to this structure, by limiting the number of appearance image data stored in association with one identification information to a predetermined number or less, it is possible to reduce the storage capacity required by the storage unit and retain appearance image data in the storage unit corresponding to appearance designs that are more likely to be reused.

[0071] Preferably, the determination unit performs the consistency export process according to the order of the appearance image data stored in the storage unit from newest to oldest time points. When appearance image data with a consistency of more than or equal to the determination threshold is detected, the consistency export process ends and the data is determined to be in the normal state.

[0072] According to this structure, consistency derivation processing can be performed on appearance image data in descending order of probability of consistency. Therefore, when appearance image data with a consistency of ≥ 100% is stored in the storage unit, the processing time before being determined to be in a normal state can be shortened.

[0073] Furthermore, preferably, it also includes a size measuring unit that measures the size of the object item, and a storage unit that stores the size data of the item in association with the identification information of the item. If the determination unit determines that the state is not normal, the learning processing unit performs the learning processing on the condition that the size of the object item measured by the size measuring unit is within the allowable size range. If the size of the object item measured by the size measuring unit is not within the allowable size range, it is determined to be an abnormal state and the learning processing is not performed. The allowable size range is based on the size data stored in the storage unit in association with the identification information obtained by the identification information acquisition unit.

[0074] According to this structure, when the determination unit determines that the object is not in a normal state, it is possible to determine whether the object is an item corresponding to the identification information obtained by the identification information acquisition unit (hereinafter referred to as "identification item") by checking whether the size of the object measured by the size measurement unit falls within the aforementioned allowable size range. Then, learning processing is performed based on the condition that the size of the object measured by the size measurement unit falls within the aforementioned allowable size range. Therefore, a structure can be adopted where learning processing is not performed when the probability that the object is an identification item is low. This easily avoids the situation where, when the object is not an identification item, the data of the captured image of the object is mistakenly stored in the storage unit as new appearance image data associated with identification information of an item different from the object.

[0075] In the above structure, it is preferable to further include an operation unit that moves the object article, which is an article included in a group of multiple articles of the same kind placed on a support and on which no other articles are placed. When the object article is moved by the operation unit such that a gap is formed between it and other adjacent articles, the size measuring unit measures the size of the object article.

[0076] According to this structure, when the object article is included in a group of multiple articles of the same type placed on a support, the size measuring unit can easily measure the size of the object article with high precision. Therefore, it is possible to improve the accuracy of determining whether the object article is an identification article.

[0077] Furthermore, it is preferable that the size measuring unit is equipped with a TOF (Time of Flight) camera or a stereo camera.

[0078] According to this structure, the three-dimensional dimensions of an object can be measured using the size measuring unit. Therefore, in determining whether the dimensions of the object measured by the size measuring unit fall within the aforementioned permissible size range, the three-dimensional shape of the object can be taken into account, thereby improving the accuracy of determining whether the object is an identification item.

[0079] In the above-described item identification system, preferably, the target item is an item included in a group of multiple items placed on a support, i.e., an item group, and no other items are placed on top of it. The system also includes a removal processing unit that performs a removal process of sequentially selecting the target item from the item group and removing the selected target item from the support. Whenever the removal processing unit selects the target item to be removed, the determination unit determines whether it is in the normal state based on the captured image of the newly selected target item. If the determination unit determines that it is in the normal state, or if the determination unit determines that it is not in the normal state and the learning processing unit performs the learning processing, the removal processing unit removes the selected target item from the support.

[0080] According to this structure, during the execution of a removal process in which object items are sequentially selected from a group of items placed on a support and removed from the support, a determination unit can be used to determine whether each object item is in a normal state. If the appearance design of an object item changes, appearance image data corresponding to the changed appearance design can be generated and registered. That is, according to this structure, the generation and registration of appearance image data in cases where the appearance design of an object item changes can be performed during the removal process. Furthermore, according to this structure, for example, the appearance image data can be used to identify the orientation of the object item during the execution of the removal process, thereby ensuring that the object items removed from the support are in the same orientation.

[0081] Regarding the item identification system disclosed herein, it is sufficient to achieve at least one of the aforementioned effects.

[0082] Explanation of reference numerals in the attached figures

[0083] 1: Item Identification System

[0084] 8: Item Group

[0085] 10: Support body

[0086] 21: Storage Department

[0087] 22: Identification Information Acquisition Department

[0088] 23: Filming Department

[0089] 24: Dimensioning Department

[0090] 25: Operations Department

[0091] 31: Judgment Department

[0092] 32: Learning Processing Department

[0093] 33: Remove processing unit

[0094] 80: Items

[0095] 81: Object item.

Claims

1. An item identification system, characterized in that: The item identification system has the following features: The storage unit stores the data of the appearance image of the item, i.e., the appearance image data, in association with the identification information of the item; The identification information acquisition unit acquires the identification information of the object to be identified, i.e., the object object. The camera unit photographs the appearance of the object. The determination unit extracts the appearance image data stored in the storage unit and associated with the identification information of the object item obtained by the identification information acquisition unit, and the consistency between the appearance image data and the image of the object item captured by the shooting unit, and determines whether there is a normal state where the consistency is above the determination threshold. as well as The learning processing unit performs learning processing when the determination unit determines that the state is not the normal state. The learning process involves associating new appearance image data with the identification information of the object obtained by the identification information acquisition unit and storing it in the storage unit. The new appearance image data refers to the data of the captured image of the object that does not have appearance image data with a consistency degree higher than the determination threshold. The storage unit stores two or more predetermined numbers of appearance image data associated with one identification information in order from newest to oldest time when they were stored in the storage unit.

2. The item identification system according to claim 1, wherein, The determination unit performs consistency export processing according to the order of the appearance image data stored in the storage unit from newest to oldest time points. When appearance image data with a consistency of more than the determination threshold is detected, the consistency export processing ends and is determined to be in the normal state.

3. The item identification system according to claim 1, wherein, It also includes a size measuring unit that measures the size of the object. The size data of items that may become the object are pre-collected and stored in the storage unit in association with the item's identification information. If the determination unit determines that the state is not normal, the learning processing unit performs the learning processing on the condition that the size of the object measured by the size measurement unit is within the allowable size range. If the size of the object measured by the size measurement unit is not within the allowable size range, the state is determined to be abnormal and the learning processing is not performed. The allowable size range is based on the size data stored in the storage unit in association with the identification information obtained by the identification information acquisition unit.

4. The item identification system according to claim 3, wherein, It also includes an operating unit that moves the object. The object article is an article included in a group of multiple articles of the same kind placed on a support and on which no other articles are placed. The object is moved using the operating unit to create a gap between it and other adjacent objects, and the size measuring unit measures the size of the object.

5. The item identification system according to claim 3, wherein, The dimensional measuring unit is equipped with a TOF (Time of Flight) camera or a stereo camera.

6. The article discrimination system according to any one of claims 1 to 5, wherein, The object article is an article included in a group of multiple articles placed on a support, i.e., an article group, and which is not topped with other articles. It also includes a removal processing unit, which performs a removal process of sequentially selecting the target item from the item group and removing the selected target item from the support. Whenever the extraction processing unit selects an object item as the object to be extracted, the determination unit determines whether it is in the normal state based on the captured image of the newly selected object item. If the determination unit determines that the state is normal, or if the determination unit determines that the state is not normal and the learning processing unit performs the learning processing, the extraction processing unit extracts the selected object item from the support.

Images