Processing device, processing method, and recording medium

Abstract

This processing device includes: an acquisition unit that acquires information indicating the packaging status of an article that is stored inside a package; a determination unit that, in accordance with the information indicating the packaging status of the article acquired by the acquisition unit, determines, as a parameter for performing recognition processing using image data in which the article is a subject, a recognition target region which is a region in the image data and which is subjected to recognition processing; and a recognition unit that performs recognition processing on the recognition target region determined by the determination unit.

Description

Processing device, processing method, and recording medium

[0001] The present disclosure relates to a processing device, a processing method, and a recording medium.

[0002] In warehouses and other locations, machines may be used to carry out optional processing, such as pick-and-place (P&P) processing, in which machines such as arm robots are used to transfer the contents of packages such as cardboard boxes to another location.

[0003] For example, Patent Document 1 describes a picking facility that transfers items stored in an item storage unit (package) to a container using a picking robot. According to Patent Document 1, the picking facility includes a robot cart that is equipped with a picking robot and that can travel along rails, a container cart that can carry a container and that can travel separately from the robot cart, and a recognition unit cart that has a recognition unit that recognizes the items stored in the item storage unit and that can travel. Furthermore, the container cart and the recognition unit cart are configured to be movable in the same direction as the traveling direction of the robot cart.

[0004] Japanese Patent Application Laid-Open No. 2022-093572

[0005] When performing any processing such as pick-and-place processing, an object to be processed, such as gripped, may be recognized using image data acquired by an imaging device attached to the tip of an arm robot performing the processing. To perform such recognition more accurately, it is desirable to perform recognition processing according to the packaging status of the object to be processed. However, depending on the warehouse where the processing is performed, it may be difficult to uniquely determine the status of the object to be processed, for example, because packages containing individually packaged objects may be mixed with packages containing multiple bundled objects. As a result, a problem has arisen in that it may be difficult to perform appropriate recognition processing according to the object to be processed.

[0006] Therefore, one of the objects of the present disclosure is to provide a processing device, a processing method, and a recording medium that can solve the above-mentioned problems.

[0007] In order to achieve this object, the processing device of the present disclosure is configured to have an acquisition unit that acquires information indicating the packaging status of an item stored inside a package; a determination unit that determines a recognition target area, which is an area of ​​the image data that is the target for recognition processing, as a parameter when performing recognition processing using image data of the item as a subject, in accordance with the information indicating the packaging status of the item acquired by the acquisition unit; and a recognition unit that performs recognition processing on the recognition target area determined by the determination unit.

[0008] In addition, the processing method disclosed herein is configured such that an information processing device acquires information indicating the packaging status of an item stored inside a package, determines a recognition target area, which is an area of ​​the image data that is the target of recognition processing, as a parameter when performing recognition processing using image data of the item as a subject, according to the acquired information indicating the packaging status of the item, and performs recognition processing on the determined recognition target area.

[0009] In addition, the recording medium in the present disclosure is a computer-readable recording medium having recorded thereon a program for causing an information processing device to acquire information indicating the packaging status of an item stored inside a package, determine a recognition target area, which is an area of ​​the image data that is the target of the recognition processing, as a parameter when performing recognition processing using image data of the item as a subject, according to the acquired information indicating the packaging status of the item, and perform recognition processing on the determined recognition target area.

[0010] According to the above-described configurations, it is possible to perform appropriate recognition processing according to the processing target.

[0011] 1 is a diagram illustrating an overview of a processing system in the present disclosure. FIG. 1 is a diagram for explaining an example of recognition processing. FIG. 2 is a block diagram illustrating an example of a configuration of a processing device according to the present disclosure. FIG. 2 is a diagram illustrating an example of item / packaging information. FIG. 3 is a diagram illustrating an example of processing when determining the height of an imaging device. FIG. 4 is a diagram illustrating an example of processing when determining the position of an imaging device. FIG. 5 is a diagram illustrating an example of processing when determining the position of an imaging device. FIG. 6 is a diagram illustrating an example of processing to determine a recognition target area. FIG. 7 is a block diagram illustrating an example of a configuration of a management device. FIG. 8 is a flowchart illustrating an example of operation of a processing device. FIG. 9 is a flowchart illustrating an example of operation of a management device. FIG. 10 is a diagram illustrating an example of a hardware configuration of a processing device according to the present disclosure. FIG. 11 is a block diagram illustrating an example of a configuration of a processing device. FIG. 12 is a flowchart illustrating an example of operation of a processing device. FIG. 13 is a block diagram illustrating an example of a configuration of a management device.

[0012] [First Embodiment] An example configuration of a processing system 100 according to the present disclosure will be described with reference to FIGS. 1 to 12. FIG. 1 is a diagram illustrating an overview of the processing system 100. FIG. 2 is a diagram illustrating an example of recognition processing. FIG. 3 is a block diagram illustrating an example configuration of a processing device 200. FIG. 4 is a diagram illustrating an example of item / packaging information 241. FIG. 5 is a diagram illustrating an example processing when determining the height of an image capturing device 410. FIGS. 6 and 7 are diagrams illustrating an example processing when determining the position of the image capturing device 410. FIGS. 8 and 9 are diagrams illustrating an example of recognition target area determination processing. FIG. 10 is a block diagram illustrating an example configuration of a management device 300. FIG. 11 is a flowchart illustrating an example operation of the processing device 200. FIG. 12 is a flowchart illustrating an example operation of the management device 300. Note that in the present disclosure, the drawings may be associated with one or more embodiments.

[0013] In a first embodiment of the present disclosure, as shown in FIG. 1 , a processing system 100 that performs processing such as pick-and-place processing using an arm robot 400 will be described. As will be described later, the processing system 100 of the present disclosure checks the packaging status of items, such as products to be processed, stored inside a package such as a cardboard box, by, for example, referencing item / packaging information 241. Then, based on the check results, the processing system 100 determines a recognition target area, which is a parameter used when performing recognition processing using image data. Thereafter, the processing system 100 recognizes the items to be processed (a group of items) by performing recognition processing on the recognition target area. As a result, the processing system 100 can perform processing such as pick-and-place processing on the recognized group of items.

[0014] For example, the processing system 100 checks whether the items to be processed are packaged in a bundle as the packaging status of the items, by referring to the item / packaging information 241. In other words, the processing system 100 checks whether the items are individually packaged or whether multiple items are packaged together. Then, based on the result of the check, the processing system 100 determines a recognition target area, which is a processing target area when performing recognition processing.

[0015] For example, suppose the recognition engine is trained to recognize a bundle of multiple items packaged together as a single unit. In this case, if multiple bundles of multiple items exist within a package, as illustrated in FIG. 2 , the recognition engine may have difficulty distinguishing between the bundled items. This situation could lead to the processing system 100 being unable to properly identify the target for processing, such as grasping. To address this risk, when the processing system 100 determines that multiple items are packaged together, it predetermines a recognition target area based on information indicating the packaging status of the items and the progress of the processing. This allows the processing system 100 to perform recognition processing within the recognition target area, enabling appropriate recognition processing even when the items are bundled together. As a result, the processing system 100 can properly perform pick-and-place processing, etc.

[0016] In addition, the processing system 100 can check the size of the item to be processed, and determine the height of the imaging device 410 when capturing image data for recognition based on the confirmed size of the item.

[0017] For example, if the position of the imaging device 410 is fixed, the size of the item in the image data captured of the inside of the package may be small, which may result in a decrease in recognition accuracy. Therefore, the processing system 100 determines the height of the imaging device 410 according to the size of the item. As a result, for example, when the processing system 100 determines that the size of the item is small, the imaging device 410 can acquire image data while positioning the imaging device 410 closer to the item. The relationship between the size of the item and the height of the imaging device 410 may be, for example, predetermined heights associated with each predetermined size range, or the height may be configured to decrease as the size of the item decreases.

[0018] In this disclosure, the processing system 100 performs pick-and-place (P&P) processing as an example of an arbitrary process. That is, the processing system 100 can perform a recognition process using image data, and then grasp the recognized item and transfer it to another location. However, the processing system 100 may perform any process other than pick-and-place processing. For example, the processing system 100 may be configured to perform an inspection process, a counting process, a screw tightening or assembly process, or any other process or combination of processes that can be performed in any environment, such as a warehouse.

[0019] FIG. 1 shows an example of the overall configuration of a processing system 100. Referring to FIG. 1, the processing system 100 includes, for example, a processing device 200, a management device 300, and an arm robot 400. An imaging device 410 can be provided at any location, such as the tip of the arm robot 400. As shown in FIG. 1, the processing device 200 and the management device 300 are connected to each other so as to be able to communicate with each other, either wired or wirelessly. The processing device 200 and the arm robot 400 may also be connected to each other so as to be able to communicate with each other, either wired or wirelessly. The arm robot 400 and the imaging device 410, and the processing device 200 and the imaging device 410 may also be connected to each other so as to be able to communicate with each other, either wired or wirelessly.

[0020] The configuration of the processing system 100 may be other than that exemplified in FIG. 1 . For example, the processing system 100 may have an imaging device 410 that is an independent device separate from the arm robot 400 and that can adjust its height and move by itself. The processing system 100 may also have an information processing device that has the functions of the processing device 200 and the management device 300. A plurality of devices may also be configured integrally, such as the arm robot 400 and the processing device 200 being configured integrally. The processing system 100 may have modified examples other than those exemplified above.

[0021] The processing device 200 is an information processing device that determines a recognition target area, which is a parameter for performing recognition processing, based on the results of checking the packaging status of the item to be processed. The processing device 200 also performs recognition processing on the determined recognition target area from image data captured of the interior of the package, and can control the arm robot 400 to perform any processing, such as pick-and-place processing, using the results of the recognition processing. For example, the processing device 200 checks whether the items to be processed are packaged in a bundle, as the packaging status of the items to be processed, by referencing the item / packaging information 241. The processing device 200 then determines the recognition target area based on the results of checking the packaging status of the items. The processing device 200 can also determine the height and position of the imaging device 410 based on the results of checking the size of the items.

[0022] Fig. 3 shows an example of the configuration of the processing device 200. Referring to Fig. 3, the processing device 200 has, as main components, for example, an operation input unit 210, a screen display unit 220, a communication I / F unit 230, a storage unit 240, and an arithmetic processing unit 250.

[0023] 3 illustrates an example in which the functions of the processing device 200 are realized using one information processing device. However, at least some of the functions of the processing device 200 may be realized using multiple information processing devices, for example, on the cloud. Furthermore, the processing device 200 may not include some of the components illustrated above, such as not having the operation input unit 210 or the screen display unit 220, or may have a configuration other than those illustrated above.

[0024] The operation input unit 210 is made up of operation input devices such as a keyboard, a mouse, etc. The operation input unit 210 detects operations of the operator operating the processing device 200 and outputs the operations to the arithmetic processing unit 250.

[0025] The screen display unit 220 is composed of a screen display device such as a liquid crystal display, an organic electroluminescence (EL) display, etc. The screen display unit 220 can display various information stored in the storage unit 240 on the screen in response to instructions from the arithmetic processing unit 250.

[0026] The communication I / F unit 230 includes a data communication circuit, etc. The communication I / F unit 230 performs data communication with external devices such as the management device 300 and the arm robot 400 connected via a communication line.

[0027] The storage unit 240 is a storage device such as a hard disk or memory. The storage unit 240 stores processing information and programs 243 required for various processes in the arithmetic processing unit 250. The programs 243 are read into the arithmetic processing unit 250 and executed to realize various processing units. The programs 243 are read in advance from an external device or recording medium via a data input / output function such as the communication I / F unit 230, and are stored in the storage unit 240. Main information stored in the storage unit 240 includes, for example, product / packaging information 241 and image data information 242.

[0028] The item / packaging information 241 includes information indicating the status of items, such as products, the packaging status of the items, and the packaging in which the items are stored. For example, the item / packaging information 241 may indicate information indicating the status of items stored in each package, such as a cardboard box. As an example, the item / packaging information 241 may be configured to enable confirmation of information indicating the status of items stored in a package based on an identifier, such as a barcode, attached to the package. The item / packaging information 241 is acquired by the item status information acquisition unit 251 using a method such as acquiring information indicating the status of items from the management device 300, and is stored in the storage unit 240.

[0029] FIG. 4 shows an example of information included in the item / packaging information 241. Referring to FIG. 4 , the item / packaging information 241 may include information indicating the status of the item, such as an item ID (identifier), size, manufacturer, weight, image, packaging size, item packaging material, number of items in the package, packaging condition, number of packages, packaging orientation, packaging pattern, and barcode position. Here, the item ID is identification information assigned in advance to each item, such as a commodity. The size may indicate the size of the item. For example, the size may indicate the dimensions of the item, such as length, width, and height. The manufacturer may indicate the manufacturer of the item. The manufacturer may be identification information corresponding to the manufacturer. The weight may indicate the weight of one item. The image may indicate image data of the item. The image may include multiple image data of the item captured from different directions. The package size may indicate the size of a package, such as a cardboard box. The package size may indicate the dimensions of the package, such as length, width, and height. Furthermore, the item packaging material can indicate the material of the item packaging (e.g., commercial packaging), such as plastic wrap or paper. The number of items in a package can indicate the number of items stored in the package. The packaging state can indicate whether the item is individually wrapped (e.g., single item, strip wrap, full wrap), whether it is bundled, and how it is bundled. The number of packages can indicate how many items are contained in one package when multiple items are bundled. In other words, the number of packages can indicate how many items are bundled together. In this way, the packaging status of the items can be indicated by information indicating the packaging state and the number of packages. The packaging orientation can indicate the inclination of the package or the items stored in the package, such as rotated x degrees. The packaging pattern can indicate how a group of packaged items is stored in the package, such as 4 rows x 2 columns x 3 rows. The barcode position can also indicate the position of the barcode attached to the product (or a package containing multiple products), such as the side, top, or bottom.The barcode position may indicate both the position of the barcode attached to the item and the position of the barcode attached to a package that bundles multiple items together.

[0030] For example, at least some of the information listed above is included in the product / packaging information 241. The product / packaging information 241 may also include information about products and packages other than those listed above, as well as other information that may be held by a warehouse management system or the like.

[0031] The image data information 242 includes image data captured by the imaging device 410. For example, the image data information 242 is updated in response to the image data acquisition unit 254 acquiring image data from the imaging device 410.

[0032] The arithmetic processing unit 250 has an arithmetic device such as a CPU (Central Processing Unit) and its peripheral circuits. The arithmetic processing unit 250 reads and executes a program 243 from the storage unit 240, thereby realizing various processing functions by causing the above hardware and the program 243 to cooperate with each other. Major processing functions realized by the arithmetic processing unit 250 include, for example, an item status information acquisition unit 251, an imaging position determination unit 252, an imaging device control unit 253, an image data acquisition unit 254, a recognition target area determination unit 255, a recognition unit 256, and an execution control unit 257.

[0033] In addition, the arithmetic processing unit 250 may have a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination of these, instead of the above-mentioned CPU.

[0034] The item status information acquisition unit 251 acquires information indicating the status of an item, such as information indicating the packaging status of the item and information indicating the size of the item. For example, the item status information acquisition unit 251 can acquire information indicating the status of an item, such as the packaging status of an item stored in a package, by referring to the item / packaging information 241. The item status information acquisition unit 251 may identify a package by, for example, reading a barcode attached to the side of the package, and acquire information indicating the status of the item corresponding to the identified package from the item / packaging information 241. The item status information acquisition unit 251 can acquire the above information at any timing, such as when it is recognized using a sensor or the like that a package to be subjected to pick-and-place (P&P) processing has arrived at a predetermined position.

[0035] Note that the product status information acquisition unit 251 may acquire at least a part of the information indicating the product status based on image data capturing images of the package and the products inside the package, instead of or in addition to referring to the product / packaging information 241. The product status information acquisition unit 251 may acquire at least a part of the information indicating the product status using any means other than those exemplified above, such as using a sensor that measures weight.

[0036] The imaging position determination unit 252 determines the height and position of the imaging device 410 when acquiring image data to be subjected to recognition processing, using information indicating the status of the item acquired by the item status information acquisition unit 251. For example, the imaging position determination unit 252 determines the height of the imaging device 410, using information indicating the status of the item. After determining the height, the imaging position determination unit 252 can determine the position of the imaging device 410 on a two-dimensional plane at the determined height.

[0037] FIG. 5 shows an example of processing performed by the imaging position determination unit 252 when determining the height of the imaging device 410. As illustrated in FIG. 5, the imaging position determination unit 252 checks the size of the object to be processed as the state of the object. The imaging position determination unit 252 can then determine the height of the imaging device 410 based on the results of this check. For example, the imaging position determination unit 252 can determine the height of the imaging device 410 so that the distance between the imaging device 410 and the object becomes shorter as the size of the object becomes smaller. As an example, the height of the imaging device 410 is predetermined for each predetermined range of object sizes. For example, the imaging position determination unit 252 can identify a range to which the size of the object belongs and determine a height corresponding to the identified range. The imaging position determination unit 252 may also determine the height of the imaging device 410 based on the size of the object, for example, so that the height becomes lower as the size of the object becomes smaller.

[0038] The imaging position determination unit 252 can determine, for example, the distance between the top end of the package (or the top end of the item stored in the topmost layer of the package) and the imaging device 410 as a value indicating the height of the imaging device 410. The imaging position determination unit 252 may determine, as a value indicating the height of the imaging device 410, a value that does not depend on the size of the package, such as the distance between the bottom end of the package, such as the top surface position of a belt conveyor, and the imaging device 410. The imaging position determination unit 252 can also confirm the size of each item as information indicating the size of the item. In cases where multiple items are packaged together, for example, the imaging position determination unit 252 may confirm the size of the bundled items. In other words, the imaging position determination unit 252 may determine the height based on the size of each item, or based on the size of the bundled and packaged items.

[0039] After determining the height, the imaging position determination unit 252 can determine the position of the imaging device 410 on a two-dimensional plane at the determined height. For example, as illustrated in Fig. 6, the imaging position determination unit 252 can determine the position of the imaging device 410 on a two-dimensional plane according to the range that the imaging device 410 can capture at the determined height and the size of the package.

[0040] For example, if the size of the package fits within the imageable range, the imaging position determination unit 252 can determine the position of the imaging device 410 so that the package fits within the imageable range. The imaging position determination unit 252 may determine the position of the imaging device 410 so that the position of the imaging device 410 is as close to the center of the package as possible within the imageable range. On the other hand, if the size of the package does not fit within the imageable range, the imaging position determination unit 252 determines that image data will be acquired multiple times, as shown in FIG. 7 . Furthermore, when acquiring image data multiple times, the imaging position determination unit 252 can determine the position of the imaging device 410 so that the entire package can be imaged in as few times as possible. For example, depending on the size of the item, the imaging device 410 may get too close to the package and may not be able to capture the entire package in a single image taken from above the package. The imaging position determination unit 252 determines the height and then determines the position of the imaging device 410 on a two-dimensional plane at the determined height, so that image data can be appropriately acquired even in the above-mentioned cases.

[0041] The imaging device control unit 253 moves the imaging device 410 to the height and position determined by the imaging position determination unit 252. For example, the imaging device control unit 253 can move the imaging device 410 to the height and position determined by the imaging position determination unit 252 by instructing the arm robot 400 to move the imaging device 410 to the determined position and height in accordance with the determination result by the imaging position determination unit 252.

[0042] Furthermore, the imaging device control unit 253 may instruct the imaging device 410 to acquire image data at the height and position determined by the imaging position determination unit 252 .

[0043] The image data acquisition unit 254 acquires image data captured by the imaging device 410 from the imaging device 410. The image data acquisition unit 254 can also store the acquired image data in the storage unit 240 as image data information 242.

[0044] The recognition target area determination unit 255 uses information indicating the packaging status of the item acquired by the item status information acquisition unit 251 to determine a recognition target area, which is an area of ​​the image data acquired by the image data acquisition unit 254 that is the target for recognition processing.

[0045] For example, the recognition target area determination unit 255 checks whether the items to be processed are packaged in a bundle as the packaging status of the items. In other words, the recognition target area determination unit 255 checks whether the items are individually packaged or whether multiple items are packaged together. Then, based on the result of the check, the processing system 100 determines the recognition target area, which is the processing target area when performing the recognition process.

[0046] For example, if the above check determines that the items are not bundled, the recognition target area determination unit 255 determines not to determine a recognition target area. In other words, if the items are individually packaged, the recognition target area determination unit 255 determines not to determine a recognition target area. In this case, the recognition target area determination unit 255 can subject the entire image data to recognition processing, as shown in FIG. 8. In other words, if it is determined that the items are not bundled, the recognition target area determination unit 255 can determine the entire image data as the recognition target area.

[0047] On the other hand, if the above check determines that the items are bundled, the recognition target area determination unit 255 calculates the size of the items in the bundled state from the number of packages and the size of the items included in the information indicating the item status. Then, as shown in FIG. 9 , the recognition target area determination unit 255 determines the recognition target area based on the calculated size, the progress of the pick-and-place process, and other factors. For example, the recognition target area determination unit 255 checks how the bundled items (items in the bundle) are stored within the package based on the size of the items in the bundled state and the information indicating the packaging pattern. Then, the recognition target area determination unit 255 determines, as the recognition target area, a range corresponding to the items in the bundle selected according to predetermined criteria from among multiple items in the bundle present within the package. Furthermore, the recognition target area determination unit 255 can determine, as the recognition target area, a range corresponding to the items in the bundle to be grasped next each time a pick-and-place process is performed. In other words, the recognition target area determination unit 255 can determine a new recognition target area each time a pick-and-place process is performed. Note that the criteria for selecting an item in a bundle may be set arbitrarily, such as the item located in the upper left corner of the image data.

[0048] The recognition unit 256 performs recognition processing to recognize the range of an article to be grasped at one time within the range determined by the recognition target area determination unit 255. That is, the recognition unit 256 performs recognition processing on the recognition target area determined by the recognition target area determination unit 255. For example, the recognition unit 256 may perform the recognition processing using any means, such as using a recognition engine that has been trained in advance. Note that the recognition engine used by the recognition unit 256 when performing the recognition processing may be common to all articles, or may be selected appropriately depending on the article to be recognized, for example.

[0049] For example, if the recognition target area determination unit 255 determines the entire image data as the recognition target area, the recognition unit 256 performs recognition processing on the entire image data, and as a result, the recognition unit 256 can recognize each individual item stored inside the package as a target to be grasped.

[0050] Furthermore, when the recognition target area determination unit 255 determines the range corresponding to the bundle of items as the recognition target area, the recognition unit 256 performs recognition processing on the recognition target area in the image data. Furthermore, at this time, the recognition unit 256 performs recognition processing using the recognition engine used to recognize the bundle of items. As a result, the recognition unit 256 can recognize the bundle of items corresponding to the recognition target area as the object to be grasped, among the items stored inside the package.

[0051] The execution control unit 257 controls the execution of processes such as pick-and-place processing using the result of the recognition processing by the recognition unit 256. For example, the execution control unit 257 can realize the above control by instructing the arm robot 400 to grasp the object to be grasped recognized by the recognition unit 256 and move the article to a predetermined location. The execution control unit 257 may control the execution of processes such as pick-and-place processing using any means.

[0052] The execution control unit 257 may instruct the recognition target area determination unit 255 to determine the next recognition target area each time the arm robot 400 executes a process such as a pick-and-place process. Furthermore, when the recognition unit 256 recognizes each of the individual objects stored inside the package as a gripping target, the execution control unit 257 may select, by any means, from the recognized articles, an article to be gripped when the next process is executed.

[0053] The above is an example of the configuration of the processing device 200. The processing device 200 may perform any processing according to information indicating the status of the item, in addition to determining the recognition target area and the height and position of the imaging device 410. For example, the processing device 200 may instruct the arm robot 400 to change the type of hand used when the arm robot 400 grasps the item, according to information indicating the packaging material, etc., among the information indicating the status of the item. The processing device 200 can perform any processing in addition to the above.

[0054] The management device 300 is an information processing device that manages information indicating the status of packages and items stored in the packages, packaging status, etc. in a warehouse, other logistics base, etc. The management device 300 can have general functions that are included in a management system such as a WMS (Warehouse Management System) that manages information about items.

[0055] Fig. 10 shows an example of the configuration of the management device 300. Referring to Fig. 10, the management device 300 has, as main components, for example, an operation input unit 310, a screen display unit 320, a communication I / F unit 330, a storage unit 340, and an arithmetic processing unit 350.

[0056] The management device 300 may have various modified examples similar to those of the processing device 200 described with reference to Fig. 3. The configurations of the operation input unit 310, the screen display unit 320, and the communication I / F unit 330 may be similar to those of the operation input unit 210, the screen display unit 220, and the communication I / F unit 230 described above.

[0057] The storage unit 340 is a storage device such as a hard disk or memory. The storage unit 340 stores processing information and programs 342 required for various processes in the arithmetic processing unit 350. The programs 342 are read into the arithmetic processing unit 350 and executed to realize various processing units. The programs 342 are read in advance from an external device or recording medium via a data input / output function such as the communication I / F unit 330, and are stored in the storage unit 340. Main information stored in the storage unit 340 includes, for example, item and packaging information 341.

[0058] The product / packaging information 341 includes information indicating the status of the product, such as information about the product, the packaging status of the product, the packaging in which the product is stored, etc. The product / packaging information 341 can include at least a portion of the information included in the product / packaging information 241 described above.

[0059] Of the information included in the item and packaging information 341, information about the items stored inside the package, such as the item ID, size, manufacturer, weight, image, and package size, is held by a general WMS or other management system. Therefore, the management device 300 may acquire information about the items from a general WMS or other management system and store it in the memory unit 340. On the other hand, among the information included in the item and packaging information 341, information indicating the packaging status of the items, such as the item packaging material, the number of items in the package, the packaging condition, the number of packages, the packaging orientation, the packaging pattern, and the barcode position, as well as information about the package, are not held by a general WMS or other management system. Therefore, the management device 300 can acquire the above-mentioned information using various functions realized by the calculation processing unit 350 (described below) that function as an acquisition unit for acquiring information. For example, as described above, the item and packaging information 341 can manage the information acquired by each function (described below) by adding it to the information about the items held by a management system such as a WMS.

[0060] The arithmetic processing unit 350 includes an arithmetic device such as a CPU and its peripheral circuits. The arithmetic processing unit 350 reads and executes a program 342 from the storage unit 340, thereby causing the hardware and the program 342 to cooperate to realize various processing units. Major processing units realized by the arithmetic processing unit 350 include, for example, an input receiving unit 351, an estimation unit 352, an information acquisition unit 353, and a transmission unit 354. For example, of the above processing units, the input receiving unit 351, the estimation unit 352, and the information acquisition unit 353 function as acquisition units that acquire information indicating the status of the item. Note that the arithmetic processing unit 350 may include a GPU or the like instead of a CPU, similar to the arithmetic processing unit 250 described above.

[0061] The input receiving unit 351 receives information indicating the status of the item by receiving input from an operator using the operation input unit 310, etc. For example, the input receiving unit 351 can receive information indicating the status of the item by displaying predetermined check boxes, etc. on the screen display unit 320 and receiving input from the operator using the operation input unit 310. The input receiving unit 351 also stores the received information in the storage unit 340 as item / packaging information 341.

[0062] For example, the input receiving unit 351 can receive input of at least a portion of information indicating the status of an item, such as the material of the item packaging, the state of the packaging, the packing pattern, etc. In other words, the input receiving unit 351 can receive input of information indicating the packaging status of an item. The input receiving unit 351 may also receive input of information other than the above examples.

[0063] The estimation unit 352 uses information included in the product and packaging information 341, arbitrary sensing results, and the like to estimate information indicating the status of the product that is not included in the product and packaging information 341. In other words, the estimation unit 352 can use information indicating the status of the product that is already stored, and the like to estimate information indicating the status of the product that is different from the information indicating the status of the product that is already stored. Furthermore, the estimation unit 352 stores the estimation result in the storage unit 340 as the product and packaging information 341.

[0064] For example, the estimation unit 352 uses a sensor or the like to measure the weight of the entire package with items stored inside. Then, the estimation unit 352 estimates the number of items in the package based on the weight or the like included in the item / packaging information 341. For example, the estimation unit 352 can estimate the number of items by dividing the weight of the entire package by the weight of each item. Note that the estimation unit 352 may perform estimations other than those exemplified above, such as estimating a packaging pattern based on information such as size, package size, number of items in the package, and number of packages. Note that if there are multiple estimable packaging patterns, the estimation unit 352 may estimate each of the multiple estimable packaging patterns, or may narrow down the packaging patterns to be estimated based on any criteria.

[0065] The information acquisition unit 353 acquires information indicating the status of the item from an external device connected via the communication I / F unit 330. For example, the information acquisition unit 353 can acquire information indicating the status of the item from a predetermined external device, such as a database of a manufacturer that produced the item, such as a commercial product. The information acquisition unit 353 also stores the acquired information in the storage unit 340 as item / packaging information 341.

[0066] For example, the information acquisition unit 353 can acquire at least some of the information indicating the status of the item, such as the number of packages, the orientation when packaged, the position of the barcode, etc., from an external device. In other words, the information acquisition unit 353 can acquire information indicating the packaging status of the item. The information acquisition unit 353 may acquire information other than the above-mentioned examples from an external device.

[0067] The transmitting unit 354 transmits the product and packaging information 341 to the processing device 200. The transmitting unit 354 may transmit the product and packaging information 341 to the processing device 200 at any timing.

[0068] The above is an example of the configuration of the management device 300. The management device 300 may acquire information indicating the status of an item using a method other than the above example. For example, the management device 300 may acquire information indicating the status of an item using a method other than the above example, such as performing image analysis on image data of the inside of a package to acquire information indicating the material or condition of the packaging, information indicating the position of a barcode, etc.

[0069] The arm robot 400 performs any process, such as pick-and-place processing, in response to an instruction from the processing device 200. As described above, an imaging device 410 is provided near the tip of the arm robot 400, for example. Note that in the present disclosure, the configuration of the arm robot 400 is not particularly limited. The arm robot 400 may have any configuration.

[0070] The imaging device 410 captures image data of the package or the items stored in the package in response to instructions from the processing device 200. The imaging device 410 may be any device with an imaging function, such as a camera. The imaging device 410 can also transmit the captured image data to the processing device 200.

[0071] The above is an example of the configuration of the processing system 100. Note that at least some of the functions of the management device 300 may be provided by the processing device 200. In other words, the processing device 200 may have a means for acquiring information indicating the status of an item.

[0072] Next, an example of the operation of the processing device 200 and the management device 300 will be described with reference to Fig. 11 and Fig. 12. First, an example of the operation of the processing device 200 will be described with reference to Fig. 11.

[0073] 11 is a flowchart showing an example of the operation of the processing device 200. Referring to FIG. 11, the product status information acquisition unit 251 acquires information indicating the status of the product (step S101). For example, the product status information acquisition unit 251 can acquire information indicating the status of the product stored in the package by referring to the product / packaging information 241.

[0074] The imaging position determination unit 252 determines the height and position of the imaging device 410 when acquiring image data to be subjected to recognition processing, using the information indicating the status of the item acquired by the item status information acquisition unit 251 (step S102). For example, the imaging position determination unit 252 determines the height of the imaging device 410, using the information indicating the status of the item. After determining the height, the imaging position determination unit 252 can determine the position of the imaging device 410 on a two-dimensional plane at the determined height.

[0075] The imaging device control unit 253 moves the imaging device 410 to the height and position determined by the imaging position determination unit 252. Thereafter, the imaging device 410 captures image data. Then, the image data acquisition unit 254 acquires the image data captured by the imaging device 410 from the imaging device 410 (step S103).

[0076] The recognition target area determination unit 255 checks the information indicating the packaging status of the item acquired by the item status information acquisition unit 251. For example, the recognition target area determination unit 255 checks, as the packaging status of the item, whether or not the items to be processed are packaged in a bundle (step S104).

[0077] If it is determined as a result of the above check that the items are not bundled (step S104, individually wrapped), the recognition target area determination unit 255 determines not to determine a recognition target area. In this case, the recognition target area determination unit 255 can target the entire image data for recognition processing, as shown in Fig. 8. In other words, if it is determined that the items are not bundled, the recognition target area determination unit 255 can determine the entire image data as the recognition target area.

[0078] On the other hand, if it is determined that the items are bundled (step S104, multiple items bundled), the recognition target area determination unit 255 calculates the size of the bundled items from the number of packages and the size of the items included in the information indicating the item status. Then, as shown in Figure 9, the recognition target area determination unit 255 determines the recognition target area based on the calculated size, the progress of the pick-and-place process, etc. (step S105).

[0079] The recognition unit 256 performs a recognition process to recognize the article to be grasped in one attempt (step S106) for the range determined by the recognition target area determination unit 255. For example, the recognition unit 256 may perform the recognition process using any means, such as using a recognition engine that has been trained in advance.

[0080] The execution control unit 257 controls the execution of processes such as pick-and-place processing using the result of the recognition process by the recognition unit 256 (step S107). For example, the execution control unit 257 can realize the above control by instructing the arm robot 400 to grasp the object to be grasped recognized by the recognition unit 256 and move the object to a predetermined location.

[0081] The above is an example of the operation of the processing device 200. Next, an example of the operation of the management device 300 will be described with reference to FIG.

[0082] FIG. 12 is a flowchart showing an example of the operation of the management device 300. Referring to FIG. 12, the input receiving unit 351 receives information indicating the status of an item by receiving input from an operator using the operation input unit 310 or the like. The information acquiring unit 353 acquires information indicating the status of the item from an external device connected via the communication I / F unit 330. For example, as described above, the management device 300 acquires information indicating the status of the item using the input receiving unit 351, the information acquiring unit 353, and the like (step S201). Furthermore, the management device 300 can acquire information about the items stored inside the package from a management system such as a general WMS.

[0083] The estimation unit 352 uses information indicating the status of the items acquired by the input reception unit 351, the information acquisition unit 353, etc. to estimate information indicating the status of the items that differs from the information indicating the status of the items acquired by the input reception unit 351, the information acquisition unit 353, etc. (step S202). For example, the estimation unit 352 can estimate the number of items in a package based on information indicating the weight of the items or the number of items in the package.

[0084] The above is an example of the operation of the management device 300. The management device 300 may transmit information indicating the acquired and estimated status of the item to the processing device 200 at any timing.

[0085] As described above, the processing device 200 includes an item status information acquisition unit 251 and a recognition target area determination unit 255. With this configuration, the recognition target area determination unit 255 can determine a recognition target area, which is an area of ​​the image data that is the target of recognition processing, using information indicating the packaging status of the items acquired by the item status information acquisition unit 251. With this configuration, for example, when it is determined that multiple items are bundled together, the recognition target area determination unit 255 can determine the recognition target area according to the information indicating the status of the items. This makes it possible to perform appropriate recognition processing even when the items are bundled together.

[0086] The processing device 200 also has an imaging device control unit 253. With this configuration, the imaging device control unit 253 can use information indicating the status of the item acquired by the item status information acquisition unit 251 to determine the height and position of the imaging device 410 when acquiring image data to be subjected to recognition processing. With this configuration, the imaging device 410 can capture image data at an appropriate position according to the size of the item. This allows for more appropriate recognition processing.

[0087] Furthermore, the management device 300 includes an input receiving unit 351, an estimation unit 352, an information acquisition unit 353, etc. With this configuration, the management device 300 can acquire information indicating the status of each item using the input receiving unit 351, the estimation unit 352, and the information acquisition unit 353. As a result, the processing device 200 can determine the recognition target area, etc., using the information indicating the status of the item acquired by the management device 300. This enables more appropriate recognition processing to be performed.

[0088] Second Embodiment Next, a processing device 500 and a management device 600 will be described with reference to Figs. 13 to 16. Fig. 13 is a diagram showing an example of the hardware configuration of the processing device 500. Fig. 14 is a block diagram showing an example of the configuration of the processing device 500. Fig. 15 is a flowchart showing an example of the operation of the processing device 500. Fig. 16 is a block diagram showing an example of the configuration of the management device 600.

[0089] In a second embodiment of the present disclosure, a processing device 500 will be described, which is an information processing device that determines a recognition target area, which is a parameter for performing the recognition process, in accordance with the confirmation result of information indicating the packaging status of an item, and performs the recognition process. Also in the second embodiment, a management device 600 will be described. Fig. 13 shows an example of the hardware configuration of the processing device 500. Referring to Fig. 13, the processing device 500 has, as an example, the following hardware configuration. CPU (Central Processing Unit) 501 (arithmetic device) ROM (Read Only Memory) 502 (storage device) RAM (Random Access Memory) 503 (storage device) Programs 504 loaded into RAM 503 Storage device 505 for storing the programs 504 Drive device 506 for reading and writing data from and to a storage medium 510 external to the information processing device Communication interface 507 for connecting to a communication network 511 external to the information processing device Input / output interface 508 for inputting and outputting data Bus 509 for connecting the various components

[0090] 14 by the CPU 501 acquiring the program group 504 and executing it. The program group 504 is stored in advance in the storage device 505 or the ROM 502, for example, and is loaded into the RAM 503 or the like by the CPU 501 for execution as needed. The program group 504 may be supplied to the CPU 501 via the communication network 511, or may be stored in advance in the recording medium 510, and the drive device 506 may read out the program and supply it to the CPU 501.

[0091] 13 shows an example of the hardware configuration of the processing device 500. The hardware configuration of the processing device 500 is not limited to the above-described case. For example, the processing device 500 may be configured with only a part of the above-described configuration, such as excluding the drive device 506. Furthermore, the CPU 501 may be a GPU or the like exemplified in the first embodiment.

[0092] The acquisition unit 521 acquires information indicating the packaging status of the items stored in the package. For example, the acquisition unit 521 may acquire the information indicating the packaging status of the items by referring to information stored in advance in a storage device.

[0093] The determining unit 522 determines a recognition target area, which is an area of ​​the image data that is a target for recognition processing, as a parameter for performing recognition processing using image data of the item as a subject, in accordance with the information indicating the packaging status of the item acquired by the acquiring unit 521. The determining unit 522 may determine, as a parameter for performing recognition processing, the recognition target area, which is an area of ​​the image data that is a target for recognition processing, as well as the height of the imaging device when acquiring the image data.

[0094] The recognition unit 523 performs recognition processing on the recognition target area determined by the determination unit 522 .

[0095] The above is an example of the configuration of the processing device 500. Next, an example of the operation of the processing device 500 will be described with reference to FIG.

[0096] Fig. 15 shows an example of the operation of the processing device 500. Referring to Fig. 15, the acquisition unit 521 acquires information indicating the packaging status of the items stored in the package (step S301).

[0097] The determination unit 522 determines a recognition target area, which is an area of ​​the image data that is the target for recognition processing, as a parameter when performing recognition processing using image data with the item as the subject, in accordance with the information indicating the packaging status of the item acquired by the acquisition unit 521 (step S302).

[0098] The recognition unit 523 performs a recognition process on the recognition target area determined by the determination unit 522 (step S303).

[0099] As described above, the processing device 500 includes an acquisition unit 521, a determination unit 522, and a recognition unit 523. With this configuration, the determination unit 522 can determine a recognition target area, which is a parameter for performing recognition processing using image data of the item as a subject, in accordance with information indicating the packaging status of the item acquired by the acquisition unit 521. This allows the recognition unit 523 to perform recognition processing in accordance with the determined recognition target area. In other words, with the above configuration, the processing device 500 can perform appropriate recognition processing in accordance with the status of the item.

[0100] The processing device 500 described above can be realized by incorporating a predetermined program into an information processing device such as the processing device 500. Specifically, a program according to another aspect of the present disclosure is a program for realizing processing in an information processing device, which acquires information indicating the packaging status of an item stored inside a package, determines a recognition target area, which is an area of ​​the image data that is the target of the recognition processing, as a parameter for performing recognition processing using image data of the item as a subject, according to the acquired information indicating the packaging status of the item, and performs recognition processing on the determined recognition target area.

[0101] Furthermore, a processing method executed by an information processing device such as the processing device 500 described above is a method in which the information processing device acquires information indicating the packaging status of an item stored inside a package, and, in accordance with the acquired information indicating the packaging status of the item, determines a recognition target area, which is an area of ​​the image data that is the target of recognition processing, as a parameter when performing recognition processing using image data with the item as the subject, and performs recognition processing on the determined recognition target area.

[0102] Any program having the above-described configuration, or a computer-readable recording medium having the program recorded thereon, or a processing method, etc., can achieve the same functions and effects as the above-described processing device 500, and therefore can achieve the above-described objective of the present disclosure.

[0103] Furthermore, the management device 600 adds information indicating the packaging status of the above-mentioned items to information about the items managed by a warehouse management system (WMS) and manages the information. FIG. 16 shows an example of the configuration of the management device 600. Referring to FIG. 16, the management device 600 has a storage device 622. Furthermore, the management device 600 can realize the function of the acquisition unit 621 shown in FIG. 16 by having a CPU acquire and execute a group of programs. The hardware configuration of the management device 600 may be the same as that of the processing device 500 described with reference to FIG. 13.

[0104] The acquisition unit 621 acquires information indicating the status of items that is used when performing recognition processing using image data of items stored inside a package, which is different from information about items managed by a warehouse management system (WMS) that manages information about items. For example, the acquisition unit 621 can acquire, as information indicating the status of items, at least some of the following: the size of the package, information indicating the packaging status of multiple items in the package, and information indicating the number of items stored inside the package. The acquisition unit 621 also stores the acquired information indicating the status of items in the storage device 622.

[0105] For example, the acquisition unit 621 may acquire information indicating the status of the item using at least one of the following methods: making an estimation using information stored in the storage device 622, acquiring the information from a specified external device, or accepting input of information.

[0106] As described above, the management device 600 includes the acquisition unit 621 and the storage device 622. With this configuration, the storage device 622 can store and manage information indicating the status of the item acquired by the acquisition unit 621. As a result, the above-described processing device 500 and the like can determine parameters according to the information acquired from the management device 600. This allows for appropriate recognition processing.

[0107] The above-described management device 600 can be realized by incorporating a predetermined program into an information processing device such as the management device 600. Specifically, a program according to another embodiment of the present disclosure is a program for realizing a process in which an information processing device acquires, in a storage device, information indicating the status of an item to be used when performing recognition processing using image data of an item stored inside a package, which information is different from information about the item managed by a management system (WMS: Warehouse Management System) that manages information about the item, and the storage device adds the acquired information indicating the status of the item to information managed by the management system and manages the information, and when acquiring the information indicating the status of the item, acquires at least some of the size of the package, information indicating the packaging status of multiple items in the package, and information indicating the number of items stored inside the package as the information indicating the status of the item.

[0108] Furthermore, a management method executed by an information processing device such as the management device 600 described above is a method in which the information processing device acquires information indicating the status of items used when performing recognition processing using image data of items stored inside a package, which is different from information about items managed by a management system (WMS: Warehouse Management System) that manages information about items, and in a storage device, adds the acquired information indicating the status of the items to the information managed by the management system and manages it, and when acquiring the information indicating the status of the items, acquires at least some of the information indicating the status of the items, including the size of the package, information indicating the packaging status of multiple items in the package, and information indicating the number of items stored inside the package, as the information indicating the status of the items.

[0109] Any program having the above-described configuration, or a computer-readable recording medium having the program recorded thereon, or a management method, etc., can achieve the same functions and effects as the above-described management device 600, and therefore can achieve the above-described objective of the present disclosure.

[0110] <Supplementary Notes> Part or all of the above-described embodiments can also be described as follows: Below, an outline of the processing device and the like in the present disclosure will be described. However, the present disclosure is not limited to the following configuration.

[0111] (Supplementary Note 1) A processing device comprising: an acquisition unit that acquires information indicating the packaging status of items stored inside a package; a determination unit that determines a recognition target area in the image data that is a target area for recognition processing as a parameter for processing image data of the items as a subject, in accordance with the information indicating the packaging status of the items acquired by the acquisition unit; and a recognition unit that performs recognition processing on the recognition target area determined by the determination unit. (Supplementary Note 2) The processing device according to Supplementary Note 1, wherein the acquisition unit acquires information indicating whether the items are packaged in a bundled state within the package as information indicating the packaging status of the items, and the determination unit determines the recognition target area in the image data that is a target area for processing recognition, in accordance with whether the items are packaged in a bundled state. (Supplementary Note 3) The processing device according to Supplementary Note 2, wherein, in the case where the items are packaged in a bundled state, the determination unit checks how the bundled items are stored in the package, and determines the recognition target area in accordance with the check result. (Supplementary Note 4) The processing device according to Supplementary Note 3, wherein the determination unit determines, as the recognition target area, an area corresponding to a bundled item selected according to a predetermined criterion from among a plurality of bundled items present in the package. (Supplementary Note 5) The processing device according to any one of Supplementary Notes 2 to 4, wherein, when the plurality of items are packaged in a bundled state, the determination unit determines a new recognition target area each time a process is executed according to a result of recognition processing by the recognition unit. (Supplementary Note 6) The processing device according to any one of Supplementary Notes 2 to 5, wherein, when the items are not bundled, the determination unit determines the entire image data as the recognition target area. (Supplementary Note 7) The processing device according to any one of Supplementary Notes 1 to 6, wherein the acquisition unit acquires information indicating the size of the item, and includes an imaging position determination unit that determines a height of an imaging device when acquiring image data to be subjected to recognition processing according to the size of the item.(Supplementary Note 8) The processing device according to Supplementary Note 7, wherein the imaging position determination unit determines a height of the imaging device so that the distance between the imaging device and the item becomes shorter as the size of the item becomes smaller. (Supplementary Note 9) The processing device according to Supplementary Note 7 or Supplementary Note 8, wherein the imaging position determination unit determines the height of the imaging device and then determines the position of the imaging device on a two-dimensional plane at the determined height. (Supplementary Note 10) The processing device according to Supplementary Note 9, wherein the imaging position determination unit determines the position of the imaging device on a two-dimensional plane according to a range that the imaging device can capture at the determined height and the size of the package. (Supplementary Note 11) A processing method in which an information processing device acquires information indicating a packaging state of an item stored inside a package, and determines a recognition target area, which is an area in the image data that is to be subjected to recognition processing, as a parameter for performing recognition processing using image data having the item as a subject, according to the acquired information indicating the packaging state of the item, and performs recognition processing on the determined recognition target area. (Supplementary Note 12) A computer-readable recording medium having recorded thereon a program for causing an information processing device to acquire information indicating the packaging status of an item stored inside a package, determine a recognition target area, which is an area of ​​the image data that is the target of the recognition processing, as a parameter when performing recognition processing using image data of the item as a subject, according to the acquired information indicating the packaging status of the item, and perform recognition processing on the determined recognition target area.(Supplementary Note 21) A management device comprising: an acquisition unit that acquires information indicating the status of an item to be used when performing recognition processing using image data of an item stored inside a package, the information being different from information about the item managed by a management system (WMS: Warehouse Management System) that manages information about the item; and a storage device that adds the information indicating the status of the item acquired by the acquisition unit to information managed by the management system and manages the information, wherein the acquisition unit acquires, as the information indicating the status of the item, at least some of the size of the package, information indicating the packaging status of the multiple items in the package, and information indicating the number of the items stored inside the package. (Supplementary Note 22) The management device according to Supplementary Note 21, wherein the acquisition unit acquires the information indicating the status of the item by using the information indicating the status of the item stored in the storage device to estimate information indicating the status of the item that is different from the information indicating the status of the item stored in the storage device. (Supplementary Note 23) A management device according to Supplementary Note 21 or Supplementary Note 22, wherein the storage device stores information indicating the weight of each of the items as information indicating the status of the items, and the acquisition unit acquires the information indicating the status of the items by estimating the number of items stored in the packaging as information indicating the status of the items based on the weight of the entire packaging acquired by any method and the information indicating the weight of each of the items. (Supplementary Note 24) A management device according to any one of Supplementary Note 21 to Supplementary Note 23, wherein the storage device stores information indicating the status of the items, including information indicating the size of the items, information indicating the size of the packages, and information indicating the number of the items bundled together; the acquisition unit acquires the information indicating the status of the items by estimating a packaging pattern indicating how the items are stored in the packages based on the information stored in the storage device and the estimated number of the items; and the packaging pattern estimated by the acquisition unit is used when determining a recognition target area, which is an area of ​​the image data that is the target for the recognition process, when performing the recognition process.(Supplementary Note 25) The management device according to any one of Supplements 21 to 24, wherein the storage device stores information indicating whether multiple items are bundled together in the package, and the information indicating whether multiple items are bundled together in the package is used when determining a recognition target area in image data that is a target area for the recognition process when performing the recognition process. (Supplementary Note 26) The management device according to any one of Supplements 21 to 25, wherein the storage device stores information indicating a size of the item, and the information indicating the size of the item is used when determining a height of an imaging device when acquiring image data that is the target for the recognition process when performing the recognition process. (Supplementary Note 27) The management device according to any one of Supplements 21 to 26, wherein the acquisition unit acquires information indicating the status of the item by accepting input from an operator. (Supplementary Note 28) The management device according to any one of Supplementary Notes 21 to 27, wherein the acquisition unit acquires information indicating the status of the items from a predetermined external device, etc. (Supplementary Note 29) A management method, wherein an information processing device acquires information indicating the status of the items to be used when performing recognition processing using image data of items stored inside a package, the information being different from information about the items managed by a management system (WMS: Warehouse Management System) that manages information about the items, and a storage device adds the acquired information indicating the status of the items to information managed by the management system and manages it, and when acquiring the information indicating the status of the items, acquires at least some of the size of the package, information indicating the packaging status of the multiple items in the package, and information indicating the number of the items stored inside the package as the information indicating the status of the items.(Supplementary Note 30) A computer-readable recording medium having recorded thereon a program for implementing the following processes: (1) acquiring, in an information processing device, information indicating the status of an item stored inside a package, which is different from information about the item managed by a management system (WMS: Warehouse Management System) that manages information about the items, and which is used when performing recognition processing using image data of the item stored inside the package; (2) managing, in a storage device, adding the acquired information indicating the status of the item to information managed by the management system and managing the information; and (3) acquiring, as the information indicating the status of the item, at least some of the size of the package, information indicating the packaging status of the multiple items in the package, and information indicating the number of the items stored inside the package.

[0112] Note that some or all of the configurations described in Supplementary Notes 2 to 10 that are dependent on the processing device described in Supplementary Note 1 may also be dependent on the processing method described in Supplementary Note 11, the recording medium described in Process 12, and the like in a similar dependent relationship. Furthermore, not limited to Supplementary Notes 11 and 12, but within the scope of each of the above-mentioned embodiments, some or all of the configurations described as the Supplements may also be dependent on various hardware, software, various recording means for recording software, or systems. Similarly, some or all of the configurations described in Supplementary Notes 22 to 28 that are dependent on the management device described in Supplementary Note 21 may also be dependent on the management method described in Supplementary Note 29, the recording medium described in Process 30, and the like in a similar dependent relationship. Similarly, not limited to Supplementary Notes 29 and 30, but within the scope of each of the above-mentioned embodiments, some or all of the configurations described as the Supplements may also be dependent on various hardware, software, various recording means for recording software, or systems.

[0113] The programs described in the above embodiments and appendices may be stored in a storage device or a computer-readable recording medium, such as a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0114] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0115] 100 Processing system 200 Processing device 210 Operation input unit 220 Screen display unit 230 Communication I / F unit 240 Memory unit 241 Product and packaging information 242 Image data information 243 Program 250 Arithmetic processing unit 251 Product status information acquisition unit 252 Imaging position determination unit 253 Imaging device control unit 254 Image data acquisition unit 255 Recognition target area determination unit 256 Recognition unit 257 Execution control unit 300 Management device 310 Operation input unit 320 Screen display unit 330 Communication I / F unit 340 Memory unit 341 Product and packaging information 342 Program 350 Arithmetic processing unit 351 Input reception unit 352 Estimation unit 353 Information acquisition unit 354 Transmission unit 400 Arm robot 410 Imaging device 500 Processing device 501 CPU 502 ROM 503 RAM 504 Program group 505 Storage device 506 Drive device 507 Communication interface 508 Input / output interface 509 Bus 510 Recording medium 511 Communication network 521 Acquisition unit 522 Determination unit 523 Recognition unit 600 Management device 621 Acquisition unit 622 Storage device

Claims

1. An acquisition unit that acquires information indicating the packaging status of items stored inside the packaging, A determination unit determines a recognition target area, which is the area of ​​the image data that is to be processed for recognition, as a parameter when performing recognition processing using image data of the item, in accordance with the information indicating the packaging status of the item acquired by the acquisition unit. A recognition unit performs recognition processing on the recognition target region determined by the determination unit, has Processing device.

2. The apparatus according to claim 1, The acquisition unit acquires information indicating whether or not multiple articles are bundled together within the packaging, as information indicating the packaging status of the articles. The determination unit determines the recognition target area, which is the area of ​​the image data to be processed for recognition, depending on whether or not the multiple articles are packaged together in a bundle. Processing device.

3. The apparatus according to claim 2, If the multiple articles are bundled together and packaged, the determination unit checks how the bundled articles are stored within the packaging and determines the recognition target area according to the result of the check. Processing device.

4. The apparatus according to claim 3, The determination unit determines the area to be recognized as the area corresponding to bundled articles selected from among multiple bundled articles present in the packaging according to predetermined criteria. Processing device.

5. The apparatus according to claim 2, When multiple items are packaged together, the determination unit determines a new recognition target area each time a process is performed according to the result of the recognition process by the recognition unit. Processing device.

6. The apparatus according to claim 2, If the items are not bundled, the determination unit determines the entire image data as the recognition target area. Processing device.

7. The apparatus according to claim 1, The acquisition unit acquires information indicating the size of the article, The imaging position determination unit determines the height of the imaging device when acquiring image data to be used for recognition processing, according to the size of the aforementioned article. Processing device.

8. The apparatus according to claim 7, The imaging position determination unit determines the height of the imaging device such that the distance between the imaging device and the article becomes shorter as the size of the article decreases. Processing device.

9. Information processing device, We obtain information indicating the packaging status of the items stored inside the package. In accordance with the information indicating the packaging status of the acquired item, the recognition target area, which is the area of ​​the image data to be processed for recognition, is determined as a parameter when performing recognition processing using image data of the item as the subject. Perform recognition processing on the determined recognition target area. Processing method.

10. In an information processing device, We obtain information indicating the packaging status of the items stored inside the package. In accordance with the information indicating the packaging status of the acquired item, the recognition target area, which is the area of ​​the image data to be processed for recognition, is determined as a parameter when performing recognition processing using image data of the item as the subject. Perform recognition processing on the determined recognition target area. A program to perform a process.