Work support device and non-temporary storage medium

The work support device employs a three-dimensional sensor to enhance detection accuracy and reduce processing load, addressing inefficiencies in article retrieval and storage tasks.

JP7583956B2Active Publication Date: 2024-11-14AIOI SYSTKK
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
JP2023554261
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-10-20
Filing Date
2022-07-06
Publication Date
2024-11-14
Estimated Expiration
2042-07-06

AI Technical Summary

Technical Problem

Existing work support devices face issues with inaccurate detection of worker's hands due to environmental illuminance variations and increased processing load, leading to inefficiencies in identifying the correct articles to be picked or stored.

Method used

A work support device utilizing a three-dimensional sensor to detect the position of a worker's hand and generate a distance image, determining article retrieval or storage based on pixel count within a detection area, with adjustable detection areas and thresholds to enhance accuracy and reduce processing load.

Benefits of technology

Improves work efficiency by accurately detecting article retrieval or storage actions with reduced processing load, minimizing errors and enhancing operational precision.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007583956000001
    Figure 0007583956000001
  • Figure 0007583956000002
    Figure 0007583956000002
  • Figure 0007583956000003
    Figure 0007583956000003
Patent Text Reader

Abstract

A work support device according to the present invention includes a plurality of storage boxes, a three-dimensional sensor, and a computational device. The plurality of storage boxes store articles. The three-dimensional sensor detects, in a first surface of a first object present in a plurality of first detection regions provided corresponding to the plurality of storage boxes, the position of at least a part of the first surface and generates a distance image representing the distance to at least the part of the first surface that is detected. The computational device determines whether or not the first object takes out the articles stored in the plurality of storage boxes or whether or not the first object stores the articles in the plurality of storage boxes on the basis of the position of at least the part of the first surface. Furthermore, the arithmetic device determines that the first object takes out the articles or stores the articles in the plurality of storage boxes when the number of pixels representing the at least the part of the first surface is larger than a first threshold value in the distance image.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a work support device and a non-transitory storage medium. [Background technology]

[0002] The work support device is a device that instructs a worker which of various items (e.g., parts or products) lined up on a shelf to pick up, and is widely used on production lines and logistics bases. As one example, the work support device is used on a production line to instruct a worker which parts and the number of parts should be assembled into a target product from among various parts stored in a parts warehouse (typically, a parts shelf). As another example, the work support device is used at a logistics base to instruct a worker which products and the number of products should be stored in a target shipper (container) from among various products stocked in a product warehouse (typically, a product shelf).

[0003] For example, Patent Document 1 discloses a technology that uses detection light to determine whether or not a worker's hand has been inserted into an opening in which an item is stored. The detection area in which the detection light detects the hand is a planar area. When an item is stored in a storage box arranged on a shelf with an opening, the detection area is set at a position away from the storage box. For this reason, when a worker takes out an item from a storage box adjacent to the target storage box, it may not be detected that the item being taken out is the wrong one. Also, if a part of the item enters the detection area, the worker's hand may not be detected.

[0004] Patent Documents 2 to 4 disclose techniques that use a camera to determine whether or not a worker has put his or her hand into an opening where an item is stored. The image captured by the camera changes depending on the surrounding illuminance environment. For this reason, erroneous detection may occur depending on the surrounding illuminance environment.

[0005] Patent Document 5 discloses a technique for determining whether a user has placed an item in an inventory location or taken an item out of an inventory location based on an image of the user's hand before entering the inventory location and an image of the user's hand when the user leaves the inventory location. In this case, the processing load increases because it is necessary to track the hand.

[0006] Patent Document 6 discloses a technology that uses a distance image sensor provided in a head-mounted display to determine whether or not a worker has put his or her hand into an opening where items are stored. In this case, each worker needs to wear a head-mounted display. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent No. 5441393 [Patent Document 2] Patent No. 6496082 [Patent Document 3] Patent No. 6656328 [Patent Document 4] Patent No. 6678790 [Patent Document 5] Special Publication No. 2016-532932 [Patent Document 6] JP 2019-156647 A Summary of the Invention

[0008] In view of the above, an object of the present disclosure is to provide a task support device that has a small processing load and allows tasks to be performed efficiently. Other objects can be understood from the following description and the explanation of the embodiments.

[0009] A task support device according to an embodiment for achieving the above object includes a plurality of storage boxes, a three-dimensional sensor, and a computing device. The plurality of storage boxes store items. The three-dimensional sensor detects the position of at least a portion of the first surfaces of a first object present in a plurality of first detection areas provided corresponding to the plurality of storage boxes, and generates a distance image representing the distance to the detected at least a portion of the first surfaces. The computing device determines whether the first object will take out the items stored in the plurality of storage boxes or whether the first object will store the items in the plurality of storage boxes, based on the position of the at least a portion of the first surfaces. Furthermore, the computing device determines that the first object will take out the items or store the items in the plurality of storage boxes when the number of pixels representing at least a portion of the first surfaces in the distance image is greater than a first threshold value.

[0010] A control program stored in a non-transitory storage medium according to one embodiment for achieving the above object causes a calculation device to calculate, from a three-dimensional sensor, a number of pixels representing at least a part of a first surface of a first object present in a plurality of first detection areas provided corresponding to a plurality of storage boxes for storing items. The control program also causes the calculation device to determine that the first object will take out an item or store an item in a plurality of storage boxes when the calculated number of pixels is greater than a first threshold value.

[0011] According to the above embodiment, the efficiency of the work can be improved. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a front view of a work support device according to an embodiment. [Diagram 2] FIG. 2 is a side view of the work support device according to the embodiment. [Diagram 3] FIG. 3 is a configuration diagram of a work support device according to an embodiment. [Figure 4] FIG. 4 is a flowchart showing a process performed by the work support device according to an embodiment. [Diagram 5]FIG. 5 is a diagram for explaining a method in which a three-dimensional sensor detects an object in one embodiment. [Figure 6A] FIG. 6A illustrates a distance display image that is displayed when setting a detection region in one embodiment. [Figure 6B] FIG. 6B illustrates a distance display image that is displayed when setting a detection region in one embodiment. [Figure 7] FIG. 7 is a diagram for explaining a full detection region and an empty detection region in one embodiment. [Figure 8] FIG. 8 is a front view of the work support device according to one embodiment. [Figure 9] FIG. 9 is a side view of a work support device according to an embodiment. [Figure 10] FIG. 10 is a diagram showing the detection area when the storage box is pulled out in one embodiment. [Figure 11A] FIG. 11A is a flowchart showing a process performed by a work support device according to an embodiment. [Figure 11B] FIG. 11B is a flowchart showing a process performed by the work support device according to an embodiment. [Figure 12] FIG. 12 is a diagram illustrating a cross-sectional distance image displayed when a detection region is set in one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (Embodiment 1) As shown in Fig. 1, a task support device 1000 according to this embodiment includes an item shelf 10 installed on a floor F, a projector 20, a three-dimensional sensor 30, and a storage box 50. For ease of understanding, a description will be given using Cartesian coordinates. The X1 direction is parallel to the floor F and is the right direction when the item shelf 10 is viewed from the front. The Y1 direction is parallel to the floor F and is the rear direction of the item shelf 10. The Z1 direction is perpendicular to the floor F and is the upward direction.

[0014] The article shelf 10 includes a plurality of base plates 11 arranged in the Z1 direction (vertical direction). Storage boxes 50 are arranged on the plurality of base plates 11 (e.g., first base plate 11-1, second base plate 11-2, and third base plate 11-3) that are provided in the -Z1 direction (downward direction) relative to one another. The angles of the base plates 11 with respect to the floor F may be different from one another, for example, as shown in FIG. 2. For example, the angle between the floor F and the base plates 11 may be smaller for base plates 11 in the -Z1 direction.

[0015] Further, a screen 13 is provided on each of the base plates 11 on which the storage boxes 50 are arranged. The screen 13 is provided, for example, on an end of the base plate 11 in the -Y1 direction so as to protrude from the base plate 11 in the +Z1 direction. The screen 13 protrudes in the +Z1 direction to prevent the storage boxes 50 from falling. An image is projected from the projector 20 onto the surface of the screen 13 in the -Y1 direction, and the projected image is configured to be visible to the worker W.

[0016] Projector 20 is provided in the +Z1 direction (upward) of item shelf 10, and displays instructions to worker W (e.g., designation of items to be picked) on screen 13 when taking out items (e.g., parts or products) from storage box 50. For example, projector 20 displays the quantity of items to be taken out in display area 100 corresponding to storage box 50 as shown in FIG. 1. For example, when taking out items stored in first storage box 50-1, projector 20 displays the quantity of items to be taken out in first display area 100-1 of screen 13, which is in the -Z1 direction of first storage box 50-1 when viewed from the -Y1 direction. Projector 20 is attached to an arm provided on, for example, fourth base plate 11-4 on the top shelf.

[0017] The three-dimensional sensor 30 is provided in the +Z1 direction (upward) of the item shelf 10, and detects the hands of the worker W when removing an item from a storage box 50. For example, as shown in FIG. 2, the three-dimensional sensor 30 detects an object, such as the hand of the worker W, present in a detection area 110 corresponding to each of the multiple storage boxes 50. For example, the detection area 110 is provided at the front of the storage box 50. Therefore, the three-dimensional sensor 30 can detect the storage box 50 from which the worker W has removed an item, even if the storage boxes 50 are adjacent to each other. The three-dimensional sensor 30 is attached to an arm provided on the fourth base plate 11-4 on the top shelf, for example.

[0018] Furthermore, the three-dimensional sensor 30 detects an object present in the detection area 110, such as the hand of the worker W, in a manner that allows it to be distinguished from the item 1 stored in the storage box 50. Therefore, even if the item 1 is within the detection area 110, the three-dimensional sensor 30 can detect the hand of the worker W.

[0019] For example, the three-dimensional sensor 30 is a distance image sensor that measures the distance to an object in each direction using light waves and generates an image representing the measured distance. The three-dimensional sensor 30 may generate a distance image using, for example, a Time of Flight (TOF) method. The distance image may represent the distance from the three-dimensional sensor 30 to the object in the optical axis direction of the light waves output by the three-dimensional sensor 30. For example, when the optical axis direction of the light waves output by the three-dimensional sensor 30 is vertical, the distance represented by the distance image represents the difference in altitude between the three-dimensional sensor 30 and the object.

[0020] The shape of the detection area 110 may be determined according to the direction in which an object, for example, the hand of the worker W, enters. For example, the detection area 110 may be formed such that at least a part of the detection area 110, the area of ​​a cross section perpendicular to the direction in which the object enters decreases as the object moves in the direction in which the object advances, for example, in the +Y1 direction. For example, the detection area 110 is formed such that the area of ​​a cross section perpendicular to the direction in which the object enters decreases as the object moves in the direction in which the object advances, for example, at an end of the detection area 110 in the direction in which the object advances, for example, in the +Y1 direction. The detection area 110 may also be formed such that the area of ​​a cross section perpendicular to the direction in which the object enters decreases as the object moves in the direction in which the object advances, for example, between the end of the +Y1 direction and the end of the opposite direction.

[0021] For example, the length of detection area 110 in the Z1 direction may monotonically increase, for example, monotonically increase in a broad sense or monotonically increase in a narrow sense, in the -Y1 direction as shown in second detection area 110-2 and third detection area 110-3. Because the hand of worker W enters detection area 110 from the -Y1 direction of storage box 50, three-dimensional sensor 30 can more reliably detect the hand of worker W and reduce detection area 110.

[0022] Furthermore, in a direction perpendicular to the surface, for example the top surface, where the article 1 is taken out of the storage box 50, the length of the detection area 110 may monotonically increase, for example, monotonically increase in a broad sense or monotonically increase in a narrow sense, in the -Y1 direction as shown in the second detection area 110-2 and the third detection area 110-3. The three-dimensional sensor 30 can detect the hand of the worker W more reliably.

[0023] Furthermore, the area of ​​a cross section of detection area 110 perpendicular to the direction in which an object enters may be constant regardless of the position in the direction in which the object enters. For example, the length of detection area 110 in the Z1 direction may be a fixed value regardless of the position in the Y1 direction, as shown in first detection area 110-1.

[0024] The end face in the +Z1 direction of the detection area 110 may be set so that the distance to each point on the end face represented in the distance image is equal. For example, when the optical axis direction of the light wave output by the three-dimensional sensor 30 is the vertical direction, the end face may represent a horizontal plane.

[0025] The detection area 110 may be configured to have different shapes depending on the base plate 11 on which the storage box 50 is placed. Since the direction in which the hand of the worker W enters the detection area 110 differs depending on the height of the base plate 11, the detection area 110 may be determined depending on the direction in which the hand of the worker W enters. Furthermore, the detection area 110 may be configured to have the same shape regardless of the base plate 11 on which the storage box 50 is placed.

[0026] 3, the work support device 1000 further includes an output device 60 and a control device 70. The output device 60 notifies the worker W of a work error, for example, that the item 1 picked up by the worker W is different from the item 1 that should be picked up. For example, the output device 60 includes a speaker and notifies the worker W of the work error by voice. The output device 60 may be incorporated in the projector 20 or may be omitted.

[0027] The control device 70 controls the projector 20, the three-dimensional sensor 30, and the output device 60. The control device 70 includes, for example, a control terminal 200 and a management terminal 300. The management terminal 300 determines the item 1 that the worker W should take out, and outputs the determined item information to the control terminal 200. Based on the item information received from the management terminal 300, the control terminal 200 outputs instruction information to the projector 20 for instructing the worker W on the item 1 that he or she should take out. Furthermore, based on the detection information acquired from the three-dimensional sensor 30, the control terminal 200 determines whether the worker W has taken out the instructed item 1.

[0028] The control terminal 200 includes a communication device 220, a computing device 230, and a storage device 240. The control terminal 200 is, for example, a computer. The control terminal 200 may further include, for example, input / output devices such as a keyboard, a mouse, a microphone, a display, a speaker, and a touch panel.

[0029] The communication device 220 is electrically connected to the projector 20, the three-dimensional sensor 30, the output device 60, and the management terminal 300, and communicates with each of the devices. For example, the communication device 220 transfers instruction information acquired from the management terminal 300 to the arithmetic device 230. Also, the communication device 220 transfers signals generated by the arithmetic device 230 to each of the devices. The communication device 220 includes various interfaces, for example, a network interface card (NIC) and a universal serial bus (USB).

[0030] The storage device 240 stores various data for determining whether the worker W has picked up the instructed item 1, for example, a control program 250. The storage device 240 is used as a non-transitory tangible storage medium for storing the control program 250. The control program 250 may be provided as a computer program product recorded in a computer-readable storage medium 5, or may be provided as a computer program product downloadable from a server.

[0031] The arithmetic device 230 executes the control program 250 to perform various data processing for determining whether the worker W has picked up the instructed item 1. For example, the arithmetic device 230 includes a central processing unit (CPU) and the like. The arithmetic device 230 also executes the control program 250 to generate image information representing an image to be projected on the screen 13 by the projector 20.

[0032] The management terminal 300 includes an input / output device 310, a communication device 320, a calculation device 330, and a storage device 340. The management terminal 300 is, for example, a computer. Information for the calculation device 330 to execute processing is input to the input / output device 310. In addition, the input / output device 310 outputs the results of the processing executed by the calculation device 330. The input / output device 310 includes various input devices and output devices, for example, a keyboard, a mouse, a microphone, a display, a speaker, a touch panel, etc.

[0033] The communication device 320 is electrically connected to the control terminal 200 and communicates with the control terminal 200. For example, the communication device 320 transfers a signal generated by the arithmetic device 330 to the communication device 220 of the control terminal 200. In addition, the communication device 320 transfers a signal acquired from the communication device 220 of the control terminal 200 to the arithmetic device 330. The communication device 320 includes various interfaces, such as a network interface card (NIC) and a universal serial bus (USB).

[0034] The storage device 340 stores various data, such as a management program 350, for determining the article 1 that the worker W should pick up. The storage device 340 is used as a non-transitory tangible storage medium for storing the management program 350. The management program 350 may be provided as a computer program product recorded in a computer-readable storage medium 6, or may be provided as a computer program product downloadable from a server. The storage medium 6 may be the same as the storage medium 5.

[0035] The arithmetic device 330 executes the management program 350 and performs various data processing to determine the item 1 to be picked up by the worker W. For example, the arithmetic device 330 includes a central processing unit (CPU) and the like.

[0036] (Operation of work support device) When the work support device 1000, for example the control terminal 200 and the management terminal 300, are started, a process of supporting a picking work is started. When the arithmetic device 330 of the management terminal 300 shown in FIG. 3 is started, it executes the management program 350 and starts the process shown in FIG. 4. In step S110, the arithmetic device 330 determines the items 1 and the quantity to be taken out by the worker W, and outputs instruction information representing the determined items 1 and the quantity to the communication device 220 of the control terminal 200. The information representing the items 1 and the quantity to be taken out by the worker W is stored in the storage device 340. The arithmetic device 330 reads out the information representing the items 1 and the quantity stored in the storage device 340, and outputs the read out information to the communication device 220 of the control terminal 200 as instruction information.

[0037] In step S120, the arithmetic device 230 of the control terminal 200 instructs the worker W on the item 1 to be removed based on the instruction information acquired from the management terminal 300. For example, the arithmetic device 230 determines the storage box 50 corresponding to the item 1 represented in the instruction information. As shown in FIG. 1, the arithmetic device 230 generates an image that displays the quantity represented in the instruction information in the display area 100 corresponding to the determined storage box 50. The instruction information representing the generated image is output to the projector 20. Note that the information representing the item 1, the information representing the storage box 50, and the information representing the display area 100 are stored in the storage device 240 in association with each other.

[0038] Based on the instruction information, the projector 20 projects an image representing the quantity of the articles 1 to be taken out onto a display area 100 of the screen 13 corresponding to the storage box 50 in which the articles 1 to be taken out are stored. An image representing the articles 1 to be taken out may also be projected onto the display area 100.

[0039] In step S130 shown in FIG. 4, the arithmetic device 230 detects an object to take out the article 1. The worker W takes out the article 1 from the storage box 50 corresponding to the display area 100 on which the image is projected. When the hand of the worker W enters the detection area 110 shown in FIG. 2, the three-dimensional sensor 30 detects the hand of the worker W. For example, the three-dimensional sensor 30 irradiates light waves, for example, infrared rays, to the multiple detection areas 110 as shown in FIG. 5. For example, when the control terminal 200 is started, the three-dimensional sensor 30 irradiates light waves to the multiple detection areas 110. The light waves irradiated by the three-dimensional sensor 30 are reflected by an object 500 in the detection area 110, for example, the hand of the worker W. The three-dimensional sensor 30 receives the reflected light waves as reflected waves and detects the object 500 present in the detection area 110.

[0040] Specifically, the three-dimensional sensor 30 receives a reflected wave reflected by at least a part of the surface 501 of the object 500. The three-dimensional sensor 30 measures the distance to each position on the surface 501 where the light wave is irradiated, based on the time from irradiating the light wave to receiving the reflected wave. The three-dimensional sensor 30 also measures the relative direction at each position on the surface 501, based on the angle with the direction in which the reflected wave is received, for example, the Z1 direction. Each position on the surface 501 of the object 500 is measured based on the relative direction and distance at each position on the surface 501. Detection information representing the position of the surface 501 of the object 500 is output to the arithmetic device 230 of the control terminal 200. The detection information represents a distance image in which the distance from the three-dimensional sensor 30 to the detected object 500 is mapped according to the relative direction. The position of a pixel represented in the distance image represents the relative direction from the three-dimensional sensor 30, and the value added to the pixel represents the distance to the object 500 that exists in the corresponding relative direction. For example, when an image is captured by a camera from the three-dimensional sensor 30 in the optical axis direction of the irradiated light wave, the value represented by each pixel contained in the captured image represents the distance from the three-dimensional sensor 30 to the object represented by that pixel.

[0041] The arithmetic device 230 of the control terminal 200 determines whether the object 500 that takes out the article 1 is present in the detection area 110 based on the detection information. The arithmetic device 230 determines whether the object 500 takes out the article 1 based on the number of pixels of the area representing the detected object 500 in the distance image. For example, the arithmetic device 230 calculates the number of pixels of at least a part of the surface 501 of the detected object 500. The arithmetic device 230 compares the calculated number of pixels of the surface 501 with a first threshold value to determine whether the object 500 that takes out the article 1 is present in the detection area 110. When the calculated number of pixels is greater than the first threshold value, the arithmetic device 230 determines that the object 500 that takes out the article 1 is present in the detection area 110. Here, the surface 501 represents a set of consecutive positions among the positions detected by the three-dimensional sensor 30. For example, the arithmetic device 230 determines that the adjacent pixels form the same surface 501 when the difference in distance represented by adjacent pixels in the distance image is smaller than the first threshold value.

[0042] The first threshold is determined in advance so as to be able to distinguish between the article 1 stored in the storage box 50 shown in FIG. 2 and the object 500, for example, the hand of the worker W, which takes out the article 1. For example, when the arithmetic device 230 calculates the number of pixels of the surface 501 of the object 500, the first threshold is determined so as to be greater than the number of pixels of the part of the article 1 stored in the storage box 50 that is included in the detection area 110 and smaller than the number of pixels of the object 500. The first threshold may be different depending on the corresponding storage box 50. For example, the first threshold determined for one detection area 110 may be different from the first threshold determined for at least one other detection area 110. The first threshold may be determined depending on the article 1 stored in the storage box 50. In addition, the first threshold may be different depending on the distance from the three-dimensional sensor 30 to the detection area 110, for example, for each base plate 11. For example, the first threshold is smaller as the distance from the three-dimensional sensor 30 to the detection area 110 is larger.

[0043] The arithmetic device 230 may determine that the object 500 taking out the article 1 is present in the detection area 110 when the calculated number of pixels of the surface 501 is included between the first threshold value and the second threshold value. Here, the second threshold value is greater than the first threshold value and represents an upper limit value for determining that the object 500 is present. For example, the arithmetic device 230 determines that the object 500 taking out the article 1 is present in the detection area 110 when the calculated number of pixels of the surface 501 is greater than the first threshold value and smaller than the second threshold value.

[0044] The second threshold is determined in advance so as to be able to distinguish between the object 500 removing the article 1 into the detection area 110 and other objects, such as the worker's head. For example, the second threshold is determined to a value that maximizes the number of pixels of the object 500 when removing the article 1 in the detection area 110. Similarly to the first threshold, the second threshold may differ depending on the distance from the three-dimensional sensor 30 to the detection area 110. For example, the second threshold is smaller as the distance from the three-dimensional sensor 30 to the detection area 110 is larger.

[0045] When the object 500 removing the article 1 is not present in the detection area 110, the arithmetic device 230 waits until the object 500 removing the article 1 enters the detection area 110. When the object 500 removing the article 1 is present in the detection area 110, the arithmetic device 230 executes the process of step S140.

[0046] In step S140, the arithmetic device 230 judges whether the item 1 taken by the object 500 is correct. Specifically, the arithmetic device 230 judges whether the item 1 taken by the object 500 is the same as the item 1 represented in the instruction information acquired from the management terminal 300. For example, the arithmetic device 230 searches for the storage box 50 corresponding to the detection area 110 in which the object 500 exists. The arithmetic device 230 searches for the item 1 corresponding to the searched storage box 50, and judges whether the searched item 1 is the same as the item 1 represented in the instruction information. The information representing the detection area 110, the information representing the storage box 50, and the information representing the item 1 are stored in advance in the storage device 240 in association with each other.

[0047] The arithmetic device 230 executes step S150 when the item 1 taken out by the object 500 is the same as the item 1 represented in the instruction information acquired from the management terminal 300. When the item 1 taken out by the object 500 is different from the item 1 represented in the instruction information acquired from the management terminal 300, the arithmetic device 230 executes step S160.

[0048] When the object 500 is to take out the correct item 1, in step S150, the calculation device 230 determines that the worker W has taken out the correct number of items 1, and requests information on the next item 1 to be taken out from the calculation device 330 of the management terminal 300. The process returns to step S110 and is repeated.

[0049] When the object 500 takes out the wrong article 1, in step S160, the arithmetic device 230 notifies the worker W of an operational error indicating that the worker W has taken out the wrong article 1. For example, the arithmetic device 230 controls the projector 20 to display an image that flashes red. The arithmetic device 230 may also control the output device 60 to output an alarm sound indicating an operational error. When the arithmetic device 230 no longer detects the object 500 in the detection area 110, it executes the process of step S130 and repeats the process.

[0050] In this way, the work support device 1000 can determine with high accuracy the item 1 to be taken out by the worker W by setting the detection area 110 according to the openings of the multiple storage boxes 50. Even if the item 1 stored in the storage box 50 is present in the detection area 110, the work support device 1000 can detect the item 1 and the object 500 from which the item 1 is to be taken out, in a distinguished manner.

[0051] (Detection area setting) The detection area 110 may be set based on a distance image obtained by the three-dimensional sensor 30. For example, image information representing the distance image obtained by the three-dimensional sensor 30 is transferred from the arithmetic device 230 of the control terminal 200 shown in FIG. 3 to the arithmetic device 330 of the management terminal 300. The arithmetic device 330 of the management terminal 300 displays a distance display image representing the distance to the detected object 500 on the input / output device 310 based on the acquired image information. The distance display image represents an object included in a range specified by the user in the distance image. The user checks the displayed distance display image and inputs information representing the detection area 110 to the input / output device 310. The arithmetic device 330 of the management terminal 300 outputs the input information representing the detection area 110 to the arithmetic device 230 of the control terminal 200. The arithmetic device 230 sets the detection area 110 based on the input information representing the detection area 110.

[0052] Furthermore, when setting the detection area 110, the arithmetic device 330 displays a distance display image according to the distance range input by the user. For example, when the distance range represents the distance from the end face in the +Z1 direction (upward direction) of the storage box 50 placed on the first base plate 11-1 shown in Fig. 1 to the end face in the +Z1 direction of the first detection area 110-1, the arithmetic device 330 displays the first distance display image 600 shown in Fig. 6A. The first distance display image 600 represents the distance to an object included in the input distance range by the difference in pixel color, for example, hue, saturation, or brightness.

[0053] For example, when the first distance display image 600 uses four colors to indicate differences in distance, the position where an object is detected at a distance equal to or greater than the minimum value of the input distance range and less than a first threshold is displayed in a first color 601. The first distance display image 600 also displays the position where an object is detected at a distance equal to or greater than the first threshold and less than a second threshold in a second color 602. No.The first distance display image 600 shows the position where an object is detected at a distance equal to or greater than the second threshold and less than the third threshold in a third color 603. The first distance display image 600 shows the position where an object is detected at a distance equal to or greater than the third threshold and less than the maximum value of the input distance range in a fourth color 604. In this way, by the computing device 330 displaying the first distance display image 600 showing an object detected within the distance range input by the user, the user can easily grasp the detection area 110 corresponding to the input distance range.

[0054] For example, the user specifies a range in the X1 direction and a range in the Y1 direction of the detection area 110 shown in FIG. 2 according to the area of ​​the storage box 50 displayed in the first distance display image 600. The X1 direction shown in FIG. 2 represents the X2 direction in the first distance display image 600 shown in FIG. 6A. Similarly, the Y1 direction shown in FIG. 2 represents the Y2 direction in the first distance display image 600 shown in FIG. 6A. Therefore, the range in the X1 direction of the detection area 110 corresponds to the range in the X2 direction in which the storage box 50 is displayed in the first distance display image 600. Moreover, the range in the Y1 direction of the detection area 110 corresponds to the range in the Y2 direction in which the storage box 50 is displayed in the first distance display image 600.

[0055] 2 in accordance with the distance to the storage box 50 displayed in the first distance display image 600. The detection area 110 is set in the +Z1 direction from the end (upper end) of the storage box 50 in the +Z1 direction. Therefore, the end of the detection area 110 in the -Z1 direction is set at a position shorter than the distance to the storage box 50 displayed in the first distance display image 600. The detection area 110 may include the end (upper end) of the storage box 50 in the +Z1 direction. In this case, the threshold for detecting the object 500 is set to include the number of pixels of the detected storage box 50.

[0056] Furthermore, the user specifies the end (upper end) in the +Z1 direction of the detection area 110 shown in FIG. 2 according to the distance to the surface 501 of the object 500 displayed in the first distance display image 600. The detection area 110 is set so as to detect the object 500 from which the article 1 stored in the storage box 50 is to be removed. Therefore, when removing the article 1 stored in the storage box 50, the user specifies the end (upper end) according to the distance to the surface 501 displayed in the first distance display image 600. For example, when the user removes the article 1, the user may specify the end (upper end) in the +Z1 direction of the detection area 110 shown in FIG. Movie The position where the distance of the surface 501 to be detected is smallest is set to the end (upper end) of the detection area 110 in the +Z1 direction.

[0057] The first distance display image 600 may also display in real time the number of pixels representing the surface 501 of the object 500 placed in one of the multiple detection areas 110, for example, the first detection area 110-1. The user checks the displayed number of pixels and sets a threshold for determining the presence of the object 500 placed in the detection area 110 and the detection area 110. This allows the user to easily set the threshold and the detection area 110.

[0058] Next, when the user sets the detection area 110 corresponding to the storage box 50 arranged on the first base plate 11-1 shown in FIG. 2, the user changes the distance range of the distance display image displayed by the arithmetic device 330 in order to set the detection area 110 corresponding to the storage box 50 arranged on the second base plate 11-2. For example, the user inputs the distance range from the end face in the +Z1 direction (upward direction) of the storage box 50 arranged on the second base plate 11-2 shown in FIG. 1 to the end face in the +Z1 direction of the second detection area 110-2. The arithmetic device 330 displays the second distance display image 650 shown in FIG. 6B according to the input distance range. In the second distance display image 650, the storage box 50 is displayed in the +Y2 direction from the storage box 50 displayed in the first distance display image 600. The user specifies the second detection area 110-2 based on the second distance display image 650 in the same way as the first detection area 110-1. 2. Similarly, the user designates a third detection area 110-3 corresponding to the storage box 50 placed on the third base plate 11-3 shown in FIG.

[0059] In this way, the calculation device 330 displays the first distance display image 600 representing the range of distances input by the user based on the distance image, allowing the user to easily set the detection area 110 and a threshold value for detecting the object 500.

[0060] (Full detection) As shown in FIG. 7, the three-dimensional sensor 30 detects the item 1 present in the fullness detection area 120 provided at the front of the storage box 50. The fullness detection area 120 is provided at the end of the storage box 50 in the +Z1 direction. The fullness detection area 120 may extend in the +Z1 direction from the end of the storage box 50 in the +Z1 direction. For example, the three-dimensional sensor 30 detects the position of at least a part of the surface of the item 1 present in the fullness detection area 120 and generates a distance image representing the distance to the detected at least a part of the surface. When the number of pixels representing at least a part of the surface of the item 1 in the distance image is greater than the fullness threshold, the calculation device 230 of the control terminal 200 determines that the corresponding storage box 50 is filled with the item 1. When the calculation device 230 determines that the corresponding storage box 50 is filled with the item 1, it notifies the worker W that the storage box 50 is full. For example, the calculation device 230 projects an image representing that the storage box 50 is full onto the screen 13 of the corresponding storage box 50. The image indicating that the container is full may be expressed by text or color.

[0061] (Sky detection) The three-dimensional sensor 30 detects an article 1 present in an empty detection area 130 provided at the end of the storage box 50 in the -Z1 direction, for example, at the bottom. For example, the three-dimensional sensor 30 detects the position of at least a part of the surface of the article 1 present in the empty detection area 130, and generates a distance image representing the distance to the detected at least a part of the surface. The arithmetic device 230 of the control terminal 200 determines that the corresponding storage box 50 is empty when the number of pixels representing at least a part of the surface of the article in the distance image is smaller than the empty threshold. When the arithmetic device 230 determines that the corresponding storage box 50 is empty, it notifies the worker W that the storage box 50 is empty. For example, the arithmetic device 230 projects an image representing that the storage box 50 is empty on the screen 13 of the corresponding storage box 50. The image representing that the storage box is empty may be represented by characters or colors.

[0062] (Embodiment 2) 8, the item shelf 10 may store a plurality of storage boxes 50 so that the storage boxes 50 can be pulled out in the -Y1 direction (forward). The item shelf 10 may also store a plurality of types of storage boxes 50, 51 of different sizes, for example, a relatively small storage box 50 and a relatively large storage box 51. In this case, the work support device 1000 includes the item shelf 10, a projector 20, a three-dimensional sensor 30, and the storage boxes 50, 51.

[0063] The relatively small storage box 50 is configured to be stored side by side in a plane perpendicular to the Y1 direction, in the +Z1 direction of the relatively large storage box 51. For example, the storage boxes 50 are stored so as to be lined up along the X1 direction. Also, the storage boxes 50 are stored so as to be lined up along the Z1 direction. Relatively large storage box 51 is a relatively small storage box 50 and floor F, the storage boxes 51 are arranged in a plane perpendicular to the Y1 direction and stored in the storage rack 10. As shown in FIG. 9, the lower portion of the storage rack 10 that stores the storage boxes 51 protrudes in the -Y1 direction from the upper portion that stores the storage boxes 50.

[0064] 8 and 9, the storage boxes 50 and 51 have a rectangular parallelepiped shape with an open top, and are configured to store multiple items 1 therein to be taken out by a worker W. The storage box 50 may be provided with a partition therein so that multiple types of items 1 can be stored separately.

[0065] Further, the storage boxes 50, 51 are provided with grips on the surface (front surface) in the -Y1 direction. The grips are gripped by the worker W when the storage boxes 50, 51 are pulled out from the article shelf 10.

[0066] The projector 20 is provided in the +Z1 direction (upward) of the item shelf 10, and is configured to project an image onto the front surface of the item shelf 10. The projector 20 notifies the worker W of instructions by projecting an image onto the label of each storage box 50.

[0067] The three-dimensional sensor 30 is provided in the +Z1 direction (upward direction) of the item shelf 10, and detects the hands of the worker W when pulling out the storage box 50. For example, as shown in FIG. 9, the three-dimensional sensor 30 detects an object 500, such as the hand of the worker W, present in a fourth detection area 110-4 corresponding to each of the multiple storage boxes 50. The fourth detection area 110-4 is provided in the -Y1 direction of the storage box 50. The three-dimensional sensor 30 also detects an object 500 present in a fifth detection area 110-5 corresponding to each of the multiple storage boxes 51. The fifth detection area 110-5 is provided in the -Y1 direction of the storage box 51. For example, one three-dimensional sensor 30 detects the object 500 in the fourth detection area 110-4 and the object 500 in the fifth detection area 110-5 present in the -Y1 direction from the fourth detection area 110-4.

[0068] The shape of the detection area 110 may be determined according to the direction in which the object 500 enters. For example, the detection area 110 may be formed such that at least a part of the area of ​​a cross section perpendicular to the direction in which the object 500 advances in the detection area 110 monotonically decreases as the object 500 advances. For example, the detection area 110 corresponding to the storage box 51 located relatively in the -Z1 direction (lower) may be formed such that the cross-sectional area of ​​a plane perpendicular to the Z1 direction monotonically decreases as the object 500 advances. For example, the detection area 110 corresponding to the storage box 51 may be formed such that the length in the Y1 direction monotonically decreases as the object 500 advances. Furthermore, the detection area 110 corresponding to the storage box 50 located relatively in the +Z1 direction (upper) may be formed such that the cross-sectional area of ​​a plane perpendicular to the Z1 direction monotonically decreases as the object 500 advances. For example, the detection area 110 corresponding to the storage box 50 may be formed such that the length in the Y1 direction monotonically decreases as the object 500 advances in the +Z1 direction. Each detection region 110 may be formed in a different shape depending on its position in the Z1 direction.

[0069] When the storage box 50 is pulled out, the three-dimensional sensor 30 sets the detection area 110 to a sixth detection area 110-6 that exists in the +Z1 direction of the storage box 50, as shown in FIG. 10. The sixth detection area 110-6 may be divided, for example, according to a partition provided in the storage box 50. In this way, even if a plurality of items 1 are stored in the storage box 50 and separated by partitions, the three-dimensional sensor 30 can detect the item 1 taken out by the worker W. The shape of the set detection area 110 may be formed similarly to the detection area 110 of the first embodiment, for example, the first detection area 110-1, the second detection area 110-2, or the third detection area 110-3 shown in FIG. 2.

[0070] The work assistance device 1000 includes the control device 70 shown in Fig. 3, similarly to the first embodiment. The configuration of the control device 70 is similar to that of the first embodiment, and therefore a detailed description thereof will be omitted.

[0071] (Operation of work support device) When the work support device 1000, for example the control terminal 200 and the management terminal 300, are started, a process of supporting a picking work is started. When started, the arithmetic device 330 of the management terminal 300 executes the management program 350 and starts the process shown in Figures 11A and 11B. The process of step S110 and the process of step S120 are the same as those in the first embodiment, and therefore detailed description thereof will be omitted.

[0072] In step S121, the arithmetic device 230 of the control terminal 200 detects an object 500, such as the hand of the worker W, pulling out the storage box 50, 51. For example, the arithmetic device 230 determines whether the object 500 is present in the detection area 110, such as the fourth detection area 110-4, the fifth detection area 110-5, etc. The process of determining whether the object 500 is present in the detection area 110 is similar to that in the first embodiment, and therefore a detailed description thereof will be omitted.

[0073] When it is determined that the object 500 is present in the detection area 110, in step S122, the arithmetic device 230 determines whether the storage box 50, 51 to be pulled out by the detected object 500 is correct. For example, the arithmetic device 230 searches for the storage box 50, 51 corresponding to the detection area 110 in which the object 500 is present. The arithmetic device 230 determines whether the item 1 represented by the instruction information acquired from the management terminal 300 is included in the item 1 stored in the searched storage box 50, 51. When the item 1 represented by the instruction information is not included in the searched storage box 50, 51, the arithmetic device 230 executes the process of step S123. When the item 1 represented by the instruction information is included in the searched storage box 50, 51, the arithmetic device 230 executes the process of step S124.

[0074] In step S123, the arithmetic device 230 notifies the worker W of the work error. The process by which the arithmetic device 230 notifies the worker W of the work error is similar to the process of step S160 shown in Fig. 4, and therefore detailed description will be omitted. When the arithmetic device 230 no longer detects the object 500 in the detection area 110, it executes the process of step S121 and repeats the process.

[0075] When the storage box 50, 51 pulled out by the object 500 is correct, in step S124, the arithmetic device 230 sets the detection area 110 in the +Z1 direction of the pulled-out storage box 50, 51. For example, as shown in FIG. 10, the arithmetic device 230 sets the detection area 110 in the +Z1 direction of the pulled-out storage box 50.

[0076] In step S130 shown in FIG. 11B, the arithmetic device 230 detects an object 500 that takes out an article 1 in a set detection area 110, for example, the sixth detection area 110-6 shown in FIG. 10. For example, the arithmetic device 230 determines whether the object 500 is present in the detection area 110, for example, the sixth detection area 110-6. The process of determining whether the object 500 is present in the detection area 110 is similar to that in the first embodiment, and therefore a detailed description thereof will be omitted. When the sixth detection area 110-6 is divided according to partitions provided on the storage boxes 50 and 51, it is determined in which of the divided detection areas that divide the sixth detection area 110-6 the object 500 is present.

[0077] When the object 500 is present in the sixth detection area 110-6, the arithmetic device 230 executes the process of step S140. The process of step S140 is similar to that of the first embodiment, and therefore a detailed description thereof will be omitted. In addition, when the sixth detection area 110-6 is divided according to the partitions provided in the storage boxes 50 and 51, it is determined that the article 1 corresponding to the divided detection area in which the object 500 is detected is to be removed.

[0078] The process of steps S150 and S160 is similar to that of the first embodiment, and therefore a detailed description thereof will be omitted.

[0079] In this way, the work support device 1000 can detect the item 1 that the worker W takes out from the storage boxes 50, 51 that are stored so as to be able to be pulled out in the -Y1 direction. Even if the storage boxes 50, 51 are separated by partitions and store different types of items 1, the work support device 1000 can detect the item 1 that the worker W takes out.

[0080] It should be noted that the detection regions 110, for example the fourth detection region 110-4, the fifth detection region 110-5, and the sixth detection region 110-6, are set using a distance display image, similar to the detection region 110 in the first embodiment.

[0081] (Modification) The above-described process is an example, and the order of each step and the process contents may be changed as long as the function is not impaired. Furthermore, the described configuration may be changed as desired as long as the function is not impaired. For example, the screen 13 may be provided at any position as long as the display area 100 is configured to correspond to the storage box 50. For example, the screen 13 may be provided above the storage box 50, or may be provided in front of the storage box 50.

[0082] The three-dimensional sensor 30 may be provided at any position as long as it can detect the object 500 in the detection area 110 corresponding to the storage box 50. For example, the three-dimensional sensor 30 may be provided in the Y1 direction of the item shelf 10.

[0083] Furthermore, the detection areas 110 may be formed in any shape as long as they can detect the object 500 removing the article 1 from the storage box 50. For example, the detection areas 110 may all be formed in the same shape.

[0084] The three-dimensional sensor 30 may further detect a worker W who takes out an item 1. For example, the three-dimensional sensor 30 searches for an area where the worker W is present when taking out the item 1. The arithmetic device 230 of the control terminal 200 identifies the worker W based on information about the worker W detected by the three-dimensional sensor 30. The arithmetic device 230 specifies the worker W corresponding to the object 500 detected in the detection area 110. In this way, the arithmetic device 230 can detect the item 1 taken out by each worker W even if multiple workers W take out an item 1 stored in one item shelf 10.

[0085] The above-described embodiments and modifications are merely examples, and the configurations described in each embodiment and modification may be arbitrarily modified and / or combined as long as the functions are not impaired. Furthermore, some of the functions described in the embodiments and modifications may be omitted as long as the necessary functions can be realized.

[0086] For example, as shown in Fig. 8, when the item shelf 10 stores a storage box 50 so that the storage box 50 can be pulled out in the -Y1 direction, the arithmetic unit 230 of the control terminal 200 may determine that the item 1 stored in the storage box 50 is removed by pulling out the storage box 50. In this case, as shown in Fig. 9, when an object 500 is detected in a detection area 110 provided in front of the storage boxes 50 and 51, the arithmetic unit 230 may determine that the storage box 50 is pulled out in the -Y1 direction. 、51 It is determined that item 1 stored in is to be taken out.

[0087] The arithmetic unit 230 of the control terminal 200 may execute all or part of the processing of the arithmetic unit 330 of the management terminal 300. Furthermore, the arithmetic unit 330 of the management terminal 300 may execute all or part of the processing of the arithmetic unit 230 of the control terminal 200. Furthermore, the control program 250 may include a management program 350.

[0088] In addition, the work support device 1000 has been described as an example of notifying the worker W of a work error when removing the item 1 from the storage box 50, but it may also notify the worker W of a work error when storing the item 1 in the storage box 50.

[0089] The computing device 330 of the management terminal 300 may display a cross-sectional distance image 700 representing a cross-section of the first distance display image 600 or the second distance display image 650 detected by the three-dimensional sensor 30 together with the first distance display image 600 shown in FIG. 6A or the second distance display image 650 shown in FIG. 6B. For example, the management terminal 300 displays the second distance display image 650 and the cross-sectional distance image 700 side by side as shown in FIG. 12. The cross-sectional distance image 700 represents a position detected by the three-dimensional sensor 30 on a plane represented in the second distance display image 650, for example, a cross-section perpendicular to the X1-Y1 plane, for example, the Y1-Z1 plane shown in FIG. 2. The position of the cross-section of the displayed cross-sectional distance image 700 is selected by, for example, a cross-sectional position selection unit 660. The cross-sectional position selection unit 660 is composed of, for example, a slider. The position of the cross-section displayed in the cross-sectional distance image 700 is determined by moving a knob 665 on the slider in the X2 direction. For example, cross-sectional distance image 700 represents a position detected by three-dimensional sensor 30 on cross-section 667 corresponding to the position where knob 665 is displayed. Here, the Y1 direction shown in Fig. 2 represents the Y3 direction in cross-sectional distance image 700 shown in Fig. 12, and the Z1 direction shown in Fig. 2 represents the Z3 direction in cross-sectional distance image 700 shown in Fig. 12.

[0090] In the cross-sectional distance image 700 shown in Fig. 12, the inner surface of the storage box 50 placed furthest in the +Z1 direction in Fig. 2 represents the recess 710 furthest in the +Z3 direction. In Fig. 2, the inner surface of the storage box 50 placed second from the +Z1 direction represents the recess 710 furthest in the +Z3 direction. In Fig. 2, the inner surface of the storage box 50 placed furthest in the -Z1 direction represents the recess 710 furthest in the -Z3 direction.

[0091] A detection range 720 representing the detection area 110 may be displayed in the second distance display image 650 and the cross-sectional distance image 700. The user checks the detection area 110 based on the detection range 720 in the second distance display image 650 and the detection range 720 in the cross-sectional distance image 700. The detection area 110 may be changed by changing the detection range 720 displayed in the second distance display image 650 or the cross-sectional distance image 700. For example, the user changes the detection area 110 by dragging a vertex of the detection range 720 displayed in the second distance display image 650 or the cross-sectional distance image 700 to change the detection range 720. This allows the user to easily check the detection area 110 and also to easily change the detection area 110.

[0092] This application claims priority based on Japanese Patent Application No. 2021-171499, filed on October 20, 2021, the disclosure of which is incorporated herein in its entirety.

Claims

1. A plurality of storage boxes for storing items; a three-dimensional sensor that detects a position of at least a portion of a first surface of a first object present in a plurality of first detection areas provided corresponding to the plurality of storage boxes, and generates a distance image representing a distance to the detected at least a portion of the first surface; a computing device that determines whether the first object will take out the item stored in the plurality of storage boxes or whether the first object will store the item in the plurality of storage boxes based on a position of the at least a portion of the first surface; Equipped with The computing device includes: When the number of pixels representing the at least a part of the first surface in the distance image is greater than a first threshold, it is determined that the first object is taking out the item or storing the item in the plurality of storage boxes. Work support equipment.

2. The computing device outputs a distance display image that represents an object included in the input distance range from among the distance images. The work support device according to claim 1 .

3. The computing device outputs the number of pixels in real time. The work support device according to claim 1 or 2.

4. The computing device outputs a distance cross-sectional image representing a position detected by the three-dimensional sensor in a cross section perpendicular to the plane represented in the distance image. The work support device according to any one of claims 1 to 3.

5. The computing device outputs an image representing a cross-sectional position selection unit that sets a position of the cross-section represented in the distance cross-sectional image. The work support device according to claim 4.

6. The first threshold value is smaller as the distance from the three-dimensional sensor to the corresponding first detection area among the plurality of first detection areas is larger. The work support device according to any one of claims 1 to 5.

7. The computing device determines that the first object takes out the item or stores the item in the plurality of storage boxes when a number of pixels representing the at least a portion of the first surface in the distance image is smaller than a second threshold value. The work support device according to any one of claims 1 to 6.

8. The shape of one of the plurality of first detection areas is such that, in at least a part of the shape, an area of ​​a first cross section of the one detection area perpendicular to a first direction monotonically decreases in accordance with a first direction in which the first object moves when the first object takes out the article or when the first object puts the article back in. The work support device according to any one of claims 1 to 7.

9. a storage shelf that stores the plurality of storage boxes so that the storage boxes can be pulled out forward, the three-dimensional sensor is provided in front of the plurality of storage boxes and detects a position of at least a part of a second surface of a second object present within a plurality of second detection areas; The computing device includes: When a first storage box of the plurality of storage boxes is pulled out forward, one of the plurality of first detection areas is set above the first storage box. The work support device according to any one of claims 1 to 8.

10. The three-dimensional sensor detects a worker who takes out the item or puts away the item, The computing device includes: Identifying the worker based on the detected information of the worker; Identifying the worker corresponding to the first object detected in the plurality of first detection areas. The work support device according to any one of claims 1 to 9.

11. the three-dimensional sensor detects a position of at least a part of a first object surface of an object present in a fullness detection area provided at an entrance of the plurality of storage boxes, and generates the distance image representing a distance to the detected at least a part of the first object surface; The computing device determines that a storage box corresponding to the full detection region among the plurality of storage boxes is full when a number of pixels representing the at least a portion of the first article surface in the distance image is greater than a full threshold value. The work support device according to any one of claims 1 to 10.

12. The three-dimensional sensor detects a position of at least a part of a second article surface of an article present in an empty detection area provided at the bottom of the plurality of storage boxes, and generates the distance image representing a distance to the detected at least a part of the second article surface; The computing device determines that a storage box corresponding to the empty detection area among the plurality of storage boxes is empty when a number of pixels representing the at least a portion of the second article surface in the distance image is smaller than an empty threshold value. The work support device according to any one of claims 1 to 11.

13. calculating a number of pixels representing at least a portion of a first surface in a distance image representing a distance from the three-dimensional sensor to at least a portion of a first surface of a first object present in a plurality of first detection areas provided corresponding to a plurality of storage boxes for storing items; When the calculated number of pixels is greater than a first threshold value, it is determined that the first object takes out the item or stores the item in the plurality of storage boxes; A non-transitory storage medium that stores a control program that causes a computing device to execute the above.

Citation Information

Patent Citations

  • Novel antibiotic substance aa5945 and preparation

    JP1979041393A

  • Oxide superconductor

    JP1989096082A

  • Article interaction and movement detection method

    JP2016532932A

  • Support system

    JP2017048052A

  • Work determination system and work determination method

    JP2019156647A