Article search system and calibration method for article search system

Abstract

To accurately notify a position of an article.SOLUTION: An identification unit configured to identify the article from the feature of the article acquired by the identification information acquisition unit, a position recognition unit configured to allocate plane coordinates to the information of the space of the placement area acquired by the identification information acquisition unit and acquire a position of the article by associating the article repeatedly identified by the identification unit with the plane coordinates at an arbitrary timing, a notification unit configured to control a notification direction so as to notify the position of the article acquired by the position recognition unit, and a calibration unit configured to associate the plane coordinates allocated by the position recognition unit with a notification direction of the position of the article by the notification unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an item search system and a calibration method for an item search system, and is suitable for application to, for example, an item search system and a calibration method for an item search system that detects the location of an item using images from multiple cameras and notifies the location of the item. [Background technology]

[0002] An example of a calibration method is described in Patent Document 1. Patent Document 1 discloses a technique for improving the accuracy of position detection by determining distortion parameters of a camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 5-277973 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in Patent Document 1 cannot ensure sufficient position detection accuracy when an obstacle is present.

[0005] The present invention has been made in consideration of the above points, and aims to propose an article search system and a calibration method for an article search system that can notify the position of an article with high accuracy. [Means for solving the problem]

[0006] In order to solve this problem, the present invention comprises an identification information acquisition unit that acquires characteristics of items that can be randomly placed within the space of a placement area by acquiring spatial information of the placement area where items can be placed; an identification unit that identifies the items from the characteristics of the items acquired by the identification information acquisition unit; a position recognition unit that assigns planar coordinates to the spatial information of the placement area acquired by the identification information acquisition unit and acquires the position of the items by repeatedly associating the items identified by the identification unit with the planar coordinates at any timing; a notification unit whose notification direction is controlled to notify the position of the item acquired by the position recognition unit; and a calibration unit that associates the planar coordinates assigned by the position recognition unit with the notification direction of the item's position by the notification unit.

[0007] In addition, the present invention includes an identification information acquisition step in which an identification information acquisition unit acquires spatial information of a placement area in which an item can be placed, thereby acquiring characteristics of an item that can be randomly placed within the space of the placement area; an identification step in which an identification unit identifies the item from the characteristics of the item acquired by the identification information acquisition unit; a position recognition step in which a position recognition unit assigns planar coordinates to the spatial information of the placement area acquired by the identification information acquisition unit, and acquires the position of the item by repeatedly associating the item identified by the identification unit with the planar coordinates at any timing; a notification step in which the notification direction of the notification unit is controlled so as to notify the position of the item acquired by the position recognition unit; and a calibration step in which a calibration unit associates the planar coordinates assigned by the position recognition unit with the notification direction of the position of the item by the notification unit. [Effects of the Invention]

[0008] According to the present invention, the location of an item can be notified with high accuracy. [Brief explanation of the drawings]

[0009] [Figure 1]1 is a schematic configuration diagram showing an example of the system configuration of an article search system according to a first embodiment. [Figure 2] FIG. 1 is a block diagram showing an example of a simplified representation of the functions of an item search system. [Figure 3] FIG. 10 is a plan view showing an example of a simplified map of the placement area. [Figure 4] FIG. 4 is a schematic diagram showing an example of managing the map of the placement area shown in FIG. 3 using a matrix coordinate system. [Figure 5] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 6] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 7] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 8] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 9] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 10] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 11] FIG. 10 is a diagram illustrating an example of how the position of a movable lamp is associated with a planar image. [Figure 12] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 13] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 14] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 15] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 16] FIG. 10 is a diagram illustrating an example of how to determine the position of a movable lamp. [Figure 17A] 10 is a sequence chart showing an example of a calibration method for an item search system. [Figure 17B] 20B is a sequence chart showing a continuation of the sequence chart shown in FIG. 20A. [Figure 17C] 20C is a sequence chart showing a continuation of the sequence chart shown in FIG. 20B. [Figure 18]10 is a sequence chart showing an example of a procedure for linking the position information of an item within an arrangement area to a map of the arrangement area. [Figure 19] FIG. 10 is a diagram showing an example of an item search sequence. [Figure 20] 10 is a sequence chart for searching for an item that is a component part that makes up a parent item from the identification code of a desired item. [Figure 21] 10 is a sequence chart for carrying a new item into a placement area. [Figure 22] 10 is a sequence chart for carrying out an article from a placement area. [Figure 23] 10 is a sequence chart showing an example of a measure to be taken when an identification code of an item that should be found within the placement area is not found. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the present invention will be described in detail below with reference to the drawings.

[0011] (1) First embodiment FIG. 1 is a schematic diagram showing an example of the system configuration of an item search system 100 according to a first embodiment. The item search system 100 is a system for searching for items that may be placed at random positions in a space. In the first embodiment, an arrangement area 99 is illustrated as an example of a space, and parts that make up a finished product (hereinafter referred to as "items") are illustrated as an example of an item. In addition to the arrangement area 99, work areas and aisle areas may also exist. In the following description, when there is no need to distinguish between each item 5, they will be simply referred to as items 5.

[0012] In the placement area 99, in the illustrated example, an entrance 100a is provided on the back left side wall for bringing in items 5 as parts that make up products manufactured by the factory's manufacturing process, and in the illustrated example, an exit 100b is provided on the right side of the front side wall for transporting items 5 out of the placement area 99.

[0013] A plurality of racks 10 are arranged on the floor of the placement area 99, and each rack 10 has, for example, four wheels on the underside. This allows each rack 10 to be moved to a random position on the floor of the placement area 99, and therefore, can be arranged in a random position within the placement area 99. Therefore, in the placement area 99, the items 5 that can be placed on each rack 10 can be arranged in a random position. In the following description, when it is not necessary to distinguish each rack 10 from another, they will be simply referred to as rack 10.

[0014] The racks 10 each have a flat shelf or multiple shelves, and at least one type of item 5 can be placed on each shelf. For example, the rack 10a shown in the lower left of the figure has two shelves, and different types of items 5a and 5b are placed on each shelf. A work slip corresponding to each item 5 to be placed is attached to the top surface of each rack 10. An identification code 9 for identifying the type of item 5 to be placed on each rack 10 is printed on each work slip. This identification code 9 is, for example, a barcode such as a QR (Quick Response) code (registered trademark). This makes it possible to determine what type of item 5 is placed on the rack 10 by reading the identification code 9.

[0015] Meanwhile, multiple cameras 3 as an example of an identification information acquisition unit are fixed to the ceiling of the placement area 99 in a manner that allows the camera to change the shooting direction. The cameras 3 acquire information about the space of the placement area 99 where the items 5 can be placed by capturing images of the space of the placement area 99, thereby acquiring an identification code 9 as an example of a characteristic of the items 5 that can be randomly placed within the space. Note that instead of the identification code 9, the characteristic of the items 5 may be, for example, the shape of the items 5 themselves.

[0016] When the placement area 99 is divided into a plurality of areas, a camera 3 is provided on the ceiling of the placement area for each area. That is, one camera 3 is provided on the ceiling of each area. By combining the angles of view of the cameras 3, an image can be obtained that covers the entire space of the placement area 99. In the first embodiment, the placement area 99 is managed by dividing it into two areas, left and right, in the orientation shown in the figure, and one camera 3 is provided on the ceiling of the placement area 99 for each area.

[0017] In the first embodiment, an imaging device having a so-called autofocus function or a deep focal depth is used as the camera 3. This allows the camera 3 to reliably read the identification code 9 on the work slip of the item 5 (hereinafter also simply referred to as "the identification code 9 of the item 5") attached to each rack 10 that may be placed at random positions within the placement area 99, even if the height from the floor of the identification code 9 is not always constant.

[0018] Instead of camera 3 taking pictures downward from the ceiling of the placement area 99 in which camera 3 is installed as described above, camera 3 (or movable lamp 7) may be installed facing a vertical surface (e.g., a side wall) within placement area 99, and the identification code 9 of a work slip installed on the side of a vertical rack on which item 5 can be placed may be photographed from the side.

[0019] Furthermore, a movable lamp 7 is provided on the ceiling of the placement area 99 as an example of a notification unit that notifies the location of the desired item 5. At least one movable lamp 7 is provided on the ceiling. The movable lamp 7 includes a lamp unit 7a and a movable unit 7b. The lamp unit 7a has the function of emitting light with a certain degree of directionality.

[0020] The movable unit 7b has a fixed part that is fixed to the ceiling of the placement area 99, and the movable unit 7b that supports the lamp unit 7a is configured to be able to change the orientation of the lamp unit 7a so that it can be tilted, for example, within a range of 180° left and right and 45° up and down, relative to the fixed part. As a result, the movable lamp 7 is configured to be able to irradiate light onto, for example, each rack 10 in accordance with the input attitude control data. This allows a worker to recognize the position of a specific rack 10 within the placement area 99 based on the manner in which light is irradiated by the movable lamp 7.

[0021] In the article search system 100 according to the first embodiment, the above-mentioned movable lamp 7, camera 3, and terminal 6 are connected to one another via a wireless or wired network.

[0022] The terminal 6 is a so-called tablet or computer equipped with a display unit, and in response to an identification code (corresponding to the display of "XXX" in the illustrated example) input by the worker as an example of a characteristic of the desired item 5, the display unit displays the position of the item 5 within the placement area 99 when the position of the item 5 within the placement area 99 is identified by a search, as will be described in detail below. As described above, the characteristic of the item 5 may be the shape of the item 5 instead of the identification code. In this case, the type of item 5 is identified based on the shape of the item 5, and the identification code of the item 5 is determined.

[0023] Here, the worker manually inputs the identification code of the desired item 5, but instead, the code may be input using voice recognition or by reading another type of barcode. When inputting using voice recognition, the input unit 35 is equipped with a voice recognition unit, which receives voice as the characteristics of the item 5 to be input when searching for the position of the item 5 within the placement area 99, and recognizes the received voice as the characteristics of the item 5 using the voice recognition function of this voice recognition unit.

[0024] It should be noted that the method of presenting the search results for the item 5 to the worker is not limited to this, and instead of displaying on the display unit of the terminal 6 described above, for example, a screen display unit 95 may be provided on the side wall of the placement area 99, and the notification unit 37 may display on this screen display unit 95 the position of the item 5 within the placement area 99 searched for by the search unit 36. In this way, the worker can directly grasp the position of the item 5 within the placement area 99 without having to operate the terminal 6.

[0025] 2 is a block diagram showing an example of a simplified representation of the functions of the item search system 100. The item search system 100 includes a control unit 31, an identification information acquisition unit 32, an identification unit 33, a position recognition unit 34, an input unit 35, a search unit 36, a notification unit 37, a memory unit 38, and a calibration unit 39. The control unit 31 is connected to the identification information acquisition unit 32, the identification unit 33, the position recognition unit 34, the input unit 35, the search unit 36, the notification unit 37, the memory unit 38, and the calibration unit 39, and controls the identification information acquisition unit 32, etc. while exchanging data, etc. with these units.

[0026] The storage unit 38 stores the following product configuration table and placement history table. The product configuration table manages the relationship between the product (finished product) manufactured in the manufacturing process in the placement area 99 and the multiple parts that make up that product. On the other hand, the placement history table manages the placement history of the items 5 that have been stored in the placement area 99. The positions of the items 5 that have been recognized in the past by the position recognition unit 34 are written into this placement history table.

[0027] As described above, the identification information acquisition unit 32 has a function of acquiring the identification code 9 of the item 5 that may be placed at a random position within the placement area 99, and one example thereof is the camera 3. The identification information acquisition unit 32 is not limited to the camera 3, and may be a contactless reader or a laser reading device. The identification information acquisition unit 32 may also be an airborne aircraft equipped with the camera 3. This airborne aircraft is an aircraft having at least one propeller, and is equipped with a GPS (Global Positioning System) function so that its position in space within the placement area 99 to which it is moving can be determined.

[0028] The identification information acquisition unit 32 acquires the characteristics of the articles 5 that can be randomly placed within the space of the placement area 99 by acquiring information about the space of the placement area 99 where the articles 5 can be placed.

[0029] The identification unit 33 identifies the article 5 from the characteristics of the article 5 acquired by the identification information acquisition unit 32 .

[0030] Furthermore, when the identification code 9 of the item 5 includes information that can identify the orientation of the identification code itself, the identification unit 33 adjusts the orientation of the identification code to identify the item 5 from the identification code 9 regardless of the orientation of the item 5. Here, it is desirable that the rack 10 on which the item 5 with an identification code is placed is placed in a position that allows the item 5 to be easily photographed by the camera 3 when it is carried into the placement area 99.

[0031] Furthermore, when the identification code 9 contains information that can identify the orientation of the identification code itself of the item 5, the identification unit 33 adjusts the orientation of the identification code 9 to identify the item 5 from the identification code regardless of the orientation of the item 5. Here, it is desirable that the rack 10 on which the item 5 with the identification code 9 is placed is placed in a position that allows the item 5 to be easily photographed by the camera 3 when it is brought into the placement area 99.

[0032] Under the control of the control unit 31, the position recognition unit 34 assigns planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32, and repeatedly associates the item 5 identified by the identification unit 33 with these planar coordinates at any timing to acquire the position of the item 5. Specifically, the position recognition unit 34, for example, combines two images captured by the two ceiling cameras 3 described above to create a map of the placement area 99 that covers the entire floor surface of the placement area 99. In this case, the position recognition unit 34 combines the two images so that they overlap each other, thereby creating a map of the placement area 99 that can completely cover the floor surface of the placement area 99. Furthermore, the position recognition unit 34 manages each matrix-like section of the map of the placement area 99 using a single coordinate system. When overlaying the two images, the position recognition unit 34 and the control unit 31 perform an overlay adjustment process, which will be described later.

[0033] The notification unit 37 includes, for example, an actuator that can change the notification direction, and the notification direction is controlled so as to notify the position of the article 5 acquired by the position recognition unit 34. The notification unit 37 corresponds to, for example, the movable lamp 7 that includes the lamp unit 7a. The notification unit 37 will be described later.

[0034] The calibration unit 39 associates the plane coordinates assigned by the position recognition unit 34 with the direction in which the notification unit 37 notifies the position of the article 5. The processing by the calibration unit 39 will be described later.

[0035] Under the control of the control unit 31, the input unit 35 receives an identification code 9 as an example of a characteristic of the desired item 5 in order to search for the position of the item 5 within the placement area 99. The characteristic of the item 5 may be at least one of a character and a symbol attached to the item 5. The input unit 35 receives input of the identification code via a terminal 6 operated by a worker. Based on the received identification code, the input unit 35 searches for the position of the desired item 5 within the placement area 99 from the association result by the position recognition unit 34.

[0036] Under the control of the control unit 31, the search unit 36 ​​searches for the position of the desired item 5 within the placement area 99 from the association result by the position recognition unit 34 based on the identification code 9 received by the input unit 35. Specifically, the search unit 36 ​​searches for the position of the item 5 within the placement area 99 from the association result by the position recognition unit 34 based on the identification code 9 of the item 5 received by the input unit 35, using the above-mentioned planar coordinates as a reference.

[0037] The search unit 36, under the control of the control unit 31, searches for and understands the relationship between a product (finished product) and the multiple parts that make up that product, based on the product configuration table in the storage unit 38, and identifies the component parts that have that product as a parent item. Note that this identification may also be performed by the control unit 31.

[0038] In this embodiment, the memory unit 38 has a placement history table that manages the placement history of items 5 that have been brought into the placement area 99, and if the item 5 cannot be found within the placement area 99 as a result of photographing by the camera 3, the search unit 36 ​​checks the history of the location information of the item 5 from the above-mentioned placement history table.

[0039] The notification unit 37 has a function of optically notifying the worker of the position of the desired item 5 within the placement area 99 in accordance with the search result by the search unit 36. Specifically, the notification unit 37 outputs, via the control unit 31, attitude control data including control information for controlling the attitude of the lamp unit 7a of the movable lamp 7 so that the lamp unit 7a of the movable lamp 7 faces the rack 10 on which the target desired item 5 is placed. This attitude control data also includes lighting control data for the lamp unit 7a to emit light in a predetermined manner (for example, lit or blinking) for a certain period of time. As a result, the lamp unit 7a of the movable lamp 7 emits light from the lamp unit 7a for a certain period of time so as to cover the rack 10 on which the target item 5 is placed.

[0040] The above-mentioned search unit 36 ​​searches for the identification code 9 to be searched for, based on the planar coordinates of the planar image created by the position recognition unit 34, and obtains position information indicating the position of the identification code 9. The search unit 36 ​​notifies the notification unit 37 of the position information of the identification code 9.

[0041] The notification unit 37 outputs attitude control data including control information for controlling the attitude of the lamp unit 7a of the movable lamp 7 based on the position information of the identification code 9 received from the search unit 36. The attitude control data includes control data such as tilt and pan for driving the movable unit 7b of the movable lamp 7. Specifically, the notification unit 37 calculates control information for the lamp unit 7a from the mounting position of the movable lamp 7 on the ceiling within the placement area 99 and the position information of the identification code 9 in planar coordinates. In order for the notification unit 37 to calculate the control information for the lamp unit 7a corresponding to the position information of the identification code 9, it is necessary to associate the positional relationship between the planar coordinates and the coordinates of the movable lamp 7, as shown in FIG. 34A . The coordinates of the movable lamp 7 here refer to coordinates that represent the movable range, such as the pitch, within which the movable lamp 7 can operate.

[0042] The mounting positions and orientations of the camera 3 and the movable lamp 7 depend on their installation environment in the placement area 99. That is, the camera 3 and the movable lamp 7 are not necessarily installed perfectly parallel to each other due to, for example, errors during installation on the ceiling. Installed parallel here means, for example, that a coordinate system according to the direction in which the camera 3 takes an image (hereinafter also referred to as a "camera coordinate system") and a coordinate system according to the direction in which the lamp unit 7a of the movable lamp 7 faces (hereinafter also referred to as a "lamp coordinate system (planar coordinate system)") match or nearly match. In this embodiment, taking such installation errors into consideration, the planar coordinates of the planar image created by the position recognition unit 34 are associated with the control information for the lamp unit 7a.

[0043] For example, first, the camera 3 mounted on the ceiling captures an image of the floor of the placement area 99 as shown in FIG. 6 without emitting light from the lamp unit 7a of the movable lamp 7 mounted on the ceiling. The position recognition unit 34 acquires a planar image 300A based on the captured image. Next, as shown in FIG. 5, the movable unit 7b of the above-described movable lamp 7 mounted on the ceiling is controlled to point the lamp unit 7a directly downward from the ceiling, and light is emitted from the lamp unit 7a onto the floor surface of the placement area 99, forming a spot of light SP on the floor. In this state, the camera 3 captures a planar image of the floor of the placement area 99 as shown in FIG. 6. The position recognition unit 34 creates a planar image 300B shown in FIG. 7 based on the captured image. Note that the above description only refers to an overview and simplifies the illustration, so there are some parts in FIG. 5 that do not strictly correspond to FIG. 6.

[0044] The position recognition unit 34 calculates the difference in RGB component values ​​between this planar image 300B and the planar image 300A (see FIG. 6) before light irradiation, and creates a difference image 300C shown in FIG. 8 as an image in which the light irradiation position is highlighted with the light spot SP. The position recognition unit 34 considers the location where the difference in RGB component values ​​is large to be the location where light was irradiated (spot SP).

[0045] The position recognition unit 34 determines the center position of the image of the portion irradiated with light (spot SP) in this difference image 300C as the coordinates on the XY plane (lamp coordinate system) of the installation position of the lamp 7 in the camera coordinate system of the camera 3. At this time, the image of the portion irradiated with light (spot SP) captured by the camera 3 usually becomes an ellipse like spot SP1 shown in FIG. 5 due to the positional relationship of the camera 3. Therefore, the position recognition unit 34 can calculate the coordinates of point O', which is estimated to be the installation position of the lamp unit 7a, by determining the center position P' of the ellipse. Alternatively, the position recognition unit 34 may determine the coordinates of the center of a circumscribing rectangle circumscribing the ellipse as the coordinates of point O'.

[0046] Next, under the control of the control unit 31, the position recognition unit 34 controls the movable unit 7b of the movable lamp 7 to tilt the lamp unit 7a by, for example, 30 degrees in the X'-axis direction shown in FIG. 5, causing the lamp unit 7a to emit light, and causes the camera 3 to capture an image of the area irradiated with the light. Based on the captured image, the position recognition unit 34 creates a planar image 300D shown in FIG. 9. The planar image 300D includes an elliptical spot SP (SP1). The position recognition unit 34 calculates the difference between the planar image 300D shown in FIG. 9 and the planar image 300A shown in FIG. 6, and controls the movable unit 7b to tilt the lamp unit 7a by 30 degrees, causing the lamp unit 7a to emit light, and creates a difference image 300E shown in FIG. 10 on the floor of the placement area 99 as an image of the area irradiated with the light.

[0047] 5, the position recognition unit 34 calculates the center position O' of the image of the light-irradiated portion of this differential image 300E in the same manner as described above. The position recognition unit 34 determines the line connecting the position P' and the point O' as the X' axis of the lamp coordinate system of the lamp unit 7a defined by the movable unit 7b, and calculates a line perpendicular to the line passing through point O' and defines this as the Y' axis of the lamp coordinate system. In this way, the position recognition unit 34 can determine the lamp coordinate system as a coordinate system representing the movable range of the lamp unit 7a defined by the movable unit 7b of the movable lamp 7.

[0048] That is, camera 3 as an example of identification information acquisition unit 32 photographs placement area 99 in a state before notification unit 37 notifies the position of item 5 as a spot of light on the floor surface of placement area 99, and acquires planar image 300A as an example of a first image, and photographs placement area 99 in a state after notification unit 37 notifies the position of item 5 as a spot of light SP1 on the floor surface of placement area 99, and acquires planar image 300B or planar image 300D as an example of a second image. Calibration unit 39, for example, calculates the difference in color components of each pixel in these two images, the first image and the second image, and if the difference is greater than a specified value, sets this as the predetermined position (notification position) to be notified by notification unit 37.

[0049] Here, when the size of the above-mentioned placement area 99 is large, it is not easy to obtain the position within the placement area 99, and therefore it is desirable to automate the position obtaining as in this embodiment. On the other hand, when the size of the placement area 99 is small, the mounting position of the movable lamp 7 within the placement area 99 may be obtained by actually measuring it with a measuring instrument such as a tape measure.

[0050] In this way, the position recognition unit 34 associates the camera coordinate system of the camera 3 with the lamp coordinate system of the movable lamp 7, and when a perpendicular line is drawn from position Q in Figure 11, which indicates the position of the identification code 9 within the placement area 99, to the X' axis of the lamp coordinate system, and the intersection point is defined as A, the coordinate of intersection A in the lamp coordinate system can be calculated by the intersection point between the X' axis and a straight line parallel to the Y' axis and passing through point Q.

[0051] Furthermore, the position recognition unit 34 can use, for example, Pythagoras' theorem to calculate the distance between the center position Oc when the movable part 7b of the movable lamp 7 is controlled to point the lamp 7a directly downward to emit light, and point Q indicating the position of the identification code 9. This makes it possible to determine the position of point P on the X' axis when point Q indicating the position of the identification code 9 in Figure 11 is rotated so that it overlaps with the X' axis. In this case, the distance Oc-P and the distance Oc-Q are equal. Note that the height HL of the installation position of the movable lamp 7, i.e., the height from the floor to the ceiling of the placement area 99, is fixed depending on the installation environment of the placement area 99.

[0052] This allows the position recognition unit 34 to determine the rotation angle θ (tilt rotation angle) and the rotation angle φ (pan rotation angle), which are control information including the control parameters of the movable unit 7b. That is, the position recognition unit 34 calculates the rotation angles θ and φ as the corresponding control information for the movable unit 7b from the X and Y coordinates of the camera coordinate system that indicate the position of the identification code 9. This makes it possible to orient the lamp unit 7a of the movable lamp 7 on the ceiling in the direction of the identification code 9, and to irradiate light from the lamp unit 7a.

[0053] In this embodiment, the following processing is performed to illuminate the floor surface of the placement area 99 with light from the movable lamp 7 and associate the position of the light spot SP with the planar image. First, the notification unit 37 issues a notification in a direction perpendicular to the surface of the placement area 99 where the item 5 is to be placed. The camera 3, which is an example of the identification information acquisition unit 32, acquires an image of the surface of the placement area 99. The calibration unit 39 acquires the position of the item 5 to be notified by the notification unit 37 from the image, considers the tip of the perpendicular line to be the mounting position of the notification unit 37, generates control information for the notification unit 37 corresponding to the position of the item 5 on the planar coordinate system based on the mounting position of the notification unit 37, and controls the notification unit 37 based on the control information.

[0054] In this embodiment, the following process is executed to determine the tilt of the axis by irradiating light from the movable lamp 7. First, the notification unit 37 notifies a predetermined position away from the foot of the perpendicular line, and sets the coordinate axes as a line connecting the center of the notified predetermined position and the center of the notified predetermined position in the vertical direction, and a line perpendicular to the line and passing through the center of the notified predetermined position in the vertical direction.

[0055] In this embodiment, in order to be able to determine the mounting position of the movable lamp 7, for example, on the ceiling, even if there is a step or obstacle on the floor of the placement area 99 when the movable lamp 7 shines light directly downward, the light irradiated from the lamp portion 7a of the movable lamp 7 forms a circular pattern on the floor of the placement area 99 as shown in Fig. 16, and each spot SP is photographed by multiple cameras 3 (for example, "camera 1" to "camera 4" in the figure), and the positions of each opposing spot SP are connected to determine the center. The movable lamp 7 is estimated to be located at this center.

[0056] Specifically, first, the notification unit 37 performs, for example, an even number of notifications circumferentially on the floor surface of the placement area 99 where the item 5 is placed, i.e., forms spots SP of light irradiated by the lamp unit 7a on the floor surface. The identification information acquisition unit 32 acquires an image of the floor surface of the placement area 99 captured using the camera 3. The calibration unit 39 acquires all of the predetermined positions (hereinafter also referred to as "notification positions") notified by the notification unit 37 from the image, calculates the positions of the intersections of lines connecting each mounting position of the notification unit 37 with the opposing position on the floor surface of the placement area 99, obtains the center of a circle from the average value of the intercepts of the lines, and regards the position of the center of the circle as the mounting position of the notification unit 37, i.e., the movable lamp 7. Based on the mounting positions of the notification unit 37, the calibration unit 39 generates control information for the notification unit 37 corresponding to the position of the item 5 on the plane coordinate system, and controls the notification unit 37 based on the control information.

[0057] Here, when the lamp 7 is installed on the upper part of the floor of the placement area 99, which has obstacles and steps, as in the planar image 300F shown in Fig. 12, the shape of the light emitted from the lamp unit 7a of the movable lamp 7 will not be elliptical as in the planar image 300G shown in Fig. 13. Furthermore, when the irradiation position of the light emitted from the lamp unit 7a is hidden by an obstacle such as an article 5 as in the planar image 300H shown in Fig. 14, it may happen that part or all of the spot SP at the irradiation position of the light emitted from the lamp unit 7a becomes invisible as in the planar image 300I shown in Fig. 15.

[0058] In this case, the present embodiment handles the situation as follows: Specifically, first, the position recognition unit 34 controls the movable unit 7b of the movable lamp 7 to point the lamp unit 7a directly downward, and irradiates light from the lamp unit 7a onto the floor surface of the placement area 99.

[0059] The position recognition unit 34 changes the angles of multiple cameras 3 installed on the ceiling of the placement area 99, for example, to capture images of the floor of the placement area 99. The position recognition unit 34 acquires the angle of the cameras 3 when an image of a spot SP formed on the floor surface of the placement area 99 by light irradiated from the lamp unit 7a is captured in the center on the left and right. This allows the position recognition unit 34 to grasp the approximate angle of the movable lamp 7 equipped with the lamp unit 7a.

[0060] The position recognition unit 34 directs the camera 3 at that angle. The position recognition unit 34 controls the movable portion 7b of the movable lamp 7 to move the lamp portion 7a in a circular motion, and causes the camera 3 to capture images of multiple locations.

[0061] The position recognition unit 34 acquires the position of the intercept of the line connecting the positions of the lamp units 7a of the multiple opposing movable lamps 7. If there is a step in the spot SP formed on the floor surface of the placement area 99 by the light irradiated from the lamp units 7a, the position recognition unit 34 determines the approximate position of the lamp unit 7a as follows. Specifically, in this case, the position of the lamp unit 7a will be closer than the theoretical value. However, since the line connecting the opposing points does not change, an abnormal value will be obtained if there is an obstacle that is not easily affected. However, because information on multiple points is acquired, the position of the movable lamp 7 can be determined more or less accurately by excluding the top and bottom 15% of abnormal values ​​and calculating the average.

[0062] In this embodiment, the notification unit 37 may display the results of the search by the search unit 36 ​​for the location of the item 5 within the placement area 99 on the display unit of the portable terminal 6 using augmented reality (AR).

[0063] In addition, in this embodiment, the notification unit 37 may project an image of the spatial information of the placement area 99 acquired by the identification information acquisition unit 32 onto a display unit (not shown) using, for example, projection mapping.

[0064] Furthermore, in this embodiment, the notification unit 37 may be configured to have a guidance unit that guides the position of the item 5 within the placement area 99 by using at least one of sound and voice.

[0065] FIG. 3 is a plan view showing an example of a simplified map of the placement area 99. The map of the placement area 99 is made up of the image capture areas 33a and 33b captured by the two cameras 3 described above. As indicated by the two-dot chain line, the image capture areas 33a and 33b are configured so that their adjacent areas overlap, and the angle of view of the two cameras 3 is configured to be larger than the image capture area so that no items 5 are missed. In other words, the angles of view of the two cameras 3 overlap so that no gaps are left between them. As a result, images of the placement area 99 captured using the two cameras 3 are stitched together to form a single image of the placement area 99.

[0066] In the illustrated example, a new rack 10 carrying an item 5 can be carried in through an inlet 100a of the placement area 99, and a rack 10 in the placement area 99 can be carried out through an outlet 100b. Even when a rack 10 is being carried in or out in this way, the item 5 on the rack 10 being carried in or out is photographed by the camera 3 as shown by the two-dot chain line on the bronze medal.

[0067] Fig. 4 is a schematic diagram showing an example of managing the map of the placement area 99 shown in Fig. 3 using a matrix coordinate system. The method of managing the coordinate system is performed by the position recognition unit 34 described above. The position recognition unit 34 manages the map of the placement area 99 using a single coordinate system with a large number of matrix-shaped rectangular sections corresponding to the map of the placement area 99. The position recognition unit 34 joins together images captured by the two cameras 3 described above to create a single coordinate system.

[0068] The general configuration of the item search system 100 has been described above, and an example of the operation of the item search system 100 will now be described.

[0069] 17A to 17C are sequence charts showing an example of a calibration method for the item search system 100. Note that Fig. 20B is a sequence chart showing a continuation of the sequence chart shown in Fig. 20A, and Fig. 20C is a sequence chart showing a continuation of the sequence chart shown in Fig. 20B. The calibration method includes an identification information acquisition step in which the identification information acquisition unit 32 acquires spatial information of the placement area 99 in which the item 5 can be placed, thereby acquiring characteristics of the item 5 that can be randomly placed within the space of the placement area 99; an identification step in which the identification unit 33 identifies the item 5 from the characteristics of the item 5 acquired by the identification information acquisition unit 32; a position recognition step in which the position recognition unit 34 assigns planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32, and acquires the position of the item 5 by repeatedly associating the item 5 identified by the identification unit 33 with the planar coordinates at any timing; a notification step in which the notification direction of the notification unit 37 is controlled so as to notify the position of the item 5 acquired by the position recognition unit 34; and a calibration step in which the calibration unit 39 associates the planar coordinates assigned by the position recognition unit 34 with the notification direction of the position of the item 5 by the notification unit 37.

[0070] When installing the camera 3 and the movable ramp 7, the notification unit 37 associates the position information on the map of the placement area 99 with the control information of the movable part 7b of the movable ramp 7. This enables the control unit 31 to output to the notification unit 37 attitude control data for the movable part 7b that corresponds to the position information of the identification code 9.

[0071] In step S701, the installation worker starts the calibration process using the input unit 35. In step S702, the input unit 35 sends an instruction to start the calibration process to the control unit 31. Next, a map creation process is executed. The map creation process includes steps S703 to S706.

[0072] First, in step S703, the control unit 31 outputs an image acquisition instruction to the identification information acquisition unit 32. In step S704, the identification information acquisition unit 32, in response to the image acquisition instruction, causes the multiple cameras 3 to capture images of the placement area 99. In step S705, the control unit 31 outputs an instruction to the position recognition unit 34 to create a map image by stitching together the captured images as described above. In step S706, in response to the instruction, the position recognition unit 34 stitches together the captured images to create a map image of the placement area 99.

[0073] In step S707, the storage unit 38 saves the map image as the planar image 300A. In step S708, the control unit 31 outputs an instruction to the notification unit 37 to control the movable unit 7b of the movable lamp 7 to point the lamp unit 7a directly downward and irradiate light from the lamp unit 7a. In step S709, the notification unit 37 instructs the movable unit 7b to point the lamp unit 7a directly downward.

[0074] Thereafter, the map creation process (steps S703 to S706) is executed once again. Next, in step S710, the storage unit 38 stores the map image as the planar image 300B.

[0075] Next, in step S711, the control unit 31 instructs the calibration unit 39 to create a difference image 300C between the planar image 300A and the planar image 300B as input information. In step S712, the calibration unit 39 calculates the difference in RGB values ​​of each pixel of the two input images.

[0076] In step S713, if the difference in any one of the R component (red), G component (green), and B component (blue) of the RGB values ​​is 50 or more, the calibration unit 39 sets the RGB values ​​of the pixel to (0, 255, 0) and designates the pixel as a green pixel. If the differences in all of the R component, G component, and B component are less than 50, the calibration unit 39 sets the RGB values ​​of the pixel to (0, 0, 0) and designates the pixel as a black pixel. In this way, the calibration unit 39 creates a difference image 300C in which only the area irradiated by the lamp unit 7a is green and the rest is black.

[0077] In step S714, the control unit 31 instructs the notification unit 37 to change the orientation of the lamp unit 7a, for example, to point 30 degrees upward. In step S715, the notification unit 37 instructs the movable unit 7b to tilt the lamp unit 7a so that it points 30 degrees upward, causing the lamp unit 7a to emit light. Thereafter, the map creation process (steps S703 to S706) is executed. In step S716, the storage unit 38 saves the map image created by the map creation process as a planar image 300D.

[0078] In step S717, the control unit 31 instructs the calibration unit 39 to create a difference image 300E between the planar image 300A and the planar image 300D as input information.

[0079] The calibration unit 39 performs a differential image creation process. The differential image creation process includes steps S712 to S713. In step S718, the control unit 31 outputs an instruction to determine the mounting position of the movable lamp 7 in the placement area 99 and the coordinate system of the movable lamp 7 (lamp coordinate system) based on the two differential images 300C and 300E created by the calibration unit 39.

[0080] In step S719, the calibration unit 39 determines the center points of the positions where light is irradiated from the lamp unit 7a (hereinafter also referred to as "lamp irradiation positions") from the input difference images 300C and 300E. In step S720, the calibration unit 39 calculates the equation of a line connecting the two center points determined above, and the equation of a line that is perpendicular to the line and passes through the center point of the lamp irradiation position in the difference image 300C, and sets these as the X and Y coordinate axes of the lamp coordinate system, respectively.

[0081] The control unit 31 outputs an instruction to the calibration unit 39 to request conversion processing to calculate control information for the movable unit 7b of the movable lamp 7 corresponding to the position information of an arbitrary identification code 9. The calibration unit 39 calculates the tilt angle θ and the pan angle φ by determining the distance between the position information of the identification code 9 and the origin of the lamp coordinate system, and the coordinates of the point where a perpendicular line is dropped from the position information of the identification code 9 to the X-axis of the lamp coordinate system. This origin is the center point of the lamp irradiation position in the difference image 300C.

[0082] FIG. 18 is a sequence chart showing an example of a procedure for linking the position information of an item 5 within a placement area 99 to a map of the placement area 99. In step S101, the control unit 31 controls two cameras to capture images of the placement area 99 using the two cameras 3, respectively. In step S102, the position recognition unit 34 stitches together the two images of the placement area 99 to create a map of the placement area 99. In step S103, the control unit 31 assigns the above-mentioned coordinate system to the generated map.

[0083] Meanwhile, in step S104, the worker carries the item 5 into any position in the placement area 99 from the entrance 100a and places the item 5 in a position where the identification code 9 attached to the work slip for the item 5 can be photographed by the camera 3.

[0084] Next, in step S105, the control unit 31 outputs an instruction to the camera 3 to read the identification code 9 of the item 5 at an arbitrary timing. In step S106, the camera 3 reads the identification code of the item 5. Note that the method of identifying the item 5 is not limited to the above-mentioned method, and methods using characters or symbols attached to the item 5, the shape of the item 5 itself, or a non-contact tag (RFID: Radio Frequency IDentification) may also be used. In step S107, the control unit 31 causes the position recognition unit 34 to link the position information within the placement area 99 relating to the identification code 9 read by the camera 3 to the map. Linking here means managing the map in the coordinate system.

[0085] On the other hand, in step S108, the worker moves the position of the rack 10 on which the item 5 is mounted within the placement area 99.

[0086] Next, in step S109, the control unit 31 outputs an instruction to the camera 3 to read the identification code 9 of the item 5 at an arbitrary timing. In step S110, the camera 3 reads the identification code 9 within the placement area 99. Note that in step S110, the size of the identification code itself may be increased so that it can be read, or the placement location of the item 5 may be moved to a position visible to the camera 3. Note that in steps S109 to S110, since there is a high possibility that a worker is involved when the position of the item 5 on the rack 10 changes, the presence of a worker may be recognized and the movement of the identification code within a certain range from the worker may be tracked. Also, in steps S109 to S110, reading of the identification code may be stopped outside of business hours, such as at night. Next, in step S111, the control unit 31 links the location information within the placement area 99 for the identification code 9 read by the camera 3 to the map.

[0087] FIG. 19 is a diagram showing an example of a search sequence for an article 5. In step S201, the worker inputs the identification code of the desired article 5 from the terminal 6. In step S202, the control unit 31 compares the input identification code with the identification code 9 on the map. In step S203, the control unit 31 outputs attitude control data to the notification unit 37 for the movable lamp 7 corresponding to the coordinates of the article on the map whose identification code matches. This attitude control data includes control information for setting a predetermined rotation angle left / right and up / down.

[0088] In step S204, the notification unit 37 outputs attitude control data to the movable lamp 7. The movable lamp 7 then controls the movable unit 7b in accordance with the input attitude control data to control the attitude of the lamp unit 7a so that it is oriented at a predetermined rotation angle left and right and up and down, and causes the lamp unit 7a to emit light for a certain period of time. In the first embodiment, when emitting light in this manner, in order to improve visibility for workers, methods such as changing the color of the lit light or flashing the light may be adopted, rather than simply turning it on. In the first embodiment, if the same type of desired item 5 is present on multiple racks 10, the color of the light may be changed depending on the purpose.

[0089] Meanwhile, in step S205, the worker finds the desired item 5 using the light emitted from the lamp portion 7a of the movable lamp 7, and moves the desired item 5 from the placement area 99 to the work area. The work area is where assembly is performed using the moved desired item 5.

[0090] In step S206, the control unit 31 outputs an instruction to the notification unit 37 to turn off the lamp unit 7a of the movable lamp 7 after a predetermined input or after a certain time has elapsed. In step S207, the notification unit 37 outputs the instruction to turn off the movable lamp 7, turning off the lamp unit 7a.

[0091] The method of notifying the position of the article 5 within the placement area 99 is not limited to the spotlight method using the movable lamp 7 described above, but may include shining a laser at the article 5 with a pinpoint, suggesting the position as a silhouette of the article using light, displaying a map including the position of the article 5 on a tablet, smartphone, or monitor as an example of the terminal 6, superimposing the article 5 on augmented reality (AR), displaying the position of the article 5 in cooperation with a global positioning system (GPS), or simply displaying it on a screen. Furthermore, methods of notifying the position of the article 5 within the placement area 99 may include projecting the position of the article 5 within the placement area 99 using so-called projection mapping, indicating the position of the article 5 using a beacon, suggesting the position of the article 5 using a so-called patrol lamp (registered trademark) or searchlight, or guiding the position of the article 5 using sound or voice.

[0092] In the first embodiment, if the identification codes 9 are not recognized due to unintentional overlapping of the identification codes 9 or the like, and the control unit 31 determines that the item 5 is not present in the placement area 99 in the latest history, the movable lamp 7 may be configured to emit light according to the position information of the item 5 going back to the time when the desired item 5 was last present in the placement area 99. In this way, the worker can identify the location when searching for the desired item 5.

[0093] 20 is a sequence chart for searching for an item that is a component part that makes up a parent item from the identification code of a desired item 5. In step S301, an operator inputs, from terminal 6, the identification code of a finished product that is a parent item made up of a combination of multiple component parts (items 5). In step S302, control unit 31 compares the part numbers of items 5 that are component parts that make up the parent item with the identification codes 9 on the map.

[0094] In step S303, the control unit 31 controls the notification unit 37 to output attitude control data to all movable lamps 7 corresponding to the map coordinates that match for all items of goods 5. In step S304, the notification unit 37 outputs the input attitude control data and turns on or blinks the movable lamps 7 for all items of goods 5. In step S304, the notification unit 37 may notify multiple items 5 at once, or may notify multiple items 5 one by one by shining a light on them in order of proximity to the position of the search unit 36 ​​(or terminal 6), whose position relative to the placement area 99 is preset, or may notify multiple items 5 in a preset order.

[0095] Meanwhile, in step S305, the worker searches for all of the above-mentioned items 5 using the guidance of the light emitted from the movable lamp 7 and moves them from the placement area 99 to the work area.

[0096] In step S306, the control unit 31 controls the notification unit 37 so that the movable lamp 7 is turned off after a certain time has elapsed. In step S307, the notification unit 37 controls the movable lamp 7 to be turned off.

[0097] 21 is a sequence chart for newly carrying in an item 5 into the placement area 99. In step S401, a worker passes through the entrance 100a of the placement area 99 while moving a rack 10 in a state where the identification code 9 attached to the work slip of the item 5 can be photographed by the camera 3.

[0098] In step S402, the identification information acquisition unit 32 controls the camera 3 capable of photographing the entrance 100a of the placement area 99 to read the identification code 9 on the work slip for the item 5. In step S403, the control unit 31 recognizes that the item 5 corresponding to the read identification code 9 has been brought into the placement area 99. In step S404, the notification unit 37 notifies the worker that the reading at the time of delivery has been completed by turning on the movable lamp 7.

[0099] 22 is a sequence chart for carrying out an item from the placement area 99. In step S501, a worker places the item 5 on the rack 10 in a position where it can be photographed by the camera 3, and passes through the outgoing entrance 100b of the placement area 99.

[0100] In step S502, the identification information acquisition unit 32 reads the identification code of the item 5 using the camera 3 capable of photographing the exit 100b of the placement area 99. In step S503, the control unit 31 recognizes that the item with the read identification code 9 has been removed from the placement area 99. In step S504, the notification unit 37 turns on the movable lamp 7 to notify the worker that the reading at the time of removal has been completed.

[0101] In the first embodiment, as described above, in addition to lighting or flashing the movable lamp 7 described above, methods of notifying workers that an item 5 has been brought into (stored in) or taken out (dispatched from) the placement area 99 may also be used, for example, to display a map on a terminal 6 such as a tablet, smartphone, or monitor, to link with virtual reality (AR: Augmented Reality) or GPS (Global Positioning System), to display on a screen, to display using projection mapping, to notify with a patrol lamp (registered trademark), or to notify with sound or voice.

[0102] 23 is a sequence chart showing an example of a countermeasure when the identification code 9 of an item 5 that should be found in the placement area 99 is not found. In step S601, the worker inputs the identification code of the desired item 5 from the terminal 6. In step S602, the control unit 31 compares the identification code 9 of the item 5 that is a component part that makes up the parent item with the identification code 9 on the map.

[0103] In step S603, the control unit 31 determines whether or not the item 5 of the specified part is present in the placement area 99, and if not, executes step S604, whereas if present, skips step S604.

[0104] In step S604, the control unit 31 refers to the placement history in the placement history table (not shown) described above for the item 5, determines whether or not there is a placement history, and if there is a placement history, executes step S605, whereas if there is no placement history, executes step S606.

[0105] In step S605, the notification unit 37 notifies the worker that the item has been taken out by lighting the movable lamp 7. In step S606, the control unit 31 goes back to the most recent history of recognition of the target item 5 and outputs attitude control data to the movable lamp 7 corresponding to the coordinates of the item 5 within the placement area 99. In step S607, the notification unit 37 moves the movable lamp 7 in accordance with the input attitude control data and lights or flashes it. At this time, if the movable lamp 7 is irradiating light onto multiple items 5, the light may be irradiated onto each item 5 in any order or a predetermined order or cycle, or the light may be irradiated onto each item 5 in order of proximity to the control unit 31, whose installation position has been determined in advance.

[0106] In step S608, the control unit 31 outputs, after a certain time has elapsed, an instruction to turn off the movable lamp 7. In step S609, the notification unit 37 turns off the movable lamp 7.

[0107] Meanwhile, in step S610, the worker searches for the desired item 5 using the guidance of the light emitted from the movable lamp 7 and moves it from the placement area 99 to the work area.

[0108] The item search system 100 of this embodiment includes an identification information acquisition unit 32 that acquires characteristics of items 5 that may be randomly placed in the space by acquiring spatial information of the placement area where items may be placed, an identification unit 33 that identifies items from the item characteristics acquired by the identification information acquisition unit 32, a position recognition unit 34 that assigns planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32 and repeatedly associates items identified by the identification unit 33 with the planar coordinates at any timing, an input unit 35 that accepts the characteristics of the desired item 5 in order to search for its position within the placement area 99, a search unit 36 ​​that searches for the position of the desired item 5 within the placement area 99 from the position recognition results by the position recognition unit 34 based on the characteristics of the desired item 5 input via the input unit 35, and a notification unit 37 that notifies the position of the desired item 5 searched for by the search unit 36.

[0109] With this configuration, the identification information acquisition unit 32 acquires spatial information of the placement area 99 where items can be placed, and thereby acquires characteristics of the items 5 that can be randomly placed within the space. The identification unit 33 identifies the items 5 from the characteristics of the items 5 acquired by the identification information acquisition unit 32. The position recognition unit 34 assigns planar coordinates to the spatial information of the placement area 99, and repeatedly associates the items 5 identified by the identification unit 33 with the planar coordinates at any timing. The search unit 36 ​​searches for the position of the desired item within the placement area 99 based on the characteristics of the desired item 5 input via the input unit 35. The notification unit 37 notifies the user of the position of the desired item searched for by the search unit 36.

[0110] In this embodiment, the identification information acquisition unit 32 is an airborne flying object equipped with a camera 3 as an example of an imaging unit. In this way, the identification information acquisition unit 32 can more reliably acquire the identification code of the article 5 by changing its position and orientation within the placement area 99.

[0111] In this embodiment, the identification unit 33 identifies the article 5 from the characteristics of the article 5 regardless of the orientation of the article 5. In this way, the identification information acquisition unit 32 can more reliably acquire the identification code of the article 5 that may be placed in a random position and orientation within the placement area 99.

[0112] In this embodiment, the feature of the item 5 is an identification code that allows the item 5 to be distinguished from other items, and when the item 5 is carried into the placement area 99, the item 5 with the identification code is placed in a position that makes it easy to acquire by the identification information acquisition unit 32. In this way, the identification code of the item 5 can be read more reliably.

[0113] In this embodiment, the memory unit 38 has a placement history table (not shown) that manages the placement history of items 5 that have been brought into the placement area 99, and if the identification information acquisition unit 32 cannot acquire the item 5 within the placement area 99, the search unit 36 ​​checks the history of the location information of the item 5 from this placement history table. In this way, even if the search unit cannot find the desired item 5 within the placement area 99, the worker can be notified of the location where the item 5 was most recently placed, making it easier to find the desired item 5 within the placement area 99.

[0114] In this embodiment, the notification unit 37 optically notifies the location of the item 5 within the placement area 99. In this way, it becomes easier for the worker to visually find the desired item 5 within the placement area 99.

[0115] In this embodiment, the notification unit 37 may be configured to display the position of the desired item 5 in the placement area 99, which has been searched for by the search unit 36, on a screen display unit 95 provided on a side wall within the placement area 99. In this way, it is not necessary to provide a dedicated terminal for displaying the position of the item 5 in the placement area 99, and the worker can visually grasp the position of the desired item 5 in the placement area 99 even if he or she does not have the terminal 6 in his or her hand.

[0116] In this embodiment, the notification unit 37 uses augmented reality to display the results of the search performed by the search unit 36 ​​to find the location of the item 5 within the placement area 99 on the display unit of the portable terminal 6. In this way, the worker can easily grasp the location of the item 5 using augmented reality, which is easy for the worker to grasp intuitively.

[0117] In this embodiment, the notification unit 37 projects an image of the spatial information of the placement area 99 acquired by the identification information acquisition unit 32 onto the display unit using, for example, projection mapping.

[0118] In this embodiment, the notification unit 37 has a guidance unit that uses at least one of sound and voice to guide the worker as to the location of the item 5 within the placement area 99. In this way, the worker can auditorily grasp the location of the desired item 5 within the placement area 99.

[0119] In this embodiment, the identification information acquisition unit 32 stitches together images of the placement area 99 taken by the multiple cameras 3 to create a single image of the placement area 99. In this way, the space of the placement area 99 can be accurately covered without any omissions.

[0120] In this embodiment, the input unit 35 has a voice recognition unit that receives voice as a characteristic of the item 5 to be input when searching for the position of the item 5 within the placement area 99, and uses a voice recognition function to recognize the received voice as a characteristic of the item 5. In this way, the worker does not need to manually input the identification code of the desired item 5, and the effort required to search for the position of the item 5 can be reduced.

[0121] As described above, the item search system 100 according to this embodiment comprises an identification information acquisition unit 32 that acquires characteristics of items 5 that may be randomly placed within the space of the placement area 99 by acquiring spatial information of the placement area 99 where the items 5 may be placed; an identification unit 33 that identifies the items 5 from the characteristics of the items 5 acquired by the identification information acquisition unit 32; a position recognition unit 34 that acquires the position of the items 5 by assigning planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32 and repeatedly associating the items 5 identified by the identification unit 33 with the planar coordinates at any timing; a notification unit 37 whose notification direction is controlled to notify the position of the item 5 acquired by the position recognition unit 34; and a calibration unit 39 that associates the planar coordinates assigned by the position recognition unit 34 with the notification direction of the position of the item 5 by the notification unit 37.

[0122] In this way, the position recognition unit 34 acquires the position of the article 5 by repeatedly associating the article 5 with planar coordinates at any timing. The notification direction of the notification unit 37 is controlled so as to notify the position of the article 5 acquired by the position recognition unit 34. Therefore, since the position of the article 5 is acquired by repeatedly associating the article 5 with planar coordinates at any timing, the notification unit 37 can accurately notify the position of the article 5 regardless of the presence or absence of an obstacle.

[0123] In this embodiment, the notification unit 37 issues a notification in a direction perpendicular to the surface of the placement area 99 where the item 5 is placed. The identification information acquisition unit 32 acquires an image of the surface of the placement area 99 using a camera 3, which is an example of an imaging unit. The calibration unit 39 acquires the position of the item 5 to be notified by the notification unit 37 from the image, considers the tip of the perpendicular line to be the mounting position of the notification unit 37, generates control information for the notification unit 37 corresponding to the position of the item 5 on the planar coordinate system based on the mounting position of the notification unit 37, and controls the notification unit 37 based on the control information. In this way, the light from the movable lamp 7 can be used to illuminate the floor surface of the placement area 99, and the position of the light spot can be linked to the planar image.

[0124] In this embodiment, the notification unit 37 notifies a predetermined position away from the foot of the perpendicular line, and the coordinate axes are a line connecting the center of the notified predetermined position and the center of the notified predetermined position in the vertical direction, and a line perpendicular to the line and passing through the center of the notified predetermined position in the vertical direction. In this way, the tilt of the axis can be obtained by irradiating light from the movable lamp 7.

[0125] In this embodiment, the notification unit 37 performs notification in a circular pattern, for example, an even number of times, on the floor surface of the placement area 99 where the item 5 is placed (i.e., forms spots SP of light irradiated by the lamp unit 7a on the floor surface). The identification information acquisition unit 32 acquires an image of the floor surface of the placement area 99 captured by the camera 3. The calibration unit 39 acquires all of the predetermined positions (hereinafter also referred to as "notification positions") notified by the notification unit 37 from the image, calculates the positions of the intersections of lines connecting each mounting position of the notification unit 37 with the opposing position on the floor surface of the placement area 99, obtains the center of a circle from the average value of the intercepts of the line, and regards the position of the center of the circle as the mounting position of the notification unit 37, i.e., the movable lamp 7. Based on the mounting positions of the notification unit 37, the calibration unit 39 generates control information for the notification unit 37 corresponding to the position of the item 5 on the plane coordinate system, and controls the notification unit 37 based on the control information. In this way, even if there is a step or obstacle on the floor of the placement area 99 when the movable lamp 7 shines light directly downward, the mounting position of the movable lamp 7 on the ceiling, for example, can be identified.

[0126] In this embodiment, the identification information acquisition unit 32 uses the camera 3 to acquire a planar image 300A as an example of a first image obtained by capturing the placement area 99 in a state before the notification unit 37 notifies the position of the item 5 as a spot of light on the floor surface of the placement area 99, and a planar image 300B as an example of a second image obtained by capturing the placement area 99 in a state after the notification unit 37 notifies the position of the item 5 as a spot of light on the floor surface of the placement area 99. The calibration unit 39 calculates the difference in color components of each pixel between the first image and the second image, and if the difference is greater than a specified value, the calibration unit 39 calculates the predetermined position (referred to as the notification position) to be notified by the notification unit 37. In this way, the position of the item 5 can be more accurately notified by the notification unit 37.

[0127] (2) Second embodiment The item search system according to the second embodiment has almost the same configuration and operation as the item search system 100 according to the first embodiment, so a description of the similar configuration and operation will be omitted and the following description will focus on the differences.

[0128] The item search system according to the second embodiment includes a placement area 99a corresponding to the placement area 99 of the item search system 100 according to the first embodiment, and a passage area 99b which is a space continuous with the placement area 99a.

[0129] Dedicated rails 98, which are an example of guide units, are laid on the ceilings of the placement area 99a and the passage area 99b so as to cover the entire floor surface of the placement area 99a and the passage area 99b. A placement area dedicated rail 98a is installed on the ceiling of the placement area 99a, and a passage area dedicated extension rail 98b is installed on the ceiling of the passage area 99b. In the following description, unless there is a need to particularly distinguish between the placement area dedicated rail 98a and the passage area dedicated extension rail 98b, they will be simply referred to as dedicated rails 98.

[0130] The dedicated rail 98 is provided with a movable part 97 that can move along the dedicated rail 98, and this movable part 97 can be driven by, for example, a rechargeable battery, and is configured to be able to move along the dedicated rail 98 while supporting the camera 3. Therefore, as the movable part 97 moves along the dedicated rail 98, the camera 3 can capture images of the space within the placement area 99a and the passage area 99b.

[0131] A maintenance area 99c is provided in part of the placement area 99a. In this maintenance area 99c, a part of the placement area dedicated rail 98a described above descends from the ceiling to the floor of the placement area 99a. This allows a worker to reach the movable part 97 when the movable part 97 moving on the placement area dedicated rail 98a enters the maintenance area 99c, enabling a worker to perform maintenance work such as replacing the battery of the movable part 7b.

[0132] Furthermore, in the article search system according to the modified example of the second embodiment, the movable unit 97, which is an example of a guide unit, is provided with a remote-controlled electric lift that can move the camera 3 up and down in a direction perpendicular to the floor surface of the maintenance area 99c. In this way, maintenance such as battery replacement for the movable unit 7b can be performed in the placement area 99a other than the above-mentioned maintenance area 99c.

[0133] (3) Other variations In this embodiment, a QR code (registered trademark) is used as an example of an identification code (barcode) attached to identify each item 5 from another item, but the present invention is not limited to this and other types of barcodes may also be used.

[0134] In addition, in this embodiment, as a measure to be taken when an item 5 is intentionally removed from the placement area 99, the camera 3 as the identification information acquisition unit 32 may be linked to follow the actions of the worker, follow the identification code of the item 5, always read the identification code of the item 5 at the exit 100b, or be linked to various sensors.

[0135] In addition, in this embodiment, the timing for acquiring the location information of the item 5 within the placement area 99 can be any timing as described above, or can be random at a fixed time, at a time when the item 5 is moving, only during the day or only at night, or in cooperation with human detection such as that of workers, at short intervals during times when people are detected.

[0136] Furthermore, in this embodiment, the fixed portion of the movable lamp 7 is mainly fixed to the ceiling, but the present invention is not limited to this, and the movable lamp 7 may be configured to be movable on the ceiling.

[0137] In addition, in this embodiment, methods for making the identification code of the item 5 easier to read include determining the placement position of the rack 10 in the placement area 99 in advance and determining the placement position of the identification code in advance, as well as reducing the information in the identification code and making the code itself larger.

[0138] In addition, in this embodiment, as a countermeasure when the identification code cannot be read, the position recognition unit 34 may refer to the placement history table in the memory unit 38 and notify the notification unit 37 about the item 5 on a date close to the date on which the identification code was read, or may notify the notification unit 37 about the data one step before in chronological order, or may automatically notify the notification unit 37 about the items 5 up to the point where they have been recognized.

[0139] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, the elements described in parallel in the present embodiment may be configured such that at least one of the elements is connected in series to the other elements. [Industrial Applicability]

[0140] The present invention can be applied to an article search system and a calibration method for an article search system that can accurately search for articles that may be stored in random positions within a placement area. [Explanation of symbols]

[0141] 3...camera, 7...movable lamp, 5...item, 6...terminal, 9...identification code, 10...rack, 31...control unit, 32...identification information acquisition unit, 33...identification unit, 37...notification unit, 99...placement area, 100...item search system.

Claims

1. an identification information acquisition unit that acquires information about the space of a placement area in which items can be placed, thereby acquiring characteristics of items that can be randomly placed within the space of the placement area; an identification unit that identifies the article based on the characteristics of the article acquired by the identification information acquisition unit; a position recognition unit that assigns planar coordinates to the spatial information of the placement area acquired by the identification information acquisition unit, and repeatedly associates the item identified by the identification unit with the planar coordinates at any timing to acquire the position of the item; a notification unit whose notification direction is controlled so as to notify the position of the item acquired by the position recognition unit; a calibration unit that associates the plane coordinates assigned by the position recognition unit with a notification direction of the position of the item by the notification unit; An object search system comprising:

2. the notification unit issues a notification in a direction perpendicular to a surface of the placement area in which the item is placed, the identification information acquisition unit acquires an image of the surface of the placement area using an imaging unit; The calibration unit The position of the item to be notified by the notification unit is acquired from the image, and the tip of the perpendicular line is regarded as the mounting position of the notification unit; generating control information for the notification unit corresponding to the position of the item on the plane coordinate system based on the mounting position of the notification unit, and controlling the notification unit based on the control information; 2. The object search system according to claim 1.

3. The notification unit a predetermined position away from the foot of the perpendicular line is notified, and a line connecting the center of the notified predetermined position and the center of the predetermined position when notified in the vertical direction, and a line perpendicular to the line and passing through the center of the predetermined position when notified in the vertical direction are set as coordinate axes; 3. The object search system according to claim 2.

4. the notification unit notifies the user an even number of times in a circular pattern on the surface of the placement area where the item is placed, the identification information acquisition unit acquires an image of the placement area using an imaging unit; The calibration unit acquire, from the image, all predetermined positions notified by the notification unit, calculate the positions of intersections of lines connecting the mounting positions of each of the notification units and the opposing positions on the surface of the placement area, find the center of a circle from the average value of the intercepts of the line, regard the position of the center of the circle as the mounting position of the notification unit, generate control information for the notification unit corresponding to the position of the item on the plane coordinate system based on the mounting positions of the notification units, and control the notification unit based on the control information; 2. The object search system according to claim 1.

5. The identification information acquisition unit using an imaging unit, acquiring a first image obtained by capturing the placement area in a state before the notification unit notifies the user, and a second image obtained by capturing the placement area in a state after the notification unit notifies the user; The calibration unit a difference between color components of each pixel of the first image and the second image is calculated, and when the difference is greater than a specified value, the predetermined position is notified by the notifying unit; 4. The object search system according to claim 2 or 3.

6. an identification information acquisition step in which an identification information acquisition unit acquires information about the space of an arrangement area in which an item can be arranged, thereby acquiring characteristics of the item that can be randomly arranged in the space of the arrangement area; an identification step in which an identification unit identifies the item based on the characteristics of the item acquired by the identification information acquisition unit; a position recognition step in which a position recognition unit assigns planar coordinates to the spatial information of the placement area acquired by the identification information acquisition unit, and repeatedly associates the item identified by the identification unit with the planar coordinates at any timing to acquire the position of the item; a notification step of controlling a notification direction of a notification unit so as to notify the position of the item acquired by the position recognition unit; a calibration step in which a calibration unit associates the plane coordinates assigned by the position recognition unit with a notification direction of the position of the item by the notification unit; A calibration method for an object searching system, comprising:

7. In the notification step, the notification unit issues a notification in a direction perpendicular to a surface of the placement area in which the item is placed, In the identification information acquisition step, the identification information acquisition unit acquires an image of the surface of the placement area using an imaging unit, In the calibration step, the calibration unit acquires, from the image, the position of the item to be notified by the notification unit, regards the tip of the perpendicular line as the mounting position of the notification unit, generates control information for the notification unit corresponding to the position of the item on the plane coordinate system based on the mounting position of the notification unit, and controls the notification unit based on the control information.

7. The calibration method according to claim 6.

8. In the notification step, the notifying unit notifies a predetermined position away from the foot of the perpendicular line, and a line connecting the center of the notified predetermined position and the center of the predetermined position when notified in the vertical direction, and a line perpendicular to the line and passing through the center of the predetermined position when notified in the vertical direction are set as coordinate axes; 8. The calibration method according to claim 7.

9. In the notification step, the notification unit notifies the user an even number of times in a circular pattern on the surface of the placement area where the item is placed, In the identification information acquisition step, the identification information acquisition unit acquires an image of the placement area using an imaging unit, In the calibration step, the calibration unit acquires from the image all of the predetermined positions notified by the notification unit, calculates the positions of intersections of lines connecting the mounting positions of the notification units with opposing positions on the surface of the placement area, finds the center of a circle from the average value of the intercepts of the line, regards the position of the center of the circle as the mounting position of the notification unit, generates control information for the notification unit corresponding to the position of the item on the plane coordinate system based on the mounting positions of the notification units, and controls the notification unit based on the control information.

7. The calibration method according to claim 6.

10. In the identification information acquisition step, the identification information acquisition unit acquires, using an imaging unit, a first image of the placement area taken before the notification unit notifies the user, and a second image of the placement area taken after the notification unit notifies the user, In the calibration step, the calibration unit calculates a difference in color components of each pixel of the first image and the second image, and when the difference is greater than a specified value, the notification unit determines that the position is the predetermined position.

9. The calibration method according to claim 7 or 8.

Images