Article search system and map image creation method for article search system

Abstract

To prepare one map image in a wide range by using an inexpensive camera.SOLUTION: An identification information acquisition unit which acquires information on a space in a placement area in which an article can be placed, thereby acquiring characteristics of the article which can be placed at random in the space; an identification unit which identifies the article from the characteristics of the article acquired by the identification information acquisition unit; and an information processing unit which assigns plane coordinates to the information on the space in the placement area acquired by the identification information acquisition unit, and a position recognition unit that associates an article repeatedly identified by the identification unit at an arbitrary timing with plane coordinates, in which the identification information acquisition unit includes one imaging unit and changes a zoom magnification according to a distance from the one imaging unit to an imaging position while changing the imaging position of the one imaging unit to capture a plurality of enlarged videos, and the identification unit connects the plurality of enlarged videos such that parts of the images overlap each other to form an image of one disposition area or an image of a part of the disposition area.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an article search system and a map image creation method for an article search system, and is suitable for application to an article search system that creates a single map image by stitching together multiple captured images. [Background technology]

[0002] A technique for creating a map image by stitching together camera images is disclosed in Patent Document 1. Patent Document 1 discloses a method for converting satellite images into a map image and a technical method for creating a panoramic image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 5-277973 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in Patent Document 1 requires an expensive, high-performance camera and does not take into consideration stitching together high-resolution images captured by zoom shooting.

[0005] The present invention has been made in consideration of the above points, and aims to propose an item search system and a map image creation method for an item search system that can create a single wide-area map image using an inexpensive camera. [Means for solving the problem]

[0006] In order to solve this problem, the present invention comprises an identification information acquisition unit that acquires characteristics of items that can be randomly placed within a placement area by acquiring spatial information of the area where items can be placed, an identification unit that identifies the items from the characteristics of the items acquired by the identification information acquisition unit, and a position recognition unit that assigns planar coordinates to the spatial information of the placement area acquired by the identification information acquisition unit and repeatedly associates the items identified by the identification unit with the planar coordinates at any timing, wherein the identification information acquisition unit has a single imaging unit that captures a plurality of enlarged images while changing the shooting position of the single imaging unit and changing the zoom magnification depending on the distance from the single imaging unit to the shooting position, and the identification unit stitches the multiple enlarged images together so that parts of the images overlap to create a single image of the placement area or an image of part of the placement area.

[0007] In the present invention, the method includes an identification information acquisition step in which an identification information acquisition unit acquires spatial information of a placement area where items can be placed, thereby acquiring characteristics of items that can be randomly placed within the space; an identification step in which an identification unit identifies the items from the characteristics of the items acquired by the identification information acquisition unit; and a position recognition step in which a position recognition unit assigns planar coordinates to the spatial information of the placement area acquired by the identification information acquisition unit, and repeatedly associates the items identified by the identification unit with the planar coordinates at any timing.In the identification information acquisition step, the identification information acquisition unit, which has one imaging unit, captures a plurality of enlarged images while changing the shooting position of the one imaging unit and changing the zoom magnification according to the distance from the one imaging unit to the shooting position, and in the identification step, the identification unit connects the multiple enlarged images so that parts of the images overlap to create a single image of the placement area or an image of part of the placement area. [Effects of the Invention]

[0008] According to the present invention, a single map image covering a wide area can be created using an inexpensive camera. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic configuration diagram showing an example of the system configuration of an article search system according to a first embodiment. [Figure 2] FIG. 1 is a block diagram showing an example of a simplified representation of the functions of an item search system. [Figure 3] FIG. 10 is a plan view showing an example of a simplified map of the placement area. [Figure 4] FIG. 4 is a schematic diagram showing an example of managing the map of the placement area shown in FIG. 3 using a matrix coordinate system. [Figure 5] FIG. 10 is a diagram showing an example of the overlap width of the overlap area of ​​the images. [Figure 6] FIG. 10 is a diagram showing an example of the overlap width of the overlap area of ​​the images. [Figure 7] FIG. 10 is a diagram showing an example of the overlap width of the overlap area of ​​the images. [Figure 8] FIG. 10 is a diagram showing an example of the overlap width of the overlap area of ​​the images. [Figure 9] FIG. 10 is a diagram showing an example of a map image formed by joining together a single image. [Figure 10] FIG. 10 is a diagram illustrating an example of a partial space image to be cut out. [Figure 11] FIG. 10 is a diagram illustrating an example of a subspace image. [Figure 12] FIG. 10 is a diagram illustrating an example of a subspace image. [Figure 13] FIG. 10 is a diagram illustrating an example of a subspace image. [Figure 14] FIG. 10 is a diagram illustrating an example of a subspace image. [Figure 15] FIG. 10 is a diagram showing an example of how a large number of fragmentary images are arranged and a subspace image is extracted. [Figure 16] FIG. 10 is a diagram showing an example of how a large number of fragmentary images are arranged and a subspace image is extracted. [Figure 17] FIG. 10 is a diagram illustrating an example of how a plurality of subspace images are stitched together. [Figure 18]FIG. 10 is a diagram illustrating an example of how a plurality of subspace images are stitched together. [Figure 19] FIG. 10 is a diagram illustrating an example of how a plurality of subspace images are stitched together. [Figure 20] FIG. 10 is a diagram illustrating an example of how a plurality of subspace images are stitched together. [Figure 21] FIG. 1 is a diagram showing an example of an image captured by a camera expressed in a pixel coordinate system. [Figure 22] FIG. 1 is a diagram showing an example of an image captured by a camera expressed in a world coordinate system. [Figure 23] 10 is a flowchart showing an example of a process for acquiring position information of a camera 3. [Figure 24] 4 is a sequence chart showing an example of a map image creation method according to the present embodiment. [Figure 25] 10 is a sequence chart showing an example of a procedure for linking the position information of an item within an arrangement area to a map of the arrangement area. [Figure 26] FIG. 10 is a diagram showing an example of an item search sequence. [Figure 27] 10 is a sequence chart for searching for an item that is a component part that makes up a parent item from the identification code of a desired item. [Figure 28] 10 is a sequence chart for carrying a new item into a placement area. [Figure 29] 10 is a sequence chart for carrying out an article from a placement area. [Figure 30] 10 is a sequence chart showing an example of a measure to be taken when an identification code of an item that should be found within the placement area is not found. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the present invention will be described in detail below with reference to the drawings.

[0011] (1) First embodiment FIG. 1 is a schematic diagram showing an example of the system configuration of an item search system 100 as an example of a map image creation device according to the first embodiment. The item search system 100 is a system for searching for items that may be placed at random positions in a space. In the first embodiment, an arrangement area 99 is shown as an example of a space, and parts that make up a finished product (hereinafter referred to as "items") are shown as an example of an item. Note that in addition to the arrangement area 99, work areas and passage areas may also exist. In the following description, when there is no need to distinguish between each item 5, they will be simply referred to as items 5.

[0012] In the placement area 99, in the illustrated example, an entrance 100a is provided on the back left side wall for bringing in items 5 as parts that make up products manufactured by the factory's manufacturing process, and in the illustrated example, an exit 100b is provided on the right side of the front side wall for transporting items 5 out of the placement area 99.

[0013] A plurality of racks 10 are arranged on the floor of the placement area 99, and each rack 10 has, for example, four wheels on the underside. This allows each rack 10 to be moved to a random position on the floor of the placement area 99, and therefore, can be arranged in a random position within the placement area 99. Therefore, in the placement area 99, the items 5 that can be placed on each rack 10 can be arranged in a random position. In the following description, when it is not necessary to distinguish each rack 10 from another, they will be simply referred to as rack 10.

[0014] The racks 10 each have a flat shelf or multiple shelves, and at least one type of item 5 can be placed on each shelf. For example, the rack 10a shown in the lower left of the figure has two shelves, and different types of items 5a and 5b are placed on each shelf. A work slip corresponding to each item 5 to be placed is attached to the top surface of each rack 10. An identification code 9 for identifying the type of item 5 to be placed on each rack 10 is printed on each work slip. This identification code 9 is, for example, a barcode such as a QR (Quick Response) code (registered trademark). This makes it possible to determine what type of item 5 is placed on the rack 10 by reading the identification code 9.

[0015] Meanwhile, a plurality of cameras 3, which are an example of an identification information acquisition unit, are fixed to the ceiling of the placement area 99. The cameras 3 photograph the space of the placement area 99 to acquire information about the space of the placement area 99 where the items 5 can be placed, thereby acquiring an identification code 9 as an example of a characteristic of the items 5 that can be randomly placed in the space. Note that instead of the identification code 9, the characteristic of the items 5 may be, for example, the shape of the items 5 themselves.

[0016] When the placement area 99 is divided into a plurality of areas, a camera 3 is provided on the ceiling of the placement area for each area. That is, one camera 3 is provided on the ceiling of each area. By combining the angles of view of the cameras 3, an image can be obtained that covers the entire space of the placement area 99. In the first embodiment, the placement area 99 is managed by dividing it into two areas, left and right, in the orientation shown in the figure, and one camera 3 is provided on the ceiling of the placement area 99 for each area.

[0017] In the first embodiment, an imaging device having a so-called autofocus function or a deep focal depth is used as the camera 3. This allows the camera 3 to reliably read the identification code 9 on the work slip of the item 5 (hereinafter also simply referred to as "the identification code 9 of the item 5") attached to each rack 10 that may be placed at random positions within the placement area 99, even if the height from the floor of the identification code 9 is not always constant.

[0018] Instead of camera 3 taking pictures downward from the ceiling of the placement area 99 in which camera 3 is installed as described above, camera 3 (or movable lamp 7) may be installed facing a vertical surface (e.g., a side wall) within placement area 99, and the identification code 9 of a work slip installed on the side of a vertical rack on which item 5 can be placed may be photographed from the side.

[0019] Furthermore, a movable lamp 7 is provided on the ceiling of the placement area 99 as an example of a notification unit that notifies the location of the desired item 5. At least one movable lamp 7 is provided on the ceiling. The movable lamp 7 includes a lamp unit 7a and a movable unit 7b. The lamp unit 7a has the function of emitting light with a certain degree of directionality.

[0020] The movable unit 7b has a fixed part that is fixed to the ceiling of the placement area 99, and the movable unit 7b that supports the lamp unit 7a is configured to be able to change the orientation of the lamp unit 7a so that it can be tilted, for example, within a range of 180° left and right and 45° up and down, relative to the fixed part. As a result, the movable lamp 7 is configured to be able to irradiate light onto, for example, each rack 10 in accordance with the input attitude control data. This allows a worker to recognize the position of a specific rack 10 within the placement area 99 based on the manner in which light is irradiated by the movable lamp 7.

[0021] In the article search system 100 according to the first embodiment, the above-mentioned movable lamp 7, camera 3, and terminal 6 are connected to one another via a wireless or wired network.

[0022] The terminal 6 is a so-called tablet or computer equipped with a display unit, and in response to an identification code (corresponding to the display of "XXX" in the illustrated example) input by the worker as an example of a characteristic of the desired item 5, the display unit displays the position of the item 5 within the placement area 99 when the position of the item 5 within the placement area 99 is identified by a search, as will be described in detail below. As described above, the characteristic of the item 5 may be the shape of the item 5 instead of the identification code. In this case, the type of item 5 is identified based on the shape of the item 5, and the identification code of the item 5 is determined.

[0023] Here, the worker manually inputs the identification code of the desired item 5, but instead, the code may be input using voice recognition or by reading another type of barcode. When inputting using voice recognition, the input unit 35 is equipped with a voice recognition unit, which receives voice as the characteristics of the item 5 to be input when searching for the position of the item 5 within the placement area 99, and recognizes the received voice as the characteristics of the item 5 using the voice recognition function of this voice recognition unit.

[0024] It should be noted that the method of presenting the search results for the item 5 to the worker is not limited to this, and instead of displaying on the display unit of the terminal 6 described above, for example, a screen display unit 95 may be provided on the side wall of the placement area 99, and the notification unit 37 may display on this screen display unit 95 the position of the item 5 within the placement area 99 searched for by the search unit 36. In this way, the worker can directly grasp the position of the item 5 within the placement area 99 without having to operate the terminal 6.

[0025] 2 is a block diagram showing an example of a simplified representation of the functions of the item search system 100. The item search system 100 includes a control unit 31, an identification information acquisition unit 32, an identification unit 33, a position recognition unit 34, an input unit 35, a search unit 36, a notification unit 37, a memory unit 38, and a calibration unit 39. The control unit 31 is connected to the identification information acquisition unit 32, the identification unit 33, the position recognition unit 34, the input unit 35, the search unit 36, the notification unit 37, the memory unit 38, and the calibration unit 39, and controls the identification information acquisition unit 32, etc. while exchanging data, etc. with these units.

[0026] The storage unit 38 stores the following product configuration table and placement history table. The product configuration table manages the relationship between the product (finished product) manufactured in the manufacturing process in the placement area 99 and the multiple parts that make up that product. On the other hand, the placement history table manages the placement history of the items 5 that have been stored in the placement area 99. The positions of the items 5 that have been recognized in the past by the position recognition unit 34 are written into this placement history table.

[0027] As described above, the identification information acquisition unit 32 has a function of acquiring the identification code 9 of the item 5 that may be placed at a random position within the placement area 99, and one example thereof is the camera 3. The identification information acquisition unit 32 is not limited to the camera 3, and may be a contactless reader or a laser reading device. The identification information acquisition unit 32 may also be an airborne aircraft equipped with the camera 3. This airborne aircraft is an aircraft having at least one propeller, and is equipped with a GPS (Global Positioning System) function so that its position in space within the placement area 99 to which it is moving can be determined.

[0028] Furthermore, when the identification code 9 of the item 5 contains information that can identify the orientation of the identification code 9 itself, the identification unit 33 adjusts the orientation of the identification code 9 to identify the item 5 from the identification code 9 regardless of the orientation of the item 5. Here, it is desirable that the rack 10 on which the item 5 with the identification code 9 is placed is placed in a position that allows the item 5 to be easily photographed by the camera 3 when it is carried into the placement area 99.

[0029] Furthermore, when the identification code 9 of the item 5 contains information that can identify the orientation of the identification code 9 itself, the identification unit 33 adjusts the orientation of the identification code 9 to identify the item 5 from the identification code regardless of the orientation of the item 5. Here, it is desirable that the rack 10 on which the item 5 with the identification code 9 is placed is placed in a position that allows the item 5 to be easily photographed by the camera 3 when it is carried into the placement area 99.

[0030] Under the control of the control unit 31, the position recognition unit 34 assigns planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32, and associates the items 5 repeatedly identified by the identification unit 33 with these planar coordinates at any timing. Specifically, the position recognition unit 34, for example, combines two images captured by the two ceiling cameras 3 described above to create a map of the placement area 99 that covers the entire floor surface of the placement area 99. In this case, the position recognition unit 34 combines the two images so that they overlap each other, thereby creating a map of the placement area 99 that can completely cover the floor surface of the placement area 99. Furthermore, the position recognition unit 34 manages each matrix-like section of the map of the placement area 99 using a single coordinate system. Under the control of the control unit 31, the position recognition unit 34 performs an overlay adjustment process, which will be described later, when overlaying two images.

[0031] The calibration unit 39 associates the planar coordinates assigned by the position recognition unit 34 with the direction in which the notification unit 37 notifies the position of the item 5. The calibration unit 39 calculates the difference in color components of each pixel in the first image and the second image, and if the difference is greater than a specified value, sets the position as the predetermined position (notification position) to be notified by the notification unit 37. For these first and second images, the camera 3, which is an example of the identification information acquisition unit 32, captures the placement area 99 in a state before the notification unit 37 notifies the position of the item 5 as a spot of light on the floor of the placement area 99, and obtains a planar image as an example of the first image, and a planar image as an example of the second image, which is captured by the camera 3 in a state after the notification unit 37 notifies the position of the item 5 as a spot of light SP1 on the floor of the placement area 99.

[0032] Under the control of the control unit 31, the input unit 35 receives an identification code as an example of a characteristic of the desired item 5 in order to search for the position of the item 5 within the placement area 99. The characteristic of the item 5 may be at least one of a character and a symbol attached to the item 5. The input unit 35 receives input of the identification code via a terminal 6 operated by a worker. Based on the received identification code, the input unit 35 searches for the position of the desired item 5 within the placement area 99 from the association result by the position recognition unit 34.

[0033] Under the control of the control unit 31, the search unit 36 ​​searches for the position of the desired item 5 within the placement area 99 from the association result by the position recognition unit 34 based on the identification code received by the input unit 35. Specifically, the search unit 36 ​​searches for the position of the item 5 within the placement area 99 from the association result by the position recognition unit 34 based on the identification code of the item 5 received by the input unit 35, using the above-mentioned planar coordinates as a reference.

[0034] The search unit 36, under the control of the control unit 31, searches for and understands the relationship between a product (finished product) and the multiple parts that make up that product, based on the product configuration table in the storage unit 38, and identifies the component parts that have that product as a parent item. Note that this identification may also be performed by the control unit 31.

[0035] In this embodiment, the memory unit 38 has a placement history table that manages the placement history of items 5 that have been brought into the placement area 99, and if the item 5 cannot be found within the placement area 99 as a result of photographing by the camera 3, the search unit 36 ​​checks the history of the location information of the item 5 from the above-mentioned placement history table.

[0036] The notification unit 37 has a function of optically notifying the worker of the position of the desired item 5 within the placement area 99 in accordance with the search result by the search unit 36. Specifically, the notification unit 37 outputs, via the control unit 31, attitude control data for controlling the attitude of the lamp unit 7a of the movable lamp 7 so that the lamp unit 7a of the movable lamp 7 faces the rack 10 on which the target desired item 5 is placed. This attitude control data also includes lighting control data for the lamp unit 7a to emit light in a predetermined manner (for example, lit or blinking) for a certain period of time. As a result, the lamp unit 7a of the movable lamp 7 emits light from the lamp unit 7a for a certain period of time so as to cover the rack 10 on which the target item 5 is placed.

[0037] In this embodiment, the notification unit 37 may display the results of the search by the search unit 36 ​​for the location of the item 5 within the placement area 99 on the display unit of the portable terminal 6 using augmented reality (AR).

[0038] In addition, in this embodiment, the notification unit 37 may project an image of the spatial information of the placement area 99 acquired by the identification information acquisition unit 32 onto a display unit (not shown) using, for example, projection mapping.

[0039] Furthermore, in this embodiment, the notification unit 37 may be configured to have a guidance unit that guides the position of the item 5 within the placement area 99 by using at least one of sound and voice.

[0040] FIG. 3 is a plan view showing an example of a simplified map of the placement area 99. The map of the placement area 99 is made up of the image capture areas 33a and 33b captured by the two cameras 3 described above. As indicated by the two-dot chain line, the image capture areas 33a and 33b are configured so that their adjacent areas overlap, and the angle of view of the two cameras 3 is configured to be larger than the image capture area so that no items 5 are missed. In other words, the angles of view of the two cameras 3 overlap so that no gaps are left between them. As a result, images of the placement area 99 captured using the two cameras 3 are stitched together to form a single image of the placement area 99.

[0041] In the illustrated example, a new rack 10 carrying an item 5 can be carried in through an inlet 100a of the placement area 99, and a rack 10 in the placement area 99 can be carried out through an outlet 100b. When a rack 10 is carried in or out in this way, the camera 3 also captures an image of the item 5 on the rack 10 being carried in or out, as shown by the two-dot chain line.

[0042] Fig. 4 is a schematic diagram showing an example of managing the map of the placement area 99 shown in Fig. 3 using a matrix coordinate system. The method of managing the coordinate system is performed by the position recognition unit 34 described above. The position recognition unit 34 manages the map of the placement area 99 using a single coordinate system with a large number of matrix-shaped rectangular sections corresponding to the map of the placement area 99. The position recognition unit 34 joins together images captured by the two cameras 3 described above to create a single coordinate system.

[0043] First, as described above, the item search system 100 includes at least an identification information acquisition unit 32, an identification unit 33, and a position recognition unit 34. The identification information acquisition unit 32 acquires spatial information of the placement area 99 in which the item 5 can be placed, thereby acquiring characteristics of the item 5 that can be randomly placed within the space. The identification unit 33 identifies the item 5 from the characteristics of the item 5 acquired by the identification information acquisition unit 32. The position recognition unit 34 assigns planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32, and repeatedly associates the item 5 identified by the identification unit 33 with the planar coordinates at any timing.

[0044] Furthermore, the identification information acquisition unit 32 has, for example, a camera 3 as an example of one imaging unit and an actuator (not shown) that changes the shooting direction of the camera 3, and while changing the shooting direction of the camera 3, i.e., its shooting position, using the actuator, changes the zoom magnification according to the distance from the camera 3 to the shooting position, and captures multiple subspace images 201 as examples of multiple enlarged images.

[0045] The identification unit 33 connects multiple partial space images 201 so that parts of the images overlap, to create a map image as an example of an image of one placement area 99 or a map image as an example of an image of part of the placement area 99.

[0046] When joining the images so that the overlap areas B (an example of a part of an image) overlap, the recognition unit 33 makes the overlap width of the overlap areas B larger than the maximum outer dimension of the article 5.

[0047] More specifically, as part of this adjustment process during overlapping, when the two images P1 and P2 are overlapped, the identification unit 33 makes the width of the overlap area B of the images shown in Figure 5 larger than the length of the diagonal of the rectangle circumscribing the identification code 9 to be searched.

[0048] Specifically, the recognition unit 33 sets the width of the overlap area B to, for example, twice the length of the long side of the circumscribing rectangle. By doing so, even if the identification code 9 is located at the joint between two images P1 and P2 as shown in Fig. 5, the entire image of the identification code 9 is included in either image P1 or image P2 shown in Fig. 5, and the image of the identification code 9 itself is not divided. This allows the recognition unit 33 to correctly recognize the identification code 9.

[0049] After identifying the article 5, the recognition unit 33 overlaps the overlap area B, which is an example of a part of the image, to join the images together.

[0050] When combining two images P1 and P2, the recognition unit 33 overlays the images so that the image containing the entire identification code 9 is on top. Image P1 contains a portion of the image of the identification code 9 that is separated by a seam. When overlaying image P1 on image P2, the recognition unit 33 must seamlessly join the portion of the image of the identification code 9 contained in image P1 with the portion of the image of the identification code 9 contained in image P2, which requires highly accurate alignment. In this embodiment, the recognition unit 33 overlays image P2, which contains the entire image of the identification code 9, on top, thereby eliminating the need for highly accurate alignment. This image combining method is also applicable when combining multiple images P1 and P2 captured using a single camera 3 while changing the zoom magnification.

[0051] As shown in Figure 6, if multiple identification codes 9, 9a are each partially contained in the overlap area B of the images, when two images P1, P2 are stitched together and then the identification codes 9, 9a are recognized, regardless of whether image P1 or image P2 is overlaid on top when combining the images, it will be necessary to stitch together the image of one of the identification codes 9, 9a, which will require highly accurate alignment.

[0052] When such alignment is performed, it is conceivable that a slight misalignment will occur, as shown in Fig. 7, making it impossible to correctly recognize the identification code 9. Therefore, in this embodiment, assuming that a plurality of identification codes 9, 9a are included in the overlap area B of the images as shown in Fig. 6, the position recognition unit 34 recognizes the identification codes 9, 9a for each of the images P1 and P2 before combining the images, obtains position information for the identification codes 9, 9a based on the recognition result, and then combines the two images P1 and P2 so that the positions of these identification codes 9, 9a match in the images P1 and P2.

[0053] As shown in Figure 8, when one identification code 9 is completely contained within the overlap area B of the images, the position recognition unit 34 acquires the position information of the identification code 9 in image P1, and then similarly acquires the position information of the identification code 9 in image P2, thereby detecting one overlapping identification code 9.

[0054] In this embodiment, the identification code 9 corresponding to the item 5 is associated with unique identification information, which will be described later. Specifically, in this embodiment, in order to prevent the duplicate registration of the identification code 9 in the storage unit 38, identification information such as the product number, order number, and individual identification number is recorded in the identification information of the identification code 9, and if the result of recognizing the identification code 9 is the same as previously recorded information, the result is overwritten with the identification information of the identification code 9 contained in the subspace image 201 captured later, thereby preventing duplicate registration of the identification information of the identification code 9.

[0055] When multiple subspace images 201, which are an example of multiple enlarged images, contain images of an item 5 having the same identification information, the position recognition unit 34 overwrites the identification information of the subspace image 201 captured earlier with the identification information of the subspace image 201 captured later, or retains the identification information of the subspace image 201 captured earlier and discards the identification information of the subspace image 201 captured later.

[0056] Hereinafter, the control unit 31 causes one camera 3, which is an example of the identification information acquisition unit 32, to capture a plurality of fragmentary images while changing the shooting direction of the camera 3. The position recognition unit 34 combines the fragmentary images to create one map image. This will be explained in detail below.

[0057] 10, the control unit 31 changes the shooting direction of the camera 3, for example, in a manner similar to drawing a circle with a compass, and continuously shoots images so that the angles of view of the camera 3 overlap each other to obtain a large number of fragmentary images. When shooting with the camera 3 while changing the shooting direction of the camera 3 in the radial direction (θ direction) and circumferential direction (φ direction), the control unit 31 causes the seams between each fragmentary image to overlap.

[0058] The illustrated example shows that fragmentary images 201a, 201b, and 201c are acquired in an xy plane including the x and y axes, arranged in a circular pattern in triplicate from the inside to the outside, centered on the origin. At the innermost position closest to the origin, fragmentary image 201c is arranged in a circular pattern. Outside fragmentary image 301c, a large number of fragmentary images 201b are arranged in a circular pattern. Outside fragmentary image 301b, a large number of fragmentary images 201a are arranged in a circular pattern.

[0059] The position recognition unit 34 cuts out, for example, a rectangular partial space image 201 centered on the origin from the numerous fragmentary images 201a, 201b, and 201c arranged in a circular pattern, and stores the cutout partial space image 201 in the storage unit 38. Note that if the partial space images 201 are arranged, for example, in two columns and seven columns with some overlap as shown in Fig. 9, they form a single joined map image.

[0060] As shown in Fig. 11 to Fig. 14 by the angle of view of a rectangle including the identification code 9, the orientation of the acquired fragmentary image of the partial space image 201 changes depending on the shooting direction of the camera 3. However, the saved partial space image 201 is saved in a pixel coordinate system with the upper left corner of the pixel coordinate system as the origin shown in Fig. 12, with the x-axis and y-axis of the pixel coordinate system being parallel to the orthogonal sides of the rectangle. Note that the pixel coordinate system refers to a coordinate system according to the orientation of the image.

[0061] The identification unit 33 finds the position of the identification code 9 from the partial space image 201 shown in Fig. 12 that has been saved in this way. The identification unit 33 rotates the partial space image 201, which is tilted as shown in Fig. 13, by 90°-θ around the origin of the pixel coordinates to return the angle of view to its original orientation as shown in Fig. 12.

[0062] Furthermore, the identification unit 33 obtains the world coordinates of the identification code 9 by translating the distance (a0, b0) so that it matches the coordinate system shown in FIG. 14 (hereinafter also referred to as the "world coordinate system"). Note that the world coordinate system refers to an absolute coordinate system regardless of the orientation of the image. The identification unit 33 repeats the above process for all captured images. This allows the identification unit 33 to obtain the position information of the identification code 9 for all fragmentary images captured by one camera 3.

[0063] In this embodiment, the method of stitching together multiple fragmentary images is also similar. As shown in Fig. 11, the identification unit 33 can determine the world coordinates (a0, b0), (a1, b1), (a2, b2), and (a3, b3) of the vertices of the shooting range of the camera 3 from the tilt angle θ, pan angle φ, and zoom magnification of the camera 3 when the multiple fragmentary images were captured.

[0064] When the identification unit 33 completes the recognition process of the identification codes 9 of the multiple fragmentary images, it creates a subspace image 201 by stitching together the multiple fragmentary images based on the world coordinates calculated as described above. The origin (0,0) of the subspace image 201 is at the top left as shown in FIG. 15. The identification unit 33 sets the center of the first captured fragmentary image as the center of the subspace image 201, and stitches together the multiple fragmentary images 201c by overlapping parts of the captured multiple fragmentary images 201c in order from that center. In doing so, the identification unit 33 overlaps the multiple fragmentary images 201c so that the fragmentary images 201c captured later are on top.

[0065] At this time, the image of the identification code 9 is positioned at the joint. If only a part of the identification code 9 is shown in the fragmentary image 201c that is overlaid later, misalignment may occur at the joint when the images are pasted together, as shown in Fig. 7, but the position of the identification code 9 has already been identified, so a slight misalignment of the images does not appear to be a problem.

[0066] The identification unit 33 connects all the fragmentary images in this way, and when a circular image as shown in FIG. 16 is completed, cuts out a partial space image 201 of the required angle of view from the circular image.

[0067] After creating the subspace images 201 for all the cameras 3, the identification unit 33 stitches together, for example, four subspace images 201, as shown in Fig. 17. At this time, the identification unit 33 performs the stitching while taking into consideration, for example, the position information of the cameras 3. Note that, although the multiple subspace images 201 are originally arranged in rows and columns as shown in Fig. 9 above, the following description will be simplified to assume that four subspace images 201 are arranged in two rows and two columns.

[0068] To obtain the position information of the camera 3, distances and the like may be actually measured using a measuring instrument, etc., but in this embodiment, in order to reduce the workload when installing on-site, a method is used in which an identification code 9 for alignment is photographed by each camera 3, and the identification unit 33 aligns the coordinate positions of the corners of the identification code 9 contained in each captured image, thereby determining the relative position of the camera 3.

[0069] For example, as shown in Fig. 17, when photographing is performed using four cameras 3 installed on the ceiling of the placement area 99, the identification code 9 for positioning is placed on the floor of the placement area 99 so that the identification code 9 is captured in the corner of the field of view of each camera 3. The subspace images 201 photographed by each camera 3 are as shown in Figs. 18(A) to 18(D), and the identification code 9 is recorded in a different position in each image.

[0070] For example, in the fragmentary image 201a shown in Figure 18(A), the upper left corner of the positioning identification code 9 is located at the coordinates (750,550) in the lower right. In the fragmentary image 201b shown in Figure 18(B), the upper left corner of the positioning identification code 9 is located at the coordinates (50,560) in the lower left. In the fragmentary image 201c shown in Figure 18(C), the upper left corner of the positioning identification code 9 is located at the coordinates (740,40) in the upper right. In the fragmentary image 201d shown in Figure 18(D), the upper left corner of the positioning identification code 9 is located at the coordinates (60,50) in the upper left.

[0071] The identification unit 33 recognizes the identification code 9 for each of the fragmentary images 201a to 201d in Figures 18(A) to 18(D) and obtains position information for this identification code 9. Using the central coordinates (m0, n0) of the fragmentary image 201a in Figure 18(A) as a reference, the identification unit 33 determines the central coordinates (m1, n1), (m2, n2), and (m3, n3) of the remaining three fragmentary images 201b to 201d shown in Figures 18(N) to 18(D).

[0072] Specifically, the identification unit 33 calculates the difference between the position information of the identification code 9 shown in Fig. 18(A) and the position information of each of the identification codes 9 shown in Fig. 18(B), 18(C), and 18(D), and adds the difference to the center coordinates (m0, n0) of the image in Fig. 18(A) to obtain the center coordinates (m1, n1), (m2, n2), and (m3, n3) of each of the fragmentary images 201a-201d in Fig. 18(B), 18(C), and 18(D). In other words, the identification unit 33 calculates the mounting positions (m1, n1), (m2, n2), and (m3, n3) of the cameras 3 (for example, cameras 3a, 3b, 3c, and 3d in the case of four cameras). When the four fragmentary images 201a-201d are superimposed, the identification codes 9 are located at the four corners, as shown in Fig. 19.

[0073] Based on this position information, the identification unit 33 translates the fragmentary images 201b to 201d in Figures 18(B), 18(C), and 18(D) to the positions shown in Figures 20(B), 20(C), and 20(D), respectively, to create a single map image. Note that the fragmentary image 201a shown in Figure 18(A) is not subject to translation because its position is not changed. Note that in this embodiment, the fragmentary images 201a to 201d are also collectively referred to as subspace images 201.

[0074] Fig. 21 is a diagram showing an example of the imaging range captured by camera 3 installed on the ceiling of placement area 99, expressed in a pixel coordinate system, and Fig. 22 is a diagram showing an example of the imaging range of camera 3 expressed in world coordinates. Fig. 22 shows an example of an actual image captured obliquely by camera 3, and Fig. 21 shows an example of the actual image shown in Fig. 22 corrected to a frontal view, as if the image were captured by camera 3 pointing directly downward.

[0075] In this embodiment, the recognition unit 33 corrects distortion in an image of the item 5 in the pixel coordinate system captured by one camera 3, and recognizes the item 5 based on the image in the world coordinate system after the distortion has been corrected.

[0076] The outline of the configuration of item search system 100 has been described above, and the following describes a map image creation method as an example of the operation of item search system 100. Below, details of the camera position information acquisition process for acquiring position information of camera 3 and the map image creation process for creating a map image, which are included in the map image creation method, will be described.

[0077] 23 is a flowchart showing an example of a process for acquiring position information of the cameras 3. In step S801, after an installation worker installs multiple cameras 3 within the installation area 99, the installation worker places an identification code 9 for alignment in a specific position where multiple cameras 3 installed on the ceiling can simultaneously capture images. At this time, the installation worker places the identification code 9 in a more suitable specific position while viewing images captured by each camera 3, for example, using a tablet or the like having a function for displaying images from each camera 3. Once the identification code 9 has been placed, in step S802, the installation worker operates the input unit 35 to start the calibration process.

[0078] In step S803, input unit 35 outputs to control unit 31 an instruction to perform calibration processing by calibration unit 39. In step S804, control unit 31 outputs an instruction to change the zoom magnification, tilt angle (radial direction), and pan angle (circumferential direction) to multiple cameras 3 as an example of multiple identification information acquisition units 32. In step S805, the multiple cameras 3 change their zoom magnification, tilt angle (radial direction), and pan angle (circumferential direction) so that the identification code 9 falls within their angles of view.

[0079] In step S806, the control unit 31 outputs an image acquisition instruction to the multiple cameras 3. In step S807, the multiple cameras 3 each acquire an image of the identification code 9. In step S808, the identification unit 33 recognizes the identification code 9 in each acquired image and acquires position information of the identification code 9 in each image.

[0080] In step S809, the identification unit 33 performs at least one of rotational movement and translational movement on the acquired image based on the zoom magnification, tilt angle (radial direction), and pan angle (circumferential direction), to convert the image into a world coordinate system.

[0081] In step S810, the calibration unit 39 calculates the amount of translation of each image so that the positions of the identification codes 9 are aligned with each other. This amount of translation represents the relative position of each camera 3. In step S811, the storage unit 38 stores the position information of the camera 3.

[0082] 24 is a sequence chart showing an example of a map image creation method according to this embodiment. In this map image creation method, a map image is created based on the position information of the camera 3 acquired as described above. First, an overview of the map image creation method will be described. The map image creation method includes an identification information acquisition step in which an identification information acquisition unit 32 acquires spatial information of a placement area 99 in which an item 5 can be placed, thereby acquiring characteristics of the item 5 that can be randomly placed within the space; an identification step in which an identification unit 33 identifies the item 5 from the characteristics of the item 5 acquired by the identification information acquisition unit 32; and a position recognition step in which a position recognition unit 34 assigns planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32, and repeatedly associates the item 5 identified by the identification unit 33 with the planar coordinates at any timing. In the above-mentioned identification information acquisition step, the identification information acquisition unit 32 having the camera 3 as an example of one imaging unit changes the shooting position of the camera 3 while changing the zoom magnification according to the distance from the camera 3 to the shooting position, and captures a plurality of subspace images 201 as an example of a plurality of enlarged images, and in the identification step, the identification unit 33 connects the plurality of subspace images 201 so that parts of the images overlap, to create a map image as an example of an image of one placement area 99 or a map image as an example of an image of part of the placement area 99.

[0083] In the map image creation method, first, a process of continuously capturing fragmentary images is executed. In the continuous capturing process, the following steps S801 to S905 are executed.

[0084] First, in step S901, the control unit 31 outputs instructions to change the zoom magnification, tilt angle (radial direction), and pan angle (circumferential direction) to the multiple cameras 3. In step S902, the multiple cameras 3 change, for example, the zoom magnification, tilt angle (radial direction), and pan angle (circumferential direction) of the cameras 3.

[0085] In step S903, the control unit 31 outputs an instruction to acquire an image to the multiple cameras 3 as an example of the multiple identification information acquisition units 32. In step S904, the multiple cameras 3 acquire an image including the identification code 9 using the zoom ratio or the like changed as described above. The control unit 31 and each camera 3 repeat the above steps S901 to S904 until all images have been acquired (step S905).

[0086] In step S906, each time each camera 3 captures an image, the identification unit 33 executes a recognition process for the identification code 9 based on the image. Furthermore, the identification unit 33 refers to the position information of the identification code 9 stored in the storage unit 38 as described above, and acquires the position information of the identification code 9 in the captured image.

[0087] In step S907, the position recognition unit 34 performs at least one of rotational and parallel translation of the coordinates of the identification code 9 and the image based on the zoom magnification, tilt angle (radial direction), and pan angle (circumferential direction) when each camera 3 acquired the image, and converts them into the coordinate system (also called "world coordinates") of the map image to be created.

[0088] In step S908, the position recognition unit 34 connects the images that have been subjected to at least one of rotational movement and translational movement to create multiple subspace images 201. In step S909, the control unit 31 outputs an instruction to connect the multiple subspace images 201 created by the position recognition unit 34 so that the positions of the identification codes 9 match to create a map image.

[0089] In step S910, the position recognition unit 34 stitches together multiple subspace images 201 based on the position information of the camera 3 stored in the storage unit 38 in step S810 shown in Fig. 23 to create a map image that is an image of the entire placement area 99. In step S911, the position recognition unit 34 saves the map image in the storage unit 38.

[0090] 25 is a sequence chart showing an example of a procedure for linking the position information of an item 5 within a placement area 99 to a map of the placement area 99. In step S101, the control unit 31 controls two cameras to capture images of the placement area 99 using the two cameras 3, respectively. In step S102, the position recognition unit 34 stitches together the two images of the placement area 99 to create a map of the placement area 99. In step S103, the control unit 31 assigns the above-mentioned coordinate system to the generated map.

[0091] Meanwhile, in step S104, the worker carries the item 5 into any position in the placement area 99 from the entrance 100a and places the item 5 in a position where the identification code 9 attached to the work slip for the item 5 can be photographed by the camera 3.

[0092] Next, in step S105, the control unit 31 outputs an instruction to the camera 3 to read the identification code 9 of the item 5 at an arbitrary timing. In step S106, the camera 3 reads the identification code of the item 5. Note that the method of identifying the item 5 is not limited to the above-mentioned method, and methods using characters or symbols attached to the item 5, the shape of the item 5 itself, or a non-contact tag (RFID: Radio Frequency IDentification) may also be used. In step S107, the control unit 31 causes the position recognition unit 34 to link the position information within the placement area 99 relating to the identification code 9 read by the camera 3 to the map. Linking here means managing the map in the coordinate system.

[0093] On the other hand, in step S108, the worker moves the position of the rack 10 on which the item 5 is mounted within the placement area 99.

[0094] Next, in step S109, the control unit 31 outputs an instruction to the camera 3 to read the identification code 9 of the item 5 at an arbitrary timing. In step S110, the camera 3 reads the identification code 9 within the placement area 99. Note that in step S110, the size of the identification code itself may be increased so that it can be read, or the placement location of the item 5 may be moved to a position visible to the camera 3. Note that in steps S109 to S110, since there is a high possibility that a worker is involved when the position of the item 5 on the rack 10 changes, the presence of a worker may be recognized and the movement of the identification code within a certain range from the worker may be tracked. Also, in steps S109 to S110, reading of the identification code may be stopped outside of business hours, such as at night. Next, in step S111, the control unit 31 links the location information within the placement area 99 for the identification code 9 read by the camera 3 to the map.

[0095] FIG. 26 is a diagram showing an example of a search sequence for an article 5. In step S201, the worker inputs the identification code of the desired article 5 from the terminal 6. In step S202, the control unit 31 compares the input identification code with the identification code 9 on the map. In step S203, the control unit 31 outputs attitude control data to the notification unit 37 for the movable lamp 7 corresponding to the coordinates of the article on the map whose identification code matches. This attitude control data includes control information for setting a predetermined rotation angle left and right and up and down.

[0096] In step S204, the notification unit 37 outputs attitude control data to the movable lamp 7. The movable lamp 7 then controls the movable unit 7b in accordance with the input attitude control data to control the attitude of the lamp unit 7a so that it is oriented at a predetermined rotation angle left and right and up and down, and causes the lamp unit 7a to emit light for a certain period of time. In the first embodiment, when emitting light in this manner, in order to improve visibility for workers, methods such as changing the color of the lit light or flashing the light may be adopted, rather than simply turning it on. In the first embodiment, if the same type of desired item 5 is present on multiple racks 10, the color of the light may be changed depending on the purpose.

[0097] Meanwhile, in step S205, the worker finds the desired item 5 using the light emitted from the lamp portion 7a of the movable lamp 7, and moves the desired item 5 from the placement area 99 to the work area. The work area is where assembly is performed using the moved desired item 5.

[0098] In step S206, the control unit 31 outputs an instruction to the notification unit 37 to turn off the lamp unit 7a of the movable lamp 7 after a predetermined input or after a certain time has elapsed. In step S207, the notification unit 37 outputs the instruction to turn off the movable lamp 7, turning off the lamp unit 7a.

[0099] The method of notifying the position of the article 5 within the placement area 99 is not limited to the spotlight method using the movable lamp 7 described above, but may include shining a laser at the article 5 with a pinpoint, suggesting the position as a silhouette of the article using light, displaying a map including the position of the article 5 on a tablet, smartphone, or monitor as an example of the terminal 6, superimposing the article 5 on augmented reality (AR), displaying the position of the article 5 in cooperation with a global positioning system (GPS), or simply displaying it on a screen. Furthermore, methods of notifying the position of the article 5 within the placement area 99 may include projecting the position of the article 5 within the placement area 99 using so-called projection mapping, indicating the position of the article 5 using a beacon, suggesting the position of the article 5 using a so-called patrol lamp (registered trademark) or searchlight, or guiding the position of the article 5 using sound or voice.

[0100] In the first embodiment, if the identification codes 9 are not recognized due to unintentional overlapping of the identification codes 9 or the like, and the control unit 31 determines that the item 5 is not present in the placement area 99 in the latest history, the movable lamp 7 may be configured to emit light according to the position information of the item 5 going back to the time when the desired item 5 was last present in the placement area 99. In this way, the worker can identify the location when searching for the desired item 5.

[0101] 27 is a sequence chart for searching for an item that is a component part that makes up a parent item from the identification code of a desired item 5. In step S301, an operator inputs, from terminal 6, the identification code of a finished product that is a parent item made up of a combination of multiple component parts (items 5). In step S302, control unit 31 compares the part numbers of items 5 that are component parts that make up the parent item with the identification codes 9 on the map.

[0102] In step S303, the control unit 31 controls the notification unit 37 to output attitude control data to all movable lamps 7 corresponding to the map coordinates that match for all items of goods 5. In step S304, the notification unit 37 outputs the input attitude control data and turns on or blinks the movable lamps 7 for all items of goods 5. In step S304, the notification unit 37 may notify multiple items 5 at once, or may notify multiple items 5 one by one by shining a light on them in order of proximity to the position of the search unit 36 ​​(or terminal 6), whose position relative to the placement area 99 is preset, or may notify multiple items 5 in a preset order.

[0103] Meanwhile, in step S305, the worker searches for all of the above-mentioned items 5 using the guidance of the light emitted from the movable lamp 7 and moves them from the placement area 99 to the work area.

[0104] In step S306, the control unit 31 controls the notification unit 37 so that the movable lamp 7 is turned off after a certain time has elapsed. In step S307, the notification unit 37 controls the movable lamp 7 to be turned off.

[0105] 28 is a sequence chart for newly carrying in an item 5 into the placement area 99. In step S401, a worker passes through the entrance 100a of the placement area 99 while moving a rack 10 in a state where the identification code 9 attached to the work slip of the item 5 can be photographed by the camera 3.

[0106] In step S402, the identification information acquisition unit 32 controls the camera 3 capable of photographing the entrance 100a of the placement area 99 to read the identification code 9 on the work slip for the item 5. In step S403, the control unit 31 recognizes that the item 5 corresponding to the read identification code 9 has been brought into the placement area 99. In step S404, the notification unit 37 notifies the worker that the reading at the time of delivery has been completed by turning on the movable lamp 7.

[0107] 29 is a sequence chart for carrying out an item from the placement area 99. In step S01, a worker places the item 5 on the rack 10 in a position where it can be photographed by the camera 3, and passes through the exit 100b of the placement area 99.

[0108] In step S502, the identification information acquisition unit 32 reads the identification code of the item 5 using the camera 3 capable of photographing the exit 100b of the placement area 99. In step S503, the control unit 31 recognizes that the item with the read identification code 9 has been removed from the placement area 99. In step S504, the notification unit 37 turns on the movable lamp 7 to notify the worker that the reading at the time of removal has been completed.

[0109] In the first embodiment, as described above, in addition to lighting or flashing the movable lamp 7 described above, methods of notifying workers that an item 5 has been brought into (stored in) or taken out (dispatched from) the placement area 99 may also be used, for example, to display a map on a terminal 6 such as a tablet, smartphone, or monitor, to link with virtual reality (AR: Augmented Reality) or GPS (Global Positioning System), to display on a screen, to display using projection mapping, to notify with a patrol lamp (registered trademark), or to notify with sound or voice.

[0110] 30 is a sequence chart showing an example of a measure to be taken when the identification code 9 of an item 5 that should have been found in the placement area 99 is not found. In step S601, the worker inputs the identification code of the desired item 5 from the terminal 6. In step S602, the control unit 31 compares the identification code 9 of the item 5 that is a component part that makes up the parent item with the identification code 9 on the map.

[0111] In step S603, the control unit 31 determines whether or not the item 5 of the specified part exists in the placement area 99, and if it does not exist, executes step S604, but if it does exist, skips step S604.

[0112] In step S604, the control unit 31 refers to the placement history in the placement history table (not shown) described above for the item 5, determines whether or not there is a placement history, and if there is a placement history, executes step S605, whereas if there is no placement history, executes step S606.

[0113] In step S605, the notification unit 37 notifies the worker that the item has been taken out by lighting the movable lamp 7. In step S606, the control unit 31 goes back to the most recent history of recognition of the target item 5 and outputs attitude control data to the movable lamp 7 corresponding to the coordinates of the item 5 within the placement area 99. In step S607, the notification unit 37 moves the movable lamp 7 in accordance with the input attitude control data and lights or flashes it. At this time, if the movable lamp 7 is irradiating light onto multiple items 5, the light may be irradiated onto each item 5 in any order or a predetermined order or cycle, or the light may be irradiated onto each item 5 in order of proximity to the control unit 31, whose installation position has been determined in advance.

[0114] In step S608, the control unit 31 outputs, after a certain time has elapsed, an instruction to turn off the movable lamp 7. In step S609, the notification unit 37 turns off the movable lamp 7.

[0115] Meanwhile, in step S610, the worker searches for the desired item 5 using the guidance of the light emitted from the movable lamp 7 and moves it from the placement area 99 to the work area.

[0116] As described above, the item search system 100 of this embodiment comprises an identification information acquisition unit 32 that acquires characteristics of items 5 that may be randomly placed in space by acquiring spatial information of the placement area where items may be placed; an identification unit 33 that identifies items from the item characteristics acquired by the identification information acquisition unit 32; a position recognition unit 34 that assigns planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32 and repeatedly associates items identified by the identification unit 33 with the planar coordinates at any timing; an input unit 35 that accepts the characteristics of the desired item 5 in order to search for its position within the placement area 99; a search unit 36 ​​that searches for the position of the desired item 5 within the placement area 99 from the position recognition results by the position recognition unit 34 based on the characteristics of the desired item 5 input via the input unit 35; and a notification unit 37 that notifies the position of the desired item 5 searched for by the search unit 36.

[0117] With this configuration, the identification information acquisition unit 32 acquires spatial information of the placement area 99 where items can be placed, and thereby acquires characteristics of the items 5 that can be randomly placed within the space. The identification unit 33 identifies the items 5 from the characteristics of the items 5 acquired by the identification information acquisition unit 32. The position recognition unit 34 assigns planar coordinates to the spatial information of the placement area 99, and repeatedly associates the items 5 identified by the identification unit 33 with the planar coordinates at any timing. The search unit 36 ​​searches for the position of the desired item within the placement area 99 based on the characteristics of the desired item 5 input via the input unit 35. The notification unit 37 notifies the user of the position of the desired item searched for by the search unit 36.

[0118] In this embodiment, the identification information acquisition unit 32 is an airborne flying object equipped with a camera 3 as an example of an imaging unit. In this way, the identification information acquisition unit 32 can more reliably acquire the identification code of the article 5 by changing its position and orientation within the placement area 99.

[0119] In this embodiment, the identification unit 33 identifies the article 5 from the characteristics of the article 5 regardless of the orientation of the article 5. In this way, the identification information acquisition unit 32 can more reliably acquire the identification code of the article 5 that may be placed in a random position and orientation within the placement area 99.

[0120] In this embodiment, the feature of the item 5 is an identification code that allows the item 5 to be distinguished from other items, and when the item 5 is carried into the placement area 99, the item 5 with the identification code is placed in a position that makes it easy to acquire by the identification information acquisition unit 32. In this way, the identification code of the item 5 can be read more reliably.

[0121] In this embodiment, the memory unit 38 has a placement history table (not shown) that manages the placement history of items 5 that have been brought into the placement area 99, and if the identification information acquisition unit 32 cannot acquire the item 5 within the placement area 99, the search unit 36 ​​checks the history of the location information of the item 5 from this placement history table. In this way, even if the search unit 36 ​​cannot find the desired item 5 within the placement area 99, the worker can be notified of the location where the item 5 was most recently placed, making it easier to find the desired item 5 within the placement area 99.

[0122] In this embodiment, the notification unit 37 optically notifies the location of the item 5 within the placement area 99. In this way, it becomes easier for the worker to visually find the desired item 5 within the placement area 99.

[0123] In this embodiment, the notification unit 37 may be configured to display the position of the desired item 5 in the placement area 99, which has been searched for by the search unit 36, on a screen display unit 95 provided on a side wall within the placement area 99. In this way, it is not necessary to provide a dedicated terminal for displaying the position of the item 5 in the placement area 99, and the worker can visually grasp the position of the desired item 5 in the placement area 99 even if he or she does not have the terminal 6 in his or her hand.

[0124] In this embodiment, the notification unit 37 uses augmented reality to display the results of the search performed by the search unit 36 ​​to find the location of the item 5 within the placement area 99 on the display unit of the portable terminal 6. In this way, the worker can easily grasp the location of the item 5 using augmented reality, which is easy for the worker to grasp intuitively.

[0125] In this embodiment, the notification unit 37 projects an image of the spatial information of the placement area 99 acquired by the identification information acquisition unit 32 onto the display unit using, for example, projection mapping.

[0126] In this embodiment, the notification unit 37 has a guidance unit that uses at least one of sound and voice to guide the worker as to the location of the item 5 within the placement area 99. In this way, the worker can auditorily grasp the location of the desired item 5 within the placement area 99.

[0127] In this embodiment, the identification information acquisition unit 32 stitches together images of the placement area 99 taken by the multiple cameras 3 to create a single image of the placement area 99. In this way, the space of the placement area 99 can be accurately covered without any omissions.

[0128] In this embodiment, the input unit 35 has a voice recognition unit that receives voice as a characteristic of the item 5 to be input when searching for the position of the item 5 within the placement area 99, and uses a voice recognition function to recognize the received voice as a characteristic of the item 5. In this way, the worker does not need to manually input the identification code of the desired item 5, and the effort required to search for the position of the item 5 can be reduced.

[0129] The item search system 100 according to this embodiment includes at least an identification information acquisition unit 32, an identification unit 33, and a position recognition unit 34. The identification information acquisition unit 32 acquires spatial information of the placement area 99 in which the item 5 can be placed, thereby acquiring characteristics of the item 5 that can be randomly placed within the space. The identification unit 33 identifies the item 5 from the characteristics of the item 5 acquired by the identification information acquisition unit 32. The position recognition unit 34 assigns planar coordinates to the spatial information of the placement area 99 acquired by the identification information acquisition unit 32, and repeatedly associates the item 5 identified by the identification unit 33 with the planar coordinates at any timing.

[0130] Furthermore, the identification information acquisition unit 32 has a camera 3 as an example of one imaging unit, and while changing the shooting position of the camera 3, changes the zoom magnification according to the distance from the camera 3 to the shooting position to shoot a plurality of subspace images 201 as an example of a plurality of enlarged images. The identification unit 33 stitches the plurality of subspace images 201 together so that parts of the images overlap, to create a map image as an example of an image of one placement area 99 or a map image as an example of an image of part of the placement area 99.

[0131] In this way, a single map image can be created using multiple inexpensive cameras, each with a narrow angle of view.

[0132] In this embodiment, when stitching together images so that overlapping areas B (an example of a part of an image) overlap, the recognition unit 33 makes the overlapping width of the overlapping area B of the images shown in Fig. 5 larger than the maximum outer dimension of the article 5. This makes it possible to prevent the image of the article 5 from being cut off midway.

[0133] In this embodiment, when stitching images together so that the images partially overlap, the recognition unit 33 makes the overlap width of the overlap area B of the images shown in Fig. 5 larger than the maximum outer dimension of the article 5. This makes it possible to prevent the image of the article 5 from being cut off midway.

[0134] In this embodiment, after identifying the article 5, the recognition unit 33 joins the images by overlapping the overlap area B, which is an example of a part of the image. In this way, the recognition unit 33 can read the recognition code 9 of the article 5 even if the recognition code 9 is located at the seam of the images.

[0135] In this embodiment, the identification code 9 corresponding to the item 5 is associated with unique identification information, which will be described later. When multiple subspace images 201, which are an example of multiple enlarged images, contain images of the item 5 having the same identification information, the position recognition unit 34 overwrites the identification information of the subspace image 201 captured earlier with the identification information of the subspace image 201 captured later, or retains the identification information of the subspace image 201 captured earlier and discards the identification information of the subspace image 201 captured later. In this way, if the result of recognizing the identification code 9 is the same as the identification information recorded previously, it is possible to prevent duplicate registration of the identification information of the identification code 9 by overwriting it with the identification information of the identification code 9 included in the subspace image 201 captured later.

[0136] In this embodiment, the recognition unit 33 corrects distortion in an image of the article 5 captured by one camera 3, and identifies the article 5 based on the distortion-corrected image. In this way, the recognition accuracy of at least one of the article 5 and the identification code 9 can be improved.

[0137] (2) Second embodiment The item search system according to the second embodiment has almost the same configuration and operation as the item search system 100 according to the first embodiment, so a description of the similar configuration and operation will be omitted and the following description will focus on the differences.

[0138] The item search system according to the second embodiment includes a placement area 99a corresponding to the placement area 99 of the item search system 100 according to the first embodiment, and a passage area 99b which is a space continuous with the placement area 99a.

[0139] Dedicated rails 98, which are an example of guide units, are laid on the ceilings of the placement area 99a and the passage area 99b so as to cover the entire floor surface of the placement area 99a and the passage area 99b. A placement area dedicated rail 98a is installed on the ceiling of the placement area 99a, and a passage area dedicated extension rail 98b is installed on the ceiling of the passage area 99b. In the following description, unless there is a need to particularly distinguish between the placement area dedicated rail 98a and the passage area dedicated extension rail 98b, they will be simply referred to as dedicated rails 98.

[0140] The dedicated rail 98 is provided with a movable part 97 that can move along the dedicated rail 98, and this movable part 97 can be driven by, for example, a rechargeable battery, and is configured to be able to move along the dedicated rail 98 while supporting the camera 3. Therefore, as the movable part 97 moves along the dedicated rail 98, the camera 3 can capture images of the space within the placement area 99a and the passage area 99b.

[0141] A maintenance area 99c is provided in part of the placement area 99a. In this maintenance area 99c, a part of the placement area dedicated rail 98a described above descends from the ceiling to the floor of the placement area 99a. This allows a worker to reach the movable part 97 when the movable part 97 moving on the placement area dedicated rail 98a enters the maintenance area 99c, enabling a worker to perform maintenance work such as replacing the battery of the movable part 7b.

[0142] Furthermore, in the article search system according to the modified example of the second embodiment, the movable unit 97, which is an example of a guide unit, is provided with a remote-controlled electric lift that can move the camera 3 up and down in a direction perpendicular to the floor surface of the maintenance area 99c. In this way, maintenance such as battery replacement for the movable unit 7b can be performed in the placement area 99a other than the above-mentioned maintenance area 99c.

[0143] (3) Other variations In this embodiment, a QR code (registered trademark) is used as an example of an identification code (barcode) attached to identify each item 5, but the present invention is not limited to this and other types of barcodes may also be used.

[0144] In addition, in this embodiment, as a measure to be taken when an item 5 is intentionally removed from the placement area 99, the camera 3 as the identification information acquisition unit 32 may be linked to follow the actions of the worker, follow the identification code of the item 5, always read the identification code of the item 5 at the exit 100b, or be linked to various sensors.

[0145] In addition, in this embodiment, the timing for acquiring the location information of the item 5 within the placement area 99 can be any timing as described above, or can be random at a fixed time, at a time when the item 5 is moving, only during the day or only at night, or in cooperation with human detection such as that of workers, at short intervals during times when people are detected.

[0146] Furthermore, in this embodiment, the fixed portion of the movable lamp 7 is mainly fixed to the ceiling, but the present invention is not limited to this, and the movable lamp 7 may be configured to be movable on the ceiling.

[0147] In addition, in this embodiment, methods for making the identification code of the item 5 easier to read include determining the placement position of the rack 10 within the placement area 99 in advance and determining the placement position of the identification code 9 in advance, as well as reducing the identification information of the identification code 9 and making the code itself larger.

[0148] In addition, in this embodiment, as a countermeasure when the identification code cannot be read, the position recognition unit 34 may refer to the placement history table in the memory unit 38 and notify the notification unit 37 about the item 5 on a date close to the date on which the identification code was read, or may notify the notification unit 37 about the data one step before in chronological order, or may automatically notify the notification unit 37 about the items 5 up to the point where they have been recognized.

[0149] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, the elements described in parallel in the present embodiment may be configured such that at least one of the elements is connected in series to the other elements. [Industrial Applicability]

[0150] The present invention can be applied to a map image creation device and a map image creation method that create one map image by stitching together a plurality of captured images. [Explanation of symbols]

[0151] 3...camera, 7...movable lamp, 5...item, 6...terminal, 9...identification code, 10...rack, 31...control unit, 32...identification information acquisition unit, 33...identification unit, 37...notification unit, 99...placement area, 100...item search system.

Claims

1. an identification information acquisition unit that acquires information about the space of an arrangement area in which items can be arranged, and thereby acquires characteristics of items that can be randomly arranged in the space; an identification unit that identifies the article based on the characteristics of the article acquired by the identification information acquisition unit; a position recognition unit that assigns planar coordinates to the spatial information of the placement area acquired by the identification information acquisition unit, and repeatedly associates the item identified by the identification unit with the planar coordinates at any timing; Equipped with The identification information acquisition unit It has one imaging unit, While changing the photographing position of the one photographing unit, a zoom magnification is changed according to the distance from the one photographing unit to the photographing position, and a plurality of enlarged images are photographed; The identification unit The plurality of enlarged images are joined together so that parts of the images overlap, to form a single image of the arrangement area or an image of part of the arrangement area. An object search system characterized by:

2. The identification unit When the images are joined together so that a portion of the image overlaps, the width of the overlap of the portion of the image is made larger than the maximum outer dimension of the article.

2. The object search system according to claim 1.

3. The identification unit When the images are joined together so that a portion of the image overlaps, the width of the overlap of the portion of the image is made larger than the maximum outer dimension of the article.

3. The object search system according to claim 2.

4. The identification unit stitching the images together by overlapping portions of the images after identifying the item; 3. The object search system according to claim 2.

5. unique identification information is associated with the identification code corresponding to the item; The position recognition unit When the plurality of enlarged images contain images of an article having the same identification information, the identification information of the enlarged image captured earlier is overwritten on the identification information of the enlarged image captured later, or the identification information of the enlarged image captured earlier is retained and the identification information of the enlarged image captured later is discarded.

2. The object search system according to claim 1.

6. The identification unit correcting distortion of the image of the item photographed by the one imaging unit, and identifying the item based on the image whose distortion has been corrected; 2. The object search system according to claim 1.

7. an identification information acquisition step in which an identification information acquisition unit acquires information about the space of an arrangement area in which an item can be arranged, thereby acquiring characteristics of an item that can be randomly arranged in the space; an identification step in which an identification unit identifies the item based on the characteristics of the item acquired by the identification information acquisition unit; a position recognition step in which a position recognition unit assigns planar coordinates to the spatial information of the placement area acquired by the identification information acquisition unit, and repeatedly associates the item identified by the identification unit with the planar coordinates at any timing; and In the identification information acquisition step, the identification information acquisition unit having one imaging unit captures a plurality of enlarged images while changing the imaging position of the one imaging unit and changing the zoom magnification in accordance with the distance from the one imaging unit to the imaging position; In the identifying step, the identification unit connects the plurality of enlarged images so that parts of the images overlap, to form a single image of the placement area or an image of part of the placement area; A map image creation method characterized by:

8. In the identifying step, When the identification unit joins the images together so that the images are partially overlapped, the overlap width of the images is made larger than the maximum outer dimension of the article.

8. The map image generating method according to claim 7.

9. In the identifying step, When the identification unit joins the images together so that the images are partially overlapped, the overlap width of the images is made larger than the maximum outer dimension of the article.

9. The map image generating method according to claim 8.

10. In the identifying step, After the identification unit identifies the item, the images are joined together by overlapping portions of the images.

9. The map image generating method according to claim 8.

11. unique identification information is associated with the identification code corresponding to the item; In the position recognition step, When the plurality of enlarged images contain images of an item having the same identification information, the position recognition unit overwrites the identification information of the enlarged image captured earlier with the identification information of the enlarged image captured later, or retains the identification information of the enlarged image captured earlier and discards the identification information of the enlarged image captured later.

8. The map image generating method according to claim 7.

12. In the identifying step, the identification unit corrects distortion in the image of the item photographed by the one imaging unit, and identifies the item based on the distortion-corrected image.

8. The map image generating method according to claim 7.

Images