Photographing device and photographing method

Abstract

To prevent an illumination light source from being reflected in a captured image.SOLUTION: An imaging apparatus comprises a movement mechanism, a light source, and a camera. The movement mechanism moves in a first direction. The light source is mounted on the movement mechanism. The camera is fitted to the movement mechanism such that the camera is disposed linearly or almost linearly to the light source along a height direction to perform imaging by facing a third direction turned about the height direction with respect to a second direction orthogonal to the first direction and the height direction.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] An embodiment of the present invention relates to an imaging device and an imaging method. [Background technology]

[0002] 2. Description of the Related Art As part of efforts to save labor, stores such as supermarkets use devices that take images of product shelves in the store with cameras and automatically check whether the price tags on the product shelves are correct.

[0003] This device checks the price tags on the shelves outside of business hours when there are no shoppers in the store. From the perspective of automation, it is highly likely that the device will automatically patrol the store at night. When capturing images at night, the lights in the store may not be on, so a set of lighting and a camera is required to capture the images.

[0004] On the other hand, price tags on product shelves are often placed between transparent resin and installed at the edge of the product shelves, so that shoppers can see them through the transparent resin. Therefore, when photographing with the above-mentioned device, the light source may be reflected on the transparent resin, and the light source may be reflected on the price tag on the product shelf. If the light source is reflected in this way, the photographed image will have overexposed highlights, making it impossible to see the price tag information printed on the price tag. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2012-053711 A Summary of the Invention [Problem to be solved by the invention]

[0006] The problem to be solved by the embodiments of the present invention is to provide a photographing device and a photographing method in which an illumination light source is not captured in a photographed image. [Means for solving the problem]

[0007] In one embodiment, the image capturing device includes a moving mechanism, a light source, and a camera. While maintaining a predetermined distance from one or more objects to be photographed aligned along a first direction, The camera moves in a first direction. The light source is mounted on a moving mechanism. The camera is disposed in a straight line or substantially straight line with the light source along the height direction, and is attached to the moving mechanism so as to face a second direction perpendicular to the first direction and the height direction. The moving mechanism moves in a third direction that is at least 1 / 2 of the angle of view of the camera with the height direction as an axis with respect to the second direction. With the camera After rotating, the camera moves in the first direction while maintaining the rotated state in the third direction. The camera faces the third direction and takes a picture.

[0008] In one embodiment, the photographing method includes a moving mechanism that is attached so that the light source and the camera are arranged in a straight line or substantially straight line along a height direction, and the camera faces a third direction rotated about the height direction with respect to a second direction perpendicular to the first direction and the height direction, and photographs the third direction rotated about the height direction with respect to the second direction, the third direction being at least 1 / 2 of the angle of view of the camera with respect to the height direction as an axis, while maintaining a predetermined distance from one or more photographing objects lined up along the first direction. With the camera After rotating the camera, the camera is moved in the first direction while rotated in a third direction, and the camera is caused to capture an image facing the third direction. The method further comprises correcting distortion of the image captured by the camera, and acquiring an image equivalent to an image captured when the camera is facing the second direction. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a block diagram showing the hardware configuration of the imaging apparatus according to the first embodiment. [Diagram 2] FIG. 1 is a schematic diagram showing the installation state of an illumination light source and a camera on a running robot. [Diagram 3] FIG. 13 is a schematic diagram showing the travel direction of the traveling robot relative to the product shelves as viewed from above the product shelves. [Figure 4] FIG. 4 is a schematic diagram showing the shooting range of a camera with respect to a product shelf. [Diagram 5]FIG. 4 is a schematic diagram showing an example of photographed image data before correction captured by a camera. [Figure 6] FIG. 11 is a schematic diagram showing an example of corrected captured image data. [Figure 7] FIG. 11 is a schematic diagram showing photographed image data of a comparative example. [Figure 8] 4 is a flowchart showing a main control procedure of a processor of the imaging control device. [Figure 9] 9 is a flowchart showing a control procedure for the image correction process in FIG. 8 . [Figure 10] FIG. 13 is a schematic diagram showing an example of the arrangement of a camera and an illumination light source in a first modified example of the first embodiment. [Figure 11] FIG. 11 is a schematic diagram showing the shooting range of a camera with respect to a product shelf in the first modified example. [Figure 12] 11 is a flowchart showing a main control procedure of a processor of an imaging control device in a second modified example of the first embodiment. [Figure 13] 13 is a schematic diagram showing a state in which a camera and a light source are attached to a running robot in the imaging device of the second embodiment. FIG. [Figure 14] 13 is a schematic diagram showing the traveling direction of a traveling robot relative to the product shelves in the second embodiment, as viewed from above the product shelves. FIG. [Figure 15] 10 is a flowchart showing a main control procedure of a processor of an imaging control device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of a photographing device that does not capture an illumination light source in a photographed image will be described with reference to the drawings.

[0011] [First embodiment] 1 is a block diagram showing a hardware configuration of an image capturing device 1 according to a first embodiment. The image capturing device 1 includes an image capturing control device 10, a traveling robot 20, lighting 30, and a camera 40. The image capturing device 1 may further include a touch panel 50 and / or an image recognition device 60.

[0012] The photography control device 10 controls each part of the photography device 1. The traveling robot 20 autonomously travels along a predetermined travel route according to instructions from the photography control device 10. The lighting 30 includes a plurality of lighting light sources. The lighting light source may be a white light source or an infrared light source. The camera 40 photographs the subject. The camera 40 is, for example, a digital camera that uses a CCD (Charge Coupled Device) as an image sensor. When the lighting light source is a white light source, the camera 40 may be a color camera or a monochrome camera. When the lighting light source is an infrared light source, the camera 40 may be an infrared camera. The touch panel 50 allows the user of the photography device 1 to input instructions to the photography control device 10 and displays various information generated by the photography device 1. The image recognition device 60 recognizes the subject from the photographed image data photographed by the camera 40.

[0013] The imaging control device 10 includes a processor 11, a memory 12, a storage 13, a robot interface 14, a lighting interface 15, and a camera interface 16. The imaging control device 10 may further include an instruction interface 17 and / or a transmission interface 18. In FIG. 1, the interfaces are abbreviated as I / F. The processor 11, the memory 12, the storage 13, the robot interface 14, the lighting interface 15, the camera interface 16, the instruction interface 17, and the transmission interface 18 are each connected to a system transmission path 19. The system transmission path 19 includes an address bus, a data bus, a control signal line, and the like. In the imaging control device 10, the processor 11 and the memory 12 are connected by the system transmission path 19 to configure a computer that performs information processing for controlling the imaging control device 10.

[0014] The robot interface 14 is connected to the traveling robot 20. The lighting interface 15 is connected to the lighting 30. The camera interface 16 is connected to the camera 40. The instruction interface 17 is connected to the touch panel 50. The transmission interface 18 is connected to the image recognition device 60.

[0015] The processor 11 of the imaging control device 10 corresponds to the central part of the computer. The processor 11 controls each part to realize various functions of the imaging control device 10 according to an operating system or an application program. The processor 11 is, for example, a CPU (Central Processing Unit).

[0016] The memory 12 corresponds to the main memory portion of the computer. The memory 12 includes a non-volatile memory area and a volatile memory area. The memory 12 stores an operating system or application programs in the non-volatile memory area. The memory 12 stores data required for the processor 11 to execute processes for controlling each part in the volatile memory area. The processor uses the volatile memory area of ​​the memory 12 as a work area where data can be rewritten as appropriate. The non-volatile memory area is, for example, a ROM (Read Only Memory). The volatile memory area is, for example, a RAM (Random Access Memory).

[0017] The storage 13 non-volatilely stores the captured image data captured by the camera 40. The storage 13 may use any of well-known storage devices such as an EEPROM (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive), either singly or in combination.

[0018] The robot interface 14 is an interface for the processor 11 to exchange data with the traveling robot 20. The robot interface 14 includes a wireless communication unit for transmitting and receiving data by wireless communication such as wireless LAN (Local Area Network) and Bluetooth (registered trademark). The imaging control device 10 can be placed in a management room of a store and remotely control the traveling robot 20 by using the robot interface 14. Data transmitted from the processor 11 to the traveling robot 20 can include travel instruction data for instructing the traveling robot 20 to travel. The traveling robot 20 stores a travel route indicating where in the store the traveling robot 20 will travel, and the travel instruction data is data for specifying one of a plurality of travel start points included in the travel route. The travel instruction data may include data for specifying a stop point for travel. Data received by the processor 11 from the traveling robot 20 can include status data related to the autonomous travel of the traveling robot 20. The status data includes arrival data indicating that the traveling robot 20 has reached a stop point corresponding to a specified travel start point or a specified stop point. The status data can also include notification data such as a malfunction.

[0019] The lighting interface 15 is an interface for transmitting lighting control data and the like that instructs the processor 11 to turn on and off the lighting light source provided in the lighting 30. The lighting interface 15 includes a wireless communication unit for transmitting the lighting control data via wireless communication such as wireless LAN or BlueTooth. The lighting interface 15 allows the photography control device 10 to remotely control the lighting state of the lighting 30.

[0020] The camera interface 16 is an interface through which the processor 11 exchanges data with the camera 40. The camera interface 16 includes a wireless communication unit for transmitting and receiving data by wireless communication such as wireless LAN and BlueTooth. The camera interface 16 allows the shooting control device 10 to remotely control the camera 40. Data transmitted from the processor 11 to the camera 40 may include shooting instruction data for causing the camera 40 to shoot an image, etc. Data received by the processor 11 from the camera 40 may include image data captured by the camera, etc. The captured image data may be stored in the storage 13.

[0021] The instruction interface 17 is an interface through which the processor 11 receives instruction data from the touch panel 50. The instruction data can include, for example, an instruction to start the operation of the photographing device 1, an instruction to select photographed image data photographed by the camera 40, output of the selected photographed image data to the image recognition device 60, and the like.

[0022] The transmission interface 18 is an interface for the processor 11 to transmit the captured image data stored in the storage 13 to the image recognition device 60. The transmission interface 18 includes a wireless communication unit for transmitting the captured image data by wireless communication such as wireless LAN or Bluetooth. This transmission interface 18 enables the photography control device 10 to cause the image recognition device 60 to recognize information on the photographed subject, such as price tag information printed on the price tag, from the captured image data.

[0023] The positional relationship between the traveling robot 20, the illumination light source included in the lighting 30, and the camera 40 will be described below with reference to Figs. 2 to 4. Fig. 2 is a schematic diagram showing the mounting state of the illumination light source 31 and the camera 40 on the traveling robot 20. Fig. 3 is a schematic diagram showing the traveling direction of the traveling robot 20 relative to the product shelf 70, as viewed from above the product shelf. Fig. 4 is a schematic diagram showing the shooting range of the camera 40 relative to the product shelf 70.

[0024] The traveling robot 20 has a robot body 21 with three or four wheels, and an attachment part 22 extending upward from the robot body 21. The robot body 21 is capable of moving forward and backward, and of rotating left and right by two-wheel differential drive. A plurality of illumination light sources 31 and a camera 40 are attached to the attachment part 22. The illumination light sources 31 and the cameras 40 are arranged in a straight line along the height direction. The attachment part 22 holds the cameras 40 so that they face in a direction shifted from the normal direction of the side surface of the robot body 21.

[0025] Specifically, the traveling robot 20 travels from a travel start point 81 to a stop point 82, approximately parallel to the product shelves 70. Hereinafter, this direction is referred to as a first direction. Thus, the traveling robot 20 functions as a moving mechanism that moves in the first direction.

[0026] Although not shown, the product shelf 70 has multiple display shelves. Each display shelf displays the same products aligned in the depth direction of the display shelf (the up-down direction in FIG. 3). Each display shelf displays different products aligned in a first direction, which is the left-right direction of the display shelf. Each display shelf displays a price tag on which price tag information indicating the price of each product is printed, corresponding to each product. Here, the product shelf 70 has a height since it has multiple display shelves. The mounting unit 22 of the traveling robot 20 holds the camera 40 at a mounting angle such that the camera 40 can photograph the price tags presented on the multiple display shelves of the product shelf 70. The distance between the camera 40 and the price tag presentation surface of the product shelf 70 in a second direction perpendicular to the first direction and the height direction, that is, in the normal direction of the side surface of the robot body 21, is specified so that the camera 40 can photograph such a way. That is, the distance between the camera 40 and the product shelf 70 is regulated according to the angle of view in the height direction determined by the mounting angle of the camera 40. In this way, the distance from the product shelf 70 along which the traveling robot 20 travels is determined. Also, the mounting part 22 of the traveling robot 20 holds a plurality of illumination light sources 31 lined up in the height direction with the camera 40 in between, so that the camera 40 can photograph a plurality of price tags distributed in the height direction.

[0027] The mounting unit 22 of the traveling robot 20 holds the camera 40 at a mounting angle that faces a third direction rotated with respect to the second direction around the height direction as an axis. That is, the mounting unit 22 holds the camera 40 so that the imaging optical axis O of the camera 40 faces a third direction rotated by an angle θ with respect to the second direction. The imaging width W of the product shelf 70 photographed by the camera 40 at this time is determined according to the rotation angle θ of the camera 40 in the yaw direction and the distance between the camera 40 and the product shelf 70. The mounting unit 22 holds the camera 40 so that the rotation angle θ of the camera 40 in the yaw direction is 1 / 2 or more with respect to the angle of view AoV of the camera 40 in the first direction. If the yaw direction rotation angle θ is smaller than 1 / 2 of the angle of view AoV, the reflected light from the price tag of the illumination light source 31 will be captured in the image data captured by the camera 40.

[0028] Fig. 5 is a schematic diagram showing an example of photographed image data before correction captured by the camera 40. Fig. 6 is a schematic diagram showing an example of photographed image data after correction captured by the camera 40. Fig. 7 is a schematic diagram showing photographed image data of a comparative example.

[0029] As described above, when the camera 40 is attached to the mounting portion 22 of the traveling robot 20 with a rotation angle θ in the yaw direction of the camera 40 without facing the price tag display surface of the product shelf 70, the captured image data 41 of the camera 40 has a keystone distortion as shown in FIG. 5. The processor 11 of the photography control device 10 performs a keystone distortion correction process on the captured image data 41 by an application program stored in the memory 12. As a result, the processor 11 can convert the captured image data 41 having the distortion as shown in FIG. 5 into captured image data 42 without keystone distortion as shown in FIG. 6. That is, the processor 11 can obtain captured image data 42 that corresponds to the captured image data obtained when the camera 40 is facing the price tag display surface of the product shelf 70 and photographs the image, that is, the captured image data 42 is equivalent to the captured image data. When the camera 40 is facing the price tag display surface of the product shelf 70 and photographs the image, the obtained captured image data 43 has a highlight blowout 44 due to the illumination light source 31 being reflected as shown in FIG. 7. In contrast, the corrected captured image data 42 is an image without such highlight blowout as shown in FIG. 6.

[0030] 8 is a flow chart showing the main control procedure of the processor 11 of the imaging control device 10. The operation of the imaging control device 10 will be described below with reference to this diagram. Note that the operation procedure and its contents described below are merely examples. As long as similar results are obtained, the procedure and its contents are not limited.

[0031] For example, when the processor 11 receives an instruction to start shooting with the photographing device 1 from the touch panel 50 via the instruction interface 17, the processor 11 starts the operation shown in Fig. 8. First, the processor 11 transmits running instruction data to instruct the running robot 20 to start running via the robot interface 14 (ACT11).

[0032] The travel instruction data transmitted from the photography control device 10 to the traveling robot 20 may be data specifying a travel start point 81. The traveling robot 20 stores a travel route indicating where in the store the traveling robot 20 will travel. In other words, since the traveling robot 20 stores a stop point 82 corresponding to the specified travel start point 81, it is not necessary to specify the stop point 82. Of course, the travel instruction data may include data specifying a stop point 82 for travel. The specified travel start point 81 (and stop point 82) is the travel start point 81 (and stop point 82) of any one of a plurality of travel routes for the traveling robot 20 to travel around all of the stores. Of course, the travel instruction data may be the travel start point 81 of the first travel route (and stop point 82 of the last travel route) of all travel routes.

[0033] Then, the processor 11 judges whether the running robot 20 has reached the running start point 81 (ACT12). The running robot 20 autonomously runs from the waiting position to the running start point 81. Then, when the running robot 20 reaches the running start point 81, the running robot 20 transmits status data including arrival data indicating that to the imaging control device 10. Therefore, the processor 11 can judge whether the running robot 20 has reached the running start point 81 depending on whether the arrival data has been received via the robot interface 14. If the running robot 20 has not yet reached the running start point 81 (NO in ACT12), the processor 11 repeats the processing of this ACT12. In this way, the processor 11 waits for the running robot 20 to reach the running start point 81.

[0034] In addition, there are cases where the travel start point 81 of one instructed travel route is a waiting position of the traveling robot 20, or is the same point as the stopping point 82 of the previous travel route. In such cases, the processor 11 immediately determines that the traveling robot 20 has reached the travel start point 81.

[0035] When the running robot 20 reaches the running start point 81 in this way (YES in ACT 12), the processor 11 starts measuring time using the photography timer provided in the memory 12 (ACT 13).

[0036] Then, the processor 11 judges whether the running robot 20 has reached the stop point 82 (ACT14). The running robot 20 autonomously runs at a constant speed from the running start point 81 to the stop point 82. Then, when the running robot 20 reaches the stop point 82, the running robot 20 transmits status data including arrival data indicating that to the shooting control device 10. Therefore, the processor 11 can judge whether the running robot 20 has reached the stop point 82 depending on whether the arrival data has been received via the robot interface 14.

[0037] If the traveling robot 20 has not yet reached the stop point 82 (NO in ACT14), the processor 11 determines whether or not a certain time has elapsed since the start of counting the photography timer (ACT15). If the certain time has not yet elapsed (NO in ACT15), the processor 11 returns to the process of ACT14.

[0038] Then, when a certain period of time has elapsed (YES in ACT15), the processor 11 outputs a shooting instruction (ACT16). The shooting instruction includes shooting instruction data for the camera 40 and lighting control data for the lighting 30. The processor 11 transmits the shooting instruction data for causing the camera 40 to shoot to the camera 40 via the camera interface 16. The processor 11 also transmits lighting control data for instructing the lighting 30 to turn on the multiple lighting light sources 31 provided in the lighting 30 via the lighting interface 15.

[0039] The processor 11 receives the photographed image data captured by the camera 40 via the camera interface 16, and temporarily stores it in the memory 12 (ACT 17).

[0040] Then, the processor 11 performs image correction processing on the captured image data temporarily stored in this memory (ACT18). FIG. 9 is a flow chart showing the control procedure of this image correction processing. In this image correction processing, the processor 11 first performs keystone correction processing on the captured image data (ACT181). Thus, the processor 11 functions as a correction means for correcting distortion of the captured image data captured by the camera 40 and acquiring captured image data equivalent to the case where the camera 40 is facing the second direction. After that, the processor 11 performs light amount unevenness correction processing on the captured image data after the keystone correction processing (ACT182). Then, the processor 11 ends this image correction processing and returns to the processing of FIG. 8. Note that the keystone correction processing and the light amount unevenness correction processing are well-known correction processing, and therefore their description will be omitted.

[0041] Once the captured image data has been subjected to the image correction process, the processor 11 stores the captured image data that has been subjected to the image correction process in the storage 13 (ACT19).

[0042] After that, the processor 11 clears the shooting timer once (ACT20). Then, the shooting timer starts counting again (ACT21). After that, the processor 11 returns to the process of ACT14.

[0043] In this way, while the traveling robot 20 autonomously travels along the travel route at a constant speed, the camera 40 captures images of the product shelves 70 including the price tags illuminated by the illumination light source 31 at regular time intervals. The traveling robot 20 travels at a constant speed. Therefore, the camera 40 captures images at regular distance intervals. The processor 11 corrects the image data of the multiple images captured at these regular distance intervals and then stores them in the storage 13.

[0044] In this manner, photographing is continued with the camera 40, and when the traveling robot 20 reaches the stopping point 82 (YES in ACT14), the processor 11 clears the photographing timer (ACT22). Then, the operation shown in FIG. 8 ends.

[0045] Thereafter, by receiving an instruction to start photographing from the photographing device 1 again, the processor 11 can photograph the next travel route.

[0046] Furthermore, the processor 11 can receive an instruction to select photographed image data stored in the storage 13 from the touch panel 50 via the instruction interface 17 and output the photographed image data to the image recognition device 60. In this case, the processor 11 transmits the selected photographed image data to the image recognition device 60 via the transmission interface 18. The image recognition device 60 can recognize information about the photographed subject, such as price tag information printed on the price tag, from the transmitted photographed image data.

[0047] In addition, in the photographed image data photographed at regular intervals, the price tag portion may be included at the edge of the image, and the entire price tag may not be captured. Therefore, the interval for photographing the photographed image data is set so that two consecutive photographed image data pieces include an overlapping portion. In addition, the processor 11 may combine two consecutive photographed image data pieces into one photographed image data piece by a well-known image matching process, and then transmit the combined image data piece to the image recognition device 60.

[0048] [Variation 1] In the first embodiment, the camera 40 and the illumination light source 31 are arranged linearly along the height direction as shown in Fig. 2. However, the camera 40 and the illumination light source 31 may be arranged approximately linearly. Fig. 10 is a schematic diagram showing an example of the arrangement of the camera 40 and the illumination light source 31 in the first modification. Fig. 11 is a schematic diagram showing the shooting range of the camera 40 with respect to the product shelf 70 in the first modification.

[0049] As shown in FIG. 10, the camera 40 and the illumination light source 31 may be spaced apart from each other by a distance L in the first direction. This distance L is sufficiently smaller than the imaging width W of the camera. Specifically, the distance L in the first direction is L≦2×atan(Aov / 2-θ)×wd Here, as shown in FIG. 11, AoV is the photographing angle of view of the camera 40. wd is the distance in the second direction between the camera 40 and the price tag display surface of the product shelf 70, which is the photographing target on the photographing optical axis O of the camera 40. θ is the rotation angle of the camera 40 in the yaw direction in the third direction. If the camera 40 and the illumination light source 31 are arranged at such a distance L, it is possible to prevent the illumination light source 31 from being captured in the photographed image data, as in the first embodiment.

[0050] The illumination light source 31 must be located on the opposite side in the first direction to the side facing the camera 40. If the illumination light source 31 is located on the same side facing the camera 40, the illumination light source 31 will be captured in the captured image data.

[0051] When the camera is placed directly facing the price tags on the product shelf 70 to take pictures, the illumination light source 31 must be placed at a location farther away than the shooting width W of the camera 40, otherwise the illumination light source 31 will be included in the image data captured by the camera 40. If the distance between the camera 40 and the illumination light source 31 is placed farther away than the shooting width W in this way, the entire device will become larger.

[0052] In contrast to this, in the present modified example 1, it is only necessary to separate the illumination light source 31 from the camera 40 by a distance L that is sufficiently smaller than the photographing width W, so that the device can be made compact.

[0053] In this case, the distance L is determined based on the shooting angle of view AoV, the distance wd in the second direction, and the yaw direction rotation angle θ, and the camera 40 and the illumination light source 31 are disposed at this distance L. However, the camera 40 and the illumination light source 31 may be disposed in a substantially linear manner at an arbitrary distance L in advance. In such a case, the rotation angle θ in the yaw direction is determined based on the above formula, and the camera 40 is attached to the traveling robot 20 so as to achieve this rotation angle θ. Alternatively, the distance wd in the second direction may be adjusted.

[0054] [Variation 2] 12 is a flow chart showing the main control procedure of the processor 11 of the imaging control device 10 in the second modified example of the first embodiment. The operation of the imaging control device 10 will be described with reference to this diagram. Note that the operation procedure and its contents described below are merely examples. As long as similar results are obtained, the procedure and its contents are not limited.

[0055] Here, the same processes as those in the first embodiment are denoted by the same reference symbols as those in Fig. 8, and the description thereof will be omitted. In this modified example 2, when the running robot 20 reaches the running start point 81 (YES in ACT12), the processor 11 acquires the reference position of the running robot 20 (ACT31). The processor 11 stores the acquired reference position in the memory 12. The reference position here is the position of the running start point 81, for example, the XY coordinates on a map showing the arrangement of the product shelves 70 inside the store and the running route of the running robot 20.

[0056] Then, in ACT 14, the processor 11 determines whether or not the running robot 20 has reached the stopping point 82, as in the first embodiment. Then, if the running robot 20 has not yet reached the stopping point 82 (NO in ACT 14), in this modified example 2, the processor 11 acquires the position of the running robot 20 (ACT 32). The running robot 20 always measures its own position in order to run autonomously. Therefore, the processor 11 can acquire the position by having the running robot 20 output the position and receiving it via the robot interface 14. Of course, the position of the running robot 20 may be acquired by other means, such as using a beacon.

[0057] The processor 11 calculates the difference between the acquired position of the running robot 20 and the reference position stored in the memory 12, and determines whether the running robot 20 has moved a certain distance (ACT33). If the running robot 20 has not yet moved a certain distance (NO in ACT33), the processor 11 returns to the process of ACT14.

[0058] Then, when the running robot 20 has moved a certain distance (YES in ACT33), the processor 11 proceeds to ACT16 and outputs a shooting instruction. Then, the processor 11 executes the processes of ACT17 to ACT19 described in the first embodiment. Then, the processor 11 rewrites the reference position of the running robot 20 stored in the memory 12 to the position acquired in ACT32 (ACT33). Then, the processor 11 returns to the process of ACT14.

[0059] In this way, while the traveling robot 20 autonomously travels along the travel route at a constant speed, the camera 40 captures an image of the product shelf 70 including the price tag illuminated by the illumination light source 31 every time the traveling robot travels a certain distance. By acquiring captured image data every certain distance in this way, it is possible to synthesize two consecutive captured image data without performing image matching processing or the like. Furthermore, the traveling robot 20 travels at a constant speed, but if an obstacle is present, it may stop at that position. Then, when the obstacle is no longer present, it starts traveling again. In such a case, it is preferable to capture images every certain distance rather than every certain time as in the first embodiment.

[0060] In this manner, the camera 40 takes pictures, and when the traveling robot 20 reaches the stopping point 82 (YES in ACT14), the processor 11 clears the picture-taking timer (ACT22). Then, the operation shown in FIG. 8 ends.

[0061] The photographing device 1 having the configuration according to the first embodiment or each of the modified examples described above includes a running robot 20 which is a mobile mechanism that moves in a first direction, an illumination light source 31 mounted on the running robot 20, and a camera 40 which is arranged in a straight line or approximately in a straight line with the illumination light source 31 along the height direction and is attached to the running robot 20 so as to capture an image facing a third direction rotated around the height direction as an axis with respect to a second direction perpendicular to the first direction and the height direction.

[0062] In this way, by having the camera 40 face in the third direction and capture images as the traveling robot 20 moves, it is possible to prevent the illumination light source 31 from appearing in the captured image data.

[0063] Furthermore, the processor 11 corrects distortion of the image captured by the camera 40, and acquires captured image data corresponding to the image captured by the camera 40 facing in the second direction.

[0064] As a result, even if the camera 40 takes an image facing the third direction, it is possible to obtain the same captured image data as when the camera 40 takes an image facing the second direction.

[0065] [Second embodiment] Fig. 13 is a schematic diagram showing the mounting state of the camera 40 and the illumination light source 31 on the traveling robot 20 in the imaging device 1 of the second embodiment. Fig. 14 is a schematic diagram seen from above the product shelf 70, showing the traveling direction of the traveling robot 20 relative to the product shelf 70 in the second embodiment.

[0066] In the first embodiment and its modified examples, the camera 40 is attached to the running robot 20 at a rotation angle θ in the yaw direction relative to the running robot 20. In contrast, in the second embodiment, the camera 40 is attached to the running robot 20 so as to face in the normal direction to the side surface of the robot body 21. That is, the mounting part 22 of the running robot 20 holds the camera 40 so that it is disposed in a straight line with the illumination light source 31 along the height direction, and faces in a second direction perpendicular to the first direction and the height direction.

[0067] In this case, the traveling robot 20 is one that can move straight in a first direction parallel to the product shelf 70 while maintaining the orientation of the robot body 21 rotated in the yaw direction as shown in FIG. 14. The traveling robot 20 that can move straight in this way uses, for example, an omni wheel / mecanum wheel. The omni wheel / mecanum wheel is a wheel that does not turn using steering like a general tire, but turns using the rotation difference of the drive wheels. The omni wheel / mecanum wheel is capable of turning and moving parallel in all directions, as well as moving in the rotation direction of the wheel like a general tire, by controlling the rotation difference of the wheels. In addition, the traveling robot 20 using the omni wheel / mecanum wheel has a simpler moving mechanism than a tire using steering or crawlers, and is expected to be smaller and lighter in weight.

[0068] 15 is a flow chart showing the main control procedure of the processor 11 of the imaging control device 10 in the second embodiment. Hereinafter, the operation of the imaging control device 10 will be described with reference to this diagram. Note that the operation procedure and its contents described below are merely examples. As long as similar results are obtained, the procedure and its contents are not limited.

[0069] Here, the same processes as those in the first embodiment are denoted by the same reference symbols as those in Fig. 8, and the description thereof will be omitted. In this second embodiment, the processor 11 first transmits robot rotation start instruction data to the traveling robot 20 via the robot interface 14 (ACT41). This robot rotation start instruction data instructs the traveling robot 20 to rotate by a rotation angle θ in the yaw direction.

[0070] Thereafter, similarly to the first embodiment, the processes in ACT11 to ACT21 are executed, and photographed image data is acquired by the camera 40 at regular time intervals.

[0071] Then, when the running robot 20 reaches the stopping point 82 (YES in ACT14), the processor 11 proceeds to ACT22 and clears the photography timer. After that, the processor 11 transmits robot rotation end instruction data to the running robot 20 via the robot interface 14 (ACT42). This robot rotation end instruction data instructs the running robot 20 to end rotation at the rotation angle θ in the yaw direction. Then, the processor 11 ends the operation shown in FIG. 15.

[0072] As described above, the photographing device 1 configured in the second embodiment includes a running robot 20 which is a moving mechanism that moves in a first direction, an illumination light source 31 mounted on the running robot 20, and a camera 40 attached to the running robot 20 arranged in a straight line with the illumination light source 31 along the height direction and facing a second direction perpendicular to the first direction and the height direction, and the running robot 20 moves in the first direction in a rotated state so that the camera 40 takes photographs facing a third direction rotated around the height direction as an axis relative to the second direction.

[0073] Even with this configuration, by having the camera 40 face in the third direction and take images as the traveling robot 20 moves, it is possible to prevent the illumination light source 31 from appearing in the captured image data.

[0074] Needless to say, in the second embodiment as well, modifications similar to the first and second modifications of the first embodiment can be made.

[0075] Although the embodiment of the photographing device 1 and the photographing method in which the illumination light source is not captured in the photographed image data has been described above, the embodiment is not limited to this.

[0076] For example, the subject photographed by camera 40 may not be a price tag but may be the product itself. By photographing the product illuminated by illumination light source 31 with camera 40 facing the third direction, photographed image data in which the texture of the product is more easily distinguishable can be obtained.

[0077] Furthermore, the photography control device 10 and the image recognition device 60 may be built into the robot body 21 of the traveling robot 20. This makes it possible to omit the wireless communication unit, and to configure the photography device at low cost.

[0078] In addition, each embodiment is presented as an example and is not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the invention and its equivalents described in the claims. In addition, the claims as originally filed in this application are set forth below. [C1] A moving mechanism that moves in a first direction; A light source mounted on the moving mechanism; a camera that is disposed in a straight line or substantially in a straight line with the light source along a height direction and is attached to the moving mechanism so as to capture an image in a third direction that is rotated around the height direction with respect to the first direction and a second direction perpendicular to the height direction; An imaging device comprising: [C2] A moving mechanism that moves in a first direction; A light source mounted on the moving mechanism; a camera that is disposed in a straight line or substantially in a straight line with the light source along a height direction and is attached to the moving mechanism so as to face a second direction perpendicular to the first direction and the height direction; Equipped with The moving mechanism moves in the first direction in a rotated state so that the camera takes an image facing a third direction rotated around the height direction as an axis relative to the second direction. [C3] 3. The photographing device according to claim 1, further comprising a correction unit that corrects distortion of an image captured by the camera to obtain an image equivalent to an image captured by the camera facing in the second direction. [C4] a moving mechanism that is attached so that a light source and a camera arranged in a straight line or substantially straight line along a height direction are photographed with the camera facing a first direction and a second direction perpendicular to the height direction and a third direction rotated about the height direction as an axis, and that moves the moving mechanism in the first direction while maintaining a predetermined distance from one or more photographing objects arranged along the first direction; correcting distortion of an image captured by the camera to obtain an image equivalent to an image captured by the camera facing in the second direction; The photographing method includes: [C5] a moving mechanism is attached to the light source and the camera, which are arranged in a straight line or substantially in a straight line along a height direction, so that the camera faces a first direction and a second direction perpendicular to the height direction; and moving the moving mechanism in the first direction while rotating the camera so that the camera faces a third direction rotated about the height direction as an axis with respect to the second direction and captures an image while maintaining a predetermined distance from one or more shooting objects arranged along the first direction; correcting distortion of an image captured by the camera to obtain an image equivalent to an image captured by the camera facing in the second direction; The photographing method includes: [Explanation of symbols]

[0079] 1...imaging device, 10...imaging control device, 11...processor, 12...memory, 13...storage, 14...robot interface, 15...lighting interface, 16...camera interface, 17...instruction interface, 18...transmission interface, 19...system transmission path, 20...traveling robot, 21...robot body, 22...mounting part, 30...lighting, 31...lighting light source, 40...camera, 41,42,43...captured image data, 50...touch panel, 60...image recognition device, 70...product shelf, 81...travel starting point, 82...stopping point.

Claims

1. A moving mechanism that moves in a first direction while maintaining a predetermined distance from one or more subjects to be photographed that are arranged along the first direction; A light source mounted on the moving mechanism; a camera that is disposed in a straight line or substantially in a straight line with the light source along a height direction and is attached to the moving mechanism so as to face a second direction perpendicular to the first direction and the height direction; the moving mechanism rotates together with the camera in a third direction by half or more of an angle of view of the camera with the height direction as an axis with respect to the second direction, and then moves in the first direction while maintaining the rotated state in the third direction; The camera is configured to face the third direction and capture an image.

2. 2. The photographing device according to claim 1, further comprising a correction unit that corrects distortion of the image photographed by the camera to obtain an image equivalent to an image photographed by the camera facing in the second direction.

3. a moving mechanism for moving a light source and a camera arranged in a straight line or substantially in a straight line along a height direction, the moving mechanism being attached so that the camera faces a first direction and a second direction perpendicular to the height direction, the moving mechanism rotating the light source and a camera together with the camera in a third direction by half or more of the angle of view of the camera with the height direction as an axis relative to the second direction while maintaining a predetermined distance from one or more shooting objects lined up along the first direction, and then moving the moving mechanism in the first direction while rotated in the third direction, so that the camera is directed toward the third direction to shoot; correcting distortion of an image captured by the camera to obtain a captured image equivalent to an image captured by the camera facing in the second direction; The photographing method includes:

Images