Information processing device and information processing method

Abstract

To provide an information processing device and information processing method that can improve the detection accuracy of two-dimensional codes. [Solution] The information processing device according to the embodiment comprises an acquisition unit, a detection unit, a specification unit, and an adjustment unit. The acquisition unit acquires an image captured by a camera. The detection unit detects a two-dimensional code that appears in the image. The specification unit identifies coordinate data indicating the position of the detected two-dimensional code in the image. The adjustment unit adjusts the orientation of the display surface of the two-dimensional code based on the coordinate data.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus and an information processing method.

Background Art

[0002] Conventionally, a technique for detecting an object such as a person based on an image captured by a camera built into a lighting device is known (see, for example, Patent Document 1). In addition, in object detection using a camera image, a technique for reading a two-dimensional code arranged for the purpose of improving detection accuracy and adding attribute information from the camera image has been proposed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, there is a risk that the detection accuracy of the two-dimensional code may decrease. Specifically, the appearance of the two-dimensional code in the image varies depending on the lens information of the camera, the installation height of the camera, the size of the two-dimensional code, or the installation position of the two-dimensional code. For example, when a two-dimensional code located at the edge of an image is captured using a fisheye lens with strong distortion, the two-dimensional code is displayed distorted, and the information embedded in the two-dimensional code cannot be accurately read, that is, the detection accuracy of the two-dimensional code may decrease.

[0005] An object of the present invention is to provide an information processing apparatus and an information processing method capable of improving the detection accuracy of a two-dimensional code.

Means for Solving the Problems

[0006] The information processing device according to this embodiment comprises an acquisition unit, a detection unit, a specification unit, and an adjustment unit. The acquisition unit acquires an image captured by a camera. The detection unit detects a two-dimensional code that appears in the image. The specification unit identifies coordinate data indicating the position of the detected two-dimensional code in the image. The adjustment unit adjusts the orientation of the display surface of the two-dimensional code based on the coordinate data. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows an example of the configuration of an information processing system according to an embodiment. [Figure 2] Figure 2 is a block diagram showing an example configuration of an information processing device according to the embodiment. [Figure 3] Figure 3 shows an example of code information. [Figure 4] Figure 4 is a diagram illustrating the angle adjustment process performed by the adjustment unit. [Figure 5] Figure 5 is a diagram illustrating the angle adjustment process performed by the adjustment unit. [Figure 6] Figure 6 is a diagram illustrating the angle adjustment process performed by the adjustment unit. [Figure 7] Figure 7 is a diagram illustrating the angle adjustment process performed by the adjustment unit. [Figure 8] Figure 8 is a flowchart showing the processing procedure of the information processing device executed according to the embodiment. [Modes for carrying out the invention]

[0008] The information processing device 100 described below comprises an acquisition unit 121, a detection unit 122, a specification unit 123, and an adjustment unit 124. The acquisition unit 121 acquires the captured image taken by the camera 11. The detection unit 122 detects the two-dimensional code C that appears in the captured image. The specification unit 123 identifies the coordinate data indicating the position of the detected two-dimensional code C in the captured image. The adjustment unit 124 adjusts the orientation of the display surface of the two-dimensional code C based on the coordinate data.

[0009] The adjustment unit 124, described below, calculates the angle between the camera 11's field of view center axis and the display surface based on the coordinate data, and adjusts the orientation of the display surface based on this angle.

[0010] The adjustment unit 124, described below, tilts the orientation of the display surface toward the camera 11 by the amount of the angle it makes, so that the display surface faces the camera.

[0011] The adjustment unit 124, described below, changes the orientation of the display surface to a predetermined reference orientation when the position of the two-dimensional code C in the captured image is at a predetermined position.

[0012] The predetermined position described below is the edge position in the captured image.

[0013] The reference orientation described below is the orientation perpendicular to the floor surface.

[0014] The reference orientation described below is the orientation of camera 11 and other cameras adjacent to it.

[0015] The information processing method described below is performed by the information processing device 100 and includes an acquisition step, a detection step, a specification step, and an adjustment step. The acquisition step acquires the captured image captured by the camera 11. The detection step detects the two-dimensional code C that appears in the captured image. The specification step identifies the coordinate data indicating the position of the detected two-dimensional code C in the captured image. The adjustment step adjusts the orientation of the display surface of the two-dimensional code C based on the coordinate data.

[0016] Hereinafter, an information processing apparatus and information processing method according to an embodiment will be described with reference to the drawings. In the embodiment, the same parts are denoted by the same reference numerals, and redundant descriptions are omitted.

[0017] FIG. 1 is a diagram showing a configuration example of an information processing system according to an embodiment. The information processing system S shown in FIG. 1 is arranged, for example, in a space SP of a facility such as a factory or an office, which is relatively large and where there are a large number of people and various types of equipment. The information processing system S is thus a system that detects an object from a camera image in a facility where various information (such as the flow of people and equipment) is included in the camera image. In addition, in the information processing system S of the present disclosure, the detection process of the object is performed by detecting a two-dimensional code arranged for the purpose of improving the detection accuracy of the object and adding attribute information of the object.

[0018] As shown in FIG. 1, the information processing system S includes a lighting device 1, an information processing device 100, and an angle adjustment device 200. In the information processing system S shown in FIG. 1, the lighting device 1, the information processing device 100, and the angle adjustment device 200 are connected by a communication network such as Ethernet (registered trademark). Note that in FIG. 1, an example is shown in which the information processing device 100 is configured as a dedicated server device, but the information processing device 100 may be incorporated into a lighting control device that controls the lighting of a plurality of lighting devices 1.

[0019] The lighting device 1 is a lighting device that illuminates a predetermined space SP of a facility or the like. The lighting device 1 is, for example, a lighting device that is directly attached to the ceiling surface and irradiates the floor surface from the ceiling. Note that in FIG. 1, it is assumed that there is an object O equipped with an angle adjustment device 200 in the space SP. The object O is, for example, a movable facility such as a forklift or a cart, but may be a facility whose position is basically fixed in the space SP, such as a desk or a copy machine. In addition, a two-dimensional code C in which various information such as the attribute information of the object O is embedded is installed on the object O. The two-dimensional code C is, for example, a QR (Quick Response) code (registered trademark).

[0020] As shown in FIG. 1, the lighting device 1 includes a light source unit 10 and a camera 11. The light source unit 10 includes a light emitting element such as an LED (Light Emitting Diode), for example, and irradiates light from the ceiling toward the floor surface of the space SP.

[0021] Camera 11 is an imaging unit capable of capturing images of the spatial SP and is built into the lighting device 1. Camera 11 has a field of view that is directed from the ceiling towards the floor of the spatial SP. Camera 11 is, for example, a camera equipped with a fisheye lens. Camera 11 is not limited to a camera built into the lighting device 1, but may also be a camera installed independently in the spatial SP, such as a security camera.

[0022] The information processing device 100 is a server device that detects objects O and two-dimensional codes C present in spatial SP. The information processing device 100 may be a server device installed within a facility having spatial SP, or it may be a server device installed outside of said facility. Furthermore, the information processing device 100 may be implemented as a cloud system. In addition, the information processing device 100 may be incorporated into the lighting device 1, the lighting control device described above, and the angle adjustment device 200. The information processing device 100 controls the angle adjustment device 200 based on the image captured by the camera 11 so that the camera 11 can easily read the two-dimensional code C, but details of this will be described later.

[0023] The angle adjustment device 200 is installed on each object O on which a two-dimensional code C is provided, and adjusts the orientation of the display surface of the two-dimensional code C. Specifically, the angle adjustment device 200 changes the inclination angle (θ) with respect to the floor surface and the rotation angle (φ) in the plane of the display surface according to the control instructions of the information processing device 100. In other words, the rotation angle is the angle at which the display surface rotates around its center point as the axis of rotation when viewed from the front. The angle adjustment device 200 can be configured with any device capable of changing the inclination angle and the rotation angle.

[0024] Next, an example of the operation of the information processing system S will be described using Figure 1. The information processing system S in this disclosure detects a two-dimensional code C that appears in the captured image captured by the camera 11, identifies coordinate data indicating the position of the two-dimensional code C in the captured image, and adjusts the orientation of the display surface of the two-dimensional code C by controlling the angle adjustment device 200 based on the identified coordinate data.

[0025] Specifically, first, the information processing device 100 acquires the captured image taken by the camera 11 (step S1). The information processing device 100 acquires the captured image taken by the camera 11 frame by frame.

[0026] Next, the information processing device 100 detects a two-dimensional code C that appears in the acquired image (step S2). Specifically, the information processing device 100 detects the two-dimensional code C that appears in the image by image analysis such as template matching, and reads the information contained in the detected two-dimensional code C. The information contained in the two-dimensional code C includes information such as the type of object O (type of equipment (forklift, etc.)) and attribute information of object O (equipment name, owning department, etc.).

[0027] Next, the information processing device 100 identifies coordinate data indicating the position of the detected two-dimensional code C in the captured image (step S3). Specifically, the information processing device 100 identifies the two-dimensional coordinates of the X and Y axes, with the center of the captured image, which was captured from the ceiling to the floor, as the origin, as coordinate data.

[0028] Next, the information processing device 100 adjusts the orientation of the display surface of the two-dimensional code C by outputting a control instruction to the angle adjustment device 200 based on the identified coordinate data (step S4). Specifically, the information processing device 100 adjusts the display surface of the two-dimensional code C so that it faces the camera 11 (facing forward), but the details of this will be described later.

[0029] As described above, the information processing system S according to this embodiment automatically adjusts the orientation of the display surface of the two-dimensional code C based on the captured image of the current frame captured by the camera 11, so that in the captured image of the next frame, the display surface of the two-dimensional code C is oriented in a way that makes it easy to detect. In other words, the information processing system S according to this embodiment can improve the detection accuracy of the two-dimensional code C in the next frame.

[0030] Next, an example of the configuration of the information processing device 100 will be described using Figure 2. Figure 2 is a block diagram showing an example of the configuration of the information processing device 100 according to the embodiment. As shown in Figure 2, the information processing device 100 comprises a communication unit 110, a control unit 120, and a storage unit 130. The control unit 120 comprises an acquisition unit 121, a detection unit 122, a specification unit 123, and an adjustment unit 124. The storage unit 130 stores code information 131.

[0031] The communication unit 110 is a network device for performing wireless communication processing and wired communication processing. For example, the communication unit 110 transmits and receives various information to and from the lighting device 1 and the angle adjustment device 200 via a predetermined network.

[0032] Here, the information processing device 100 includes, for example, a computer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), flash memory, input / output ports, and various circuits.

[0033] The computer's CPU functions as the acquisition unit 121, detection unit 122, identification unit 123, and adjustment unit 124 of the control unit 120, for example, by reading and executing a program stored in ROM.

[0034] Furthermore, at least one or all of the functions of the acquisition unit 121, detection unit 122, identification unit 123, and adjustment unit 124 of the control unit 120 can be configured using hardware such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0035] Furthermore, the storage unit 130 can, for example, correspond to RAM or flash memory. RAM or flash memory can store code information 131, information on various programs, etc. The information processing device 100 may also acquire the above-mentioned programs and various information via other computers or portable recording media connected by wired or wireless networks.

[0036] Code information 131 is information about the registered two-dimensional code C. Figure 3 is a diagram showing an example of code information 131. As shown in Figure 4, code information 131 has items such as "code ID", "angle adjustment device", "coordinate data", and "current orientation".

[0037] "Code ID" is identification information that identifies each two-dimensional code C. "Angle adjustment device" is information that identifies the angle adjustment device 200 that adjusts the angle of the two-dimensional code C, and for example, a uniquely determined device ID is entered for each angle adjustment device 200. "Coordinate data" is information that indicates the position in the captured image, and is identified by the identification unit 123 described later, and is expressed by two-dimensional coordinates of the X and Y axes. "Current orientation" is information that indicates the current orientation of the display surface of the two-dimensional code C, and is the content of the last control instruction by the adjustment unit 124 described later, and is expressed by the tilt angle θ and rotation angle φ.

[0038] Next, the functions of the control unit 120 (acquisition unit 121, detection unit 122, identification unit 123, and adjustment unit 124) will be described.

[0039] The acquisition unit 121 acquires captured images of the spatial SP captured by the camera 11 of the lighting device 1. More specifically, the acquisition unit 121 acquires captured images of the spatial SP continuously at regular frame intervals, frame by frame.

[0040] The detection unit 122 detects a two-dimensional code C present in the spatial SP based on the acquired image. Specifically, the detection unit 122 detects the two-dimensional code C in the image by image analysis such as template matching, and reads the information contained in the detected two-dimensional code C. The information contained in the two-dimensional code C includes, for example, the type of object O (type of equipment (forklift, etc.)) and attribute information of object O (equipment name, owning department, etc.). The information contained in the two-dimensional code C may also include information that identifies the angle adjustment device 200 (such as the equipment ID mentioned above).

[0041] The identification unit 123 identifies coordinate data indicating the position of the detected two-dimensional code C in the captured image. Specifically, the identification unit 123 sets the X and Y axes with the center of the captured image, which is captured from the ceiling to the floor, as the origin. Then, the identification unit 123 identifies the center of the area of ​​the two-dimensional code C detected by the detection unit 122 as coordinate data represented in two dimensions.

[0042] The adjustment unit 124 adjusts the orientation of the display surface of the two-dimensional code C by outputting control instructions to the angle adjustment device 200 based on the identified coordinate data. Specifically, the adjustment unit 124 calculates the angle between the camera 11's field of view center axis (center of the captured image: origin) and the display surface of the two-dimensional code C based on the coordinate data, and adjusts the tilt angle θ and rotation angle φ, which are the orientation of the display surface, based on the angle.

[0043] Here, the angle adjustment process by the adjustment unit 124 will be explained using Figures 4 to 7. Figures 4 to 7 are diagrams illustrating the angle adjustment process by the adjustment unit 124.

[0044] First, using Figure 4, we will explain the concept of the angle between the camera 11's field of view central axis (center of the captured image: origin) and the display surface of the two-dimensional code C.

[0045] The upper part of Figure 4 shows the captured image, and the lower part shows a simulated elevation view of the captured image from the side. As shown in the upper part of Figure 4, it is assumed that a two-dimensional code C is visible in the captured image at position A (X:100, Y:200). The elevation view in the lower part of Figure 4 shows a first range R1, which covers the distance W1 from the center of the captured image to the edge of the image, and a second range R2, which covers the distance W2 from the center of the captured image to position A.

[0046] As shown in the lower part of Figure 4, the angle between the center axis of the camera 11's field of view and the display surface of the two-dimensional code C is expressed as the angle θ' between the line indicating the center of the captured image and the line from position A to the camera 11. The adjustment unit 124 then calculates the angle θ' using the coordinate data of position A identified by the identification unit 123 and the length from the camera 11 to the floor (the length of the line indicating the center of the captured image). The length from the camera 11 to the floor may be a predetermined value, or the two-dimensional code C may contain information about the height position of the two-dimensional code C, and the length from the camera 11 to the floor may be determined from that height position.

[0047] Next, we will explain the concepts of the tilt angle θ and rotation angle φ of the two-dimensional code C using Figure 5. Figure 5 shows a simplified representation of the captured image. As shown in Figure 5, the tilt angle θ is expressed as the tilt angle with respect to the XY plane, and its value range is -90° < θ < 90°. The rotation angle φ is expressed as the angle of rotation around the central axis of the display surface of the two-dimensional code C when it is positioned horizontally to the XY plane, and its value range is 90° < φ < 90°.

[0048] The adjustment unit 124 adjusts the orientation of the display surface by adjusting the tilt angle θ and rotation angle φ so that the two-dimensional code C faces the camera 11. In other words, the adjustment unit 124 adjusts the orientation of the display surface so that the display surface of the two-dimensional code C is perpendicular to the lens of the camera 11. Specifically, for the rotation angle φ, the adjustment unit 124 creates a line connecting the camera 11 and the center of the display surface, and determines the rotation angle φ such that this line passes through the center of any one side of the rectangular display surface. Next, the tilt angle θ will be explained using Figures 6 and 7.

[0049] Figures 6 and 7 show only the second range R2 of a simulated elevation view of the captured image from the side. As shown in Figure 6, the adjustment unit 124 tilts the display surface of the two-dimensional code C by an inclination angle θ, which is the same as the angle θ' described above. In other words, as shown in Figure 6, the inclination angle θ is set so that the display surface is perpendicular to the camera 11. As a result, the information processing device 100 can improve the detection accuracy of the two-dimensional code C because the camera 11 can capture an image of the front of the two-dimensional code C.

[0050] Here, we will explain the point where the angle θ' and the inclination angle θ are equal, using Figure 7. First, draw a line passing through the center of the display surface and parallel to the line that is the center of the camera 11's field of view. From this, we can see that the angle between the line passing through the center of the display surface and the line from the camera 11 to the display surface is θ'. Next, draw a line parallel to the display surface. The angle between this line parallel to the display surface and the line passing through the center of the display surface is 90°-θ'. Next, draw a line passing through the center of the display surface and parallel to the floor. As a result, the angle between the line passing through the center of the display surface and parallel to the floor and the line parallel to the display surface is the inclination angle θ, so θ = 90° - (90° - θ') = θ'. In other words, the adjustment unit 124 can orient the display surface perpendicular to the camera 11 by tilting the display surface by an inclination angle θ, which is the same angle as the angle θ'.

[0051] The adjustment unit 124 basically tilts the display surface by the angle θ', but if the position of the two-dimensional code C in the captured image is at a predetermined position, it does not tilt the display surface by the angle θ', but instead changes it to a predetermined reference orientation.

[0052] For example, if object O is a moving object, the adjustment unit 124 changes the orientation of the display surface to be perpendicular to the floor when the position of the two-dimensional code C in the captured image reaches the edge of the image. In other words, the adjustment unit 124 changes the orientation of the display surface so that it is parallel to the floor. This is to bring it closer to the orientation of the other camera 11 when the object will no longer be visible in the captured image in the next frame and will be visible in the captured image of the other camera 11. In other words, this is to avoid a situation where the display surface is not facing the other camera 11 if the current state of being pointed at the camera 11 is maintained. As a result, when the two-dimensional code C begins to appear in the captured image of the other camera 11, the two-dimensional code C can be detected from the captured image of the other camera 11 at an earlier stage.

[0053] The reference orientation was set to be perpendicular to the floor surface, but for example, if the position information of other cameras 11 can be obtained, the reference orientation may be set to the orientation of other cameras 11. In other words, when the position of the two-dimensional code C in the captured image reaches the edge of the image, the adjustment unit 124 orients its display surface so that it is perpendicular to the camera 11 of the captured image and to other cameras 11 adjacent to it. This makes it possible to detect the two-dimensional code C from captured images taken by other cameras 11 at an earlier stage.

[0054] Next, the processing procedure of the information processing device 100 according to the embodiment will be described using Figure 8. Figure 8 is a flowchart showing the processing procedure of the information processing device 100 according to the embodiment.

[0055] As shown in Figure 8, the control unit 120 acquires the captured image from the lighting device 1 captured by the camera 11 (step S101).

[0056] Next, the control unit 120 performs image analysis on the acquired image and detects the two-dimensional code C that appears in the image (step S102).

[0057] Next, the control unit 120 identifies coordinate data indicating the position of the detected two-dimensional code C in the captured image (step S103).

[0058] Next, the control unit 120 calculates the angle θ' formed from the identified coordinate data (step S104).

[0059] Next, the control unit 120 determines the tilt angle θ and rotation angle φ, which are the orientation of the display surface, based on the coordinate data and the angle θ' (step S105).

[0060] Next, the control unit 120 notifies the angle adjustment device 200 of an angle adjustment instruction using the determined inclination angle θ and rotation angle φ as control instructions, thereby adjusting the orientation of the display surface of the two-dimensional code C (step S106), and then terminates the process.

[0061] As described above, the information processing device 100 according to the embodiment comprises an acquisition unit 121, a detection unit 122, a specification unit 123, and an adjustment unit 124. The acquisition unit 121 acquires the captured image taken by the camera 11. The detection unit 122 detects the two-dimensional code C that appears in the captured image. The specification unit 123 identifies coordinate data indicating the position of the detected two-dimensional code C in the captured image. The adjustment unit 124 adjusts the orientation of the display surface of the two-dimensional code C based on the coordinate data. This improves the detection accuracy of the two-dimensional code C.

[0062] While embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0063] 1. Lighting device 10 Light source section 11 Cameras 100 Information Processing Devices 110 Communications Department 120 Control Unit 121 Acquisition Department 122 Detection unit 123 Specific part 124 Adjustment section 130 Storage section 131 Code Information 200 Angle adjustment device C 2D code S Information Processing System

Claims

1. An acquisition unit that acquires the captured image taken by the camera; A detection unit for detecting a two-dimensional code that appears in the captured image; A unit for identifying coordinate data that indicates the position of the detected two-dimensional code in the captured image; An adjustment unit that adjusts the orientation of the display surface of the two-dimensional code based on the coordinate data; An information processing device equipped with the following.

2. The adjustment unit is, Based on the coordinate data, the angle between the camera's field of view center axis and the display surface is calculated, and the orientation of the display surface is adjusted based on this angle. The information processing apparatus according to claim 1.

3. The adjustment unit is, The orientation of the display surface is tilted toward the camera by the amount of the aforementioned angle, so that the display surface faces the camera. The information processing apparatus according to claim 2.

4. The adjustment unit is, If the position of the two-dimensional code in the captured image is at a predetermined position, the orientation of the display surface is changed to a predetermined reference orientation. The information processing apparatus according to claim 1.

5. The aforementioned predetermined position is, The edge position in the aforementioned captured image. The information processing apparatus according to claim 4.

6. The aforementioned reference orientation is, It is oriented perpendicular to the floor surface. The information processing apparatus according to claim 4.

7. The aforementioned reference orientation is, This is the orientation of the aforementioned camera and other cameras adjacent to it. The information processing apparatus according to claim 4.

8. An information processing method performed by an information processing device, The acquisition process involves obtaining the captured image taken by the camera; A detection step for detecting a two-dimensional code that appears in the captured image; A process of identifying coordinate data indicating the position of the detected two-dimensional code in the captured image; An adjustment step to adjust the orientation of the display surface of the two-dimensional code based on the coordinate data; Information processing methods including

Images