POSITION TRACKING DEVICE, POSITION TRACKING METHOD AND POSITION TRACKING PROGRAM

Abstract

Position detection device (30), comprising: a person detection unit (14) for receiving an image recorded by a surveillance camera (11_1), for performing a process for detecting a person in the image and for outputting two-dimensional camera coordinates that indicate a position of the detected person; a coordinate transformation unit (15) for transforming the two-dimensional camera coordinates into three-dimensional coordinates that are represented in a predetermined common coordinate system; a character recognition unit (34) for recognizing a string (43) on a nameplate (42) of a device in an image taken by a portable camera (31) when the portable camera is carried by the person; a character search unit (35) for searching a layout diagram of the device for the recognized character string (43); a map coordinate determination unit (16b) for determining two-dimensional map coordinates based on a position where the string (43) is found if the recognized string (43) is found in the layout diagram, and for calculating the two-dimensional map coordinates based on the three-dimensional coordinates if the recognized string (43) is not found in the layout diagram; and a display control unit (17) for capturing a map and outputting image data in which position information is superimposed on the two-dimensional map coordinates of the map.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a position detecting apparatus, a position detecting method, and a position detecting program. BACKGROUND TO THE STATE OF THE ART

[0002] A system has been proposed that includes a plurality of surveillance cameras mounted on the ceiling of a factory or the like, such that an area as the image pickup area of ​​the plurality of surveillance cameras combined with each other includes a target area without any omission. In this system, the position coordinates of a worker, as a person, moving within the target area are acquired based on images captured by a large number of surveillance cameras facing the same direction. See, for example, Patent Literature 1.

[0003] Patent Literature 2 discloses an apparatus and a method for specifying installation conditions of a rotatable pan and tilt camera in a three-dimensional space.

[0004] Patent Literature 3 describes a device for better extraction of object features such as a person from an image with background, wherein the person is simultaneously captured by a plurality of image acquisition units with different viewing angles and the image elements inside and outside the person are evaluated taking into account the camera parameters. REFERENCES TO THE PRIOR ART OF THE TECHNICAL PATENT LITERATURE Patent Literature 1: JP 2017-34511 A (paragraph 0025 and Fig. 2, as an example). Patent Literature 2: WO 2009 / 110417 A1 Patent Literature 3: JP 2014-146247 A SUMMARY OF THE INVENTION TASK TO BE SOLVED BY THE INVENTION

[0005] However, the conventional system described above has the problem that a large number of surveillance cameras facing the same direction are required to determine the worker's position coordinates.

[0006] An object of the present disclosure is to provide a position detecting apparatus, a position detecting method, and a position detecting program that make it possible to solve the problem described above. MEANS TO SOLVED THE TASK

[0007] A position detection device in the present disclosure includes a person detection unit that receives a plurality of images captured by a plurality of surveillance cameras, executes a process for detecting a person in each of the plurality of images, and outputs a plurality of sets of two-dimensional camera coordinates indicating a position of the detected person, a coordinate transformation unit that transforms the plurality of sets of two-dimensional camera coordinates into a plurality of sets of three-dimensional coordinates represented in a predetermined common coordinate system, a map coordinate determination unit that generates two-dimensional map coordinates based on the plurality of sets of three-dimensional coordinates, and a display control unit that acquires a map and outputs image data,in which position information is superimposed over the two-dimensional map coordinates of the map.,

[0008] Another position detection device in the present disclosure includes a person detection unit that receives an image captured by a surveillance camera, performs a process for detecting a person in the image, and outputs two-dimensional camera coordinates indicating a position of the detected person; a coordinate transformation unit that transforms the two-dimensional camera coordinates into three-dimensional coordinates represented in a predetermined common coordinate system; a terminal position calculation unit that calculates terminal position coordinates indicating a position of a mobile terminal carried by the person based on a detection value of an inertial sensor of the mobile terminal; a map coordinate determination unit that calculates two-dimensional map coordinates based on the three-dimensional coordinates in periods in which the person is detected;and calculates the two-dimensional map coordinates based on the terminal position coordinates during periods when the person is not detected, and a display control unit that captures a map and outputs image information in which position information is superimposed on the two-dimensional map coordinates of the map.

[0009] Another position detection device in the present disclosure includes a person detection unit that receives an image captured by a surveillance camera, executes a process for detecting a person in the image, and outputs two-dimensional camera coordinates indicating a position of the detected person, a coordinate transformation unit that transforms the two-dimensional camera coordinates into three-dimensional coordinates represented in a predetermined common coordinate system, a character recognition unit that recognizes a character string on a nameplate of a device in a portable camera image captured by a portable camera when the portable camera is worn by the person, a character search unit that searches a layout diagram of the device for the recognized character string, a map coordinate determination unit,which determines two-dimensional map coordinates based on a position where the character string is found when the recognized character string is found in the layout diagram, and determines the two-dimensional map coordinates based on the three-dimensional coordinates when the recognized character string is not found in the layout diagram, and a display control unit that acquires a map and outputs image information in which position information is superimposed on the two-dimensional map coordinates of the map. EFFECT OF THE INVENTION

[0010] According to the present disclosure, the position coordinates of the worker can be detected with a simple configuration. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing a plurality of monitoring cameras as a configuration used for position detection by a position detection apparatus according to a first embodiment. Fig. 2A and Fig. 2B are diagrams showing the two-dimensional coordinates of a person detected on a map based on a plurality of images captured by the plurality of surveillance cameras. Fig. 3 is a functional block diagram schematically showing the configuration of the position detecting device according to the first embodiment. Fig. 4 is a diagram showing a hardware configuration of the position detecting device according to the first embodiment. Fig. 5 is a flowchart showing a process executed by a human detection unit of the position detection apparatus according to the first embodiment. Fig. 6 is a flowchart showing a process executed by a two-dimensional / three-dimensional coordinate transformation unit of the position detecting apparatus according to the first embodiment. Fig. 7 is a flowchart showing a process executed by a map coordinate determination unit of the position detecting apparatus according to the first embodiment. Fig. 8 is a diagram showing surveillance cameras and a mobile terminal in a configuration used for position detection by a position detection apparatus according to a second embodiment. Fig. 9A is a diagram showing the two-dimensional coordinates of a person detected on a map based on the images taken by the surveillance cameras, and Fig. Figure 9B is a diagram showing the two-dimensional coordinates of a person detected by pedestrian dead reckoning (PDR) on a map. Fig. 10 is a functional block diagram schematically showing the configuration of the position detecting device according to the second embodiment. Fig. 11 is a flowchart showing a process executed by a coordinate calculation unit of the position detecting device according to the second embodiment. Fig. 12 is a flowchart showing a process executed by a map coordinate determination unit of the position detecting apparatus according to the second embodiment. Fig. 13 is a diagram showing monitoring cameras and a wearable camera as a configuration used for position detection by a position detection device according to a third embodiment. Fig. Figure 14A is a diagram showing an example of an instrument panel, instrument, and nameplate. Fig. Figure 14B is a diagram showing 2D map coordinates obtained from an image of the vehicle nameplate on a map. Fig. 15 is a functional block diagram schematically showing the configuration of the position detecting device according to the third embodiment. Fig. 16 is a flowchart showing a process executed by a character recognition unit of the position detecting apparatus according to the third embodiment. Fig. 17 is a flowchart showing a process executed by a character search unit of the position detecting apparatus according to the third embodiment. MODE FOR CARRYING OUT THE INVENTION

[0011] A position detection device, a position detection method, and a position detection program according to each embodiment will be described below with reference to the drawings. The following embodiments are only examples, and it is possible to combine embodiments as appropriate and modify each embodiment as appropriate. In the drawings, components that are the same as or similar to each other are assigned the same reference numerals. First embodiment

[0012] Fig. 1 is a diagram showing a plurality of monitoring cameras 11_1-11_n (n is a positive integer) as a configuration used for position detection by a position detection device 10 according to a first embodiment. The monitoring cameras 11_1-11_n are installed in a predetermined area 80. The instrument panels 40 and the machines 50 are placed in the area 80. The monitoring cameras 11_1-11_n are fixed cameras. Although the monitoring cameras 11_1-11_n may be panning PTZ cameras, in this case, the monitoring cameras 11_1-11_n must be equipped with a function for notifying the position detection device 10 of camera parameters. The surveillance cameras 11_1 - 11_n each record images from image recording areas R1 - Rn and transmit images I1- I nto the position detection device 10. The position detection device 10 calculates two-dimensional (2D) map coordinates (X, Y) of a person 90 based on the images I1 - I n and generates image data to cause a display device to display information indicating the 2D map coordinates (X, Y) on a map 81 of the area 80. The area is, for example, the interior of a factory. The person 90 is, for example, a worker.

[0013] Fig. 2A and Fig. 2B are diagrams showing the 2D map coordinates (X, Y) of the person 90 obtained based on a plurality of images I1 - I n recorded by the multitude of surveillance cameras 11_1 - 11_n on map 81 of area 80. Fig. Figure 2A shows an example of the 2D map coordinates (X, Y) calculated using three surveillance cameras 11_1, 11_2 and 11_n, and Fig. Figure 2B shows an example of the 2D map coordinates (X, Y) calculated using two surveillance cameras 11_1 and 11_n. The map 81 of the area 80 in the Fig. 2A and Fig. 2B is obtained from an external storage device. However, the map 81 can also be stored in a storage device of the position detection device 10.

[0014] Fig. Figure 3 is a functional block diagram schematically showing the configuration of the position detecting device 10 according to the first embodiment. The position detecting device 10 is a device capable of executing a position detecting method according to the first embodiment. The position detecting device 10 is capable of executing the position detecting method according to the first embodiment by executing a position detecting program. As shown in Fig. 3, the position detection device 10 includes an image receiving unit 13, a person detection unit 14, a coordinate transformation unit 15, a map coordinate determination unit 16, and a display control unit 17.

[0015] The image receiving unit 13 receives the plurality of images I1 - I recorded by the plurality of surveillance cameras 11_1 - 11_n and transmitted by the image transmission units 12_1 - 12_n. n and gives the images I1 - I n to the person detection unit 14. The image receiving unit 13 is also referred to as a communication circuit or communication interface.

[0016] The person detection unit 14 receives the images I1 - I n , performs a process of detecting the person 90 in each of the images I1 - I n and outputs a variety of sets of two-dimensional (2D) camera coordinates (u1, v1) - (u n , v n) that indicate the position of the detected person 90. The coordinate systems of the 2D camera coordinates (u1, v1) - (u n , v n ) differ from each other.

[0017] The coordinate transformation unit 15 is a coordinate transformation unit that transforms two-dimensional (2D) coordinates into three-dimensional (3D) coordinates. The coordinate transformation unit 15 transforms the plurality of sets of 2D camera coordinates (u1, v1) - (u n , v n ) into a variety of sets of 3D coordinates (X1, Y1, Z1) - (X n , Y n , Z n ), which are represented in a given common coordinate system. The common coordinate system is, for example, a world coordinate system.

[0018] The map coordinate determination unit 16 generates 2D map coordinates (X, Y) based on the plurality of sets of 3D coordinates (X1, Y1, Z1) - (X n , Y n, Z n ). The map coordinate determination unit 16 calculates the 2D map coordinates (X, Y) using a sum average of the coordinate values ​​of the plurality of sets of 3D coordinates (X1, Y1, Z1) - (X n , Y n , Z n ).

[0019] The display control unit 17 outputs image data in which position information on the 2D map coordinates (X, Y) is superimposed on the map 81 of the area 80. A display device 18 displays the map 81 of the area 80 and the position information on the 2D map coordinates (X, Y). In the example of Fig. 3, the map 81 is displayed based on map information L obtained from an external storage device.

[0020] Fig. 4 is a diagram showing a hardware configuration of the position detecting device 10 according to the first embodiment. As shown in Fig. As shown in Figure 4, the position detection device 10 includes a processor 101, such as a CPU (Central Processing Unit), a memory 102 as a volatile storage device, a non-volatile storage device 103, such as a hard disk drive (HDD) or a solid state drive (SSD), and a communication unit 104 that performs communication with the outside world. The memory 102 is a volatile semiconductor memory, such as a RAM (Random Access Memory).

[0021] The functions of position detection device 10 are implemented by processing circuits. The processing circuits can be either dedicated hardware or processor 101, which executes a program stored in memory 102. Processor 101 can be a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (digital signal processor).

[0022] In the case where the processing circuit is dedicated hardware, the processing circuit is, for example, a single circuit, a combined circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of some of these circuits.

[0023] If the processing circuit is processor 101, the position detection program is implemented by software, firmware, or a combination of software and firmware. The software and firmware are described as programs and stored in memory 102. The processor 101 performs the functions of the Fig. 3 by reading and executing the position detection program stored in memory 102.

[0024] It is also possible to implement part of the position detection device 10 using dedicated hardware and part of the position detection device 10 using software or firmware. As described above, the processing circuits can implement the functions described above using hardware, software, firmware, or a combination of some of these means.

[0025] Fig. Figure 5 is a flowchart showing a process executed by the human detection unit 14 of the position detection device 10 according to the first embodiment. The human detection unit 14 first receives the images I1 - I n(Step S11). Subsequently, the person detection unit 14 performs a process for detecting the person 90 by sequentially moving a detection window 92 in an image 91 of each frame (steps S12-S14). Specifically, the person detection unit 14 repeats a process (steps S12-S15) of calculating an HOG (Histograms of Oriented Gradients) feature value as a feature value obtained by representing gradient directions of luminance (color, brightness) in a local area as a histogram by successively moving the detection window 92 in the image 91 of each frame, making a judgment by SVM (Support Vector Machine) as a pattern recognition model using supervised learning, and judging whether the person 90 has been successfully detected or not. The person detection unit 14 then outputs the 2D camera coordinates (u, v) as 2D coordinates of the person 90.

[0026] Fig. 6 is a flowchart showing a process performed by the coordinate transformation unit 15 of the position detection device 10 according to the first embodiment. First, the coordinate transformation unit 15 receives the 2D camera coordinates (u, v) of the person 90 on the captured image from the person detection unit 14 (step S21), acquires [R|t] representing position and attitude information about the surveillance camera (step S22), and acquires the internal parameters A of the surveillance camera (step S23). Subsequently, an inverse transformation of the perspective projection is performed according to the following expression (1) (step S24): (uv1)=A[R|t→](XYZ1)

[0027] Here it says A for the internal parameters of the surveillance camera, and [R|t→] stands for external parameters of the surveillance camera.

[0028] Subsequently, the coordinate transformation unit 15 outputs the 3D coordinates (X, Y, Z) of the person 90.

[0029] Fig. 7 is a flowchart showing a process executed by the map coordinate determination unit 16 of the position detection device 10 according to the first embodiment. The map coordinate determination unit 16 acquires the 3D coordinates (X1, Y1, Z1) according to the image of the monitoring camera No. 1, the 3D coordinates (X2, Y2, Z2) according to the image of the monitoring camera No. 2, and the 3D coordinates (X n , Y n , Z n ) according to the image of surveillance camera No. n (steps S31 to S33).

[0030] Then, the map coordinate determination unit 16 counts the number of coordinate sets that have a value among the 3D coordinates (X1, Y1, Z1) - (X n , Y n , Z n ) (ie count M).

[0031] Then, the map coordinate determination unit 16 outputs the values ​​of the 2D map coordinates (X, Y) based on the values ​​of the 3D coordinates (X1, Y1, Z1) - (X n , Y n , Z n ) using weighted average calculation formulas represented by the following expressions: X=Sum(X1+X2+⋯+Xn)M Y=Sum(Y1+Y2+⋯+Yn)M.

[0032] Then, the map coordinate determination unit 16 outputs the values ​​of the 2D map coordinates (X, Y).

[0033] As described above, by using the position detecting device 10, the position detecting method and the position detecting program according to the first embodiment, the position of the person 90 can be detected based on the images I1 - I n are recorded by the multitude of surveillance cameras 11_1 - 11_n, which differ in their position.

[0034] In addition, the accuracy of position detection can be increased by using multiple surveillance cameras 11_1 - 11_n.

[0035] Since the camera parameters of the multiple surveillance cameras 11_1 - 11_n do not have to match, the position of a person can be determined with high accuracy by using images from existing surveillance cameras. Second embodiment

[0036] Fig. 8 is a diagram showing the monitoring cameras 11_1-11_n and a mobile terminal 21 in a configuration used for position detection by a position detection device 20 according to a second embodiment. The mobile terminal 21 is carried by the person 90 as a detection target. One or more monitoring cameras 11_1-11_n have been installed in the predetermined area 80 where the instrument panels 40 and the machines 50 have been placed. The monitoring cameras 11_1-11_n may also be existing cameras. The monitoring cameras 11_1-11_n each capture images of the image capture areas R1-Rn and transmit the images I1-I n to the position detection device 20. The position detection device 20 calculates the 2D map coordinates (X, Y) of the person 90 based on the images I1 - I nand generates the image data to cause the display device 18 to display the information indicating the 2D map coordinates (X, Y) on the map 81 of the area 80.

[0037] Fig. 9A is a diagram showing the 2D map coordinates (X, Y) of the person 90 obtained based on the images I1 - I taken by the surveillance cameras 11_1 - 11_n. n recorded on map 81 of area 80. Fig. 9B is a diagram showing 2D map coordinates (X, Y) based on 2D coordinates (Xp, Yp) of person 90 detected by pedestrian dead reckoning (PDR) on map 81 of area 80. Fig. Figure 9A shows an example of the 2D map coordinates (X, Y) calculated using two surveillance cameras 11_1 and 11_n, and Fig. 9B shows the 2D map coordinates (X, Y) obtained as a result of position calculation by PDR after the person 90 moved outside the image pickup areas of the surveillance cameras.

[0038] Fig. Figure 10 is a functional block diagram schematically showing the configuration of the position detecting device 20 according to the second embodiment. The position detecting device 20 is a device capable of executing a position detecting method according to the second embodiment. The position detecting device 20 is capable of executing the position detecting method according to the second embodiment by executing a position detecting program. As shown in Fig. 10, the position detection device 20 includes the image receiving unit 13, the person detection unit 14, the coordinate transformation unit 15, a detection value receiving unit 23, a terminal position calculation unit 24, a map coordinate determination unit 16a, and the display control unit 17. The hardware configuration of the position detection device 20 is the same as in Fig. 4.

[0039] The image receiving unit 13 receives the images I1 captured by one or more surveillance cameras 11_1 and transmitted by the image transmitting unit 12_1, and sends the images I1 to the person detection unit 14. The person detection unit 14 receives the images I1, performs a process for detecting the person 90 in the images I1, and outputs the 2D camera coordinates (u1, v1) indicating the position of the detected person 90. The coordinate transformation unit 15 is a 2D / 3D coordinate transformation unit. The coordinate transformation unit 15 transforms the 2D camera coordinates (u1, v1) into 3D coordinates (X1, Y1, Z1) represented in a predetermined common coordinate system.

[0040] The detection value receiving unit 23 receives detection values ​​as sensor values ​​from an inertial sensor 21a of the mobile terminal 21 worn by the person 90 and outputs the detection values ​​to the terminal position calculation unit 24. The inertial sensor 21a is a device capable of detecting the translational movement and the rotational movement in directions of, for example, three mutually orthogonal axes. Generally, the inertial sensor is a device that detects the translational movement with an acceleration sensor [m / s 2 ] and the rotational movement is recorded with an angular velocity sensor (gyroscope) [deg / sec].

[0041] The terminal position calculation unit 24 calculates the terminal position coordinates (X p , Y p ) representing the position of the mobile terminal 21.

[0042] The map coordinate determination unit 16a calculates the 2D map coordinates (X, Y) based on the 3D coordinates in periods in which the person 90 is detected, and calculates the 2D map coordinates (X, Y) based on the terminal position coordinates (X p , Y p ) in periods when person 90 is not recorded.

[0043] The display control unit 17 outputs the image data in which the position information at the 2D map coordinates (X, Y) is superimposed on the map 81 of the area 80. The display device 18 displays the map 81 of the area 80 and the position information at the 2D map coordinates (X, Y). The map 81 is displayed based on the map information L acquired from the external storage device.

[0044] Fig. I1 is a flowchart showing a process executed by the terminal position calculation unit 24 of the position detecting device 20 according to the second embodiment. In Fig. 11, the terminal position calculation unit 24 calculates a rotation matrix from the detection values ​​(step S42), transforms the attitude of the mobile terminal 21 (step S43), calculates the acceleration of the mobile terminal 21 (step S44), calculates the displacement by double integration of the acceleration (step S45), and outputs the terminal position coordinates (X p , Y p ) based on the displacement (step S46). The terminal position calculation unit 24 repeats the above process (steps S42 to S46) until the position detection by the monitoring cameras is restarted (step S41).

[0045] Fig. 12 is a flowchart showing a process executed by the map coordinate determination unit 16a of the position detection device 20 according to the second embodiment. The map coordinate determination unit 16a receives images in a loop process (step S51) and outputs the 2D map coordinates (X, Y) based on the images (step S54) when the person 90 is detected (YES in step S53), or outputs the terminal position coordinates (X p , Y p ) obtained by PDR based on the detection values ​​of the inertial sensor 21a as 2D map coordinates (X, Y) (step S55) when the person is not detected (NO in step S53).

[0046] As described above, by using the position detecting device 20, the position detecting method, and the position detecting program according to the second embodiment, it is possible to output the 2D map coordinates (X, Y) by a method with relatively high accuracy based on the images I1 or the like when the position of the person 90 can be detected based on the images from one or more monitoring cameras, and it is possible to output the terminal position coordinates calculated based on the detection values ​​of the inertial sensor 21a as the 2D map coordinates (X, Y) when the person 90 is outside the image pickup areas.

[0047] In addition, the disadvantage of position detection by using PDR with relatively low accuracy can be mitigated by a countermeasure such as comparing the PDR calculation result with calculation results in the past and reporting that the accuracy is low when an error is greater than or equal to a predetermined value. Third embodiment

[0048] Fig. 13 is a diagram showing the monitoring cameras 11_1 - 11_n and a portable camera 31 in a configuration used for position detection by a position detection device 30 according to a third embodiment. The portable camera 31 is a small camera that captures an image in the direction of the line of sight of the worker as person 90 and is also referred to as smart glasses. The portable camera 31 is worn by the person 90. One or more monitoring cameras 11_1 - 11_n have been installed in the predetermined area 80 in which the instrument panels 40 and the machines 50 have been set up. The monitoring cameras 11_1 - 11_n may also be pre-existing cameras. The monitoring cameras 11_1 - 11_n each capture images of the image capture areas R1 - Rn and transmit the images I1 - I nto the position detection device 10. The position detection device 30 calculates the 2D map coordinates (X, Y) of the person 90 based on the images I1 - I n and generates image data to cause the display device to display the information indicating the 2D map coordinates (X, Y) on the map 81 of the area 80.

[0049] Fig. 14A is a schematic diagram of an example of the instrument panel 40, a device 41, e.g., an instrument, and a nameplate 42. Fig. 14B is a diagram showing the 2D map coordinates (X, Y) on the map 81 of the area 80 based on a character string recognized using an image of the nameplate 42.

[0050] Fig. Figure 15 is a functional block diagram schematically showing the configuration of the position detecting device 30 according to the third embodiment. The position detecting device 30 is a device capable of executing a position detecting method according to the third embodiment. The position detecting device 30 is capable of executing the position detecting method by executing a position detecting program. As shown in Fig. 15, the position detection device 30 includes the image receiving unit 13, the person detection unit 14, the coordinate transformation unit 15, an image receiving unit 33, a character recognition unit 34, a character search unit 35, a map coordinate determination unit 16b, and the display control unit 17. The hardware configuration of the position detection device 30 is the same as in Fig. 4.

[0051] The person detection unit 14 receives an image I1 captured by the surveillance camera 11_1, executes the process for detecting the person 90 in the images I1, and outputs the 2D camera coordinates (u1, v1) indicating the position of the detected person 90.

[0052] The coordinate transformation unit 15 transforms the 2D camera coordinates into 3D coordinates (X1, Y1, Z1), which are represented in a predetermined common coordinate system.

[0053] The character recognition unit 34 recognizes the character string 43 on the type plate 42 of the device 41 in an image I taken by the portable camera 31 w when the portable camera 31 is carried by the person 90.

[0054] The character search unit 35 searches for the recognized character string 43 in a layout diagram of the devices in the area 80 (e.g., in the map 81 describing the device layout).

[0055] If the recognized character string 43 is found in the layout diagram, the map coordinate determination unit 16b determines the 2D map coordinates (X, Y) based on the position where the character string 43 is found. If the recognized character string 43 is not found in the layout diagram, the map coordinate determination unit 16b calculates the 2D map coordinates (X, Y) based on the 3D coordinates.

[0056] The display control unit 17 outputs the image data in which the position information of the 2D map coordinates (X, Y) on the map 81 of the area 80 is displayed.

[0057] Fig. 16 is a flowchart showing a process performed by the character recognition unit 34 of the position detection device 30 according to the third embodiment. The character recognition unit 34 recognizes a character string region (step S61), divides the character string region into one-character regions (step S62), performs character pattern matching (step S63), determines a character (step S64), and judges whether or not there is the next character (step S65). If the next character is present (YES in step S65), the character recognition unit 34 repeats steps S63 to S65. If there is no next character (NO in step S65), the character recognition unit 34 outputs the character string (step S66).

[0058] Fig.17 is a flowchart showing a process executed by the character search unit 35 of the position detection device 30 according to the third embodiment. The character search unit 35 acquires the layout diagram of the map 81 of the area 80 (step S71), acquires the character string (step S72), and searches the layout diagram for a character string that matches the acquired character string (step S73). If there is a matching character string (YES in step S74), the character search unit 35 converts the character string into 2D coordinates (step S75) and outputs the 2D map coordinates (X, Y) indicating the person's position. If there is no matching character string (NO in step S74), the character search unit 35 ends the character search.

[0059] As described above, using the position detection device 30, the position detection method, and the position detection program according to the third embodiment, when the character string 43 on the nameplate 42 is found based on the image from the portable camera 31, the position of the nameplate 42 is output as 2D map coordinates (X, Y). If the character string 43 on the nameplate 42 is not found, the position of the person 90 is output as 2D map coordinates (X, Y) based on the image from the surveillance camera. Through such control, the accuracy of position detection can be increased. In other words, the accuracy of position detection can be increased because the position of the nameplate 42 is used for position detection with priority over the calculation result of the captured image.

[0060] Even if the person is outside the image capture range, the coordinates can be calculated from the image of the portable camera 31. DESCRIPTION OF REFERENCE SYMBOLS

[0061] 10, 20, 30: Position detection device, 11_1 - 11_n: Surveillance camera, 14: Person detection unit, 15: Coordinate transformation unit, 16, 16a, 16b: Map coordinate determination unit, 17: Display control unit, 18: Display device, 21: Mobile terminal, 21a: Inertial sensor, 24: Terminal position calculation unit, 31: Portable camera, 34: Character recognition unit, 35: Character search unit, 41: Device (instrument), 42: Nameplate, 43: Character string, 90: Person, I1 - I n : Image, I w : Image of the portable camera, (u1, v1) - (u n , v n ): two-dimensional camera coordinates (2D camera coordinates), (X1, Y1, Z1) - (X n , Y n , Z n): three-dimensional coordinates (3D coordinates), (X, Y): two-dimensional map coordinates (2D map coordinates), (X p , Y p ): Terminal position coordinates, (X w , Y w ): Coordinates of the portable camera.

Claims

[1] Position detecting device (30) comprising: a person detection unit (14) for receiving an image captured by a surveillance camera (11_1), for executing a process for detecting a person in the image, and for outputting two-dimensional camera coordinates indicating a position of the detected person; a coordinate transformation unit (15) for transforming the two-dimensional camera coordinates into three-dimensional coordinates represented in a predetermined common coordinate system; a character recognition unit (34) for recognizing a character string (43) on a nameplate (42) of a device in an image of a portable camera (31) taken by a portable camera (31) when the portable camera is worn by the person; a character search unit (35) for searching a layout diagram of the device for the recognized character string (43); a map coordinate determining unit (16b) for determining two-dimensional map coordinates based on a position where the character string (43) is found when the recognized character string (43) is found in the layout diagram, and for calculating the two-dimensional map coordinates based on the three-dimensional coordinates when the recognized character string (43) is not found in the layout diagram; and a display control unit (17) for capturing a map and outputting image data in which position information is superimposed on the two-dimensional map coordinates of the map. [2] A position detecting method performed by a position detecting device (30), the method comprising: a step of receiving an image captured by a surveillance camera (11_1), executing a process for detecting a person in the image, and outputting two-dimensional camera coordinates indicating a position of the detected person; a step of converting the two-dimensional camera coordinates into three-dimensional coordinates represented in a predetermined common coordinate system; a step of recognizing a character string (43) on a nameplate (42) of a device in an image of a portable camera (31) taken by a portable camera (31) when the portable camera is worn by the person; a step of searching for a position of the person based on a result of matching between the recognized character string (43) and a character string included in a layout diagram of the device; a step of determining two-dimensional map coordinates based on a position where the character string (43) is found when the recognized character string (43) is found in the layout diagram, and calculating the two-dimensional map coordinates based on the three-dimensional coordinates when the recognized character string (43) is not found in the layout diagram; and a step of capturing a map and outputting image data in which position information is superimposed on the two-dimensional map coordinates of the map. [3] A position detecting program for causing a computer to execute the position detecting method of claim 2.

Images