Positioning system

The smart electronic curved mirror system addresses positioning inaccuracies by combining image recognition and wireless communication to enhance location estimation in dynamic environments, overcoming sensor malfunctions and errors.

JP7576836B2Active Publication Date: 2024-11-01NAT INST OF INFORMATION & COMM TECH
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2021006417
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-01-19
Publication Date
2024-11-01
Estimated Expiration
2041-01-19

AI Technical Summary

Technical Problem

Conventional positioning systems face difficulties in accurately estimating a moving object's position due to sensor malfunctions and accumulated errors caused by changes in the surrounding environment or conditions, especially in dynamic scenarios like self-driving vehicles.

Method used

A positioning system utilizing a smart electronic curved mirror with a camera and wireless device that generates image position information and autonomously acquires position data, combining image recognition with wireless communication to enhance location accuracy.

Benefits of technology

The system effectively suppresses sensor malfunctions and accumulated errors, ensuring precise positioning even in changing environments by integrating image and wireless data to improve location estimation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007576836000001
    Figure 0007576836000001
  • Figure 0007576836000002
    Figure 0007576836000002
  • Figure 0007576836000003
    Figure 0007576836000003
Patent Text Reader

Abstract

To suppress a malfunction and accumulated errors in a sensor even if the environment or the situation changes according to the movement.SOLUTION: A positioning system according to an embodiment comprises: a sensor unit that acquires image information including a moving body and generates image position information indicating a position of the moving body, based on a relative position of the moving body in the image information; a server unit that acquires position information of the moving body based on the image position information in response to a request from a radio device provided in the moving body; and a radio unit that receives a request from the radio device and provides the position information of the moving body to the radio device.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The embodiments relate to a positioning system. [Background technology]

[0002] There is a strong desire to realize smart mobility for the elderly that will eliminate their anxieties about mobility and promote their participation in society and independence. Smart mobility for the elderly is a system that utilizes a wide variety of sensor information, such as cameras, LiDAR (Light Detection and Ranging), and GNSS / IMU devices, as well as infrastructure sensor information, in vehicles such as wheelchairs and nursing care assistance robots to achieve autonomous and automatic driving, primarily to support the mobility of the elderly.

[0003] While location information from satellite positioning devices is effective outdoors, a means of autonomously acquiring one's own position is necessary indoors. Known location acquisition means that can be used indoors include indoor positioning sensors, triangulation using wireless devices, and indoor positioning technologies using cameras, etc. Combining these positioning technologies is also effective for systems that require precise location information, such as self-driving cars indoors.

[0004] However, when considering moving vehicles such as self-driving cars, overcoming positioning errors that occur as the vehicle moves is a challenge. In particular, sensor malfunctions and accumulated errors caused by changes in the surrounding environment and conditions as the vehicle moves cannot be ignored, making it difficult to estimate the vehicle's own position. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent application 2019-18117 [Patent Document 2] Patent Publication No. 2008-176504 [Patent Document 3] Patent No. 5454573 Summary of the Invention [Problem to be solved by the invention]

[0006] As described above, conventional positioning systems have a problem in that it is difficult to estimate the self-position due to sensor malfunctions and accumulated errors caused by changes in the surrounding environment and situation. The embodiment described below has been made to solve this problem, and aims to provide a positioning system that can suppress sensor malfunctions and accumulated errors even when the environment and situation change due to movement. [Means for solving the problem]

[0007] The system includes a sensor unit that acquires image information including a moving object and generates image position information indicating the position of the moving object based on the relative position of the moving object in the image information, a server unit that acquires position information of the moving object based on the image position information in response to a request from a wireless device provided on the moving object, and a wireless unit that receives requests from the wireless device and provides the wireless device with position information of the moving object.

[0008] The sensor unit may acquire first image information and second image information obtained by photographing the moving object from different positions, and generate image position information based on the relative position of the area including the moving object in the first image information and the relative position of the area including the moving object in the second image information. Here, the sensor unit may generate the image position information based on the overlapping ratio of the area including the moving object in the first image information and the area including the moving object in the second image information.

[0009] The wireless unit may transmit to the wireless device operation instruction information indicating an operation to be performed by a visually identifiable moving object, and the sensor unit may determine an area including the moving object in the image information based on an operation performed by the moving object in response to the operation instruction information to generate image position information. In this case, the operation instruction information may include one or more of the moving mode, moving time, moving distance, and lighting status of the lights on the moving object. Here, the wireless unit may acquire, from the wireless device provided in the moving object, autonomous position information that is acquired by the wireless device autonomously and indicates the position of the wireless device, and the server unit may acquire the position information of the moving object based on the autonomous position information and the image position information. Effect of the Invention

[0010] According to the positioning system of the embodiment, it is possible to suppress malfunctions and accumulated errors of sensors even when the environment or situation changes due to movement. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an overview of a positioning system according to an embodiment. [Diagram 2] 1 is a block diagram showing a configuration of a positioning system according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an image recognition process in the positioning system according to the embodiment. [Figure 4] FIG. 1 is a diagram illustrating an application example of a positioning system according to an embodiment. [Diagram 5] FIG. 1 is a diagram illustrating an application example of a positioning system according to an embodiment. [Figure 6] FIG. 1 is a diagram illustrating an application example of a positioning system according to an embodiment. [Figure 7] 4 is a flowchart showing the operation of the positioning system according to the embodiment. [Figure 8] 4 is a flowchart showing the operation of the positioning system according to the embodiment. [Figure 9] 4 is a flowchart showing the operation of the positioning system according to the embodiment. [Figure 10] 4 is a flowchart showing the operation of the positioning system according to the embodiment. [Figure 11] 4 is a flowchart showing the operation of the positioning system according to the embodiment. [Figure 12] 4 is a flowchart showing the operation of the positioning system according to the embodiment. [Figure 13] FIG. 2 is a diagram illustrating the operation of the positioning system according to the embodiment. [Figure 14] FIG. 2 is a diagram illustrating the operation of the positioning system according to the embodiment. [Figure 15] FIG. 2 is a diagram illustrating the operation of the positioning system according to the embodiment. [Figure 16] FIG. 4 is a diagram illustrating an example of operation instruction information in the positioning system of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] (Overview of the embodiment) The positioning system of the embodiment mainly measures and tracks the position of a moving object indoors. As shown in FIG. 1, the positioning system 1 of the embodiment can measure the position of a moving object 30 using a smart electronic traffic mirror 10.

[0013] The smart electronic convex mirror 10 has a sensor device 11, a wireless device 12, and an antenna 13. The sensor device 11 has a camera as a sensor for photographing a moving body 30 such as a wheelchair, a pedestrian 41, an object 42, etc., and has a function of identifying the photographed object. The wireless device 12 and the antenna 13 correspond to a terminal device 32 and an antenna 33 mounted on the moving body 30, and have a function of communicating with the terminal device 32. The smart electronic convex mirror 10 is connected to a server 20 by wireless or wired connection. The server 20 has a function of measuring the position of the moving body 30 based on information acquired through the smart electronic convex mirror 10, and providing information on the measured position to the moving body 30. In the example shown in FIG. 1, the server 20 is connected to a plurality of smart electronic convex mirrors 10 via a network NW.

[0014] The terminal device 32 mounted on the moving body 30 may have a means for autonomously measuring its own position. When the terminal device 32 has a means for autonomously measuring its own position, the measurement accuracy does not need to be high. The terminal device 32 can obtain a level of position accuracy that enables automatic driving by using its own position information obtained by itself and its own position information obtained via the smart electronic traffic mirror 10.

[0015] (Configuration of the embodiment) Next, the configuration of the positioning system 1 of the embodiment will be described in detail with reference to Fig. 2. As shown in Fig. 2, the positioning system 1 of the embodiment includes a smart electronic convex mirror 10 and a server 20. The smart electronic convex mirror 10 includes a sensor device 11 and a wireless device 12.

[0016] The sensor device 11 has a camera CM, an image processing unit 112, a control unit 114, and a network interface (NW I / F) 116. The camera CM is, for example, a sensor capable of optically photographing a moving object. As shown in FIG. 2, the camera CM of the embodiment is configured to be capable of photographing a moving object 30 equipped with a terminal device 32. The image processing unit 112 is a calculation block that performs predetermined image processing on image information related to a moving image acquired by the camera CM. The image processing unit 112 also has a function of identifying the moving object 30 based on the image information processed by the image processing unit 112. The control unit 114 is a calculation block that controls the entire sensor device 11. The network interface 116 is an interface for connecting to the server 20 via a network NW. The network interface 116 may be wireless or wired.

[0017] FIG. 3 shows how the sensor device 11 photographs and identifies the moving body 30 and the like. When the camera CM photographs an image 50, the image processing unit 112 identifies the moving body 30, the pedestrian 41, and the object 42, and generates image information in which the identification frame 51 is superimposed on each of them. The identification frame 51 occupies a certain position in the image information photographed by the camera CM, and therefore makes it possible to obtain the relative position in the image information. That is, the image processing unit 112 generates image position information indicating the relative position of the moving body 30, the pedestrian 41, and the like identified by the identification frame 51 through the identification frame 51. For example, when the moving body 30 moves, the image processing unit 112 tracks the movement, and sequentially generates image information in which the identification frame 51 is also moved to follow the movement. That is, the image processing unit 112 can identify objects such as the moving body 30 in the image 50 acquired by the camera CM and track their movement. The image information on which the identification frame 51 is superimposed is sent to the server 20 via the network NW.

[0018] The wireless device 12 functions as a wireless access point that communicates with the terminal device 32. The wireless device 12 has an antenna 13, a wireless interface (wireless I / F) 122, a control unit 124, and a network interface (NW I / F) 126. The wireless interface 122 is an interface for connecting with the terminal device 32 by radio waves. The antenna 13 is connected to the wireless interface 122 and realizes wireless communication with the terminal device 32. The control unit 124 is an arithmetic block that identifies the terminal device 32 as a communication partner, and generates and decodes transmission and reception data. The network interface (NW I / F) 126 corresponds to the network interface 116, and is an interface for connecting with the server 20 via the network NW.

[0019] The server 20 has a function of identifying the moving object 30 and measuring its position via the sensor device 11. As shown in FIG. 2, the server 20 has a map information processing unit 210, a position information processing unit 220, an identification processing unit 230, a terminal registration unit 240, a storage unit 250, and a network interface (NW I / F) 260.

[0020] The map information processing unit 210 is a calculation block that generates map information of an area where the mobile object 30 (terminal device 32) is located and provides it to the mobile object 30. The position information processing unit 220 is a calculation block that acquires position information of the mobile object 30 via the sensor device 11. The identification processing unit 230 is a calculation block that identifies the mobile object 30 based on image information acquired via the sensor device 11. The terminal registration unit 240 is a calculation block that registers the mobile object 30 to be positioned in the storage unit 250 as a tracking target. The storage unit 250 is a storage medium that stores map information to be provided to the mobile object 30, position information of the mobile object 30, terminal information on the terminal device 32 provided in the mobile object 30, and the like. The network interface (NW I / F) 260 corresponds to the network interfaces 116 and 126, and is an interface for connecting to the sensor device 11 and the wireless device 12 via the network NW.

[0021] 2, the terminal device 32 is provided in the moving body 30 and has a wireless communication function and a self-position acquisition function. The terminal device 32 has an antenna 33, a map management unit 310, a position management unit 320, a control unit 330, a wireless interface (wireless I / F) 340, an input interface (input I / F) 350, and an output interface (output I / F) 360.

[0022] The map management unit 310 is a calculation block that acquires map information of a map including its own position and provides it to the user. The position management unit 320 is a calculation block that acquires position information related to its own position. The control unit 330 is a calculation block that controls communication with the wireless device 12 and the processes of each element. The wireless interface (wireless I / F) 340 corresponds to the wireless interface 122 and is an interface for connecting to the wireless device 12 via radio waves. The antenna 33 is connected to the wireless interface 340 and realizes wireless communication with the wireless device 12. The input interface 350 is an interface such as a keyboard or a touch panel that allows the user to input commands, etc. The output interface 360 ​​is an interface such as a display device that allows the user to provide image information.

[0023] (Application example of embodiment) Next, application examples of the embodiment and their operations will be described with reference to Fig. 4 to Fig. 16. The positioning system 1 of the embodiment can be applied to a case where a moving object 30 moves in a building B, for example, as shown in Fig. 4. The building B shown in Fig. 4 has a first floor FL1 having rooms R101-R104, and a second floor FL2 having rooms R501-R506, and allows movement between the upper and lower floors by an elevator EV.

[0024] On the first floor, FL1, there is sensor device 11. C1 ~11 C3 The sensor device 11 is installed on the second floor FL2 and captures the corridor of the first floor FL1. C4 ~11 C6 The sensor device 11 is installed and captures the corridor area on the second floor FL2. C1 ~11 C6 The sensor device 11 is connected to the server 20 via the network NW. C1 ~11 C6 are the shooting area V C1 ~V C6 It is possible to capture a range of.

[0025] As shown in FIG. 5, the first floor FL1 of the building B is equipped with antennas 13 A1 ~13 A2 Radio equipment 12 A1 ~12 A2 Similarly, on the second floor FL2, antenna 13 A3 ~13 A4 Radio equipment 12 A3 ~12 A4 The wireless device 12 A1 ~12 A4 The wireless device 12 is connected to the server 20 via the network NW. A1 ~12 A4 are the communication domains W A1 ~W A4 It is possible to communicate with a terminal device 32 located within the range of the antenna 32 .

[0026] 6 is a superimposition of FIG. 4 and FIG. 5. As shown in FIG. 6, the hallway of the building B through which the moving object 30 moves is C1 ~11 C6 Shooting area V C1 ~V C6 and wireless device 12 A1 ~12 A4 Communication area W A1 ~W A4 In the following description, it is assumed that a moving object 30 (indicated as "M1" in FIG. 6) travels from point R1 to R3 along the arrow, arrives at the elevator EV, gets off the elevator EV at the second floor FL2, and travels from point R4 to point R9.

[0027] Considering a route taken by the moving object 30 to reach the elevator EV via points R1, R2, and R3, the terminal device 32 has a communication area W A1 Radio equipment with 12 A1 At this time, the moving object 30 is capable of communicating with the sensor device 11. C1 , sensor device 11 C3 , sensor device 11 C2 Each shooting area V C1 , shooting area V C3 , shooting area V C2 and are photographed by each sensor device.

[0028] Similarly, when considering a route in which the moving object 30 moves from point R4 to point R9 and arrives at the room R506, the terminal device 32 first detects the communication area W A3 Radio equipment with 12 A3 Then, communication with the communication area W A4 Radio equipment with 12 A4 At this time, the moving object 30 is capable of communicating with the sensor device 11. C4 , sensor device 11 C5 , sensor device 11 C6 Each shooting area V C4 , shooting area V C5 , shooting area V C6 and are photographed by each sensor device.

[0029] The server 20 acquires autonomous position information indicating its own position autonomously acquired from the terminal device 32 via the wireless device 12. The server 20 also acquires image position information acquired based on image information including the terminal device 32 captured by the sensor device 11. As described above, the image position information indicates the relative position of the terminal device 32 in the image information captured by the sensor device 11, so the absolute position of the terminal device 32 can also be obtained through the relative position. In other words, the positioning system of the embodiment makes it possible to interpolate the position information acquired by the target object itself with position information objectively obtained from outside.

[0030] (Location registration operation) In this embodiment, at a start point (Start), the terminal device 32 acquires map information, acquires position information, and registers it in the server 20. The operation of the positioning system 1 in the application example of the embodiment will be described with reference to Figs. 6 and 7.

[0031] At the start point (Start), the terminal device 32 requests map information for the first floor FL1 from the server 20. The map management unit 310 of the terminal device 32 generates a map request signal including its own ID. The map request signal is transmitted to the server 20 by the terminal device 32 within the communication range W A1 Includes wireless devices 12 A1 Radio device 12 A1 The control unit 124 sends a map request signal to the server 20, and the map information processing unit 210 of the server 20 receives the map request signal (S100).

[0032] The map information processing unit 210 reads out the map information for the first floor FL1 from the storage unit 250. The map information processing unit 210 transmits the read out map information to the wireless device 12. A1 Send to radio equipment 12 A1 The map information is sent to the terminal device 32 (S110). The map management unit 310 of the terminal device 32 provides the received map information to the user via the output interface 360.

[0033] The position management unit 320 of the terminal device 32 autonomously acquires and holds position information (autonomous position information) indicating its own position. The position management unit 320 generates a position notification signal based on the autonomous position information (X1, Y1) indicating the position of the terminal device 32. The position notification signal includes its own ID and the autonomous position information (X1, Y1). The position notification signal indicates the terminal device 32 within a communication range W A1 Includes wireless devices 12 A1 Radio device 12 A1 The control unit 124 sends a position notification signal to the server 20, and the position information processing unit 220 of the server 20 receives the position notification signal (S120).

[0034] Next, the position information processing unit 220 determines whether the terminal device 32 is located in the shooting area V based on the autonomous position information (X1, Y1) of the terminal device 32 included in the received position notification signal. C1 Sensor device 11 included in C1 The sensor device 11 instructs the terminal device 32 to acquire image information. C1 The image processing unit 112 identifies the terminal device 32 from the image information acquired through the camera CM, and acquires image position information (X2, Y2) indicating its position. The image position information is position information indicating the position of the target acquired from the image information by the sensor device 11. The image processing unit 112 sends the image position information (X2, Y2) of the terminal device 32 acquired through the camera CM to the server 20, and the position information acquisition unit 220 accepts the image position information (X2, Y2) (S130).

[0035] The identification processing unit 230 compares (identification process) the autonomous position information (X1, Y1) and the image position information (X2, Y2) received by the position information processing unit 220 (S140). The identification process is performed from the viewpoint of whether or not the autonomous position information autonomously acquired and held by the terminal device 32 and the image position information of the terminal device 32 acquired from the image information acquired by the sensor device 11 indicate the same object.

[0036] If it is determined that the autonomous position information and the image position information indicate the same object, the terminal registration unit 240 associates the object indicated by the autonomous position information (X1, Y1) and the image position information (X2, Y2) with the ID of the terminal device 32 and stores them in the storage unit 250. A1 (S150) via the terminal device 32. At this point, the terminal device 32 can know that the autonomous position information has been confirmed by the image position information.

[0037] In addition, the terminal registration unit 240 notifies the sensor device 11 that the target indicating the image position information (X2, Y2) is the terminal device 32. C1 The sensor device 11 C1 records that the object indicated by the image position information (X2, Y2) is the terminal device 32 with the autonomous position information (X1, Y1) (S160).

[0038] (Existence of image location information from multiple sensor devices) In the above-described example, the terminal device 32 is connected to the sensor device 11. C1 In the above description, the moving object 30 is photographed by the camera 100 and the image position information (X2, Y2) is acquired, and the position information processing unit 220 receives the image position information (X2, Y2). However, this is not limited to this. For example, in FIG. 6, when the moving object 30 is located at the point R2, the moving object 30 is located in the photographing area V C1 Not only the shooting area V C3 That is, the moving object 30 is also located at the sensor device 11. C1 and sensor device 11 C3 Therefore, the position information processing unit 220 receives two pieces of image position information.

[0039] This will be described in detail with reference to Fig. 13. As shown in Fig. 13, for example, a moving object 32 is a sensor device 11. C1 and sensor device 11 C3 When an image is captured by the image processor 122, the image processor 122 detects the discrimination frame 52 C1 and identification frame 52 C3 That is, the sensor device 11C1 is identification frame 52 C1 The image position information is acquired using the sensor device 11 C3 is identification frame 52 C3 At this time, the sensor device 11 acquires image position information. C1 The discrimination frame 52 generated by C1 and sensor device 11 C3 The discrimination frame 52 generated by C3 may not match, resulting in two different image position information.

[0040] Similarly, for example, if a moving object 33 is C1 and sensor device 11 C3 When the image is captured by the image processor 122, the image processor 122 detects the discrimination frame 53 C1 and identification frame 53 C3 That is, the sensor device 11 C1 is identification frame 53 C1 The image position information is acquired using the sensor device 11 C3 is identification frame 53 C3 At this time, the sensor device 11 acquires image position information. C1 The discrimination frame 53 generated by C1 and sensor device 11 C3 The discrimination frame 53 generated by C3 may not match, resulting in two different image position information.

[0041] In particular, as shown in FIG. 13, when multiple moving bodies 32 and 33 exist in close proximity, if the autonomous position information and the image position information diverge too much, it becomes impossible to determine whether the terminal device corresponding to the autonomous position information obtained via the wireless device 12 corresponds to moving body 32 or moving body 33.

[0042] Therefore, in the positioning system of the embodiment, the identification determination is performed based on the overlapping ratio of a plurality of identification frames. For example, as shown in FIG. 14, an identification frame 53 in which different sensor devices are overlapped on image information is used. C1 and identification frame 53 C3 Duplicate part 53 OVIf the proportion of the identification frame that each sensor device occupies exceeds a predetermined threshold (e.g., 50%), it is determined that the two devices are identifying the same object, and the image location information acquired by one of the sensor devices is used for comparison with the autonomous location information. Conversely, if the overlapping portion does not exceed the threshold, it is determined that the sensor devices are not necessarily pointing to the same object, and processing is performed assuming that image location information cannot be acquired.

[0043] According to the positioning system of this embodiment, the autonomous position information acquired via the wireless device 12 and autonomously held by the terminal device 32 is compared with the image position information acquired based on the image information acquired via the sensor device 11, thereby improving the position accuracy of the autonomous position information.

[0044] (Operation when autonomous location information cannot be obtained) Next, the operation of the positioning system when the terminal device 32 cannot acquire its own location information will be described with reference to Fig. 6 and Fig. 8. In the following description, steps common to the operation described in Fig. 7 are denoted by common reference numerals, and duplicated description will be omitted.

[0045] At the start point (Start), the terminal device 32 requests map information for the first floor FL1 from the server 20. The map management unit 310 of the terminal device 32 generates a map request signal including its own ID. The map request signal is transmitted to the server 20 by the terminal device 32 within the communication range W A1 Includes wireless devices 12 A1 Radio device 12 A1 The control unit 124 sends a map request signal to the server 20, and the map information processing unit 210 of the server 20 receives the map request signal (S100).

[0046] The map information processing unit 210 reads out the map information for the first floor FL1 from the storage unit 250. The map information processing unit 210 transmits the read out map information to the wireless device 12. A1 Send to radio equipment 12 A1 The map information is sent to the terminal device 32 (S110). The map management unit 310 of the terminal device 32 provides the received map information to the user via the output interface 360.

[0047] The position management unit 320 of the terminal device 32 shown in FIG. 2 acquires and holds the autonomous position information, but may fail to hold the autonomous position information for some reason. In this case, the terminal device 32 requests image position information from the server 20. The position management unit 320 of the terminal device 32 generates a position request signal including its own ID. For example, when the mobile object 30 is located at point R2, the position request signal is generated by transmitting the position request signal to the server 20, which indicates that the terminal device 32 is located within a communication range W A1 Includes wireless devices 12 A1 and communication range W A2 Includes wireless devices 12 A2 The wireless device 12 A1 and / or wireless devices 12 A2 The control unit 124 sends a position request signal to the server 20, and the position information processing unit 220 of the server 20 accepts the position request signal (S235).

[0048] When the server 20 receives the position request signal, the server 20 starts a request terminal identification process (S240). The request terminal identification process includes two processes: issuing an operation instruction to the terminal device 32 and issuing an identification request to the sensor device 11.

[0049] The identification processing unit 230 of the server 20 sends an operation instruction signal to the position management unit 320 of the terminal device 32 via the wireless device 12. The operation instruction signal is a signal that instructs the mobile body 30 equipped with the wireless device 32 to perform a predetermined visually identifiable operation. For example, the operation instruction signal includes one or more of the following: the time or duration to perform the operation, the content of the operation (movement mode or distance, such as moving forward a predetermined distance or moving backward a predetermined distance), the presence or absence and state of lights equipped on the mobile body 30 (presence or absence of blinking, blinking interval, color of lights, etc.). The position management unit 320 provides the content of the operation instruction signal to the user using the mobile body 30 through the output interface 360 ​​in a manner that the user can understand. In response to this, the user performs an operation according to the content of the operation instruction signal provided by the output interface 360. For example, if the content of "moving forward a predetermined distance" is included in the operation instruction signal, the user performs an operation to move the mobile body 30 forward a predetermined distance. Also, if the operation instruction signal includes "blinking lights at a predetermined interval", the user blinks the lights so that they can be seen by the sensor device 11.

[0050] When the identification processing unit 230 of the server 20 sends the operation instruction signal, it sends an identification request for the moving object 30 to the sensor device 11. The camera CM always tracks the terminal device 32 that has performed the location registration operation and the moving object 30 equipped with it, so the moving object 30 that has performed the operation according to the operation instruction signal is photographed by the camera CM. When the image processing unit 112 of the sensor device 11 receives the identification request from the server 20, it searches for a terminal device that is performing an operation corresponding to the operation instruction signal. Specifically, it searches for an identification frame that shows a movement corresponding to the operation instruction signal among the identification frames generated by the image processing unit 112, and generates image position information corresponding to the corresponding identification frame. The identification processing unit 230 of the server 20 determines the ID of the terminal device 32 that corresponds to the image position information acquired by the sensor device 11. Through this series of operations, the terminal device 32 that issued the location request signal and its image position information are determined (S245).

[0051] When the terminal device 32 and the corresponding image position information are determined, the terminal registration unit 240 associates the target indicated by the image position information (X2, Y2) with the ID of the terminal device 32 and stores them in the storage unit 250. A1 or wireless device 12 A2 The terminal device 32 is notified via the terminal device 32 that the image position information (X2, Y2) has been registered (S250). Since the terminal device 32 acquires the latest image position information at this point, the position management unit 320 of the terminal device 32 can resume generating its own autonomous position information (X1, Y1) based on the acquired image position information (S260).

[0052] Here, the request terminal identification process (S240) will be described in detail with reference to Fig. 15 and Fig. 16. The sensor device 11 superimposes an identification frame for tracking the moving object 30, etc., on the image information acquired by the camera CM. At this time, in a situation where multiple moving objects are close to each other, it is considered that multiple moving objects are included in one identification frame. In this case, the sensor device 11 cannot track the position of each moving object, and it becomes difficult to generate image position information for each moving object.

[0053] 15, when the terminal device 32 fails to acquire autonomous position information and issues a position request signal to the server 20, the sensor device 11 generates image position information for the moving object 30 equipped with the terminal device 32 by using an identification frame. However, since the moving objects 30a and 30b are close to each other, the two moving objects 30a and 30b are included in one identification frame 51, and the image position information cannot be determined.

[0054] Therefore, in the positioning system of the embodiment, the server 20 transmits an action instruction signal to the terminal device 32. The action instruction signal includes the action to be performed by the moving body 30. FIG. 16 shows an example of the action instruction signal. In the example of the action instruction signal shown in FIG. 16, the action "stop" is specified as index 0, the condition "for X seconds", and the start and end times. That is, if the action instruction signal includes index 0, the instruction is "stop for X seconds". Similarly, the action "go straight" is specified as index 2, the condition "Xm", and the start and end times. That is, if the action instruction signal includes index 2, the instruction is "go straight (forward) Xm". The action instruction signal may include the content shown in FIG. 16 as data, or may include only the index number shown in FIG. 16. When the operation instruction signal includes only an index number, the identification processing unit 230 of the server 20, the image processing unit 112 of the sensor device 11, and the position management unit 320 of the terminal device 32 may hold a common table of operation instruction signals as shown in FIG. 16.

[0055] The user of the terminal device 32 obtains the contents of the operation instruction signal sent from the server 20 through the output interface 360. The user operates the moving object 30 according to the contents of the operation instruction signal. For example, if the index number is 2, the user moves the moving object 30 forward by Xm.

[0056] The image processor 112 of the sensor device 11 determines that the moving object performing the action according to the content of the action instruction signal is the moving object related to the identification request, tracks it using the identification frame, and generates image position information. That is, among the moving objects 30a and 30b, the moving object performing the forward action of Xm is tracked using the identification frame 51, and image position information is generated.

[0057] According to the positioning system of this embodiment, even if the terminal device 32 fails to acquire autonomous position information indicating its own position, which is acquired autonomously, the position of the terminal device 32 (the position of the mobile body 30) can be acquired based on the image position information acquired by the sensor device 11. In the above-described embodiment, the position management unit 320 of the terminal device 32 has a wireless communication function and a self-position acquisition function, but this is not limited to this. That is, the terminal device 32 does not have to have a self-position acquisition function. In this case, similar to the "operation when autonomous position information cannot be acquired" in FIG. 8, the terminal device 32 sends a position request signal to the server 20 (step S235 in FIG. 8), the server 20 starts the request terminal identification process (step S240 in FIG. 8), the server 20 sends an operation instruction signal to the terminal device 32, and the sensor device 11 searches for the terminal device and generates image position information (S245). As a result, even if the terminal device 32 of the mobile body 30 does not have a self-position acquisition function, the terminal device 32 can acquire its own position based on the image position information by the sensor device 11.

[0058] Next, communication operations performed in response to the movement of the mobile unit 30 will be described in detail with reference to FIG. 6 and FIG. 9 to FIG.

[0059] When the location registration is completed (FIG. 7), the terminal device 32 transmits destination information to the server 20 to request route information, and the server 20 accepts the request for route information from the terminal device 32 (S300). The server 20 generates route information based on the destination information transmitted from the terminal device 32, and transmits related map information to the terminal device 32 (S310).

[0060] Sensor device 11 C1 uses the discrimination frame 51 to track the moving object 30 in the captured image information, and generates image position information. The server 20 receives the image position information of the moving object 30, and stores it in the storage unit 250 (S400).

[0061] At the position R2, the moving body 30 detects the sensor device 11. C1 and sensor device 11 C2When a plurality of sensor devices 11 capture the moving object 30 (Yes in S410), the server 20 executes the classification process described with reference to Fig. 13 and Fig. 14 (S420). That is, the classification process is performed using the overlapping ratio of the classification frames 51. The server 20 transmits the determined image position information to the terminal device 32 (S430).

[0062] At the position R3, the moving body 30 detects the sensor device 11. C2 The server 20 receives the image of the sensor device 11. C2 The image position information of the moving object 30 is received from the image sensor 10 (S440).

[0063] When the moving object 30 moves from the position R4 to the position R5, the moving object 30 detects the sensor device 11. C4 The server 20 receives the image of the sensor device 11. C4 The image position information (X1, Y1) of the moving object 30 is received from (S500).

[0064] When the moving object 30 reaches the position R6, the moving object 30 detects the sensor device 11. C4 Shooting area V C4 The sensor device 11 was removed. C4 If the moving object 30 is not detected, it becomes impossible to track the moving object 30 (Yes in S510). In this case, the server 20 acquires the autonomous position information (X2, Y2) from the terminal device 32 (S520).

[0065] When the moving object 30 reaches the position R7, the moving object 30 detects the sensor device 11. C6 The server 20 captures the image using the sensor device 11. C6 The server 20 receives image position information (X3, Y3) of the moving object 30 from the terminal device 32 (S530). In addition, the server 20 obtains autonomous position information (X4, Y4) from the terminal device 32 (S540).

[0066] The server 20 executes a classification process using the image position information (X3, Y3) and the autonomous position information (X4, Y4) (S550). The classification process is carried out by using the autonomous position information autonomously acquired and held by the terminal device 32 and the autonomous position information of the sensor device 11. C6The determination is made from the viewpoint of whether the image information acquired by the terminal device 32 and the image position information acquired by the terminal device 32 show the same object.

[0067] If it is determined that the autonomous location information and the image location information indicate the same object, the server 20 associates the object indicated by the autonomous location information (X4, Y4) and the image location information (X3, Y3) with the ID of the terminal device 32 and stores them in the storage unit 250. In addition, the server 20 A4 via the terminal device 32 that the autonomous position information (X4, Y4) has been registered. At this point, the terminal device 32 can know that the autonomous position information has been confirmed by the image position information (S560).

[0068] In addition, the server 20 notifies the sensor device 11 that the target indicating the image position information (X3, Y3) is the terminal device 32. C6 The sensor device 11 C6 records that the object indicated by the image position information (X3, Y3) is the terminal device 32 with the autonomous position information (X4, Y4).

[0069] The moving object 30 includes a sensor device 11 C6 The server 20 captures the image using the sensor device 11. C6 The image position information of the moving object 30 is received from the image sensor 10 (S570).

[0070] In the above-described example, the sensor device 11 C4 However, if multiple moving objects are present in close proximity, it is difficult to obtain image position information. In that case, a request terminal identification process is performed.

[0071] When the moving object 30 moves from the position R4 to the position R5, the moving object 30 detects the sensor device 11. C4 The server 20 receives the image of the sensor device 11. C4 The image position information of the moving object 30 is received from the image sensor 10 (S600).

[0072] When the moving object 30 reaches the position R6, the moving object 30 detects the sensor device 11. C4 Shooting area V C4 The sensor device 11 was removed. C4 Then, when the moving object 30 reaches the position R7, the moving object 30 is not tracked by the sensor device 11. C6 The server 20 captures the image using the sensor device 11. C6 The server 20 receives image position information (X2, Y2) of the moving object 30 from the terminal device 32 (S620). In addition, the server 20 obtains autonomous position information from the terminal device 32 (S630).

[0073] The identification processing unit 230 of the server 20 sends an operation instruction signal to the location management unit 320 of the terminal device 32 via the wireless device 12. The location management unit 320 of the terminal device 32 provides the content of the operation instruction signal to the user using the mobile object 30 through the output interface 360 ​​in a manner that the user can understand. The user performs an operation according to the content of the operation instruction signal provided by the output interface 360 ​​(S640).

[0074] When the identification processing unit 230 of the server 20 sends the operation instruction signal, it sends an identification request for the moving object 30 to the sensor device 11. The camera CM always tracks the terminal device 32 that has performed the location registration operation and the moving object 30 equipped with it, so the moving object 30 that has performed the operation according to the operation instruction signal is photographed by the camera CM. When the image processing unit 112 of the sensor device 11 receives the identification request from the server 20, it searches for a terminal device that is performing an operation corresponding to the operation instruction signal. Specifically, it searches for an identification frame that shows a movement corresponding to the operation instruction signal among the identification frames generated by the image processing unit 112, and generates image position information corresponding to the corresponding identification frame. The identification processing unit 230 of the server 20 determines the ID of the terminal device 32 that corresponds to the image position information acquired by the sensor device 11. Through this series of operations, the terminal device 32 that issued the location request signal and its image position information are determined (S650).

[0075] The server 20 associates the target indicated by the autonomous position information (X1, Y1) and the image position information (X2, Y2) with the ID of the terminal device 32 and stores them in the storage unit 250. A4 via the terminal device 32 that the autonomous position information (X1, Y1) has been registered. At this point, the terminal device 32 can know that the autonomous position information has been confirmed by the image position information (S660).

[0076] In addition, the server 20 notifies the sensor device 11 that the target indicating the image position information (X2, Y2) is the terminal device 32. C6 The sensor device 11 C6 records that the object indicated by the image position information (X2, Y2) is the terminal device 32 with the autonomous position information (X1, Y1).

[0077] The moving object 30 includes a sensor device 11 C6 The server 20 captures the image using the sensor device 11. C6 The image position information of the moving object 30 is received from the image sensor 10 (S670).

[0078] In this way, according to the positioning system of the embodiment, even if tracking of a moving object by a sensor device is interrupted and image position information can no longer be acquired, tracking of the moving object can be restored based on the autonomous position information acquired by the moving object itself.

[0079] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0080] 1...positioning system, 10...smart electronic traffic mirror, 11...sensor device, CM...camera, 112...image processing unit, 114...control unit, 116...network interface, 12...wireless device, 122...wireless interface, 124...control unit, 126...network interface, 13...antenna, 20...server, 210...map information processing unit, 220...location information processing unit, 230...identification processing unit, 240...terminal registration unit, 250...memory unit, 260...network interface, 30...mobile body, 32...terminal device, 33...antenna, 310...map management unit, 320...location management unit, 330...control unit, 340...wireless interface, 350...input interface, 360...output interface, NW...network.

Claims

1. a sensor unit that acquires image information including a moving object and generates image position information indicating a position of the moving object based on a relative position of the moving object in the image information; a server unit that acquires location information of the mobile object based on the image location information in response to a request from a wireless device provided in the mobile object; a wireless unit that receives the request from the wireless device and provides location information of the moving object to the wireless device; the wireless unit transmits to the wireless device operation instruction information indicating an operation to be performed by the visually identifiable moving object; the sensor unit searches for an area including the moving object performing an action corresponding to the action instruction information in the image information, and generates a position corresponding to the area as the image position information. A positioning system comprising:

2. The positioning system according to claim 1, characterized in that the sensor unit acquires first image information and second image information obtained by photographing the moving object from different positions, and generates the image position information based on the relative position of an area including the moving object in the first image information and the relative position of an area including the moving object in the second image information.

3. The positioning system according to claim 2 , characterized in that the sensor unit generates the image position information based on an overlap ratio between an area including the moving object in the first image information and an area including the moving object in the second image information.

4. The positioning system according to claim 1 , wherein the operation instruction information includes at least one of a moving mode, a moving time, a moving distance, and a lighting state of the moving object.

5. the wireless unit acquires, from a wireless device provided in the mobile object, autonomous location information that is acquired autonomously by the wireless device and indicates a location of the wireless device; The server unit acquires position information of the moving object based on the autonomous position information and the image position information.

5. The positioning system according to claim 1, wherein:

Citation Information

Patent Citations

  • Semiconductor device

    JP1979054573A

  • Object detector and method therefor

    JP2008176504A

  • Gesture-based car control

    JP2015521127A

  • Determination device, determination method and determination program

    JP2016085686A

  • Applicator, coating material feeding device and coating material feeding method

    JP2019018117A