Object Search System

Abstract

The present invention performs a search for an object safely and with ease.　A scope control unit can simplify and recognize the positional relationship between a scope and a fallen object, as a result of a classification which is in accordance with the range of a distance R in fig. 4a. A prescribed angle range (target recognition range X) around a central axis Y is set as a visual range where the presence or absence of a fallen object (100) is recognized. If the situation at the top portion of fig. 4a is recognized, the scope control unit causes an audio output unit to execute periodic sound output such as shown at the bottom of fig. 4a.

Description

[Technical field]

[0001] The present invention relates to an object search system, an object search device, and an object search method used when a user searches for an object, such as a fallen object, on the ground. [Background technology]

[0002] There are known systems that use radar or the like to detect obstacles and fallen objects (hereinafter referred to as "falling objects") in vast areas (work sites, grounds, facilities, parking lots, roads, railroad tracks, runways, etc.) In such systems, the position (coordinates, etc.) of a fallen object can be identified with a certain degree of accuracy based on the detection results of the radar, for example.

[0003] Conventionally, when a fallen object is detected by radar, a worker from the management company would then search for the fallen object while looking at a map based on the coordinate information (location information) of the fallen object. In this case, for example, the map can be displayed on a tablet terminal held by the worker, and the location of the fallen object can be highlighted, making it easier for the worker to perform this task.

[0004] In this case, as described in Patent Document 1, for example, a scope (head-mounted display) worn by the worker is used instead of a tablet terminal, and the position information of the falling object recognized by the radar is recognized on the scope side, and a display showing the positional relationship between the worker and the falling object is displayed on the scope in real time, making this task particularly easy to perform. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication WO2019 / 065045 DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]

[0006] However, there are various environments in which workers search for fallen objects. For example, the road conditions may be bad and normal walking may be difficult, or the road may be on a road with vehicles passing by. In such cases, the worker needs to search for fallen objects while visually checking the safety of his / her feet and surroundings. In such situations, it is not safe for the worker to keep staring at the scope screen, and even when a head-mounted display is used, it is not easy to search for fallen objects and ensure safety at the same time. For this reason, there was a demand for technology that would allow for easy and safe search for fallen objects.

[0007] The present invention has been made in view of the above circumstances, and has an object to solve the above problems. [Means for solving the problem]

[0008] The present invention provides an object search system including a detection device that detects position information of an object and a display device worn by a user, the detection device including an object position information detection unit that detects the object and acquires position information of the object, and a transmission unit that transmits the position information of the object to the display device, the display device including a reception unit that receives the position information of the object from the detection device, a position and orientation recognition unit that recognizes the position and orientation of the display device, a sound output unit that outputs a plurality of types of sounds, and a control unit that recognizes a positional relationship between the display device and the object from the position and orientation of the display device and the position information of the object, and selects one of the plurality of types of sounds in accordance with the positional relationship and causes the sound output unit to output the sound, the display device displays a substantial image of an environment including the object to the user, and the control unit recognizes a distance between the display device and the object from the received position information of the object and the position of the display device recognized by the position and orientation recognition unit, 、 A target recognition range, which is an area within a visual field range of the actual image on the display device, At a certain point in time, the field of view is set as a certain range of area around the center in the horizontal direction of the field of view, and at a point in time different from the one point, the width in the horizontal direction is set narrower than that at the one point in time when the distance is greater than that at the one point in time, and is set wider than that at the one point in time when the distance is smaller than that at the one point in time, and the target recognition range is displayed superimposed on the actual image. The sound is switched depending on the result of judging whether or not the object is present within the target recognition range, and if the object is present within the target recognition range, the sound pattern is changed depending on the distance. Effect of the Invention

[0009] According to the present invention, a search for an object can be performed easily and safely. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing a configuration of an object search system according to an embodiment. [Diagram 2] 1 is a block diagram showing a configuration of a scope (display device: object search device) in an object search system according to an embodiment. [Diagram 3] 4 is an example of an image recognized by a scope in the object search system according to the embodiment. [Figure 4a] 13 is a typical example (part 1) of a positional relationship between a recognized fallen object and a scope in the object search system according to the embodiment. [Figure 4b] 13 is a typical example (part 2) of a positional relationship between a recognized fallen object and a scope in the object search system according to the embodiment. [Figure 4c] 13 is a typical example (part 3) of a positional relationship between a recognized fallen object and a scope in the object search system according to the embodiment. [Diagram 5] 10 is an example of a change in the positional relationship between a fallen object and a scope when an operator searches for the fallen object using the object search system according to the embodiment. [Figure 6] 1 is an example of a flowchart illustrating an object searching method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An object search system according to an embodiment of the present invention will be described below. This object search system is used, for example, to search for fallen objects on the ground, similar to the technology described in Patent Document 1.

[0012] Fig. 1 is a diagram showing a schematic configuration of an object search system 1 according to this embodiment. This object search system 1 has a detection device 10 that detects a falling object (object) 100 using a radar, recognizes the positional relationship between itself and the falling object 100, and captures an image of the falling object 100, and a scope (display device: object search device) 20 that is worn or carried by an operator (user) 200 and wirelessly connected to the detection device 10. Fig. 1 also shows a block diagram of the detection device 10.

[0013] The detection device 10 is composed of a detection head 10A installed within an area where a falling object 100 can be searched, and a detection device main body 10B connected to the detection head 10A by wire or wirelessly. The detection head 10A is provided with a radar 11 and a camera 12. As is well known, the radar 11 transmits a predetermined electromagnetic wave (for example, millimeter wave band radio waves of 60 GHz or more) in a certain direction and receives the reflected wave for each direction, thereby recognizing the distance to the reflector for each direction. The presence of the falling object 100, which acts as a reflector, can be recognized based on the relationship between the direction and the distance at this time. The control of the radar 11 and its signal processing (calculation processing for recognizing the distance based on the time difference between the transmission timing and the detection timing of the reflected wave, etc.) are performed by a radar control unit 13 of the detection device main body 10B.

[0014] At this time, the radar control unit 13 recognizes the position of the detection head 10A (or the radar 11) in advance, and can recognize the position information of the recognized falling object 100 from this position and the detection result by the radar 11. Furthermore, if the falling object 100 is sufficiently larger than the detection resolution of the radar 11, the radar control unit 13 can also recognize the shape and size (shape information) of the falling object 100. In other words, the radar 11 and the radar control unit 13 function as an object position information detection unit that recognizes the position information, etc. of the falling object 100.

[0015] In addition, the camera 12 can capture an image of the falling object 100 by having its field of view controlled by the camera control unit 14 based on the detection result of the radar 11.

[0016] In addition to the radar control unit 13 and the camera control unit 14, the detection device main body 10B is provided with a detection device side display unit 15 which is a display that displays various information and images, a detection device side wireless communication unit (transmitter) 16 which wirelessly transmits acquired data to the scope 20, and a detection device control unit 17 which controls the entire detection device 10. The detection device control unit 17 can display, in real time, on the detection device side display unit 15, a radar image in which the radar detection results (detection times) for each direction by the radar 11 are mapped, and a video including the falling object 100 obtained by the camera 12. In addition, the detection results of the radar 11, position information and shape information of the recognized falling object 100, and image data captured by the camera 12 are transmitted to the scope 20 side by wireless communication by the detection device side wireless communication unit (transmitter) 16.

[0017] FIG. 2 is a block diagram showing the configuration of the scope (display device: object search device) 20 in FIG. 1. The object search method according to the embodiment of the present invention is executed by this scope 20. The scope 20 is provided with a scope-side wireless communication unit (receiving unit) 21 that receives data transmitted from the detection device 10. In addition, a position and attitude recognition unit 22 that recognizes the position and attitude (direction) of the scope 20 in a three-dimensional space is also provided. The position and attitude recognition unit 22 is provided with a position recognition unit 221 that recognizes the position information of the scope 20 by receiving a GNSS signal. In addition, in order to recognize the direction (direction) of the scope 20, a gyro sensor 222 that recognizes the rotation angle of the scope 20 is provided. Here, the position (position information) of the scope 20 is substantially equal to the position (position information) of the worker 200 wearing the scope 20, and the attitude of the scope 20 is set as the direction of the line of sight of the worker 200 wearing the scope 20. By using the gyro sensor 222, it is possible to detect changes in this attitude, especially over a short period of time.

[0018] The scope 20 is also provided with a storage unit 23, which is a non-volatile memory that stores map information within the area where the fallen object 100 is searched for, a scope side display unit 24, which is a head-mounted display, and a scope control unit (control unit) 25 that controls the entire scope 0. As described in Patent Document 1, the scope side display unit 24 allows the worker 200 to directly view a real image (actual image) through the scope side display unit 24, and can display AR (Augumented Reality) information superimposed on the image. For this reason, the scope control unit 25 can display map information recognized as being visible from the current situation of the scope 20 on the scope side display unit 24 as this AR information based on the recognized position information and attitude of the scope 20. As shown in FIG. 1, the scope 20 has a glasses-like shape, and the scope side display unit 24 actually occupies most of the area of ​​the scope 20.

[0019] At this time, the scope control unit 25 can recognize in advance the field of view of the solid image in the scope 20 (scope side display unit 24) based on the detection result of the position / posture recognition unit 22. Meanwhile, the scope control unit 25 can determine whether or not the falling object 100 is currently within the field of view of the scope 20 (scope side display unit 24) based on the position information of the falling object 100 obtained from the detection device 10 via the scope side wireless communication unit 21, and if the falling object 100 is present, a method can be used such as highlighting the range corresponding to the falling object 100 as AR information on the scope side display unit 24. This allows the worker 200 to easily search for the falling object 100.

[0020] However, what the worker 200 recognizes through the scope side display unit 24 is the actual image that the worker actually sees and the two-dimensional image displayed as AR information as described above, and the worker 200 cannot directly recognize the actual distance between himself / herself and the falling object 100 therein. In this respect, in the technology described in Patent Document 1, a display (specific display) according to the positional relationship (distance) between the worker 200 (scope 20) and the falling object 100 is performed on the scope side display unit 24, and the worker 200 can recognize the distance (positional relationship) between himself / herself and the falling object 100 simply by looking at the scope side display unit 24. Therefore, the worker 200 can gradually approach the falling object 100 while looking at the scope side display unit 24.

[0021] However, there are various environments (situations related to the road surface R in FIG. 1) in which the fallen object 100 is searched. For example, the road surface R may not be a well-maintained paved road but may be rough, or the environment may be dangerous because many vehicles are running on the road surface R. In such cases, it is not preferable for the worker 200 to constantly watch the screen of the scope side display unit 24.

[0022] For this reason, the scope 20 is provided with a sound output unit 26 that outputs a sound (sound signal) rather than a visual display such as an image, of the positional relationship between the scope 20 and the falling object 100, particularly the distance between them. The sound may include voice. In order for the worker 200 to easily determine the situation of the change in the positional relationship between the worker 200 and the falling object 100 when the relationship changes from moment to moment, it is preferable to use, as such a sound signal, a plurality of types of sound output simplified according to rules and switched between, rather than using, for example, a voice output that issues text information. It is preferable that the switching of the sound signal is performed based on the relationship between the recognized falling object 100 and the field of view of the worker 200 (scope 20).

[0023] For this reason, the object search method executed in this scope 20 includes (1) an object position information acquisition process for obtaining position information of the falling object 100 from the detection device 10 side; (2) a position and attitude recognition process for recognizing the position and attitude of the scope 20 by the position and attitude recognition unit 22; (3) a positional relationship recognition process for recognizing the positional relationship between the scope 20 and the falling object 100 from the position and attitude of the scope 20 and the position information of the falling object 100; and (4) a sound output process for selecting and outputting one of multiple preset types of sound signals depending on this positional relationship.

[0024] As described above, the scope control unit 25 can recognize the position and size of the falling object 100 from the data received from the detection device 10 (object position information acquisition process). Meanwhile, the scope control unit 25 can recognize the position and orientation of the scope 20 itself by the position and orientation recognition unit 22 (position and orientation recognition process), and therefore can recognize the position and size of the falling object 100 in the solid image (the field of view of the worker 200) that the worker 200 directly sees on the scope side display unit 24. Alternatively, it can recognize whether the falling object 100 is within the field of view of the solid image. Here, the position in the solid image displayed on the scope side display unit 24 corresponds to the direction (orientation) as seen from the scope 20, and the size of the object in this solid image corresponds to the angle of view of this object as seen from the scope 20.

[0025] FIG. 3 is an example of a solid image (two-dimensional image) seen from the worker 200 side on the scope 20 (scope side display unit 24) in this case. Here, a falling object 100 is present within this field of view. FIGS. 4a to 4c show three typical situations of the falling object 100 seen from the scope 20. The size of the falling object 100 in the image on the scope side display unit 24 corresponds to its angle of view, and when the distance to the falling object is large, this angle of view is small, and when the distance is small, this angle of view is large. FIG. 4a shows the state where the distance is the greatest (state where the angle of view is small), FIG. 4c shows the state where the distance is the smallest (state where the angle of view is large), and FIG. 4b shows an intermediate state between these two.

[0026] As described above, the scope control unit 25 obtains the position information and shape (size) information of the falling object 100 from the detection device 10, but there is a limit to the accuracy, and it is actually difficult to accurately recognize the position and size of the falling object 100 in the image of FIG. 3 from only this information, and to accurately show the falling object 100 in the image of FIG. 3, for example. For this reason, it is preferable to simply recognize the distance range to the falling object 100 and the range in the field of view of the falling object 100 (positional relationship recognition step) as shown in FIGS. 4a to 4c, and communicate this to the worker 200. At this time, as to the field of view, a narrow field of view is set when the distance is large, and a wide field of view is set when the distance is small, in accordance with the size of the falling object 100 recognized in the image as described above, thereby making it possible to more appropriately express the positional relationship between the worker 200 and the falling object 100.

[0027] The scope control unit 25 can recognize a simplified positional relationship between the scope 20 and the falling object 100 by dividing the range of the distance R in Figures 4a to 4c. In Figures 4a to 4c, the upper side shows a situation when the distance D between the scope 20 and the falling object 100 is large (20 m or more), the upper side shows a situation when the distance D is smaller (less than 20 m and 5 m or more), and the lower side shows an oscillation pattern of the sound signal emitted by the sound output unit 26 when each of these situations is recognized.

[0028] 4a to 4c, a range of a predetermined angle (target recognition range X) around the central axis Y is set as the viewing range in which the presence or absence of a falling object 100 is recognized as described above. This angle is set to, for example, 5° (target recognition range X1) in the case of FIG. 4a where the distance D is the longest, 90° (target recognition range X3) in the case of FIG. 4c where the distance D is the shortest, and 45° (target recognition range X2) in the case of FIG. 4b where the distance D is intermediate between the above.

[0029] When the situation in the upper part of Fig. 4a is recognized (when the distance D between the scope 20 and the falling object 100 is 20 m or more and the object 100 is recognized to be within the target recognition range X1), the scope control unit 25 causes the sound output unit 26 to output a periodic sound as shown in the lower part of Fig. 4a. This sound output is actually recognized by the worker 200 as a periodic sound, for example, "beep beep beep." Similarly, when the situation in the upper part of Fig. 4b is recognized (when the distance D between the scope 20 and the falling object 100 is less than 20 m and 5 m or more and the object 100 is recognized to be within the target recognition range X2), the scope control unit 25 causes the sound output unit 26 to output a periodic sound, "beep beep beep beep," as shown in the lower part of Fig. 4b. When the situation in the upper part of Figure 4c is recognized (when the distance D between the scope 20 and the falling object 100 is recognized to be less than 5 m and the object 100 is recognized to be within the target recognition range X3), a continuous "beep" sound as shown in the lower part of Figure 4c is output (this is the sound output process).

[0030] The worker 200 can easily identify these sound signals. On the other hand, when it is recognized that the falling object 100 does not exist within each of the target recognition ranges X in Figs. 4a to 4c, these sound signals are not emitted and a silent state is established. For this reason, for example, when the falling object 100 does not exist within the target recognition range X in Figs. 4a to 4c, and the worker 200 (scope 20) changes direction or moves, causing the falling object 100 to enter the target recognition range X set according to the distance D, the worker 200 can easily recognize this fact by the sound signal. After that, the worker 200 can move straight from this state to get closer to the falling object 100.

[0031] FIG. 5 is a diagram showing an example of such a work situation in correspondence with FIG. 4a to FIG. 4c. In FIG. 5, the state (1) is a case where the falling object 100 does not exist within the target recognition range X (X1) in FIG. 4a. In this case, since no sound is generated, the worker 200 can recognize this fact and change his / her direction. As a result, as shown in (2), when the falling object 100 enters the target recognition range X1, the worker 200 can recognize this fact by the sound. In this state, the distance D to the falling object 100 is not small, but since the falling object 100 exists almost in front of the worker in this state, the worker can move forward as it is and sequentially reach the states (3) and (4), and can recognize this fact by the sound. When the falling object X does not exist within the target recognition range X corresponding to the distance D therebetween when moving forward, the sound is no longer generated, so the worker 200 can realize the states (3) and (4) by changing his / her direction, similar to the case of (1).

[0032] By performing such an operation, the worker 200 can approach the falling object 100. At this time, the worker 200 can recognize the positional relationship (distance, traveling direction) between the worker 200 and the falling object 100 only by the sound. In addition, the image in FIG. 3 also shows the target recognition range X. In FIGS. 4a to 4c and 5, the target recognition range X is shown as a range of horizontal azimuth angles, but the horizontal range of the elliptical target recognition range X in FIG. 3 corresponds to the target recognition range X in FIGS. 4a to 4c and 5. The vertical width of the elliptical shape that is the target recognition range X in the image in FIG. 3 can be set appropriately depending on the size of the recognized falling object 100, etc. In addition, the display method of the target recognition range X can be set appropriately.

[0033] FIG. 6 is a flow chart showing this object search method. First, as described above, the scope 20 performs an object position information acquisition step of acquiring position information of the falling object 100 from the detection device 10 (S1). Next, the scope 20 performs a position and attitude recognition step of recognizing its own position and attitude by the position and attitude recognition unit 22 (S2), and thereby performs a positional relationship recognition step of recognizing the positional relationship between itself and the falling object 100 from its own position and attitude and the positional information of the falling object 100 (S3). Next, the scope 20 performs a sound output step of selecting and outputting one of a plurality of types of sound signals set in advance according to this positional relationship (S5). The worker 200 recognizes the sound output and performs an action such as moving forward or changing direction as described above (S5). The above actions are repeated until the worker 200 finds the falling object 100 (the worker 200 gets close enough to the falling object 100) (S6: Yes).

[0034] In the above example, the pattern of the sound signal is changed according to the positional relationship between the falling object 100 and the worker 200 (scope 20), but instead of or in addition to changing the pattern, the volume may also be changed.

[0035] Furthermore, the scope control unit 25 can display, among the information obtained from the detection device 10, information that is useful when the worker 200 searches for the falling object 100 as AR information on the scope display unit 24. As in the case of Patent Document 1, such information includes the shape of the falling object 100 captured by the camera 12 (or an image including the falling object 100), etc.

[0036] In the above example, the worker 200 can recognize the positional relationship between himself and the falling object 100 by the sound signal, but the positional relationship shown in Figures 4a to 4c may be displayed by a display method on the scope (display device) side, in addition to the sound signal. For example, there is a method of changing the blinking pattern when blinking the display depending on the positional relationship.

[0037] Even in such a case, as described above, the worker can recognize the positional relationship between the fallen object and himself (the scope) only by the sound signal, so the worker does not need to constantly watch the screen and can simply check the contents as necessary. Therefore, the worker can easily perform this work while ensuring the safety of the surroundings.

[0038] In the above example, the position and attitude recognition unit 22 is provided only on the scope (display device) 20 side, but it may also be provided on the detection device (detection head). In this case, even if the detection device moves, its own position and attitude are accurately recognized, and even if the detection device moves, the display device can obtain accurate position information of the fallen object.

[0039] In the above example, the radar 11 is used to obtain the position information of the falling object 100 in the detection device 10. However, the position information of the falling object can be recognized by other methods such as an optical method, an acoustic method, etc., and the same operation as above is possible.

[0040] Furthermore, even if the display device itself can recognize the position information of the falling object by a signal emitted by the falling object itself, for example, the display device can perform the same operation. In such a case, the above-mentioned detection device is not necessary, and the above-mentioned display device alone can be used to perform the same operation. That is, the above-mentioned display device alone can be used as an object search device. Also, as described above, the display device allows the user to visually recognize the actual image, which makes the search easier. However, even if the user cannot visually recognize the actual image on the display device (object search device), the search for the fallen object can be easily performed as long as the above-mentioned sound output is performed.

[0041] In the above object search method, the positional relationship between the falling object 100 and the scope 20 is recognized on the scope (display device) 20 side. However, this positional relationship may be recognized on a device other than the display device, and the display device may only output a sound corresponding to this. That is, in the above object search method, the object position information acquisition step, the position and attitude recognition step, and the positional relationship recognition step may be appropriately performed inside or outside the display device.

[0042] The present invention has been described above based on the embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of the respective components, and that such modifications are also within the scope of the present invention. [Explanation of symbols]

[0043] 1 Object Search System 10. Detection Device 10A detection head 10B Detector body 11 Radar (object position information detection section) 12 Camera 13 Radar control unit (object position information detection unit) 14 Camera control section 15 Detector side display unit 16 Detector side wireless communication unit (transmitter) 17 Detection device control section 20 Scope (display device: object search device) 21 Scope side wireless communication unit (receiver) 22 Position and orientation recognition unit 23 Memory section 24 Scope side display 25 Scope control section (control section) 26 Sound output section 100 Falling object (object) 200 workers (users) 221 Location recognition unit 222 Gyro sensor R road surface X, X1~X3 target recognition range Y center axis

Claims

[Claim 1] An object search system including a detection device for detecting position information of an object and a display device worn by a user, The detection device includes: an object position information detection unit that detects the object and acquires position information of the object; a transmitting unit that transmits position information of the object to the display device; Equipped with The display device includes: a receiving unit that receives position information of the object from the detection device; a position and orientation recognition unit that recognizes a position and orientation of the display device; a sound output unit that outputs a plurality of types of sound; a control unit that recognizes a positional relationship between the display device and the object based on a position and attitude of the display device and position information of the object, and selects one of the plurality of types of sounds in accordance with the positional relationship and outputs the selected sound to the sound output unit; Equipped with The display device displays to the user a real-world image of an environment including the object; The control unit is a distance between the display device and the object is recognized from the received position information of the object and the position of the display device recognized by the position / orientation recognition unit, and a target recognition range, which is an area within the field of view of the solid image on the display device, is set as an area of ​​a certain range around the center of the field of view in the horizontal direction at a single point in time, and at a time point different from the single point in time, the horizontal width is set narrower than at the single point in time if the distance is greater than at the single point in time, and wider than at the single point in time if the distance is smaller than at the single point in time, and the target recognition range is displayed superimposed on the solid image, and the sound is switched depending on the result of determining whether the object is present within the target recognition range, An object searching system characterized in that, when the object is present within the target recognition range, the sound pattern is changed according to the distance.

Images