Radar device, monitoring system, and method of calibrating radar device

Abstract

This radar device for detecting an object is provided with a radar module, a processor, and a memory. The memory stores map information about a region under monitoring by the radar device. The processor generates a movement trajectory graphic indicating a movement trajectory of a detected body moving through the interior of the region under monitoring, on the basis of radar detection information outputted upon detection of the detected body, and generates and stores correction data for aligning the movement trajectory graphic with a feature graphic corresponding to a movement path of the detected body included in a map image based on the map information.

Description

Radar device, surveillance system, and method for calibrating radar device

[0001] The present invention relates to a radar device, a surveillance system, and a method for calibrating a radar device.

[0002] Conventionally, there are surveillance systems that use radar devices to monitor the intrusion of suspicious individuals into a premises. Because the detection results of the radar device are output as point cloud data, in order to easily understand which point in the monitored area the monitoring results are for, map information of the monitored area and the point cloud information are used together.

[0003] For example, Patent Document 1 discloses a calibration method in which a user specifies two pairs of points on a map and corresponding points on a radar, and the points are matched with the map.

[0004] Patent Document 2 discloses a radar camera system that calibrates the radar using the trajectory of an object on the camera image and the trajectory on the radar.

[0005] European Patent No. 3,385,747 U.S. Patent No. 1,733,370

[0006] The calibration method in Patent Document 1 matches map information and point cloud data from two sets of information: points on a map and the corresponding detection points output by the radar device. This poses the problem that high-precision adjustments are not possible in places where precise alignment between map information and point cloud information is required, such as near the boundary of the monitored area, or in places where it is desired to draw straight lines that follow the terrain.

[0007] Furthermore, the radar camera system of Patent Document 2 requires a camera in addition to the radar device in order to calibrate the point cloud data of the radar device with map information, and furthermore, the calibration points must be within the angle of view of the camera, which poses the problem that calibration with map information cannot be performed by the radar device alone.

[0008] The present invention has been made to solve the above-mentioned problems, and aims to match the point cloud data output by a radar device with map information of the radar detection range with high accuracy without using camera images.

[0009] To achieve the above object, the present invention employs the configuration described in the claims. For example, the present invention provides a radar device for detecting objects, comprising a radar module that irradiates radio waves to objects in a monitored area, detects the objects based on the radio waves reflected by the objects, and outputs radar detection information including relative position information of the objects, a processor, and a memory, wherein the memory stores map information of the monitored area, and the processor generates a movement trajectory diagram showing the movement trajectory of a detected object based on the radar detection information output after detecting the detected object moving within the monitored area, and generates and stores correction data for matching the movement trajectory diagram with a feature diagram corresponding to the movement path of the detected object included in a map image based on the map information.

[0010] The present invention also provides a monitoring system comprising: a radar device that irradiates radio waves to an object in a monitored area, detects the object based on the radio wave reflected by the object, and outputs radar detection information including relative position information of the object; and an information processing device that receives the radar detection information, wherein the information processing device includes a processor and a memory, and the memory stores map information of the monitored area. The processor generates a movement trajectory graphic showing the movement trajectory of the detected object based on the radar detection information outputted after detecting the detected object moving within the monitored area, generates correction data for matching the movement trajectory graphic with a characteristic graphic corresponding to the movement path of the detected object, which is included in a map image based on the map information, and converts the coordinates of a radar detection point newly detected by the radar device using the correction data, and executes control to superimpose and display the converted coordinates of the radar detection point on the map image.

[0011] The present invention also provides a method for calibrating a radar device that detects objects, the method comprising the steps of: acquiring radar detection information including relative position information of an object, the radar device detecting the object irradiating radio waves to an object in a monitored area, detecting the object based on the radio wave reflected by the object, and outputting the radar detection information; reading map information of the monitored area; generating a movement trajectory diagram showing the movement trajectory of the detected object based on the radar detection information detected and output; and generating and storing correction data for matching the movement trajectory diagram with a characteristic diagram corresponding to the movement path of the detected object, which is included in a map image based on the map information.

[0012] According to the present invention, it is possible to match the point cloud data output by the radar device with map information of the radar detection range with high accuracy without using images from a camera. Objects, configurations, and effects other than those described above will be made clear in the following embodiments.

[0013] 1 is a diagram explaining the purpose of use of a correction formula (calibration data) generated in this embodiment. A hardware configuration diagram of a radar device. A diagram showing an example configuration of a monitoring system using a radar device. A software configuration diagram of a radar device according to the first embodiment. A flowchart showing the flow of calibration processing using a radar device alone. A sequence diagram showing the flow of another example of calibration processing using a radar device and a user terminal. A diagram showing an example of input of a radar detection range. A diagram showing an example of a recommended route. A flowchart showing the flow of reflected wave reception and signal analysis processing. A diagram showing an example configuration of a comparison processing unit configured using AI. A flowchart showing the flow of superposition matching processing by a comparison processing unit using AI. A diagram showing an example of an actual operation image (map reading processing). A diagram showing an example of an actual operation image (example of acquiring a walking trajectory). A diagram showing an example of an actual operation image (processing for matching with a map). A diagram showing an example of an actual operation image (handling of coordinates when matching with a map). An explanatory diagram showing an overview of the logic for converting points in three-dimensional space acquired by radar into points on a two-dimensional map. A diagram showing details of the process of aligning the Z radar axis in the vertical direction, which is part of the logic for converting points in three-dimensional space acquired by radar into points on a two-dimensional map. A diagram showing the relationship between radar and real-world coordinates, which is part of the logic for converting points in three-dimensional space acquired by radar into points on a two-dimensional map. A diagram showing the relationship between actual coordinates and map coordinates, which is part of the logic for converting points in three-dimensional space acquired by radar into points on a two-dimensional map. A software configuration diagram of a radar device and a user terminal according to a second embodiment. A sequence diagram showing the processing flow of the second embodiment.

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings for explaining the embodiment, the same components are generally designated by the same reference numerals, and repeated description thereof will be omitted.

[0015] FIG. 1 is a diagram illustrating the purpose of use of the correction formula (calibration data) generated in this embodiment. The point cloud map 1 in FIG. 1 corresponds to point cloud data in which a radar device emits radio waves, receives reflected waves from the object to detect the object, and detects the object's relative position, including the direction of the object relative to the radar device and the distance to the object, and maps the detected position as points. The point cloud map 1 indicates the relative position of the object relative to the installation position of the radar device based on the reflected waves from the object existing in three-dimensional real space. Therefore, if the installation position of the radar device and the range in which the radar device emits radio waves to detect objects (hereinafter referred to as the "radar detection range") are unknown in the monitored area, the point cloud map 1 does not indicate where on the map the object detected by the radar device is located.

[0016] Therefore, by generating a composite map 3 by superimposing it on a map image 2 illustrating the topography, shape and installation location of buildings in the monitored area, it becomes easier to understand which point in the monitored area each radar detection point shown on the point cloud map 1 corresponds to.

[0017] To create the composite map 3, it is necessary to convert the coordinates of the radar detection points detected by the radar device into coordinates on the map image 2. A first object of this embodiment is to accurately generate a correction formula for converting the coordinates of the radar detection points into coordinates on the map image 2. The "correction formula" is coordinate conversion data for converting the radar coordinate system into the map coordinate system, and its form does not need to be expressed by a determinant, and any data structure is acceptable as long as it is correction data that can convert the radar coordinate system into the map coordinate system. In the present disclosure, the process of generating the correction formula is referred to as a calibration process.

[0018] Furthermore, a second object of this embodiment is to create and display a composite map 3 using the generated correction formula.

[0019] 2 is a hardware configuration diagram of the radar device 10. In this embodiment, the radar device 10 uses a millimeter wave radar, but may also use a microwave radar, an infrared radar, or a LiDAR (Light Detection and Ranging).

[0020] The radar device 10 includes a CPU (Central Processing Unit, equivalent to a processor) 11, a ROM 12, a RAM 13, a storage medium 14, a communication I / F 15, and a radar module 16, which are connected to each other via a bus 18. The radar device 10 may also include an inclination sensor 17 that detects the installation angle of the radar device 10 with respect to a horizontal plane. The inclination sensor 17 is, for example, an acceleration sensor.

[0021] The CPU 11 controls detection by the radar module 16 , communication via the communication I / F 15 , and the overall processing of the radar device 10 .

[0022] A ROM (Read Only Memory) 12 stores a control program for the radar device 10 .

[0023] A RAM (Random Access Memory) 13 functions as a work area for loading control programs and as a primary storage area for point cloud data.

[0024] The storage medium 14 is, for example, a flash memory, and stores, for example, data required for generating calibration data, coordinate transformation data (corresponding to correction data), and the like.

[0025] The communication I / F 15 is, for example, a LAN communication interface.

[0026] The radar module 16 includes a radar IC (integrated circuit) 161, multiple radar transmitting antenna arrays ATx1 to ATxm, and radar receiving antenna arrays ARx1 to ARxn. The radar transmitting antenna arrays ATx1 to ATxm sequentially emit radio waves, and the radar receiving antenna arrays ARx1 to ARxn receive the reflected waves. The range in which radio waves can be emitted and reflected waves can be received is the radar detection range.

[0027] The radar IC (integrated circuit) 161 calculates the distance from the radar device 10 to an object based on the time (ToF) from emitting radio waves to receiving reflected waves. Hereinafter, the radar reflection point on the object will be referred to as the radar detection point. The radar IC (integrated circuit) 161 also detects the direction of the radar detection point relative to the radar device 10 from the direction in which the radar receiving antenna arrays ARx1, ..., ARxn received the reflected waves. The radar IC then outputs radar detection information including relative position information of the radar detection point, including the distance and direction to the radar detection point.

[0028] 3 is a diagram showing an example of the configuration of a monitoring system 90 using the radar device 10. The monitoring system 90 is configured by connecting the radar device 10 and a user equipment (UE) 20 via a network 30 for communication.

[0029] In a second embodiment described later, a server 40 is connected to the network 30, and the composite map 3 is created by the server 40. Furthermore, a PTZ camera 50 (PTZ is an abbreviation of Panoramac Tilt Zoom) may be connected to the network 30, and an object detected by the radar device 10 may be captured by the PTZ camera 50, thereby configuring a monitoring system 90 that combines radar detection and video.

[0030] First Embodiment The first embodiment is an embodiment in which a correction formula is generated by a radar device alone or by a user terminal 20 communicatively connected to a radar device.

[0031] 4 is a software configuration diagram of the radar device 10 according to the first embodiment. The radar device 10 includes a movement trajectory generation unit 101, a comparison processing unit 102, a superposition matching unit 103, an image rotation and enlargement processing unit 104, a correction formula generation unit 105, a radar device communication unit 106, a recommended route generation unit 107, a map information storage unit 108, a correction formula storage unit 109, and an operation mode control unit 110. The movement trajectory generation unit 101, the comparison processing unit 102, the superposition matching unit 103, the image rotation and enlargement processing unit 104, the correction formula generation unit 105, the radar device communication unit 106, the recommended route generation unit 107, and the operation mode control unit 110 are configured by the CPU 11 loading a program that realizes the function of each unit into the RAM 13 and executing it. The map information storage unit 108 and the correction formula storage unit 109 are configured using a partial area of ​​the storage medium 14. Details of the functions of each unit will be described later with reference to the respective drawings.

[0032] (Example of execution by radar device alone) The radar device 10 has two operation modes: a normal operation mode and a calibration mode. In the normal operation mode, the radar device 10 outputs only the position data of the radar-detected object at the time the object is detected as radar detection information. Therefore, the radar detection information output in the normal operation mode does not include past position data of the radar-detected object. This makes it easy to grasp the current position of the radar-detected object, and allows the radar device 10 to perform operations suitable for surveillance.

[0033] On the other hand, in the calibration mode, the radar device 10 outputs radar detection information including the position data of radar-detected objects from the start of operation in the calibration mode to the present time. This leaves a trail of movement of the radar-detected object, making it easier to align it with map coordinates and allowing advance preparation for monitoring using the radar device 10.

[0034] The radar device 10 of this embodiment is characterized by its calibration mode, and therefore the calibration mode will be described below. When a user selects the calibration mode as the operation mode of the radar device 10, the operation mode control unit 110 sets the operation mode of the radar device 10 to the calibration mode, thereby starting processing in the calibration mode. If the user then selects the normal operation mode as the operation mode, the operation mode control unit 110 ends the calibration mode and starts the normal operation mode. Alternatively, in another example, a second embodiment described below, the radar device 10 may be configured to receive an operation to start or end the calibration mode from an information processing device, such as a user terminal or server, that is communicatively connected to the radar device 10, and in response, the operation mode control unit 110 may switch to the calibration mode and start or end processing in the calibration mode.

[0035] FIG. 5 is a flowchart showing the flow of calibration processing performed by the radar device alone.

[0036] The user inputs map information indicating the topography of the area to be monitored including the radar detection range into the radar device 10, and stores the input information in the map information storage unit 108 (S101).

[0037] The radar device 10 is set to a mode that leaves a radar trail (calibration mode). This ensures that the points detected by the radar do not disappear and a trail remains. A user who is the subject of detection by the radar device 10 (an object detected by the radar device 10) walks along a predetermined path. In the radar device 10, the radar IC 161 controls the emission of radio waves and the reflected waves. Furthermore, the radar device 10 performs signal analysis processing of the received reflected waves, calculates the relative position of the radar detection point, and generates radar detection information (S102).

[0038] The movement trajectory generating unit 101 acquires radar detection information, plots point cloud information indicating the relative position of the user over time, and generates a movement trajectory diagram (S103).

[0039] The comparison processor 102 reads map information, searches for a partial area within the map that has a relatively high correlation with the shape of the movement trajectory diagram, and generates a feature diagram defined by its contour. The comparison processor 102 then superimposes the movement trajectory diagram on the feature diagram. The image rotation and enlargement processor 104 rotates and enlarges the movement trajectory diagram so that it matches the feature diagram (S104).

[0040] The correction formula generation unit 105 acquires the rotation amount, enlargement amount, and translation amount for matching the movement trajectory figure with the characteristic figure in step S104, and generates a rotation / translation matrix for matching the movement trajectory figure with the characteristic figure based on these (S105). This rotation / translation matrix corresponds to the correction formula. The correction formula generation unit 105 stores the generated correction formula in a non-volatile storage medium, for example, the storage medium 14.

[0041] Thereafter, the radar device 10 performs radar detection processing within the monitored area in a normal operating mode, i.e., an operating mode in which the point cloud data is updated at every radar scan period and the position of an object is detected in real time. When a new object is detected, the radar device 10 transmits radar detection information including relative position information of the new object expressed in the radar coordinate system to the user terminal 20. The user terminal 20 converts the relative position of the new object in the radar coordinate system into the map coordinate system using a correction formula, and superimposes it on the map image 2 to create and display a composite map 3 (see FIG. 1 ) (S106). The display device of the composite map 3 is a monitor (not shown) that is communicatively connected to the radar device 10 via the radar device communication unit 106.

[0042] According to the above example, since calibration data can be generated without using the user terminal 20, there is an advantage that calibration can be performed on a radar device installed in a location with a poor communication environment.

[0043] (Example of Implementation Using a Radar Device and a User Terminal) As another example, the radar device communication unit 106 of the radar device 10 may be communicatively connected to the user terminal 20, and a walking trajectory may be displayed to the user who is the detected object, thereby making it easier for the user to walk along the planned movement route. Figure 6 is a sequence diagram showing the flow of another example of calibration processing using a radar device and a user terminal.

[0044] Map information indicating the topography of the area to be monitored, including the radar detection range, is input to the radar device 10 and stored in the map information storage unit 108 (S101). The radar device 10 transmits the map information and a map information display instruction to the user terminal 20 (S201), and the user terminal 20 receives the map information and the map information display instruction and displays a map image on the display (S202).

[0045] The user performs an operation to start radar detection by the radar device 10 on the user terminal 20 (S203), and the user terminal 20 accepts the start operation (S204) and transmits a start instruction to the radar device 10. Thereafter, the user, who is the object to be detected by the radar device 10, walks along a predetermined movement route.

[0046] The radar device 10 starts emitting radio waves (S205) and executes the reflected wave reception and signal analysis process (S102).

[0047] The movement trajectory generating unit 101 acquires radar detection information, plots point cloud information indicating the relative position of the user over time, and generates a movement trajectory diagram (S103).

[0048] Next, the radar device 10 transmits the movement trajectory information to the user terminal 20 (S206). The user terminal 20 displays the movement trajectory diagram (S207). At this time, an image in which the movement trajectory diagram is superimposed on a two-dimensional radar coordinate system may be displayed on the display of the user terminal 20. This allows the user to visually recognize the plotted points displayed on the user terminal 20 and confirm their own walking trajectory. Note that, although the movement trajectory diagram is created by the user terminal 20 in step S206, as another example, the radar device 10 may create the movement trajectory diagram, transmit movement trajectory information including the movement trajectory diagram to the user terminal 20, and the user terminal 20 may display the movement trajectory diagram included in the received movement trajectory information.

[0049] When the user has finished walking the planned walking route, the user performs an end operation of the radar device 10 on the user terminal 20 (S208). When the user terminal 20 receives the end operation (S209), it transmits an end operation instruction to the radar device 10. When the radar device 10 receives the end operation instruction, it ends the radar detection process.

[0050] The radar device 10 then executes a superposition matching process (S104) and a rotation / translation matrix generation process (S105). When the radar device 10 then performs radar detection, it performs coordinate transformation using the generated correction formula, and superimposes the position of a newly detected object on a map image and displays it on the display of the user terminal 20 (S106).

[0051] According to this example, since the movement trajectory graphic is displayed on the user terminal 20, it becomes easier for the user to walk along the planned movement route and to match the movement trajectory graphic with the characteristic graphic.

[0052] (Map Information Input: S101) Here, the input mode of map information in S101 will be described. If the map information has scale data attached to it in advance, it is sufficient to simply input the map information to the radar device 10. When inputting a map image without scale data to the radar device 10, input of the map information is completed by specifying any two points on the map image to be input and then specifying the scale of the map image.

[0053] In addition to inputting map information and specifying a scale, a radar detection range may be input to assist the system in determining a walking route. Fig. 7 is a diagram showing an example of inputting a radar detection range.

[0054] A map image 2 based on the map information may be displayed on the display of the user terminal 20 or on a map information editing terminal before inputting the map information into the radar device 10, and the user may input a line 211 indicating the radar detection range. Alternatively, the user may input an installation position mark 212 of the radar device 10 on the map image 2. In this case, the radar detection range may be calculated automatically from the effective range of the radar.

[0055] (Map display process: S203) Only the map image 2 based on the map information may be displayed on the display of the user terminal 20. Alternatively, the recommended route generation unit 107 may detect terrain suitable for acquiring a walking trajectory based on the map information, generate a recommended walking route, and display it together with the map image. Figure 8 shows an example of a recommended route.

[0056] To generalize the recommended route, the following are preferable in order to match the characteristic figure with the movement trajectory figure: Pattern 1: A figure in which the vector indicating the walking direction changes two or more times after the start of walking, that is, an open-shaped figure with two or more vertices other than a straight line segment connecting any two points on the map image (in other words, an open-shaped figure with two or more vertices formed by a portion of the outline of a closed-shaped figure being missing), or a closed-shaped figure with two or more vertices (polygonal shape), or a circular route; and Pattern 2: A route near a structure that is a singular point on the map.

[0057] For example, matching patterns A and B in Figure 8 show a state in which rectangular routes 213a and 213b are matched with map image 2, and matching pattern D shows a state in which triangular route 213d is matched with map image 2. Routes 213a, 213b, and 213d correspond to routes that form a closed shape. Route C corresponds to open-shaped route 213c of pattern 1, which has no rectangular side. These routes A, B, C, and D are examples of recommended routes in the map display process.

[0058] On the other hand, matching patterns E and F are "L"-shaped routes 213e and 213f, which have only one vertex. With such characteristic figures, it is difficult to achieve accurate alignment when matching the characteristic figure with a movement trajectory figure. In the case of a route with only one vertex, such as route 213f, where one side is relatively long, it is easier to achieve accurate alignment. However, a route with a shape that includes two corners is more likely to achieve accurate alignment when matching the map image 2 with the walking trajectory.

[0059] In pattern 2, the recommended route is a route that follows the terrain where a clear line is visible on the map, such as "along a fence" or "the edge of a crosswalk." This is because walking trajectories that follow the terrain are easy to match with the terrain.

[0060] (Reflected Wave Reception and Signal Analysis Processing: S102) FIG. 9 is a flowchart showing the flow of reflected wave reception and signal analysis processing.

[0061] When the radar receiving antenna arrays ARx1, ..., ARx receive reflected waves (S1021), the radar device 10 analyzes the distance from the intensity of the reflected waves received by the radar IC 161 and the direction of the radar detection point relative to the radar device 10 from the reception direction, and generates point cloud data including the distance and direction of the radar detection point. The distance and direction of the radar detection point correspond to the relative position of the radar detection point relative to the radar device 10.

[0062] The trajectory generation unit 101 specifies point cloud data to be used for trajectory drawing from among the point cloud data (S1022). The trajectory generation unit 101 generates trajectory data by stacking the point cloud data specified during calibration without erasing it (S1023). This realizes signal analysis processing for generating trajectory data.

[0063] (Overlay matching process 1: S104) When the user performs the overlay matching process between map information and a walking trajectory, the user superimposes a movement trajectory figure on the map image 2 and performs rotation and translation operations. The user's operations may be performed on a display terminal (not shown) connected to the radar device 10, or may be performed on the user terminal 20 by transmitting and displaying the map image 2 and the movement trajectory figure on the user terminal 20.

[0064] (Superimposition matching process 2: S104) Fig. 10 is a diagram showing an example of the configuration of the comparison processing unit 102 configured using AI (artificial intelligence). The comparison processing unit 102 includes a trained model storage unit 1021, an AI calculation processing unit 1022, and an image superimposition unit 1023. The trained model storage unit 1021 stores trained model 1, which is trained using a machine learning model suitable for image recognition processing, such as a deep neural network (DNN) or a convolutional neural network (CNN), to learn a map image and characteristic topographical lines contained therein, and trained model 2, which is trained using matching data between a movement trajectory diagram and map information as training data.

[0065] FIG. 11 is a flowchart showing the flow of the overlap matching process by the comparison processing unit 102 using AI.

[0066] The AI ​​calculation processing unit 1022 reads out the learned model 1 from the learned model memory unit 1021, inputs a map image into the learned model 1 to detect buildings and terrain lines on the map (S1041), extracts singular points on the map (S1042), and connects several singular points to create a characteristic figure.

[0067] The AI ​​calculation processing unit 1022 reads out the learned model 2 from the learned model storage unit 1021, inputs the movement trajectory diagram and the feature diagram into the learned model 2, and compares the feature diagram with the movement trajectory diagram, for example, by assigning a score indicating the degree of similarity (S1043). If there is a feature diagram corresponding to the movement trajectory diagram (S1043: correspondence found) and there is only one corresponding feature diagram (hereinafter referred to as "matching shape") (S1044: one), the superimposition matching unit 103 matches the movement trajectory diagram with the feature diagram at the position of the map image to which the movement trajectory diagram corresponds (S1045), and displays the movement trajectory diagram superimposed on the map image (S1046).

[0068] If there are two or more matching shapes, the corresponding positions are displayed to the user (S1047), and when the user selects one (S1048), a movement trajectory figure is superimposed and displayed at the selected position (S1046).

[0069] If there is no characteristic graphic corresponding to the movement trajectory graphic (S1043: no correspondence), the movement trajectory graphic is superimposed and displayed at an arbitrary position on the map image (S1046).

[0070] If the accuracy of the match between the travel trajectory graphic and the characteristic graphic is insufficient and correction is required, the user rotates and translates the travel trajectory graphic to match the two (S1049), and the process ends.

[0071] An example of an actual operation will be described with reference to Fig. 12 to Fig. 15. Fig. 12 is a diagram showing an example of an actual operation (map reading process). Fig. 13 is a diagram showing an example of an actual operation (acquisition of a walking trajectory). Fig. 14 is a diagram showing an example of an actual operation (processing for matching with a map). Fig. 15 is a diagram showing an example of an actual operation (handling of coordinates when matching with a map).

[0072] As shown in Figure 12, a map image 2 prepared by the user is input into the radar device 10. The user is prompted to select any two points 250 and 251 on the map image 2 whose distance is known. The user is then prompted to input the distance between points 250 and 251. This ensures that the scale will match when the image is later overlaid with the radar point cloud. However, if image data with added scale data is read, it is not necessary to specify points 250 and 251.

[0073] When the map image 2 is loaded into the radar device 10, recommended routes 221, 222, and 223 generated by the recommended route generation unit 107 may be superimposed on the map image 2. The recommended routes 221, 222, and 223 are generated by the recommended route generation unit 107 by extracting partial areas of the map image 2 that are suitable as movement routes for the detected object and defining the outlines of those partial areas as recommended routes. Therefore, a movement trajectory diagram of a person walking along the recommended route matches the characteristic diagram. The radar device 10 transmits characteristic diagram information for displaying the characteristic diagram superimposed on the recommended route to the user terminal 20. When the information is displayed on the display of the user terminal 20, the map image 2 is displayed with the recommended routes 221, 222, and 223 (which match the characteristic diagram) superimposed thereon, as shown in FIG. 12 . Note that the installation position and detection range of the radar device 10 are not shown in FIG. 12 .

[0074] As shown in Fig. 13, the user walks along an arrow 230 indicating the direction of movement on the map image 2 while looking at the screen 31 of the user terminal 20 at hand. The path the user took walking along the arrow 230 is displayed as a path 311 on the screen 31. The screen 31 also displays a radar detection diagram 310, which is drawn as concentric semicircles indicating positions equidistant from the radar installation position. This allows the user looking at the screen 31 to know the distance and direction (relative position) from the radar device 10 to their own position. The acquired movement path diagram 312 is saved in the user terminal 20.

[0075] 14, the image rotation and enlargement processing unit 104 rotates and moves the map image 2 or the movement trajectory figure 312 to match the characteristic figure in the map image 2 with the movement trajectory figure 312. Once they are matched, the positional relationship between the map image 2 and the radar detection figure 310 is fixed.

[0076] 15, an origin (hereinafter referred to as the map origin) and two orthogonal axes, the Xmap axis and the Ymap axis, are defined on the map image 2. The two-axis orthogonal coordinate system consisting of the Xmap axis and the Ymap axis is called the map coordinate system. When the image rotation and enlargement processing unit 104 rotates the map image 2, the map coordinate system also rotates.

[0077] Furthermore, the image rotation and enlargement processing unit 104 defines the installation position of the radar device 10 as the origin (hereinafter referred to as the radar origin), a Y-radar axis indicating the front direction of the radar device 10, and an X-radar axis as a right-handed coordinate system perpendicular to the Y-radar axis, on the radar detection diagram 310 on which the movement trajectory figure 312 is drawn. The radar coordinate system is a three-axis Cartesian coordinate system that also includes a Z-radar axis that is perpendicular to the Y-radar axis and X-radar axis and is perpendicular to the paper surface of Fig. 21, and the radar detection diagram 310 draws the X-radar-Y-radar plane of the radar coordinate system.

[0078] Once the overlay matching unit 103 matches the feature figure of the map image 2 with the movement trajectory figure 312, the image rotation and enlargement processing unit 104 obtains the rotation angle ψ between the map coordinate system and the two axes, the X-radar axis and the Y-radar axis, of the radar coordinate system from the difference between the map coordinate system and the two axes, the X-radar axis and the Y-radar axis of the radar coordinate system, and obtains the parallel movement distance from the map origin Omap and the radar origin Oradar.

[0079] (Rotation / Translation Matrix Generation Process: S105) FIG. 16 is an explanatory diagram showing an outline of the logic for converting points in a three-dimensional space acquired by a radar into points on a two-dimensional map.

[0080] First, the angles of the radar's XZ plane and YZ plane, which correspond to the actual direction of gravity, are measured. Specifically, for example, the angle between the direction of gravity and the Z radar axis direction is determined using the tilt sensor 17 mounted on the radar device 10. The three-axis Cartesian coordinate system of the tilt sensor 17 coincides with the radar coordinate system. Alternatively, a flat surface serving as a reference may be provided on the housing of the radar device 10, and the tilt of the flat surface, i.e., the tilt of the radar device 10 itself, may be measured using an acceleration sensor. Next, a rotation matrix about the X radar axis-Y radar axis is determined (S1051) as a rotation matrix that aligns the Z radar axis with the actual direction of gravity based on the radar coordinate system.

[0081] Using a rotation matrix that matches the Z radar axis with the actual direction of gravity, the points detected by the radar device 10 are plotted on the actual XY coordinate plane (a two-dimensional coordinate plane on the map) (S1052). If no operation to match with the map has been performed, the calculation results up to this point are displayed (displayed with ψ = 0, a translation amount of 0, as described later).

[0082] The X-axis direction, Y-axis direction, and origin of the real coordinates with the gravity direction matched are rotated and translated to match the X-axis direction, Y-axis direction, and origin of the radar, respectively (S1053). The rotation and translation matrices used here are the matrices found in the operation to match with the map.

[0083] The coordinates of the point after the movement become the real coordinates, and the X and Y components become the coordinates of the point on the two-dimensional coordinate system on the map (S1054).

[0084] FIG. 17 is a diagram showing details of the process of aligning the Z radar axis with the vertical direction, which is part of the logic for converting points in a three-dimensional space acquired by radar into points on a two-dimensional map.

[0085] As shown in FIG. 17 , if the radar coordinate system 150 is defined as a three-axis Cartesian coordinate system in which the Z radar axis is the direction toward the top of the radar device 10, the Y radar axis is the direction toward the front, and the X radar axis is a right-handed system, then point A detected by the radar device 10 can be expressed in the radar coordinate system as shown in equation (1).

[0086] In order to align the actual vertically upward vector OZreal with the direction of the Z radar axis, the vector OZreal is rotated around the X radar axis and the Y radar axis. When the vector OZreal and the Z radar axis are aligned, the angle by which the vector OZreal is rotated around the X radar axis is defined as θ, and the angle by which the vector OZreal is rotated around the Y radar axis is defined as φ. That is, the angle θ is the angle between the X radar-Z radar plane and the vector OZreal, and the angle φ is the angle between the Y radar-Z radar plane and the vector OZreal. Furthermore, the vector OZreal after movement is defined as the vector OZ'real, and similarly, the X radar axis and the Y radar axis after movement are defined as the X' radar axis and the Y' radar axis.

[0087] FIG. 18 is a diagram showing the relationship between radar and real-world coordinates, part of the logic for converting points in three-dimensional space acquired by radar into points on a two-dimensional map.

[0088] As shown in Fig. 18, an arbitrary point on the map is set as the origin O, and orthogonal Xmap and Ymap axes are given. Furthermore, as the coordinate axes of a three-axis Cartesian coordinate system in real space (called the real space coordinate system), the Xreal axis and the Yreal axis are defined, which have the same origin as the origin O on the map, and are such that the Xreal axis = the Xmap axis and the Xreal axis = the Ymap axis. Let us consider superimposing this real space coordinate system on the radar coordinate system.

[0089] As in Figure 17, after the Z real axis and the Z radar axis are aligned, the two axes obtained by rotating the X real axis and the Y real axis around the Z radar axis (= Z real axis) are defined as the X' real axis and the Y' real axis, respectively. The angle at which the X' real axis and the X radar axis, and the Y' real axis and the Y radar axis are aligned is defined as ψ. The axes obtained by rotating the X' real axis and the Y' real axis by the angle ψ are represented as the X'' real axis and the Y'' real axis, respectively.

[0090] Thereafter, by translating the origin O of the real space coordinate system to the position of the origin O of the radar coordinate system, the real space coordinate system and the radar coordinate system can be made to coincide with each other.

[0091] Next, we will explain the coordinate conversion of radar detection points. When the coordinates of point A in real space are detected by radar, we will consider how to convert them into coordinates on a map. As explained above, we can rotate the radar by angles θ, φ, and ψ, and then translate it.

[0092] The coordinates of point A in the radar coordinate system are expressed by equation (1). The coordinates of point A in the real space coordinate system are expressed by equation (2) below.

[0093] The relationship between Areal and Aradar can be expressed by the following equation (3) using a rotation matrix R and a translation matrix T.

[0094] Here, the rotation matrix R is a rotational movement of angles θ, φ, and ψ, and can be expressed by the following equation (4).

[0095] Furthermore, the translation matrix T is the difference between the origin O of the radar coordinate system and the origin O of the real space coordinate system, and can be expressed by the following equation (5).

[0096] FIG. 19 is a diagram showing the relationship between actual coordinates and map coordinates in the logic for converting points in a three-dimensional space acquired by a radar into points on a two-dimensional map.

[0097] Next, the real space coordinate system is converted into the map coordinate system. As shown in Figure 19, the X real coordinate and Y real coordinate of the real space coordinate system are defined to be the same as the X map coordinate and Y map coordinate of the map coordinate system. Therefore, the point A real in the real space, which has only the X and Y components, becomes the point A map in the map coordinate system.

[0098] When a radar device detects an object, the position of the object defined in a radar coordinate system is detected. According to this embodiment, a user who is the object to be detected by the radar device walks between feature points on a map, and calibration can be performed using only the radar trajectory. Therefore, without using camera images, it is possible to match the point cloud data corresponding to the object's walking trajectory data output by the radar device with the map information of the radar detection range with high accuracy.

[0099] The generated correction formula may be used when plotting the relative position of an object newly detected by the radar device on the map image 2, or may be output to an information processing device connected to the radar device, such as a server or user terminal, to show the position of the object on the map image.

[0100] Second Embodiment In the second embodiment, the radar device 10 detects an object, generates point cloud data, and plots the point cloud data. The map information is stored and the overlay matching process is performed by a terminal externally connected to the radar device, such as a user terminal 20, a server 40, or a personal computer (not shown). Hereinafter, the processing flow of the second embodiment will be described using the user terminal 20 as an example of an information processing device. However, the main entity performing the overlay matching process may be the server 40 or a personal computer (not shown). As in the first embodiment, the calibration mode may be configured to start when the user selects the calibration mode as the operating mode of the radar device 10 and to end when the user selects the normal operating mode as the operating mode of the radar device 10. Alternatively, the calibration mode may be started or ended from the user terminal. The above description of the triggers for starting and ending the calibration mode is merely an example. Other modifications may be possible, such as starting and ending the calibration mode as appropriate in accordance with instructions from an information processing device, such as a server, that can communicate with the radar device 10.

[0101] 20 is a software configuration diagram of the radar device 10 and the user terminal 20 according to the second embodiment. The radar device 10 includes a movement trajectory generation unit 101, a radar device communication unit 106, and an operation mode control unit 110.

[0102] The user terminal 20 includes a correction formula generation unit 205, a user terminal notification unit 206, a map information storage unit 208, a correction formula storage unit 209, and a display control unit 210. The correction formula generation unit 205, the user terminal communication unit 206, and the display control unit 210 are configured by a CPU (not shown) mounted on the user terminal 20 loading programs that realize the functions of each unit into a RAM (not shown) and executing the programs. The map information storage unit 208 and the correction formula storage unit 209 are configured using partial areas of a storage medium (not shown) mounted on the user terminal 20. The functions of each unit of the correction formula generation unit 205, the map information storage unit 208, and the correction formula storage unit 209 are similar to those of the correction formula generation unit 105, the map information storage unit 108, and the correction formula storage unit 109 of the first embodiment, but differ in that the processing of each unit is executed by the user terminal 20 in the second embodiment, whereas it is executed by the radar device 10 in the first embodiment.

[0103] 21 is a sequence diagram showing the flow of processing in the second embodiment. Prior to the following processing, the operation mode control unit 110 sets the radar device 10 to a calibration mode.

[0104] The user inputs map information into the user terminal 20 (S301), and the map information is stored in the map information storage unit 208. In this map information input step, similar to step S101, the scale data of the map is set or the scale data added to the map information is read.

[0105] The display control unit 210 causes a map image based on the map information to be displayed on the display of the user terminal 20 (S202).

[0106] The user uses the user terminal 20 to perform an operation to start radar detection by the radar device 10 (S203). Upon receiving the start operation (S204), the user terminal 20 transmits a start instruction to the radar device 10. Thereafter, the user, who will be the subject of detection by the radar device 10 (the object detected by the radar device 10), walks along a predetermined movement route.

[0107] The radar device 10 starts emitting radio waves (S205) and executes the reflected wave reception and signal analysis process (S102).

[0108] The movement trajectory generation unit 101 acquires the radar detection information, plots point cloud information indicating the relative position of the user over time, and generates a movement trajectory diagram (S103). The radar device 10 transmits the movement trajectory information to the user terminal 20 (S206).

[0109] The display control unit 210 of the user terminal 20 displays the movement trajectory diagram on the display (S207). At this time, an image in which the movement trajectory diagram is superimposed on a two-dimensional radar coordinate system may be displayed on the display of the user terminal 20. This allows the user to visually recognize the plotted points displayed on the user terminal 20 and check their own walking trajectory.

[0110] When the user has finished walking the planned walking route, the user performs an end operation for the radar device 10 on the user terminal 20 (S208), the user terminal 20 accepts the end operation (S209), sends an end operation instruction to the radar device 10, and the radar device 10 ends the radar detection process.

[0111] Thereafter, the user rotates and translates the movement trajectory figure on the user terminal 20 so that the orientation and position of the movement trajectory figure matches the size of the feature figure included in the map image (S302), the user terminal 20 accepts the rotation and translation operations, and the display control unit 210 superimposes the movement trajectory figure after the operations on the map image and displays it on the user terminal 20 (S303).

[0112] The correction equation generation unit 205 of the user terminal 20 executes a rotation / translation matrix generation process based on the rotation and translation operation amounts of the movement trajectory figure accepted in step S303 (S304), thereby completing the correction equation generation process (calibration process).

[0113] Thereafter, the correction formula data is stored in the correction formula memory unit 209 of the user terminal 20, and when the operation mode control unit 110 of the radar device 10 detects a radar in normal operation mode, the user terminal 20 converts the relative position of the new object in the radar coordinate system into the map coordinate system using the correction formula, and creates and displays a composite map 3 (see Figure 1) by superimposing it on the map image 2 (S305).

[0114] According to this example, there is an advantage that the traffic load is low because map information is stored in the user terminal 20 and only point cloud data in the radar coordinate system needs to be received from the radar device 10. Furthermore, there is no need to store map information in the radar device 10, and the radar device 10 does not perform the overlay matching process, so there are advantages that the storage of the radar device 10 is not used and the calculation load of the radar device 10 can be reduced.

[0115] Furthermore, as in the first embodiment, a correction formula for converting the radar coordinate system into the map coordinate system can be generated by generating a rotation / translation matrix for matching the movement trajectory map of the detected object with the characteristic figures included in the map image. Therefore, in this embodiment as well, it is possible to match the point cloud data corresponding to the walking trajectory data of the object output by the radar device with the map information of the radar detection range with high accuracy without using camera images.

[0116] The generated correction formula may be used to plot the relative position of an object newly detected by the radar device on a map image, or may be output to an information processing device connected to the radar device, such as a server or user terminal, to show the position of the object on a map image.

[0117] The above-described embodiment is not intended to limit the present invention, and modifications that do not deviate from the spirit of the present invention are also included in the present invention. For example, in Figures 6 and 20, the user terminal 20 executes the movement trajectory diagram generation process (S103), but the radar device 10 may execute the movement trajectory diagram generation process (S103) and transmit the movement trajectory diagram to the user terminal 20. In this way, modifications in which the functions performed by one device are performed by another device are also included in the present invention.

[0118] This embodiment includes the following invention: (Supplementary Note 1) A radar device for detecting objects, comprising: a radar module that irradiates radio waves to an object in a monitored area, detects the object based on the radio wave reflected by the object, and outputs radar detection information including relative position information of the object; a processor; and a memory, wherein the memory stores map information of the monitored area, and the processor generates a movement trajectory diagram showing the movement trajectory of the detected object based on the radar detection information output after detecting the detected object moving within the monitored area, and generates and stores correction data for matching the movement trajectory diagram with a characteristic diagram corresponding to the movement path of the detected object included in a map image based on the map information.

[0119] (Supplementary Note 2) The radar device according to Supplementary Note 1, wherein the characteristic figure is a closed-shaped figure or an open-shaped figure other than a line segment connecting any two points.

[0120] (Supplementary Note 3) The radar device according to Supplementary Note 1, wherein the characteristic figure is a polygon having two or more vertices.

[0121] (Supplementary Note 4) The radar device according to Supplementary Note 1, wherein the characteristic graphic is a graphic based on the topography of the monitored area.

[0122] (Supplementary Note 5) The radar device according to Supplementary Note 1, wherein the processor determines and outputs a recommended travel route based on the map information.

[0123] (Supplementary Note 6) The radar device according to Supplementary Note 1, further comprising a communication interface for establishing a communication connection with a user terminal, wherein the processor transmits a characteristic figure corresponding to a movement path of the detected object to the user terminal.

[0124] (Supplementary Note 7) A radar device according to Supplementary Note 1, further comprising a communication interface for establishing a communication connection with an information processing device, wherein the processor converts the coordinates of a radar detection point newly detected by the radar device using the correction data, and executes control to superimpose the converted coordinates of the radar detection point on the map image and display them on the information processing device.

[0125] (Supplementary Note 8) A monitoring system comprising: a radar device that irradiates radio waves to an object in a monitored area, detects the object based on the radio wave reflected by the object, and outputs radar detection information including relative position information of the object; and an information processing device that receives the radar detection information, wherein the information processing device includes a processor and a memory, and the memory stores map information of the monitored area, and the processor generates a movement trajectory graphic showing the movement trajectory of the detected object based on the radar detection information output after detecting the detected object moving within the monitored area, generates correction data for matching the movement trajectory graphic with a characteristic graphic corresponding to the movement path of the detected object, which is included in a map image based on the map information, and converts coordinates of a radar detection point newly detected by the radar device using the correction data, and executes control for superimposing and displaying the converted coordinates of the radar detection point on the map image.

[0126] (Supplementary Note 9) The monitoring system according to Supplementary Note 8, wherein the characteristic graphic is an open-shaped graphic or a closed-shaped graphic.

[0127] (Supplementary Note 10) The surveillance system according to Supplementary Note 8, wherein the characteristic graphic is a polygon having two or more vertices.

[0128] (Supplementary Note 11) The monitoring system according to Supplementary Note 8, wherein the characteristic graphic is a graphic based on the topography of the monitored area.

[0129] (Supplementary Note 12) The monitoring system according to Supplementary Note 8, wherein the processor determines and outputs a recommended travel route based on the map information.

[0130] (Appendix 13) A monitoring system according to Appendix 8, wherein the information processing device is a user terminal used by the object to be detected, the user terminal further comprises a display, and the processor displays on the display a characteristic figure corresponding to the movement path of the object to be detected, which is included in a map image based on the map information.

[0131] (Appendix 14) A monitoring system according to Appendix 8, wherein the information processing device is a server communicatively connected to the radar device, the server is communicatively connected to a user terminal used by the detected object, the user terminal further comprising a display, the processor transmits characteristic graphic information to the user terminal for displaying a characteristic graphic corresponding to the movement path of the detected object included in a map image based on the map information, and the user terminal receives the characteristic graphic information and displays the characteristic graphic corresponding to the movement path of the detected object on the display.

[0132] (Supplementary Note 15) A monitoring system according to Supplementary Note 14, wherein the processor transmits movement trajectory information for displaying the movement trajectory diagram to the user terminal, and the user terminal receives the movement trajectory information and displays on the display the movement trajectory diagram based on the movement trajectory information together with a feature diagram corresponding to the movement path of the detected object included in a map image based on the map information.

[0133] (Supplementary Note 16) A calibration method for a radar device that detects objects, the method comprising the steps of: acquiring radar detection information including relative position information of the object, the radar device detecting the object irradiating radio waves to the object in a monitored area, detecting the object based on the reflected waves of the radio waves reflected by the object, and outputting the radar detection information; reading map information of the monitored area; generating a movement trajectory diagram showing the movement trajectory of the detected object based on the radar detection information detected and output; and generating and storing correction data for matching the movement trajectory diagram with a feature diagram corresponding to the movement path of the detected object, which is included in a map image based on the map information.

[0134] 1: Point cloud map, 2: Map image, 3: Composite map, 10: Radar device, 11: CPU, 12: ROM, 13: RAM, 14: Storage medium, 15: Communication I / F, 16: Radar module, 17: Tilt sensor, 18: Bus, 20: User terminal, 30: Network, 31: Screen, 40: Server, 50: PTZ camera, 90: Monitoring system, 101: Movement trajectory generation unit, 102: Comparison processing unit, 103: Superposition matching unit, 104: Image rotation and enlargement processing unit, 105: Correction formula generation unit, 106: Radar device communication unit, 107: Recommended route generation unit, 108: Map information storage unit, 109: Correction formula storage unit, 110: Operation mode control unit, 150: Radar coordinate system, 161: Radar IC, 1021: Learned model storage unit, 1022: AI calculation processing unit, 1023: Image superposition unit, 205: Correction formula generation unit, 206: User terminal communication unit, 208: Map information storage unit, 209: Correction formula storage unit, 210: Display control unit, 211: Line, 212: Installation position mark, 213a: Route, 213b: Route, 213c: Route, 213d: Route, 213e: Route, 213f: Route, 221: Recommended route, 222: Recommended route, 223: Recommended route, 230: Arrow, 250: Dot, 251: Dot, 310: Radar detection diagram, 311: Trajectory, 312: Movement trajectory figure, ARx1: Radar receiving antenna array, ATx1: Radar transmitting antenna array

Claims

1. A radar device for detecting objects, comprising: a radar module that irradiates radio waves onto objects in a monitored area, detects the objects based on the radio wave reflected by the objects, and outputs radar detection information including relative position information of the objects; a processor; and a memory, wherein the memory stores map information of the monitored area, and the processor generates a movement trajectory diagram showing the movement trajectory of the detected object based on the radar detection information output after detecting the detected object moving within the monitored area, and generates and stores correction data for matching the movement trajectory diagram with a characteristic diagram corresponding to the movement path of the detected object included in a map image based on the map information.

2. A radar device according to claim 1, wherein the characteristic figure is a closed figure or an open figure other than a line segment connecting any two points.

3. A radar device according to claim 1, wherein the characteristic figure is a polygon having two or more vertices.

4. A radar device according to claim 1, wherein the characteristic graphic is a graphic based on the topography of the monitored area.

5. A radar device according to claim 1, wherein the processor determines and outputs a recommended travel route based on the map information.

6. A radar device according to claim 1, further comprising a communication interface for establishing a communication connection with a user terminal, wherein the processor transmits to the user terminal a characteristic figure corresponding to the movement path of the detected object.

7. A radar device as claimed in claim 1, further comprising a communication interface for establishing a communication connection with an information processing device, wherein the processor converts the coordinates of a radar detection point newly detected by the radar device using the correction data, and executes control to superimpose the converted coordinates of the radar detection point on the map image and display it on the information processing device.

8. A monitoring system comprising: a radar device that irradiates radio waves to an object in a monitored area, detects the object based on the radio wave reflected by the object, and outputs a movement trajectory diagram showing the movement trajectory of the detected object based on radar detection information including relative position information of the object; and an information processing device that receives the radar detection information, wherein the information processing device includes a processor and a memory, and the memory stores map information of the monitored area, and the processor generates correction data for matching a characteristic diagram corresponding to the movement path of the detected object, which is included in a map image based on the map information, with the movement trajectory diagram, converts the coordinates of a radar detection point newly detected by the radar device using the correction data, and executes control to superimpose and display the converted coordinates of the radar detection point on the map image.

9. A monitoring system according to claim 8, wherein the characteristic graphic is an open-shaped graphic or a closed-shaped graphic.

10. A surveillance system according to claim 8, wherein the characteristic graphic is a polygon having two or more vertices.

11. A surveillance system according to claim 8, wherein the characteristic graphic is a graphic based on the topography of the area to be monitored.

12. A monitoring system according to claim 8, wherein the processor determines and outputs a recommended travel route based on the map information.

13. A monitoring system as described in claim 8, wherein the information processing device is a user terminal used by the detected object, the user terminal further comprises a display, and the processor displays on the display a characteristic figure corresponding to the movement path of the detected object contained in a map image based on the map information.

14. A monitoring system as described in claim 8, wherein the information processing device is a server communicatively connected to the radar device, the server is communicatively connected to a user terminal used by the detected object, the user terminal further comprising a display, the processor transmits to the user terminal characteristic figure information for displaying a characteristic figure corresponding to the movement path of the detected object contained in a map image based on the map information, and the user terminal receives the characteristic figure information and displays on the display the characteristic figure corresponding to the movement path of the detected object.

15. A monitoring system as described in claim 14, wherein the processor transmits movement trajectory information for displaying the movement trajectory diagram to the user terminal, and the user terminal receives the movement trajectory information and displays on the display the movement trajectory diagram based on the movement trajectory information together with a feature diagram corresponding to the movement path of the detected object included in a map image based on the map information.

16. A method for calibrating a radar device that detects objects, comprising the steps of: a processor executing the steps of: acquiring radar detection information including relative position information of an object, the radar device irradiating radio waves to an object in a monitored area, detecting the object based on the radio wave reflected by the object, and outputting the radar detection information; reading map information of the monitored area; generating a movement trajectory diagram showing the movement trajectory of the detected object based on the radar detection information detected and output; and generating and storing correction data for matching the movement trajectory diagram with a characteristic diagram corresponding to the movement path of the detected object, which is included in a map image based on the map information.

Images