Monitoring system, monitoring method, and monitoring program

Abstract

To provide a monitoring system, a monitoring method and a monitoring program that can easily perform real-time synchronous processing in a plurality of pieces of measurement point information.SOLUTION: According to a monitoring system, a monitoring method and a monitoring program, a plurality of pieces of point group data representing road surface within a target area and a measurement point on a surface of an object are acquired using a plurality of measuring apparatuses. The measurement point that is not included in an exclusion area which is defined using a position and a measurement direction of the measuring apparatus as a reference, of measurement points represented by the plurality of pieces of point group data is clustered for each object within the target area to generate a plurality of pieces of shape data. Obstacle data representing an object within the target area is generated on the basis of the plurality of pieces of shape data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a monitoring system, a monitoring method, and a monitoring program. [Background technology]

[0002] Patent Document 1 discloses a traffic information monitoring system that collects information on vehicles, pedestrians, etc. in a monitored area such as an intersection and provides the information to users in order to support vehicle driving and pedestrian safety. According to this technology, information on multiple measurement points is obtained by scanning the intersection with a laser beam, and vehicles present within the intersection are detected based on the information on the multiple measurement points. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-197341 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in Patent Document 1 does not allow the laser radar to emit laser light into areas (blind spots) shadowed by obstructing objects. This may result in failure to detect objects in such areas. For example, when a large truck or similar vehicle crosses an intersection, it may be impossible to detect motorcycles, pedestrians, and other vehicles in the area shadowed by the truck. This phenomenon is called occlusion. To address this problem and improve detection accuracy, multiple laser radars may be installed, and object detection may be performed using measurement point information acquired by each laser radar. However, the amount of data for multiple measurement point information acquired by a laser radar is generally large, which requires a long communication time and may make real-time synchronization difficult.

[0005] The present disclosure has been made in consideration of the above-mentioned circumstances, and aims to provide a monitoring system, a monitoring method, and a monitoring program that can easily perform real-time synchronization processing of multiple measurement point information. [Means for solving the problem]

[0006] A monitoring system according to the present disclosure includes a first measuring device, a second measuring device, and a generating unit. The first measuring device acquires first point cloud data representing measurement points on a road surface and the surface of an object within a target area. The second measuring device is installed at a location different from the first measuring device and acquires second point cloud data representing measurement points on the road surface and the surface of an object within the target area. The generating unit generates obstacle data representing objects within the target area. The first measuring device clusters measurement points represented by the first point cloud data that are not included in a first exclusion area defined based on the position and measurement direction of the first measuring device for each object within the target area to generate first shape data and transmits the first shape data to the generating unit. The second measuring device clusters measurement points represented by the second point cloud data that are not included in a second exclusion area defined based on the position and measurement direction of the second measuring device for each object within the target area to generate second shape data and transmits the second shape data to the generating unit. The generating unit generates the obstacle data based on the first shape data and the second shape data.

[0007] The first exclusion zone may be a zone whose distance from the first measurement device is greater than a predetermined length and a distance determined based on the horizontal angular resolution of the first measurement device, and the second exclusion zone may be a zone whose distance from the second measurement device is greater than a predetermined length and a distance determined based on the horizontal angular resolution of the second measurement device.

[0008] The predetermined length is d, the horizontal angular resolution of the first measurement device is θ1, and the horizontal angular resolution of the second measurement device is θ2. In this case, the first exclusion zone may be a zone whose distance from the first measurement device is greater than d / (2 tan θ1). The second exclusion zone may be a zone whose distance from the second measurement device is greater than d / (2 tan θ2).

[0009] The predetermined length may be set to the minimum width of the monitored object within the target area.

[0010] A monitoring method according to the present disclosure uses a first measuring device to acquire first point cloud data representing measurement points on the road surface and the surface of objects within a target area. Also, a second measuring device installed at a different location from the first measuring device is used to acquire second point cloud data representing measurement points on the road surface and the surface of objects within the target area. Then, among the measurement points represented by the first point cloud data, measurement points that are not included in a first exclusion area defined based on the position and measurement direction of the first measuring device are clustered for each object within the target area to generate first shape data. Also, among the measurement points represented by the second point cloud data, measurement points that are not included in a second exclusion area defined based on the position and measurement direction of the second measuring device are clustered for each object within the target area to generate second shape data. Then, obstacle data representing the objects within the target area is generated based on the first shape data and the second shape data.

[0011] A monitoring program according to the present disclosure causes a computer connected to a first measuring device and a second measuring device installed at a different location from the first measuring device to perform the following steps: Using the first measuring device, the computer acquires first point cloud data representing measurement points on the road surface and the surface of an object within a target area; and Using the second measuring device, the computer acquires second point cloud data representing measurement points on the road surface and the surface of an object within the target area. The computer then generates first shape data by clustering measurement points represented by the first point cloud data that are not included in a first exclusion area defined based on the position and measurement direction of the first measuring device for each object within the target area. The computer also generates second shape data by clustering measurement points represented by the second point cloud data that are not included in a second exclusion area defined based on the position and measurement direction of the second measuring device for each object within the target area. The computer then generates obstacle data representing objects within the target area based on the first shape data and the second shape data. [Effects of the Invention]

[0012] According to the present disclosure, real-time synchronization processing of information on a plurality of measurement points can be easily performed. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram schematically illustrating a configuration of a monitoring system according to an embodiment of the present disclosure. [Figure 2] 2 is a plan view showing an example of the arrangement of the master laser radar device and the slave laser radar device shown in FIG. 1. FIG. [Figure 3] 2 is a perspective view showing an example of the arrangement of the master laser radar device and the slave laser radar device shown in FIG. 1. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, several exemplary embodiments will be described with reference to the drawings. Note that common parts in the drawings are given the same reference numerals, and duplicated explanations will be omitted.

[0015] [Monitoring system configuration] Fig. 1 is a diagram schematically illustrating a configuration of a monitoring system according to an embodiment of the present disclosure. Fig. 2 is a plan view illustrating an example of the arrangement of the master laser radar device and the slave laser radar device illustrated in Fig. 1. Fig. 3 is a perspective view illustrating an example of the arrangement of the master laser radar device and the slave laser radar device illustrated in Fig. 1.

[0016] The monitoring system 1 shown in FIG. 1 is a system that monitors a monitoring area RD (target area) using laser sensors 4 and 6 described below. More specifically, the monitoring system 1 detects objects (moving bodies) present in the monitoring area RD. The monitoring area RD is an area to be monitored. The monitoring area RD can be set at any location on a road. For example, an intersection, a junction, or partway along a road can be selected as a setting location for the monitoring area RD. As shown in FIGS. 2 and 3, the monitoring area R is set at an intersection C. Objects to be monitored include vehicles and pedestrians (people).

[0017] The monitoring system 1 includes a slave laser radar device 2 (first measurement device) and a master laser radar device 3 (second measurement device). The slave laser radar device 2 and the master laser radar device 3 are installed near a monitoring region RD. In the example of FIGS. 2 and 3, the slave laser radar device 2 and the master laser radar device 3 are installed diagonally across an intersection C. The intersection C is a point where four roads TR1 to TR4 converge.

[0018] Roads TR1 and TR2 extend in one direction and are connected to each other via an intersection C. Roads TR3 and TR4 extend in a direction that intersects with roads TR1 and TR2 and are connected to each other via an intersection C. The slave laser radar device 2 and the master laser radar device 3 are fixed to a support member P (see FIG. 3) installed on the ground. The support member P is, for example, a columnar structure installed on the roadside near the intersection C. The support member P may also be a utility pole or a warehouse wall.

[0019] The slave laser radar device 2 and the master laser radar device 3 are connected to each other via a communication line so that they can communicate with each other. The communication line may be either wired or wireless. The communication line may be a non-dedicated line such as an Internet line or a mobile communication network, or it may be a dedicated line. Note that at intersections such as intersection C, the distance between the slave laser radar device 2 and the master laser radar device 3 is long, making it difficult to wire a dedicated line as a communication line. For this reason, in the examples of Figures 2 and 3, a wireless communication line may be used as the communication line.

[0020] [Configuration of the slave laser radar device] The slave laser radar device 2 includes a laser sensor 4 and a processing device 5. The laser sensor 4 is fixed to a support member P. In other words, the laser sensor 4 is installed on the ground.

[0021] Note that the laser sensor 4 does not need to be installed on the ground as long as its position relative to the monitoring region RD is fixed. The laser sensor 4 may be installed on a drone, for example, and may be floating in the air.

[0022] The laser sensor 4 emits laser light toward the irradiable area RA and receives reflected light of the emitted laser light to generate measurement point information for each measurement point within the irradiable area RA. The irradiable area RA is an area where the laser sensor 4 can emit laser light, and is, for example, a range of about 150 m. The irradiable area RA includes at least a part of the monitoring area RD.

[0023] The measurement point information includes time information and position information. The time information is information indicating the time when the measurement point information of the measurement point indicated by the position information was generated (when reflected light was received). The position information is information indicating the position coordinates of the measurement point. For the position coordinates, a polar coordinate system represented by a yaw angle, a pitch angle, and a depth may be used, or a three-dimensional coordinate system represented by an x-coordinate, a y-coordinate, and a z-coordinate may be used.

[0024] The coordinate system CS used in the embodiments of the present disclosure is a three-dimensional coordinate system. The x-axis of the coordinate system CS extends along roads TR1 and TR2, and is set so that the direction from road TR1 to road TR2 is positive. The y-coordinate of the coordinate system CS extends along roads TR3 and TR4, and is set so that the direction from road TR3 to road TR4 is positive. The z-axis of the coordinate system CS is set so that the ground surface is used as the reference (z=0), and that points above the ground surface are positive. The measurement point information may further include reflection intensity information. The reflection intensity information is information indicating the intensity of reflected light received from the measurement point indicated by the position information at the time indicated by the time information.

[0025] The laser sensor 4 scans the irradiatable area RA in the main scanning direction and the sub-scanning direction by changing the irradiation direction of the laser light. As a result, the laser light is sequentially irradiated onto multiple measurement points included in the irradiatable area RA. A complete irradiation of all measurement points included in the irradiatable area RA with the laser light may be referred to as one frame. The irradiation of the irradiatable area RA with the laser light is repeated at predetermined time intervals. The laser sensor 4 outputs point cloud information DM1 (first point cloud data) including information on multiple measurement points for one frame to the processing device 5 for each frame. The point cloud information DM1 may further include a frame ID (identifier). The frame ID is identification information that can uniquely identify a frame. For example, a frame number indicating the order of the frames may be used as the frame ID.

[0026] The processing device 5 processes point cloud information DM1 including information on a plurality of measurement points generated by the laser sensor 4, thereby generating partial information DP1 (first shape data).

[0027] The processing device 5 (controller) is configured, for example, by a general-purpose microcomputer equipped with a CPU (central processing unit), memory, and input / output units. A computer program (monitoring program) for functioning as a monitoring device is installed in the processing device 5. By executing the computer program, the processing device 5 functions as multiple information processing circuits (51, 52, 53, 54) equipped in the monitoring device. The computer program (monitoring program) may be stored in a storage medium readable and writable by a computer.

[0028] In this disclosure, an example is shown in which the multiple information processing circuits (51, 52, 53, 54) are realized by software. However, it is also possible to configure the information processing circuits (51, 52, 53, 54) by preparing dedicated hardware for executing each of the information processes described below. Also, the multiple information processing circuits (51, 52, 53, 54) may be configured by individual hardware.

[0029] As shown in FIG. 1, the processing device 5 includes an acquisition unit 51, a storage unit 52, a processing unit 53, and an output unit 54 as a plurality of information processing circuits (51, 52, 53, 54).

[0030] The acquisition unit 51 acquires the point cloud information DM1 from the laser sensor 4. The acquisition unit 51 outputs the acquired point cloud information DM1 to the processing unit 53.

[0031] The memory unit 52 stores various setting information. The various setting information includes exclusion information indicating a detection exclusion range (first exclusion area). The detection exclusion range is a range that is not monitored. The detection exclusion range includes a range outside the monitoring area RD and a range within the monitoring area RD that does not need to be monitored. The detection exclusion range is set in advance. For example, the detection exclusion range may be set by a user using an input device (not shown). For example, a three-dimensional space simulating intersection C is displayed on an output device (not shown) such as a display, and the user sets the detection exclusion range using a frame or the like.

[0032] For example, areas below the ground level and the airspace above the road may not be monitored. Therefore, in order to exclude areas below the ground level, a range lower than a predetermined height from the ground level may be set as the height of the detection exclusion range. For example, a range of z<20 cm may be set as the detection exclusion range. In order to exclude areas above the road, a range higher than a predetermined height from the ground level may be set as the height of the detection exclusion range. For example, a range of z>500 cm may be set as the detection exclusion range.

[0033] Furthermore, there are fixed objects (stationary objects) such as traffic lights, poles, utility poles, roadside trees, and overpasses within and around intersection C. These fixed objects do not need to be monitored, so a range of fixed objects may be set as a detection exclusion range to exclude these fixed objects. The range of fixed objects may be set as coordinates (x, y) that indicate the boundary of the fixed objects on the xy plane (horizontal plane).

[0034] Alternatively, the detection exclusion range (first exclusion area) may be determined based on the horizontal angular resolution of the laser sensor 4. Specifically, it may be determined as an area whose distance from the laser sensor 4 is greater than a predetermined length and a distance determined based on the horizontal angular resolution of the laser sensor 4. Here, the horizontal angular resolution of the laser sensor 4 is represented by the angle formed between adjacent lines connecting the measurement point and the laser sensor 4.

[0035] For example, the first exclusion zone may be defined as a zone that satisfies the constraint "S1>d / {2·tan(θ1)}" where d is the predetermined length, θ1 is the horizontal angular resolution of the laser sensor 4, and S1 is the distance from the laser sensor 4. Here, the constraint regarding the distance S1 is derived from the condition that fewer than three measurement points can be measured horizontally for an object of a predetermined length located at the distance S1 from the laser sensor 4. When three or more measurement points can be measured horizontally, it is possible to measure the area of ​​an image obtained by projecting a portion of the object onto the xy plane. By setting the first exclusion zone, the combining unit 83 (described later) can determine whether multiple detected objects are the same.

[0036] The predetermined length is set to the minimum width of the monitored object within the monitoring area RD. The monitored objects present in the monitoring area RD include vehicles and pedestrians (people), and the predetermined length is set appropriately depending on the type of monitored object.

[0037] In this way, the first exclusion area is an area where the distance from the laser sensor 4 is greater than the distance determined based on the horizontal angular resolution of the laser sensor 4. As a result, objects that are difficult to measure with sufficient angular resolution can be excluded from the measurement target, and the processing by the processing unit 53, which will be described later, can be reduced. Furthermore, the size of the data transmitted by the output unit 54 can be reduced.

[0038] The processing unit 53 processes the point cloud information DM1 to generate partial information DP1. Specifically, when the processing unit 53 receives the point cloud information DM1 from the acquisition unit 51, the processing unit 53 excludes (deletes) from the point cloud information DM1 the measurement point information of the measurement points included in the detection exclusion range indicated by the exclusion information read from the storage unit 52. This excludes measurement points that are not subject to monitoring and also excludes abnormal measurement point information caused by noise, etc. The processing unit 53 outputs the partial information DP1 including the remaining measurement point information to the output unit 54. The partial information DP1 may further include a frame ID.

[0039] Alternatively, the processing unit 53 may exclude measurement point information of measurement points included in the detection exclusion range from the point cloud information DM1 and cluster the excluded measurement point information to generate partial information DP1. For example, the processing unit 53 may perform clustering processing by connecting nearby measurement points among the multiple measurement points in the monitoring area RD to divide them into clusters (blocks). Examples of clustering techniques include, but are not limited to, Euclidean clustering. In Euclidean clustering, measurement points whose Euclidean distance is less than a predetermined distance are classified as belonging to the same cluster, and measurement points whose Euclidean distance is equal to or greater than the predetermined distance are classified as belonging to different clusters. The processing unit 53 may detect the obtained cluster as a single detected object (e.g., a vehicle or a person).

[0040] The processing unit 53 calculates the dimensions (width, depth, and height) and position of the detected object. The position of the detected object may be the coordinates of the four corners (front right end, front left end, rear right end, and rear left end) of the detected object, the average of the positions of the measurement point information included in the cluster, or the center of gravity of the detected object.

[0041] The output unit 54 outputs the partial information DP1 to the outside of the processing device 5. Specifically, when the output unit 54 receives the partial information DP1 from the processing unit 53, it transmits the partial information DP1 to the master laser radar device 3 via a communication line.

[0042] In this way, the processing device 5 generates partial information DP1 by deleting the measurement point information of the measurement points included in the detection exclusion range from the point cloud information DM1, and transmits the partial information DP1 to the master laser radar device 3. Therefore, the data amount of the partial information DP1 is smaller than the data amount of the point cloud information DM1.

[0043] [Configuration of the master laser radar device] The master laser radar device 3 includes a laser sensor 6 (second measuring device), a processing device 7, and a detection device 8. The laser sensor 6 is fixed to a support member P. In other words, the laser sensor 6 is installed on the ground.

[0044] Note that the laser sensor 6 does not need to be installed on the ground as long as its position relative to the monitoring region RD is fixed. The laser sensor 6 may be installed on a drone, for example, and may be floating in the air.

[0045] Like the laser sensor 4, the laser sensor 6 emits laser light toward the irradiable area RB and receives reflected light of the emitted laser light to generate measurement point information for each measurement point within the irradiable area RB. The irradiable area RB is an area where the laser sensor 6 can emit laser light, and is, for example, a range of about 150 m. The irradiable area RB includes at least a part of the monitoring area RD.

[0046] The laser sensor 6 also uses the same coordinate system CS as the laser sensor 4. The laser sensor 6 outputs point cloud information DM2 (second point cloud data) including information on a plurality of measurement points for one frame to the processing device 7 for each frame. The point cloud information DM2 may further include a frame ID.

[0047] The time of the laser sensor 6 is synchronized with the time of the laser sensor 4. The time of the laser sensor 4 and the time of the laser sensor 6 are synchronized using, for example, NTP (Network Time Protocol). Therefore, the start time and end time of one frame of the laser sensor 6 match the start time and end time of one frame of the laser sensor 4. The start time of one frame is the time when the first measurement point information of that frame is generated (acquired). The end time of one frame is the time when the last measurement point information of that frame is generated (acquired).

[0048] The processing device 7 is a device that processes point cloud information DM2 that includes information on a plurality of measurement points generated by the laser sensor 6, to generate partial information DP2 (second shape data).

[0049] The processing device 7 (control unit) is configured, for example, by a general-purpose microcomputer equipped with a CPU (central processing unit), memory, and input / output units. A computer program (monitoring program) for functioning as a monitoring device is installed in the processing device 7. By executing the computer program, the processing device 7 functions as multiple information processing circuits (71, 72, 73, 74) equipped in the monitoring device. The computer program (monitoring program) may be stored in a storage medium readable and writable by a computer.

[0050] In this disclosure, an example is shown in which the multiple information processing circuits (71, 72, 73, 74) are realized by software. However, it is also possible to configure the information processing circuits (71, 72, 73, 74) by preparing dedicated hardware for executing each of the information processes described below. Also, the multiple information processing circuits (71, 72, 73, 74) may be configured by individual hardware.

[0051] As shown in FIG. 1, the processing device 5 includes an acquisition unit 71, a storage unit 72, a processing unit 73, and an output unit 74 as a plurality of information processing circuits (71, 72, 73, 74).

[0052] The acquisition unit 71 acquires the point cloud information DM2 from the laser sensor 6. The acquisition unit 71 outputs the acquired point cloud information DM2 to the processing unit 73.

[0053] The memory unit 72 stores various setting information. The various setting information includes exclusion information indicating a detection exclusion range (second exclusion area). The detection exclusion range is set in advance. For example, similar to the slave laser radar device 2, the user sets the detection exclusion range using an input device. The detection exclusion range set in the processing device 7 may be the same as or different from the detection exclusion range set in the processing device 5. When the same detection exclusion range is set in the processing device 5 and the processing device 7, the processing device 7 may transmit exclusion information indicating the detection exclusion range set in the processing device 7 to the processing device 5.

[0054] Alternatively, the detection exclusion range (second exclusion area) may be determined based on the horizontal angular resolution of the laser sensor 6. Specifically, it may be determined as an area whose distance from the laser sensor 6 is greater than a predetermined length and a distance determined based on the horizontal angular resolution of the laser sensor 6. Here, the horizontal angular resolution of the laser sensor 6 is represented by the angle formed between adjacent lines connecting the measurement point and the laser sensor 6.

[0055] For example, the second exclusion zone may be defined as a zone that satisfies the constraint "S2>d / {2·tan(θ2)}" where d is the predetermined length, θ2 is the horizontal angular resolution of the laser sensor 6, and S2 is the distance from the laser sensor 6. Here, the constraint regarding the distance S2 is derived from the condition that fewer than three measurement points can be measured horizontally for an object of a predetermined length located at the distance S2 from the laser sensor 6. When three or more measurement points can be measured horizontally, it is possible to measure the area of ​​an image obtained by projecting a portion of the object onto the xy plane. By setting the second exclusion zone, the combining unit 83 (described later) can determine whether multiple detected objects are the same.

[0056] In this way, the second exclusion area is an area where the distance from the laser sensor 6 is greater than the distance determined based on the horizontal angular resolution of the laser sensor 6. As a result, objects that are difficult to measure with sufficient angular resolution can be excluded from the measurement target, and the processing by the processing unit 73, which will be described later, can be reduced. Furthermore, the size of the data transmitted by the output unit 74 can be reduced.

[0057] The processing unit 73 processes the point cloud information DM2 to generate partial information DP2. Specifically, when the processing unit 73 receives the point cloud information DM2 from the acquisition unit 71, it excludes from the point cloud information DM2 the measurement point information of the measurement points included in the detection exclusion range indicated by the exclusion information read from the storage unit 72. This excludes measurement points that are not subject to monitoring and also excludes abnormal measurement point information caused by noise, etc. The processing unit 73 outputs the partial information DP2 including the remaining measurement point information to the output unit 74. The partial information DP2 may further include a frame ID.

[0058] Alternatively, the processing unit 73 may exclude measurement point information of measurement points included in the detection exclusion range from the point cloud information DM2, and cluster the plurality of measurement point information after the exclusion to generate partial information DP2. For example, the processing unit 73 may perform the clustering process by connecting nearby measurement points among the plurality of measurement points in the monitoring area RD to divide them into clusters (blocks). The processing unit 73 may detect the obtained clusters as single detected objects (vehicles, people, etc.).

[0059] The processing unit 73 calculates the dimensions (width, depth, and height) and position of the detected object. The position of the detected object may be the coordinates of the four corners of the detected object (front right end, front left end, rear right end, and rear left end), the average of the positions of the measurement point information included in the cluster, or the center of gravity of the detected object.

[0060] The output unit 74 outputs the partial information DP2 to the outside of the processing device 7. Specifically, upon receiving the partial information DP2 from the processing unit 73, the output unit 74 outputs the partial information DP2 to the detection device 8.

[0061] In this way, the processing device 7 generates partial information DP2 by deleting the measurement point information of the measurement points included in the detection exclusion range from the point cloud information DM2, and outputs the partial information DP2 to the detection device 8. Therefore, the data amount of the partial information DP2 is smaller than the data amount of the point cloud information DM2.

[0062] The detection device 8 is a device that generates a detection result in the monitoring region RD based on the partial information DP1 and the partial information DP2. The detection device 8 is configured, for example, by a general-purpose microcomputer equipped with a CPU (Central Processing Unit), memory, and an input / output unit. A computer program (monitoring program) that functions as a monitoring device is installed in the detection device 8. By executing the computer program, the detection device 8 functions as multiple information processing circuits (81, 82, 83, 84, 85) that the monitoring device has. Note that the computer program (monitoring program) may be stored in a storage medium that can be read and written by a computer.

[0063] In this disclosure, an example is shown in which multiple information processing circuits (81, 82, 83, 84, 85) are realized by software. However, it is also possible to configure the information processing circuits (81, 82, 83, 84, 85) by preparing dedicated hardware for executing each of the information processes described below. Also, the multiple information processing circuits (81, 82, 83, 84, 85) may be configured by individual hardware.

[0064] As shown in FIG. 1, the detection device 8 includes an acquisition unit 81, a storage unit 82, a synthesis unit 83 (generation unit), a detection unit 84, and an output unit 85 as a plurality of information processing circuits (81, 82, 83, 84, 85).

[0065] The acquisition unit 81 acquires the partial information DP1 from the processing device 5 and acquires the partial information DP2 from the processing device 7. Due to a transmission delay or the like, the timing at which the acquisition unit 81 receives the partial information DP1 and the timing at which the acquisition unit 81 receives the partial information DP2 may differ from each other, even in the same frame.

[0066] 1, the partial information DP1 is transmitted via a communication line, whereas the partial information DP2 is transmitted via a communication line inside the master laser radar device 3. Therefore, the acquisition unit 81 receives the partial information DP1 after receiving the partial information DP2. Therefore, the acquisition unit 81 outputs the partial information DP2 to the storage unit 82, where it is stored. The acquisition unit 81 outputs the partial information DP1 to the synthesis unit 83.

[0067] The storage unit 82 stores the partial information DP2 for each frame. The storage unit 82 may store the partial information DP2 in a different file for each frame. The storage unit 82 may sort the multiple pieces of measurement point information included in the partial information DP2 received from the processing device 7 by the time indicated by the time information of each piece of measurement point information, and store the partial information DP2 for each frame. In this case, since the frame can be identified from the coordinates of the measurement points, the frame ID may be omitted.

[0068] The synthesis unit 83 (generation unit) generates synthesized information by synthesizing (merging) the partial information DP1 with the partial information DP2 of the same frame as the frame of the partial information DP1. Specifically, when the synthesis unit 83 receives the partial information DP1 from the acquisition unit 81, it first extracts the earliest time and the latest time from among the times indicated by the time information of the multiple pieces of measurement point information included in the partial information DP1.

[0069] The start time and end time of one frame of the laser sensor 4 match the start time and end time of one frame of the laser sensor 6. Therefore, for the same frame, the earliest and latest times of the partial information DP1 roughly match the earliest and latest times of the partial information DP2. Therefore, the synthesis unit 83 acquires, from the partial information DP2 of multiple frames stored in the memory unit 82, the partial information DP2 of the frame having the earliest and latest times close to the extracted earliest and latest times.

[0070] The storage unit 82 may store the start time and end time of each frame in advance. In this case, the synthesis unit 83 identifies a frame whose start time and end time include the time period defined by the earliest and latest times of the partial information DP1, and acquires the partial information DP2 of that frame. Furthermore, if the partial information DP1 and the partial information DP2 include a frame ID, the synthesis unit 83 acquires, from the partial information DP2 of multiple frames stored in the storage unit 82, the partial information DP2 that includes the same frame ID as the frame ID of the partial information DP1.

[0071] The combining unit 83 combines the partial information DP1 and the partial information DP2. More specifically, the combining unit 83 generates combined information including the plurality of pieces of measurement point information included in the partial information DP1 and the plurality of pieces of measurement point information included in the partial information DP2. The combined information can be expressed in the form of a table that lists (arranges) the plurality of pieces of measurement point information included in the partial information DP1 and the plurality of pieces of measurement point information included in the partial information DP2, for example.

[0072] The partial information DP1 and the partial information DP2 may include measurement point information of the same measurement point. Therefore, if the two pieces of measurement point information are measurement point information of the same measurement point, the combining unit 83 may delete one of them. For example, if the distance between the position coordinates of the two pieces of measurement point information is equal to or less than a preset distance, the combining unit 83 may determine that these two pieces of measurement point information are measurement point information of the same measurement point. The combining unit 83 outputs the combined information to the detection unit 84.

[0073] If the partial information DP1 and the partial information DP2 are data after clustering, the synthesis unit 83 may determine whether the clusters represent the same detected object (a vehicle, a person, etc.) based on the overlap between the clusters.

[0074] For example, the synthesis unit 83 calculates images obtained by projecting onto the xy plane a cluster included in the partial information DP1 and a cluster included in the partial information DP2. The synthesis unit 83 may determine that the cluster included in the partial information DP1 and the cluster included in the partial information DP2 are the same detected object when the ratio of the area of ​​the overlapping portion of the calculated images to the entire area of ​​the original images is equal to or greater than a predetermined threshold.

[0075] Alternatively, the synthesis unit 83 may determine a common part between an area surrounded by a cluster included in the partial information DP1 and an area surrounded by a cluster included in the partial information DP2, and may determine that the cluster included in the partial information DP1 and the cluster included in the partial information DP2 are the same detected object if the ratio of the volume of the common part to the volume of the area surrounded by the original clusters is equal to or greater than a predetermined threshold.

[0076] When the detected objects are determined to be the same by the above-described method, the synthesis unit 83 may integrate both clusters to calculate synthesis information.

[0077] The detection unit 84 (generation unit) generates obstacle data representing objects within the target area based on the synthesis information. Specifically, when the detection unit 84 receives the synthesis information from the synthesis unit 83, it clusters the information on multiple measurement points included in the synthesis information. That is, the detection unit 84 connects nearby measurement points among the multiple measurement points within the monitoring area RD and divides them into clusters (lumps) as obstacle data. The detection unit 84 detects the obtained clusters as single detected objects (vehicles, people, etc.).

[0078] The detection unit 84 may calculate the dimensions (width, depth, and height) and position of a detected object as obstacle data. The position of a detected object may be the coordinates of the four corners of the detected object (front right end, front left end, rear right end, and rear left end), the average of the positions of the measurement point information included in the cluster, or the center of gravity of the detected object.

[0079] The detection unit 84 outputs the detection result for the detected object to the output unit 85. The detection result includes dimensional information indicating the dimensions of the detected object, position information indicating the position of the detected object, and detection time information indicating the detection time when the detected object was detected. The detection time is, for example, the average time of the times indicated by the time information included in the measurement point information of each measurement point included in the cluster.

[0080] The detection unit 84 may track the detected object. That is, the detection unit 84 may associate an object ID with a detected object detected in a different frame (different time). The object ID is identification information that can uniquely identify a detected object. Specifically, the detection unit 84 determines whether a detected object detected in the current frame corresponds to any of the detected objects detected in the past frames, based on the position and dimensions of the detected object, as well as the velocity and angular velocity estimated from past observation results.

[0081] When the detection unit 84 determines that a detected object detected in the current frame does not correspond to any of the detected objects detected in past frames, it assigns a new object ID to the detected object as a new detected object. When the detection unit 84 determines that a detected object detected in the current frame corresponds to a detected object detected in a past frame, it assigns the object ID assigned to the corresponding detected object to the detected object detected in the current frame. The detection unit 84 deletes the object ID of a detected object that has not been detected for a long time, among the detected objects that have been assigned an object ID.

[0082] The problem of tracking (assigning IDs to) multiple detected objects is called the multi-target tracking problem. The detection unit 84 tracks each detected object using a known algorithm. Known algorithms include SNN (Suboptimal Nearest Neighbor) and GNN (Global Nearest Neighbor). Other known algorithms include JPDAF (Joint Probabilistic Data Association Filter). In these cases, the detection unit 84 outputs the object ID, object position information, detection time information, etc. to the output unit 85 as the detection result.

[0083] The output unit 85 outputs the detection results to the outside of the detection device 8 (master laser radar device 3). The output unit 85 transmits the detection results to, for example, an external device (not shown). Examples of the external device include a higher-level management system and a traffic control system. The detection results may be transmitted by wireless communication or by wired communication. The output unit 85 may convert the data format of the detection results into a data format that is easy for the external device to process, and transmit the converted detection results to the external device.

[0084] [Effects of the embodiment] As described above in detail, the monitoring system, monitoring method, and monitoring program according to the present disclosure use a first measurement device to acquire first point cloud data representing measurement points on the road surface and the surface of an object within a target area. Also, a second measurement device installed at a different location from the first measurement device is used to acquire second point cloud data representing measurement points on the road surface and the surface of an object within the target area. Then, among the measurement points represented by the first point cloud data, measurement points that are not included in a first exclusion area defined based on the position and measurement direction of the first measurement device are clustered for each object within the target area to generate first shape data. Also, among the measurement points represented by the second point cloud data, measurement points that are not included in a second exclusion area defined based on the position and measurement direction of the second measurement device are clustered for each object within the target area to generate second shape data. Then, obstacle data representing the objects within the target area is generated based on the first shape data and the second shape data.

[0085] This makes it easy to perform real-time synchronization processing for multiple pieces of measurement point information. In particular, even when multiple measurement devices are installed and object detection is performed using measurement point information acquired by each measurement device, the amount of data required to transmit multiple pieces of measurement point information can be reduced. As a result, communication time is shortened and real-time synchronization processing can be easily performed. In addition, because obstacle data is generated based on shape data generated by clustering each object in the target area, the amount of data required to transmit multiple pieces of measurement point information can be further reduced.

[0086] The first exclusion zone may be a zone whose distance from the first measurement device is greater than a predetermined length and a distance determined based on the horizontal angular resolution of the first measurement device, and the second exclusion zone may be a zone whose distance from the second measurement device is greater than a predetermined length and a distance determined based on the horizontal angular resolution of the second measurement device.

[0087] This allows distant objects that are difficult to track due to low detection accuracy to be excluded from the measurement target. As a result, the load associated with object tracking processing can be reduced. Furthermore, the amount of data required to transmit information on multiple measurement points can be reduced, shortening communication time and facilitating real-time synchronization processing.

[0088] The predetermined length is d, the horizontal angular resolution of the first measurement device is θ1, and the horizontal angular resolution of the second measurement device is θ2. In this case, the first exclusion zone may be a zone whose distance from the first measurement device is greater than d / (2 tan θ1). The second exclusion zone may be a zone whose distance from the second measurement device is greater than d / (2 tan θ2).

[0089] This makes it possible to exclude from the measurement target distant objects that are difficult to track due to low detection accuracy. As a result, the load associated with the object tracking process can be reduced. Furthermore, it is guaranteed that at least three or more measurement points are measured in the horizontal direction for the object to be measured. As a result, the object tracking process can be reliably performed for the object to be measured. Furthermore, the object tracking process can be continued for the object to be measured.

[0090] The predetermined length may be set to the minimum width of the monitored object within the target area. This allows the exclusion area to be set to match the monitored object that is expected to pass through the target area. Furthermore, it is guaranteed that at least three or more measurement points are measured horizontally on the monitored object. As a result, the tracking process of the monitored object can be continued.

[0091] Each function described in the above embodiments may be implemented by one or more processing circuits, including programmed processors, electrical circuits, and even devices such as application specific integrated circuits (ASICs), or circuit components arranged to perform the described functions.

[0092] Although several embodiments have been described, the embodiments can be modified or varied based on the above disclosure. All components of the above embodiments and all features described in the claims may be individually extracted and combined, unless they contradict each other. [Explanation of symbols]

[0093] 1. Surveillance System 2. Slave laser radar device (first measuring device) 3 Master laser radar device (second measuring device) 4 laser sensors 5 Processing equipment 6 laser sensors 7 Processing equipment 8. Detection Device DM1 Point cloud information (first point cloud data) DM2 Point Cloud Information (Second Point Cloud Data) DP1 Part information (first shape data) DP2 Partial information (second shape data) RA irradiation area RB irradiation area RD monitoring area

Claims

1. a first measuring device that acquires first point cloud data representing measurement points on a road surface and an object surface within a target area; a second measuring device that is installed at a position different from the first measuring device and that acquires second point cloud data representing measurement points on a road surface and an object surface within the target area; a generator for generating obstacle data representing objects within the target area; A monitoring system comprising: The first measuring device is generating first shape data by clustering measurement points represented by the first point cloud data that are not included in a first exclusion area, where the distance from the first measurement device is greater than a distance determined based on the minimum width of the monitored object within the target area and the horizontal angular resolution of the first measurement device, for each object within the target area; transmitting the first shape data to the generating unit; The second measuring device is generating second shape data by clustering measurement points represented by the second point cloud data that are not included in a second exclusion area, where the distance from the second measurement device is greater than a distance determined based on the minimum width of the monitored object and the horizontal angular resolution of the second measurement device, for each object within the target area; transmitting the second shape data to the generating unit; The generation unit generates the obstacle data based on the first shape data and the second shape data.

2. the first exclusion area is an area whose distance from the first measurement device is greater than a distance determined based on a minimum width of the monitored object within the target area and a horizontal angular resolution of the first measurement device, 2. The monitoring system of claim 1, wherein the second exclusion area is an area whose distance from the second measurement device is greater than a distance determined based on the minimum width of the monitored object within the target area and the horizontal angular resolution of the second measurement device.

3. The minimum width of the monitored object is d; The horizontal angular resolution of the first measuring device is θ1, The horizontal angular resolution of the second measurement device is θ2, the first exclusion region is a region whose distance from the first measurement device is greater than d / (2 tan θ1), The monitoring system according to claim 2 , wherein the second exclusion zone is a zone whose distance from the second measurement device is greater than d / (2·tan θ 2 ).

4. using a first measurement device to acquire first point cloud data representing measurement points on a road surface and an object surface within a target area; acquiring second point cloud data representing measurement points on a road surface and an object surface within the target area using a second measurement device installed at a position different from that of the first measurement device; generating first shape data by clustering measurement points represented by the first point cloud data that are not included in a first exclusion area, where the distance from the first measurement device is greater than a distance determined based on the minimum width of the monitored object within the target area and the horizontal angular resolution of the first measurement device, for each object within the target area; generating second shape data by clustering measurement points represented by the second point cloud data that are not included in a second exclusion area, where the distance from the second measurement device is greater than a distance determined based on the minimum width of the monitored object and the horizontal angular resolution of the second measurement device, for each object within the target area; generating obstacle data representing objects within the region of interest based on the first shape data and the second shape data.

5. a computer connected to a first measurement device and a second measurement device installed at a different location from the first measurement device; acquiring first point cloud data representing measurement points on a road surface and an object surface within a target area using the first measurement device; acquiring second point cloud data representing measurement points on a road surface and an object surface within the target area using the second measurement device; generating first shape data by clustering measurement points represented by the first point cloud data that are not included in a first exclusion area, where the distance from the first measurement device is greater than a distance determined based on a minimum width of the monitored object in the target area and a horizontal angular resolution of the first measurement device, for each object in the target area; generating second shape data by clustering, for each object within the target area, measurement points represented by the second point cloud data that are not included in a second exclusion area, where the distance from the second measurement device is greater than a distance determined based on the minimum width of the monitored object and the horizontal angular resolution of the second measurement device; generating obstacle data representing objects within the target region based on the first shape data and the second shape data; A monitoring program to run the program.

Images