Information processing device

Abstract

To provide an information processing device capable of detecting a vehicle from image data with high accuracy.SOLUTION: An information processing device 10 includes: an object extraction unit 42 that extracts an object indicating a vehicle from image data for a traffic intersection acquired by an imaging device 60; and a counting unit 46 that sets a first counting line on an entrance side of the traffic intersection and a second counting line on a traffic lane on a front side as viewed from the imaging device 60 on an exit side of the traffic intersection on the image data, and counts the number of objects that have passed through the second counting line without passing through the first counting line.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device that counts objects representing vehicles extracted from image data. [Background technology]

[0002] When measuring traffic volume, vehicles traveling on a road are extracted from image data and counted.

[0003] Patent Document 1 discloses an invention that measures the amount of movement of a vehicle from a captured image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-086467 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the invention described in Patent Document 1 was prone to a phenomenon in which the ID of a vehicle detected from an image was switched when the vehicle was temporarily blocked by an obstruction, which could result in a decrease in the accuracy of vehicle counting.

[0006] In consideration of the above, an object of the present invention is to provide an information processing device that can accurately detect a vehicle from image data. [Means for solving the problem]

[0007] To achieve the above objectives According to the first aspect The information processing device includes an object extraction unit that extracts an object representing a vehicle from image data of the intersection acquired by the imaging device; Corresponding to the upstream of the movement trajectory of the object not to be countedThe intersection includes a counting unit that sets a first counting line on the entrance side of the intersection and a second counting line on the exit side of the intersection, and counts the objects that pass through the second counting line without passing through the first counting line.

[0008] First Aspect According to the method, by setting up a second counting line in a position where the vehicle is less likely to be blocked, and counting only objects that pass through the second counting line without passing through the first counting line, it is possible to suppress changes in the object's ID due to blocking objects. Furthermore, according to the first aspect, by providing the first counting line upstream of the movement trajectory of the object that is not to be counted, it is possible to measure the traffic volume excluding the object that is not to be counted.

[0016] The second aspect is the first aspect, The object extraction unit detects the objects for each class set for each type of vehicle, and the counting unit counts the objects for each class.

[0017] Second Aspect According to this method, objects can be counted by class, such as light vehicles, passenger cars, and freight vehicles. [Effects of the Invention]

[0018] As described above, the information processing device according to the present invention can accurately detect a vehicle from image data. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 2 is a block diagram showing an example of a specific configuration of the information processing device according to the present embodiment. [Figure 2] 10 is a flowchart illustrating an example of processing by the information processing device according to the present embodiment. [Figure 3] 10A shows a state in which an object to be counted is occluded by another object, and FIG. 10B shows a state in which the object to be counted has passed through a second counting line. [Figure 4] FIG. 10A is an explanatory diagram of a case where an object that is not a counting target is about to pass through a first counting line, and FIG. 10B is an explanatory diagram of a case where an object that is not a counting target has passed through a second counting line. [Figure 5]10A is an explanatory diagram illustrating a case where a movement trajectory to be counted is displayed briefly in a corner of a captured image, and FIG. 10B is an enlarged view of FIG. 10A. [Figure 6] FIG. 10 is an explanatory diagram showing an example of counting in a modified example of the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] An information processing device 10 according to this embodiment will be described below with reference to Fig. 1. Fig. 1 is a block diagram showing an example of a specific configuration of the information processing device 10 according to this embodiment. The information processing device 10 is a type of computer, and includes a CPU (Central Processing Unit) 14B, a ROM (Read Only Memory) 14A, a RAM (Random Access Memory) 14C, and an input / output port 14D.

[0021] In the arithmetic unit 14, the CPU 14B, ROM 14A, RAM 14C, and input / output port 14D are connected to one another via various buses such as an address bus, a data bus, and a control bus. The input / output port 14D is connected to various input / output devices, such as the input device 12, the output device 16 such as a display or a printer, the storage device 18 such as a hard disk (HDD), and the communication unit 36. The input device 12 is also connected to an imaging device 60 that acquires image data of an intersection or the like where traffic volume is to be measured as a time-series series of continuously captured images (video). The storage device 18 has installed therein a traffic volume measurement program that measures traffic volume at an intersection or the like from image data.

[0022] Next, various functions realized by the CPU 14B of the calculation device 14 executing the traffic volume measurement program will be described. The traffic volume measurement program functions as an image acquisition function for acquiring image data from the imaging device 60 via the input device 12, an object extraction function for extracting objects representing vehicles from the acquired image data, an ID assignment function for assigning IDs to the extracted objects, a counting function for counting the objects to which IDs have been assigned, and an output function for outputting the counting results indicating traffic volume to the output device 16 and the storage device 18. By executing the traffic volume measurement program, the CPU 14B functions as an image acquisition unit 40, an object extraction unit 42, an ID assignment unit 44, a counting unit 46, and an output unit 48.

[0023] 2 is a flowchart showing an example of processing by the information processing device 10 according to this embodiment. In step S100, processing for extracting an object representing a vehicle from all frames of continuously captured images acquired by the imaging device 60 is started.

[0024] In step S102, one frame of the captured image is acquired, and in step S104, an object is extracted from the acquired frame of the captured image. In step S104, objects are detected for each class to which each object in the captured image belongs, and a boundary box, which is a partial region surrounding the object, is assigned to each detected object. In this embodiment, the object class refers to a vehicle type, such as a passenger car, a light vehicle, or a freight truck. The object class is classified, for example, by the size (number of pixels) of the object or the object's boundary box. As an example, if the number of pixels of the object or boundary box is less than a predetermined first threshold, the object is classified into the light vehicle class. If the number of pixels of the object or boundary box is equal to or greater than the first threshold but less than a predetermined second threshold that is greater than the first threshold, the object is classified into the passenger car class. If the number of pixels of the object or boundary box is equal to or greater than the second threshold, the object is classified into the freight truck class.

[0025] In step S106, the object extraction in steps S102 and S104 is executed for all frames of the continuously captured images, and then the procedure proceeds to step S108.

[0026] In step S108, ID assignment according to the procedure of steps S110 to S116 is executed for all extracted objects.

[0027] In step S110, one object (boundary box) is extracted. Then, in step S112, it is determined whether the object extracted in step S110 is an object whose image has been captured for the first time. If it is an object whose image has been captured for the first time in step S112, the procedure proceeds to step S116, and if it is not an object whose image has been captured for the first time, the procedure proceeds to step S114.

[0028] In step S114, the object extracted in step S110 is assigned the ID assigned in the previous frame, that is, the same ID as the ID assigned in the previous frame.

[0029] In step S116, a new ID is assigned to the object extracted in step S110. In assigning a new ID in step S116, as shown by the solid lines in FIGS. 3A and 3B, when counting only vehicles traveling from the back of an intersection and turning right at the intersection, if an object Car1, to which an ID has already been assigned, is occluded by another object Car3, a new ID, Car4, may be assigned to the object Car1. In addition, in the case shown in FIGS. 3A and 3B, vehicles traveling along the movement trajectory indicated by the dashed lines must be excluded from the counting target. In this embodiment, as described below, by counting objects that do not pass through the first counting line (first counting line 70) but pass through the second counting line (second counting line 72), a case where a new ID, Car4, is assigned to the object Car1 is handled, and vehicles traveling along the movement trajectory indicated by the dashed lines are excluded from the counting target.

[0030] In step S118, ID assignment according to the procedure of steps S110 to S116 is executed for all extracted objects, and then the procedure proceeds to step S120.

[0031] In step S120, the counting according to the procedure of steps S122 to S136 is executed for all extracted objects.

[0032] In step S122, one object (boundary box) is extracted. Then, in step S124, it is determined whether the object extracted in step S122 belongs to a class to be counted. For example, if only objects belonging to the passenger car class are to be counted, objects belonging to classes such as freight trucks are determined not to be counted. In step S124, if the object's class is to be counted, the procedure proceeds to step S126, and if the object's class is not to be counted, the procedure proceeds to step S138.

[0033] In step S126, it is determined whether the extracted object has passed through the first counting line 70. The first counting line 70, together with the second counting line 72, is virtually set to, for example, linear pixels that cross a lane in a captured image near an intersection. FIG. 3A shows a case where an object Car2 passes through the first counting line 70 (first counting line). In step S126, if the object to be counted has passed through the first counting line 70 like the object Car2 shown in FIG. 3A, the procedure proceeds to step S128, and if the object to be counted has not passed through the first counting line 70, the procedure proceeds to step S132.

[0034] In step S128, the ID of the object that passed through first counting line 70 is stored in storage device 18, and in step S130, a first passing object storage area is allocated in storage device 18, and the procedure proceeds to step S138. The first passing object storage area allocated in step S130 is referenced in the determination in step S134, which will be described later.

[0035] In step S132, it is determined whether the extracted object has passed through the second counting line 72. In Figures 3(A), 3(B), 4(A), and 4(B), object Car2 has passed through the second counting line 72. In Figure 3(B), object Car4 (= object Car1) has passed through the second counting line 72. In step S132, if an object that has passed through the second counting line 72 is detected as shown in Figure 3(A) etc., the procedure proceeds to step S134, and if an object that has passed through the second counting line 72 is not detected, the procedure proceeds to step S138.

[0036] In step S134, the first passing object storage area of ​​the storage device 18 is referenced to determine whether the object that passed the second counting line 72 has previously passed the first counting line 70. If it is determined in step S134 that the object that passed the second counting line 72 has previously passed the first counting line 70, the procedure proceeds to step S138. If the object that passed the second counting line 72 has not previously passed the first counting line 70, the procedure proceeds to step S136. Figures 3(A), 3(B), 4(A), and 4(B) show a case where an object Car2 passes the first counting line 70 and the second counting line 72. In this embodiment, the object Car4 that passed the first counting line 70 is not counted, and the procedure proceeds to step S138.

[0037] In step S136, the objects are counted for each class to be counted, the ID of the object is excluded from the counting, and the procedure proceeds to step S138. In FIG. 3B, object Car4 passes through second counting line 72 without passing through first counting line 70. In this embodiment, object Car4 shown in FIG. 3B is counted, and after counting, the ID of the counted object such as object Car4 is excluded from the counting.

[0038] In step S138, the counting according to the procedure of steps S122 to S136 is performed for all extracted objects, and then the process ends.

[0039] As described above, according to this embodiment, by counting objects that do not pass through the first counting line 70 but pass through the second counting line 72, an information processing device can be obtained that can accurately detect vehicles from image data.

[0040] In assigning IDs to objects detected from captured images, as shown in Figures 3(A) and 3(B), if an object Car1 is occluded by another object Car3, a new ID, Car4, may be assigned to the object Car1. In this embodiment, as shown in Figures 3(A) and 3(B), the second counting line 72 is set on the exit side of the intersection in the lane on the near side as viewed from the image capture device 60, where the counting target object is unlikely to be occluded by another object. Furthermore, in order to exclude vehicles traveling along the movement trajectory indicated by the dashed line in Figures 3(A) and 3(B) from the counting target, the first counting line 70 is set at a position where a vehicle traveling straight through the intersection and passing the second counting line 72 enters the intersection, i.e., a position upstream of the movement trajectory of the object Car2.

[0041] In addition, by setting up a first counting line 70 and a second counting line 72 as shown in Figures 3(A) and 3(B) and counting only objects that pass through the second counting line 72, it is possible to count an object that has been assigned a new ID, Car4, instead of Car1, and to exclude vehicles traveling along the movement trajectory shown by the dashed line in Figures 3(A) and 3(B) from the counting targets.

[0042] 5A, this embodiment can also handle a case where a movement trajectory 82 that is not to be counted is a vehicle turning right at an intersection, a movement trajectory 80 that is to be counted is a vehicle going straight through the intersection, and the movement trajectory 80 is displayed briefly in a corner of the captured image. In such a case, the first counting line 74 is set on the entrance side of the intersection, upstream of the movement trajectory 82 that is not to be counted, and the second counting line 76 is set on the exit side of the intersection, downstream of both the movement trajectory 80 that is to be counted and the movement trajectory 82 that is not to be counted. As a result, by counting only objects passing through the second counting line 76, it is possible to count only vehicles that indicate the movement trajectory 80 that is to be counted.

[0043] 5(B) is an enlarged view of portion (i) in FIG. 5(A). In FIG. 5(B), the movement trajectories 80 to be counted include a movement trajectory 80A when the vehicle is moving slowly and a movement trajectory 80B when the vehicle is moving quickly. Depending on the frame rate available with the imaging device 60, the movement trajectory 80B when the vehicle is moving quickly will only be present in the captured image for a short period of time, making it more difficult to detect than the movement trajectory 80A when the vehicle is moving slowly. However, in this embodiment, vehicles are counted based on whether or not an object passes through the second counting line 76, making it possible to measure traffic volume without being affected by the vehicle's movement speed.

[0044] [Variations] Next, a modified example of this embodiment will be described. This modified example differs from the above-described embodiment in that objects to be counted are selected based on the movement trajectories of the objects estimated based on traffic information acquired from outside, but the configuration for extracting objects by class is the same as this embodiment.

[0045] Fig. 6 is an explanatory diagram showing an example of counting in a modified example of this embodiment. In the example shown in Fig. 6, traffic light 112 on the north side turns green, and vehicle 92 on the south side of the intersection travels along movement trajectory 96. Vehicle 90 on the west side of the intersection stops just before the intersection, following traffic light 114, which has turned red in conjunction with traffic light 110.

[0046] In this modified example, only vehicles traveling along movement trajectory 96 are counted, so counting line 98 is provided on the east side of the intersection in the image captured by imaging device 60. Since counting line 98 also intersects with movement trajectory 94 of vehicle 90, which should not be counted, in this modified example, vehicles are counted by movement trajectory in conjunction with traffic lights 110, 112, 114, and 116. Specifically, in order to count only vehicles traveling along movement trajectory 96, such as vehicle 92, objects related to vehicles that pass counting line 98 when traffic light 112 turns green are counted.

[0047] In this modified example, objects that pass through the counting line 98 are counted when the traffic light 112 that controls the movement of the vehicle being counted turns green, thereby making it possible to count objects accurately using fewer counting lines while reducing the computational load of object counting.

[0048] In another variant, the objects to be counted are selected based on the movement trajectory of each vehicle estimated based on information on the position, speed, acceleration, braking operation, on-board camera footage, and steering angle of the own vehicle or other vehicles obtained by the DCM (vehicle communication device) of the own vehicle or other vehicles.

[0049] Furthermore, objects to be counted may be selected based on the movement trajectory of each vehicle estimated by referring to the position information of the subject vehicle, other vehicles, or pedestrians' smartphones. [Explanation of symbols]

[0050] 10. Information processing equipment 12 Input Devices 14 Arithmetic unit 14B CPU 42 Object Extraction Unit 46 Counting Unit 60 Imaging device 70, 74 First counting line 72, 76 Second counting line 80, 80A, 80B, 82 movement trajectory 90, 92 vehicles 94, 96 Movement trajectory 98 Counting Line 110, 112, 114, 116 traffic lights Car1, Car2, Car3, Car4 objects

Claims

1. an object extraction unit that extracts an object representing a vehicle from image data of the intersection acquired by the imaging device; a counting unit that sets a first counting line on the entrance side of the intersection corresponding to the upstream of a movement trajectory of an object that is not to be counted, and a second counting line on the exit side of the intersection, and counts the object that passes through the second counting line without passing through the first counting line; An information processing device comprising:

2. The object extraction unit detects the objects according to classes set according to the vehicle type, The information processing device according to claim 1 , wherein the counting unit counts the objects by class.

Images