Information processing apparatus, information processing method, and program

Abstract

To provide a system capable of measuring a movement situation of a mobile body with high accuracy.SOLUTION: An information processing device comprises: a detection unit that detects a moving body from a time-series image formed of a plurality of chronologically continuous images obtained by imaging a measurement area; a movement line extraction unit that extracts movement line information indicating a movement line in the time-series image of the moving body detected by the detection unit; a confidence calculation unit that calculates a confidence of the movement line indicated by the movement line information extracted by the movement line extraction unit; and a measurement unit that measures the number of measurement lines set in the measurement area that has been passed by the mobile body on the basis of a movement line for which the confidence calculated by the confidence calculation unit is higher than a predetermined threshold value.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, technologies for measuring the movement status of moving objects such as vehicles and people based on camera images have been developed. For example, Patent Document 1 below discloses a technology for detecting and tracking vehicles from moving images obtained by a camera for monitoring road conditions installed on a highway, and measuring the traffic volume and speed for each lane.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a further need for improvement in the measurement accuracy of the movement status of moving objects.

[0005] Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a mechanism capable of more accurately measuring the movement status of a moving object.

Means for Solving the Problems

[0006] In order to solve the above problems, according to an aspect of the present invention, a detection unit that detects a moving object from a time-series image composed of a plurality of images consecutive in time-series that image a measurement area, a trajectory extraction unit that extracts trajectory information indicating a trajectory of the moving object in the time-series image detected by the detection unit, and the reliability of the trajectory indicated by the trajectory information extracted by the trajectory extraction unit , based on the total length of the above-mentioned flow line A reliability calculation unit that calculates, and a measurement unit that measures the number of times the moving object has passed through the measurement line set in the measurement area based on the traffic line for which the reliability calculated by the reliability calculation unit is higher than a predetermined threshold value, are provided in an information processing apparatus.

[0008] The reliability calculation unit may calculate the reliability based on the length of the traffic line at each time.

[0009] The reliability calculation unit may calculate the reliability based on the frame rate of the time-series image.

[0010] The traffic line information includes detected coordinates that are the coordinates of the detected moving object, or predicted coordinates that are the predicted coordinates of the moving object, in each of a plurality of images included in the time-series image. The traffic line extraction unit may update the traffic line information by adding the detected coordinates of the detected moving object or the predicted coordinates of the moving object for which detection has failed in an image newly added to the time-series image over time to the traffic line information.

[0011] The traffic line extraction unit may update the traffic line information by adding the coordinates of the detected moving object corresponding to the traffic line information to the traffic line information.

[0012] The reliability calculation unit may calculate the reliability based on the configuration of the detected coordinates and the predicted coordinates in the traffic line information.

[0013] The measurement unit may determine whether or not the moving object has passed through the measurement line based on a specific part of the traffic line indicated by the traffic line information.

[0014] The measurement unit may determine whether or not the moving object has passed through the measurement line based on a part consisting of a predetermined number of coordinates among the traffic lines indicated by the traffic line information.

[0015] The measurement unit may determine whether the moving object has passed through the measurement line based on a part of the traffic flow indicated by the traffic flow information and within a predetermined distance from the measurement line.

[0016] The reliability calculation unit may calculate the reliability based on a calibration file that converts the coordinates in the time-series image into coordinates in the world coordinate system of the real space. Further, in order to solve the above problems, according to another aspect of the present invention, a detection unit that detects a moving body from a time-series image composed of a plurality of images continuous in time-series that image a measurement area, a flow line extraction unit that extracts flow line information indicating the flow line of the moving body in the time-series image detected by the detection unit, and a reliability calculation unit that calculates the reliability of the flow line indicated by the flow line information extracted by the flow line extraction unit based on a calibration file that converts the coordinates in the time-series image into coordinates in the world coordinate system of the real space, and a measurement unit that measures the number of times the moving body has passed through a measurement line set in the measurement area based on the flow line for which the reliability calculated by the reliability calculation unit is higher than a predetermined threshold value are provided.

[0017] The reliability calculation unit may calculate the reliability based on the speed of the moving object in the real space calculated based on the traffic flow information and the calibration file.

[0018] The measurement unit may measure the speed of the moving object in the real space when the moving object passes through the measurement line.

[0019] The measurement unit may measure the number of times the moving object passes through the measurement line for each direction in which the moving object passes through the measurement line.

[0020] The information processing device may further include an output unit that outputs information indicating the measurement result by the measurement unit.

[0021] The moving object may be a vehicle.

[0022] The measurement area may include a road, an intersection, or a parking lot.

[0023] Also, in order to solve the above problems, according to another aspect of the present invention, detecting a moving object from a time-series image composed of a plurality of images consecutive in time-series that image a measurement area, extracting traffic flow information indicating the traffic flow of the detected moving object in the time-series image, and the reliability of the traffic flow indicated by the extracted traffic flow information , based on the total length of the above-mentioned flow line Calculating, and measuring the number of times the moving object has passed through a measurement line set in the measurement area based on the traffic line for which the calculated reliability is higher than a predetermined threshold , information processing device An information processing method executed thereby is provided. Further, in order to solve the above problems, according to another aspect of the present invention, detecting a moving body from a time-series image composed of a plurality of images continuous in time-series that image a measurement area, extracting flow line information indicating the flow line of the detected moving body in the time-series image, calculating the reliability of the flow line indicated by the extracted flow line information based on a calibration file that converts the coordinates in the time-series image into coordinates in the world coordinate system of the real space, and measuring the number of times the moving body has passed through a measurement line set in the measurement area based on the flow line for which the calculated reliability is higher than a predetermined threshold value, and an information processing method executed by an information processing device including the above is provided.

[0024] Further, in order to solve the above problems, according to another aspect of the present invention, a computer is configured to include a detection unit that detects a moving object from a time-series image composed of a plurality of images consecutive in time-series that image a measurement area, a traffic line extraction unit that extracts traffic line information indicating a traffic line of the moving object in the time-series image detected by the detection unit, and a reliability of the traffic line indicated by the traffic line information extracted by the traffic line extraction unit , based on the total length of the above-mentioned flow line A reliability calculation unit that calculates, and a measurement unit that measures the number of times the moving object has passed through a measurement line set in the measurement area based on the traffic line for which the reliability calculated by the reliability calculation unit is higher than a predetermined threshold, and a program for causing the computer to function as such is provided. Further, in order to solve the above problems, according to another aspect of the present invention, a computer is provided with a detection unit that detects a moving object from a time-series image composed of a plurality of images consecutive in time-series obtained by imaging a measurement area, a trajectory extraction unit that extracts trajectory information indicating a trajectory of the moving object in the time-series image detected by the detection unit, a reliability calculation unit that calculates the reliability of the trajectory indicated by the trajectory information extracted by the trajectory extraction unit based on a calibration file that converts coordinates in the time-series image into coordinates in a world coordinate system of a real space, and a measurement unit that measures the number of times the moving object has passed through a measurement line set in the measurement area based on the trajectory for which the reliability calculated by the reliability calculation unit is higher than a predetermined threshold value. A program for causing the computer to function as described above is provided.

Advantages of the Invention

[0025] As described above, according to the present invention, a mechanism capable of more accurately measuring the moving state of a moving object is provided.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0027] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0028] <1. Technical Problem> Hereinafter, as an example of a technique for measuring the movement state of a moving object, the technical problem will be described while citing a technique for measuring the traffic volume of vehicles.

[0029] A measurement system that measures the traffic volume of vehicles by image processing typically performs image processing on a camera image in time series to detect the positions of vehicles, and extracts the traffic flow of the vehicles by connecting the detected coordinates. Then, the measurement system measures the number of times the traffic flow of the vehicle crosses a measurement line set in the camera image as the number of passing vehicles on the measurement line.

[0030] When a vehicle continues to stay on the measurement line and when a specific background is continuously misdetected as a vehicle, the coordinates of the detected vehicle may blur, causing the traffic flow to cross the measurement line multiple times and resulting in an overestimated number of passing vehicles being measured. Such a measurement error of counting the number of passing vehicles that should not be counted is also referred to as an incorrect count. Also, even if the vehicle is successfully detected before and after the measurement line, but a concealment or the like occurs on the measurement line and the vehicle detection fails, the count of the number of passing vehicles that should be counted may fail. Such a measurement error of not counting the number of passing vehicles that should be counted is also referred to as an uncounted count.

[0031] Although various techniques for measuring the traffic volume of vehicles, including the above-mentioned Patent Document 1, have been proposed, they have not yet been able to sufficiently suppress these incorrect counts and uncounts.

[0032] Therefore, in the present embodiment, a mechanism is provided that suppresses the occurrence of incorrect counts and uncounts by calculating the reliability for the traffic flow of a moving object and measuring the movement state of the moving object based on the reliability.

[0033] <2. First Embodiment> <2.1. Configuration Example> FIG. 1 is a block diagram showing an example of the configuration of a system 1 according to the present embodiment. As shown in FIG. 1, the system 1 according to the present embodiment includes an imaging device 10, an information processing device 20, and an output device 30.

[0034] (1) Imaging Device 10 The imaging device 10 is a device that captures a time-series image (i.e., a moving image) composed of a plurality of images continuous in time series. The imaging device 10 captures a time-series image at a predetermined frame rate. Then, the imaging device 10 outputs the captured time-series image to the information processing device 20 in real time. The time-series image may be a color image or an infrared image.

[0035] The imaging device 10 is arranged at a position and an angle that include the measurement area within the imaging range, and outputs a time-series image of the measurement area. The measurement area is a space that is the object of measurement of the movement state of the moving object. In this specification, the moving object is a vehicle, and the movement state of the moving object is the traffic situation of the vehicle. The traffic situation here is a concept that includes the traffic volume of vehicles, the moving direction of vehicles, and the speed of vehicles. The measurement area includes an area where vehicles can pass, such as a road, an intersection, or a parking lot. For example, the imaging device 10 may be a surveillance camera installed in a road, an intersection, or a parking lot.

[0036] FIG. 2 is a diagram showing an example of the time-series image 100 output from the imaging device 10. As shown in FIG. 2, the time-series image 100 shows a vehicle 200 passing through an intersection. The detection frame 210, the traffic flow line 300, and the measurement lines 400 (400A to 400D) are information given to the time-series image 100 by the information processing device 20. These will be described in detail later.

[0037] (2) Information processing device 20 The information processing device 20 is a device that measures the traffic situation of vehicles based on the time-series images captured by the imaging device 10. As shown in FIG. 1, the information processing device 20 includes an image acquisition unit 21, a detection unit 22, a traffic flow line extraction unit 23, a traffic flow line storage unit 24, a reliability calculation unit 25, a measurement line setting unit 26, a measurement line storage unit 27, a measurement unit 28, and an output unit 29.

[0038] - Image acquisition unit 21 The image acquisition unit 21 has a function of acquiring the time-series images output from the imaging device 10. The image acquisition unit 21 outputs the acquired time-series images to the detection unit 22.

[0039] - Detection unit 22 The detection unit 22 has a function of detecting vehicles from the time-series images. The detection unit 22 detects the vehicles shown in the time-series images by applying image recognition processing to each of the plurality of images included in the time-series images. For example, the detection unit 22 may detect vehicles from the time-series images by comparing the image feature amounts of vehicles learned in advance by machine learning with the image feature amounts obtained by applying image recognition processing to the time-series images.

[0040] The detection unit 22 detects the coordinates of the vehicles in the time-series images as vehicles are detected. At this time, as shown in FIG. 2, the detection unit 22 may set a detection frame 210 indicating the range occupied by the detected vehicle 200 in the time-series image 100. When the detection frame 210 is rectangular, the detection unit 22 detects the coordinates of the four corners of the detection frame 210 as the coordinates of the vehicle 200 in the time-series image 100.

[0041] Furthermore, the detection unit 22 may detect vehicle information indicating the characteristics of the vehicle. The vehicle information may include the color of the vehicle body, the number of the license plate, the vehicle type, and the like.

[0042] - Traffic flow line extraction unit 23 The traffic flow line extraction unit 23 has a function of extracting traffic flow line information indicating the traffic flow line (i.e., the movement trajectory) in the time-series image of the vehicle detected by the detection unit 22. The traffic flow line information is information including the coordinates of the vehicle in each of a plurality of images included in the time-series image, that is, the coordinates of the vehicle at each time (i.e., each frame). As shown in FIG. 2, by connecting the coordinates that are continuous on the time axis included in the traffic flow line information, a traffic flow line 300 is obtained. The traffic flow line 300 shown in FIG. 2 is a line connecting the coordinates from time t to t-4 and corresponds to the movement trajectory of the vehicle 200. Furthermore, the traffic flow line information includes vehicle information indicating the characteristics of the vehicle. That is, the traffic flow line information is information indicating the traffic flow line of the vehicle uniquely specified by the vehicle information. Therefore, the traffic flow line information is extracted for each vehicle.

[0043] The traffic flow line extraction unit 23 outputs the extracted traffic flow line information to the traffic flow line storage unit 24 and stores it in the traffic flow line storage unit 24. Each time a new image is added to the time-series image over time, the traffic flow line extraction unit 23 updates the traffic flow line information by adding the coordinates of the vehicle in the newly added current-time image to the traffic flow line information of the previous time (i.e., the previous frame) stored in the traffic flow line storage unit 24.

[0044] At that time, the traffic flow extraction unit 23 updates the traffic flow information by adding the coordinates of the detected vehicle that corresponds to the traffic flow information to the traffic flow information. As an example, the traffic flow extraction unit 23 compares the vehicle information included in the traffic flow information stored in the traffic flow storage unit 24 with the vehicle information of the vehicle detected by the detection unit 22 at the current time. Then, when these match, the traffic flow extraction unit 23 determines that the detected vehicle is the vehicle corresponding to the traffic flow information, and adds the coordinates of the detected vehicle to the traffic flow information. As another example, the traffic flow extraction unit 23 compares the coordinates at the current time predicted from the coordinates and the moving direction one time step before shown by the traffic flow information stored in the traffic flow storage unit 24 with the coordinates of the vehicle detected by the detection unit 22 at the current time. Then, when the distance between these coordinates is less than the threshold value, the traffic flow extraction unit 23 determines that the detected vehicle is the vehicle corresponding to the traffic flow information, and adds the coordinates of the detected vehicle to the traffic flow information. According to such a configuration, it becomes possible to continuously track the traffic flow of the vehicle uniquely identified by the vehicle information.

[0045] On the other hand, for a detected vehicle that does not correspond to the traffic flow information, the traffic flow extraction unit 23 determines that a new vehicle has appeared. As an example, the traffic flow extraction unit 23 compares the vehicle information included in the traffic flow information stored in the traffic flow storage unit 24 with the vehicle information of the vehicle detected by the detection unit 22 at the current time. Then, when these do not match, the traffic flow extraction unit 23 determines that the detected vehicle is a newly appeared vehicle. As another example, the traffic flow extraction unit 23 compares the coordinates at the current time predicted from the coordinates and the moving direction one time step before shown by the traffic flow information stored in the traffic flow storage unit 24 with the coordinates of the vehicle detected by the detection unit 22 at the current time. Then, when the distance between these coordinates is greater than or equal to the threshold value, the traffic flow extraction unit 23 determines that the detected vehicle is a newly appeared vehicle. When it is determined that the detected vehicle is a newly appeared vehicle, the traffic flow extraction unit 23 generates and outputs new traffic flow information including the coordinates of the vehicle determined to have newly appeared. According to such a configuration, it becomes possible to start tracking the traffic flow of a vehicle that newly appears in the time-series image.

[0046] Here, when detecting a vehicle corresponding to the traffic flow information, it may fail due to reasons such as partial occlusion of the vehicle. In that case, the traffic flow extraction unit 23 may continue to track the vehicle while adding the predicted coordinates of the vehicle to the traffic flow information. Hereinafter, the coordinates of the vehicle for which detection has succeeded are also referred to as detection coordinates. On the other hand, the coordinates predicted as the coordinates of the vehicle for which detection has failed are also referred to as predicted coordinates. When there is no need to particularly distinguish between the detection coordinates and the predicted coordinates, these are simply collectively referred to as coordinates.

[0047] The traffic flow information includes the detection coordinates of the detected vehicle or the predicted coordinates of the predicted vehicle in each of a plurality of images included in the time-series images. Then, the traffic flow extraction unit 23 updates the traffic flow information by adding the detection coordinates of the detected vehicle or the predicted coordinates of the vehicle for which detection has failed in the image newly added to the time-series images over time to the traffic flow information. The point of adding the detection coordinates to the traffic flow information is as described above. The traffic flow extraction unit 23 predicts the predicted coordinates of a vehicle corresponding to the traffic flow information and for which detection has failed. Various methods such as a Kalman filter can be applied to the prediction of the predicted coordinates. At that time, the traffic flow extraction unit 23 may set a prediction frame as an alternative to the detection frame. The prediction frame is a range predicted to be occupied by the vehicle in the time-series images. When the prediction frame is rectangular, the traffic flow extraction unit 23 detects the coordinates of the four corners of the prediction frame as the predicted coordinates of the vehicle in the time-series images. According to such a configuration, even if the detection of the vehicle temporarily fails, the tracking can be continued, so it is possible to suppress the occurrence of uncounted cases.

[0048] The traffic flow extraction unit 23 may determine whether the predicted coordinates are correct and update the traffic flow information using only the predicted coordinates determined to be correct. As an example, the traffic flow extraction unit 23 may determine whether the predicted coordinates are correct based on the similarity between the features extracted from the partial image cut out by the prediction frame in the time-series images and the features indicated by the vehicle information included in the traffic flow information. This method is effective when part of the vehicle is occluded and the other part is not occluded. According to such a configuration, even when the detection of the vehicle fails, it is possible to further improve the measurement accuracy of the traffic situation.

[0049] - Traffic flow memory unit 24 The traffic flow memory unit 24 has a function of storing the traffic flow information extracted by the traffic flow extraction unit 23. The traffic flow memory unit 24 outputs the stored traffic flow information to the traffic flow extraction unit 23, or stores (for example, overwrites) the traffic flow information output from the traffic flow extraction unit 23. Further, the traffic flow memory unit 24 outputs the stored traffic flow information to the reliability calculation unit 25.

[0050] - Reliability calculation unit 25 The reliability calculation unit 25 has a function of calculating the reliability of the traffic flow indicated by the traffic flow information extracted by the traffic flow extraction unit 23. The reliability here is an index indicating the probability that the traffic flow extracted by the traffic flow extraction unit 23 is the traffic flow of the vehicle. The reliability calculation unit 25 calculates the reliability of the traffic flow information stored in the traffic flow memory unit 24.

[0051] The reliability calculation unit 25 may calculate the reliability based on the total length of the traffic flow. The length here may mean distance. In that case, for example, the reliability calculation unit 25 calculates a higher reliability as the total distance of the traffic flow is longer, and calculates a lower reliability as the total distance of the traffic flow is shorter. This is because when the vehicle is clearly shown in the time-series image, detection and tracking are stably performed for each frame, so the total distance of the traffic flow becomes longer, and when a specific background is misdetected, the distance of the traffic flow becomes shorter. Also, the length here may mean the time length. In that case, for example, the reliability calculation unit 25 calculates a low reliability when the total time length of the traffic flow is excessively long. This is because when a signboard on which a vehicle is drawn is misdetected as a vehicle, the total time length of the traffic flow of the misdetected signboard as a vehicle becomes excessively long. As described above, according to such a configuration, it is possible to calculate a low reliability for a traffic flow considered to include misdetected coordinates.

[0052] The reliability calculation unit 25 may calculate the reliability based on the length of the traffic flow line at each time. The length here means the distance. For example, the reliability calculation unit 25 calculates a higher reliability as the variation (e.g., variance) of the length of the traffic flow line at each time is lower, and calculates a lower reliability as the variation is higher. For example, if the coordinates of a vehicle not corresponding to the traffic flow line information are erroneously added to the traffic flow line information, sudden movement of the coordinates occurs, resulting in variation in the length of the traffic flow line at each time. In this regard, according to such a configuration, it is possible to calculate a lower reliability for a traffic flow line in which sudden movement of the coordinates occurs.

[0053] The reliability calculation unit 25 may calculate the reliability based on the frame rate of the time-series image. For example, the reliability calculation unit 25 calculates a higher reliability if the length of the traffic flow line at each time is appropriate in view of the frame rate of the time-series image, and calculates a lower reliability if it is not appropriate. For example, if the coordinates of a vehicle not corresponding to the traffic flow line information are erroneously added to the traffic flow line information, sudden movement of the coordinates that is impossible in one frame may occur. In this regard, according to such a configuration, it is possible to calculate a lower reliability for a traffic flow line that has moved a distance exceeding the distance that can be moved per frame.

[0054] The reliability calculation unit 25 may calculate the reliability based on the configuration of the detected coordinates and predicted coordinates in the traffic flow line information. As an example, the reliability calculation unit 25 calculates a higher reliability as the ratio of the detected coordinates in the traffic flow line information is higher and the ratio of the predicted coordinates is lower, and calculates a lower reliability as the ratio of the detected coordinates is lower and the ratio of the predicted coordinates is higher. As another example, the reliability calculation unit 25 calculates a higher reliability for a traffic flow line that includes detected coordinates after the predicted coordinates, and calculates a lower reliability for a traffic flow line that does not include detected coordinates after the predicted coordinates. According to such a configuration, it is possible to calculate a lower reliability for a vehicle with an excessively long period of being blocked, etc., and a vehicle that has continued to fail in detection after a certain timing.

[0055] - Measurement line setting unit 26 The measurement line setting unit 26 has a function of setting a measurement line in the measurement area. The measurement line is a line for which the number of passing vehicles is counted when the vehicle passes through the line. Setting a measurement line in the measurement area is synonymous with setting the coordinates of the measurement line in the time-series image. In the example shown in FIG. 2, measurement lines 400A to 400D are set at four locations that are the entrances and exits of the intersection, respectively.

[0056] - Measurement line storage unit 27 The measurement line storage unit 27 has a function of storing information indicating the measurement line set by the measurement line setting unit 26. Specifically, the measurement line storage unit 27 stores the coordinates of the measurement line in the time-series image. The measurement line storage unit 27 outputs information indicating the stored measurement line to the measurement unit 28.

[0057] - Measurement unit 28 The measurement unit 28 has a function of measuring the traffic situation of vehicles passing through the measurement line set in the measurement area based on the traffic flow information of the vehicles. For example, the measurement unit 28 measures the number of vehicles that have passed through the measurement line based on the traffic flow information of the vehicles. Specifically, when the traffic flow and the measurement line intersect, the measurement unit 28 counts (i.e., adds 1) the number of vehicles that have passed through the measurement line. This point will be described in detail with reference to FIG. 3.

[0058] FIG. 3 is a diagram for explaining the measurement of the number of vehicles that have passed through the measurement line according to the present embodiment. As shown in FIG. 3, the traffic flow line 300 at time t is configured as a line connecting the coordinates at time t, the coordinates at time t−1, the coordinates at time t−2, and the coordinates at time t−3. Hereinafter, the measurement at time t will be described. The measurement unit 28 determines whether a straight line 300A connecting the coordinates at time t and the coordinates at time t−1 intersects the measurement line 400, and determines that the vehicle has passed through the measurement line 400 if they intersect. On the other hand, when it is determined that they do not intersect, the measurement unit 28 determines whether a straight line 300B connecting the coordinates at time t−1 and the coordinates at time t−2 intersects the measurement line 400, and determines that the vehicle has passed through the measurement line 400 if they intersect. The measurement unit 28 makes such a determination by tracing back the traffic flow line 300 until a straight line connecting coordinates that are continuous on the time axis intersects the measurement line 400. If the measurement unit 28 reaches the end of the traffic flow line 300 and the straight line connecting coordinates that are continuous on the time axis does not intersect the measurement line 400, the measurement unit 28 determines that the vehicle 200 has not passed through the measurement line 400. In the example shown in FIG. 3, the measurement unit 28 determines that a straight line 300C connecting the coordinates at time t−2 and the coordinates at time t−3 intersects the measurement line 400, and counts the number of vehicles 200 that have passed through the measurement line 400.

[0059] The measurement unit 28 may measure the number of vehicles that have passed through the measurement line for each direction in which the vehicle passes through the measurement line. Specifically, the measurement unit 28 determines the direction in which the traffic flow line intersects the measurement line as the direction in which the vehicle passes through the measurement line. Then, the number of vehicles that have passed through the measurement line is measured for each direction in which the vehicle passes through the measurement line. According to such a configuration, it becomes possible to measure the traffic situation in more detail.

[0060] Here, the measurement unit 28 measures the number of vehicles that have passed through the measurement line based on a traffic flow line in which the reliability calculated by the reliability calculation unit 25 is higher than a predetermined threshold value. By measuring the traffic situation by limiting it to traffic flow lines with high reliability, it is possible to suppress the occurrence of incorrect counting. In addition, the measurement unit 28 counts the number of vehicles that have passed through the measurement line only once for each traffic flow line. According to such a configuration, for example, even when a vehicle stays on the measurement line and the traffic flow line intersects the measurement line multiple times, it is possible to suppress the occurrence of incorrect counting.

[0061] The measurement unit 28 may determine whether or not a vehicle has passed through the measurement line based on a specific part of the traffic flow line indicated by the traffic flow line information. According to such a configuration, it is possible to reduce the processing load as compared with the case of determining whether or not a straight line connecting coordinates continuous in the time axis intersects the measurement line over the entire traffic flow line.

[0062] Specifically, the measurement unit 28 may determine whether or not a vehicle has passed through the measurement line based on a part composed of a predetermined number of coordinates among the traffic flow lines indicated by the traffic flow line information. For example, when the measurement unit 28 determines whether or not a straight line connecting coordinates continuous in the time axis intersects the measurement line while tracing the traffic flow line, the measurement unit 28 may specify the number of coordinates to be traced and cut off the determination halfway without tracing to the end of the traffic flow line. According to such a configuration, it is possible to reduce the processing load as compared with the case of tracing to the end of the traffic flow line.

[0063] In addition, the measurement unit 28 may determine whether or not a vehicle has passed through the measurement line based on a part within a predetermined distance from the measurement line among the traffic flow lines indicated by the traffic flow line information. For example, regarding the example shown in FIG. 3, the measurement unit 28 may limit to a straight line 300C connecting the coordinates at time t-2 and time t-3 close to the measurement line 400 and determine whether or not it intersects the measurement line 400. According to such a configuration, it is possible to suppress the occurrence of uncounting while reducing the processing load.

[0064] - Output unit 29 The output unit 29 has a function of outputting information indicating the measurement result by the measurement unit 28. As an example, the output unit 29 may output information indicating the number of vehicles that have passed through the measurement line. As another example, the output unit 29 may output information indicating the number of vehicles that have passed through the measurement line for each direction in which the vehicle passes through the measurement line. The output unit 29 outputs information indicating the measurement result by the measurement unit 28 to the output device 30.

[0065] (3) Output device 30 The output device 30 is a device that outputs information. The output device 30 outputs the information output from the information processing device 20.

[0066] As an example, the output device 30 may be a display device that displays an image, and may display the information output from the information processing device 20. As another example, the output device 30 may be a sound output device that outputs sound, and may output the information output from the information processing device 20 as voice.

[0067] <2.2. Flow of processing> FIG. 4 is a flowchart showing an example of the flow of processing executed in the information processing device 20 according to the present embodiment.

[0068] As shown in FIG. 4, first, the image acquisition unit 21 acquires a time-series image captured by the imaging device 10 (step S102). Specifically, the image acquisition unit 21 sequentially acquires images continuously captured and output by the imaging device 10 at a predetermined frame rate.

[0069] Next, the detection unit 22 detects a vehicle from the time-series image acquired by the image acquisition unit 21 (step S104).

[0070] Next, the traffic flow extraction unit 23 determines whether a vehicle corresponding to the traffic flow information at the previous moment stored in the traffic flow storage unit 24 has been detected (step S106). For example, the traffic flow extraction unit 23 determines whether a vehicle corresponding to the traffic flow information at the previous moment has been detected based on the vehicle information included in the traffic flow information at the previous moment or based on the coordinates at the current moment predicted from the traffic flow information at the previous moment.

[0071] If it is determined that a vehicle corresponding to the traffic flow information at the previous moment has been detected (step S106: YES), the traffic flow extraction unit 23 adds the detection coordinates of the detected vehicle to the traffic flow information (step S108). In this way, the traffic flow extraction unit 23 updates the traffic flow information stored in the traffic flow storage unit 24. Thereafter, the process proceeds to step S112.

[0072] On the other hand, if it is determined that a vehicle corresponding to the traffic flow information at the previous moment has not been detected (step S106: NO), the traffic flow extraction unit 23 adds the predicted coordinates predicted for the vehicle for which detection has failed to the traffic flow information (step S110). In this way, the traffic flow extraction unit 23 updates the traffic flow information stored in the traffic flow storage unit 24. Thereafter, the process proceeds to step S112.

[0073] Next, the reliability calculation unit 25 calculates the reliability of the traffic flow indicated by the latest traffic flow information stored in the traffic flow storage unit 24 (step S112). For example, the reliability calculation unit 25 calculates the reliability based on at least any one of the distance or time length of the entire traffic flow, the distance length at each moment of the traffic flow, the frame rate of the time-series image, and the configuration of the detection coordinates and predicted coordinates in the traffic flow information.

[0074] Next, the measurement unit 28 determines whether a traffic flow with a calculated reliability higher than a predetermined threshold has crossed the measurement line (step S114).

[0075] If it is determined that the traffic flow has crossed the measurement line (step S114: YES), the measurement unit 28 counts the number of vehicles that have passed through the measurement line (step S116).

[0076] Then, the output unit 29 outputs the measurement result by the measurement unit 28 to the output device 30 (step S118). The output device 30 displays the measurement result by the measurement unit 28 and the like. Thereafter, the process returns to step S102.

[0077] Even when it is determined that the traffic flow line does not intersect the measurement line (step S114: NO), the process also returns to step S102.

[0078] <2.3. Effect> As described above, according to the present embodiment, it is possible to accurately measure the traffic situation of the vehicle while suppressing the occurrence of incorrect counting and uncounting. Specifically, by limiting the counting target to traffic flow lines with high reliability, the occurrence of incorrect counting can be suppressed. Also, when the detection of the vehicle fails, by adding the predicted coordinates to the traffic flow information instead of the detected coordinates and continuing to track the vehicle, the occurrence of uncounting can be suppressed. The effect of suppressing incorrect counting will be described in more detail with reference to FIG. 5.

[0079] FIG. 5 is a diagram for explaining the effect of suppressing incorrect counting in the present embodiment. In FIG. 5, a time-series image 100 showing a state where the vehicle 200 stays on the measurement line 400C during a right turn is shown. In the example shown in FIG. 5, the traffic flow line 300 intersects the measurement line 400C twice from time t-2 to t-1 and from time t-1 to time t. In this way, when the vehicle 200 continues to stay on the measurement line 400C, the traffic flow line 300 intersects the measurement line 400C many times due to the deviation of the detected coordinates. Here, when the vehicle 200 is stationary, the moving distance of the coordinates of the vehicle 200 at each time shown in the time-series image 100 becomes small, so the reliability of the traffic flow line 300 becomes low and it does not become a counting target. And when the vehicle 200 starts to move, the moving distance of the coordinates of the vehicle 200 becomes long, so the reliability of the traffic flow line 300 becomes high and it becomes a counting target. Only thus, the vehicle 200 is counted as having passed through the measurement line 400C. In this way, it is possible to suppress the occurrence of incorrect counting.

[0080] <3. Second Embodiment> In this embodiment, coordinates included in the flow line information are converted into coordinates in the world coordinate system of the real space, and then the traffic situation is measured.

[0081] <3.1. Configuration Example> FIG. 6 is a block diagram showing an example of the configuration of the system 1 according to this embodiment. As shown in FIG. 6, the system 1 according to this embodiment has the same configuration as the system 1 according to the first embodiment. However, the information processing apparatus 20 according to this embodiment includes a calibration file storage unit 40 in addition to the components included in the information processing apparatus 20 according to the first embodiment. Hereinafter, the description of the same configuration as that of the first embodiment will be omitted, and the differences will be mainly described.

[0082] The calibration file storage unit 40 has a function of storing a calibration file. The calibration file is a file in which parameters for converting coordinates in a time-series image into coordinates in the world coordinate system of the real space are described. The world coordinate system is a three-dimensional coordinate system with an arbitrary point in the real space as the origin. The calibration file is generated in advance based on the position and angle at which the imaging device 10 is installed, and the magnification of the lens of the imaging device 10, etc. The calibration file storage unit 40 outputs the stored calibration file to the reliability calculation unit 25.

[0083] The reliability calculation unit 25 calculates the reliability based on the calibration file. Specifically, the reliability calculation unit 25 calculates the reliability based on the speed of the vehicle in the real space calculated based on the traffic flow information and the calibration file. For example, the reliability calculation unit 25 converts the coordinates in the time-series images included in the traffic flow information into coordinates in the world coordinate system based on the calibration file. Then, the reliability calculation unit 25 calculates the speed of the vehicle in the real space indicated by the traffic flow information based on the distance between the coordinates in the world coordinate system that are continuous in time series and the frame rate. Then, the reliability calculation unit 25 calculates a high reliability when the calculated speed of the vehicle is reasonable as the actual speed of the vehicle, and calculates a low reliability otherwise. For example, when the speed of the vehicle suddenly changes from 10 km / h to 50 km / h, the reliability calculation unit 25 determines that a false detection has occurred and calculates a low reliability.

[0084] The measurement unit 28 measures the speed of the vehicle in the real space when the vehicle passes through the measurement line. Specifically, the measurement unit 28 acquires the speed of the vehicle in the real space at the timing when the vehicle passes through the measurement line, which is calculated by the reliability calculation unit 25. According to such a configuration, it becomes possible to measure the traffic situation in more detail.

[0085] The flow of the process executed by the information processing apparatus 20 according to the present embodiment is the same as the flow of the process executed by the information processing apparatus 20 according to the first embodiment described with reference to FIG. 4, and thus the description here is omitted.

[0086] <3.2. Effects> As described above, according to the present embodiment, the speed of the vehicle in the real space can be used for calculating the reliability. Therefore, it becomes possible to improve the measurement accuracy of the traffic situation compared to the first embodiment.

[0087] <4. Hardware Configuration> Next, with reference to FIG. 7, the hardware configuration of the information processing apparatus will be described. FIG. 7 is a block diagram showing an example of the hardware configuration of the information processing apparatus 900. Note that the information processing apparatus 900 shown in FIG. 7 can implement, for example, the information processing apparatus 20 shown in FIGS. 1 and 6, respectively. The information processing by the information processing apparatus 20 according to each of the above embodiments is realized by the cooperation of software and the hardware described below.

[0088] As shown in FIG. 7, the information processing apparatus 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, and a communication device 911.

[0089] The CPU 901 functions as an arithmetic processing unit and a control unit, and controls the overall operation within the information processing apparatus 900 according to various programs. Also, the CPU 901 may be a microprocessor. The ROM 902 stores programs used by the CPU 901, arithmetic parameters, and the like. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that change as appropriate during the execution. These are interconnected by a host bus 904 composed of a CPU bus or the like. The CPU 901 can form the image acquisition unit 21, the detection unit 22, the flow line extraction unit 23, the reliability calculation unit 25, the measurement line setting unit 26, the measurement unit 28, and the output unit 29 shown in FIGS. 1 and 6.

[0090] The host bus 904 is connected to an external bus 906 such as a PCI (Peripheral Component Interconnect / Interface) bus via the bridge 905. Note that it is not always necessary to configure the host bus 904, the bridge 905, and the external bus 906 separately, and these functions may be implemented on one bus.

[0091] The input device 908 is composed of input means such as a mouse, keyboard, touch panel, buttons, microphone, switch, and lever for the user to input information, and an input control circuit that generates an input signal based on the input by the user and outputs it to the CPU 901. The user who operates the information processing device 900 can input various data to the information processing device 900 or instruct processing operations by operating this input device 908.

[0092] The output device 909 includes, for example, display devices such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, a lamp, and an audio output device such as a speaker.

[0093] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, a deleting device for deleting data recorded on the storage medium, and the like. The storage device 910 is composed of, for example, an HDD (Hard Disk Drive). This storage device 910 drives a hard disk and stores programs executed by the CPU 901 and various data. The storage device 910 can form, for example, the flow line storage unit 24, the measurement line storage unit 27, and the calibration file storage unit 40 shown in FIGS. 1 and 6.

[0094] The communication device 911 is a communication interface composed of, for example, a communication device for connecting to a network. Also, the communication device 911 may support either wireless communication or wired communication. The communication device 911 can function as an interface between, for example, the information processing device 20, the imaging device 10, and the output device 30.

[0095] Above, an example of the hardware configuration capable of realizing the functions of the information processing apparatus 900 according to the present embodiment has been shown. Each of the above-described components may be realized using general-purpose members, or may be realized by hardware specialized for the functions of each component. Therefore, it is possible to appropriately change the hardware configuration to be used according to the technical level at the time of implementing the present embodiment.

[0096] <5. Supplementary> As described above, the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present invention.

[0097] For example, in the above embodiment, a vehicle passing through an intersection has been described as an example, but the present invention is not limited to such an example. The measurement area may be a place without lanes such as a parking lot.

[0098] For example, in the above embodiment, an example in which the moving body is a vehicle has been described, but the present invention is not limited to such an example. The moving body may be a person or an animal other than a person. The measurement area may be a store, a room, or a corridor inside a building, or may be in the middle of a forest. Further, the moving body is not limited to a real object in the real space, and may be a virtual object such as a character operated by a user in the virtual space.

[0099] For example, in the above embodiment, an example of calculating the speed of a vehicle in the real space using a calibration file has been described, but the present invention is not limited to such an example. The system 1 may include a device capable of observing a three-dimensional position such as a laser sensor or a ToF (Time of Flight) camera, and the speed of the vehicle in the real space may be calculated based on the observation results of these devices.

[0100] In addition, each device described in this specification may be implemented as a single device, or some or all of them may be implemented as separate devices. As an example, among the imaging devices 10 shown in FIGS. 1 and 6, the flow line storage unit 24, the measurement line storage unit 27, and the calibration file storage unit 40 may be provided in a device such as a server connected to the information processing device 20 having the remaining components via a network or the like. As another example, the imaging device 10 and the information processing device 20, or the information processing device 20 and the output device 30, may be implemented as one device.

[0101] In addition, a series of processes performed by each device described in this specification may be implemented using any of software, hardware, and a combination of software and hardware. The program constituting the software is pre-stored, for example, in a recording medium (specifically, a non-transitory storage medium readable by a computer) provided inside or outside each device. Then, each program is read into the RAM when executed by a computer that controls each device described in this specification, for example, and is executed by a processor such as a CPU. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Also, the above computer program may be distributed via a network or the like without using a recording medium.

[0102] Also, the processes described using flowcharts and sequence diagrams in this specification do not necessarily have to be executed in the order shown in the figures. Some processing steps may be executed in parallel. Also, additional processing steps may be adopted, and some processing steps may be omitted.

Explanation of Reference Numerals

[0103] 1 System 10 Imaging Device 20 Information Processing Device 21 Image Acquisition Unit 22 Detection Unit 23 Flow Line Extraction Unit 24 Flow Line Storage Unit 25 Reliability calculation unit 26 Measurement line setting unit 27 Measurement line memory unit 28 Measurement unit 29 Output unit 30 Output device 40 Calibration file memory unit 100 Time-series image 200 Vehicle 210 Detection frame 300 Traffic flow 400 Measurement line

Claims

1. A detection unit that detects a moving object from a time-series image composed of a plurality of images consecutive in time-series obtained by imaging a measurement area; A trajectory extraction unit that extracts trajectory information indicating the trajectory of the moving object in the time-series image detected by the detection unit; A reliability calculation unit that calculates the reliability of the trajectory indicated by the trajectory information extracted by the trajectory extraction unit based on the total length of the trajectory; A measurement unit that measures the number of times the moving object has passed through a measurement line set in the measurement area based on the trajectory for which the reliability calculated by the reliability calculation unit is higher than a predetermined threshold; An information processing apparatus comprising the above.

2. The reliability calculation unit calculates the reliability based on the length of the trajectory at each time of the trajectory. The information processing apparatus according to claim 1.

3. The reliability calculation unit calculates the reliability based on the frame rate of the time-series image. The information processing apparatus according to claim 1 or 2.

4. The trajectory information includes detection coordinates that are coordinates of the detected moving object in each of a plurality of images included in the time-series image, or prediction coordinates that are predicted coordinates of the moving object, The trajectory extraction unit updates the trajectory information by adding the detection coordinates of the detected moving object or the prediction coordinates of the moving object that failed to be detected in an image newly added to the time-series image over time to the trajectory information. The information processing apparatus according to any one of claims 1 to 3.

5. The trajectory extraction unit updates the trajectory information by adding the coordinates of the detected moving object corresponding to the trajectory information to the trajectory information. The information processing apparatus according to claim 4.

6. The reliability calculation unit calculates the reliability based on the configuration of the detection coordinates and the prediction coordinates in the trajectory information. The information processing apparatus according to claim 4 or 5.

7. The measurement unit determines whether or not the moving object has passed through the measurement line based on a specific part of the trajectory indicated by the trajectory information. The information processing apparatus according to any one of claims 1 to 6.

8. The measurement unit determines whether or not the moving object has passed through the measurement line based on a part of the trajectory indicated by the trajectory information that consists of a predetermined number of coordinates. The information processing apparatus according to claim 7.

9. The measurement unit determines whether the moving object has passed through the measurement line based on a part of the traffic flow indicated by the traffic flow information and within a predetermined distance from the measurement line. The information processing apparatus according to claim 7 or 8.

10. The reliability calculation unit calculates the reliability based on a calibration file that converts the coordinates in the time-series image into coordinates in the world coordinate system of the real space. The information processing apparatus according to any one of claims 1 to 9.

11. A detection unit that detects a moving object from a time-series image composed of a plurality of images consecutive in time-series that image a measurement area, A traffic flow extraction unit that extracts traffic flow information indicating the traffic flow of the moving object in the time-series image detected by the detection unit, A reliability calculation unit that calculates the reliability of the traffic flow indicated by the traffic flow information extracted by the traffic flow extraction unit based on a calibration file that converts the coordinates in the time-series image into coordinates in the world coordinate system of the real space, A measurement unit that measures the number of times the moving object has passed through a measurement line set in the measurement area based on the traffic flow whose reliability calculated by the reliability calculation unit is higher than a predetermined threshold value, An information processing apparatus comprising:

12. The reliability calculation unit calculates the reliability based on the speed of the moving object in the real space calculated based on the traffic flow information and the calibration file. The information processing apparatus according to claim 10 or 11.

13. The measurement unit measures the speed of the moving object in the real space when the moving object passes through the measurement line. The information processing apparatus according to any one of claims 10 to 12.

14. The measurement unit measures the number of times the moving object has passed through the measurement line for each direction in which the moving object passes through the measurement line. The information processing apparatus according to any one of claims 1 to 13.

15. The information processing apparatus further includes an output unit that outputs information indicating the measurement result by the measurement unit. The information processing apparatus according to any one of claims 1 to 14.

16. The moving object is a vehicle. The information processing apparatus according to any one of claims 1 to 15.

17. The measurement area includes a road, an intersection, or a parking lot. The information processing apparatus according to any one of claims 1 to 16.

18. Detecting a moving object from a time-series image composed of a plurality of images consecutive in time-series that image a measurement area, Extracting trajectory information indicating a trajectory in the time-series image of the detected moving object; Calculating the reliability of the trajectory indicated by the extracted trajectory information based on the total length of the trajectory; Measuring the number of times the moving object has passed through a measurement line set in the measurement area based on the trajectory whose calculated reliability is higher than a predetermined threshold; An information processing method executed by an information processing apparatus, including the above.

19. Detecting a moving object from a time-series image composed of a plurality of images consecutive in time-series that image a measurement area; Extracting trajectory information indicating a trajectory in the time-series image of the detected moving object; Calculating the reliability of the trajectory indicated by the extracted trajectory information based on a calibration file that converts coordinates in the time-series image into coordinates in a world coordinate system of a real space; Measuring the number of times the moving object has passed through a measurement line set in the measurement area based on the trajectory whose calculated reliability is higher than a predetermined threshold; An information processing method executed by an information processing apparatus, including the above.

20. A computer, A detection unit that detects a moving object from a time-series image composed of a plurality of images consecutive in time-series that image a measurement area; A trajectory extraction unit that extracts trajectory information indicating a trajectory in the time-series image of the moving object detected by the detection unit; A reliability calculation unit that calculates the reliability of the trajectory indicated by the trajectory information extracted by the trajectory extraction unit based on the total length of the trajectory; A measurement unit that measures the number of times the moving object has passed through a measurement line set in the measurement area based on the trajectory whose reliability calculated by the reliability calculation unit is higher than a predetermined threshold; A program for causing the computer to function as the above.

21. A computer, A detection unit that detects a moving object from a time-series image composed of a plurality of images consecutive in time-series that image a measurement area; A trajectory extraction unit that extracts trajectory information indicating a trajectory in the time-series image of the moving object detected by the detection unit; A reliability calculation unit that calculates the reliability of the trajectory indicated by the trajectory information extracted by the trajectory extraction unit based on a calibration file that converts coordinates in the time-series image into coordinates in a world coordinate system of a real space; A measurement unit that measures the number of times the moving object has passed through the measurement line set in the measurement area based on the traffic line for which the reliability calculated by the reliability calculation unit is higher than a predetermined threshold value; A program for causing the above to function.

Images