Data acquisition system and data acquisition method
The data acquisition system improves time accuracy in time-series data by using GNSS signals and a slider mechanism to associate accurate time information with position changes, addressing inaccuracies in manual clock settings and enabling cost-effective, versatile data capture.
Patent Information
- Application Number
- JP2023075940
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-02
- Publication Date
- 2024-11-15
AI Technical Summary
Existing time-series data acquisition systems often suffer from inaccuracies due to manual setting of clock functions, leading to significant deviations in the associated acquisition times.
A data acquisition system that utilizes GNSS signals to provide highly accurate time information by associating it with position information, creating time reference data and identification data, and employing a common mark to specify the acquisition time of multiple data points using a slider mechanism to move the camera and GNSS antenna in unison.
This system enhances the accuracy of time stamps in time-series data by leveraging GNSS signals, allowing the use of commercially available devices and reducing costs while improving versatility and usability.
Smart Images

Figure 2024160724000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a system and method for acquiring time-information-attached time-series data in which time information is associated with each of a plurality of data constituting the time-series data. [Background technology]
[0002] Traditionally, it has been common to acquire and retain (store) time-series data such as video data consisting of multiple image data and numerical transition data consisting of multiple numerical data using cameras and various sensors. The acquired and retained time-series data is used in a variety of situations, including the maintenance and management of infrastructure facilities (e.g., bridges and roads), environmental assessment, and disaster prevention.
[0003] Meanwhile, by associating the acquisition time of multiple data (such as the image data and numerical data) constituting the time series data, it is possible to further increase the value of the time series data. In this regard, a clock function has been provided in a means for acquiring time series data (such as a camera), and the acquisition time of multiple data (image data) constituting the time series data has been associated with the acquisition time of the data (see, for example, Patent Document 1, etc.). [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2005-51278 A Summary of the Invention [Problem to be solved by the invention]
[0005] However, in many cases, the time in the clock function is manually set, and time deviations may occur within the clock function, so the acquisition time associated with the data may deviate relatively significantly from the actual acquisition time.
[0006] The present invention has been made in consideration of the above-mentioned circumstances, and its purpose is to provide a data acquisition system, etc. that can further improve the accuracy of the acquisition time associated with multiple data that constitute time series data. [Means for solving the problem]
[0007] The following describes each of the means suitable for achieving the above object. Note that, as necessary, the specific effects of the corresponding means will be added.
[0008] Means 1. A data acquisition system for acquiring time-series data in which time information relating to the acquisition time of a plurality of data constituting the time series data is associated with the time information of the data, A time series data acquisition means for acquiring the time series data; A receiving means for receiving a GNSS signal from a satellite; a position and time information acquisition means for acquiring position information of the receiving means based on the GNSS signal received by the receiving means and time information when the receiving means is present at a position indicated by the position information; a position change generating means for generating a change in the position information acquired by the position and time information acquiring means; a time reference data acquiring means for acquiring time reference data in which information relating to the position information changed by the position change generating means and the time information acquired by the position and time information acquiring means are associated with each other; an identification data providing means for adding information to the time series data acquired by the time series data acquiring means, the information including information that has a correlation with information related to the position information in the time reference data and information for identifying a plurality of data constituting the time series data; and a data generating means for generating the identification data from the time-series data including information for generating the identification data; a corresponding time identification means for identifying an estimated time based on the time information acquired by the position and time information acquisition means, the estimated time corresponding to the acquisition time of a plurality of data constituting the time series data, by using the time reference data acquired by the time reference data acquisition means and the identification data created by the identification data creation means; a time-information-attached time series data acquisition means for acquiring the time-information-attached time series data in which the estimated time identified by the corresponding time identification means is associated with each of a plurality of data constituting the time series data.
[0009] According to the above-mentioned means 1, the position and time information acquisition means acquires position information and time information based on the GNSS signal, and the position change generating means generates a change in the position information acquired by the position and time information acquisition means. Then, the position change generating means associates the information related to the position information that changes with the time information, thereby acquiring time reference data. The time reference data has highly accurate time information based on the GNSS signal, and associates the highly accurate time information with the changing position information.
[0010] On the other hand, the time series data is acquired by the time series data acquisition means, and the time series data acquired by the time series data acquisition means includes information capable of creating identification data in which information correlating with the position information in the time reference data and information for identifying a plurality of data constituting the time series data (e.g., frame numbers, etc.) are associated with each other by the identification data assignment means. Then, the identification data is created from the time series data including information capable of creating identification data. The identification data is correlated with the position information of the time reference data (e.g., data that changes in the same manner as the position information of the time reference data, data that has a plurality of characteristic corresponding parts that correspond to each of a plurality of characteristic parts of the position information of the time reference data, etc.). Therefore, each of the time reference data and the identification data has a common marker, so to speak.
[0011] Then, the corresponding time identification means uses the time reference data and the identification data to identify an estimated time based on the time information acquired by the position and time information acquisition means, which corresponds to the acquisition time of the multiple data constituting the time series data. That is, by using a common landmark provided in the time reference data and the identification data, an estimated time with high accuracy similar to that of the time information acquired by the position and time information acquisition means is identified as the time corresponding to the acquisition time of the multiple data by using the common landmark provided in the time reference data and the identification data. Finally, time series data with time information is obtained in which the identified estimated time is associated with each of the multiple data.
[0012] In this way, according to the above-mentioned means 1, by using a common mark provided in the time reference data and the identification data, it is possible to obtain time-information-attached time series data in which a highly accurate estimated time is associated with a plurality of data constituting the time series data. Therefore, it is possible to further improve the accuracy of the acquisition time associated with a plurality of data constituting the time series data.
[0013] Furthermore, according to the above-mentioned means 1, when acquiring time-series data with time information associated with a highly accurate estimated time, it is not necessary to modify the time-series data acquisition means, such as by incorporating a module for acquiring highly accurate time information. Therefore, for example, a general commercially available device (such as a commercially available camera) can be used as the time-series data acquisition means, rather than a special device. Therefore, it is possible to improve versatility and reduce costs.
[0014] Means 2. The data acquisition system described in Means 1, characterized in that the position change generating means and the identification data assigning means are composed of a moving means capable of simultaneously moving the time series data acquisition means and the receiving means in the same manner.
[0015] According to the above-mentioned means 2, the position change generating means and the identifying data providing means are configured by a moving means (such as a slider capable of reciprocating movement) capable of simultaneously moving the time series data acquiring means and the receiving means in the same manner. Therefore, it is possible to obtain the time reference data and the identifying data that change in the same manner and have a higher correlation.
[0016] Furthermore, since the position change generating means and the identification data providing means can be configured by a common means (that is, a moving means), it is possible to reduce costs and improve usability.
[0017] Means 3. The data acquisition system according to Means 1, wherein the time series data acquisition means is constituted by an imaging means capable of acquiring video data consisting of a plurality of image data as the time series data.
[0018] According to the above-mentioned means 3, the time series data acquisition means is constituted by an imaging means (for example, a camera), and therefore it is possible to obtain highly accurate times as the acquisition times of the image data constituting the video data.
[0019] Means 4. The data acquisition system described in Means 3, wherein the identification data creation means is configured to extract feature points in the image data and create data indicating the displacement of the extracted feature points as the identification data.
[0020] According to the above-mentioned means 4, it is possible to relatively easily create identification data from video data (a plurality of image data), and therefore it is possible to sufficiently keep the processing load involved in creating the identification data low.
[0021] Means 5. The data acquisition system described in Means 1, characterized in that the time series data acquisition means is composed of a displacement-related data measurement means capable of acquiring displacement-related data consisting of a plurality of data relating to the position of the data acquisition target as the time series data.
[0022] Incidentally, the "displacement-related data" may be displacement data, or may be data indicating an element related to the displacement (for example, acceleration data or speed data).
[0023] According to the above-mentioned means 5, the time-series data acquisition means is configured with a displacement-related data measurement means (for example, an acceleration sensor or a displacement sensor). Therefore, it is possible to obtain highly accurate times as the acquisition times of the multiple data constituting the displacement-related data.
[0024] Means 6. The position change generating means and the identifying data providing means are capable of generating waveform data having a plurality of peaks as the time reference data and the identifying data, respectively; The data acquisition system described in means 1, characterized in that the corresponding time identification means uses a peak in the waveform data to identify the estimated time corresponding to the acquisition time of multiple data constituting the time series data.
[0025] According to the above-mentioned means 6, since the peaks of the waveform data are utilized, it is possible to more accurately specify the estimated time corresponding to the acquisition time of the plurality of data constituting the time series data, thereby further improving the accuracy of the acquisition time associated with the plurality of data constituting the time series data.
[0026] Means 7. The corresponding time identification means obtains a regression equation showing the relationship between information for identifying multiple data constituting the time series data and the time information acquired by the position and time information acquisition means based on the time reference data and the identification data, and uses the regression equation to identify the estimated time corresponding to the acquisition time of the multiple data constituting the time series data.
[0027] According to the above-mentioned means 7, it is possible to more easily obtain a more accurate time as an estimated time corresponding to the acquisition time of a plurality of data constituting the time-series data.
[0028] Means 8. A data acquisition method for acquiring time-series data in which time information relating to the acquisition time of a plurality of data constituting the time series data is associated with the time information of the data, comprising: a time series data acquiring step of acquiring the time series data using a predetermined time series data acquiring means; a receiving step of receiving a GNSS signal from a satellite using a receiving means capable of receiving the GNSS signal; a position and time information acquisition step of acquiring position information of the receiving means based on the GNSS signal received by the receiving means in the receiving step, and time information when the receiving means is present at a position indicated by the position information; a position change generating step of causing a change in the position information acquired by the position and time information acquiring step; a time reference data acquiring step of acquiring time reference data in which information related to the position information that changes due to the position change occurrence step is associated with the time information acquired by the position and time information acquiring step; an identification data adding step of adding information to the time series data acquired by the time series data acquiring step, the information being capable of creating identification data in which information correlating with information related to the position information in the time reference data and information for identifying a plurality of data constituting the time series data are associated with each other; a data creation step of creating the identification data from the time-series data including information enabling the creation of the identification data; a corresponding time specifying step of specifying an estimated time based on the time information obtained by the position and time information obtaining step, the estimated time corresponding to the acquisition time of the plurality of data constituting the time series data, by using the time reference data obtained by the time reference data obtaining step and the identification data created by the identification data creating step; and acquiring time series data with time information, in which the estimated time identified by the corresponding time identification process is associated with each of a plurality of data constituting the time series data.
[0029] According to the above-mentioned means 8, the same effects as those of the above-mentioned means 1 are achieved.
[0030] The technical matters related to the above-mentioned means may be combined as appropriate. For example, the technical matters related to the above-mentioned means 2 may be combined with the technical matters related to the above-mentioned means 3. Also, the technical matters related to the above-mentioned means 8 may be combined with at least one of the technical matters related to the above-mentioned means 2 to 7. [Brief description of the drawings]
[0031] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a data acquisition system. [Diagram 2] FIG. 11 is a table showing an example of video data obtained by a camera. [Diagram 3] 2 is a schematic perspective view showing a slider and a camera etc. installed on the slider. FIG. [Figure 4] FIG. 2 is a schematic front view showing a camera and the like that are reciprocated by a slider. [Diagram 5] 10 is a table illustrating an example of information stored in a position and time information storage unit. FIG. [Figure 6] FIG. 11 is a table showing an example of time reference data. [Figure 7] 11 is a graph showing an example of time reference data. [Figure 8] FIG. 2 is a schematic diagram showing feature points and the like in image data. [Figure 9] FIG. 13 is a table showing an example of candidates for identification data. [Figure 10] 11 is a graph showing an example of identification data. [Figure 11] FIG. 11 is a table showing an example of time-series data with time information. [Figure 12] 1 is an explanatory diagram for explaining an example of a method for acquiring time-series data with time information using a data acquisition system. FIG. [Figure 13]FIG. 2 is an explanatory diagram for explaining, in a simplified manner, the steps performed to acquire time-series data with time information. [Figure 14] FIG. 13 is a table showing the relationship between the map information level and the standard deviation of the horizontal position. [Figure 15] FIG. 13 is a schematic perspective view showing an impact generating unit as a specifying data providing unit in another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0032] An embodiment will be described below with reference to the drawings. The data acquisition system 1 is for acquiring time-series data with time information in which each of a plurality of data constituting the time series data is associated with time information relating to the acquisition time of the data.
[0033] As shown in Fig. 1, the data acquisition system 1 includes a camera 2, a GNSS antenna 3, a GNSS receiver 4, a slider 5, and a control device 6. In this embodiment, the camera 2 constitutes a "time series data acquisition means" and an "imaging means", and similarly, the GNSS antenna 3 constitutes a "receiving means", the GNSS receiver 4 constitutes a "position and time information acquisition means", and the slider 5 constitutes a "position change generating means", a "identification data providing means", and a "moving means".
[0034] Camera 2 is a typical commercially available video camera equipped with a lens as a light collecting means, an imaging element that converts light passing through the lens into an electrical signal, and an image memory (not shown) that stores image data input from the imaging element. Camera 2 acquires video data having a plurality of image data as time-series data. The acquired video data (time-series data) is stored in the image memory in camera 2, and can then be sent from camera 2 to control device 6 via wired or wireless communication.
[0035] The acquired video data (time series data) is associated with a plurality of image data [Frame (Image)] and frame numbers (Frame No.), which are information for identifying these image data (see FIG. 2). Furthermore, in this embodiment, the video data (time series data) is provided with an internal time that roughly indicates the time when each image data was acquired, based on the clock function in the camera 2 (the internal clock of the camera 2). However, the time indicated by the clock function in the camera 2 may not be very accurate due to reasons such as the time being manually set, and therefore the internal time provided to the video data is often not very accurate either. Note that the video data may not be provided with an internal time.
[0036] In addition, the camera 2 in this embodiment is equipped with an operating battery (not shown) and can operate independently (no power supply is required). Therefore, the camera 2 is extremely easy to carry and install. For example, the "GoPro 9" (product name) manufactured by GoPro (registered trademark) can be used as the camera 2. In this embodiment, the process of acquiring video data by the camera 2 corresponds to the "time-series data acquisition process."
[0037] The GNSS antenna 3 is an antenna for receiving GNSS signals transmitted from positioning satellites such as GPS satellites. For example, "ANN-MB-00" (product name) manufactured by u-blox can be used as the GNSS antenna 3. In this embodiment, the process of receiving the GNSS signal by the GNSS antenna 3 corresponds to the "reception process."
[0038] The GNSS receiver 4 is connected to the GNSS antenna 3, and acquires position information of the GNSS antenna 3 and time information when the GNSS antenna 3 is located at the position indicated by the position information, based on the GNSS signal received by the GNSS antenna 3. The acquired position information and time information are sent to the control device 6. Information related to latitude and longitude is acquired as the position information, and Coordinated Universal Time (UTC) is acquired as the time information.
[0039] The GNSS antenna 3 and GNSS receiver 4 in this embodiment constitute a dual-frequency GNSS receiving system, which has better positioning accuracy than a single-frequency GNSS receiving system.
[0040] Moreover, the GNSS receiver 4 in this embodiment acquires position information using network-type RTK (Real-Time-Kinematic) positioning. In this embodiment, the horizontal position accuracy of the acquired position information is several centimeters. For example, "Cohac∞QZNEO" (product name) manufactured by CORE Corporation can be used as the GNSS receiver 4. In this embodiment, the process of acquiring position information and time information by the GNSS receiver 4 corresponds to the "position and time information acquisition process."
[0041] The slider 5 is a device for causing a change in the position information acquired by the GNSS receiver 4, and for including information capable of creating identification data (described later) in the video data (time series data) acquired by the camera 2. The slider 5 includes a pair of rotatable rollers, a drive source for rotating at least one of the rollers, and a control means (not shown) for controlling the operation of the drive source to control the rotation of each of the rollers. As shown in Fig. 3, the slider 5 includes a circular belt 51 looped around the pair of rollers, and the control means controls the direction of rotation of the rollers, allowing the upper surface of the belt 51 to move back and forth from side to side.
[0042] Furthermore, a base 52 for mounting the camera 2 and the like is fixed to the upper surface of the belt 51. The camera 2 is configured to be easily attached and detached to the base 52. As the slider 5, for example, "Micro2Plus" (product name) manufactured by Zeapon Corporation can be used.
[0043] In this embodiment, the slider 5 operates such that the upper surface of the belt 51 reciprocates left and right at a constant period with the camera 2 installed on the base 52 and the GNSS antenna 3 attached on the camera 2. By operating the slider 5 in this manner, the GNSS antenna 3 and the camera 2 can be simultaneously reciprocated in the same manner (see FIG. 4). As a result, a change in a predetermined manner can be caused in the position information acquired by the GNSS receiver 4, and a change correlating with the predetermined manner can be caused in the position of the imaged object in the image data of the video data (time-series data) captured by the camera 2.
[0044] In this embodiment, the slider 5 is controlled so that the upper surface of the belt 51 reciprocates with an amplitude of 54 cm and a period of about 40 seconds. In this embodiment, the process of reciprocating the GNSS antenna 3 and the camera 2 by the slider 5 corresponds to the "position change generating process" and the "identification data assigning process".
[0045] The control device 6 is for obtaining time-series data with time information, using the time information and position information obtained by the GNSS receiver 4, and the time-series data (video data) obtained by the camera 2. The control device 6 is made up of a computer including a CPU (Central Processing Unit) that executes predetermined arithmetic processing, a ROM (Read Only Memory) that stores various programs and fixed value data, a RAM (Random Access Memory) that temporarily stores various data when various arithmetic processing is executed, a storage device for storing information, and peripheral circuits for these.
[0046] The control device 6 includes a position and time information storage unit 61, a time reference data acquisition unit 62, a video data storage unit 63, an identification data creation unit 64, a corresponding time identification unit 65, and a final data acquisition unit 66. In this embodiment, the time reference data acquisition unit 62 constitutes a "time reference data acquisition means", and similarly, the identification data creation unit 64 constitutes an "identification data creation means", the corresponding time identification unit 65 constitutes a "corresponding time identification means", and the final data acquisition unit 66 constitutes a "time-information-attached time-series data creation means". The various functions of the time reference data acquisition unit 62, the identification data creation unit 64, and the final data acquisition unit 66 are realized by cooperation between the various hardware such as the CPU, ROM, and RAM and software pre-stored in the storage device.
[0047] The position and time information storage unit 61 stores, in association with each other, the position information and the time information sent from the GNSS receiver 4. The stored information includes the position information and the time information obtained when the GNSS antenna 3 is moved back and forth by the slider 5. In this embodiment, at least the longitude and latitude as the position information, the time information (time), and an index number (time_idx) for identifying the position information and the time information are stored in association with each other (see FIG. 5).
[0048] The time reference data acquisition unit 62 acquires reference time data that is formed by associating information relating to the position information of the GNSS antenna 3 acquired by the GNSS receiver 4, particularly information relating to the position information that changes due to the reciprocating movement of the slider 5 (referred to as "position information during reciprocating movement"), with the position information acquired by the GNSS receiver 4.
[0049] To acquire the reference time data, the time reference data acquisition unit 62 first acquires information related to the position information during the reciprocating movement described above. More specifically, the time reference data acquisition unit 62 extracts the position information obtained when the GNSS antenna 3 is reciprocated by the slider 5 from the information stored in the position and time information storage unit 61. The extraction of the position information is performed, for example, using rough times regarding the start and end of the movement of the slider 5 (belt 51) that are acquired (input) in advance.
[0050] Next, the time reference data acquisition unit 62 converts the extracted position information (longitude and latitude) into plane rectangular coordinates. Then, the time reference data acquisition unit 62 calculates the L2 norm (Euclidean norm) of the converted position information (longitude and latitude) with respect to a predetermined reference coordinate set in advance as information related to the round-trip position information.
[0051] The time reference data acquisition unit 62 then acquires reference time data by associating the calculated L2 norm [dist(L2 Norm)] with time information (time) corresponding to the position information on which the calculated L2 norm is based (see FIG. 6). The time reference data is waveform data in which the L2 norm changes periodically with respect to the time information (time) (see FIG. 7). Note that although the reference time data in this embodiment is assigned the above-mentioned index number (time_idx), it may not be assigned an index number.
[0052] The video data storage unit 63 stores video data (time-series data) obtained by the camera 2. In this embodiment, video data is stored in the video data storage unit 63 by, for example, copying acquired video data stored in the camera 2. The stored video data includes data obtained when the camera 2 is reciprocated by the slider 5. Note that a configuration may be adopted in which the video data is stored in the video data storage unit 63 in parallel with the acquisition of the video data by the camera 2.
[0053] The identification data creation unit 64 creates identification data from the moving image data (time-series data) stored in the moving image data storage unit 63. The identification data is data in which information correlating with information relating to position information in the time reference data (i.e., the above-mentioned L2 norm) is associated with information for identifying a plurality of image data constituting the moving image data (time-series data) (in this embodiment, frame numbers).
[0054] In creating the identification data, the identification data creation unit 64 first extracts, from the information stored in the video data storage unit 63, video data (time-series data) obtained when the camera 2 is moved back and forth by the slider 5. The video data is extracted using, for example, rough times regarding the start and end of the movement of the slider 5 (belt 51) that are previously acquired (input).
[0055] Next, the identification data creation unit 64 extracts feature points SP (see FIG. 8) from each image data by, for example, the Shi-Tomasi corner detection method, etc. A plurality of feature points SP can be extracted.
[0056] Furthermore, the identification data creation unit 64 creates data indicating the displacement of the extracted feature point SP as identification data. That is, the identification data creation unit 64 first calculates the L2 norm of the coordinates indicated by the feature point SP relative to a predetermined reference coordinate on the image data for each feature point SP. Next, the identification data creation unit 64 creates identification data candidates by associating the L2 norm (dist1, d-ist2, dist3...) for each feature point SP with the frame number (F-frame No.) corresponding to the image data from which the calculated L2 norm is based (see FIG. 9). In FIG. 9, the identification data candidates are shown as those in which dist1 and Frame No. are associated, those in which dist2 and Frame No. are associated, and those in which dist3 and Frame No. are associated. When multiple feature points SP are extracted, multiple identification data candidates are created. As the slider 5 reciprocates, the created identification data candidates are usually waveform data in which the L2 norm changes periodically with respect to the frame number.
[0057] Then, the identification data creating unit 64 selects one data that satisfies the "predetermined condition" from among the created identification data candidates, thereby finally creating one identification data. However, if there is only one created identification data candidate, that candidate becomes the identification data. Since the camera 2 moves in the same manner as the GNSS antenna 3 moves, the identification data becomes waveform data that changes in the same manner as the time reference data (see FIG. 10). In other words, the time reference data and the identification data each have a common marker (peak). Incidentally, as the "predetermined condition," a condition such as whether the amplitude of the waveform data is approximately constant or not can be adopted. In this embodiment, the process of creating identification data by the identification data creating unit 64 corresponds to the "identification data creating process." Incidentally, multiple pieces of identification data may be created by selecting multiple pieces of data from multiple identification data candidates.
[0058] The corresponding time identification unit 65 uses the time reference data and the identification data to identify an estimated time based on the time information acquired by the GNSS receiver 4, which corresponds to the acquisition time of multiple image data constituting the video data (time series data). More specifically, the corresponding time identification unit 65 first detects peaks in the time reference data and the identification data. Peak detection can be performed using, for example, the signal.ar-grelmax function in the scipy module of the programming language python. Note that, in the peak detection, noise parts such as high frequency components in the time reference data and the identification data may be removed by using, for example, a low pass filter.
[0059] Then, based on each peak of the time reference data and the identification data, the corresponding time identification unit 65 obtains a regression equation showing the relationship between information (frame number) for identifying multiple image data that make up the video data (time series data) and the time information acquired by the GNSS receiver 4.
[0060] In obtaining the regression equation, the corresponding time identification unit 65 first uses each peak of the detected time reference data and identification data as a marker to create a plurality of pair data (Frame No., time) each of which is a set of the frame number at each peak of the identification data and the time information at each peak of the time reference data corresponding to the frame number. Then, the corresponding time identification unit 65 performs regression analysis on the created plurality of pair data to obtain a regression equation indicating the relationship between the frame number and the time information. In this embodiment, the corresponding time identification unit 65 obtains a simple regression equation [y=f(x)], where y indicates the time information and x indicates the frame number. Note that if a plurality of identification data are created, a plurality of simple regression equations can be obtained.
[0061] Next, the corresponding time determination unit 65 uses the acquired simple regression equation to determine, for each of the multiple image data, an estimated time corresponding to the acquisition time of the image data. That is, the corresponding time determination unit 65 performs this process for all image data constituting the video data, substituting the frame number (x) into the simple regression equation [y=f(x)] to determine the acquired time information (y) as the estimated time. In this way, the estimated time corresponding to the acquisition time of the image data is determined for each image data.
[0062] In addition, when a plurality of simple regression equations are obtained, the average value or median value of the plurality of time information obtained from each simple regression equation may be used as the estimated time. In this embodiment, the process of identifying the estimated time by the corresponding time identifying unit 65 corresponds to the "corresponding time identifying process".
[0063] The final data acquisition unit 66 acquires time series data with time information (see Figure 11) in which the estimated time identified by the corresponding time identification unit 65 is associated with each of the multiple image data [Frame (Image)] that make up the video data (time series data).
[0064] When acquiring the time series data with time information, the final data acquisition unit 66 performs a correction process for all image data to correct (replace) the internal time (the time obtained by the clock function of the camera 2) in the video data stored in the video data storage unit 63 with the estimated time identified by the corresponding time identification unit 65. This allows the time series data with time information to be acquired. Note that the final data acquisition unit 66 may acquire the time series data with time information by associating the estimated time identified by the corresponding time identification unit 65 with each image data constituting the video data without correcting the internal time. In this embodiment, the process of acquiring the time series data with time information by the final data acquisition unit 66 corresponds to the "time series data with time information acquisition process."
[0065] Next, a description will be given of an example of using the above-mentioned data acquisition system 1. Prior to using the data acquisition system 1, after starting up the GNSS antenna 3 and the GNSS receiver 4 (GNSS receiving system), the GNSS antenna 3 is placed in an outdoor environment for a certain period of time, whereby an initialization process is performed in advance for obtaining a fix solution by the GNSS receiver 4. The initialization process may be performed with the GNSS antenna 3 installed relative to the slider 5 in a stopped state.
[0066] 12, in order to acquire time-series data with time information, first, a time-series data acquisition step S1, a reception step S2, and a position and time information acquisition step S3 are performed in parallel. The time-series data acquisition step S1 is a step of acquiring video data by the camera 2, and the reception step S2 is a step of receiving a GNSS signal by the GNSS antenna 3. The position and time information acquisition step S3 is a step of acquiring, by the GNSS receiver 4, position information of the GNSS antenna 3 based on the GNSS signal received by the GNSS antenna 3, and time information when the GNSS antenna 3 is in the position indicated by the position information.
[0067] Then, while these steps S1, S2, and S3 are being performed, a first position change generating step S41 is executed. The first position change generating step S41 is a step of reciprocating the slider 5 (belt 51) in a state in which the camera 2 and the GNSS antenna 3 are installed on the slider 5.
[0068] After the first position change generating step S41 continues for a predetermined time t1 (e.g., 5 minutes), the slider 5 is stopped, the camera 2 is removed from the slider 5, and the target object is imaged by the camera 2. For example, the camera 2 removed from the slider 5 is installed on a vehicle, and while the vehicle is traveling along a predetermined route, the camera 2 images the road and its surrounding environment as the target object for a predetermined time (e.g., 40 minutes).
[0069] After capturing an image of a road or the like, a second position change generating step S42 is executed while performing the time series data acquiring step S1 and the receiving step S2 (i.e., without stopping the camera 2 or the GNSS receiving system). The second position change generating step S42 is a step of reciprocating the slider 5 (the belt 51) in a state in which the camera 2 and the GNSS antenna 3 are installed on the slider 5, similar to the first position change generating step S41. These position change generating steps S41 and S42 can cause a certain change in the position information acquired by the GNSS receiver 4, and can also include information capable of creating the above-mentioned identification data (data having a correlation with the position information) in the video data (time series data) acquired by the camera 2. In this embodiment, both position change generating steps S41 and S42 correspond to a "position change generating step" and an "identification data providing step".
[0070] After the second position change generating step S42 continues for a predetermined time t2 (e.g., 5 minutes), the slider 5 is stopped, the camera 2 is removed from the slider 5, and the video data (time series data) obtained by the camera 2 is stored in the video data storage unit 63. By performing the position and time information acquiring step S3, the position and time information storage unit 61 has already stored therein the time information and position information.
[0071] When the video data is stored in the video data storage unit 63, a time reference data acquisition step S51, an identification data creation step S52, a corresponding time identification step S53, and a final data acquisition step S54 are performed by the control device 6. In this embodiment, the final data acquisition step S54 corresponds to the "time-series data with time information acquisition step".
[0072] In the time reference data acquisition step S51, the time reference data acquisition unit 62 acquires time reference data in which the position information and time information obtained during the execution of both the position change generation steps S41, S42 are associated with each other, based on the position information and time information.
[0073] In addition, in the identification data creation step S52, the identification data creation unit 64 creates identification data based on the video data (time-series data) obtained by the camera 2 during the execution of the position change occurrence steps S41 and S42. Furthermore, in a corresponding time identification step S53, the corresponding time identification unit 65 uses the time reference data and identification data acquired or created in the above steps S51 and S52 to determine a simple regression equation [y=f(x)]. Then, using the simple regression equation, the estimated time corresponding to the acquisition time of the multiple image data constituting the video data is identified.
[0074] Then, in the final data acquisition step S54, the final data acquisition unit 66 associates the estimated time identified by the corresponding time identification unit 65 with each of the image data constituting the video data, thereby acquiring time series data with time information.
[0075] In this manner, the data acquisition system 1 acquires time-series data with time information by executing steps A to E, which will be described below, as shown in FIG.
[0076] Step A: A process of moving the camera 2 and the GNSS antenna 3 back and forth so that a common marker (peak) is provided in the time reference data and the identification data. Step B: A process of acquiring identification data based on video data (time-series data) acquired by camera 2. Step C: A process of generating time reference data based on the position information and time information acquired by the GNSS receiver 4. Step D: A process of obtaining a simple regression equation showing the relationship between the frame number and the time information acquired by the GNSS receiver 4 based on the time reference data and the identification data. Step E: A step of acquiring time-series data with time information by associating the estimated time determined by the simple regression equation with each image data constituting the video data (time-series data). When the estimated time obtained by the above-mentioned use example was compared with the accurate time, the root mean square error between the two times was about 49 ms. Therefore, it can be said that it is possible to obtain an estimated time with a very high degree of accuracy. The accurate time can be obtained, for example, by using a simple regression equation that indicates the relationship between the frame number of the image data and the time captured in the image data based on the Coordinated Universal Time (UTC) time provided by a time information providing service or the like, based on the video data captured by the camera 2.
[0077] Based on the time error mentioned above, the error in the position information is approximately 0.54 m when the vehicle is traveling at 40 km / h, and approximately 0.81 m when the vehicle is traveling at 60 km / h. According to the relationship between the map information level, which indicates the map representation accuracy of digital topographical map data, and the standard deviation of horizontal position (see Figure 14) indicated by the Geospatial Information Authority of Japan, this error in the position information falls within the range of the standard deviation of horizontal position at map information levels 1000 and 2500. Therefore, it can be said that the estimation accuracy of the shooting position is also sufficiently high.
[0078] As described above in detail, according to this embodiment, by utilizing a common landmark (peak) provided in the time reference data and the identification data, the estimated time, which is based on the highly accurate time information obtained by the GNSS receiver 4 and has the same high accuracy as the time information, is identified as the time corresponding to the acquisition time of the multiple image data. Therefore, it is possible to obtain time-series data with time information in which the highly accurate estimated time is associated with the multiple image data constituting the video data (time-series data). As a result, it is possible to further improve the accuracy of the acquisition time associated with the multiple image data constituting the video data (time-series data).
[0079] Furthermore, in order to obtain time-series data with time information associated with a highly accurate estimated time, there is no need to modify the camera 2, such as by incorporating a module for obtaining highly accurate time information. Therefore, instead of a special camera, as in this embodiment, a general commercially available camera 2 can be used. This allows for improved versatility and reduced costs.
[0080] In addition, the data acquisition system 1 can be used even in areas where mobile phone signals cannot reach (dead zones). Therefore, the data acquisition system 1 is particularly effective for confirming the disaster situation in dead zones and recovering from disasters.
[0081] Furthermore, since the camera 2 and the GNSS antenna 3 are simultaneously moved in the same manner by the slider 5, it is possible to obtain time reference data and identification data that change in the same manner and have a higher correlation.
[0082] In addition, since the "position change generating means" and the "identification data providing means" are configured by a common means (that is, the slider 5), it is possible to reduce costs and improve usability.
[0083] Additionally, the identification data creation unit 64 extracts feature points SP in the image data and creates data indicating the displacement of the extracted feature points SP as identification data. Therefore, it is relatively easy to create identification data from video data (multiple image data). This makes it possible to sufficiently reduce the processing load involved in creating the identification data.
[0084] In addition, since the peaks of the waveform data are used to determine the estimated time, the estimated time can be determined more accurately, thereby further improving the accuracy of the acquisition times associated with the multiple data constituting the video data (time-series data).
[0085] Furthermore, the estimated time corresponding to the acquisition time of a plurality of data constituting the video data (time series data) is specified using the simple regression equation, so that it is possible to more easily obtain a more accurate estimated time corresponding to the acquisition time of a plurality of data constituting the video data (time series data).
[0086] The present invention is not limited to the above-described embodiment, and may be implemented as follows: Of course, other applications and modifications not exemplified below are also possible.
[0087] (a) In the above embodiment, the camera 2 is given as the “time series data acquisition means”, but the “time series data acquisition means” may be anything capable of acquiring time series data, and is not limited to the camera 2.
[0088] Therefore, for example, the "time series data acquisition means" may be configured with an acceleration sensor or a displacement measurement sensor capable of acquiring displacement-related data consisting of a plurality of data related to the position of the data acquisition target. Even in such a case, it is possible to obtain highly accurate time as the acquisition time of the plurality of data constituting the displacement-related data. The acceleration sensor or the displacement measurement sensor corresponds to the "displacement-related data measurement means." Furthermore, the "displacement-related data" may be displacement data, or may be data indicating an element related to the displacement (for example, acceleration data or speed data).
[0089] Furthermore, for example, a current sensor, a magnetic sensor, a distance measurement sensor, a proximity sensor, an illuminance sensor, a pressure sensor, an angular velocity sensor, an impact sensor, or the like may be used as the "time series data acquisition means."
[0090] (b) In the above embodiment, the "position change generating means" and the "identification data providing means" are configured by the slider 5, but the "position change generating means" and the "identification data providing means" may be configured by a device for moving the GNSS antenna 3 or the camera 2 other than the slider 5. For example, the "position change generating means" may be configured by a device that swings the GNSS antenna 3 or the like left and right along an arc-shaped path. This device also corresponds to the "moving means".
[0091] (c) In the above embodiment, the camera 2 is moved using the slider 5 to include information that can be used to create identification data for the video data (time series data). However, the video data may also be configured to include information that can be used to create identification data for the video data by using a fixed camera 2 to capture an image of something that moves using the slider 5 (such as a GNSS antenna 3).
[0092] (d) In the above embodiment, the slider 5 as the "identification data providing means" is configured to include information capable of creating identification data in the video data (time-series data) obtained by the camera 2. However, the "identification data providing means" is not limited to the slider 5 as long as it can change the data acquired by the "time-series data acquiring means" and cause the time-series data to include information correlating with information related to position information in the time reference data.
[0093] Therefore, for example, as shown in FIG. 15, when an acceleration sensor 21 is used as the "time series data acquisition means", the "identification data providing means" may be configured by an impact generating unit 71 for applying an impact to the acceleration sensor 21 at the timing when the GNSS antenna 3 installed on the slider 5 reaches both ends of the reciprocating movement range. In this configuration, the multiple data (acceleration data, etc.) constituting the time series data change suddenly at the position corresponding to the peak of the time reference data. Therefore, the time series data includes information having a correlation with information related to the position information in the time reference data (i.e., the peak position), and by using this information, the estimated time with high accuracy based on the GNSS signal can be obtained as the acquisition time of the multiple data (acceleration data, etc.). Note that when an acceleration sensor is used, "information for identifying the multiple data constituting the time series data" refers to a number or the like assigned to identify each data, similar to the frame number in the above embodiment.
[0094] Furthermore, when an acceleration sensor is used as the "time-series data acquisition means", the "identification data assignment means" may be constituted by a device that moves the acceleration sensor while changing its speed (acceleration). In this case, when the acceleration sensor is moved back and forth, different speed (acceleration) changes may be made during forward and backward movements, or the speed (acceleration) may be changed only during a reciprocating movement at a predetermined timing among multiple reciprocating movements.
[0095] Furthermore, the "identification data providing means" may be appropriately configured to generate displacement, electric current, magnetism, electromagnetic waves, sound waves, light, or the like, depending on the type of data acquired by the "time-series data acquiring means".
[0096] (e) In the above embodiment, the camera 2 is installed in a vehicle to capture images of a road and its surrounding environment as an example of use of the data acquisition system 1, but the use of the data acquisition system 1 is not limited to this. Therefore, for example, the camera 2 may be placed at a certain point and used to capture images of a bridge and vehicles passing over the bridge. In this case, it is possible to obtain, based on the video data, the transition of vibrations occurring on the bridge and the relationship between the magnitude of the vibrations and passing vehicles.
[0097] Furthermore, the state of the natural environment, such as mountains, may be captured by the camera 2. In this case, information relating to movements on the ground surface, etc., can be obtained based on the video data.
[0098] Furthermore, when an acceleration sensor or a displacement measuring sensor is used as the "time series data acquisition means", the acceleration sensor may be used to measure the displacement of a structure such as a bridge. In this case, inspection of the structure can be performed based on the acquired time series data.
[0099] In addition, in the above usage example, after acquiring the video data, an estimated time is associated with the video data, but the process of acquiring the video data and the process of associating the estimated time with the acquired video data may be performed in parallel.
[0100] (f) In the above example of use, the position change generating steps S41 and S42 are performed twice, but the position change generating step may be performed once or three or more times.
[0101] (g) In the above embodiment, the time reference data and the identification data are waveform data that change in the same manner and have periodicity, but they do not need to be periodic, and they do not need to be waveform data. It is sufficient that the time reference data and the identification data can be used to grasp the temporal correspondence between the two data by utilizing their characteristic points (i.e., common landmarks).
[0102] (h) In the above embodiment, the corresponding time identifying unit 65 identifies the estimated time by using peaks in the time reference data and the identifying data, but it is not necessary to use peaks. Therefore, the corresponding time identifying unit 65 may identify the estimated time by a method that uses correlations related to the change modes of the time reference data and the identifying data (for example, a method that uses the similarity between two waveform data). In this case, artificial intelligence (AI) may be used to grasp the correlation between the two data.
[0103] (i) In the above embodiment, a simple regression equation is obtained to specify the estimated time, but it is not necessary to obtain a simple regression equation as long as the estimated time is based on highly accurate time information obtained by the GNSS receiver 4. Therefore, in some cases (for example, when the number of time series data obtained per unit time is equal to or less than the number of pieces of time information acquired by the GNSS receiver 4 per unit time), the time information obtained by the GNSS receiver 4 may be used as it is as the estimated time.
[0104] (j) Although not mentioned in the above embodiment, 1PPS (a precise time pulse synchronized with UTC) may be used to further improve the time precision.
[0105] (k) When a time-series data acquisition means capable of moving while acquiring time-series data is used, as in the camera 2 of the above embodiment, data in which an estimated position corresponding to the acquisition position of each of a plurality of data constituting the time-series data is associated with each of the plurality of data may be acquired as time-series data with time information. The estimated position refers to a position estimated based on position information acquired by the GNSS receiver 4. The estimated position can be acquired, for example, by creating a plurality of paired data in which the position information and time information acquired by the GNSS receiver 4 are paired, and using a regression model obtained by performing a regression analysis on the created plurality of paired data. In some cases, the position information acquired by the GNSS receiver 4 may be used as it is as the estimated position. [Explanation of symbols]
[0106] 1...data acquisition system, 2...camera (time series data acquisition means, imaging means), 3...GNSS antenna (receiving means), 4...GNSS receiver (position and time information acquisition means), 5...slider (position change generating means, identification data assigning means, moving means), 21...acceleration sensor (time series data acquisition means), 62...time reference data acquisition unit (time reference data acquisition means), 64...identification data creation unit (identification data creation means), 65...corresponding time identification unit (corresponding time identification means), 66...final data acquisition unit (time series data with time information acquisition means), 71...impact generation unit (identification data assigning means).
Claims
1. A data acquisition system for acquiring time-information-attached time-series data in which time information relating to an acquisition time of each of a plurality of data constituting time-series data is associated with the data, the data acquisition system comprising: A time series data acquisition means for acquiring the time series data; A receiving means for receiving a GNSS signal from a satellite; a position and time information acquisition means for acquiring position information of the receiving means based on the GNSS signal received by the receiving means and time information when the receiving means is present at a position indicated by the position information; a position change generating means for generating a change in the position information acquired by the position and time information acquiring means; a time reference data acquiring means for acquiring time reference data in which information relating to the position information changed by the position change generating means and the time information acquired by the position and time information acquiring means are associated with each other; an identification data providing means for adding information to the time series data acquired by the time series data acquiring means, the information including information that has a correlation with information related to the position information in the time reference data and information for identifying a plurality of data constituting the time series data; and a data generating means for generating the identification data from the time-series data including information for generating the identification data; a corresponding time identification means for identifying an estimated time based on the time information acquired by the position and time information acquisition means, the estimated time corresponding to the acquisition time of a plurality of data constituting the time series data, by using the time reference data acquired by the time reference data acquisition means and the identification data created by the identification data creation means; and a time-information-attached time series data acquisition means for acquiring the time-information-attached time series data in which the estimated time identified by the corresponding time identification means is associated with each of a plurality of data constituting the time series data.
2. The data acquisition system according to claim 1, characterized in that the position change generating means and the identification data assigning means are composed of a moving means capable of simultaneously moving the time series data acquiring means and the receiving means in the same manner.
3. 2. The data acquisition system according to claim 1, wherein the time-series data acquisition means is constituted by an imaging means capable of acquiring video data consisting of a plurality of image data as the time-series data.
4. 4. The data acquisition system according to claim 3, wherein the identification data creation means is configured to extract feature points in the image data and create data indicating displacement of the extracted feature points as the identification data.
5. The data acquisition system according to claim 1, characterized in that the time series data acquisition means is configured by a displacement-related data measurement means capable of acquiring, as the time series data, displacement-related data consisting of a plurality of data relating to the position of a data acquisition target.
6. the position change generating means and the identifying data providing means are capable of generating waveform data having a plurality of peaks as the time reference data and the identifying data, respectively; 2. The data acquisition system according to claim 1, wherein the corresponding time identifying means identifies the estimated time corresponding to the acquisition times of a plurality of data constituting the time series data by using peaks in the waveform data.
7. The corresponding time identification means obtains a regression equation showing the relationship between information for identifying multiple data constituting the time series data and the time information acquired by the position and time information acquisition means, based on the time reference data and the identification data, and identifies the estimated time corresponding to the acquisition time of the multiple data constituting the time series data using the regression equation.
8. A data acquisition method for acquiring time-information-attached time-series data in which time information relating to an acquisition time of each of a plurality of data constituting time-series data is associated with the data, the method comprising: a time series data acquiring step of acquiring the time series data using a predetermined time series data acquiring means; A receiving step of receiving a GNSS signal from a satellite using a receiving means capable of receiving the GNSS signal; a position and time information acquisition step of acquiring position information of the receiving means based on the GNSS signal received by the receiving means in the receiving step, and time information when the receiving means is present at a position indicated by the position information; a position change generating step of causing a change in the position information acquired by the position and time information acquiring step; a time reference data acquiring step of acquiring time reference data in which information related to the position information that changes due to the position change occurrence step is associated with the time information acquired by the position and time information acquiring step; an identification data adding step of adding information to the time series data acquired by the time series data acquiring step, the information being capable of creating identification data in which information correlating with information related to the position information in the time reference data and information for identifying a plurality of data constituting the time series data are associated with each other; a data creation step of creating the identification data from the time-series data including information enabling the creation of the identification data; a corresponding time specifying step of specifying an estimated time based on the time information obtained by the position and time information obtaining step, the estimated time corresponding to the acquisition time of the plurality of data constituting the time series data, by using the time reference data obtained by the time reference data obtaining step and the identification data created by the identification data creating step; and acquiring time series data with time information, in which the estimated time identified by the corresponding time identification process is associated with each of a plurality of data constituting the time series data.
Citation Information
Patent Citations
Video image recording apparatus
JP2005051278A
