Information processing method, information processing device, and information processing system

Abstract

An information processing method according to an embodiment of the present technology involves acquiring detection signals detected by a tracking system and a camera that are rigidly connected to each other and have sensors capable of detecting a prescribed event, and estimating, on the basis of a first detection signal detected by the camera and a second detection signal detected by the tracking system, an offset so that the first detection signal and the second detection signal can match. Accordingly, the offset can be estimated with high accuracy.

Description

Information processing method, information processing device, and information processing system

[0001] The present technology relates to an information processing method, an information processing device, and an information processing system that can be applied to synchronization of two devices, etc.

[0002] Patent Literature 1 describes a system including a sensor that detects the movement of a first mobile device, a synchronization module that performs time synchronization with a second mobile device in response to the movement of the first mobile device, and an imaging module that captures an image and timestamps the image based on the time synchronization. In this system, the time synchronization is performed in the first mobile device without the support of a network infrastructure (see, for example, paragraphs

[0007] to

[0015] , Figures 1 and 2 of the specification of Patent Literature 1).

[0003] Special Publication No. 2013-541891

[0004] In order to synchronize such two devices, there is a demand for an information processing method, an information processing device, and an information processing system that are capable of estimating the offset with high accuracy.

[0005] In view of the above circumstances, an object of the present technology is to provide an information processing method, an information processing device, and an information processing system that are capable of estimating an offset with high accuracy.

[0006] To achieve the above object, an information processing method according to one aspect of the present technology includes acquiring detection signals detected by a camera and a tracking system that are rigidly connected to each other and have sensors capable of detecting a predetermined event, and estimating an offset based on a first detection signal detected by the camera and a second detection signal detected by the tracking system so that the first detection signal and the second detection signal coincide with each other.

[0007] In this information processing method, the offset is estimated based on the detection signals of the rigidly connected camera and tracking system, making it possible to estimate the offset with high accuracy without using an external synchronization device.

[0008] The predetermined event may include rotational movement of the camera or the tracking system, in which case the sensor may detect angular velocity of the camera and the tracking system.

[0009] The information processing method may further include determining whether the angular velocity norm exceeds a predetermined threshold value.

[0010] In the process of estimating the offset, the offset may be estimated based on a difference between the first detection signal and the second detection signal.

[0011] In the process of estimating the offset, the offset may be estimated based on a time difference between the first detection signal and the second detection signal.

[0012] In the process of estimating the offset, the offset may be estimated so that a peak of the first detection signal coincides with a peak of the second detection signal.

[0013] The information processing method may further include presenting estimated information relating to the offset estimation to a user.

[0014] The estimation information may include at least one of determination information indicating whether the sensor has detected the specified event for estimating the offset, or an estimation result indicating whether the offset has been correctly estimated.

[0015] The process of presenting the estimated information to the user may include a first determination step of presenting whether or not a state in which the angular velocity is equal to or greater than a predetermined threshold has continued for a predetermined number of seconds.

[0016] In the process of presenting the estimated information to the user, a second judgment stage may be presented based on the judgment result of the first judgment stage, which presents whether the state in which the angular velocity is below the predetermined threshold has continued for more than a predetermined number of seconds.

[0017] In the process of presenting the estimated information to the user, the estimation result may be presented as a degree of coincidence indicating whether the peak of the first detection signal and the peak of the second detection signal coincide based on the estimated offset.

[0018] The estimation information may include selection information indicating whether or not to re-execute the process of estimating the offset.

[0019] The predetermined event may include at least one of sound, light, or temperature, in which case the sensor may be capable of detecting at least one of the sound, light, or temperature.

[0020] According to one aspect of the present technology, an information processing device includes an acquisition unit and an offset estimation unit. The acquisition unit acquires detection signals detected by a camera and a tracking system that are rigidly connected to each other and have a sensor capable of detecting a predetermined event. The offset estimation unit estimates an offset based on a first detection signal detected by the camera and a second detection signal detected by the tracking system so that the first detection signal and the second detection signal coincide with each other.

[0021] According to one aspect of the present technology, there is provided an information processing system including a camera, a tracking system, and an information processing device. The camera includes a sensor capable of detecting a predetermined event. The tracking system is rigidly connected to the camera and includes a sensor capable of detecting the predetermined event. The information processing device includes an acquisition unit that acquires detection signals detected by the camera and the tracking system, and an offset estimation unit that estimates an offset based on a first detection signal detected by the camera and a second detection signal detected by the tracking system so that the first detection signal and the second detection signal coincide with each other.

[0022] FIG. 1 is an explanatory diagram of an imaging system. FIG. 2 is an explanatory diagram of a background image according to the camera position of the imaging system. FIG. 3 is an explanatory diagram of a background image according to the camera position of the imaging system. FIG. 4 is a schematic diagram showing an information processing system. FIG. 5 is a block diagram showing an example of the configuration of an information processing device. FIG. 6 is a diagram showing a processing flow of offset estimation. FIG. 7 is a diagram showing a flowchart related to an application and its UI in offset estimation. FIG. 8 is a diagram showing a schematic UI of the imaging application.

[0023] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0024] [Photography System] Fig. 1 is a diagram schematically showing a photography system 500 to which the present technology can be applied. This photography system 500 is a system that performs photography as a virtual production, and Fig. 1 shows some of the equipment arranged in a photography studio.

[0025] Virtual production is a filming system that uses a large display device installed in a studio to display a background image on the display device, and then allows actors to perform in front of it, thereby filming the actors and the background.

[0026] As shown in Fig. 1, a film studio is provided with a performance area 501 where a performer 510 performs or performs other performances. Large display devices are placed at least on the back, left and right sides, and top of this performance area 501. The type of display device is not limited, but the figure shows an example in which an LED wall 505 is used as a large display device.

[0027] A single LED wall 505 is formed by arranging a plurality of LED panels 506 connected vertically and horizontally to form a large panel. The size of the LED wall 505 is not particularly limited, but it may be of a size necessary or sufficient to display the background when filming the performer 510.

[0028] A required number of lights 580 are placed at required positions, such as above or to the side of the performance area 501, to illuminate the performance area 501.

[0029] A camera 502 is placed near the performance area 501 for shooting, for example, movies or other video content. The camera 502 can be moved by a cameraman 512, and the camera's shooting direction, angle of view, etc. can also be controlled. Of course, it is also conceivable that the movement of the camera 502, the angle of view, etc. can be controlled by remote control. The camera 502 may also be designed to move or change the angle of view automatically or autonomously. For this reason, the camera 502 may be mounted on a platform or a moving body.

[0030] Some cameras 502 capture both the performer 510 in the performance area 501 and the image displayed on the LED wall 505. For example, by displaying a landscape as background image vB on the LED wall 505, it becomes possible to capture an image that is similar to the image of the performer 510 actually being in the location of that landscape and performing.

[0031] An output monitor 503 is placed near the performance area 501. The video being shot by the camera 502 is displayed in real time on this output monitor 503 as a monitor video vM, allowing the director and staff producing the video content to check the video being shot.

[0032] Here, the background image vB will be explained with reference to Figures 2 and 3. Even if the background image vB is displayed on the LED wall 505 and filming is performed together with the performer 510, simply displaying the background image vB will result in an unnatural background in the filmed video. This is because the background, which is actually three-dimensional and has depth, is displayed as a two-dimensional background image vB.

[0033] For example, camera 502 can capture images of performer 510 in performance area 501 from a variety of directions and can also zoom. Performer 510 does not stand still in one place. Therefore, the actual appearance of the background of performer 510 should change depending on the position, shooting direction, and angle of view of camera 502, but such changes cannot be obtained with background image vB as a two-dimensional image. Therefore, background image vB is changed so that the background appears similar to how it actually appears, including parallax.

[0034] Figure 2 shows a camera 502 photographing a performer 510 from a position on the left side of the figure, and Figure 3 shows a camera 502 photographing a performer 510 from a position on the right side of the figure. In each figure, a shooting area image vBC is shown within a background image vB.

[0035] The background image vB that does not include the shooting area image vBC is called the "outer frustum," and the shooting area image vBC is called the "inner frustum."

[0036] The range of this shooting area image vBC (inner frustum) corresponds to the range actually captured by the camera 502 within the display surface of the LED wall 505. The shooting area image vBC is an image that has been transformed to represent the scene that can be seen when the viewpoint is actually the position of the camera 502, depending on the position, shooting direction, angle of view, etc. of the camera 502.

[0037] Specifically, the shooting area video vBC is prepared by preparing a 3D model (3D background data) as the background, and sequentially rendering the 3D background data in real time based on the viewpoint position of the camera 502 .

[0038] In reality, the range of the shooting area image vBC is set to be slightly wider than the range currently being shot by the camera 502. This is to prevent the image of the outer frustum from being captured due to a drawing delay when the shooting range changes slightly due to panning, tilting, zooming, etc. of the camera 502, and to avoid the influence of diffracted light from the image of the outer frustum.

[0039] The image of the shooting area image vBC rendered in real time in this way is composited with the image of the outer frustum. The image of the outer frustum used in the background image vB is rendered in advance based on 3D background data, and the image of the shooting area image vBC rendered in real time is incorporated into part of the outer frustum image to generate the overall background image vB.

[0040] As a result, even if the camera 502 is moved back and forth or left and right, or a zoom operation is performed, the background of the range photographed together with the performer 510 will be photographed as an image corresponding to the change in viewpoint position accompanying the actual movement of the camera 502.

[0041] 2 and 3, the output monitor 503 displays a monitor image vM including a performer 510 and the background, which is the captured image. The background in this monitor image vM is the shooting area image vBC. In other words, the background included in the captured image is a real-time rendered image.

[0042] In this way, in the embodiment of the shooting system 500, the background image vB is not simply displayed in a two-dimensional manner, but the background image vB including the shooting area image vBC is changed in real time so that footage similar to that obtained when actually shooting on location can be shot.

[0043] In addition, by rendering in real time only the shooting area image vBC, which is the range captured by the camera 502, rather than the entire background image vB displayed on the LED wall 505, the processing load on the system is also reduced.

[0044] Here, the person controlling the camera 502 capturing the image of the performer 510 is not limited to the cameraman 512. For example, the camera 502 may automatically track the performer 510 using a tracking system. In this embodiment, the camera 502 is rigidly connected to the tracking system. In other words, when the tracking system tracks the performer 510, the attitude and shooting direction of the camera 502 are controlled in accordance with the tracking.

[0045] 4A and 4B are schematic diagrams showing an information processing system according to this embodiment, in which Fig. 4A is a schematic diagram showing the information processing system, and Fig. 4B is a schematic diagram showing synchronization by an information processing device.

[0046] As shown in FIG. 4A, the information processing system 100 includes a camera 502, a tracking system 1, and an information processing device 10.

[0047] In this embodiment, the camera 502 is a cinema camera, and is rigidly connected to the tracking system 1. For example, the camera 502 and the tracking system 1 are integrated by being mounted on a rig or the like.

[0048] The tracking system 1 is a system that can automatically track a specific subject (e.g., a performer 510) in a video. For example, the tracking system 1 can detect a human face, etc., and automatically pan, tilt, zoom, etc. to track the target person.

[0049] In this embodiment, the camera 502 and the tracking system 1 have sensors capable of detecting a predetermined event. The predetermined event includes a rotational movement of the camera 502 or the tracking system 1. For example, the predetermined event is an action involving various rotations such as panning, tilting, or rolling.

[0050] In this embodiment, the sensor included in the camera 502 is a sensor such as an IMU (Inertial Measurement Unit) that detects the rotational motion (change in angular velocity) of the camera 502. The sensor included in the tracking system 1 is a sensor that can acquire the 6 Dof (Degrees of Freedom) camera pose of the tracking system.

[0051] In addition to this, the predetermined event may include actions of the camera 502 and the tracking system 1, such as movement of the camera 502 or the tracking system 1 in the X, Y, or Z axis directions, and various external events, such as light, sound, or temperature, that are different from the actions of the camera 502 or the tracking system 1. Of course, the sensors of the camera 502 and the tracking system 1 are configured with sensors that can detect the above-mentioned predetermined events.

[0052] In this embodiment, the camera 502 outputs video and audio data and IMU data acquired from the sensor to the information processing device 10. The tracking system 1 also outputs tracking data including the 6Dof camera pose to the information processing device 10.

[0053] The information processing device 10 synchronizes the camera 502 and the tracking system 1. In this embodiment, the information processing device 10 synchronizes the camera 502 and the tracking system 1, which are rigidly connected, in software based on observed values ​​(angular velocity norms) corresponding to the respective movements of the two devices.

[0054] 4B , the information processing device 10 synchronizes with the camera 502 and tracking system 1, which are rigidly connected, when the camera 502 and tracking system 1 are turned (a predetermined event). In this embodiment, the information processing device 10 acquires a first detection signal 5 (an observation value of the IMU) indicating the angular velocity norm of the camera 502 and a second detection signal 6 (a 6Dof estimated value) indicating the angular velocity norm of the tracking system 1. The estimated value of the tracking system is compensated for by estimating the time difference between the first and second detection signals (see arrow 7). That is, based on the time difference between the first and second detection signals, a time offset is estimated so that the first and second detection signals coincide with each other.

[0055] Fig. 5 is a block diagram showing an example of the configuration of the information processing device 10. Fig. 6 is a diagram showing a processing flow of offset estimation.

[0056] As shown in FIG. 5 , the information processing device 10 includes an acquisition unit 11 , a calculation unit 12 , a sampling unit 13 , an offset estimation unit 14 , an interpolation unit 15 , a presentation unit 16 , and a determination unit 17 .

[0057] The acquisition unit 11 acquires IMU data of the camera 502 acquired by the sensor (IMU) of the camera 502. The acquisition unit 11 also acquires the 6Dof camera pose acquired by the sensor of the tracking system 1.

[0058] 6, the acquisition unit 11 acquires IMU data C1 from the camera 502 and a 6Dof camera pose T1 from the tracking system 1. The acquired 6Dof camera pose T1 is recorded in the DB 18 for interpolation processing.

[0059] The calculation unit 12 calculates an angular velocity norm C2 of the camera 502 based on the IMU data C1. The calculation unit 12 also calculates an angular velocity norm T2 of the tracking system based on the 6Dof camera pose T1.

[0060] For example, the calculation unit 12 calculates the angular velocity norm at time t_i using the attitude R_i at each time according to the following equation (Equation 1).

[0061] In this embodiment, the 6Dof camera pose T1, the IMU data C1, and the angular velocity norms T2 and C2 are each data with a time stamp.

[0062] Because the sampling timings of the calculated angular velocity norms T2 and C2 are different, the sampling unit 13 performs re-sampling to align the sampling timings of the two. For example, the sampling unit 13 aligns the sampling timings of the two angular velocity norms by linear interpolation. The sampled angular velocity norm C3 of the camera 502 and the angular velocity norm T3 of the tracking system are output to the offset estimation unit 14.

[0063] The offset estimator 14 estimates an offset for matching the angular velocity norm C3 and the angular velocity norm T3. In this embodiment, the offset estimator 14 estimates the time offset O1 so that the peaks of the detection signals indicating the angular velocity norms C3 and T3 match.

[0064] For example, the offset estimation unit 14 estimates the timestamp offset Δt using the following equation (Equation 2) for the norm sequences of both the camera 502 and the tracking system 1.

[0065] The interpolation unit 15 interpolates a camera pose at a timing synchronized with the frame of the camera 502 based on the estimated time offset O1, the 6Dof camera pose T1, and the camera frame timestamp C4 of the camera 502. The interpolation unit 15 outputs a synchronized 6Dof camera pose P1.

[0066] The presentation unit 16 presents estimation information related to the estimation of the time offset to the user. The estimation information includes at least one of determination information indicating whether a predetermined event for estimating the time offset has been detected and an estimation result indicating whether the time offset has been correctly estimated.

[0067] The determination unit 17 determines whether the estimated information is satisfied. In this embodiment, the determination unit 17 determines whether the amplitudes of a first detection signal when a sensor included in the camera 502 detects a predetermined event and a second detection signal when a sensor included in the tracking system 1 detects a predetermined event exceed a predetermined threshold. For example, if the predetermined event is a head shake (rotational movement), it is determined whether the amplitude (angular velocity norm) of the detection signal exceeds a predetermined threshold.

[0068] In this embodiment, the presentation unit 16 presents the estimated information to the user by displaying it on a screen. A specific example will be described later with reference to FIG. 8 . The screen on which the estimated information is presented may be provided by the information processing device 10 or the tracking system 1. Alternatively, the estimated information may be presented on the screen of a mobile terminal carried by the user by downloading an application to the mobile terminal.

[0069] Fig. 7 is a flowchart showing an application and its UI for offset estimation, and Fig. 8 is a diagram showing a typical UI of the application.

[0070] For example, the flowchart shown in Fig. 7 starts when a user launches an application for estimating an offset when capturing an image using the camera 502 or the tracking system 1. As shown in Fig. 7, the presentation unit 16 displays a plot screen A (step 101).

[0071] FIG. 8A is a schematic diagram showing the plot screen A.

[0072] 8A, plot screen A is a screen indicating that the angular velocity of camera 502 (tracking system) is not sufficiently large. That is, in step 101, determination information as to whether the change in angular velocity is equal to or greater than a predetermined threshold is displayed as estimated information.

[0073] Also displayed on plot screen A are a first detection signal 5 indicating the angular velocity (amplitude) of camera 502 and a second detection signal 6 indicating the angular velocity of tracking system 1. As shown in Fig. 8A, the amplitudes of the first detection signal 5 and the second detection signal 6 do not exceed a predetermined threshold (dotted line 7). Also, because the time offset has not yet been estimated, the peaks of the two detection signals do not match.

[0074] While checking plot screen A, the user changes the angular velocity of camera 502 by, for example, turning the camera's head so that it becomes sufficiently large. Furthermore, on plot screen A, because the predetermined threshold is not exceeded, comment 8 is displayed urging the user to "rotate the camera more widely" to satisfy the estimated information. In addition, on plot screen A, comment 8 or screen frame 9 may be displayed in red as information indicating that the estimated information is not satisfied.

[0075] The determination unit 17 determines whether the change in angular velocity of the camera 502 is sufficiently large (step 102). If the angular velocity of the camera 502 exceeds a predetermined threshold (YES in step 102), the presentation unit 16 displays the plot screen B (step 103).

[0076] FIG. 8B is a schematic diagram showing the plot screen B.

[0077] 8B, a plot screen B is displayed, indicating that the amplitudes of the first detection signal 5 and the second detection signal 6 exceed the predetermined threshold. Also, a comment 8 is displayed on the plot screen B, indicating that the angular velocity, which is the condition of step 102, exceeds the predetermined threshold.

[0078] 7, the determination unit 17 determines whether the state in which the amplitudes of the first detection signal 5 and the second detection signal 6 are equal to or less than a predetermined threshold continues for M seconds or more. That is, after transition from the state of plot screen A to plot screen B, if the state in which the amplitudes of the first detection signal 5 and the second detection signal 6 are below the predetermined threshold continues for M seconds or more (YES in step 104), the screen returns to plot screen A.

[0079] Furthermore, after transitioning to plot screen B, the judgment unit 17 judges whether or not the state in which the amplitudes of the first detection signal 5 and the second detection signal 6 exceed the predetermined threshold has continued for more than N seconds (step 105).

[0080] In addition, on the plot screen B, as information indicating that the judgment information is met, a comment 8 or a screen frame 9 may be displayed in green, or a check mark, a circle, OK, etc. may be displayed.

[0081] If the state in which the amplitudes of the first detection signal 5 and the second detection signal 6 exceed the predetermined threshold has continued for N seconds or more (YES in step 105), the offset estimation unit 14 estimates the time offset (step 106). Furthermore, the presentation unit 16 displays the plot screen C (step 107).

[0082] FIG. 8C is a schematic diagram showing the plot screen C.

[0083] As shown in Fig. 8C, the plot screen C displays a state in which the first detection signal 5 and the second detection signal 6 match due to the time offset. The plot screen C also displays the estimation results, mutual compatibility, and absolute error of the first detection signal 5 and the second detection signal 6. For example, if the first detection signal 5 and the second detection signal 6 match highly, "Excellent" is displayed. Conversely, if the first detection signal 5 and the second detection signal 6 are significantly different, "Bad" or the like is displayed.

[0084] Furthermore, on the plot screen C, a selection screen (yes or no) for whether or not to apply the estimated value of the time offset is displayed as comment 8. The user refers to the above estimation result, and selects "yes" if the estimated value is to be applied (YES in step 108). On the other hand, if the degree of match is low even after applying the time offset, the user selects "no" (NO in step 108).

[0085] Note that the setting of N seconds or the like may be set arbitrarily. For example, the IMU observation value of the camera 502 may be stored in the queue for an interval of ±n seconds (N=2n) based on the time of the tracking system 1. The value of n is a length (e.g., 2 to 3 seconds) that is long enough to fully encompass the actual time it takes from when the IMU observation value is recorded in the camera 502 until it is transmitted to the tracking system 1.

[0086] As described above, in the information processing method according to this embodiment, detection signals detected by the camera 502 and tracking system 1, which are rigidly connected to each other and have sensors capable of detecting a predetermined event, are acquired, and an offset is estimated based on the first detection signal 5 detected by the camera 502 and the second detection signal 6 detected by the tracking system 1 so that the first detection signal 5 and the second detection signal 6 coincide with each other. This makes it possible to estimate the offset with high accuracy.

[0087] Traditionally, in the field of video production, such as virtual production, synchronization of cameras, monitors, microphones, etc. is required. For example, if you want to synchronize a tracking system with a cinema camera, you need to input a Genlock signal into each device to synchronize the clock. However, since signal wires must be connected to the devices to be synchronized, which makes handling difficult, there is a desire to reduce the wiring in the camera rig as much as possible. Furthermore, with outside-in systems, while the number of wires in the camera rig does not increase, additional accessories for synchronization are required, making it necessary to build a system that supports external synchronization.

[0088] For example, in the past, a clapperboard would be used to record a synchronization sound at the start and end of filming, and then the video and audio would be synchronized using that sound during editing. In this case, the time is adjusted using a sample taken at the moment the clapperboard sounds.

[0089] In this technology, in order to support time synchronization such as clock synchronization, synchronization of time-stamping timing, and synchronization of event trigger timing without using the above-mentioned Genlock signal, two rigidly connected devices are each equipped with a sensor that outputs observation values ​​according to the device's movement, and these observation values ​​are used to synchronize the devices via software.

[0090] This allows for high-fps observations to be sampled over several seconds using sensors such as an IMU, enabling extremely high resolution and accuracy in offset estimation. Furthermore, because this technology uses the norm of angular velocity, it is not necessary to consider the relationship between the coordinate systems of the camera's IMU and the tracking system (the relative relationship between their mounting positions). Furthermore, because the cross-correlation peak is detected, the time offset can be accurately determined even if there is a slight deviation in the calibration of the angular velocity sensor or if the scale is off.

[0091] Furthermore, since the system only detects the fluctuation pattern of angular velocity, the user can estimate the offset simply by rotating the camera rig, thereby reducing the burden on the user.

[0092] Other Embodiments The present technology is not limited to the above-described embodiments, and various other embodiments can be realized.

[0093] In the above embodiment, angular velocity is used to estimate the offset, but this is not limiting. For example, velocity may be used, but in this case, the IMU observation value cannot be used directly and the accelerometer observation value must be integrated, which makes the process more complicated.

[0094] In the above embodiment, the camera 502 is equipped with an IMU, but this is not limited to this. For example, if the tracking system 1 is equipped with an IMU and gyro observation is available, the IMU observation value may be used instead of the 6Dof camera pose.

[0095] In the above embodiment, the calculation of the angular velocity and the calculation of the offset are performed by the information processing device 10, but this is not limiting, and these calculations may be performed by the tracking system 1, or part of these calculations may be performed by the information processing device 10 and other parts may be performed by the tracking system 1. For example, the estimation of the time offset may be performed by the tracking system 1, and the resampling may be performed by the information processing device 10.

[0096] In the above embodiment, the information processing system includes the information processing device 10, the camera 502, and the tracking system 1. However, the information processing system is not limited to these, and may further include various other devices. For example, the information processing system may include a synchronization signal generator that outputs a Genlock signal for synchronizing the camera 502 with an LED panel or other external devices. Of course, as shown in FIG. 4 , the synchronization signal generator may not be used depending on the application. For example, if the purpose is to acquire a 6Dof camera pose synchronized with the video, the synchronization signal generator may not be required.

[0097] The configurations of the calculation unit, offset estimation unit, presentation unit, determination unit, etc. described with reference to the drawings are merely one embodiment and can be arbitrarily modified without departing from the spirit of the present technology. In other words, any other configurations, algorithms, etc. for implementing the present technology may be adopted.

[0098] It should be noted that the effects described in this disclosure are merely examples and are not limiting, and other effects may also be present. The description of multiple effects above does not necessarily mean that these effects are exhibited simultaneously. It means that at least one of the effects described above can be obtained depending on the conditions, etc., and of course, effects not described in this disclosure may also be exhibited.

[0099] It is also possible to combine at least two of the characteristic features of each embodiment described above. In other words, the various characteristic features described in each embodiment may be combined in any manner without distinguishing between the embodiments.

[0100] Note that the present technology can also adopt the following configurations. (1) An information processing method comprising: acquiring detection signals detected by a camera and a tracking system that are rigidly connected to each other and have a sensor capable of detecting a predetermined event; and estimating an offset based on a first detection signal detected by the camera and a second detection signal detected by the tracking system so that the first detection signal and the second detection signal match. (2) The information processing method according to (1), in which the predetermined event includes rotational motion of the camera or the tracking system, and the sensor detects angular velocities of the camera and the tracking system. (3) The information processing method according to (1), in which the offset estimation process estimates the offset based on a difference between the first detection signal and the second detection signal. (4) The information processing method according to (3), in which the offset estimation process estimates the offset based on a time difference between the first detection signal and the second detection signal. (5) The information processing method according to (4), wherein the process of estimating the offset estimates the peak of the first detection signal to coincide with the peak of the second detection signal. (6) The information processing method according to (2), further comprising presenting estimation information regarding the offset estimation to a user. (7) The information processing method according to (6), wherein the estimation information includes at least one of determination information indicating whether the sensor has detected the predetermined event for estimating the offset or an estimation result indicating whether the offset has been estimated correctly. (8) The information processing method according to (7), wherein the process of presenting the estimation information to a user includes presenting a first determination step indicating whether a state in which the angular velocity is equal to or greater than a predetermined threshold has continued for a predetermined number of seconds. (9) An information processing method according to (8), wherein the process of presenting the estimated information to the user includes a second determination step of presenting, based on the determination result of the first determination step, whether or not the state in which the angular velocity is equal to or less than the predetermined threshold value has continued for a predetermined number of seconds or more.(10) The information processing method according to (7), wherein the process of presenting the estimated information to a user presents, as the estimation result, a degree of coincidence indicating whether or not peaks of the first detection signal and peaks of the second detection signal coincide due to the estimated offset. (11) The information processing method according to (6), wherein the estimation information includes selection information indicating whether or not to execute the process of estimating the offset again. (12) The information processing method according to (1), wherein the predetermined event includes at least one of sound, light, or temperature, and the sensor is capable of detecting at least one of the sound, the light, or the temperature. (13) An information processing device comprising: an acquisition unit that acquires detection signals detected by a camera and a tracking system that are rigidly connected to each other and have a sensor that can detect a predetermined event; and an offset estimation unit that estimates an offset based on a first detection signal detected by the camera and a second detection signal detected by a tracking system so that the first detection signal and the second detection signal coincide. (14) An information processing system having an information processing device including: a camera having a sensor capable of detecting a predetermined event; a tracking system rigidly connected to the camera and having a sensor capable of detecting the predetermined event; an acquisition unit that acquires detection signals detected by the camera and the tracking system; and an offset estimation unit that estimates an offset based on a first detection signal detected by the camera and a second detection signal detected by the tracking system so that the first detection signal and the second detection signal match.

[0101] REFERENCE SIGNS LIST 1 Tracking system 10 Information processing device 12 Calculation unit 14 Offset estimation unit 16 Presentation unit 17 Determination unit 502 Camera

Claims

1. An information processing method comprising: acquiring detection signals detected by a camera and a tracking system that are rigidly connected to each other and have sensors capable of detecting a predetermined event; and estimating an offset based on a first detection signal detected by the camera and a second detection signal detected by the tracking system so that the first detection signal and the second detection signal coincide with each other.

2. An information processing method according to claim 1, wherein the predetermined event includes rotational movement of the camera or the tracking system, and the sensor detects angular velocities of the camera and the tracking system.

3. An information processing method according to claim 1, wherein the offset estimation process estimates the offset based on the difference between the first detection signal and the second detection signal.

4. An information processing method according to claim 3, wherein the offset estimation process estimates the offset based on the time difference between the first detection signal and the second detection signal.

5. An information processing method according to claim 4, wherein the offset estimation process estimates the peak of the first detection signal to coincide with the peak of the second detection signal.

6. An information processing method according to claim 2, further comprising presenting estimated information relating to the offset estimation to a user.

7. An information processing method according to claim 6, wherein the estimation information includes at least one of determination information indicating whether or not the sensor has detected the predetermined event for estimating the offset, and an estimation result indicating whether or not the offset has been correctly estimated.

8. An information processing method according to claim 7, wherein the process of presenting the estimated information to the user includes a first determination step of presenting whether the state in which the angular velocity is equal to or greater than a predetermined threshold has continued for a predetermined number of seconds.

9. An information processing method according to claim 8, wherein the process of presenting the estimated information to the user includes a second determination step of presenting, based on the determination result of the first determination step, whether the state in which the angular velocity is equal to or less than the predetermined threshold value has continued for a predetermined number of seconds or more.

10. An information processing method according to claim 7, wherein the process of presenting the estimated information to a user presents, as the estimation result, a degree of coincidence indicating whether or not the peak of the first detection signal and the peak of the second detection signal coincide due to the estimated offset.

11. An information processing method according to claim 6, wherein the estimation information includes selection information indicating whether or not to execute the process of estimating the offset again.

12. An information processing method according to claim 1, wherein the predetermined event includes at least one of sound, light, or temperature, and the sensor is capable of detecting at least one of the sound, light, or temperature.

13. An information processing device comprising: an acquisition unit that acquires detection signals detected by a camera and a tracking system that are rigidly connected to each other and have sensors capable of detecting a predetermined event; and an offset estimation unit that estimates an offset based on a first detection signal detected by the camera and a second detection signal detected by the tracking system so that the first detection signal and the second detection signal match.

14. An information processing system comprising: a camera having a sensor capable of detecting a predetermined event; a tracking system rigidly connected to the camera and having a sensor capable of detecting the predetermined event; an acquisition unit that acquires detection signals detected by the camera and the tracking system; and an offset estimation unit that estimates an offset based on a first detection signal detected by the camera and a second detection signal detected by the tracking system so that the first detection signal and the second detection signal match.

Images