Information embedding method

Abstract

To provide an information embedding method that can read out desired information simply and easily.SOLUTION: A device 2 includes a built-in acceleration sensor 2b. An acoustic wave 3a of a specific frequency that resonates with the acceleration sensor 2b is applied to the acceleration sensor 2b. This allows an arbitrary bit string to be embedded in the acceleration data measured by the acceleration sensor 2b. This arbitrary bit string may be information that can be recognized as behavior estimation information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information embedding method. [Background technology]

[0002] In recent years, acceleration sensors, a type of inertial sensor, have been adopted in a variety of products, including portable digital electronic devices such as smartphones and laptops, as well as automobiles, robots, construction machinery, and medical equipment. These acceleration sensors measure vibration and gravity to detect the movement and orientation of equipment and people, and are used for control, making them an important component that supports daily life and various industries.

[0003] Incidentally, it is known that inputting acoustic waves into such acceleration sensors can produce an output corresponding to the acoustic waves in addition to an acceleration signal. Utilizing this knowledge, technology has been announced that attacks acceleration sensors with acoustic waves (see, for example, Non-Patent Document 1), and technology that records slight vibrations from a smartphone's ear speaker using a built-in acceleration sensor and infers voice from those vibrations (see, for example, Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] T. Trippel, O. Weisse, W. Xu, P. Honeyman, and K. Fu “WALNUT: Waging doubt on the integrity of MEMS accelerometers with acoustic injection attacks,” IEEE European symposium on security and privacy (EuroS&P 2017), pp. 3-18 (2017). [Non-patent document 2] Ahmed Tanvir Mahdad, Cong Shi, Zhengkun Ye, Tianming Zhao, Yan Wang, Yingying Chen, Nitesh Saxena, “EarSpy: Spying Caller Speech and Identity through Tiny Vibrations of Smartphone Ear Speakers”, Fri, 23 Dec 2022, https: / / doi.org / 10.48550 / arXiv.2212.12151 Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, it is conceivable to apply such technology to add metadata such as event and location information to acceleration sensor signals.

[0006] However, currently, metadata such as text must be prepared separately from the acceleration sensor output signal, and annotation must be performed while comparing it with the acceleration sensor signal. Therefore, when there are multiple acceleration sensors, it is not only very difficult to associate them with the annotation file, but there is also the possibility of incorrect association. Therefore, currently, there is a problem that it is difficult to read out the desired information.

[0007] In view of the above problems, an object of the present invention is to provide an information embedding method that allows desired information to be read out simply and easily. [Means for solving the problem]

[0008] The above object of the present invention can be achieved by the following means: Note that the parentheses indicate reference symbols of embodiments to be described later, but the present invention is not limited to these.

[0009] The information embedding method of claim 1 is characterized in that it applies an acoustic wave (e.g., acoustic wave 3a shown in FIG. 1) of a predetermined frequency to which an inertial sensor (e.g., acceleration sensor 2b shown in FIG. 1) built into a housing (e.g., device 2 shown in FIG. 1) responds, thereby embedding an arbitrary bit string into measurement data measured by the inertial sensor.

[0010] The information embedding method according to claim 2 is the information embedding method according to claim 1, characterized in that the predetermined frequency is a specific frequency at which the inertial sensor (for example, the acceleration sensor 2b shown in Figure 1) resonates.

[0011] The information embedding method according to claim 3 is the information embedding method according to claim 1, characterized in that the arbitrary bit string is information that can be recognized as behavior estimation information.

[0012] The information embedding method according to claim 4 is characterized in that, in the information embedding method according to claim 2, when the inertial sensor (for example, the acceleration sensor 2b shown in FIG. 1) has multiple axes (for example, the X-axis, Y-axis, and Z-axis shown in FIG. 3) and the specific frequencies that resonate with each axis are different, the specific frequency of the applied acoustic wave (for example, the acoustic wave 3a shown in FIG. 1) is a composite wave of the different specific frequencies.

[0013] The information embedding method according to claim 5 is characterized in that, in the information embedding method according to claim 2, after applying an acoustic wave of a specific frequency (e.g., acoustic wave 3a shown in Figure 1) that resonates with the inertial sensor (e.g., acceleration sensor 2b shown in Figure 1), the application of the next acoustic wave of the specific frequency is stopped until the reverberation caused by the resonance of the inertial sensor disappears.

[0014] The information embedding method according to claim 6 is characterized in that, in the information embedding method according to claim 2, an acoustic wave of a specific frequency (e.g., acoustic wave 3a shown in FIG. 1) that resonates with the inertial sensor (e.g., acceleration sensor 2b shown in FIG. 1) is applied, and then an acoustic wave of the opposite phase to the applied acoustic wave of the specific frequency is applied to the inertial sensor. [Effects of the Invention]

[0015] Next, the effects of the present invention will be described with reference to the drawings. Note that the reference symbols in parentheses are those of the embodiments described below, but the present invention is not limited to these.

[0016] According to the invention of claim 1, an arbitrary bit string is embedded in the measurement data measured by an inertial sensor (for example, acceleration sensor 2b shown in Figure 1), so that desired information can be read out simply and easily.

[0017] According to the invention of claim 2, the predetermined frequency is a specific frequency at which an inertial sensor (for example, acceleration sensor 2b shown in FIG. 1) resonates, making it easy to embed any bit string.

[0018] According to the invention of claim 3, by making the information recognizable as behavior estimation information as an arbitrary bit string, it becomes possible to easily estimate the behavior of a person or robot carrying a device (e.g., device 2 shown in Figure 1).

[0019] According to the invention of claim 4, at least one of the axes resonates, making it possible to embed any bit string.

[0020] According to the invention of claim 5, it is possible to improve the accuracy when decoding an arbitrary bit string.

[0021] According to the invention of claim 6, not only is it not necessary to stop applying the next acoustic wave of a specific frequency (for example, acoustic wave 3a shown in Figure 1), but the accuracy when decoding an arbitrary bit string can be improved. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a schematic overall view showing an embodiment of an information embedding system according to the present invention; [Figure 2] FIG. 2 is a block diagram of a device according to the embodiment. [Figure 3] FIG. 10 is a diagram showing a waveform when an arbitrary bit string is embedded in acceleration data. DETAILED DESCRIPTION OF THE INVENTION

[0023] An embodiment of an information embedding system for an information embedding method according to the present invention will be described below with reference to the drawings. In the following description, when directions such as up, down, left, and right are indicated, they refer to the up, down, left, and right directions when viewed from the front of the illustration.

[0024] <Outline of the information embedding system> The information embedding system according to this embodiment can read out desired information simply and easily. Specifically, as shown in Fig. 1, the information embedding system 1 is mainly composed of a portable device 2 and a speaker 3. Each component will be described in detail below.

[0025] <Device description> Device 2 is a smartphone, smartwatch, etc. (Figure 1 shows an example of a smartphone) that can be carried by a person or robot, and as shown in Figure 2, it is composed of a CPU 2a that executes and controls each function of device 2, an acceleration sensor 2b that is composed of a capacitance type, a piezo-resistance type, etc., an input / output unit 2c that can input predetermined data into device 2 or output predetermined data to the outside of device 2, a ROM 2d that is composed of a writable flash ROM or the like that stores predetermined programs, etc., a RAM 2e that functions as a working area, buffer memory, etc., and a display unit 2f that is composed of an LCD (Liquid Crystal Display), etc.

[0026] <Speaker description> The speaker 3 is installed on a wall, ceiling, or television, and can emit sound to the device 2, as shown in FIG. 1. Specifically, as shown in FIG. 1, the speaker 3 outputs an acoustic wave 3a to the acceleration sensor 2b built into the device 2, thereby applying the acoustic wave 3a to the acceleration sensor 2b. This acoustic wave 3a has a specific frequency with which the acceleration sensor 2b resonates. To explain this in more detail, for example, if the acceleration sensor 2b is a capacitive type, the specific frequency is a frequency near the resonant frequency of the movable electrode of the acceleration sensor 2b, and this frequency is output from the speaker 3 as the acoustic wave 3a. As a result, the acoustic wave 3a output from the speaker 3 is applied to the acceleration sensor 2b, causing the movable electrode to vibrate, i.e., resonate. This resonance causes fluctuations in the amplitude of the acceleration data measured by the acceleration sensor 2b, as shown in FIG. 3.

[0027] Therefore, in this embodiment, an acoustic wave 3a having a specific frequency with which the acceleration sensor 2b resonates is instantaneously and continuously output from the speaker 3 so as to embed an arbitrary bit string (for example, "0101") in the acceleration data. As a result, the acoustic wave 3a is instantaneously and continuously applied to the acceleration sensor 2b, and as shown in FIG. 3, a pulse-like waveform consisting of a substantially rectangular wave is applied to the acceleration data. In other words, an arbitrary bit string is embedded in the acceleration data.

[0028] Thus, if this arbitrary bit string is used as annotation information, that is, information that can be used to estimate the behavior of a person, robot, or the like carrying device 2, then the behavior of the person, robot, or the like carrying device 2 can be estimated simply by reading the acceleration data of acceleration sensor 2b from input / output unit 2c shown in Fig. 2. That is, in this embodiment, since the arbitrary bit string is embedded in the acceleration data itself, reading the acceleration data of acceleration sensor 2b from input / output unit 2c shown in Fig. 2 can read the arbitrary bit string simultaneously with the acceleration data. This makes it possible to extract and analyze only the arbitrary bit string from acceleration data in which the arbitrary bit string is embedded, and furthermore, by removing the embedded arbitrary bit string, the data can be used as the original acceleration data.

[0029] Thus, this eliminates the need for annotation while checking against the acceleration sensor signal, as in the conventional method, and also eliminates the risk of mismatching when multiple acceleration sensors are used. Therefore, in this embodiment, desired information can be easily read. Furthermore, since separate recording is not required as in the conventional method, the data size can be reduced compared to the conventional method. Note that the specific frequency of the acoustic wave 3a in this embodiment is equal to or higher than the cutoff frequency of the low-pass filter built into the acceleration sensor 2b. This allows the acceleration data and any bit string data generated by the acoustic wave 3a to be reliably separated. Therefore, deterioration in the accuracy of the acceleration data can be suppressed.

[0030] Incidentally, the information that can be used to estimate the behavior of a person or robot carrying device 2 can have a correspondence relationship determined in advance, such as "0001" indicating that the person was in room A and "0011" indicating that the person was in theme park B, or any bit string can be decoded to become a character code.

[0031] The acoustic wave 3a may be output by, for example, a person or a robot pressing a button, or may be output at regular intervals. [Example]

[0032] Here, the inventors conducted the following experiment to prove that an arbitrary bit string can be embedded in the acceleration data itself.

[0033] The acceleration sensor 2b was an ATR-Promotions multi-function sensor TSND151 (with an MPU-9250 acceleration sensor built in). An acoustic wave 3a with a frequency of 5200 Hz and a sound pressure of approximately 125 dB was output from a speaker 3 to the acceleration sensor 2b. The acoustic data for the acoustic wave 3a consisted of a 6-bit recognition header [101010] and a 10-bit random number embedded in the acceleration data, creating a total of 16 bits. When expressing 1 bit, a "1" represented 10 ms of sound and 10 ms of silence, while a "0" represented 20 ms of silence. The acoustic wave 3a was output, starting with the most significant bit, along with the 6-bit recognition header and the 10-bit random number embedded in the acceleration data. When expressing 1 bit, 10 ms of silence is included for "1" because in a preliminary experiment, when an acoustic wave 3a was output from the speaker 3 to the acceleration sensor 2b, the effect of the acoustic wave 3a subsided after about 10 ms had passed.

[0034] In this way, acoustic wave 3a was output from speaker 3 to acceleration sensor 2b, resulting in acceleration data such as that shown in Figure 3. To explain this in more detail, the acceleration sensor MPU-9250 is a capacitance type with three axes: X, Y, and Z. In this experiment, as shown in Figure 3, it was confirmed that acoustic wave 3a was applied to the Y-axis movable electrode, causing the Y-axis movable electrode to vibrate, or resonate, resulting in an acceleration change of -1G or more.

[0035] Therefore, when extracting the embedded bit string from acceleration data such as that shown in Figure 3, if there was an acceleration change of -1G or more in 10 ms, it was determined that there was an influence of the acoustic wave 3a. Then, based on this determination criterion, the acceleration data was scanned in chronological order, and when the recognition header pattern was confirmed, a 10-bit pattern was extracted from the 200 ms immediately following the recognition header.

[0036] Thus, when this extracted 10-bit pattern was checked against the 10-bit random number created as sound data, it was confirmed that they matched.

[0037] Therefore, the above experimental results prove that it is possible to embed any bit string into the acceleration data itself.

[0038] Thus, according to the present embodiment described above, desired information can be read out simply and easily.

[0039] Incidentally, when the acceleration sensor 2b has multiple axes (e.g., X-axis, Y-axis, and Z-axis) as shown in the embodiment, the specific resonant frequency may differ for each of the multiple axes. Therefore, in this case, the acoustic wave 3a may be a composite wave of different specific frequencies. In this way, at least one of the axes will resonate, making it possible to embed any bit string. Alternatively, the acoustic wave 3a may be output to the acceleration sensor 2b in a time-division manner, for example, at 3000 Hz to 5000 Hz. In this way, at least one of the axes will resonate, making it possible to embed any bit string. However, using a composite wave has the advantage of simplifying the work, since there is no need to output the acoustic wave 3a unnecessarily.

[0040] Furthermore, if the acceleration sensor 2b has multiple axes (e.g., X-axis, Y-axis, and Z-axis), the positional relationship between the acceleration sensor 2b and the speaker 3 can also be estimated from the bit string embedded in the acceleration data. For example, as shown in Fig. 3, if there is fluttering in the amplitude of the Y-axis, it can be estimated that the Y-axis direction of the acceleration sensor 2b is pointing toward the speaker 3. Furthermore, if the amplitude fluttering is greater than a predetermined threshold, the acceleration sensor 2b is located closer to the speaker 3 than a predetermined reference position, and if the amplitude fluttering is smaller than the predetermined threshold, the acceleration sensor 2b is located farther from the speaker 3 than the predetermined reference position. This allows for an approximate estimation of the positional relationship.

[0041] On the other hand, as described in the embodiment, when the acceleration sensor 2b resonates with the acoustic wave 3a, the resonance of the acceleration sensor 2b continues for a while, i.e., there is reverberation. Therefore, it is preferable to wait for the output of the acoustic wave 3a until the reverberation disappears, i.e., to stop applying the acoustic wave 3a. That is, when embedding an arbitrary bit string, for example, "0101," in acceleration data, if the acoustic wave 3a of the arbitrary bit string is output to the acceleration sensor 2b without waiting for the output of the acoustic wave 3a, the acceleration sensor 2b will continue to resonate even for the "0" part. Therefore, when the embedded bit string is extracted and decoded, it will not be decoded into an accurate bit string, which will reduce the decoding accuracy. Therefore, to improve the decoding accuracy, when the acceleration sensor 2b resonates with the acoustic wave 3a, the resonance of the acceleration sensor 2b continues for a while, i.e., there is reverberation. Therefore, it is preferable to wait for the output of the acoustic wave 3a until the reverberation disappears, i.e., to stop applying the acoustic wave 3a.

[0042] On the other hand, to eliminate the waiting time for the output of the acoustic wave 3a, when embedding an arbitrary bit string, for example, by embedding "1" to resonate the acceleration sensor 2b, it is also possible to immediately output an acoustic wave 3a of the same intensity but opposite phase to the acceleration sensor 2b to cancel the reverberation of the acceleration sensor 2b. This not only eliminates the waiting time for the output of the acoustic wave 3a, but also improves the accuracy of decoding.

[0043] Furthermore, in this embodiment, since the acoustic wave 3a is output to the acceleration sensor 2b, the device 2 does not need an environment such as an application or a communication function (for example, Bluetooth (registered trademark), Wi-Fi (registered trademark), etc. Therefore, in this embodiment, regardless of the interface on the device 2 side, the acoustic wave 3a can be output unilaterally to embed an arbitrary bit string in the acceleration sensor 2b.

[0044] <Description of Modifications> Note that the shapes and the like shown in this embodiment are merely examples, and various modifications and alterations are possible within the scope of the gist of the present invention as set forth in the claims. For example, in this embodiment, an example has been shown in which an arbitrary bit string is used as information that can estimate the behavior of a person, a robot, or the like. However, the present invention is not limited to this, and the arbitrary bit string may be information that automatically starts or processes an application or camera of device 2. Alternatively, the arbitrary bit string may be information for grouping or timing multiple devices 2. For example, at an amusement park or theme park, multiple devices 2 may be grouped or timing-synchronized by simultaneously embedding an arbitrary bit string in multiple devices 2.

[0045] Furthermore, in this embodiment, an example has been shown in which the acceleration sensor 2b and the speaker 3 are provided separately, but this is not limiting, and the speaker 3 may be built into the device 2. In this case, the acoustic wave 3a has the strongest effect on the acceleration data when the wavefront thereof is perpendicular to the resonance direction of the acceleration sensor 2b (the direction of movement in the case of a movable electrode). Therefore, it is necessary to build the speaker 3 into the device 2 in consideration of the orientation of the resonance direction of the acceleration sensor 2b. In this way, even if the sound pressure of the acoustic wave 3a is small, a large S / N ratio can be achieved, and signal degradation due to noise can be suppressed.

[0046] Furthermore, in this embodiment, a portable device is exemplified as the device 2, but the device is not limited to this and may be a non-portable device.

[0047] Furthermore, in this embodiment, the acoustic wave 3a is exemplified as a specific frequency at which the acceleration sensor 2b resonates, but this is not limiting. As long as the acceleration sensor 2b responds, i.e., vibrates, the acoustic wave 3a does not have to be a specific frequency at which the acceleration sensor 2b resonates. However, it is preferable that the acoustic wave 3a be a specific frequency at which the acceleration sensor 2b resonates. This is because applying a specific resonating frequency to the acceleration sensor 2b maximizes the amplitude of the movable electrode, making it easier to embed a bit string.

[0048] Furthermore, in this embodiment, the acceleration sensor 2b has been described as an example, but the present invention is not limited to this and can also be applied to a gyro sensor. [Explanation of symbols]

[0049] 1. Information Embedding System 2 Device (Housing) 2b Acceleration sensor (inertial sensor) 3 speakers 3a acoustic waves

Claims

1. An information embedding method in which an acoustic wave of a predetermined frequency to which an inertial sensor built into a housing responds is applied to the inertial sensor, thereby embedding an arbitrary bit string into measurement data measured by the inertial sensor.

2. 2. The information embedding method according to claim 1, wherein the predetermined frequency is a specific frequency at which the inertial sensor resonates.

3. The information embedding method according to claim 1 , wherein the arbitrary bit string is information that can be recognized as behavior estimation information.

4. 3. The information embedding method according to claim 2, wherein when the inertial sensor has a plurality of axes and the specific frequencies that resonate with each axis are different, the specific frequency of the acoustic wave that is applied is a composite wave of the different specific frequencies.

5. 3. The information embedding method according to claim 2, wherein after applying an acoustic wave of a specific frequency at which the inertial sensor resonates, application of the next acoustic wave of the specific frequency is stopped until reverberation due to the resonance of the inertial sensor disappears.

6. 3. The information embedding method according to claim 2, wherein an acoustic wave of a specific frequency at which the inertial sensor resonates is applied, and then an acoustic wave of an opposite phase to the applied acoustic wave of the specific frequency is applied to the inertial sensor.

Images