Receiving terminal and method

Abstract

The present technology relates to a receiving terminal and method capable of realizing stable audio transmission regardless of the device. The receiving terminal: receives a packet sent from a transmitting terminal, the packet includes main data and redundant data, the redundant data is generated based on redundancy, the redundancy is determined based on a retransmission timeout period, the retransmission timeout period is a time determined by a combination of the transmitting terminal and the receiving terminal and allows the retransmission processing of the main data to be repeatedly performed, and the redundant data is used for the recovery or interpolation of the main data; and decodes the reproduced data by decoding the main data or the redundant data. The present technology can be applied to a communication system that performs voice transmission through Bluetooth Low Energy (BLE).

Description

Technical Field

[0001] The present technology relates to a receiving terminal and method, and more particularly, to a receiving terminal and method capable of realizing stable voice transmission independent of a device. Background Art

[0002] In voice transmission over Bluetooth Low Energy (hereinafter, BLE) under standardization, packets are transmitted at regular intervals called ISO intervals. Packets that are not successfully transmitted during this period are retransmitted at the next ISO interval in response to a request from a receiving terminal. The time during which the retransmission process can be repeated is called a retransmission timeout period.

[0003] Since packets that are not successfully transmitted within the retransmission timeout period are discarded, it is necessary to perform packet loss concealment for recovering or interpolating data in the discarded packets (lost packets). However, since the retransmission timeout period is determined for each combination of a transmitting terminal and a receiving terminal, the necessary quality (concealment quality) of packet loss concealment varies depending on the combination of devices even under the same radio wave environment.

[0004] In view of the above situation, Patent Document 1 proposes a method of measuring line quality information in a receiving terminal, transmitting data to a transmitting terminal, and then changing a calculation method of a forward error correction (FEC) code on the transmitting side.

[0005] Furthermore, Patent Document 2 proposes a method of controlling the number of frames of encoded data to be transmitted by observing the amount of data waiting to be transmitted in a buffer without measuring line quality information.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: International Patent Application Publication No. 2015 / 072005

[0009] Patent Document 2: Japanese Patent Application Publication No. 2016-208215 Summary of the invention

[0010] Problems to be solved by the present invention

[0011] However, in the case of using the method described in Patent Document 1 in communication using BLE, it is necessary to provide a packet loss rate transmission function separately on the receiving side, and it is difficult and unrealistic to have an indefinite number of vendors implement such a function in actual operation.

[0012] In the method described in Patent Document 2, since control is performed based on the buffer amount waiting to be transmitted, it is difficult to appropriately control, for example, the number of frames of encoded data transmitted immediately after starting transmission.

[0013] Therefore, a certain period of time is required before appropriate control is performed, and depending on the setting of the initial value, the concealment quality may become insufficient and auditory noise or discomfort may be perceived.

[0014] The present technology is made in view of such circumstances, and its purpose is to achieve stable voice transmission regardless of the device.

[0015] Solution to the problem

[0016] According to one aspect of the present technology, a receiving terminal includes: a receiving unit, which receives a packet sent from a sending terminal, the packet including main data and redundant data, the redundant data being generated based on redundancy and used for recovery or interpolation of the main data, the redundancy being determined based on a retransmission timeout period, the retransmission timeout period being a time determined by a combination of the sending terminal and the receiving terminal, and the retransmission processing of the main data can be repeatedly performed during the retransmission timeout period; and a decoding unit, which decodes and reproduces the data by decoding the main data or the redundant data.

[0017] In one aspect of the present technology, a packet transmitted from a transmitting terminal is received, the packet including main data and redundant data, the redundant data is generated based on redundancy and used for recovery or interpolation of the main data, the redundancy is determined based on a retransmission timeout period, the retransmission timeout period is a time determined by a combination of the transmitting terminal and the receiving terminal, and a retransmission process of the main data can be repeatedly performed during the retransmission timeout period. Then, the main data or the redundant data is decoded to reproduce the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a diagram showing an overview of packet loss concealment.

[0019] Figure 2 is a diagram showing the relationship between the number of pieces of redundant data and a possible hiding range.

[0020] Figure 3 is a diagram showing the relationship between the number of pieces of redundant data and a possible hiding range.

[0021] Figure 4 is a diagram illustrating a method of determining the number of pieces of redundant data according to the present technology.

[0022] Figure 5 : is a diagram showing a configuration example of an audio transmission system according to an embodiment of the present technology.

[0023] Figure 6 is a block diagram showing a configuration example of a transmitting terminal.

[0024] Figure 7is a flowchart for describing the system setting process of the transmitting terminal.

[0025] Figure 8 is a diagram showing a specific example of calculation of the target data amount.

[0026] Fig. 9 It is a flowchart for describing the transmission packet generation processing of the transmission terminal.

[0027] Fig.10 Is used to describe Fig. 9 Flowchart of the redundant data generation process in step S34.

[0028] Fig.11 Is used to describe Fig.10 An illustration of the process of step S60 in FIG.

[0029] Fig.12 is a block diagram showing a configuration example of a receiving terminal.

[0030] Fig.13 is a flowchart for describing the data acquisition process of the receiving terminal.

[0031] Fig.14 is a diagram for describing a method of adding data numbers.

[0032] Fig.15 is a flowchart for describing the audio reproduction process of the receiving terminal.

[0033] Fig.16 Is used to describe Fig.15 Flowchart of the reproduction data selection process in step S101.

[0034] Fig.17 is a block diagram showing another configuration example of the transmitting terminal.

[0035] Fig.18 is a flowchart for describing the system setting process of the transmitting terminal.

[0036] Fig.19 is a diagram showing a specific example of calculation of the target data amount.

[0037] Fig. 20 is a flowchart for describing the setting update processing of the transmitting terminal.

[0038] Fig.21 It is a flowchart for describing the transmission packet generation processing of the transmission terminal.

[0039] Fig. 22 is a diagram showing the configuration of frames and packets in the case where one or more pieces of main data exist.

[0040] Fig.23 Is used to describe Fig.21 Flowchart of the sending data number adding process in step S247.

[0041] Fig.24 : is a diagram showing a specific example of the transmission data number adding process.

[0042] Fig.25 is a flowchart for describing the data acquisition process of the receiving terminal.

[0043] Fig.26 is a diagram showing an example of a method of adding frame_id.

[0044] Fig. 27 : is a diagram showing a configuration example of an audio transmission system according to a third embodiment of the present technology.

[0045] Fig.28 is a block diagram showing a configuration example of a transmitting terminal.

[0046] Fig.29 is a block diagram showing a configuration example of a receiving terminal.

[0047] Fig.30 is a flowchart for describing the inter-device communication control processing of the receiving terminal.

[0048] Fig.31 is used to describe Fig.30 Flowchart of the request data determination process in step S307 of FIG.

[0049] Fig.32 is a block diagram showing a configuration example of a computer. DETAILED DESCRIPTION

[0050] Hereinafter, a mode for carrying out the present technology will be described. The description will be given in the following order.

[0051] 0. Overview of this technology

[0052] 1. First Embodiment (System Including Packet Loss Concealment)

[0053] 2. Second Embodiment (Combination of Dynamic Control of Data Amount According to Communication Status)

[0054] 3. Third Embodiment (Control in One-to-Many Communication)

[0055] 4. Others

[0056] <0. Overview of this technology>

[0057] <Term Description>

[0058] First, terms used in this specification will be described.

[0059] Among forward error correction schemes, "media-specific forward error correction (FEC)" is a scheme that holds a plurality of audio coded data having different encoding formats or settings as error correction data.

[0060] "Packet loss concealment" is a method of recovering or interpolating data in packets that are lost during transmission.

[0061] A "frame" is a processing unit standardized by the encoding scheme.

[0062] "Main data" is data (encoded data) of one frame that is usually transmitted in packets. "Redundant data" is data (encoded data) of one frame that is added to the main data alone for packet loss concealment or the like.

[0063] A "packet" is a transmission unit standardized in a transmission scheme, and indicates a data block transmitted in one communication. Specifically, in the present technology, a packet includes primary data and redundant data.

[0064] "Redundancy" is redundancy adjusted by the number and quality of redundant data to be added. In this specification, unless otherwise specified, "redundancy" means the number of redundant data added to one packet.

[0065] In BLE voice transmission, packets are sent at regular intervals called ISO intervals. Packets that were not successfully sent during this period are retransmitted at the next ISO interval in response to a request from the receiving terminal. Hereinafter, the time during which the retransmission process can be repeated is referred to as the retransmission timeout period.

[0066] Since the packets that are not successfully sent within the retransmission timeout period are discarded, lost packets are generated. Thus, when the retransmission timeout period is short, the packet loss frequency, which is the frequency of lost packets, increases. On the other hand, when the retransmission timeout period is long, the packet loss frequency decreases.

[0067] When a lost packet is generated, if reproduction is performed as it is, a discontinuous point is generated in the audio signal, and the discontinuous point is perceived as auditory discomfort. In order to prevent this, for example, packet loss concealment is performed.

[0068] <Overview of Packet Loss Concealment>

[0069] Figure 1 A diagram for explaining an overview of packet loss concealment using media-specific FEC (hereinafter referred to simply as packet loss concealment).

[0070] exist Figure 1In FIG. 1 , a rectangle shown between the transmitting terminal 1 and the receiving terminal 2 represents a packet transmitted from the transmitting terminal 1 to the receiving terminal 2 .

[0071] First, the transmitting terminal 1 adds the audio data of the frame before or after the main data as redundant data to the audio data as the main data, and sends the audio data. The redundant data is usually different from the main data in the encoding method or setting, and has a lower quality than the main data. For example, in the Nth packet, the redundant data of the N+1th frame is added to the main data of the Nth frame. In the previously transmitted packet of the N-1th frame, the redundant data of the Nth frame is added to the main data of the N-1th frame. Similarly, in the N-2th packet, the redundant data of the N-1th frame is added to the main data of the N-2th frame.

[0072] like Figure 1 As shown by the cross in , when the N-1th packet is discarded because the retransmission timeout period has exceeded, if the receiving terminal 2 reproduces only the main data as it is, a lost packet is generated due to the discarding of the N-1th packet, and a discontinuity occurs.

[0073] Therefore, when the main data of the N-1th frame included in the N-1th group is lost, the receiving terminal 2 performs interpolation using the redundant data of the N-1th frame included in the N-2th group as concealment. As a result, although the main data of the N-1th frame is not completely restored, the noise is reduced and it is less likely to perceive errors (auditory discomfort).

[0074] It should be noted that although Figure 1 An example in which audio data of a future frame is added as redundant data is shown, but the redundant data to be added may be audio data of a past frame.

[0075] <Relationship between the number of redundant data and possible hidden range>

[0076] Figure 2 and Figure 3 is a diagram showing the relationship between the number of redundant data and the possible hidden range. It should be noted that Figure 2 and Figure 3 An example of one frame / one packet is shown.

[0077] Figure 2 A grouping having grouping configuration A is shown, and Figure 3 A packet having a packet configuration A and a packet having a packet configuration B are shown. In the packet having the packet configuration A, one piece of redundant data (the number of pieces of redundant data: one) is added to one piece of primary data. In the packet having the packet configuration B, three pieces of redundant data (the number of pieces of redundant data: three) are added to one piece of primary data.

[0078] In addition, Figure 2 and Figure 3 , the packets before concealment in the receiving terminal 2 are shown on the left, and the packets after concealment in the receiving terminal 2 are shown on the right.

[0079] like Figure 2 As shown, in the case where the packet loss frequency is low, interpolation of the main data in the lost packets can be performed by using packets with packet configuration A.

[0080] On the other hand, in the case of high packet loss frequency, such as Figure 3 As shown by arrow a in , when packets having packet configuration A are used, the number of pieces of redundant data is insufficient relative to the number of lost packets, and interpolation of primary data in the lost packets becomes insufficient.

[0081] Meanwhile, when using a grouping with grouping configuration B, such as Figure 3 As shown by arrow b in , even if the packet loss frequency is high, each primary data in the lost packet can be interpolated with redundant data.

[0082] That is, by adding redundant data, the possible concealment range is widened.

[0083] However, specifically, in voice transmission using BLE, the size of data that can be stored in each packet is a fixed length. Therefore, by comparing Figure 3 It can be seen from the grouping configuration A and grouping configuration B in that in order to increase the number of redundant data and widen the possible hiding range, it is necessary to reduce the data size of the main data and sacrifice the quality.

[0084] In addition, since the retransmission timeout period is determined for each combination of transmitting and receiving terminals, even in the same radio wave environment, the necessary concealment quality varies depending on the combination of devices.

[0085] Therefore, as described above, using the fact that the retransmission timeout period is related to the packet loss frequency, in the present technology, the retransmission timeout period is obtained when the BLE is connected (when the transmission starts), and the number of redundant data (i.e., redundancy) is set according to the obtained retransmission timeout period. Therefore, an appropriate possible hiding range can be set according to the combination of devices.

[0086] <Method for determining the number of redundant data in this technology>

[0087] Figure 4 is a diagram illustrating a method of determining the number of pieces of redundant data according to the present technology.

[0088] exist Figure 4 In FIG. 1 , the retransmission timeout period becomes shorter from left to right. For such a retransmission timeout period, the number of redundant data is determined to increase from left to right.

[0089] That is, in the case of a long retransmission timeout period, the frequency of expected packet loss is low. Therefore, in the present technology, it is determined to reduce the number of redundant data and narrow the possible hiding range. As a result, less original main data to be transmitted is sacrificed.

[0090] On the other hand, when the retransmission timeout period is short, the frequency of packet loss is expected to be high. Therefore, in the present technology, it is determined to increase the number of redundant data and widen the possible concealment range. As a result, the interpolation of the main data in the lost packet is less likely to become insufficient.

[0091] The retransmission timeout period can be calculated according to a parameter determined by a combination of a transmitting terminal and a receiving terminal when connecting BLE communication. The number of redundant data can be calculated using a pre-prepared calculation formula or table about the relationship between the retransmission timeout period and the appropriate redundancy.

[0092] It should be noted that the purpose of this technology is to ensure high-quality concealment of packet losses that occur during communication at about a few percent. Therefore, situations where burst packet losses occur (such as situations where the communication state is poor and the amount of data required for voice transmission cannot be continuously ensured) are not affected by this technology.

[0093] <1. First Embodiment (System Including Packet Loss Concealment)>

[0094] <Configuration of audio transmission system>

[0095] Figure 5 : is a diagram showing a configuration example of an audio transmission system according to an embodiment of the present technology.

[0096] The audio transmission system is configured by connecting the transmitting terminal 11 and the receiving terminal 12 via BLE communication.

[0097] For example, the transmitting terminal 11 includes a multifunctional mobile phone such as a smart phone, a mobile terminal, etc. At the beginning of transmission, the transmitting terminal 11 calculates a retransmission timeout period and determines the redundancy of redundant data. The transmitting terminal 11 generates redundant data based on the determined redundancy, and encodes the main data and the redundant data including the audio data. The transmitting terminal 11 transmits the packet including the encoded main data and the redundant data to the receiving terminal 12.

[0098] The receiving terminal 12 includes, for example, headphones, a head-mounted device, etc. The receiving terminal 12 receives packets transmitted from the transmitting terminal 11, and temporarily accumulates the main data and the redundant data, respectively. In the case where the main data corresponding to the reproduction data to be reproduced exists, the receiving terminal 12 decodes and reproduces the main data. In the case where the main data corresponding to the reproduction data to be reproduced does not exist, the receiving terminal 12 decodes and reproduces the redundant data corresponding to the main data.

[0099] <Configuration example of sending terminal>

[0100] Figure 6 is a block diagram showing a configuration example of the transmission terminal 11.

[0101] It should be noted that the details of the layers of wireless transmission are omitted in the configuration example of the transmitting terminal 11. In addition, in this specification, it is assumed that one frame of main data and one or more frames of redundant data are stored and transmitted in one packet.

[0102] The transmitting terminal 11 includes a BLE controller 21 , a retransmission timeout period calculation unit 22 , a signal processing unit 23 , an input buffer 24 , a redundant data generation unit 25 , an encoding unit 26 , a multiplexing (MUX) unit 27 , and a transmission buffer 28 .

[0103] The BLE controller 21 controls operations related to BLE communication. After the connection via BLE communication is completed, the BLE controller 21 obtains parameters determined according to the combination of the transmitting and receiving terminals when connecting via BLE communication, and outputs the parameters to the retransmission timeout period calculation unit 22. Details of the parameters are omitted.

[0104] The retransmission timeout period calculation unit 22 calculates the retransmission timeout period based on the parameters supplied from the BLE controller 21. The retransmission timeout period calculation unit 22 outputs the retransmission timeout period to the redundant data generation unit 25.

[0105] The signal processing unit 23 obtains audio data required to generate one frame from the audio file, performs pre-processing for audio encoding, and accumulates the resulting data in the input buffer 24. The pre-processing for audio encoding is, for example, a time-frequency conversion process.

[0106] When the signal processing unit 23 accumulates data and the input buffer 24 is full, the input buffer 24 discards the data having the lowest priority (usually, discards the oldest data).

[0107] The redundant data generating unit 25 acquires original data corresponding to the main data and the redundant data from the input buffer 24. The original data corresponding to the main data is one frame of audio data necessary to generate the main data. The original data corresponding to the redundant data is one or more frames of audio data necessary to generate the redundant data.

[0108] In addition, the encoding unit 26 notifies the redundant data generating unit 25 of the encoded data amount of the redundant data.

[0109] The redundant data generating unit 25 determines redundancy based on the retransmission timeout period supplied from the retransmission timeout period calculating unit 22. The redundant data generating unit 25 generates main data and redundant data based on the encoded data amount notification supplied from the encoding unit 26 and the determined redundancy.

[0110] At this time, which of the previous packet and the subsequent packet (N-2, N-1, N+1, or N+2) will be restored by the redundant data of the Nth packet can be set by the data accumulated in the input buffer 24. For example, multiple future data can be accumulated in the input buffer 24, and the accumulated future data can be used to generate redundant data. In addition, multiple past data can be accumulated in the input buffer 24, and the accumulated past data can be used to generate redundant data. To this end, the redundant data generation method can be set according to the allowable delay amount or other conditions.

[0111] The redundant data generating unit 25 outputs the generated main data and redundant data to the encoding unit 26. In addition, the redundant data generating unit 25 determines encoding settings corresponding to the main data and redundant data output to the encoding unit 26, and notifies the determined encoding settings to the encoding unit 26. The encoding settings are determined by calculating the target data amount.

[0112] The encoding unit 26 encodes the main data and the redundant data based on the acquired encoding setting. The encoding unit 26 outputs the encoded main data and the redundant data to the multiplexing unit 27. In addition, the encoding unit 26 outputs the encoded data amount of the redundant data to the redundant data generating unit 25.

[0113] For example, in the case where three future packets are added as redundant data to the main data, in order to generate the Nth packet, the encoding unit 26 obtains the main data of the Nth frame, the redundant data generated from the main data of the N+1th frame to the N+3th frame, and the encoding settings of each data from the redundant data generation unit 25.

[0114] The multiplexing unit 27 performs multiplexing processing on the encoded main data and redundant data supplied from the encoding unit 26. That is, the multiplexing unit 27 stores the encoded main data and redundant data in packets suitable for BLE communication. The multiplexing unit 27 accumulates packets in the transmission buffer 28 for BLE communication.

[0115] <Operation of the sending terminal>

[0116] Hereinafter, the basic processing flow of the transmitting terminal 11 of the first embodiment will be described for the system setting process and the transmission packet generation process. The system setting process is a process that is performed only once when the BLE communication is connected. On the other hand, the transmission packet generation process is a process that is performed multiple times during voice transmission via BLE communication.

[0117] <System settings processing>

[0118] Figure 7 1 is a flowchart for describing the system setting process of the transmitting terminal 11.

[0119] In step S11 , the BLE controller 21 starts BLE communication by executing BLE connection processing. At this time, the BLE controller 21 acquires parameters related to the retransmission timeout period determined by combination with the receiving terminal 12 from the receiving terminal 12 and outputs the parameters to the retransmission timeout period calculation unit 22 .

[0120] In step S12 , the retransmission timeout period calculation unit 22 performs a retransmission timeout period calculation process based on the parameters supplied from the BLE controller 21 , and calculates the retransmission timeout period.

[0121] In step S13, the redundant data generating unit 25 determines the number of pieces of redundant data (redundancy) based on the retransmission timeout period provided from the retransmission timeout period calculating unit 22. The number of pieces of redundant data is determined by using a pre-prepared relational expression or table indicating the corresponding relationship between the retransmission timeout period and the number of pieces of redundant data.

[0122] In the case where the retransmission timeout period is long, the expected packet loss frequency is low. Therefore, in the present technology, the range of frames corresponding to redundant data is adjusted and determined so that the number of redundant data is reduced and the possible hiding range is narrowed. At this time, compared with the case where the retransmission timeout period is short, the proportion of redundant data in the packet is reduced and the proportion of primary data is increased.

[0123] In the case where the retransmission timeout period is short, the expected packet loss frequency is high. Therefore, in the present technology, the range of frames corresponding to redundant data is adjusted and determined so that the number of redundant data is increased and the possible hiding range is widened. At this time, compared with the case where the retransmission timeout period is long, the proportion of redundant data in the packet increases, and the proportion of primary data decreases.

[0124] In step S14, the redundant data generating unit 25 determines the target data volume of the main data based on the determined number of redundant data and the predetermined target data volume of the redundant data. In addition, in step S14, when the target data volume of the main data is determined, the encoding setting of the main data and the encoding setting of the redundant data are determined so as to fall within the target data volume.

[0125] Note that the default target data volume of each redundant data is predetermined. Thus, the target data volume of the primary data is set based on the target data volume of the redundant data, the number of redundant data and the data size of the packet.

[0126] After step S14, the system setting process ends.

[0127] <Calculation of target data volume>

[0128] Figure 8 is a diagram showing a specific example of calculation of the target data amount.

[0129] Figure 8 A grouping is shown. For example, Figure 8 As shown, when the target data amount of redundant data is 20 [bytes], the number of redundant data is 3, and the data size of the packet is 240 [bytes], the target data amount of the main data is 240-20×3=180 [bytes].

[0130] <Transmission Packet Generation Process>

[0131] Fig. 9 2 is a flowchart for describing a transmission packet generation process of the transmission terminal 11 .

[0132] In step S31 , the signal processing unit 23 acquires an audio file or audio data from the outside.

[0133] In step S32 , the signal processing unit 23 performs signal processing such as time-frequency conversion on the audio data.

[0134] In step S33 , the signal processing unit 23 accumulates data obtained as a result of the signal processing in the input buffer 24 .

[0135] In step S34, the redundant data generating unit 25 and the encoding unit 26 perform redundant data generating processing. Fig.10 The details of the redundant data generation process are described. Through the process of step S34, the generation of redundant data and the updating of the encoding setting of the main data are performed.

[0136] In step S35, the encoding unit 26 performs encoding processing on the main data based on the encoding setting of the main data. The encoding unit 26 outputs the encoded main data and redundant data to the multiplexing unit 27.

[0137] In step S36, the multiplexing unit 27 performs a multiplexing process. Specifically, the multiplexing unit 27 stores the encoded main data and redundant data supplied from the encoding unit 26 in a packet suitable for BLE communication.

[0138] The multiplexing unit 27 accumulates the packet in the transmission buffer 28 for BLE communication, and the transmission packet generation process ends. Thereafter, the packet is transmitted to the receiving terminal 12 by BLE communication.

[0139] <Redundant Data Generation Processing>

[0140] Fig.10 Is used to describe Fig. 9 Flowchart of the redundant data generation process in step S34.

[0141] In step S51 , the redundant data generating unit 25 acquires a piece of original data corresponding to redundant data from the input buffer 24 .

[0142] In step S52 , the redundant data generating unit 25 extracts audio signal components necessary for redundant data from the acquired original data, and generates redundant data.

[0143] Here, based on Figure 7 The audio signal components necessary for the redundant data in the original data are determined based on a preset table or calculation formula corresponding to the encoding setting determined in step S14 of the above, or a feature amount such as waveform information of the data. At this time, the audio signal components necessary for the redundant data are determined so that even if the audio signal components are replaced by the original data during reproduction, degradation is hardly perceived.

[0144] Furthermore, a process of deleting components other than necessary audio signal components based on encoding settings (such as excluding components from encoding targets in an encoding process in a spectrum of main data) greatly reduces redundant data to be encoded in a subsequent stage.

[0145] In step S53, the redundant data generating unit 25 determines whether raw data to be processed remains in the input buffer 24. If it is determined in step S53 that raw data to be processed remains in the input buffer 24, the process returns to step S51, and the processes in step S51 and subsequent steps are repeated.

[0146] If it is determined in step S53 that there is no raw data to be processed in the input buffer 24, the process proceeds to step S54.

[0147] In step S54, the redundant data generating unit 25 determines whether there is redundant data to be converted into the correlation data. If it is determined in step S54 that there is redundant data to be converted into the correlation data, the process proceeds to step S55.

[0148] Here, the correlation data is data about the correlation between the main data in the group and one or more redundant data. Since it is assumed that the generated redundant data have time series correlation, only the correlation data can be encoded as needed.

[0149] In step S55, the redundant data generating unit 25 determines whether to refer to the master data for related data generation. If it is determined in step S55 that the master data is referred to for related data generation, the process proceeds to step S56.

[0150] In step S56 , the redundant data generating unit 25 acquires the main data (original data itself because no processing is performed) from the input buffer 24 .

[0151] If it is determined in step S55 that the master data is not referred to for related data generation, the processing in step S56 is skipped and the processing proceeds to step S57.

[0152] In step S57, the redundant data generating unit 25 performs the related data generating process and generates the related data corresponding to the redundant data. Hereinafter, the related data is processed as new redundant data. Note that at this time, as long as the redundant data is within an acceptable range in terms of sound quality, it is not always necessary to completely restore the original redundant data from the related data.

[0153] In step S58, the redundant data generating unit 25 determines whether redundant data to be processed remains. If it is determined in step S58 that redundant data to be processed remains, the process returns to step S57, and the processes in step S57 and subsequent steps are repeated.

[0154] If it is determined in step S58 that there is no redundant data to be processed, the process proceeds to step S59.

[0155] Furthermore, if it is determined in step S54 that there is no redundant data to be converted into related data, the process also proceeds to step S59.

[0156] In step S59, the encoding unit 26 generates Figure 7 The redundant data acquired from the redundant data generating unit 25 is encoded according to the encoding setting of the redundant data determined in step S14. That is, the redundant data is encoded so as to fall within the target data amount.

[0157] In step S60, the redundant data generation unit 25 updates the target data amount of the main data using the free space in the packet. The free space in the packet is the sum of the packet data size minus the amount of redundant data that has been encoded. At this time, since the target data amount of the main data is updated, the encoding setting of the main data is updated to fall within the data amount. After step S60, the redundant data generation process ends.

[0158] < Fig.10 Details of the processing in step S60>

[0159] Fig.11 Is used to describe Fig.10 An illustration of the process of step S60 in FIG.

[0160] Fig.11 An example of generating three redundant data for primary data is shown. That is, Fig.11 A and Fig.11 B shows a packet including primary data N and redundant data N+1 to N+3.

[0161] Fig.11 A shows an example of setting the target data amount of the main data N to the target data amount determined in the process of step S14. Here, depending on the content of the redundant data, the encoding result may be significantly smaller than the target data amount. In this case, by the redundant data encoding process in step S59, Fig.11 A free space E is generated in the grouping shown in A.

[0162] Therefore, in step S60, if Fig.11 As shown by the arrow in A, the target data amount of the main data N is increased to fill the free space E in the group.

[0163] Results, such as Fig.11 As shown in B, the target data amount of the main data N is increased. Thus, the main data can be sent in packets with the highest possible quality.

[0164] <Configuration Example of Receiving Terminal>

[0165] Fig.12 is a block diagram showing a configuration example of a receiving terminal.

[0166] The receiving terminal 12 includes a BLE controller 51 , a receiving buffer 52 , a demultiplexing (demultiplexing) unit 53 , a main data buffer 54 , a redundant data buffer 55 , a reproduction data selection unit 56 , a decoding unit 57 , a signal processing unit 58 , and an audio DAC 59 .

[0167] The BLE controller 51 controls operations related to BLE communication. The BLE controller 51 counts packet losses due to BLE communication. The BLE controller 21 supplies packet loss information indicating packet loss to the demultiplexing unit 53 in response to an inquiry from the demultiplexing unit 53.

[0168] The reception buffer 52 receives and accumulates packets through BLE communication.

[0169] The demultiplexing unit 53 acquires the packets accumulated in the reception buffer 52 .

[0170] The demultiplexing unit 53 separates the acquired packets into main data and redundant data. The demultiplexing unit 53 adds a data number (frame_id) to each piece of data, accumulates the main data in the main data buffer 54, and accumulates the redundant data in the redundant data buffer 55. The data number is used to identify the data selected by the reproduction data selection unit 56.

[0171] In the case where data to be reproduced (hereinafter referred to as reproduction data) exists in the main data buffer 54 or the redundant data buffer 55, the reproduction data selection unit 56 acquires the reproduction data and outputs the reproduction data to the decoding unit 57. The reproduction data selection unit 56 notifies the signal processing unit 58 of the presence or absence of the reproduction data.

[0172] The decoding unit 57 acquires and decodes the main data or one or more pieces of redundant data supplied from the reproduction data selection unit 56. The decoding unit 57 outputs the decoded main data or one or more pieces of redundant data to the signal processing unit 58.

[0173] According to the reproduction data presence or absence notification provided by the reproduction data selection unit 56, in the case where the reproduction data exists, the signal processing unit 58 acquires data from the decoding unit 57 and performs post-processing of decoding on the acquired data. The post-processing of decoding is, for example, signal processing including frequency-time conversion. The processed audio signal is output to a reproduction device in a subsequent stage, for example, an audio DAC 59.

[0174] The audio DAC 59 converts the digital audio signal supplied from the signal processing unit 58 into an analog audio signal and outputs the analog audio signal to a speaker or the like at a subsequent stage.

[0175] <Operation of Receiving Terminal>

[0176] Hereinafter, a basic processing flow of the reception terminal 12 of the first embodiment will be described separately for the data acquisition process and the audio reproduction process.

[0177] <Data acquisition processing>

[0178] Fig.13 : is a flowchart for describing the data acquisition process of the receiving terminal 12 .

[0179] Hereinafter, for convenience of description, it is assumed that the data in the reception buffer 52 is in time sequence when the process is started, and the redundant data is generated based on the future original data with respect to the main data.

[0180] Furthermore, in order to identify data to be reproduced in the audio reproduction process described later, the receiving terminal 12 adds a data number (frame_id) to each piece of data in chronological order and counts frame_id using a variable main_id. It is assumed that neither frame_id nor main_id overflows.

[0181] In step S81 , the demultiplexing unit 53 acquires the packet from the reception buffer 52 .

[0182] In step S82, the demultiplexing unit 53 updates main_id to main_id+1, and obtains the main_id of the acquired packet (frame_id of the main data). The main_id of the packet obtained here is the main_id in the case where no packet loss occurs.

[0183] In step S83 , the demultiplexing unit 53 inquires of the BLE controller 51 whether a packet loss has occurred.

[0184] In step S84, the demultiplexing unit 53 determines whether packet loss has occurred between the last acquired packet and the currently acquired packet based on the packet loss information acquired from the BLE controller 51. If it is determined in step S84 that packet loss has occurred between the last acquired packet and the currently acquired packet, the process proceeds to step S85.

[0185] In step S85 , the demultiplexing unit 53 updates main_id to main_id+the number of occurrences of packet loss.

[0186] If it is determined in step S84 that packet loss has not occurred between the packet acquired last time and the packet acquired this time, the processing in step S85 is skipped and the processing proceeds to step S86.

[0187] In step S86, the demultiplexing unit 53 performs demultiplexing processing. Specifically, the demultiplexing unit 53 separates the acquired packet into main data and redundant data.

[0188] In step S87 , the demultiplexing unit 53 sets the frame_id to be added to the main data as main_id, and accumulates the frame_id in the main data buffer 54 .

[0189] In step S88, the demultiplexing unit 53 sets frame_id to be added to each piece of redundant data to main_id+1 to main_id+the number of pieces of redundant data in time sequence, and accumulates frame_id in the redundant data buffer 55. After step S88, the data acquisition process ends.

[0190] <How to add data number>

[0191] Fig.14 is a diagram for describing a method of adding data numbers.

[0192] Fig.14 1 shows a received packet received by the reception buffer 52. The received packet includes main data and three pieces of redundant data, and in the case where no packet loss has occurred since the last reception, main_id=37 is set.

[0193] The demultiplexing unit 53 sets the frame_id to be added to the main data to main_id=37, and accumulates the frame_id in the main data buffer 54 .

[0194] The demultiplexing unit 53 sets the frame_id to be added to the respective pieces of redundant data to main_id+1 ( 38 ) to main_id+3 ( 40 ) in time sequence, and accumulates the frame_id in the redundant data buffer 55 .

[0195] Here, if no packet loss occurs, the frame_id set to the next main data is 38, and if n packet losses occur, the frame_id is 38+n.

[0196] <Audio Reproduction Processing>

[0197] Fig.15 1 is a flowchart for describing the audio reproduction process of the receiving terminal 12.

[0198] In step S101, the reproduction data selection unit 56 performs reproduction data selection processing. Note that this will be referred to later. Fig.16 The details of the reproduction data selection process are described. Through the process of step S101, which one of the main data and redundant data is to be reproduced is selected according to the occurrence of packet loss. At this time, data for related data recovery is also obtained as needed. The selected data or the obtained data is output to the decoding unit 57.

[0199] In step S102, the decoding unit 57 performs a decoding process on the data selected by the reproduction data selection unit 56. When the data to be decoded is related data, the main data or redundant data for related data recovery acquired in step S101 is also decoded and restored to independently reproducible data. The decoding unit 57 outputs the decoded main data or redundant data to the signal processing unit 58.

[0200] In step S103, based on the reproduction data presence or absence notification provided by the reproduction data selection unit 56, if reproduction data exists, the signal processing unit 58 acquires data from the decoding unit 57 and performs signal processing such as frequency-time conversion on the acquired data.

[0201] In step S104, the signal processing unit 58 outputs the waveform of the processed audio signal in a format (such as I2S) that can be received by the audio DAC 59. The audio DAC 59 converts the digital audio signal provided from the signal processing unit 58 into an analog signal and outputs the analog signal to a speaker, etc. at a subsequent stage. After step S104, the audio reproduction process ends.

[0202] <Reproduction Data Selection Processing>

[0203] Fig.16 Is used to describe Fig.15 Flowchart of the reproduction data selection process in step S101.

[0204] For ease of description, it is assumed that the frame_id to be reproduced is set to the same as in Fig.13 The frame_id added during the data acquisition process is consistent.

[0205] In step S121, the reproduction data selection unit 56 determines whether main data having frame_id to be reproduced exists in the main data buffer 54. If it is determined in step S121 that main data having frame_id to be reproduced exists in the main data buffer 54, the process proceeds to S122.

[0206] In step S122, the reproduction data selection unit 56 acquires the main data from the main data buffer 54. After step S122, the reproduction data selection process ends.

[0207] On the other hand, if it is determined in step S121 that main data having the frame_id to be reproduced does not exist in the main data buffer 54, the process proceeds to step S123.

[0208] In step S123, the reproduction data selection unit 56 determines whether redundant data having the frame_id to be reproduced exists in the redundant data buffer 55. If it is determined in step S123 that redundant data having the frame_id to be reproduced exists in the redundant data buffer 55, the process proceeds to step S124.

[0209] In step S124 , the reproduction data selection unit 56 acquires the redundant data from the redundant data buffer 55 .

[0210] In step S125, the reproduction data selection unit 56 determines whether the redundant data is relevant data. If it is determined in step S125 that the redundant data is relevant data, the process proceeds to step S126.

[0211] In step S126, if redundant data for redundant data recovery is also necessary, the reproduction data selection unit 56 additionally acquires redundant data.

[0212] If it is determined in step S125 that the redundant data is not relevant data, or after step S126, the reproduction data selection process ends.

[0213] In addition, if it is determined in step S123 that redundant data having the frame_id to be reproduced does not exist in the redundant data buffer 55, the process proceeds to step S127.

[0214] In step S127, the reproduction data selection unit 56 notifies the signal processing unit 58 that there is no redundant data. After step S127, the reproduction data selection process ends.

[0215] As described above, according to the first embodiment of the present technology, it is possible to suppress the variation in the probability of occurrence of packet loss caused by devices in the same communication state. Therefore, it is possible to realize stable voice transmission regardless of the device.

[0216] Furthermore, since a value determined at the time of connection is used, the present invention is also applicable under conditions where dynamic control during transmission is unnecessary or impossible.

[0217] <2. Second Embodiment (Combination of Dynamic Control of Data Amount According to Communication Status)>

[0218] <Overview of Second Embodiment>

[0219] The second embodiment of the present technology can be used in combination with the method described in Patent Document 2.

[0220] In the method described in Patent Document 2, the number of pieces of transmission standby data in the transmission buffer is observed at a constant cycle during communication. By observing the number of pieces of transmission standby data, an effect corresponding to the measurement of the communication state can be obtained. This is for the following reasons.

[0221] Generally, when the communication state is good, it is expected that the number of sent standby data is reduced. On the other hand, when the communication state is poor and retransmission is repeated, it is expected that the number of sent standby data is increased. Thus, when the number of sent standby data is large, it can be determined that the communication state tends to deteriorate, and when the number of sent standby data is small, it can be determined that the communication state tends to be good.

[0222] Therefore, when it is determined that the number of waiting data to be sent is large (condition 1), the number of data to be stored in one packet is increased by reducing the amount of data per frame, thereby extending the effective retransmission time of each data and improving the probability of successful transmission.

[0223] On the other hand, when it is determined that the number of pieces of data to be sent is small (condition 2), the amount of data to be stored in one packet is reduced by increasing the amount of data per frame. As a result, although the effective retransmittable time of each piece of data becomes shorter, data can be sent with the highest possible quality in the current communication state.

[0224] The second embodiment of the present technology is obtained by applying the above-described method to the first embodiment, and has the following two points.

[0225] First point: the number of pieces of primary data and the number of pieces of redundant data stored in a packet may be set according to the communication status using a retransmission timeout period and the number of packets (or the number of pieces of primary data) accumulated in a transmission buffer.

[0226] Second point: By changing the threshold for determining whether the number of packets in the transmission buffer is large (condition 1) or small (condition 2) according to the retransmission timeout period, the method described in Patent Document 2 can be optimized for each device.

[0227] The retransmission timeout period is actually a "retransmission possible time". In the case where the retransmission timeout period is short, the probability of successful transmission may decrease when the communication state deteriorates. In this case, by making it easy to satisfy condition 1, control is performed as sensitively as possible to increase the number of pieces of primary data per packet in response to changes in the communication state. As a result, the probability of successful transmission increases, and packet loss is less likely to occur.

[0228] When the retransmission timeout period is relatively long, the probability of successful transmission is predicted to be high even when the communication state is deteriorated compared to when the retransmission timeout period is short. Thus, by making it difficult to satisfy condition 1 and easy to satisfy condition 2, reproduction is performed with priority given to reproduction quality as much as possible.

[0229] Note that, hereinafter, according to the operating principle of the second embodiment of the present technology, it is assumed that one or more pieces of master data are stored for one packet. Other assumptions are similar to those of the first embodiment.

[0230] The configuration of the audio transmission system of the second embodiment is the same as that in Figure 5 The configuration of the audio transmission system of the first embodiment shown in FIG. 1 is similar to that of the audio transmission system of the first embodiment shown in FIG. 10 . Furthermore, the receiving terminal 12 of the second embodiment also has a configuration similar to that of the receiving terminal 12 of the first embodiment.

[0231] Therefore, in the second embodiment, only the configuration of the transmission terminal 11 that is different from that of the first embodiment will be described.

[0232] <Configuration example of sending terminal>

[0233] Fig.17 is a block diagram showing another configuration example of the transmission terminal 11.

[0234] Fig.17 The sending terminal 11 and Figure 6 The difference between the transmitting terminal 11 in FIG. 1 and FIG. 2 is that the redundant data generating unit 25 becomes the redundant data generating unit 101, the encoding unit 26 becomes the encoding unit 102, and the multiplexing unit 27 becomes the multiplexing unit 103. Fig.17 In the configuration shown in the reference Figure 6 The same configurations as those described are denoted by the same reference numerals, and repeated descriptions are appropriately omitted.

[0235] The redundant data generating unit 101 acquires original data corresponding to the main data and redundant data accumulated in the input buffer 24. Furthermore, the redundant data generating unit 101 is supplied with the retransmission timeout period from the retransmission timeout period calculating unit 22, and with the encoded data amount of the redundant data from the encoding unit 102.

[0236] Furthermore, the redundant data generating unit 101 periodically observes the transmission buffer 28 and acquires the accumulated number of packets.

[0237] The redundant data generating unit 101 determines the redundancy based on the retransmission timeout period supplied from the retransmission timeout period calculating unit 22. The redundant data generating unit 101 generates one or more pieces of main data and one or more pieces of redundant data necessary to generate one packet based on the encoded data amount notification supplied from the encoding unit 102, the number of packets accumulated in the transmission buffer 28, and the determined redundancy.

[0238] The redundant data generating unit 101 outputs the generated one or more pieces of main data and redundant data to the encoding unit 102. In addition, the redundant data generating unit 101 determines encoding settings corresponding to the main data and redundant data output to the encoding unit 102, and notifies the encoding unit 102 of the determined encoding settings.

[0239] The encoding unit 102 encodes the main data and the redundant data based on the acquired encoding setting. The encoding unit 102 outputs the encoded main data and the redundant data to the multiplexing unit 103. In addition, the encoding unit 102 outputs the encoded data amount of the redundant data to the redundant data generating unit 101.

[0240] The multiplexing unit 103 performs multiplexing processing on the encoded one or more pieces of primary data and redundant data provided from the encoding unit 102. That is, the multiplexing unit 103 stores the encoded one or more pieces of primary data and redundant data in packets suitable for BLE communication. The multiplexing unit 103 accumulates the packets in the transmission buffer 28 for BLE communication.

[0241] <Operation of the sending terminal>

[0242] Hereinafter, the basic processing flow of the transmitting terminal 11 of the second embodiment will be described separately for the system setting processing, the setting update processing, and the transmission packet generation processing. The system setting processing is a processing that is performed only once when the BLE communication is connected. The transmission packet generation processing is a processing that is performed multiple times during voice transmission via BLE communication.

[0243] The setting update process is a process executed in parallel with the transmission packet generation process at an observation interval of a predetermined fixed time (such as every 20 ms). Through this process, the number of pieces of primary data and the number of pieces of redundant data are dynamically changed according to changes in the communication state.

[0244] <System settings processing>

[0245] Fig.18 1 is a flowchart for describing the system setting process of the transmitting terminal 11.

[0246] Because steps S201 and 202 are Figure 7 Steps S11 and S12 in are basically the same processing, so their description will be omitted.

[0247] In step S203, the redundant data generating unit 101 determines the number of pieces of primary data and the number of pieces of redundant data (redundancy) based on the retransmission timeout period provided from the retransmission timeout period calculating unit 22. The number of pieces of primary data and the number of pieces of redundant data are determined by using a pre-prepared relational expression or table indicating the correspondence between the retransmission timeout period and the number of pieces of primary data and the number of pieces of redundant data.

[0248] In step S204, the redundant data generating unit 101 determines the target data volume of the primary data based on the determined number of primary data and the determined number of redundant data. In addition, in step S204, when the target data volume of the primary data is determined, the encoding setting of the primary data and the encoding setting of the redundant data are determined so as to fall within the target data volume.

[0249] Although the above processing is similar to Figure 7However, since the number of pieces of main data is one or more, the target data amount of one piece of main data is obtained by dividing the remainder obtained by subtracting the sum of the target data amounts of redundant data from the packet data size by the number of pieces of main data.

[0250] Note that the table for determining the number of data pieces and the target data amount of redundant data used in steps S203 and S204 are set so that the target data amount of primary data always exceeds the target data amount of redundant data.

[0251] In step S205, the redundant data generating unit 101 determines the threshold th1 and the threshold th2 (th1>th2), which are used to determine the increase or decrease in the number of pieces of primary data used in the setting update process to be described later. These thresholds can be set to certain fixed default values, or, according to the above advantages, can be set based on a calculation formula or table indicating the correspondence between the retransmission timeout period and the threshold value prepared in advance. After step S205, the system setting process ends.

[0252] <Calculation of target data volume>

[0253] Fig.19 is a diagram showing a specific example of calculation of the target data amount.

[0254] Fig.19 A grouping is shown. For example, Fig.19 As shown, when the number of main data is two, the number of redundant data is three, the target data volume of redundant data is 20 [bytes], and the data size of the group is 240 [bytes], the target data volume of the main data is (240-20×3) / 2=90 [bytes].

[0255] <Settings Update Processing>

[0256] Fig. 20 2 is a flowchart for describing the setting update process of the transmitting terminal 11.

[0257] In step S221 , the redundant data generating unit 101 acquires the number of packets accumulated in the transmission buffer 28 .

[0258] In step S222, the redundant data generating unit 101 determines whether the number of packets in the transmission buffer 28 exceeds the threshold th1 as condition 1. If it is determined in step S222 that the number of packets in the transmission buffer 28 exceeds the threshold th1, it is assumed that the communication state is poor, and therefore the process proceeds to step S223.

[0259] In step S223 , the redundant data generating unit 101 increases the number of pieces of primary data in one packet.

[0260] Furthermore, if it is determined in step S222 that the number of packets in the transmission buffer 28 does not exceed the threshold value th1, the process proceeds to step S224.

[0261] In step S224, as condition 2, it is determined whether the number of packets in the transmission buffer 28 is less than threshold th2. If it is determined in step S224 that the number of packets in the transmission buffer 28 is less than threshold th2, it is assumed that the communication state is good, and therefore the process proceeds to step S225.

[0262] In step S225 , the redundant data generating unit 101 reduces the number of pieces of primary data in one packet.

[0263] After step S223 or step S225, the process proceeds to step S226.

[0264] On the other hand, if it is determined in step S224 that the number of packets of the transmission buffer 28 is not less than the threshold value th2 either, the process proceeds to step S226. In this case, the number of pieces of main data in the packet does not change.

[0265] Note that the determination of condition 1 and condition 2 does not necessarily need to be compared with the set threshold value, and for example, past information (such as the number of packets in the past transmission buffer or the transition of the number of pieces of set main data) may be used as the threshold value. In addition, such as in the case where the current number of packets is n and equal to or greater than m (m is the average value of the number of packets in the past), either the set threshold value and the threshold value including the past information may be used, or both of the set threshold value and the threshold value may be used.

[0266] In step S226 , the redundant data generating unit 101 resets the number of pieces of redundant data based on the number of pieces of primary data and a table indicating a correspondence relationship between a retransmission timeout period and the number of pieces of redundant data.

[0267] For example, when the number of pieces of primary data stored in one packet increases, the number of pieces of redundant data required when a packet is lost also increases, and therefore, the process of step S226 is performed as needed.

[0268] Note that when the number of pieces of redundant data is reset, the redundant data generating unit 101 may determine the number of pieces of redundant data in consideration of the transition history of the number of pieces of primary data, the number of pieces of primary data scheduled in the next packet, and other information.

[0269] In step S227 , the redundant data generating unit 101 resets the target data amount of the main data.

[0270] and Fig.18The target data volume of the main data is obtained based on the number of main data, the number of redundant data, the target data volume of the redundant data, and the data size of the group. In addition, in step S227, when the target data volume of the main data is determined, the encoding settings of the main data and the encoding settings of the redundant data are updated so as to fall within the target data volume.

[0271] After step S227, the setting update process ends.

[0272] <Transmission Packet Generation Process>

[0273] Fig.21 2 is a flowchart for describing a transmission packet generation process of the transmission terminal 11 .

[0274] It should be noted that due to Fig.21 Steps S241 to S246 in Fig. 9 Steps S31 to S36 in are basically similar, so their description is omitted.

[0275] However, it should be noted that in steps S244 to S246, as will be referred to later Fig. 22 For example, in step S244, since the number of main data is one or more, the value obtained by dividing the free space by the number of main data is set as the target data amount of the main data.

[0276] exist Fig.21 In step S247, the multiplexing unit 103 adds the transmission data number (send_id) to the packet. The transmission data number is a number used to calculate the above-mentioned data number (frame_id) in the first embodiment. Fig.23 Describe the details of the transmission data number adding process.

[0277] The multiplexing unit 103 accumulates the packet to which the transmission data number is added in the transmission buffer 28 for BLE communication, and the transmission packet generation process ends. Thereafter, the packet is transmitted to the receiving terminal 12 by BLE communication.

[0278] <Configuration Example of Frames and Packets>

[0279] Fig. 22 is a diagram showing the configuration of frames and packets in the case where one or more pieces of main data exist.

[0280] exist Fig. 22 In A, original data frames N to N+7 are shown along the time axis. In the case where the number of pieces of main data per packet is 2, two pieces of original data indicated by dotted lines are included in one packet.

[0281] Fig. 22 B shows a configuration example of packets in the case where the number of pieces of main data per packet is 2.

[0282] In a case where the number of pieces of redundant data is three packets with respect to primary data and the number of pieces of primary data per packet is 2, the number of pieces of redundant data is 6 future frames.

[0283] Thus, the configuration of the packet for transmitting the original data N and N+1 includes the main data N and N+1, and the redundant data N+2 to N+7 of the original data N+2 to N+7.

[0284] <Send data number addition process>

[0285] Fig.23 Is used to describe Fig.21 Flowchart of the sending data number adding process in step S247.

[0286] In step S261, the multiplexing unit 103 adds a send data number (send_id) to the current packet.

[0287] Here, send_id is a value corresponding to the main data at the packet head when each piece of main data stored in the packet after the reproduction starts is numbered in time series order from the initial value 0. Note that the possible maximum value of send_id is ID_MAX, and when 1 is added to ID_MAX, ID_MAX overflows and becomes 0. In addition, send_id is information to be added to the packet, so it has the minimum necessary bit length. Specifically, the bit length of send_id can be any length as long as it can represent the maximum number of lost data that can be generated continuously.

[0288] In the following, it is assumed that the number of pieces of main data included in a packet is main_num, and consecutive packet losses of ID_MAX or more do not occur.

[0289] In step S262, the multiplexing unit 103 obtains send_id to be added to the next packet as in the following equation (1).

[0290] [Expression 1]

[0291] send_id=(send_id+main_num)%(ID_MAX+1)···(1)

[0292] After step S262, the transmission data number adding process ends.

[0293] <Specific Example of Send Data Number Addition Processing>

[0294] Fig.24 : is a diagram showing a specific example of the transmission data number adding process.

[0295] exist Fig.24 In , when represented by 5 bits, send_id takes a value of 0 to 31, and ID_MAX is 31. In addition, packets 1 to 3 are obtained by arranging packets to be transmitted in time order.

[0296] The send_id of the packet 1 = 27 and main_num (the number of pieces of main data) = 4. In the packet 1, five pieces of redundant data are added to the packet.

[0297] When send_id=27 and main_num (the number of pieces of main data)=4 in packet 1, the send_id of the next (packet 2) is (27+4)%(31+1)=31.

[0298] The send_id of the packet 2 = 31 and main_num (the number of pieces of main data) = 3. In the packet 2, three pieces of redundant data are added to the packet.

[0299] When send_id=31 and main_num=3 in packet 2, send_id added to packet 3 is (31+3)%(31+1)=2.

[0300] The send_id of the packet 3 = 2 and main_num (the number of pieces of main data) = 4. In the packet 3, four pieces of redundant data are added to the packet.

[0301] Thus, send_id is added to all packets to be sent in a similar manner.

[0302] <Operation of Receiving Terminal>

[0303] Hereinafter, in the basic processing flow of the receiving terminal 12 of the second embodiment, a data acquisition process including a difference from the first embodiment will be described. Note that the audio reproduction process includes operations similar to those of the first embodiment.

[0304] In the second embodiment, since the number of pieces of main data stored in one packet varies, the send data number send_id is confirmed when calculating the data number frame_id to be added to each piece of received data.

[0305] In the following, the send data number added to the received packet is send_id, its maximum value is ID_MAX, the number of main data included is main_num, and the variable indicating the send_id of the previous received packet is send_id_before. In addition, similar to the example of the first embodiment, the variable indicating the frame_id of the main data at the head of the packet is main_id.

[0306] It should be noted that, for ease of description, it is assumed that more than ID_MAX consecutive packet losses do not occur, redundant data is generated from future original data with respect to main data, and neither frame_id nor main_id overflows.

[0307] <Data acquisition processing>

[0308] Fig.25 : is a flowchart for describing the data acquisition process of the receiving terminal 12 .

[0309] The receiving terminal 12 updates main_id by using the send_id of the initially received packet. Since the presence or absence of packet loss or the number of data lost when packet loss is identified based on send_id, send_id_before, and the number of main data of the previous packet, it is not necessary to confirm the presence or absence of packet loss as in steps S83 and S84 described in the first embodiment.

[0310] In step S281, the demultiplexing unit 53 determines whether send_id of the received packet is equal to or greater than send_id_before. If it is determined in step S281 that send_id is equal to or greater than send_id_before, the process proceeds to step S282.

[0311] In step S282, the process in the case where send_id does not overflow is performed. That is, the demultiplexing unit 53 obtains main_id=main_id+send_id-send_id_before.

[0312] On the other hand, if it is determined in step S281 that send_id is smaller than send_id_before, the process proceeds to step S283.

[0313] In step S283, processing in the case where send_id overflows is performed. That is, the demultiplexing unit 53 obtains main_id=main_id+(ID_MAX+1+send_id)-send_id_before.

[0314] After step S282 or S283 ends, the processing proceeds to step S284.

[0315] In step S284, as a demultiplexing process, the demultiplexing unit 53 separates the acquired packet into main data and redundant data.

[0316] In step S285 , the demultiplexing unit 53 sets the frame_id to be added to the respective pieces of main data to main_id to main_id+main_num−1 in time sequence, and accumulates the frame_id in the main data buffer 54 .

[0317] In step S286 , the demultiplexing unit 53 sets the frame_id to be added to the respective pieces of redundant data to main_id+main_num to main_id+main_num+(the number of pieces of redundant data−1) in time sequence, and accumulates the frame_id in the redundant data buffer 55 .

[0318] In step S287, the demultiplexing unit 53 updates send_id_before to send_id. After step S287, the data acquisition process ends.

[0319] <Method to add frame_id>

[0320] Fig.26 is a diagram showing an example of a method of adding frame_id.

[0321] Fig.26 Groups 1 to 3 are shown Fig.24 In the case where the sequence shown in is transmitted and packet 2 is lost during BLE communication. At this time, the demultiplexing unit 53 obtains the frame_id of each piece of main data and each piece of redundant data added to packet 3.

[0322] Packet 1 has send_id = 27 when it is received. The main_id immediately after receiving packet 1 is a value less than 150, and is received by Fig.25 The process of step S282 in updates main_id to main_id = 150. Based on the updated main_id = 150, frame_id to be added to the four pieces of main data is set to 150 to 153, and frame_id to be added to the five pieces of redundant data is set to 154 to 158.

[0323] When receiving Packet 3, the demultiplexing unit 53 compares the send_id of the received packet (Packet 3) with the send_id of the previous received packet (Packet 1). Since send_id = 2 and send_id_before (send_id of Packet 1) = 27, send_id < send_id_before.

[0324] Therefore, since send_id < send_id_before, when main_id = 150 at the previous reception, the demultiplexing unit 53 updates main_id to 150 + (31 + 1 + 2) - 27 = 157 in Fig.25 step S283 as described above. According to this value, the frame_id of the four main data added to Packet 3 is set to 157 to 160, and the frame_id of the four redundant data added is set to 161 to 164.

[0325] If the lost Packet 2 with send_id = 31 is received, main_id = 154 is updated. According to this value, the frame_id of the three main data added to Packet 2 is set to 154 to 156, and the frame_id of the three redundant data added is set to 157 to 159.

[0326] Note that in the second embodiment, for ease of description, it is assumed that redundant data is generated from future raw data in chronological order with respect to the main data, and the number of main frames held in the next packet cannot be confirmed.

[0327] For this reason, in Fig.24 and Fig.26 instances, when the number of main data in a packet changes according to the communication state, frames with overlapping redundant data (such as Fig.26 frame_id = 157, 158 in) or frames with unprepared redundant data (such as frame_id = 160) occur.

[0328] Therefore, in practice, the following operations are performed: predetermine the number of main data in the next packet so that the number of redundant data is set to be the same as the number of main data in the next packet to be sent, or do not add redundant data that has been added at least once again. As a result, waste and omission of redundant data are suppressed.

[0329] As described above, according to the second embodiment of the present technology, the change in the packet loss occurrence probability caused by the device in the same communication state can be suppressed. Therefore, stable voice transmission independent of the device can be achieved.

[0330] Furthermore, even when a mechanism for checking the communication status and dynamically changing the data amount is used, interruption of sound immediately after the start of connection or the like can be avoided by setting the two thresholds to appropriate values ​​in advance.

[0331] Furthermore, since the threshold for comparing the number of packets in the transmission buffer is changed according to the retransmission timeout period, appropriate control can be performed according to a device that is sensitive to a change in the communication state or a device that is not sensitive to a change in the communication state.

[0332] <3. Third Embodiment (Control in One-to-Many Communication)>

[0333] <Overview of Third Embodiment>

[0334] In actual voice transmission using BLE, one transmitting terminal can communicate with multiple receiving terminals (such as completely wireless headphones). In the third embodiment, an example in which the present technology is applied to such a case will be described.

[0335] In addition, generally, devices such as completely wireless headphones usually perform control and communicate audio signals between devices in a dedicated scheme as inter-device communication. Since the redundant data used in the present technology is part of the encoding scheme and does not depend on the type of communication scheme, the redundant data can also be used for communication in the dedicated scheme.

[0336] For this reason, by adding redundant data of the present technology to inter-device data for performing inter-device communication and transmitting and receiving data between reception terminals, the possible concealment range can be expanded compared with a single reception terminal.

[0337] In order to realize the above control, in the third embodiment, a process of requesting necessary data between receiving terminals is added. The details will be described below.

[0338] Each receiving terminal checks its own primary data buffer and redundant data buffer and checks whether there is missing data. In the case of missing data, the receiving terminal notifies another receiving terminal of information (such as frame_id) that can identify the missing data as request information through inter-device communication.

[0339] A receiving terminal that receives request information from another receiving terminal through inter-device communication checks whether the requested redundant data exists in its own redundant data buffer. If redundant data exists, the receiving terminal adds the redundant data to the information used for inter-device communication and sends the information to the requesting receiving terminal.

[0340] The receiving terminal that receives the requested redundant data uses the redundant data to interpolate and reproduce the missing data. At this time, even if the receiving terminal can only obtain data of a channel different from the originally reproduced channel, an effect equivalent to single-channel reproduction can be obtained.

[0341] As described above, from the perspective of the transmitting terminal, the possible concealment range can be widened without requiring additional communication costs.

[0342] Note that as the redundant data used in the third embodiment, it is obvious that the related data between channels can be pre-added on the transmitting terminal side. For example, the transmitting terminal adds the related data of the left channel and the right channel, and sends the result data to the receiving terminal of the left channel. After this, if there is a request from the receiving terminal of the right channel, the receiving terminal of the left channel adds the redundant data and the related data of the left channel, and sends the result data to the receiving terminal of the right channel. The method for processing related data is similar to the method for processing related data described in the first embodiment, and its description will be omitted.

[0343] Each receiving terminal performs the above-described processing, making it possible to realize packet loss concealment optimized for a system of multiple terminals (channels).

[0344] <Configuration of audio transmission system>

[0345] Fig. 27 : is a diagram showing a configuration example of an audio transmission system according to a third embodiment of the present technology.

[0346] The audio transmission system is configured by connecting the transmission terminal 211 and the reception terminals 212-1 to 212-3 via BLE communication. In addition, in the audio transmission system, the reception terminals 212-1 to 212-3 perform inter-device communication by a dedicated communication method.

[0347] The transmission terminal 211 includes a multifunctional mobile phone such as a smartphone, a mobile terminal, etc. The transmission terminal 211 generates a packet in a manner similar to those in the first and second embodiments, and transmits the packet to each of the reception terminals 212-1 to 212-3 through BLE communication.

[0348] In the case of a completely wireless headset, the receiving terminals 212-1 to 212-3 output sounds corresponding to audio channels (eg, left and right channels).

[0349] Note that in the case of a device capable of performing stereo reproduction by one device such as a wireless speaker, data of both the left channel and the right channel may be transmitted in one packet by a method similar to that in the second embodiment.

[0350] The receiving terminals 212-1 to 212-3 receive the packets sent from the transmitting terminal 211 via BLE communication, decode the packets by methods similar to those in the first and second embodiments, and reproduce the packets. In addition, in addition to the control information, the receiving terminals 212-1 to 212-3 also send and receive the above-mentioned request information and related redundant data via inter-device communication.

[0351] It should be noted that, hereinafter, in the case where there is no need to distinguish the receiving terminals 212-1 to 212-3, they are referred to as receiving terminal 212. Furthermore, hereinafter, for convenience of description, it is assumed that one receiving terminal 212 receives one channel.

[0352] <Configuration example of sending terminal>

[0353] Fig.28 211 is a block diagram showing a configuration example of the transmission terminal 211.

[0354] Fig.28 The sending terminal 211 and Figure 6 The difference between the transmitting terminal 11 in FIG. 1 and FIG. 2 is that the signal processing unit 23 becomes the signal processing unit 221, the redundant data generating unit 25 becomes the redundant data generating unit 222, the encoding unit 26 becomes the encoding unit 223, and the multiplexing unit 27 becomes the multiplexing unit 224. Fig.28 In the configuration shown in the reference Figure 6 The same configurations as those described are denoted by the same reference numerals, and repeated descriptions are appropriately omitted.

[0355] The signal processing unit 221 is connected to Figure 6 The signal processing unit 23 in FIG. 1 performs preprocessing for audio encoding in a similar manner, adds information indicating which audio channel data a frame includes, and accumulates the data in the input buffer 24 .

[0356] The redundant data generating unit 222 and the encoding unit 223 obtain the original data required to generate one packet of a specific audio channel from the input buffer 24, and Figure 6 The main data and redundant data are generated and encoded in a similar manner to the redundant data generating unit 25 and the encoding unit 26 in the multiplexing unit 224. The encoded main data and redundant data are provided to the multiplexing unit 224.

[0357] The multiplexing unit 224 is connected to Figure 6The multiplexing unit 224 accumulates the packets for BLE communication in the transmission buffer 28.

[0358] Note that the operation of the transmitting terminal 211 is similar to that of the first or second embodiment except for consideration of which audio channel each piece of data belongs to, and thus a description thereof will be omitted.

[0359] <Configuration Example of Receiving Terminal>

[0360] Fig.29 is a block diagram showing a configuration example of the receiving terminal 212. Fig.29 In the configuration shown in the reference Fig.12 The same configurations as those described are denoted by the same reference numerals, and repeated descriptions are appropriately omitted.

[0361] Fig.29 The sending terminal 211 and Fig.12 The receiving terminal 12 in FIG. 1 is different in that a receiving buffer 251, an inter-device data demultiplexing unit 252, an inter-device communication control unit 253, a redundant data adding unit 254, and a sending buffer 255 are added.

[0362] The reception buffer 251 receives and accumulates packets through inter-device communication.

[0363] The inter-device data demultiplexing unit 252 acquires packets of the inter-device data accumulated in the receiving buffer 251. In the case where redundant data is added to the acquired packets, the inter-device data demultiplexing unit 252 separates the redundant data, accumulates the redundant data in the redundant data buffer 55, and outputs other information (hereinafter, referred to as inter-device control information) to the inter-device communication control unit 253.

[0364] The inter-device communication control unit 253 acquires the inter-device control information provided from the inter-device data demultiplexing unit 252 and transmitted from the other receiving terminal 212. The inter-device control information may include a request for redundant data from the other receiving terminal 212. In this case, the inter-device communication control unit 253 checks the data existing in the redundant data buffer 55 among the data of the frame_id requested from the other receiving terminal 212. The inter-device communication control unit 253 sets the redundant data existing in the redundant data buffer 55 as the redundant data to be added to the inter-device control information. The inter-device communication control unit 253 outputs the inter-device control information transmitted by itself and the frame_id of the redundant data to be added to the redundant data adding unit 254.

[0365] The redundant data adding unit 254 acquires redundant data corresponding to the frame_id of the redundant data to be added from the redundant data buffer 55. The redundant data adding unit 254 adds the acquired redundant data to the inter-device control information, stores the information in a packet suitable for inter-device communication, and accumulates the packet in the transmission buffer 255.

[0366] Furthermore, the inter-device communication control unit 253 checks the main data buffer 54 and the redundant data buffer 55, and acquires the frame_id requested from the inter-device communication control unit 253 itself to the other receiving terminal 212. The inter-device communication control unit 253 adds information of the frame_id of the redundant data to be requested to the inter-device control information transmitted by itself, and outputs the information to the redundant data adding unit 254.

[0367] The redundant data adding unit 254 stores the inter-device control information in a packet suitable for inter-device communication, and accumulates the packet in the transmission buffer 255 .

[0368] The packets stored in the transmission buffer 255 are transmitted to another receiving terminal 212 through inter-device communication.

[0369] <Operation of Receiving Terminal>

[0370] In the following, in the basic processing flow of the receiving terminal 212 of the third embodiment, only the inter-device communication control processing different from the first or second embodiment will be described. The inter-device communication control processing is performed at a timing independent of the BLE communication and data acquisition processing and audio reproduction processing described in the first and second embodiments.

[0371] <Inter-device communication control processing>

[0372] Fig.30 2 is a flowchart for describing the inter-device communication control processing of the receiving terminal 212.

[0373] Hereinafter, for convenience of description, it is assumed that the reproduction timing is synchronized with allowable accuracy among the receiving terminals 212. In addition, as the number indicating the frame of the same timing in the audio data of the transmitting terminal 211, it is assumed that frame_id can be correctly shared by all the receiving terminals 212.

[0374] In step S301 , the inter-device data demultiplexing unit 252 acquires a packet of inter-device data transmitted from another reception terminal 212 and accumulated in the reception buffer 251 .

[0375] In step S302, the inter-device data demultiplexing unit 252 determines whether redundant data is added to the inter-device data. If it is determined in step S302 that redundant data has been added to the inter-device data, the process proceeds to step S303.

[0376] In step S303 , the inter-device data demultiplexing unit 252 performs inter-device data demultiplexing processing to separate redundant data.

[0377] In step S304, the inter-device data demultiplexing unit 252 determines whether data of the same frame_id exists in the primary data buffer 54 and the redundant data buffer 55. If it is determined that data of the same frame_id exists in the primary data buffer 54 and the redundant data buffer 55, the process proceeds to step S305.

[0378] In step S305, when the priority of the redundant data received from another receiving terminal 212 is higher, the inter-device data demultiplexing unit 252 replaces the data with the same frame_id existing in the redundant data buffer 55 with the redundant data already received from another receiving terminal 212 and accumulated in the receiving buffer 251.

[0379] For example, in the case of the reception terminal 212 receiving left channel audio data, the left channel audio signal has a higher priority, while the right channel audio signal has a lower priority.

[0380] On the other hand, in the case where the priority of the redundant data received from the other receiving terminal 212 is lower, the inter-device data demultiplexing unit 252 discards the received redundant data.

[0381] For example, when the receiving terminal reproduces the audio signal of ch1 and redundant data corresponding to {frame_id, ch}={150, 1} exists in the redundant data buffer 55, even if redundant data corresponding to {frame_id, ch}={150, 2} is newly received, the received redundant data is discarded.

[0382] If it is determined in step S304 that data of the same frame_id does not exist in the primary data buffer 54 and the redundant data buffer 55 , the process proceeds to step S306 .

[0383] In step S306 , the inter-device data demultiplexing unit 252 accumulates the redundant data received from the other receiving terminal 212 in the redundant data buffer 55 .

[0384] On the other hand, if it is determined in step S302 that redundant data is not added to the inter-device data, the processing of steps S303 to S306 is skipped, and the processing proceeds to step S307.

[0385] In step S307, the inter-device communication control unit 253 performs request data determination processing. Fig.31 The details of the request data determination process are described. Through the process of step S307, the main data buffer 54 and the redundant data buffer 55 of the inter-device communication control unit 253 itself are checked, and a list of frame_ids to be requested is obtained.

[0386] The inter-device communication control unit 253 sets the list of frame_ids to be requested together with its own channel information as request information for the receiving terminal 212. For example, for the receiving terminal 212 receiving ch2, the request information is configured as {frame_id, ch}={150, 2}, {167, 2}, {180, 2}, etc.

[0387] For example, assuming that the own terminal is a receiving terminal that reproduces the audio signal of ch1, and the request target terminal is a receiving terminal that reproduces the audio signal of ch2 but also receives the audio signal of ch1. In this case, the own terminal requests {frame_id, ch} = {150, 1} with a high priority from the receiving terminal 212.

[0388] Note that a request for a channel may be made indiscriminately. In this case, the request information is configured as, for example, {frame_id, ch} = {150, x (don't care)}.

[0389] In step S308, the inter-device communication control unit 253 performs a process of generating inter-device control information. Specifically, in addition to the request information, the inter-device communication control unit 253 also generates the inter-device control information to be sent by itself. In addition, the inter-device communication control unit 253 adds the above request information to the generated inter-device control information as needed.

[0390] In step S309, the inter-device communication control unit 253 acquires request information of another receiving terminal 212 from the inter-device data acquired in step S301. The inter-device communication control unit 253 outputs the inter-device control information and frame_id of the redundant data to be added to the redundant data adding unit 254.

[0391] In step S310, the redundant data adding unit 254 determines whether redundant data corresponding to the frame_id in the request information from the other receiving terminal 212 exists in the redundant data buffer 55. If it is determined in step S310 that redundant data corresponding to the frame_id exists in the request information from the other receiving terminal 212, the process proceeds to step S311.

[0392] In step S311, the redundant data adding unit 254 selects redundant data to be added according to the amount that can be added and the adding priority, and adds the selected redundant data to the inter-device control information. The adding priority is used to give priority to the transmission of the old frame_id as much as possible. Note that the priority related to the above channel can also be considered.

[0393] In addition, if it is determined in step S310 that redundant data corresponding to frame_id does not exist in the request information from the other receiving terminal 212, the processing in step S311 is skipped, and the processing proceeds to step S312.

[0394] In step S312 , the redundant data adding unit 254 stores the inter-device control information in a packet suitable for inter-device communication, and accumulates the packet in the transmission buffer 255 .

[0395] The packets stored in the transmission buffer 255 are transmitted to another receiving terminal 212 through the inter-device communication. Thereafter, the inter-device communication control process ends.

[0396] <Request data confirmation processing>

[0397] Fig.31 is used to describe Fig.30 Flowchart of the request data determination process in step S307 of FIG.

[0398] In this process, in order to confirm whether the requested data exists, a variable id that temporarily holds frame_id is prepared.

[0399] In step S341, the inter-device communication control unit 253 determines whether a request for redundant data is necessary. For example, in the case where no packet loss occurs for a long time, a request does not need to be made. Therefore, it is determined that a request for redundant data is not necessary, and the request data determination process is ended.

[0400] If it is determined in step S341 that a request for redundant data is necessary, the process proceeds to step S342.

[0401] In step S342 , the inter-device communication control unit 253 replaces the frame_id of the oldest data currently in its own primary data buffer 54 and redundant data buffer 55 with id and initializes id.

[0402] In step S343, the inter-device communication control unit 253 determines whether data of frame_id matching id exists in the primary data buffer 54 and the redundant data buffer 55. If it is determined in step S343 that data of frame_id matching id does not exist in the primary data buffer 54 and the redundant data buffer 55, the process proceeds to step S344.

[0403] In step S344 , the inter-device communication control unit 253 adds id to the request information.

[0404] If it is determined in step S343 that data of frame_id matching id exists in the primary data buffer 54 and the redundant data buffer 55, the processing of step S344 is skipped and the processing proceeds to step S345.

[0405] In step S345 , the inter-device communication control unit 253 updates id to id+1.

[0406] In step S346, the inter-device communication control unit 253 determines whether id is equal to or less than the latest frame_id in the main data buffer 54 and whether the requested data is equal to or less than the pre-specified number. If it is determined in step S346 that id is equal to or less than the latest frame_id in the main data buffer 54 and the requested data is equal to or less than the specified number, the process returns to step S343, and the processes in step S343 and subsequent steps are repeated.

[0407] If it is determined in step S346 that id has become larger than the latest frame_id in the main data or the request data has become larger than the specified number, the request data determination processing ends.

[0408] In this way, by specifying the number in advance and setting the number of pieces of request data to be equal to or smaller than the specified number, it is possible to suppress an unnecessary increase in the amount of data communicated between devices.

[0409] As described above, according to the third embodiment of the present technology, it is possible to suppress the variation in the probability of occurrence of packet loss caused by the device in the same communication state. Therefore, it is possible to realize stable voice transmission regardless of the device.

[0410] Furthermore, according to the third embodiment, in the system of voice transmission to a plurality of devices (channels), the frequency of sound interruption can be reduced compared to a single device.

[0411] It should be noted that the technology of the third embodiment can be applied not only to the first embodiment but also to the second embodiment.

[0412] <4. Others>

[0413] <Computer Configuration Example>

[0414] The above-described series of processes can be executed by hardware or software. In the case where the series of processes are executed by software, a program forming the software is installed from a program recording medium into a computer incorporated in dedicated hardware, a general-purpose personal computer, or the like.

[0415] Fig.32 : is a block diagram showing a hardware configuration example of a computer that executes the above-described series of processes according to a program.

[0416] A central processing unit (CPU) 301 , a read only memory (ROM) 302 , and a random access memory (RAM) 303 are connected to each other via a bus 304 .

[0417] The input / output interface 305 is also connected to the bus 304. An input unit 306 including a keyboard, a mouse, etc. and an output unit 307 including a display, a speaker, etc. are connected to the input / output interface 305. In addition, a storage unit 308 such as a hard disk and a nonvolatile memory, a communication unit 309 such as a network interface, and a drive 310 for driving a removable medium 311 are connected to the input / output interface 305.

[0418] In the computer configured as described above, for example, the CPU 301 loads a program stored in the storage unit 308 onto the RAM 303 through the input / output interface 305 and the bus 304 , and executes the program to perform the above-described series of processing.

[0419] For example, the program executed by the CPU 301 is provided by being recorded in the removable medium 311 or through a wired or wireless transmission medium (eg, a local area network, the Internet, or digital broadcasting), and is installed in the storage unit 308 .

[0420] Note that the program executed by the computer may be a program that is processed in time series according to the order described in this specification, or a program that is processed in parallel, or a program that is processed at necessary timing (for example, when a call is made).

[0421] Note that in this specification, a system means a collection of multiple components (devices, modules (components), etc.), and it does not matter whether all components are in the same case. Therefore, multiple devices housed in separate housings and connected via a network, and a device that houses multiple modules in one housing are both systems.

[0422] In addition, the effects described in this specification are only illustrative and not restrictive. Therefore, other effects can be obtained.

[0423] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made without departing from the scope of the present technology.

[0424] For example, the present technology may have a cloud computing configuration in which one function is shared and processed by a plurality of devices through a network.

[0425] Furthermore, each step described in the above flowchart may be performed by one device or performed by a plurality of devices in a shared manner.

[0426] Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in one step may be executed by one device or may be executed by a plurality of devices in a shared manner.

[0427] <Configuration instance combination>

[0428] The present technology may also be configured as follows. (1)

[0430] A receiving terminal, comprising:

[0431] a receiving unit that receives a packet transmitted from a transmitting terminal, the packet including primary data and redundant data, the redundant data being generated based on redundancy and used for recovery or interpolation of the primary data, the redundancy being determined based on a retransmission timeout period, the retransmission timeout period being a time determined by a combination of the transmitting terminal and the receiving terminal, and in which a retransmission process of the primary data may be repeatedly performed; and

[0432] The decoding unit decodes the reproduced data by decoding the main data or the redundant data. (2)

[0434] The receiving terminal according to (1) above further includes:

[0435] a master data buffer to accumulate master data; and

[0436] A redundant data buffer accumulates redundant data, wherein

[0437] In a case where main data corresponding to the reproduction data does not exist in the main data buffer, the decoding unit decodes the reproduction data by decoding redundant data corresponding to the reproduction data read from the redundant data buffer. (3)

[0439] The receiving terminal according to (1) or (2) above, wherein

[0440] The redundancy is represented by the number of redundant data items in each packet. (4)

[0442] The receiving terminal according to (1) or (2) above, wherein

[0443] The redundant data corresponds to a frame before or after a frame corresponding to the main data included in the same packet, and

[0444] The redundancy is adjusted by the range of frames corresponding to the redundant data in the packet. (5)

[0446] A receiving terminal according to any one of (1) to (4) above, wherein

[0447] The ratio of the amount of primary data to the amount of redundant data in the packet has been adjusted. (6)

[0449] A receiving terminal according to any one of (1) to (5) above, wherein

[0450] An area in the packet freed by encoding the redundant data is allocated to an area for main data. (7)

[0452] The receiving terminal according to any one of (1) to (6) above, wherein the redundant data is generated based on the redundancy and the feature amount of the main data. (8)

[0454] A receiving terminal according to any one of (1) to (7) above, wherein

[0455] The redundant data is related data regarding the correlation between the main data and the redundant data or the correlation between the redundant data and another redundant data. (9)

[0457] A receiving terminal according to any one of (1) to (8) above, wherein

[0458] The receiving unit adds a data number for identifying data to be reproduced to the main data and the redundant data based on the packet loss information indicating the loss of the packet. (10)

[0460] The receiving terminal according to (1) above, wherein

[0461] The redundancy is determined based on the retransmission timeout period and the number of packets accumulated in the transmit buffer. (11)

[0463] The receiving terminal according to (10) above, wherein

[0464] The number of pieces of primary data is determined based on the retransmission timeout period and the number of packets accumulated in the transmission buffer, the number of pieces of primary data being the number of pieces of primary data per packet. (12)

[0466] The receiving terminal according to (11) above, wherein

[0467] A threshold value used for comparison with the number of packets accumulated in the transmission buffer to determine the number of pieces of primary data is changed according to the retransmission timeout period. (13)

[0469] The receiving terminal according to (11) or (12) above, wherein

[0470] The redundancy is changed based on the number of primary data and the retransmission timeout period. (14)

[0472] The receiving terminal according to (13) above, wherein

[0473] The data amount of the main data is changed based on the number of pieces of the main data, the number of pieces of the redundant data in each packet, and the data amount of the redundant data. (15)

[0475] A receiving terminal according to any one of (11) to (14) above, wherein

[0476] A transmission data number is added to the packet, the transmission data number being a number corresponding to the main data at the head of the packet when the pieces of main data are numbered in time-series order. (16)

[0478] The receiving terminal according to (15) above, wherein

[0479] The receiving unit adds a data number for identifying data to be reproduced to the main data and the redundant data based on the transmitted data number. (17)

[0481] The receiving terminal according to (2) above further comprises

[0482] a communication unit for communicating with another receiving terminal for receiving a packet transmitted from the transmitting terminal, wherein

[0483] The communication unit receives redundant data from another receiving terminal. (18)

[0485] The receiving terminal according to (17) above further comprises

[0486] a control unit that requests the redundant data corresponding to the reproduction data from another receiving terminal when the main data corresponding to the reproduction data is not in the main data buffer and the redundant data corresponding to the reproduction data is not in the redundant data buffer, wherein

[0487] The communication unit receives redundant data transmitted from another receiving terminal in response to the request. (19)

[0489] The receiving terminal according to (17) or (18) above, wherein

[0490] The communication unit reads redundant data requested from another receiving terminal from the redundant data buffer and transmits the redundant data to the other receiving terminal. (20)

[0492] A receiving method, comprising:

[0493] From the receiving terminal:

[0494] receiving a packet transmitted from a transmitting terminal, the packet including primary data and redundant data, the redundant data being generated based on redundancy and used for recovery or interpolation of the primary data, the redundancy being determined based on a retransmission timeout period, the retransmission timeout period being a time determined by a combination of the transmitting terminal and the receiving terminal, and a retransmission process of the primary data can be repeatedly performed during the retransmission timeout period; and

[0495] The reproduced data is decoded by decoding the main data or the redundant data. (twenty one)

[0497] A sending terminal, comprising:

[0498] a redundant data generating unit that determines the redundancy of redundant data used for recovery or interpolation of primary data based on a retransmission timeout period that is a time determined by a combination of a transmitting terminal and a receiving terminal and in which a retransmission process of the primary data can be repeatedly performed; and

[0499] The sending unit sends a packet including the main data and the redundant data generated based on the redundancy. (twenty two)

[0501] The transmitting terminal according to (21) above, wherein

[0502] The redundancy is represented by the number of redundant data items in each packet. (twenty three)

[0504] The transmitting terminal according to (21) above, wherein

[0505] The redundant data corresponds to a frame before or after a frame corresponding to the main data included in the same packet, and

[0506] The redundant data generating unit adjusts the redundancy according to the range of the frame corresponding to the redundant data in the packet. (twenty four)

[0508] A transmitting terminal according to any one of (21) to (23) above, wherein

[0509] The redundant data generating unit adjusts the ratio between the amount of main data and the amount of redundant data in the packet. (25)

[0511] A transmitting terminal according to any one of (21) to (24) above, wherein

[0512] The redundant data generating unit allocates an area in the packet released by encoding the main data and the redundant data to an area of ​​the main data. (26)

[0514] A transmitting terminal according to any one of (21) to (25) above, wherein

[0515] The redundant data generating unit generates redundant data based on the redundancy and the feature amount of the main data. (27)

[0517] A transmitting terminal according to any one of (21) to (26) above, wherein

[0518] The redundant data generating unit sets correlation data regarding correlation between the main data and the redundant data or correlation between the redundant data and another redundant data as the redundant data. (28)

[0520] The transmitting terminal according to (21) above, wherein

[0521] The redundant data generating unit determines the redundancy of the redundant data based on the retransmission timeout period and the number of packets accumulated in the transmission buffer. (29)

[0523] The transmitting terminal according to (28) above, wherein

[0524] The redundant data generating unit determines the number of pieces of primary data based on the retransmission timeout period and the number of packets accumulated in the transmission buffer, the number of pieces of primary data being the number of pieces of primary data in each packet. (30)

[0526] The transmitting terminal according to (29) above, wherein

[0527] The redundant data generating unit changes a threshold value used for comparison with the number of packets accumulated in the transmission buffer according to the retransmission timeout period to determine the number of pieces of primary data. (31)

[0529] The transmitting terminal according to (29) or (30) above, wherein

[0530] The redundant data generating unit changes the redundancy based on the number of pieces of primary data and the retransmission timeout period. (32)

[0532] The transmitting terminal according to (29) above, wherein

[0533] The redundant data generating unit changes the data amount of the main data based on the number of pieces of the main data, the number of pieces of the redundant data in each packet, and the data amount of the redundant data. (33)

[0535] The transmitting terminal according to any one of (28) to (32) above, further comprising:

[0536] A control unit that, when the pieces of main data are numbered in time series order, adds a value corresponding to the main data at the head of a packet as a transmission data number to the packet. (34)

[0538] The transmitting terminal according to (21) above further includes:

[0539] The signal processing unit adds identification information for identifying a channel of the frame, wherein

[0540] The transmission unit transmits the packet corresponding to the frame by acquiring a receiving terminal as a transmission destination from among the plurality of receiving terminals based on the identification information. (35)

[0542] A transmission method, comprising:

[0543] From the sending terminal:

[0544] determining the redundancy of redundant data used for recovery or interpolation of primary data based on a retransmission timeout period which is a time determined by a combination of a transmitting terminal and a receiving terminal and in which a retransmission process of the primary data may be repeatedly performed; and

[0545] A packet including primary data and redundant data generated based on the redundancy is transmitted.

[0546] Reference numerals list

[0547] 11 Sending terminal

[0548] 12 Receiving Terminal

[0549] 21 BLE controller

[0550] 22 Retransmission timeout calculation unit

[0551] 23 Signal Processing Unit

[0552] 24 Input Buffer

[0553] 25 Redundant data generation unit

[0554] 26 coding units

[0555] 27 Multiplexing Unit

[0556] 28 Send Buffer

[0557] 51 BLE controller

[0558] 52 Receive Buffer

[0559] 101 Redundant data generation unit

[0560] 102 coding units

[0561] 103 Multiplexing Unit

[0562] 28 Send Buffer

[0563] 211 Sending terminal

[0564] 221 Signal Processing Unit

[0565] 222 Redundant data generation unit

[0566] 223 coding units

[0567] 251 Receive Buffer

[0568] 252 Inter-device data demultiplexing unit

[0569] 253 Inter-device communication control unit

[0570] 254 Redundant data adding unit

[0571] 255 send buffer.

Claims

1. A receiving terminal, comprising: a receiving unit, receiving a packet sent from a transmitting terminal, the packet including primary data and redundant data, the redundant data being generated based on redundancy and used for recovery or interpolation of the primary data, the redundancy being determined based on a retransmission timeout period, the retransmission timeout period being a time determined by a combination of the transmitting terminal and the receiving terminal, and a retransmission process of the primary data being able to be repeatedly performed during the retransmission timeout period; as well as a decoding unit, decoding the reproduced data by decoding the main data or the redundant data; a main data buffer for accumulating the main data; A redundant data buffer for accumulating the redundant data; as well as A control unit that requests the redundant data corresponding to the reproduced data from another receiving terminal that receives the packet sent from the sending terminal, when the main data corresponding to the reproduced data is not in the main data buffer and the redundant data corresponding to the reproduced data is not in the redundant data buffer.

2. The receiving terminal according to claim 1, wherein: The redundancy is represented by the number of redundant data in each packet.

3. The receiving terminal according to claim 1, wherein: The redundant data corresponds to a frame before or after a frame corresponding to the main data included in the same packet, and The redundancy is adjusted by a range of frames corresponding to the redundant data in the packet.

4. The receiving terminal according to claim 1, wherein: The ratio of the amount of the primary data and the amount of the redundant data in the packet is adjusted.

5. The receiving terminal according to claim 1, wherein: An area in the packet released by encoding the redundant data is allocated to an area of ​​the main data.

6. The receiving terminal according to claim 1, wherein: The redundant data is generated based on the redundancy and a feature amount of the main data.

7. The receiving terminal according to claim 1, wherein: The redundant data is correlation data regarding correlation between the main data and the redundant data, or correlation between the redundant data and another redundant data.

8. The receiving terminal according to claim 1, wherein: The receiving unit adds a data number for identifying data to be reproduced to the main data and the redundant data based on packet loss information indicating a loss of a packet.

9. The receiving terminal according to claim 1, wherein: The redundancy is determined based on the retransmission timeout period and the number of packets accumulated in a transmission buffer.

10. The receiving terminal according to claim 9, wherein: The number of pieces of the primary data is determined based on the retransmission timeout period and the number of the packets accumulated in the transmission buffer, the number of pieces of the primary data being the number of pieces of the primary data in each packet.

11. The receiving terminal according to claim 10, wherein: A threshold value for comparison with the number of the packets accumulated in the transmission buffer to determine the number of pieces of the primary data is changed according to the retransmission timeout period.

12. The receiving terminal according to claim 10, wherein: The redundancy changes based on the number of pieces of the primary data and the retransmission timeout period.

13. The receiving terminal according to claim 12, wherein: The data amount of the main data changes based on the number of pieces of the main data, the number of pieces of the redundant data in each packet, and the data amount of the redundant data.

14. The receiving terminal according to claim 10, wherein: A transmission data number is added to the packet, the transmission data number being a number corresponding to the main data at the head of the packet when the pieces of main data are numbered in time series order.

15. The receiving terminal according to claim 14, wherein: The receiving unit adds a data number for identifying data to be reproduced to the main data and the redundant data based on the transmitted data number.

16. The receiving terminal according to claim 1, further comprising: A communication unit communicates with the other receiving terminal, wherein: The communication unit receives the redundant data transmitted from the other receiving terminal in response to the request.

17. The receiving terminal according to claim 16, wherein: The communication unit reads the redundant data requested from the other receiving terminal from the redundant data buffer, and transmits the redundant data to the other receiving terminal.

18. A receiving method, comprising: From the receiving terminal: receiving a packet transmitted from a transmitting terminal, the packet including primary data and redundant data, the redundant data being generated based on redundancy and used for recovery or interpolation of the primary data, the redundancy being determined based on a retransmission timeout period, the retransmission timeout period being a time determined by a combination of the transmitting terminal and the receiving terminal, and a retransmission process of the primary data being able to be repeatedly performed during the retransmission timeout period; decoding reproduction data by decoding the main data or the redundant data; as well as When the main data corresponding to the reproduced data is not in the main data buffer and the redundant data corresponding to the reproduced data is not in the redundant data buffer, the redundant data corresponding to the reproduced data is requested from another receiving terminal that receives the packet sent from the sending terminal.

Images