Information transmission method and device based on audio two-dimensional code, equipment and medium

Abstract

The embodiment of the invention provides an information transmission method and device based on an audio two-dimensional code, equipment and a medium, and belongs to the technical field of communication. The method comprises the following steps: acquiring a preset initial audio and to-be-transmitted target information; separately encoding the initial audio and the target information to obtain audio encoding features and information encoding features; performing hybrid coding on the audio coding feature and the information coding feature to obtain an initial audio two-dimensional code; and sending the initial audio two-dimensional code to a client, so that the client receives the initial audio two-dimensional code sent by the terminal, configures multiple types of distorted audios for the initial audio two-dimensional code, introduces the multiple types of distorted audios into the initial audio two-dimensional code to obtain an enhanced audio two-dimensional code, and decodes the enhanced audio two-dimensional code to obtain target information. The accuracy of transmitting the target information based on the audio two-dimensional code can be improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an information transmission method, device, equipment and medium based on an audio QR code. Background Art

[0002] An audio QR code is an audio message that carries QR code information. It is designed to use sound to quickly transmit information such as text, images or files.

[0003] In the related art, the target information is transmitted by reading the audio that carries the QR code information in the audio QR code. However, the noise generated in different reading scenarios can easily interfere with the information transmission, thereby distorting the audio QR code, and ultimately causing the target information obtained based on the audio QR code reading to be inaccurate. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose an information transmission method, device, equipment and medium based on audio QR code, aiming to improve the accuracy of target information transmitted based on audio QR code.

[0005] To achieve the above-mentioned purpose, the first aspect of the embodiment of the present application proposes an information transmission method based on an audio QR code, which is applied to a terminal, and the terminal is connected to a client for communication, and the method includes:

[0006] Obtaining the preset initial audio and target information to be transmitted;

[0007] Encode the initial audio and target information separately to obtain audio coding features and information coding features;

[0008] Mix and encode the audio coding features and the information coding features to obtain an initial audio QR code;

[0009] An initial audio QR code is sent to a client so that the client receives the initial audio QR code sent by a terminal, multiple types of distorted audio are configured for the initial audio QR code, and multiple distorted audios are introduced into the initial audio QR code to obtain an enhanced audio QR code, and the enhanced audio QR code is decoded to obtain target information.

[0010] In some embodiments, the initial audio QR code is output by a pre-trained encoder, and the encoder is trained by the following steps, comprising:

[0011] Obtaining preset sample initial audio and sample target information to be transmitted;

[0012] Encode the sample initial audio and the sample target information separately to obtain the sample audio encoding feature and the sample information encoding feature;

[0013] Mix and encode the sample audio coding features and the sample information coding features to obtain the sample initial audio QR code;

[0014] Based on the difference between the sample initial audio and the sample initial audio QR code in the time domain, a time domain loss value is calculated, and based on the difference between the sample initial audio and the sample initial audio QR code in the frequency domain, a frequency domain loss value is calculated;

[0015] According to the time domain loss value and the frequency domain loss value, the encoder parameters of the encoder are adjusted to obtain a trained encoder.

[0016] In some embodiments, the method is applied to a client, the client is connected to a terminal for communication, and includes:

[0017] The initial audio QR code sent by the receiving terminal is obtained by mixing the audio coding features and the information coding features; the audio coding features and the information coding features are obtained by separately coding the acquired preset initial audio and the target information to be transmitted;

[0018] Configuring multiple types of distorted audio for the initial audio QR code, and introducing multiple distorted audio into the initial audio QR code to obtain an enhanced audio QR code;

[0019] Decode the enhanced audio QR code to obtain the target information.

[0020] In some embodiments, multiple distorted audios are introduced into the initial audio QR code to obtain an enhanced audio QR code, including:

[0021] Acquire the surrounding scene audio when the receiving terminal sends the initial audio;

[0022] Selecting at least one target distorted audio that matches the scene audio from a plurality of configured distorted audios, introducing at least one target distorted audio into an initial audio QR code, and obtaining an enhanced audio QR code;

[0023] Alternatively, an adjustable probability weight is assigned to each distorted audio, at least one target distorted audio is randomly selected from multiple distorted audios based on the probability weight, and at least one target distorted audio is introduced into the initial audio QR code to obtain an enhanced audio QR code.

[0024] In some embodiments, decoding the enhanced audio QR code to obtain target information includes:

[0025] Extracting at least one sinusoidal signal from the enhanced audio QR code;

[0026] Select one of a plurality of preset decoders that matches the periodic characteristics of the sinusoidal signal, decode the enhanced audio QR code, and obtain target information;

[0027] When there are multiple sinusoidal signals, multiple decoders matching the periodic characteristics of each sinusoidal signal are used to decode the audio QR code respectively to obtain multiple sub-target information;

[0028] Multiple sub-target information is fused to obtain target information.

[0029] In some embodiments, the target information is output by a pre-trained decoder, and the decoder is trained by the following steps, comprising:

[0030] The sample initial audio QR code sent by the receiving terminal, wherein the sample initial audio QR code is obtained by mixing the sample audio coding features and the sample information coding features; the sample audio coding features and the sample information coding features are obtained by separately coding the acquired preset sample initial audio and the sample target information to be transmitted;

[0031] Decoding the sample initial audio QR code to obtain first sample target information;

[0032] Configuring multiple types of sample distorted audio for the sample initial audio QR code, and introducing multiple types of sample distorted audio into the sample initial audio QR code to obtain a sample enhanced audio QR code;

[0033] Decoding the sample enhanced audio QR code to obtain second sample target information;

[0034] Based on the sample initial information and the first sample target information, a first reconstruction loss value is calculated, wherein the sample initial information is used to generate the sample initial audio QR code;

[0035] Based on the sample initial information and the second sample target information, a first distortion loss value is calculated, and based on the first sample target information and the second sample target information, a second distortion loss value is calculated, and the first distortion loss value and the second distortion loss value are superimposed to obtain a second reconstruction loss value;

[0036] According to the first reconstruction loss value and the second reconstruction loss value, a decoder parameter of the decoder is adjusted to obtain a trained decoder.

[0037] In some embodiments, the method further comprises:

[0038] Based on a preset first balancing coefficient, a result of superimposing the time domain loss value and the frequency domain loss value is updated to obtain a first updated loss value, wherein the time domain loss value and the frequency domain loss value are obtained when the terminal trains an encoder for generating an initial audio QR code;

[0039] Based on a preset second balance coefficient, updating a result of superimposing the first reconstruction loss value and the second reconstruction loss value to obtain a second updated loss value;

[0040] Add the first updated loss value and the second updated loss value to get the total loss value;

[0041] According to the total loss value, the encoder parameters of the encoder and the decoder parameters of the decoder are jointly adjusted to obtain the trained encoder and decoder.

[0042] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes an information transmission device based on an audio QR code, the device comprising:

[0043] An acquisition module, used to acquire preset initial audio and target information to be transmitted;

[0044] A separate encoding module, used to separately encode the initial audio and the target information to obtain audio encoding features and information encoding features;

[0045] A hybrid coding module, used for hybrid coding of audio coding features and information coding features to obtain an initial audio QR code;

[0046] The target information determination module is used to send an initial audio QR code to the client so that the client receives the initial audio QR code sent by the terminal, configure various types of distorted audio for the initial audio QR code, and introduce various distorted audio into the initial audio QR code to obtain an enhanced audio QR code, decode the enhanced audio QR code, and obtain the target information.

[0047] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the method of the above-mentioned first aspect when executing the computer program.

[0048] To achieve the above objectives, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0049] The information transmission method, device, equipment and medium of the audio QR code proposed in the present application, when applied to a terminal, communicates with a client, obtains a preset initial audio and a target information to be transmitted; separately encodes the initial audio and the target information to obtain an audio encoding feature and an information encoding feature; mixes and encodes the audio encoding feature and the information encoding feature to obtain an initial audio QR code; sends the initial audio QR code to the client so that the client receives the initial audio QR code sent by the terminal, configures the initial audio QR code with distorted audio of various types, introduces various distorted audio into the initial audio QR code, obtains an enhanced audio QR code, decodes the enhanced audio QR code, and obtains the target information; it can be understood that by introducing various distorted audio into the initial audio QR code received by the client, the client can remove the signal that is the same as or similar to the introduced distorted audio from the audio signal of the initial audio QR code, thereby achieving the purpose of enhancing the audio signal of the initial audio QR code, and decodes the enhanced audio QR code obtained thereafter to obtain the target information with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a schematic diagram of an optional application scenario of the information transmission device based on audio QR code provided in an embodiment of the present application;

[0051] Figure 2 This is an optional flow chart of the information transmission method based on audio QR code provided in the embodiment of the present application;

[0052] Figure 3 It is an optional encoding flow chart of the information transmission method based on audio QR code provided in the embodiment of the present application;

[0053] Figure 4 It is an optional audio encoder structure diagram of the information transmission method based on audio QR code provided in the embodiment of the present application;

[0054] Figure 5 is another optional flow chart of the information transmission method based on audio QR code provided in an embodiment of the present application;

[0055] Figure 6 It is another optional flow chart of the information transmission method based on audio QR code provided in the embodiment of the present application;

[0056] Figure 7 It is an optional decoding flow chart of the information transmission method based on audio QR code provided in the embodiment of the present application;

[0057] Figure 8 yes Figure 6 A flowchart of an implementation of step 302 in FIG.

[0058] Fig. 9 yes Figure 6 A flowchart of an implementation of step 303 in FIG.

[0059] Fig.10 This is another optional flowchart of the information transmission method based on audio QR code provided in the embodiment of the present application;

[0060] Fig.11 This is another optional flow chart of the information transmission method based on audio QR code provided in the embodiment of the present application;

[0061] Fig.12 This is an optional flow chart of an information transmission device based on an audio QR code provided in an embodiment of the present application;

[0062] Fig.13 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] An audio QR code is an audio message that carries QR code information. It is designed to use sound to quickly transmit information such as text, images or files.

[0067] In the related art, the target information is transmitted by reading the audio that carries the QR code information in the audio QR code. However, the noise generated in different reading scenarios can easily interfere with the information transmission, thereby distorting the audio QR code, and ultimately causing the target information obtained based on the audio QR code reading to be inaccurate.

[0068] Based on this, the embodiments of the present application provide an information transmission method, device, equipment and medium based on audio QR code, aiming to improve the accuracy of target information transmitted based on audio QR code.

[0069] It should be noted that in the embodiments of the present application, when it comes to information related to user characteristics such as user basic information or user identity, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the user's separate permission or separate consent will be obtained first. After the user's separate permission or separate consent is clearly obtained, the necessary data for the normal operation of the embodiments of the present application will be obtained. For example, when the target information to be transmitted proposed in the embodiments of the present application, the authorization or consent of the relevant user will be obtained first, otherwise it cannot be applied to the target information of the embodiments of the present application. In addition, the other relevant data obtained by the information transmission device based on audio QR code proposed in the present application (for the convenience of description, it may also be referred to as "information transmission device" below) are all legal data after obtaining authorization or consent, which will not be repeated here.

[0070] The information transmission method, device, equipment and medium based on audio QR code provided in an embodiment of the present application are specifically described through the following embodiments. First, the application scenario of the information transmission device in the embodiment of the present application is described, such as Figure 1 As shown, Figure 1 This is a schematic diagram of an optional application scenario of the information transmission device based on audio QR code provided in an embodiment of the present application, wherein the information transmission device can be applied to a terminal or a client. In one example, the terminal is a computer and the client is a mobile phone. The terminal receives the target information to be transmitted and the initial audio for embedding the target information, and encodes the initial audio and the target information to generate an initial audio QR code; the initial audio QR code can be transmitted through broadcasting or the like and received by the client, and the client decodes the received initial audio QR code. In this way, the target information can be quickly transmitted without manually scanning the QR code with a mobile phone.

[0071] After understanding the example application scenarios of the information transmission device proposed in the present application, in order to further understand the beneficial effects of the information transmission method based on audio QR code proposed in the embodiment of the present application (for the sake of ease of description, it may also be referred to as "information transmission method" below) applied to the information transmission device, the information transmission method proposed in the embodiment of the present application is described below.

[0072] In the embodiments of the present application, the information transmission device will be described from the perspective of the information transmission device, which is applied to the terminal, and the terminal is connected to the client for communication; the information transmission device can be integrated into a computer device, such as a server. Figure 2 As shown, Figure 2 is an optional flowchart of the information transmission method based on audio QR code provided in an embodiment of the present application. Figure 2 The method may include but is not limited to the following steps 101 to 104. When the information transmission device executes the prediction method, the specific process is as follows. It should be noted that the present embodiment Figure 2 The order of step 101 to step 104 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0073] Step 101, obtaining preset initial audio and target information to be transmitted.

[0074] The following is a detailed description of step 101.

[0075] The target information to be transmitted refers to data in the form of text, image or file that needs to be transmitted to the client; or, the target information can also be QR code information in binary format converted from data in the form of text, image or file to be transmitted.

[0076] Among them, the initial audio is a piece of audio prepared in advance by the information transmission device before encoding processing. The initial audio has specific properties (such as length, frequency, sound quality, etc.). The initial audio is usually imperceptible to the human ear. The initial audio will be used as a carrier for subsequent embedding of the target information to be transmitted and will be encoded together with the target information.

[0077] Furthermore, since the information transmission device is set in the terminal, the acquired initial audio and the target information to be transmitted will be received and encoded by the terminal. The terminal can be a device capable of transmitting audio signals, such as a computer, a server, a smart speaker, etc. Of course, the selected terminal can also be adaptively adjusted according to different application scenarios, and the embodiments of the present application do not limit this.

[0078] Furthermore, the number of target information to be transmitted can be one or more. When there are multiple target information, the terminal can encode multiple target information into the initial audio at the same time, so that when the client receives and parses the corresponding initial audio QR code, multiple target information can be presented in sequence.

[0079] Step 102, encode the initial audio and the target information separately to obtain audio encoding features and information encoding features.

[0080] The following is a detailed description of step 102.

[0081] In some embodiments, Figure 3 As shown, Figure 3 It is an optional encoding flow chart of the information transmission method based on audio QR code provided in the embodiment of the present application. After obtaining the initial audio and target information, the initial audio can be input into a pre-trained encoder, and the encoder encodes the initial audio and target information. Among them, the encoder includes an audio encoder, an information encoder and a hybrid encoder. The audio encoder encodes the initial audio to obtain audio encoding features; the information encoder encodes the target information to obtain information encoding features. In this way, the terminal can adopt different encoding methods to encode the initial audio and target information in different data formats separately, and further make the encoded result more compact in structure through hybrid encoding, which increases the difficulty for malicious attackers to destroy the encoding process, thereby improving the security of the encoded features.

[0082] Furthermore, the audio encoder may be a pulse code modulation encoder (PCM), an advanced audio encoder (AAC), etc. Of course, the audio encoder may also be selected according to actual conditions, and the embodiments of the present application do not limit this. The information encoder may be a convolutional neural network-based QR code encoder, an autoencoder-based QR code encoder, a quick response code encoder (Quick Response Code), etc. The information encoder may also be selected according to actual conditions, and the embodiments of the present application do not limit this.

[0083] like Figure 4 As shown, Figure 4 It is an optional audio encoder structure diagram of the information transmission method based on audio two-dimensional code provided by an embodiment of the present application, wherein the audio encoder is composed of five one-dimensional convolutional layers (Conv1D), and each convolutional layer is connected to a leaky rectified linear unit (Leaky ReLU) activation function, and after multiple sequentially connected convolutional layers and leaky rectified linear units, a flattening layer (flatten) and a multilayer perceptron (Multilayer Perceptron, MLP) are also provided. The functions of each layer in the audio encoder are as follows:

[0084] (1) One-dimensional convolution layer: used to process one-dimensional data (such as time series data or audio signals) and extract local features in the data through convolution operations. Each convolution kernel (or filter) slides on the input data and calculates the dot product between the input data and the convolution kernel, thereby generating a feature map that captures information at different frequencies and locations in the input data. Figure 4 The numbers in brackets after Conv1D are adjustable parameters for the convolutional layer. In this way, the convolutional layer can process the input data based on the pre-set parameters.

[0085] (2) Leaky ReLU: Leaky ReLU is a nonlinear activation function used to increase the nonlinear expression capability of audio encoders. Compared with the traditional ReLU activation function, Leaky ReLU allows a small gradient (i.e., "leakage") to pass when the input is less than 0, which helps solve the problem of neuron death caused by input less than 0. Multiple convolutional layers and leaky rectified linear units are connected in sequence to form a hierarchical feature extraction structure. Each convolutional layer can further extract higher-level features from the features extracted by the previous layer, thereby capturing more detailed features in the data.

[0086] (3) Flattening layer: used to convert multi-dimensional input data (such as the feature map output by the convolutional layer) into a one-dimensional vector to obtain data that meets the input requirements of the multi-layer perceptron, so that the subsequent multi-layer perceptron can process the converted one-dimensional vector;

[0087] (4) Multi-layer Perceptron: MLP is used to further learn the latent representation of the data from the features extracted by the flattening layer. Then, the MLP layer can capture the global information in the data and map it to the output space.

[0088] Furthermore, the audio encoder can also use weight normalization technology to decouple the size of the weight tensor from the direction of each one-dimensional convolution layer to accelerate the audio encoder's processing of input data. Finally, the audio encoder outputs audio coding features.

[0089] Furthermore, the encoder structure of the information encoder is similar to that of the audio encoder, which will not be described here. In one example, the initial audio x is input into the audio encoder preset by the terminal, and the target information z (the target information obtained in this example is set to be the QR code information) is input into the information encoder preset by the terminal, where the dimensions of x and z are B·L and B·D respectively, L represents the length of the initial audio filling waveform, D represents the length of the target information, B is the batch size, and B, L, and D can be adaptively adjusted according to the actual situation; then, the audio encoder encodes the initial audio x to obtain the audio encoding feature f x , the information encoder encodes the target information z and obtains the audio coding feature fx Information encoding features f with the same feature dimension z .

[0090] Step 103, perform mixed encoding on the audio encoding features and the information encoding features to obtain an initial audio QR code.

[0091] The following is a detailed description of step 103.

[0092] In some embodiments, Figure 3 As shown, the terminal can also be provided with a hybrid encoder, which is connected to the audio encoder and the information encoder. When the audio encoder and the information encoder output the encoding results, the hybrid encoder can encode the audio encoding features and the information encoding features (f x +f z ) is mixed and encoded to map the mixed representation of audio coding features and information coding features to the waveform perturbation, and obtain the initial audio QR code. The specific processing process is as follows: <1> As shown:

[0093] x'=EF(EA(x)⊕EQR(z)) <1>

[0094] Among them, x represents the initial audio; z represents the target information; x' represents the initial audio QR code output by the encoder; EA represents the audio encoder; EQR represents the information encoder; EF represents the hybrid encoder; ⊕ represents the connection operation.

[0095] Furthermore, the hybrid encoder includes a plurality of encoding processing structure layers, each of which includes a transposed convolution layer (ConvTranspose), which is used to upsample and restore the input feature map until the processed audio waveform δ x When the dimension is the same as the audio waveform of the initial audio x, the initial audio QR code is output.

[0096] Furthermore, in order to increase the versatility of the encoder to adapt to target information of different dimensions, a linear projection layer can be added to the information encoder to map target information of different dimensions to a unified feature dimension.

[0097] Step 104, sending the initial audio QR code to the client, so that the client receives the initial audio QR code sent by the terminal, configuring multiple types of distorted audio for the initial audio QR code, and introducing multiple distorted audio into the initial audio QR code to obtain an enhanced audio QR code, and decoding the enhanced audio QR code to obtain the target information.

[0098] Step 104 is described in detail below.

[0099] Among them, the client can be a mobile communication device, such as a mobile phone, a computer, a tablet, etc. The terminal plays the audio signal carrying the QR code information in the audio QR code to achieve the purpose of sending the initial audio QR code to the client. In this way, the client can easily complete the reception of the audio QR code when the audio receiving component is turned on. For example, when the user wants to use the mobile phone to obtain the target information (such as the target information is the QR code information), the user can transmit the information without scanning the QR code, which greatly facilitates the daily life of the visually impaired, or can realize the rapid transmission of information in dim scenes with insufficient light.

[0100] Furthermore, since the terminal is disturbed by the surrounding environment when sending the initial audio QR code to the client, the audio signal of the initial audio QR code is mixed with other noises, such as but not limited to unnecessary human voices, water sounds, whistle sounds, etc. In order to improve the accuracy of the target information finally decoded by the client, the embodiment of the present application configures multiple types of distorted audio for the initial audio QR code, wherein the distorted audio refers to the noise audio under the preset scene. The client introduces multiple distorted audios into the received initial audio QR code so that the client can remove the same or similar signals as the introduced distorted audio from the audio signal of the initial audio QR code, thereby enhancing the initial audio QR code so that the enhanced audio QR code can be decoded later to obtain the target information with higher accuracy.

[0101] In some embodiments, Figure 5 As shown, Figure 5 This is another optional flow chart of the information transmission method based on the audio QR code provided in the embodiment of the present application. The initial audio QR code is output by a pre-trained encoder, and the encoder is trained by the following steps 201 to 205:

[0102] Step 201: Obtain preset sample initial audio and sample target information to be transmitted.

[0103] Step 202: Encode the sample initial audio and the sample target information separately to obtain the sample audio encoding feature and the sample information encoding feature.

[0104] Step 203, perform mixed encoding on the sample audio encoding features and the sample information encoding features to obtain the sample initial audio QR code.

[0105] Step 204, based on the difference between the sample initial audio and the sample initial audio QR code in the time domain, calculate the time domain loss value, and based on the difference between the sample initial audio and the sample initial audio QR code in the frequency domain, calculate the frequency domain loss value.

[0106] Step 205 , adjusting encoder parameters of the encoder according to the time domain loss value and the frequency domain loss value to obtain a trained encoder.

[0107] Steps 201 to 205 are described in detail below.

[0108] In some embodiments, the encoder in the information transmission device needs to be trained before it is formally deployed to the terminal to ensure the accuracy of its operation in the terminal. Steps 101 to 103 of training the encoder are similar and will not be repeated here. In particular, the sample initial audio and the sample target information to be transmitted can be manually input by the user, or automatically imported from an open source database, and this application does not specifically limit it.

[0109] Furthermore, after obtaining the sample initial audio QR code, the following formula is used <2> Calculate the time domain loss value L ti :

[0110]

[0111] in, is the L1 norm, which refers to the sum of the absolute values ​​of each element in the vector; x represents the initial audio of the sample; x ′ Represents the sample initial audio QR code output by the encoder; the following formula <3> to <5> In the formula <2> Parameters with the same characters have the same meaning and will not be described in detail later.

[0112] Furthermore, by the following formula <3> Calculate the frequency domain loss value L fi :

[0113]

[0114] Here, φ(·) represents a preset function that converts a waveform into a spectrum.

[0115] Furthermore, after obtaining the time domain loss value and the frequency domain loss value, the encoder parameters of the encoder are adjusted based on these two loss values, where the encoder parameters can be network structure parameters, such as the number of encoder layers, the number of neurons, the activation function, etc.; they can also be optimization parameters, such as learning rate, batch size, and regularization parameters, etc.; they can also be feature parameters, such as feature vectors that characterize sparsity in the encoder, feature vectors that characterize quantization accuracy, etc.; of course, the specific adjusted encoder parameters can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0116] Furthermore, since the encoder includes an audio encoder, an information encoder and a hybrid encoder, the adjusted encoder parameters can be any one of the audio encoder, the information encoder and the hybrid encoder, or the encoder parameters of the audio encoder, the information encoder and the hybrid encoder can be jointly adjusted.

[0117] In another embodiment of the present application, the information transmission device is described from the perspective of an information transmission device, which is applied to a client, and the client is connected to a terminal for communication; the information transmission device can also be integrated into a computer device, such as a server. Figure 6 As shown, Figure 6 is another optional flowchart of the information transmission method based on audio QR code provided in the embodiment of the present application. Figure 6 The method in the embodiment may include but is not limited to the following steps 301 to 303. When the information transmission device executes the prediction method, the specific process is as follows. It should be noted that the present embodiment Figure 6 The order of step 301 to step 303 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0118] Step 301, receiving the initial audio QR code sent by the terminal, wherein the initial audio QR code is obtained by mixed encoding of audio encoding features and information encoding features; the audio encoding features and information encoding features are obtained by separately encoding the acquired preset initial audio and the target information to be transmitted.

[0119] Among them, the client and the terminal complete the communication connection in advance before information transmission, so that the client and the terminal can subsequently transmit information through the audio QR code. The communication connection between the client and the terminal includes but is not limited to client-server (C / S) mode connection, connection based on communication protocol, Bluetooth connection and other methods. The embodiment of the application does not limit the specific method of communication connection, and can be adaptively adjusted according to actual conditions.

[0120] Step 302: configure multiple types of distorted audio for the initial audio QR code, and introduce multiple types of distorted audio into the initial audio QR code to obtain an enhanced audio QR code.

[0121] Step 303: decode the enhanced audio QR code to obtain target information.

[0122] Steps 301 to 303 are described in detail below.

[0123] In some embodiments, Figure 7 As shown, Figure 7It is an optional decoding flow chart of the information transmission method based on audio QR code provided in an embodiment of the present application. When receiving the initial audio QR code as input, the decoder introduces multiple distorted audios so that the client can remove signals that are the same as or similar to the introduced distorted audio from the audio signal of the initial audio QR code, thereby enhancing the initial audio QR code so that the enhanced audio QR code can be decoded later to obtain target information with higher accuracy, that is, the robustness of the information transmission device is improved.

[0124] For example, when a visually impaired person uses a mobile phone to receive an audio signal carrying QR code information played by a coffee shop bar, he or she may be affected by the surrounding human voice noise, machine noise and music played in the store. The embodiment of the present application introduces distorted audio to denoise the audio signal of the initial audio QR code. The mobile phone (client) of the visually impaired person can quickly and accurately identify the target information that the bar (terminal) wants to transmit based on the denoised audio signal.

[0125] Among them, the audio types of distorted audio can be: ① environmental background distorted audio (such as animal calls, urban traffic noise, machine engine sounds, external rain sounds); ② room impulse response distorted audio in different places (such as shopping malls, living rooms, kitchens, coffee shops, offices, hospitals); ③ music background distorted audio; ④ Gaussian noise distorted audio, etc.

[0126] It should be noted that the audio type of the distorted audio can be adaptively adjusted according to actual conditions. For example, when the location for transmitting the target information is a school playground, the audio type of the distorted audio can also be the sound of reading, birds singing, etc. The embodiment of the present application does not impose any specific restrictions on this.

[0127] Furthermore, based on the following formula <4> and <5> Complete the enhancement and decoding of the initial audio QR code and obtain the target information z′ aug :

[0128] δ aug =δ e +δ r +δ m +δ g <4>

[0129] z′ aug =D mp (x′+δ aug ) <5>

[0130] Among them, δ e Indicates the ambient background distorted audio, δ r represents the room impulse response distorted audio, δ m Indicates music background distorted audio, δ g Indicates Gaussian noise distorted audio; Dmp is a pre-trained decoder.

[0131] In some embodiments, Figure 8 As shown, Figure 8 yes Figure 6 A flowchart for implementing step 302 in the embodiment of the present invention introduces multiple distorted audios into the initial audio QR code to obtain an enhanced audio QR code, including the following steps 401 to 403:

[0132] Step 401, obtaining the surrounding scene audio when the receiving terminal sends the initial audio.

[0133] Step 402: Select at least one target distorted audio that matches the scene audio from the configured multiple distorted audios, introduce the at least one target distorted audio into the initial audio QR code, and obtain an enhanced audio QR code.

[0134] Step 403, alternatively, assign an adjustable probability weight to each distorted audio, randomly select at least one target distorted audio from multiple distorted audios based on the probability weight, introduce at least one target distorted audio into the initial audio QR code, and obtain an enhanced audio QR code.

[0135] Steps 401 to 403 are described in detail below.

[0136] In some embodiments, in order to improve the efficiency of the decoder in generating enhanced audio QR codes, the scene audio around the client receiving terminal when the initial audio is sent can be obtained, and at least one target distorted audio that matches the scene audio can be selected from multiple distorted audios to enhance the initial audio QR code.

[0137] Exemplarily, the distorted audio includes environmental background distorted audio, room impulse response distorted audio, music background distorted audio and Gaussian noise distorted audio. When a visually impaired person uses a mobile phone to receive the initial audio QR code sent by the coffee shop bar, the coffee shop is playing music, and the music background interferes with the audio signal transmission of the initial audio QR code. At this time, the acquired scene audio includes music noise, so the information transmission device can determine that the music background distorted audio is the target distorted audio that matches the scene audio, and introduce the target distorted audio into the initial audio QR code to obtain an enhanced audio QR code. Of course, when it is identified that there are multiple distorted audios that match the scene audio, the number of target distorted audios is multiple.

[0138] In other embodiments, the scene audio obtained in actual applications usually includes multiple types and quantities of noise. In this case, multiple target distorted audios matching the scene audio can be selected from the multiple configured distorted audios, and the multiple target distorted audios can be introduced into the initial audio QR code to obtain an enhanced audio QR code. Specifically, by the following formula: <6> Select the target distorted audio from the configured multiple distorted audios δ aug :

[0139]

[0140] in, Represented by probability weight p e To distort audio e For sampling, and Use a similar sampling approach.

[0141] Furthermore, the probability weight p can be adaptively adjusted according to actual conditions. For example, the information transmission device can receive the scene audio within a preset period of time, and readjust each probability weight according to the duration of each distorted audio in the scene audio to obtain an updated formula: <6> , based on the updated formula <6> At least one target distorted audio is randomly selected from multiple distorted audios; then, based on the sum formula <5> In a similar manner, the initial audio QR code after the target distorted audio is introduced is enhanced to obtain an enhanced audio QR code.

[0142] It can be understood that randomly cascading different distorted audios can better remove noise in the real world, thereby enhancing the robustness of the information transmission device and improving the generalization ability of the information transmission device in dealing with different initial audio QR codes.

[0143] In some embodiments, Fig. 9 As shown, Fig. 9 yes Figure 6 A flowchart for implementing step 303 in the embodiment of the present invention is provided, wherein the enhanced audio QR code is decoded to obtain target information, including the following steps 501 to 504:

[0144] Step 501: extract at least one sinusoidal signal from the enhanced audio QR code.

[0145] Step 502: Select one of a plurality of preset decoders that matches the periodic characteristics of the sinusoidal signal, decode the enhanced audio QR code, and obtain target information.

[0146] Step 503: When there are multiple sinusoidal signals, the audio two-dimensional code is decoded respectively based on multiple decoders matching the periodic characteristics of each sinusoidal signal to obtain multiple sub-target information.

[0147] Step 504, fusing multiple sub-target information to obtain target information.

[0148] Steps 501 to 504 are described in detail below.

[0149] In some embodiments, since the audio signal of the enhanced audio QR code is composed of a plurality of sinusoidal signals of different frequencies, different amplitudes and different phases, the audio signal of the enhanced audio QR code is first parsed to extract at least one sinusoidal signal; then, one of the multiple decoders is selected to decode the corresponding target information, which can improve the accuracy and efficiency of decoding. Specifically, the process of decoding based on the pre-trained decoder to obtain the target information z′ is as follows: <7> As shown:

[0150] z′=D mp (x′) <7>

[0151] Where x′ is used to represent the enhanced audio QR code; D mp is a pre-trained decoder.

[0152] Furthermore, the periodic feature refers to the time period used by a sinusoidal signal to repeat a complete sinusoidal wave, that is, the length of time required from a certain phase point to the next same phase point in the sinusoidal signal. The sinusoidal signals of different sound signals are usually different.

[0153] Furthermore, a sinusoidal signal can be separated from the audio signal of the enhanced audio QR code by using filtering, Fourier transform and other methods; correspondingly, the decoder for decoding the enhanced audio QR code can be a filtering decoder, a Fourier transform-based decoder, etc.; of course, the method for extracting the sinusoidal signal and the decoder used for decoding can be adjusted according to actual conditions, and the embodiments of the present application do not limit this.

[0154] In some embodiments, the pre-trained decoder is composed of multiple sub-decoders, each of which is composed of a set of hierarchical convolutional layers with Leaky ReLU activation. When the audio signal of the enhanced audio QR code includes multiple sinusoidal signals, each sub-decoder will process the audio of different sinusoidal signals to obtain multiple sub-target information; then, the multiple sub-target information will be stacked and average-pooled by a preset activation function (such as a sigmoid function) to finally obtain the fused and reconstructed target information z′.

[0155] In some embodiments, Fig.10 As shown, Fig.10This is another optional flow chart of the information transmission method based on audio QR code provided in the embodiment of the present application. The target information is output by a pre-trained decoder, and the decoder is trained by following steps 601 to 607:

[0156] Step 601, receiving a sample initial audio QR code sent by a terminal, wherein the sample initial audio QR code is obtained by mixing and encoding a sample audio coding feature and a sample information coding feature. The sample audio coding feature and the sample information coding feature are obtained by separately encoding the acquired preset sample initial audio and the sample target information to be transmitted.

[0157] Step 602: decode the sample initial audio QR code to obtain first sample target information.

[0158] Step 603, configure multiple types of sample distorted audio for the sample initial audio QR code, and introduce multiple types of sample distorted audio into the sample initial audio QR code to obtain a sample enhanced audio QR code.

[0159] Step 604: decode the sample enhanced audio QR code to obtain second sample target information.

[0160] Step 605: Calculate a first reconstruction loss value based on the sample initial information and the first sample target information, wherein the sample initial information is used to generate a sample initial audio QR code.

[0161] Step 606, based on the sample initial information and the second sample target information, a first distortion loss value is calculated, and based on the first sample target information and the second sample target information, a second distortion loss value is calculated, and the first distortion loss value and the second distortion loss value are superimposed to obtain a second reconstruction loss value.

[0162] Step 607: Adjust decoder parameters of the decoder according to the first reconstruction loss value and the second reconstruction loss value to obtain a trained decoder.

[0163] Steps 601 to 607 are described in detail below.

[0164] In some embodiments, the decoder in the information transmission device needs to be trained before it is formally deployed to the client to ensure the accuracy of its operation on the client. Steps 601 to 604 of training the decoder are similar to steps 301 to 303 and are not repeated here. In particular, in order to enhance the robustness of the decoder, corresponding output results are obtained by introducing distorted audio and not introducing distorted audio (step 602), that is, the first sample target information and the second sample target information to determine the loss value of adjusting the decoder parameters.

[0165] Furthermore, without introducing distorted audio, the following formula is used <8> Calculate the first reconstruction loss value L v :

[0166]

[0167] Among them, D mp For a pre-built decoder, x ′ is the sample initial audio QR code, z1 is the first sample target information, is the L1 norm.

[0168] Furthermore, when distorted audio is introduced, the following formula is used: <9> Calculate the second reconstruction loss value L a :

[0169]

[0170] Among them, D mp For a pre-built decoder, x ′ +δ aug To enhance the initial audio QR code, z2 is the second sample target information, is the L1 norm; is the first distortion loss value, is the second distortion loss value.

[0171] Furthermore, after obtaining the first reconstruction loss value and the second reconstruction loss value, the decoder parameters of the decoder are adjusted based on the two loss values, wherein the decoder parameters may be network structure parameters, such as the number of decoder layers, the number of neurons, the activation function, etc.; they may also be optimization parameters, such as the learning rate, the batch size, and the regularization parameter, etc.; they may also be feature parameters, such as the feature vector that characterizes the sparsity in the decoder, the feature vector that characterizes the quantization accuracy, etc.; of course, the specific adjusted decoder parameters may be set according to actual conditions, and the embodiments of the present application do not impose any restrictions on this.

[0172] Furthermore, different loss weights can be set for the first reconstruction loss value and the second reconstruction loss value, respectively, and the first reconstruction loss value and the second reconstruction loss value can be updated according to the loss weights. After that, the decoder parameters of the decoder can be adjusted based on the updated first reconstruction loss value and the second reconstruction loss value to obtain a trained decoder.

[0173] In some embodiments, Fig.11 As shown, Fig.11 This is another optional flowchart of the information transmission method based on the audio QR code provided in the embodiment of the present application. The information transmission method also includes the following steps 701 to 704:

[0174] Step 701, based on a preset first balance coefficient, the result of superimposing the time domain loss value and the frequency domain loss value is updated to obtain a first updated loss value, wherein the time domain loss value and the frequency domain loss value are obtained by the terminal when training the encoder to generate the initial audio QR code.

[0175] Step 702: Based on a preset second balance coefficient, the result of superimposing the first reconstruction loss value and the second reconstruction loss value is updated to obtain a second updated loss value.

[0176] Step 703: superimpose the first updated loss value and the second updated loss value to obtain a total loss value.

[0177] Step 704 , jointly adjusting the encoder parameters of the encoder and the decoder parameters of the decoder according to the total loss value to obtain the trained encoder and decoder.

[0178] Steps 701 to 704 are described in detail below.

[0179] In some embodiments, by the formula <10> Calculate the total loss value L total :

[0180] L total =λ imp (L ti +L fi )+λ mr (L v +L a ) <10>

[0181] Among them, λ imp is the first balance coefficient, λ mr is the second balance coefficient; imp (L ti +L fi ) is the first update loss value, λ mr (L v +L a ) is the second update loss value; the first balance coefficient and the second balance coefficient can be adaptively adjusted according to actual conditions.

[0182] In this way, the encoder parameters and decoder parameters that are associated with each other are jointly adjusted based on the total loss value to achieve collaborative optimization of the encoder and decoder, thereby improving the performance of the information transmission devices deployed on the terminal and the client, respectively. Exemplarily, the encoder parameters and the decoder parameters can be updated simultaneously in each iteration of the training process by the gradient descent method.

[0183] like Fig.12 As shown, Fig.12This is an optional flow chart of an information transmission device based on an audio QR code provided in an embodiment of the present application. The information transmission device includes the following modules 801 to 804:

[0184] The acquisition module 801 is used to acquire the preset initial audio and the target information to be transmitted.

[0185] The separate encoding module 802 is used to separately encode the initial audio and the target information to obtain audio encoding features and information encoding features.

[0186] The hybrid coding module 803 is used to perform hybrid coding on the audio coding features and the information coding features to obtain an initial audio QR code.

[0187] The target information determination module 804 is used to send an initial audio QR code to the client so that the client receives the initial audio QR code sent by the terminal, configure multiple types of distorted audio for the initial audio QR code, and introduce multiple distorted audio into the initial audio QR code to obtain an enhanced audio QR code, decode the enhanced audio QR code, and obtain the target information.

[0188] In some embodiments, the target information to be transmitted refers to data in the form of text, image or file that needs to be transmitted to the client; or, the target information can also be QR code information in binary format converted from data in the form of text, image or file to be transmitted.

[0189] Among them, the initial audio is a piece of audio prepared in advance by the information transmission device before encoding processing. The initial audio has specific properties (such as length, frequency, sound quality, etc.). The initial audio is usually imperceptible to the human ear. The initial audio will be used as a carrier for subsequent embedding of the target information to be transmitted and will be encoded together with the target information.

[0190] Furthermore, since the information transmission device is set in the terminal, the acquired initial audio and the target information to be transmitted will be received and encoded by the terminal. The terminal can be a device capable of transmitting audio signals, such as a computer, a server, a smart speaker, etc. Of course, the selected terminal can also be adaptively adjusted according to different application scenarios, and the embodiments of the present application do not limit this.

[0191] Furthermore, the number of target information to be transmitted can be one or more. When there are multiple target information, the terminal can encode multiple target information into the initial audio at the same time, so that when the client receives and parses the corresponding initial audio QR code, multiple target information can be presented in sequence.

[0192] In some embodiments, after obtaining the initial audio and target information, the initial audio can be input into a pre-trained encoder, and the encoder encodes the initial audio and target information. The encoder includes an audio encoder, an information encoder, and a hybrid encoder. The audio encoder encodes the initial audio to obtain audio encoding features; the information encoder encodes the target information to obtain information encoding features. In this way, the terminal can use different encoding methods to encode the initial audio and target information in different data formats separately, and further use hybrid encoding to make the encoded result more compact in structure, increase the difficulty for malicious attackers to destroy the encoding process, and thus improve the security of the encoded features.

[0193] In some embodiments, the terminal may also be provided with a hybrid encoder, which is connected to the audio encoder and the information encoder. When the audio encoder and the information encoder output encoding results, the hybrid encoder can perform mixed encoding on the audio coding features and the information coding features to map the mixed representation of the audio coding features and the information coding features onto the waveform perturbation and obtain an initial audio QR code.

[0194] Furthermore, the hybrid encoder includes multiple encoding processing structure layers, each encoding processing structure layer includes a transposed convolution layer, and the transposed convolution layer is used to upsample and restore the input feature map until the processed audio waveform has the same dimension as the audio waveform of the initial audio, and then output the initial audio QR code.

[0195] Furthermore, in order to increase the versatility of the encoder to adapt to target information of different dimensions, a linear projection layer can be added to the information encoder to map target information of different dimensions to a unified feature dimension.

[0196] Among them, the client can be a mobile communication device, such as a mobile phone, a computer, a tablet, etc. The terminal plays the audio signal carrying the QR code information in the audio QR code to achieve the purpose of sending the initial audio QR code to the client. In this way, the client can easily complete the reception of the audio QR code when the audio receiving component is turned on. For example, when the user wants to use the mobile phone to obtain the target information (such as the target information is the QR code information), the user can transmit the information without scanning the QR code, which greatly facilitates the daily life of the visually impaired, or can realize the rapid transmission of information in dim scenes with insufficient light.

[0197] Furthermore, since the terminal is disturbed by the surrounding environment when sending the initial audio QR code to the client, the audio signal of the initial audio QR code is mixed with other noises, such as but not limited to unnecessary human voices, water sounds, whistle sounds, etc. In order to improve the accuracy of the target information finally decoded by the client, the embodiment of the present application configures multiple types of distorted audio for the initial audio QR code, wherein the distorted audio refers to the noise audio under the preset scene. The client introduces multiple distorted audios into the received initial audio QR code so that the client can remove the same or similar signals as the introduced distorted audio from the audio signal of the initial audio QR code, thereby enhancing the initial audio QR code so that the enhanced audio QR code can be decoded later to obtain the target information with higher accuracy.

[0198] The information transmission method, device, equipment and medium of the audio QR code proposed in the present application, when applied to a terminal, communicates with a client, obtains a preset initial audio and a target information to be transmitted; separately encodes the initial audio and the target information to obtain an audio encoding feature and an information encoding feature; mixes and encodes the audio encoding feature and the information encoding feature to obtain an initial audio QR code; sends the initial audio QR code to the client so that the client receives the initial audio QR code sent by the terminal, configures the initial audio QR code with distorted audio of various types, introduces various distorted audio into the initial audio QR code, obtains an enhanced audio QR code, decodes the enhanced audio QR code, and obtains the target information; it can be understood that by introducing various distorted audio into the initial audio QR code received by the client, the client can remove the signal that is the same as or similar to the introduced distorted audio from the audio signal of the initial audio QR code, thereby achieving the purpose of enhancing the audio signal of the initial audio QR code, and decodes the enhanced audio QR code obtained thereafter to obtain the target information with higher accuracy.

[0199] The specific implementation of the information transmission device is basically the same as the specific implementation of the above-mentioned information transmission method, and will not be repeated here.

[0200] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above information transmission method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0201] like Fig.13 As shown, Fig.13 : is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application, the electronic device includes:

[0202] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0203] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the information transmission method of the embodiment of this application;

[0204] Input / output interface 903, used to implement information input and output;

[0205] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0206] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0207] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0208] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned information transmission method is implemented.

[0209] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0210] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0211] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0212] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0213] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0214] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0215] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0216] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0217] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0218] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0219] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0220] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. An information transmission method based on audio QR code, characterized in that: Applied to a terminal, the terminal is connected to a client for communication, and the method comprises: Obtaining the preset initial audio and target information to be transmitted; Separately encoding the initial audio and the target information to obtain audio coding features and information coding features; Mixing and encoding the audio coding feature and the information coding feature to obtain an initial audio QR code; The initial audio QR code is sent to the client so that the client receives the initial audio QR code sent by the terminal, multiple types of distorted audio are configured for the initial audio QR code, and multiple types of distorted audio are introduced into the initial audio QR code to obtain an enhanced audio QR code, and the enhanced audio QR code is decoded to obtain the target information.

2. The method according to claim 1, characterized in that The initial audio QR code is output by a pre-trained encoder, and the encoder is trained by the following steps, the steps comprising: Obtaining preset sample initial audio and sample target information to be transmitted; Separately encoding the sample initial audio and the sample target information to obtain sample audio encoding features and sample information encoding features; Mixing and encoding the sample audio coding feature and the sample information coding feature to obtain a sample initial audio QR code; Based on the difference between the sample initial audio and the sample initial audio QR code in the time domain, a time domain loss value is calculated, and based on the difference between the sample initial audio and the sample initial audio QR code in the frequency domain, a frequency domain loss value is calculated; According to the time domain loss value and the frequency domain loss value, the encoder parameters of the encoder are adjusted to obtain the trained encoder.

3. An information transmission method based on audio QR code, characterized in that: Applied to a client, the client is connected to a terminal for communication, and the method comprises: An initial audio QR code sent by a receiving terminal, wherein the initial audio QR code is obtained by mixing audio coding features and information coding features; the audio coding features and the information coding features are obtained by separately coding the acquired preset initial audio and the target information to be transmitted; Configuring multiple types of distorted audio for the initial audio QR code, and introducing multiple types of distorted audio into the initial audio QR code to obtain an enhanced audio QR code; The enhanced audio two-dimensional code is decoded to obtain the target information.

4. The method according to claim 3, characterized in that: The step of introducing a plurality of distorted audios into the initial audio two-dimensional code to obtain an enhanced audio two-dimensional code comprises: Acquire the surrounding scene audio when the receiving terminal sends the initial audio; Selecting at least one target distorted audio that matches the scene audio from the configured multiple distorted audios, introducing at least one target distorted audio into the initial audio QR code, and obtaining an enhanced audio QR code; Alternatively, an adjustable probability weight is assigned to each of the distorted audios, at least one target distorted audio is randomly selected from the multiple distorted audios based on the probability weight, and at least one target distorted audio is introduced into the initial audio QR code to obtain an enhanced audio QR code.

5. The method according to claim 3, characterized in that: The step of decoding the enhanced audio two-dimensional code to obtain the target information includes: Extracting at least one sinusoidal signal from the enhanced audio QR code; Selecting one that matches the periodic characteristics of the sinusoidal signal from a plurality of preset decoders, decoding the enhanced audio QR code, and obtaining the target information; When there are multiple sinusoidal signals, the audio two-dimensional code is decoded respectively based on multiple decoders matching the periodic characteristics of each sinusoidal signal to obtain multiple sub-target information; The plurality of sub-target information are fused to obtain the target information.

6. The method according to claim 5, characterized in that The target information is output by a pre-trained decoder, and the decoder is trained by the following steps, the steps comprising: Receive a sample initial audio QR code sent by the terminal, wherein the sample initial audio QR code is obtained by mixing and encoding a sample audio encoding feature and a sample information encoding feature; the sample audio encoding feature and the sample information encoding feature are obtained by separately encoding an acquired preset sample initial audio and a sample target information to be transmitted; Decoding the sample initial audio QR code to obtain first sample target information; Configuring multiple types of sample distorted audio for the sample initial audio QR code, and introducing multiple types of sample distorted audio into the sample initial audio QR code to obtain a sample enhanced audio QR code; Decoding the sample enhanced audio QR code to obtain second sample target information; Based on the sample initial information and the first sample target information, a first reconstruction loss value is calculated, wherein the sample initial information is used to generate the sample initial audio QR code; Based on the sample initial information and the second sample target information, a first distortion loss value is calculated, and based on the first sample target information and the second sample target information, a second distortion loss value is calculated, and the first distortion loss value and the second distortion loss value are superimposed to obtain a second reconstruction loss value; According to the first reconstruction loss value and the second reconstruction loss value, the decoder parameters of the decoder are adjusted to obtain the trained decoder.

7. The method according to claim 6, characterized in that The method further comprises: Based on a preset first balancing coefficient, updating a result of superimposing a time domain loss value and a frequency domain loss value to obtain a first updated loss value, wherein the time domain loss value and the frequency domain loss value are obtained by the terminal when training an encoder for generating the initial audio QR code; Based on a preset second balancing coefficient, updating a result of superimposing the first reconstruction loss value and the second reconstruction loss value to obtain a second updated loss value; Add the first updated loss value and the second updated loss value to obtain a total loss value; According to the total loss value, encoder parameters of the encoder and decoder parameters of the decoder are jointly adjusted to obtain the trained decoder and encoder.

8. An information transmission device based on audio QR code, characterized in that: The device comprises: An acquisition module, used to acquire preset initial audio and target information to be transmitted; A separate encoding module, used to separately encode the initial audio and the target information to obtain audio encoding features and information encoding features; A hybrid encoding module, used for hybrid encoding the audio encoding feature and the information encoding feature to obtain an initial audio QR code; The target information determination module is used to send the initial audio QR code to the client so that the client receives the initial audio QR code sent by the terminal, configure multiple types of distorted audio for the initial audio QR code, and introduce multiple types of distorted audio into the initial audio QR code to obtain an enhanced audio QR code, decode the enhanced audio QR code, and obtain the target information.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images