5g rich media message speech-to-text system, method, electronic device, and medium

Abstract

Embodiments of the present application disclose a 5G rich media message voice-to-text system, method, electronic device and medium, comprising: obtaining video information in a 5G rich media message, constructing a video message information set Y based on the video information, and constructing a training set based on the video message information set Y; constructing an improved CTC model, wherein the improved CTC model is an improved CTC original loss function in the CTC model by introducing a maximum entropy function based on the CTC model; training the improved CTC model through the training set to obtain a trained improved CTC model; and converting the video message information set Y into a video text set y through the trained improved CTC model. The 5G rich media message voice-to-text method solves the problem of increasing CTC model recognition period with increasing noise in the voice environment in the prior art.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a 5G rich media messaging voice-to-text system, method, electronic device, and medium. Background Technology

[0002] 5G rich media messaging represents a significant leap forward in the communication capabilities of the SMS industry. Compared to traditional text messages, 5G rich media messaging supports more media formats and offers richer presentation. It can send not only long text, images, voice, and video, but also user interaction and feedback capabilities such as official accounts and mini-programs, greatly enhancing the application scenarios, content quality, and scope of use of 5G rich media messaging.

[0003] 5G rich media messages are often found in marketing video materials, with background audio mixed with a lot of noise. While traditional CTC (Content-Based Transcription) decoding schemes effectively solve the misalignment problem in attention-based speech recognition methods, the probability of feasible paths learned by the traditional CTC model through maximum likelihood estimation increases exponentially with the increase of noise in the speech environment. Furthermore, once the traditional CTC model finds the main feasible path, error messages will concentrate around that path, drastically amplifying the recognition cycle. Considering the efficiency and timeliness of 5G rich media message delivery, deep learning neural network algorithms with excessively long processing times cannot be chosen for 5G rich media message speech-to-text conversion.

[0004] Therefore, there is an urgent need for a method to convert 5G rich media messages from speech to text with a shorter processing time. Summary of the Invention

[0005] The purpose of this invention is to provide a 5G rich media message voice-to-text system, method, electronic device and medium to solve the problem in the prior art that the recognition cycle of CTC model increases with the increase of noise in the voice environment.

[0006] To achieve the above objectives, embodiments of the present invention provide a 5G rich media message voice-to-text method, the method specifically including:

[0007] Obtain video information from 5G rich media messages, construct a video message information set Y based on the video information, and construct a training set based on the video message information set Y;

[0008] An improved CTC model is constructed, wherein the improved CTC model is based on the CTC model by introducing a maximum entropy function to improve the original CTC loss function in the CTC model;

[0009] The improved CTC model is trained using the training set to obtain a trained improved CTC model;

[0010] The improved CTC model, after being trained, converts the video message information set Y into a video text set y.

[0011] Based on the above technical solution, the present invention can be further improved as follows:

[0012] Furthermore, the 5G rich media message voice-to-text method also includes:

[0013] Retrieve text messages from 5G rich media messages;

[0014] Construct a text message information set X based on the text message and the video text set y;

[0015] Construct a sensitive word variant recognition model;

[0016] The text messages in the text message information set X are sequentially input into the sensitive word variant recognition model to determine whether all text messages in the text message information set X are compliant.

[0017] Furthermore, the 5G rich media message voice-to-text method also includes:

[0018] Obtain image messages from 5G rich media messages, and construct an image message information set Z based on the image messages;

[0019] Construct a model for classifying and extracting harmful images;

[0020] Based on the feature extraction model, feature extraction is performed on the image message information set Z to obtain the image depth feature set z;

[0021] The images in the image depth feature set z are sequentially input into the bad image classification model to determine whether all images in the image depth feature set z are compliant.

[0022] Furthermore, the 5G rich media message voice-to-text method also includes:

[0023] When all text messages in the text message information set X are compliant, and all images in the image depth feature set z are compliant, it is determined that the 5G rich media message can be sent normally.

[0024] Furthermore, the construction of the improved CTC model, wherein the improved CTC model is based on the CTC model by introducing a maximum entropy function to improve the original CTC loss function in the CTC model, including:

[0025] The original loss function of CTC is improved using Formula 1;

[0026]

[0027] In the formula, The loss function of the improved CTC model is... Let α be the original loss function of CTC, α be the coefficient of maximum conditional entropy regularization, and H(p(π|l,X)) be the entropy of the feasible path given the input sequence and the target sequence.

[0028] Furthermore, the construction of the improved CTC model, wherein the improved CTC model is based on the CTC model by introducing a maximum entropy function to improve the original CTC loss function in the CTC model, including:

[0029] Solve for H(p(π|l,X)) using Formula 2;

[0030]

[0031] In the formula, (p(π|l,X)) represents the conditional probability of a feasible path π given 5G voice information X and the actual output I;

[0032] logp(π|X) represents the logarithm of the conditional probability of a feasible path π given 5G voice information X; It represents the sum of all output conditional probabilities of 5G voice information X, regardless of whether the actual output I is given.

[0033] Further, the step of training the improved CTC model using the training set to obtain the trained improved CTC model includes:

[0034] The video message information set Y is divided into a training set, a test set, and a validation set;

[0035] The improved CTC model is trained based on the training set.

[0036] The improved CTC model is validated based on the validation set, and the improved CTC model that meets the performance conditions is saved.

[0037] The conversion results of the improved CTC model are evaluated based on the test set.

[0038] A 5G rich media messaging voice-to-text system includes:

[0039] The acquisition module is used to acquire video information from 5G rich media messages;

[0040] The first construction module is used to construct a video message information set Y based on the video information, and to construct a training set based on the video message information set Y;

[0041] An improved CTC model is proposed, wherein the improved CTC model is based on the CTC model and introduces a maximum entropy function to improve the original CTC loss function in the CTC model;

[0042] The training module is used to train the improved CTC model using the training set to obtain the trained improved CTC model.

[0043] The improved CTC model, after being trained, converts the video message information set Y into a video text set y.

[0044] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the steps of the method described herein.

[0045] A non-transitory computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0046] The embodiments of the present invention have the following advantages:

[0047] This invention discloses a 5G rich media message speech-to-text method. It acquires video information from 5G rich media messages, constructs a video message information set Y based on the video information, and builds a training set based on the video message information set Y. An improved CTC model is constructed, wherein the improved CTC model is based on the original CTC model by introducing a maximum entropy function to improve the original CTC loss function. The improved CTC model is trained using the training set to obtain a trained improved CTC model. The trained improved CTC model is then used to convert the video message information set Y into a video text set y. This solves the problem in existing technologies where the recognition cycle of the CTC model increases with increasing noise in the speech environment. Attached Figure Description

[0048] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0049] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0050] Figure 1 This is a flowchart of the 5G rich media message voice-to-text method of the present invention;

[0051] Figure 2 This is the first architecture diagram of the 5G rich media message voice-to-text system of the present invention;

[0052] Figure 3 This is a second architecture diagram of the 5G rich media message voice-to-text system of the present invention;

[0053] Figure 4 This is a schematic diagram showing the experimental results of comparing the loss function of the present invention;

[0054] Figure 5 This is a schematic diagram of the physical structure of the electronic device provided by the present invention.

[0055] The attached figures are labeled as follows:

[0056] Acquisition module 10, first construction module 20, improved CTC model 30, sensitive word variant recognition model 40, feature extraction model 50, bad image classification model 60, electronic device 70, processor 701, memory 702, bus 703. Detailed Implementation

[0057] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example

[0059] Figure 1 This is a flowchart of an embodiment of the 5G rich media message voice-to-text method of the present invention, as shown below. Figure 1 As shown, the 5G rich media message voice-to-text method provided in this embodiment of the invention includes the following steps:

[0060] S101, acquire video information from 5G rich media messages, construct video message information set Y based on video information, and construct training set based on video message information set Y;

[0061] Specifically, when acquiring 5G rich media messages, since the maximum size of a single 5G rich media message is 3MB, it is possible to include multiple text messages, multiple images, and multiple video / audio clips within a single 5G rich media message. A single 5G rich media message can be represented as T... xyz T can contain multiple sets of text messages X(x1,x2,...), multiple sets of video messages Y(y1,y2,...), and multiple sets of image messages Z(z1,z2,...).

[0062] The video message information set Y contains video content and audio content. In security compliance review, both video and audio information need to be reviewed. The video message information set Y(y1,y2,...) is transformed into a video text set y and a video sample image set y', respectively, and a training set is constructed based on the video message information set Y.

[0063] S102, Construct an improved CTC model, wherein the improved CTC model is based on the CTC model by introducing a maximum entropy function to improve the original CTC loss function in the CTC model;

[0064] Specifically, CTC (Connectionist Temporal Classification) can be understood as temporal classification based on neural networks; assuming 5G voice information is a sequence of length T, X... 1:T Then, the expression for the conditional probability of the true output l given by the CTC speech-to-text model is:

[0065]

[0066] In the formula, π represents the label path that the CTC model can observe in all concatenated information through maximum likelihood estimation; B is the set of many-to-one mappings from all feasible paths π to the true output l; p(π|X 1:T X is the input sequence for 5G voice information. 1:T The conditional probability of all feasible paths π; The loss encountered when choosing this path π is exponentially increased when there is too much noise in the 5G voice information.

[0067]

[0068] This represents the 5G voice information input sequence X under all feasible paths. 1：T The summation of the conditional probabilities of all feasible paths; the summation result p(I|X)1：T ) equals the 5G voice information input sequence X 1：T The conditional probability of the true input I; the core of the CTC model is to find the best speech recognition effect by solving for the maximum of this conditional probability.

[0069] CTC original loss function The reason it exhibits exponential growth when noise is too high is that its solution process involves calculating the conditional probability p(I|X) of all feasible paths π. 1：T Take the logarithm, that is:

[0070]

[0071] When there is a lot of noise in the 5G voice message If it expands exponentially, then solving for the maximum conditional probability in CTC will take a huge amount of time.

[0072] The original loss function of CTC is improved using Formula 1;

[0073]

[0074] In the formula, The loss function of the improved CTC model 30 is... Let α be the original loss function of CTC, α be the coefficient of maximum conditional entropy regularization, and H(p(π|l,X)) be the entropy of the feasible path given the input sequence and the target sequence.

[0075] like Figure 4 To compare the experimental results of the loss function, it can be understood that the original exponential loss function of CTC was used, but with a fluctuation entropy subtracted, thus making the loss function proposed in this patent... The range of values ​​for is reduced to 0 and This avoids the problem of infinite expansion of the loss function caused by a large amount of noise in 5G voice information. The video text set y can be obtained quickly and accurately from the video message information set Y.

[0076] Solve for H(p(π|l,X)) using Formula 2;

[0077]

[0078] In the formula, (p(π|l,X)) represents the conditional probability of a feasible path π given 5G voice information X and the actual output I;

[0079] logp(π|X) represents the logarithm of the conditional probability of a feasible path π given 5G voice information X; It represents the sum of all output conditional probabilities of 5G voice information X, regardless of whether the actual output I is given.

[0080] The loss function used in this invention can be L1Loss, MSEloss, CrossEntropyLoss, etc., and will not have a significant difference in the final effect of improving the CTC model 30.

[0081] S103, The improved CTC model is trained using the training set to obtain the trained improved CTC model;

[0082] Specifically, the video message information set Y is divided into a training set, a test set, and a validation set;

[0083] The improved CTC model 30 is trained based on the training set; the performance of the improved CTC model 30 is verified based on the validation set, and the improved CTC model 30 that meets the performance conditions is saved; the conversion result of the improved CTC model 30 is evaluated based on the test set.

[0084] S104, using the trained improved CTC model, transforms the video message information set Y into a video text set y.

[0085] The 5G rich media message voice-to-text method also includes:

[0086] Obtain text messages from 5G rich media messages; construct a text message information set X based on the text messages and the video text set y;

[0087] A sensitive word variant recognition model 40 is constructed; preferably, the sensitive word variant recognition model 40 is a Text CNN model. The method of identifying bad short texts based on the Text CNN model has been used in short message text review in a relatively mature manner.

[0088] The sensitive word variant recognition model 40 used in this invention can be replaced by models such as CRNN and LSTM+CTC, in addition to the Text CNN model, without significant difference in recognition performance.

[0089] First, the 5G rich media messages to be processed need to undergo preprocessing such as standardization of digital characters, standardization of English characters, conversion of traditional Chinese to simplified Chinese, processing of special symbols, removal of mixed symbol noise, unified representation of continuous digital payments, and string segmentation.

[0090] Secondly, the short text is vectorized using word2vec, and the text vectors are then subjected to high-dimensional convolution and stretching in convolutional layers. Pooling layers and fully connected layers are used to activate the vectors of sensitive words, and the probability of hitting sensitive words is calculated using the SoftMax function. The expression for the SoftMax function chosen here is as follows:

[0091] Where x represents a word vector.

[0092] Finally, the text messages in the text message information set X are sequentially input into the sensitive word variant recognition model 40 to determine whether all text messages in the text message information set X are compliant. If a text message is determined to be non-compliant, it is transferred to manual judgment or warning. If a text message is determined to be compliant, it proceeds to the subsequent judgment process.

[0093] Obtain image messages from 5G rich media messages, and construct an image message information set Z based on the image messages;

[0094] Construct a bad image classification model 60 and a feature extraction model 50;

[0095] Based on the feature extraction model 50, feature extraction is performed on the image message information set Z to obtain the image depth feature set z; preferably, the original image feature extraction method used in this invention is LBP, HOG, or SIFT, and other similar feature extraction algorithms can be used to replace them. The replacement effect will not have a significant impact on the final poor image classification model 60.

[0096] The images in the image depth feature set z are sequentially input into the bad image classification model 60 to determine whether all images in the image depth feature set z are compliant. If an image or any feature information within an image is determined to be non-compliant, then the image is determined to be non-compliant. If an image or any feature information within an image is determined to be compliant, then the image is determined to be compliant.

[0097] When all text messages in the text message information set X are compliant, and all images in the image depth feature set z are compliant, it is determined that the 5G rich media message can be sent normally.

[0098] This 5G rich media message speech-to-text method acquires video information from 5G rich media messages, constructs a video message information set Y based on the video information, and constructs a training set based on the video message information set Y. An improved CTC model 30 is constructed, wherein the improved CTC model 30 is based on the CTC model by introducing a maximum entropy function to improve the original loss function of the CTC model. The improved CTC model 30 is trained using the training set to obtain a trained improved CTC model 30. The trained improved CTC model 30 is then used to convert the video message information set Y into a video text set y. This solves the problem in existing technologies where the recognition cycle of the CTC model increases with the increase of noise in the speech environment.

[0099] This 5G rich media message voice-to-text method has the following advantages compared with traditional text or image review methods.

[0100] The review process is more comprehensive. Traditional text or image review methods can only review single pieces of information, resulting in limited scope. This invention utilizes a multimodal recognition algorithm to innovatively propose a security review framework for 5G message video, text, and image content, providing a holistic solution to the security review of rich media information in 5G messages.

[0101] By subtracting a fluctuation entropy from the original exponential loss function of CTC, the loss function proposed in this patent becomes... The range of values ​​for is reduced to 0 and This avoids the problem of infinite expansion of the loss function caused by a large amount of noise in 5G voice information. The video text set y can be obtained quickly and accurately from the video message information set Y.

[0102] Figures 2-3 This is a flowchart of an embodiment of the 5G rich media message voice-to-text system of the present invention; as follows: Figures 2-3 As shown in the figure, a 5G rich media message voice-to-text system provided by an embodiment of the present invention includes the following steps:

[0103] Module 10 is used to acquire video information from 5G rich media messages;

[0104] The first construction module 20 is used to construct a video message information set Y based on the video information, construct a training set based on the video message information set Y, divide the video message information set Y into a training set, a test set and a validation set, and train the improved CTC model 30 based on the training set.

[0105] The improved CTC model 30 is validated based on the validation set, and the improved CTC model 30 that meets the performance conditions is saved; the conversion result of the improved CTC model 30 is evaluated based on the test set.

[0106] An improved CTC model 30 is provided, wherein the improved CTC model 30 is based on the CTC model and introduces a maximum entropy function to improve the original CTC loss function in the CTC model;

[0107] The original loss function of CTC is improved using Formula 1;

[0108]

[0109] In the formula, The loss function of the improved CTC model 30 is... Let α be the original loss function of CTC, α be the coefficient of maximum conditional entropy regularization, and H(p(π|l,X)) be the entropy of the feasible path given the input sequence and the target sequence.

[0110] Solve for H(p(π|l,X)) using Formula 2;

[0111]

[0112] In the formula, (p(π|l,X)) represents the conditional probability of a feasible path π given 5G voice information X and the actual output I;

[0113] logp(π|X) represents the logarithm of the conditional probability of a feasible path π given 5G voice information X; It represents the sum of all output conditional probabilities of 5G voice information X, regardless of whether the actual output I is given.

[0114] The training module is used to train the improved CTC model 30 using the training set to obtain the trained improved CTC model 30.

[0115] The improved CTC model 30, after being trained, converts the video message information set Y into a video text set y.

[0116] The 5G rich media messaging voice-to-text system also includes:

[0117] The acquisition module 10 is further configured to:

[0118] Retrieve text and image messages from 5G rich media messages;

[0119] The first building module 20 is also used for:

[0120] Construct a text message information set X based on the text message and the video text set y;

[0121] Construct an image message information set Z based on the image messages;

[0122] Sensitive word variant recognition model 40 inputs the text messages in the text message information set X into the sensitive word variant recognition model 40 in turn to determine whether all text messages in the text message information set X are compliant.

[0123] Feature extraction model 50 is used to extract features from the image message information set Z to obtain the image depth feature set z;

[0124] The problematic image classification model 60 inputs the images in the image depth feature set z into the problematic image classification model 60 in turn to determine whether all the images in the image depth feature set z are compliant.

[0125] When all text messages in the text message information set X are compliant, and all images in the image depth feature set z are compliant, it is determined that the 5G rich media message can be sent normally.

[0126] This invention discloses a 5G rich media message speech-to-text system. The system acquires video information from 5G rich media messages via an acquisition module 10; constructs a video message information set Y based on the video information via a first construction module 20, and constructs a training set based on the video message information set Y; improves the CTC model 30 by introducing a maximum entropy function to improve the original loss function of the CTC model; trains the improved CTC model 30 using the training set via a training module, obtaining a trained improved CTC model 30; and converts the video message information set Y into a video text set y using the trained improved CTC model 30. This 5G rich media message speech-to-text method solves the problem in existing technologies where the recognition cycle of the CTC model increases with increasing noise in the speech environment.

[0127] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5 As shown, the electronic device 70 includes: a processor 701, a memory 702, and a bus 703;

[0128] The processor 701 and the memory 702 communicate with each other via the bus 703.

[0129] The processor 701 is used to call program instructions in the memory 702 to execute the methods provided in the above-described method embodiments, such as: acquiring video information in 5G rich media messages; constructing a video message information set Y based on the video information; constructing a training set based on the video message information set Y; constructing an improved CTC model 30, wherein the improved CTC model 30 is based on the CTC model by introducing a maximum entropy function to improve the original CTC loss function in the CTC model; training the improved CTC model 30 using the training set to obtain a trained improved CTC model 30; and converting the video message information set Y into a video text set y using the trained improved CTC model 30.

[0130] This embodiment provides a non-transitory computer-readable medium that stores computer instructions. These instructions cause a computer to execute the methods provided in the above embodiments, including, for example: acquiring video information from 5G rich media messages; constructing a video message information set Y based on the video information; constructing a training set based on the video message information set Y; constructing an improved CTC model 30, wherein the improved CTC model 30 is based on the CTC model by introducing a maximum entropy function to improve the original CTC loss function; training the improved CTC model 30 using the training set to obtain a trained improved CTC model 30; and converting the video message information set Y into a video text set y using the trained improved CTC model 30.

[0131] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the prior art, can be embodied in the form of software products. This computer software product can be stored in a computer-readable medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0134] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A 5G rich media message speech-to-text method, characterized in that, The method specifically comprises: Obtaining video information in a 5G rich media message, constructing a video message information set Y based on the video information, and constructing a training set based on the video message information set Y; An improved CTC model is constructed, wherein the improved CTC model is an improved CTC model in which a maximum entropy function is introduced into a CTC model to improve a CTC original loss function in the CTC model, and the improvement comprises: The CTC original loss function is improved through formula 1; Formula 1 ; In the formula, is the loss function of the improved CTC model, is the original loss function of CTC, is the coefficient of maximum conditional entropy regularization, is the entropy of the feasible path of the given input sequence and target sequence; Solving by equation 2 below ; wherein, denotes the conditional probability of a certain feasible path p given the 5G voice information X and the true output Y. log P (π | X) represents the log of the conditional probability of a given feasible path π given the 5G voice information X; represents the sum of all output condition probabilities of the 5G voice information X regardless of whether the true output is given represents the sum of all output condition probabilities of the 5G voice information X regardless of whether the true output is given The improved CTC model is trained through the training set to obtain a trained improved CTC model; The video message information set Y is converted into a video text set y through the trained improved CTC model.

2. The 5G rich media message speech-to-text method of claim 1, wherein, The 5G rich media message voice-to-text method further comprises: Obtaining a text message in a 5G rich media message; Constructing a text message information set X based on the text message and the video text set y; An sensitive word variant recognition model is constructed; Text messages in the text message information set X are input into the sensitive word variant recognition model one by one to determine whether all the text messages in the text message information set X are compliant.

3. The 5G rich media message speech-to-text method of claim 2, wherein, The 5G rich media message voice-to-text method further comprises: Obtaining a picture message in a 5G rich media message, and constructing a picture message information set Z based on the picture message; An improper picture classification model and a feature extraction model are constructed; Picture deep feature sets z are obtained by performing feature extraction on the picture message information set Z based on the feature extraction model; Pictures in the picture deep feature set z are input into the improper picture classification model one by one to determine whether all the pictures in the picture deep feature set z are compliant.

4. The 5G rich media message speech-to-text method of claim 3, wherein, The 5G rich media message voice-to-text method further comprises: When all the text messages in the text message information set X are compliant, and all the pictures in the picture deep feature set z are compliant, it is determined that the 5G rich media message can be normally sent.

5. The 5G rich media message speech-to-text method of claim 1, wherein, The improved CTC model is trained through the training set to obtain a trained improved CTC model, comprising: The video message information set Y is divided into a training set, a test set and a validation set; The improved CTC model is trained based on the training set; The performance of the improved CTC model is verified based on the validation set, and the improved CTC model meeting the performance condition is saved; The conversion result of the improved CTC model is evaluated based on the test set.

6. A 5G rich media messaging speech-to-text system, characterized by, Comprise: An acquisition module is configured to acquire video information in a 5G rich media message; A first construction module is configured to construct a video message information set Y based on the video information, and construct a training set based on the video message information set Y; An improved CTC model is constructed, wherein the improved CTC model is an improved CTC model in which a maximum entropy function is introduced into a CTC model to improve a CTC original loss function in the CTC model; The CTC original loss function is improved through formula 1; Formula 1 ; In the formula, is the loss function of the improved CTC model, is the original loss function of CTC, is the coefficient of maximum conditional entropy regularization, is the entropy of the feasible path of the given input sequence and target sequence; Solving by equation 2 below ; wherein, represents the conditional probability of a certain feasible path π given the 5G voice information X and the true output Y. log P (π | X) represents the log of the conditional probability of a given feasible path π given the 5G voice information X; represents the sum of all output condition probabilities of the 5G voice information X regardless of whether the true output is given is given, 5G voice information X A training module is configured to train the improved CTC model through the training set to obtain a trained improved CTC model; convert the set of video message information Y into a set of video text y by using the trained improved CTC model.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1 to 5 when executing the computer program.

8. A non-transitory computer readable medium having stored thereon a computer program, characterized in that, The computer program, which when executed by the processor, implements the steps of the method of any one of claims 1 to 5. The computer program, which when executed by the processor, implements the steps of the method of any one of claims 1 to 5.

Images