Course recommendation method and device, electronic equipment and medium

Abstract

The invention provides a course recommendation method and device, electronic equipment and a medium, and can be applied to the technical field of big data and the technical field of artificial intelligence. The method comprises the following steps: acquiring voice information input by a user; obtaining an adjustment audio of the voice information; obtaining a target voice feature; a course audio feature library is obtained, and the course audio feature library comprises course audio features extracted from course video resources in advance; calculating a semitone difference between the target voice feature and the course audio feature, and performing first screening to obtain a first course audio feature; calculating an editing distance between the target voice feature and the first course audio feature, and performing second screening to obtain a second course audio feature; and inputting the target voice feature and the second course audio feature into a pre-constructed course recommendation model to generate a recommendation score, and outputting a recommended course based on the recommendation score.

Description

Technical Field

[0001] The present invention relates to the fields of big data technology and artificial intelligence technology, and more specifically to a course recommendation method, device, electronic device and medium. Background Art

[0002] In today's digital learning environment, training platforms provide a wide range of learning resources covering a variety of topics and fields. Users often use keyword text searches to discover the resources they are interested in. However, keyword searches can present some challenges in certain situations.

[0003] On the one hand, users may not be familiar with the scientific names or fields of certain knowledge, resulting in the input keywords not matching the names of the actual resources, thus limiting users' discovery of learning resources, especially when users explore new fields or topics; on the other hand, the text input method is not conducive to users' quick search, not only requiring users to spend time thinking about keywords, but also possible spelling errors when inputting. In addition, video resources usually contain a large amount of audio and video content, which is usually difficult to accurately match with keywords. Users may need to watch the entire video or a large amount of content to determine whether the resource is relevant to their needs, resulting in low efficiency.

[0004] Therefore, there is a need to explore smarter, user-friendly ways to make learning resources more accessible and discoverable. Summary of the invention

[0005] In view of the above problems, according to a first aspect of the present invention, a course recommendation method is provided, the method comprising: obtaining voice information input by a user; adjusting the time axis of the voice information using an improved dynamic time stretching algorithm to obtain adjusted audio of the voice information; performing feature extraction on the adjusted audio to obtain target voice features; obtaining a course audio feature library, wherein the course audio feature library includes course audio features pre-extracted from course video resources; calculating a semitone difference between the target voice feature and the course audio feature, performing a first screening of the course audio feature based on the semitone difference to obtain a first course audio feature; calculating an edit distance between the target voice feature and the first course audio feature, performing a second screening of the first course audio feature based on the edit distance to obtain a second course audio feature; and inputting the target voice feature and the second course audio feature into a pre-built course recommendation model to generate a recommendation score, and outputting a recommended course based on the recommendation score.

[0006] According to some exemplary embodiments, the calculating of the semitone difference between the target speech feature and the course audio feature and the first screening of the course audio feature based on the semitone difference specifically include: calculating the first pitch information and the second pitch information corresponding to the target speech feature and the course audio feature using discrete Fourier transform technology; obtaining the semitone difference by calculating the ratio of the first pitch information and the second pitch information, taking the logarithm of the ratio and multiplying the logarithm by a semitone coefficient; and in response to the absolute value of the semitone difference not exceeding a preset first threshold, retaining the corresponding course audio feature to obtain the first course audio feature.

[0007] According to some exemplary embodiments, the computing of the edit distance between the target speech feature and the first course audio feature, performing a second screening of the first course audio feature based on the edit distance, and obtaining a second course audio feature specifically includes: based on the first pitch information and the second pitch information corresponding to the first course audio feature, converting the target speech feature and the first course audio feature into a first character string sequence and a second character string sequence, respectively; computing the edit distance between the first character string sequence and the second character string sequence using a dynamic programming algorithm; and in response to the edit distance not exceeding a preset second threshold, retaining the corresponding first course audio feature, and obtaining the second course audio feature.

[0008] According to some exemplary embodiments, the use of a dynamic programming algorithm to calculate the edit distance between the first string sequence and the second string sequence specifically includes: creating a distance matrix, wherein the number of rows and the number of columns of the distance matrix are determined based on the lengths of the first string sequence and the second string sequence, respectively; initializing the distance matrix, and gradually filling elements from the upper left corner of the distance matrix according to the similarity of the first string sequence and the second string sequence until the lower right corner of the distance matrix is ​​reached; and using the value corresponding to the lower right corner of the distance matrix as the edit distance.

[0009] According to some exemplary embodiments, the time axis of the voice information is adjusted using an improved dynamic time warping algorithm to obtain the adjusted audio of the voice information, specifically including: obtaining an audio template for reference; calculating the time similarity between the voice information and the audio template using an improved dynamic time warping algorithm; and adjusting the time axis of the voice information based on the time similarity to obtain the adjusted audio.

[0010] According to some exemplary embodiments, the course recommendation model is trained using a Transformer model; the target speech feature and the second course audio feature are input into a pre-built course recommendation model to generate a recommendation score, and a recommended course is output based on the recommendation score, specifically including: feature encoding and position encoding the target speech feature and the second course audio feature to obtain a first encoding feature and a second encoding feature; the correlation between the first encoding feature and the second encoding feature is obtained using a multi-encoder layer and a self-attention mechanism of the course recommendation model; and based on the correlation, the recommendation score is generated using the output layer of the course recommendation model.

[0011] According to some exemplary embodiments, the feature extraction of the adjusted audio to obtain the target speech feature specifically includes: extracting acoustic features from the adjusted audio based on Mel-cephalometric coefficients; and mean normalizing or standardizing the acoustic features to obtain the target speech feature.

[0012] According to a second aspect of the present invention, a course recommendation device is proposed, the device comprising: a voice information acquisition module, used to: acquire voice information input by a user; an adjusted audio acquisition module, used to: adjust the time axis of the voice information using an improved dynamic time stretching algorithm to acquire the adjusted audio of the voice information; a target voice feature acquisition module, used to: perform feature extraction on the adjusted audio to acquire target voice features; a course audio feature library acquisition module, used to: acquire a course audio feature library, wherein the course audio feature library includes course audio features pre-extracted from course video resources; a first course audio feature acquisition module, used to: calculate the semitone difference between the target voice feature and the course audio feature, perform a first screening of the course audio feature based on the semitone difference, and acquire the first course audio feature; a second course audio feature acquisition module, used to: calculate the edit distance between the target voice feature and the first course audio feature, perform a second screening of the first course audio feature based on the edit distance, and acquire the second course audio feature; and a recommended course output module, used to: input the target voice feature and the second course audio feature into a pre-built course recommendation model to generate a recommendation score, and output a recommended course based on the recommendation score.

[0013] According to some exemplary embodiments, the adjusted audio acquisition module may include a template acquisition unit, a time similarity calculation unit and an adjusted audio acquisition unit.

[0014] According to some exemplary embodiments, the template acquiring unit may be configured to acquire an audio template for reference.

[0015] According to some exemplary embodiments, the time similarity calculation unit may be configured to calculate the time similarity between the speech information and the audio template using an improved dynamic time warping algorithm.

[0016] According to some exemplary embodiments, the adjusted audio acquisition unit may be configured to adjust the time axis of the voice information based on the time similarity to acquire the adjusted audio.

[0017] According to some exemplary embodiments, the target speech feature acquisition module may include an acoustic feature extraction unit and a target speech feature acquisition unit.

[0018] According to some exemplary embodiments, the acoustic feature extraction unit may be configured to extract acoustic features from the adjusted audio based on Mel-cepstral coefficients.

[0019] According to some exemplary embodiments, the target speech feature acquisition unit may be configured to perform mean normalization or standardization on the acoustic feature to acquire the target speech feature.

[0020] According to some exemplary embodiments, the first course audio feature acquisition module may include a pitch information calculation unit, a semitone difference calculation unit, and a first course audio feature acquisition unit.

[0021] According to some exemplary embodiments, the pitch information calculation unit may be configured to calculate the first pitch information and the second pitch information corresponding to the target speech feature and the course audio feature using a discrete Fourier transform technique.

[0022] According to some exemplary embodiments, the semitone difference calculation unit may be configured to obtain the semitone difference by calculating a ratio of the first pitch information to the second pitch information, taking a logarithm of the ratio, and multiplying the logarithm by a semitone coefficient.

[0023] According to some exemplary embodiments, the first course audio feature acquisition unit may be used to retain the corresponding course audio feature in response to the absolute value of the semitone difference not exceeding a preset first threshold, so as to obtain the first course audio feature.

[0024] According to some exemplary embodiments, the second course audio feature acquisition module may include a character string sequence conversion unit, an edit distance calculation module and a second course audio feature acquisition unit.

[0025] According to some exemplary embodiments, the character string sequence conversion unit may be used to convert the target speech feature and the first course audio feature into a first character string sequence and a second character string sequence, respectively, based on the first pitch information and the second pitch information corresponding to the first course audio feature.

[0026] According to some exemplary embodiments, the edit distance calculation module may be configured to calculate the edit distance between the first character string sequence and the second character string sequence using a dynamic programming algorithm.

[0027] According to some exemplary embodiments, the second course audio feature acquisition unit may be configured to retain the corresponding first course audio feature and obtain the second course audio feature in response to the edit distance not exceeding a preset second threshold.

[0028] According to some exemplary embodiments, the edit distance calculation module may include a distance matrix creation unit, a filling unit, and an edit distance determination unit.

[0029] According to some exemplary embodiments, the distance matrix creation unit may be used to create a distance matrix, wherein the number of rows and the number of columns of the distance matrix are determined based on the lengths of the first character string sequence and the second character string sequence, respectively.

[0030] According to some exemplary embodiments, the filling unit may be used to initialize the distance matrix and gradually fill elements from the upper left corner of the distance matrix according to the similarity between the first character string sequence and the second character string sequence until reaching the lower right corner of the distance matrix.

[0031] According to some exemplary embodiments, the edit distance determining unit may be configured to use a value corresponding to a lower right corner of the distance matrix as the edit distance.

[0032] According to some exemplary embodiments, the recommended course output module may include an encoding unit, a relevance acquisition unit, and a score generation unit.

[0033] According to some exemplary embodiments, the encoding unit may be used to perform feature encoding and position encoding on the target speech feature and the second course audio feature to obtain a first encoding feature and a second encoding feature.

[0034] According to some exemplary embodiments, the correlation acquisition unit may be configured to acquire the correlation between the first encoding feature and the second encoding feature by utilizing a multi-encoder layer and a self-attention mechanism of the course recommendation model.

[0035] According to some exemplary embodiments, the score generating unit may be configured to generate the recommendation score based on the correlation using an output layer of the course recommendation model.

[0036] According to a third aspect of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method as described above.

[0037] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method as described above.

[0038] According to a fifth aspect of the present invention, there is provided a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method as described above is implemented.

[0039] One or more of the above embodiments have the following advantages or beneficial effects: According to the course recommendation method provided by the present invention, a large amount of course audio data can be quickly and accurately screened and processed by using an improved dynamic time stretching algorithm and fast semitone difference and edit distance calculation, thereby improving the processing efficiency of the computer; at the same time, the course audio and the target voice can be matched more accurately, reducing unnecessary calculations and processing. In addition, users can obtain personalized course recommendations through descriptive voice input, reducing the time of manual input and thinking of keywords, thereby improving user satisfaction and learning effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0041] Figure 1 The application scenario diagram of the course recommendation method, device, equipment, and medium according to an embodiment of the present invention is schematically shown.

[0042] Figure 2 The flowchart of the course recommendation method according to an embodiment of the present invention is schematically shown.

[0043] Figure 3 The flowchart of the method for adjusting the time axis of voice information according to an embodiment of the present invention is schematically shown.

[0044] Figure 4 The flowchart of the method for performing feature extraction according to an embodiment of the present invention is schematically shown.

[0045] Figure 5 The flowchart of the method for calculating semitone difference and performing first screening according to an embodiment of the present invention is schematically shown.

[0046] Figure 6The flowchart of the method for calculating the edit distance and performing the second screening according to an embodiment of the present invention is schematically shown.

[0047] Figure 7 The flowchart of the method for calculating the edit distance by using the dynamic programming algorithm according to an embodiment of the present invention is schematically shown.

[0048] Figure 8 The flowchart of the method for generating recommendation scores by using the Transformer model according to an embodiment of the present invention is schematically shown.

[0049] Fig. 9 The structure block diagram of the course recommendation device according to an embodiment of the present invention is schematically shown.

[0050] Fig.10 The block diagram of an electronic device suitable for a course recommendation method according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0051] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0052] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0053] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0054] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0055] In the technical solution of the present invention, the acquisition, storage and application of user personal information involved all comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0056] First, the technical terms described in this article are explained and described as follows.

[0057] Dynamic Time Warping (DTW) is a method for comparing the similarity of two time series. The core idea is to match two time series by warping or stretching the time axis to find the best alignment between them.

[0058] Semitone difference: Also known as semitone step, it is used to indicate the difference between the pitches of two notes.

[0059] Edit Distance: A metric used to measure the similarity or difference between two strings, measuring the minimum number of edit operations required to transform one string into another.

[0060] Mel-frequency Cepstral Coefficients (MFCC): A feature extraction method used for audio signal processing and speech recognition. Its main purpose is to simulate the way the human auditory system perceives sound features in order to improve the representation and recognition performance of audio features.

[0061] Transformer model: A deep learning model used for natural language processing tasks. One of its core features is the self-attention mechanism, which can process different positions of the input sequence in parallel and capture long-distance dependencies.

[0062] Dynamic Programming (DP) is a method for solving optimization problems, usually used to solve problems that can be decomposed into sub-problems and have overlapping sub-problems. The usual steps are: define the problem state, determine the state transition equation, initialize the state, calculate the state according to the state transition equation, and return the result.

[0063] With the rapid development of online education and training platforms in the financial system, users' demand for learning resources has gradually increased. However, when searching and accessing these learning resources, especially when they try to find resources that match their needs and interests, users face the following challenges.

[0064] First, users need to use text keywords to search, but they may not be familiar with the scientific name of the knowledge or the name of a specific course, which may result in the keywords entered by the user not matching the content or name of the actual resource, making the search results inaccurate or incomplete. This limits the user's ability to fully utilize the rich learning resources within the platform.

[0065] The second is the retrieval problem of video resources. Although text keyword retrieval is an effective retrieval method, it requires users to accurately enter vocabulary related to resources, and users cannot get accurate results through vague descriptions. At the same time, for platforms with a large number of video resources, extracting the text of the video content to match with keywords becomes cumbersome, time-consuming and may not be flexible enough, resulting in inconvenience for users when searching and accessing video resources.

[0066] Therefore, existing search methods have certain limitations on users' learning experience, especially for users who are not familiar with subject areas and course names. In order to improve the convenience of users' search and access to learning resources, a smarter and more intuitive method is needed to enable users to find resources of interest more easily.

[0067] Based on this, an embodiment of the present invention provides a course recommendation method, characterized in that the method includes: obtaining voice information input by a user; adjusting the time axis of the voice information using an improved dynamic time warping algorithm to obtain the adjusted audio of the voice information; extracting features from the adjusted audio to obtain target voice features; obtaining a course audio feature library, wherein the course audio feature library includes course audio features extracted in advance from course video resources; calculating the semitone difference between the target voice feature and the course audio feature, performing a first screening of the course audio feature based on the semitone difference to obtain a first course audio feature; calculating the edit distance between the target voice feature and the first course audio feature, performing a second screening of the first course audio feature based on the edit distance to obtain a second course audio feature; and inputting the target voice feature and the second course audio feature into a pre-built course recommendation model to generate a recommendation score, and outputting a recommended course based on the recommendation score. According to the course recommendation method provided by the present invention, a large amount of course audio data can be quickly and accurately screened and processed using an improved dynamic time warping algorithm and fast semitone difference and edit distance calculation, thereby improving the processing efficiency of the computer; at the same time, the course audio and the target voice can be more accurately matched, reducing unnecessary calculations and processing. In addition, users can get personalized course recommendations through descriptive voice input, reducing the time spent on manual input and thinking about keywords, thereby improving user satisfaction and learning outcomes.

[0068] It should be noted that the course recommendation method, device, equipment and medium determined by the present invention can be used in the fields of big data technology and artificial intelligence technology, can also be used in the financial field, and can also be used in various fields other than the fields of big data technology and artificial intelligence technology and the financial field. The application fields of the course recommendation method, device, equipment and medium provided by the embodiments of the present invention are not limited.

[0069] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0070] Figure 1 The application scenario diagram of the course recommendation method, device, equipment, and medium according to an embodiment of the present invention is schematically shown.

[0071] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0072] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0073] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0074] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0075] It should be noted that the course recommendation method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the course recommendation device provided in the embodiment of the present invention can generally be set in the server 105. The course recommendation method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the course recommendation device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0076] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0077] Figure 2 The flowchart of the course recommendation method according to an embodiment of the present invention is schematically shown.

[0078] like Figure 2 As shown, the course recommendation method 200 of this embodiment may include operations S210 to S270.

[0079] In operation S210, voice information input by a user is acquired.

[0080] In an embodiment of the present invention, a user may be allowed to express his / her need for a course using vaguely described oral language without having to precisely specify the course name or keywords. The user may provide voice input to the system through a microphone or a voice recognition application.

[0081] In operation S220, the time axis of the voice information is adjusted using an improved dynamic time warping algorithm to obtain adjusted audio of the voice information.

[0082] According to an embodiment of the present invention, in order to better adapt to the user's original voice input and avoid situations such as inconsistent speaking speed and changes in voice length when the user describes the course by voice, an embodiment of the present invention provides a method for adjusting voice information.

[0083] Figure 3The flowchart of the method for adjusting the time axis of voice information according to an embodiment of the present invention is schematically shown.

[0084] like Figure 3 As shown, the method for adjusting the time axis of voice information in this embodiment may include operations S310 to S330, and operations S310 to S330 may at least partially execute operation S220.

[0085] In operation S310, an audio template for reference is acquired.

[0086] In the embodiments of the present invention, the audio template is generally a known audio segment used as a standard, and may be a standard pronunciation, a reference voice sample, or any audio that can be used to compare with the user's voice information.

[0087] In operation S320, the time similarity between the speech information and the audio template is calculated using an improved dynamic time warping algorithm.

[0088] In an embodiment of the present invention, the dynamic time warping algorithm can consider speech features on different time axes and find the best time stretching or compression to maximize the matching of speech information and templates, and generate a metric reflecting the similarity between them. On this basis, the improved dynamic time warping algorithm can automatically adjust the time axis to adapt to the speed difference between different time sequences, so as to cope with the alignment problem between sequences of different speeds and different time lengths without manually specifying time warping parameters, thereby being more efficient, reducing computational complexity and improving operating speed.

[0089] In an embodiment of the present invention, the temporal similarity can be calculated by a dynamic time warping method. Specifically, the temporal similarity between the two signals can be measured by searching for an optimal alignment path considering the temporal axis of the speech information, which reflects how to adjust the temporal axis of the speech information to match the temporal axis of the audio template.

[0090] In operation S330, the time axis of the voice information is adjusted based on the time similarity to obtain the adjusted audio.

[0091] In an embodiment of the present invention, the result of the time similarity can be a metric value, which represents the similarity between the speech information and the audio template. The improved dynamic time stretching algorithm can adaptively adjust the time axis of the speech information according to the metric value to adapt it to the time difference of different speech segments.

[0092] Return to reference Figure 2 In operation S230, feature extraction is performed on the adjusted audio to obtain target speech features.

[0093] Figure 4The flowchart of the method for performing feature extraction according to an embodiment of the present invention is schematically shown.

[0094] like Figure 4 As shown, the method for performing feature extraction in this embodiment may include operations S410 to S420, and operations S410 to S420 may at least partially execute operation S230.

[0095] In operation S410, an acoustic feature is extracted from the adjusted audio based on a Mel-cepstral coefficient.

[0096] In an embodiment of the present invention, sound features can be obtained based on Mel cepstral coefficients (MFCC). Specifically, it includes: pre-emphasis, framing, Fourier transform, Mel filter combination, logarithmization, discrete cosine transform, time step sequence, etc. Among them, pre-emphasis is to reduce the high-frequency noise in the sound signal to improve the effect of subsequent processing. The signal can be passed through a high-pass filter to reduce the amplitude of the low-frequency component and increase the amplitude of the high-frequency component; framing can divide the signal into frames of 20 to 30 milliseconds. There is a certain overlap between each frame to ensure continuity; a discrete Fourier transform (DFT) is applied to each frame to represent the signal as a frequency domain component; a set of Mel filters is constructed, and the frequency scales of these filters are distributed according to the Mel scale. Then, the spectrum of each frame is passed through this set of filters to obtain the energy of each filter channel; the energy of the filter channel is logarithmic to simulate the nonlinear perception of the human ear to sound; a discrete cosine transform (DCT) is applied to the logarithmic energy coefficient, and then the first few coefficients are retained as MFCC features; the MFCC coefficients extracted from each frame are arranged in chronological order to form a time series.

[0097] According to an embodiment of the present invention, the MFCC feature captures the spectral characteristics of the sound signal, can effectively represent the characteristics of the sound, and helps the model to better understand the sound information.

[0098] In operation S420, mean normalization or standardization is performed on the acoustic features to obtain the target speech features.

[0099] In an embodiment of the present invention, the extracted acoustic features may be mean normalized or standardized to ensure that the features of different speech segments have similar scales and distributions. Specifically, mean normalization includes calculating the mean of the extracted acoustic features and subtracting the mean from each feature dimension so that the mean of all feature dimensions is zero; standardization involves calculating the standard deviation of each feature dimension and dividing each feature dimension by its standard deviation to ensure that their scales are consistent. The processed acoustic features become target speech features and can be used for subsequent matching and recognition tasks.

[0100] Return to reference Figure 2In operation S240, a course audio feature library is obtained, wherein the course audio feature library includes course audio features extracted in advance from course video resources.

[0101] In an embodiment of the present invention, audio data may be acquired in advance from different course video resources, wherein the audio data may include content related to different courses. Further, the audio data may be preprocessed and feature engineered to extract course audio features, wherein each feature in the course audio features may represent the audio content of a certain course, and generate a course audio feature library.

[0102] In operation S250, a semitone difference between the target speech feature and the course audio feature is calculated, and a first screening is performed on the course audio feature based on the semitone difference to obtain a first course audio feature.

[0103] Figure 5 The flowchart of the method for calculating semitone difference and performing first screening according to an embodiment of the present invention is schematically shown.

[0104] like Figure 5 As shown, the method for calculating the single customer acquisition cost of this embodiment may include operations S510 to S530, and operations S510 to S530 may at least partially execute operation S250.

[0105] In operation S510, the first pitch information and the second pitch information corresponding to the target speech feature and the course audio feature are calculated using discrete Fourier transform technology.

[0106] In an embodiment of the present invention, a discrete Fourier transform (DFT) technique may be used to convert a time domain signal (sound) into a frequency domain signal. DFT decomposes the signal into complex representations of different frequency components, which contain spectrum information.

[0107] In an embodiment of the present invention, the pitch can be extracted by analyzing the spectrum representation. Specifically, the gene frequencies corresponding to the target speech features and the course audio features can be extracted from the spectrum representation and used as the first pitch information and the second pitch information.

[0108] In operation S520, the semitone difference is obtained by calculating a ratio of the first pitch information to the second pitch information, taking a logarithm of the ratio, and multiplying the logarithm by a semitone coefficient.

[0109] In an embodiment of the present invention, semitone difference is an important sound feature for pitch matching and analysis. By calculating semitone difference, the continuous range of pitch information can be converted into discrete semitone units for matching and screening, thereby improving the accuracy of matching. Wherein, the semitone difference calculation formula is expressed as:

[0110]

[0111] Wherein, A is a semitone difference, f1 is the first pitch information, f2 is the second pitch information, and the semitone coefficient is 12. A positive semitone difference indicates that the pitch of the second pitch information is higher, and a negative semitone difference indicates that the pitch of the second pitch information is lower.

[0112] In operation S530, in response to the absolute value of the semitone difference not exceeding a preset first threshold, the corresponding course audio feature is retained to obtain the first course audio feature.

[0113] In the embodiment of the present invention, since the semitone difference may have a negative value, the absolute value of the semitone difference can be compared with a preset first threshold to screen features with higher similarity.

[0114] Return to reference Figure 2 In operation S260, the edit distance between the target speech feature and the first course audio feature is calculated, and the first course audio feature is secondly screened based on the edit distance to obtain a second course audio feature.

[0115] Figure 6 The flowchart of the method for calculating the edit distance and performing the second screening according to an embodiment of the present invention is schematically shown.

[0116] like Figure 6 As shown, the method for calculating the edit distance and performing the second screening in this embodiment may include operations S610 to S630, and operations S610 to S630 may at least partially execute operation S260.

[0117] In operation S610, based on the first pitch information and the second pitch information corresponding to the first course audio feature, the target speech feature and the first course audio feature are converted into a first character string sequence and a second character string sequence, respectively.

[0118] In an embodiment of the present invention, the first pitch information and the second pitch information may be mapped to letters or characters to convert them into a string sequence, wherein different pitches may be mapped to different letters or characters.

[0119] In operation S620, the edit distance between the first character string sequence and the second character string sequence is calculated using a dynamic programming algorithm.

[0120] Figure 7 The flowchart of the method for calculating the edit distance by using the dynamic programming algorithm according to an embodiment of the present invention is schematically shown.

[0121] like Figure 7As shown, the method for calculating the edit distance using the dynamic programming algorithm in this embodiment may include operations S710 to S730, and operations S710 to S730 may at least partially execute operation S620.

[0122] In operation S710, a distance matrix is ​​created, wherein the number of rows and the number of columns of the distance matrix are determined based on the lengths of the first character string sequence and the second character string sequence, respectively.

[0123] In the embodiment of the present invention, the distance matrix is ​​generally a two-dimensional array, whose dimension is determined by the length of the two strings. For example, m is the length of the first string sequence, n is the length of the second string sequence, and the size of the matrix is ​​(m+1) rows and (n+1) columns, where m and n are positive integers.

[0124] In operation S720, the distance matrix is ​​initialized, and elements are gradually filled from the upper left corner of the distance matrix according to the similarity between the first character string sequence and the second character string sequence until the lower right corner of the distance matrix is ​​reached.

[0125] In an embodiment of the present invention, it is necessary to initialize values ​​in the distance matrix to ensure that the values ​​of the first row and the first column correspond to their corresponding position indexes. For example, the first row and the first column of the edit distance are consecutive integers from 0 to m and n, representing the edit distance from an empty string to the corresponding string.

[0126] In an embodiment of the present invention, the elements of the matrix can be gradually filled starting from the first position (upper left corner) of the distance matrix until the lower right corner of the distance matrix is ​​reached. The process of filling elements can follow the following principles: for each cell (i, j), the minimum value of the following three values ​​is calculated according to the dynamic programming algorithm and filled into the current cell:

[0127] a) The value on the left is increased by 1: it means inserting a character into the first string sequence and increasing the edit distance by 1;

[0128] b) Add 1 to the value above: This means deleting one character in the first string sequence and increasing the edit distance by 1;

[0129] c) Value of the upper left corner: If the character i in the first string sequence is equal to the character j in the second string sequence, the edit distance remains unchanged; if they are not equal, the edit distance is increased by 1.

[0130] In an embodiment of the present invention, the above calculation can be determined based on the similarity of the string sequence, that is, if two strings are similar (or equal), the edit distance will not increase too much. In addition, the order of filling can be from the upper left corner to the lower right corner, in the order from left to right, from top to bottom, until reaching the lower right corner of the distance matrix.

[0131] In operation S730, a value corresponding to the lower right corner of the distance matrix is ​​used as the edit distance.

[0132] In an embodiment of the present invention, the value corresponding to the lower right corner of the distance matrix represents the minimum number of editing operations required to convert the first string sequence into the second string sequence. The value represents the similarity between the two strings. The smaller the edit distance, the more similar they are.

[0133] Return to reference Figure 6 In operation S630, in response to the edit distance not exceeding a preset second threshold, the corresponding first course audio feature is retained and the second course audio feature is obtained.

[0134] In an embodiment of the present invention, the edit distance is used as a similarity metric to help screen course audio features, thereby further improving matching accuracy.

[0135] Return to reference Figure 2 In operation S270, the target speech feature and the second course audio feature are input into a pre-built course recommendation model to generate a recommendation score, and a recommended course is output based on the recommendation score.

[0136] In an embodiment of the present invention, the course recommendation model is trained using a Transformer model.

[0137] Figure 8 The flowchart of the method for generating recommendation scores by using the Transformer model according to an embodiment of the present invention is schematically shown.

[0138] like Figure 8 As shown, the method for generating recommendation scores by using the Transformer model in this embodiment may include operations S810 to S830.

[0139] In operation S810, feature encoding and position encoding are performed on the target speech feature and the second course audio feature to obtain a first encoding feature and a second encoding feature.

[0140] In an embodiment of the present invention, the target speech feature and the second course audio feature first need to be encoded in order to convert them into a form that can be processed by the model. Specifically, feature encoding can map the target speech feature and the second course audio feature into a high-dimensional vector representation, which can be performed using a convolutional neural network (CNN) or a fully connected neural network (DNN); position encoding is to introduce position information into the target speech feature and the second course audio feature so that the model can consider the feature order, and a sine-cosine function can be used to generate position-related encoding.

[0141] In operation S820, the correlation between the first encoding feature and the second encoding feature is obtained by utilizing the multi-encoder layer and the self-attention mechanism of the course recommendation model.

[0142] In an embodiment of the present invention, the multi-encoder layer of the course recommendation model may include multiple Transformer encoders, each encoder including a self-attention mechanism and a feedforward neural network, and used to capture the dependency and context information between the first encoding feature and the second encoding feature. The self-attention mechanism allows the course recommendation model to dynamically weight features according to the correlation between the first encoding feature and the second encoding feature, thereby better capturing the correlation between different features.

[0143] In operation S830, based on the correlation, the recommendation score is generated using the output layer of the course recommendation model.

[0144] In an embodiment of the present invention, a recommendation score can be generated based on the calculated correlation, which can represent the degree of match between the voice input by the user and the audio features of different courses. Specifically, the activation function and weight of the output layer can be applied to calculate the recommendation score.

[0145] In an embodiment of the present invention, the system can recommend appropriate courses to the user based on the degree of matching between the voice information input by the user and the audio of different courses.

[0146] According to the course recommendation method provided by the present invention, a large amount of course audio data can be quickly and accurately screened and processed by utilizing an improved dynamic time-stretching algorithm and fast semitone difference and edit distance calculation, thereby improving the processing efficiency of the computer; at the same time, the course audio and the target voice can be matched more accurately, reducing unnecessary calculations and processing. In addition, users can obtain personalized course recommendations through descriptive voice input, reducing the time for manual input and thinking of keywords, thereby improving user satisfaction and learning effects. Specifically, the following beneficial effects are brought about:

[0147] 1. Through the improved dynamic time stretching algorithm, voice information can be processed more effectively and the time axis can be adjusted, thereby reducing unnecessary information processing and improving the computer's computing speed;

[0148] 2. The feature extraction process is performed using the Mel-frequency cepstral coefficient method, which helps to reduce the computational complexity and alleviate the computational burden of the computer by converting the adjusted audio into a feature representation that is easier to process;

[0149] 3. By calculating the semitone difference between the target speech features and the course audio features, the pitch similarity between them can be evaluated, thereby screening out the course audio features with smaller pitch differences from the target speech, so as to narrow the scope of the final course selection and recommendation, and allow the system to quickly exclude irrelevant courses in the rough screening stage;

[0150] 4. By calculating the edit distance between the target speech features and the first course audio features, the similarity between them can be measured more accurately, which can further narrow the scope of course selection and help to quickly exclude irrelevant courses in the fine screening stage;

[0151] 5. By using the Transformer model to train the course recommendation model, the correlation between the target speech features and the second course audio features can be more accurately calculated, thereby providing more accurate course recommendations;

[0152] 6. Users can use fuzzy voice descriptions for input instead of relying on text keyword searches, which reduces the complexity of user input and improves user experience.

[0153] 7. By calculating the recommendation score, the system can provide users with personalized course recommendations based on the user's voice input and the degree of matching. Users do not need to browse a large number of courses or use keyword searches to find content of interest, reducing the time users spend on finding suitable courses, thereby improving user satisfaction.

[0154] Based on the above course recommendation method, the present invention also provides a course recommendation device. Fig. 9 The device is described in detail.

[0155] Fig. 9 The structure block diagram of the course recommendation device according to an embodiment of the present invention is schematically shown.

[0156] like Fig. 9 As shown, according to this embodiment, the course recommendation device 900 includes a voice information acquisition module 910, an adjustment audio acquisition module 920, a target voice feature acquisition module 930, a course audio feature library acquisition module 940, a first course audio feature acquisition module 950, a second course audio feature acquisition module 960 and a recommended course output module 970.

[0157] The voice information acquisition module 910 may be used to acquire the voice information input by the user. In one embodiment, the voice information acquisition module 910 may be used to perform the operation S210 described above, which will not be described in detail here.

[0158] The adjusted audio acquisition module 920 can be used to adjust the time axis of the voice information using an improved dynamic time warping algorithm to obtain the adjusted audio of the voice information. In one embodiment, the adjusted audio acquisition module 920 can be used to perform the operation S220 described above, which will not be repeated here.

[0159] The target speech feature acquisition module 930 may be used to extract features from the adjusted audio to acquire target speech features. In one embodiment, the target speech feature acquisition module 930 may be used to perform the operation S230 described above, which will not be described in detail herein.

[0160] The course audio feature library acquisition module 940 can be used to acquire a course audio feature library, wherein the course audio feature library includes course audio features pre-extracted from course video resources. In one embodiment, the course audio feature library acquisition module 940 can be used to perform the operation S240 described above, which will not be repeated here.

[0161] The first course audio feature acquisition module 950 can be used to calculate the semitone difference between the target speech feature and the course audio feature, and perform a first screening on the course audio feature based on the semitone difference to obtain the first course audio feature. In one embodiment, the first course audio feature acquisition module 950 can be used to perform the operation S250 described above, which will not be repeated here.

[0162] The second course audio feature acquisition module 960 can be used to calculate the edit distance between the target speech feature and the first course audio feature, and perform a second screening on the first course audio feature based on the edit distance to obtain the second course audio feature. In one embodiment, the second course audio feature acquisition module 960 can be used to perform the operation S260 described above, which will not be repeated here.

[0163] The recommended course output module 970 can be used to input the target speech feature and the second course audio feature into a pre-built course recommendation model to generate a recommendation score, and output a recommended course based on the recommendation score. In one embodiment, the recommended course output module 970 can be used to perform the operation S270 described above, which will not be repeated here.

[0164] According to an embodiment of the present invention, the adjusted audio acquisition module 920 may include a template acquisition unit, a time similarity calculation unit and an adjusted audio acquisition unit.

[0165] The template acquisition unit may be used to acquire an audio template for reference. In one embodiment, the template acquisition unit may be used to perform the operation S310 described above, which will not be described in detail herein.

[0166] The time similarity calculation unit may be used to calculate the time similarity between the speech information and the audio template using an improved dynamic time warping algorithm. In one embodiment, the time similarity calculation unit may be used to perform the operation S320 described above, which will not be described in detail herein.

[0167] The adjusted audio acquisition unit may be used to adjust the time axis of the voice information based on the time similarity to acquire the adjusted audio. In one embodiment, the adjusted audio acquisition unit may be used to perform the operation S330 described above, which will not be described in detail herein.

[0168] According to an embodiment of the present invention, the target speech feature acquisition module 930 may include an acoustic feature extraction unit and a target speech feature acquisition unit.

[0169] The acoustic feature extraction unit may be used to extract acoustic features from the adjusted audio based on Mel-cepstral coefficients. In one embodiment, the acoustic feature extraction unit may be used to perform the operation S410 described above, which will not be described in detail herein.

[0170] The target speech feature acquisition unit may be used to perform mean normalization or standardization on the acoustic feature to acquire the target speech feature. In one embodiment, the target speech feature acquisition unit may be used to perform the operation S420 described above, which will not be described in detail herein.

[0171] According to an embodiment of the present invention, the first course audio feature acquisition module 950 may include a pitch information calculation unit, a semitone difference calculation unit and a first course audio feature acquisition unit.

[0172] The pitch information calculation unit can be used to calculate the first pitch information and the second pitch information corresponding to the target speech feature and the course audio feature using discrete Fourier transform technology. In one embodiment, the pitch information calculation unit can be used to perform the operation S510 described above, which will not be repeated here.

[0173] The semitone difference calculation unit may be used to obtain the semitone difference by calculating the ratio of the first pitch information to the second pitch information, taking the logarithm of the ratio, and multiplying the logarithm by a semitone coefficient. In one embodiment, the semitone difference calculation unit may be used to perform the operation S520 described above, which will not be described in detail herein.

[0174] The first course audio feature acquisition unit can be used to retain the corresponding course audio feature in response to the absolute value of the semitone difference not exceeding a preset first threshold value, and obtain the first course audio feature. In one embodiment, the first course audio feature acquisition unit can be used to perform the operation S530 described above, which is not repeated here.

[0175] According to an embodiment of the present invention, the second course audio feature acquisition module 960 may include a character string sequence conversion unit, an edit distance calculation module and a second course audio feature acquisition unit.

[0176] The character string sequence conversion unit can be used to convert the target speech feature and the first course audio feature into a first character string sequence and a second character string sequence, respectively, based on the first pitch information and the second pitch information corresponding to the first course audio feature. In one embodiment, the character string sequence conversion unit can be used to perform the operation S610 described above, which will not be repeated here.

[0177] The edit distance calculation module may be used to calculate the edit distance between the first character string sequence and the second character string sequence using a dynamic programming algorithm. In one embodiment, the edit distance calculation module may be used to perform the operation S620 described above, which will not be described in detail herein.

[0178] The second course audio feature acquisition unit can be used to retain the corresponding first course audio feature in response to the edit distance not exceeding a preset second threshold value, and obtain the second course audio feature. In one embodiment, the second course audio feature acquisition unit can be used to perform the operation S630 described above, which will not be repeated here.

[0179] According to an embodiment of the present invention, the edit distance calculation module may include a distance matrix creation unit, a filling unit and an edit distance determination unit.

[0180] The distance matrix creation unit may be used to create a distance matrix, the number of rows and the number of columns of the distance matrix being determined based on the lengths of the first string sequence and the second string sequence, respectively. In one embodiment, the distance matrix creation unit may be used to perform the operation S710 described above, which will not be described in detail herein.

[0181] The filling unit can be used to initialize the distance matrix and gradually fill elements from the upper left corner of the distance matrix according to the similarity between the first string sequence and the second string sequence until reaching the lower right corner of the distance matrix. In one embodiment, the filling unit can be used to perform the operation S720 described above, which will not be repeated here.

[0182] The edit distance determination unit may be used to use the value corresponding to the lower right corner of the distance matrix as the edit distance. In one embodiment, the edit distance determination unit may be used to perform the operation S730 described above, which will not be described in detail herein.

[0183] According to an embodiment of the present invention, the recommended course output module 970 may include an encoding unit, a relevance acquisition unit, and a score generation unit.

[0184] The encoding unit may be used to perform feature encoding and position encoding on the target speech feature and the second course audio feature to obtain a first encoding feature and a second encoding feature. In one embodiment, the encoding unit may be used to perform the operation S810 described above, which will not be described in detail herein.

[0185] The correlation acquisition unit may be used to utilize the multi-encoder layer and the self-attention mechanism of the course recommendation model to acquire the correlation between the first encoding feature and the second encoding feature. In one embodiment, the correlation acquisition unit may be used to perform the operation S820 described above, which will not be described in detail herein.

[0186] The score generating unit may be used to generate the recommendation score based on the correlation using the output layer of the course recommendation model. In one embodiment, the score generating unit may be used to perform the operation S830 described above, which will not be described in detail here.

[0187] According to an embodiment of the present invention, any multiple modules among the speech information acquisition module 910, the adjustment audio acquisition module 920, the target speech feature acquisition module 930, the course audio feature library acquisition module 940, the first course audio feature acquisition module 950, the second course audio feature acquisition module 960 and the recommended course output module 970 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the speech information acquisition module 910, the adjustment audio acquisition module 920, the target speech feature acquisition module 930, the course audio feature library acquisition module 940, the first course audio feature acquisition module 950, the second course audio feature acquisition module 960 and the recommended course output module 970 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the speech information acquisition module 910, the adjustment audio acquisition module 920, the target speech feature acquisition module 930, the course audio feature library acquisition module 940, the first course audio feature acquisition module 950, the second course audio feature acquisition module 960 and the recommended course output module 970 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0188] Figure 1 FIG. 0 schematically shows a block diagram of an electronic device suitable for a course recommendation method according to an embodiment of the present invention.

[0189] like Figure 1 0, the electronic device 1000 according to an embodiment of the present invention includes a processor 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0190] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0191] According to an embodiment of the present invention, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0192] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0193] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0194] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0195] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 1001. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0196] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0197] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0198] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed completely on the user computing device, partially on the user device, partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0199] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0200] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A course recommendation method, characterized in that: The method comprises: Get the voice information input by the user; Using an improved dynamic time-stretching algorithm to adjust the time axis of the voice information, and obtain the adjusted audio of the voice information; Performing feature extraction on the adjusted audio to obtain target speech features; Acquire a course audio feature library, wherein the course audio feature library includes course audio features extracted in advance from course video resources; Calculating a semitone difference between the target speech feature and the course audio feature, and performing a first screening on the course audio feature based on the semitone difference to obtain a first course audio feature; Calculating the edit distance between the target speech feature and the first course audio feature, performing a second screening on the first course audio feature based on the edit distance to obtain a second course audio feature; and The target speech feature and the second course audio feature are input into a pre-built course recommendation model to generate a recommendation score, and a recommended course is output based on the recommendation score.

2. The method according to claim 1, characterized in that The calculating of the semitone difference between the target speech feature and the course audio feature, and performing a first screening of the course audio feature based on the semitone difference specifically includes: Calculate the first pitch information and the second pitch information corresponding to the target speech feature and the course audio feature by using discrete Fourier transform technology; The semitone difference is obtained by calculating a ratio of the first pitch information to the second pitch information, taking a logarithm of the ratio, and multiplying the logarithm by a semitone coefficient; as well as In response to the absolute value of the semitone difference not exceeding a preset first threshold, the corresponding course audio feature is retained to obtain the first course audio feature.

3. The method according to claim 2, characterized in that The calculating the edit distance between the target speech feature and the first course audio feature, and performing a second screening on the first course audio feature based on the edit distance to obtain the second course audio feature specifically includes: Based on the first pitch information and the second pitch information corresponding to the first course audio feature, converting the target speech feature and the first course audio feature into a first character string sequence and a second character string sequence respectively; Calculating the edit distance between the first character string sequence and the second character string sequence using a dynamic programming algorithm; and In response to the edit distance not exceeding a preset second threshold, the corresponding first course audio feature is retained, and the second course audio feature is obtained.

4. The method according to claim 3, characterized in that The calculating the edit distance between the first character string sequence and the second character string sequence by using a dynamic programming algorithm specifically includes: Creating a distance matrix, wherein the number of rows and the number of columns of the distance matrix are determined based on the lengths of the first character string sequence and the second character string sequence respectively; Initializing the distance matrix, and gradually filling elements from the upper left corner of the distance matrix according to the similarity between the first character string sequence and the second character string sequence until reaching the lower right corner of the distance matrix; and The value corresponding to the lower right corner of the distance matrix is ​​used as the edit distance.

5. The method according to any one of claims 1 to 4, characterized in that: Using an improved dynamic time stretching algorithm to adjust the time axis of the voice information to obtain the adjusted audio of the voice information specifically includes: Get an audio template for reference; Calculating the time similarity between the speech information and the audio template using an improved dynamic time warping algorithm; and The time axis of the voice information is adjusted based on the time similarity to obtain the adjusted audio.

6. The method according to claim 5, characterized in that Using the Transformer model to train the course recommendation model; The step of inputting the target speech feature and the second course audio feature into a pre-built course recommendation model to generate a recommendation score, and outputting a recommended course based on the recommendation score specifically includes: Performing feature coding and position coding on the target speech feature and the second course audio feature to obtain a first coding feature and a second coding feature; Obtaining the correlation between the first encoding feature and the second encoding feature by using the multi-encoder layer and the self-attention mechanism of the course recommendation model; and Based on the correlation, the recommendation score is generated using the output layer of the course recommendation model.

7. The method according to claim 6, characterized in that The feature extraction of the adjusted audio to obtain the target speech feature specifically includes: extracting acoustic features from the adjusted audio based on Mel-frequency cepstral coefficients; and The acoustic features are mean normalized or standardized to obtain the target speech features.

8. A course recommendation device, characterized in that: The device comprises: The voice information acquisition module is used to: acquire the voice information input by the user; An adjusted audio acquisition module is used to: adjust the time axis of the voice information using an improved dynamic time stretching algorithm to obtain the adjusted audio of the voice information; A target speech feature acquisition module is used to: extract features from the adjusted audio to acquire target speech features; A course audio feature library acquisition module is used to: acquire a course audio feature library, wherein the course audio feature library includes course audio features extracted in advance from course video resources; A first course audio feature acquisition module is used to: calculate the semitone difference between the target speech feature and the course audio feature, perform a first screening on the course audio feature based on the semitone difference, and obtain the first course audio feature; A second course audio feature acquisition module is used to: calculate the edit distance between the target speech feature and the first course audio feature, perform a second screening on the first course audio feature based on the edit distance, and obtain a second course audio feature; and The recommended course output module is used to: input the target speech feature and the second course audio feature into a pre-built course recommendation model to generate a recommendation score, and output a recommended course based on the recommendation score.

9. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 7.

Images