Method for adapting audio content to a user's competence profile of a motor vehicle, computer program and / or computer-readable medium, data processing device, motor vehicle

Abstract

Method (100) for adapting audio content (60) to a competence profile (95) of a user (90) of a motor vehicle (50), wherein the method (100) comprises: capturing (110) an audio content (60) comprising a speech content (61); determining (120) a competence profile (95) characterizing a competence of the user (90); determining (130) an audio output content (65) taking into account the speech content (61) and the competence profile (95); and outputting (140) an adapted audio content (60) based on the audio output content (65).

Description

[0001] The present disclosure relates to a method for adapting audio content to a competency profile of a user of a motor vehicle. The disclosure also relates to a computer program and / or computer-readable medium, a data processing device, and a motor vehicle.

[0002] Various applications are known from the state of the art for playing voice content such as podcasts, news, audio books and so on in a motor vehicle.

[0003] Adapting audio content is well known in the art. This involves reproducing speech content with an identical word sequence as it was recorded or published, with the playback speed and / or volume adjustable.

[0004] DE 10 2021 108 042 A1 discloses devices, systems, and techniques for facilitating the understanding of media content using neural networks to adjust playback speed and volume based on environmental and other factors. This involves slowing or speeding up playback of media content when audio associated with the media content is difficult to understand based on background noise, accent, difficulty of the material, and other factors that reduce the intelligibility of the media content.

[0005] Furthermore, it is known that text sections, i.e., components of the spoken content, can be skipped and / or modified. Skipping can occur, for example, due to an ad blocker in a podcast.

[0006] US 9,405,741 B1 discloses that features for detecting inappropriate content in an output are disclosed. The offensive content may be generated as a result of a language processing error. A system may identify the inappropriate elements of a generated output and select among several suitable alternatives. The system may be customized based on certain user characteristics. The system may be localized based on linguistic and cultural characteristics. The system may modify the generated output based on characteristics such as the tolerance threshold of known people in the vicinity of the system. The tolerance threshold may further be used to personalize and modify available content. Models used by the system may be further trained using input from a user.

[0007] EP 1 172 804 B1 discloses a method, performed by a device, for adapting a content of an oral presentation given by a representative of an organization and perceived by a human target of the organization based on a task of the organization, the method comprising the steps of: detecting a content of the oral presentation given by the representative and modifying the oral presentation given by the representative to compose the oral presentation perceived by the human target based on the detected content and the organizational task, wherein detecting the content of the oral presentation given by the representative comprises recognizing a spoken word content;and changing the oral presentation made by the representative involves replacing a recognized spoken word of inappropriate content with a word that is equivalent but more appropriate in content or omitting the recognized spoken word if no purpose can be recognized in the use of the word;

[0008] However, regardless of the known modifications, language content can be difficult to comprehend or follow, depending on the listener's or user's ability to understand the language content, even if the user can easily grasp the language content acoustically. Both the user's language and subject-matter competence can be crucial in this regard.

[0009] Against the background of this prior art, one object of the present disclosure is to enable a method suitable for enriching the prior art and at least overcoming the aforementioned disadvantages of the prior art. In particular, the object of the disclosure is to present audio content in such a way that a user of a motor vehicle can understand the audio content as well as possible.

[0010] The problem is solved by the features of the independent claims. The subclaims contain further developments of the disclosure.

[0011] According to this, the problem is solved by a method for adapting audio content to a competency profile of a user of a motor vehicle, the method comprising: detecting audio content comprising speech content; determining a competency profile characterizing a competency of the user; determining audio output content taking into account the speech content and the competency profile; and outputting adapted audio content based on the audio output content.

[0012] The method allows the actual spoken content of an audio recording to be adapted to the user's competence. For this purpose, the audio content can be captured. The audio content includes speech content, i.e., a section containing information spoken by one or more speakers and acoustically perceivable.

[0013] To enable targeted adaptation of the content, the disclosure recognizes that the competence of the user, listener, or consumer can be described. The user's competence can be described by a competence profile. The competence profile characterizes the user's ability to understand the language content. This competence can include language and / or subject-matter competence. The competence can depend on the user, the language content, and / or a driving situation and / or predicted driving situation in order to comprehensively assess the user's competence.

[0014] Based on the competency profile, it is possible to specifically adapt the language content and thus determine the audio output content. The audio output content can include unadapted, original sections and adapted sections. The adapted sections can feature adapted language content with a reduced complexity compared to the original language content, so that the user can understand the adapted language content and thus the audio output content. This ensures that the user can understand the content as well as possible.

[0015] Optionally, the competency profile includes a classification of the user's language level and / or a subject-specific level related to the language content. The language level can be crucial for understanding individual words and / or grammatical features. The subject-specific level can influence the adaptation of the language content, as certain terms and / or phrases may be more or less common in certain subject areas and thus more or less familiar to the user. This allows for a comprehensive characterization of the user's competence.

[0016] Optionally, the competency profile is captured through user input and / or determined by identifying the user based on camera data. Through user input, the competency profile and / or a parameter describing the competency profile or competency can be captured for each individual user. Camera data can be used to identify the user and, for example, link them to a stored competency profile and / or estimate the user's competency.

[0017] Optionally, determining the audio output content includes: converting the speech content into text; determining the complexity of the text based on a complexity analysis; determining the output content based on the complexity of the competency profile and the text; and converting the output content into the audio output content. In other words, the speech content is converted into text, the text is adapted, and then converted back into the audio output content as audio content. It has been recognized that text can be effectively classified and / or generated using machine learning. Classifiability enables effective complexity analysis, and generatability enables effective determination of the output content, for example, by modifying and / or generating sections that include simpler formulations and / or explanations.

[0018] Optionally, the output content is determined taking into account a complexity threshold defined by the competency profile. A complexity value can be assigned to a language content. If the complexity value falls below the threshold, the language content can be retained because the language content is not too complicated. If the complexity value exceeds the threshold, the language content can be adjusted because the language content is too complicated. The threshold can be user-specific, driving situation-related, and / or specific to a topic area of ​​the language content.

[0019] Optionally, the output content is converted into the audio output content, taking into account the speaker of the speech content. The speaker of the speech content can be determined by classification and / or retrieved from metadata of the audio content. By taking the speaker into account, the audio output content can be adapted to the original audio content.

[0020] Optionally, the competency profile and / or audio output content is determined taking into account a driving situation. It has been recognized that competency can be limited, for example, by the driving situation, particularly a potentially dangerous situation, darkness, travel time, etc., and thus the user's competence can decrease. The driving situation can also be a driving situation predicted based on a route, for example. Furthermore, the audio output content can be adapted to the driving situation, for example, to prevent more difficult-to-understand sections from being played when there is an increased risk potential.

[0021] In other words, and with reference to a specific embodiment, which is described as non-limiting for the present disclosure, the above description can be summarized as follows: The disclosure relates to the at least partial replacement of spoken audio content for adaptation to the listener's competency profile. Various applications exist for reproducing speech content as audio content, e.g., podcasts, audio books, for example in vehicles and / or via smartphone. Speech content is generally reproduced exactly as it was recorded or published. If playback is adapted, only the playback speed is modified; even more rarely, parts are skipped. However, depending on the listener's competency, speech content can be difficult to understand, in the sense of "comprehending" or "understanding," not in the sense of "acoustically perceiving."Both the language and subject-matter competence of the listener play a role here. According to the disclosure, it is proposed that the actual spoken content of an audio recording be adapted to the listener's competence. The goal is for the listeners to be able to understand the content as well as possible. First, the listeners' competence must be determined. This can be done in different ways, e.g.: 1. The listeners' faces are recognized using camera images: a.) The listeners can be identified; for each listener, either a stored competence profile (both language and subject-matter competence) is available, or a query is made for as yet unknown listeners and / or b.) The age of the faces is estimated, and this allows at least an assessment of the language level to be made (e.g., the older the person, the higher the competence). The minimum competence of all listeners is determined. 2.The language and subject matter expertise of the audio content to be played is entered directly by the user. The speech recording is then analyzed based on the following seven steps: 1. The speech recording is converted into text using existing speech-to-text methods. 2. A complexity analysis of the text is performed; for example, individual words as well as (partial) sentences, sentence structures, tenses, etc. can be classified according to their respective complexity. A word can, for example, be assessed according to its frequency of use (the less often it is used, the more complex it is). Technical terms that are difficult for a listener unfamiliar with a subject area to understand can also be identified. 3. The different speakers in the speech recording are identified. 4. A voice model is trained for each speaker, or, if available, an existing voice model is retrieved. 5. If the complexity of a word or(partial) sentence exceeds the listener's competency profile, alternative words, formulations and explanations are identified that are suitable for the listener's competency profile but correspond as closely as possible to the original spoken content. 6. The alternative words / phrases are synthesized into speech using the voice model of the speaker who spoke the original content to be replaced. 7. Finally, the complex words / phrases are replaced with the newly generated simplified speech content. The speech content can be adapted in advance for the entire recording or it can be done on-the-fly during playback. If the listener gives feedback on which areas were understood well and which poorly, this can be used to learn a personalized competency profile of the listener. Facial expressions and gestures during listening can also be used as feedback (e.g. a camera records whether the brow is frowning).

[0022] According to one aspect of the invention, a computer program and / or a computer-readable medium is provided. The computer program and / or the computer-readable medium comprise instructions which, when the program or instructions are executed by a computer, cause the computer to perform the method according to the invention and / or steps thereof. Optionally, the computer program and / or the computer-readable medium comprises instructions which, when the program or instructions are executed by a computer, cause the computer to perform the method steps described as advantageous or optional in order to achieve an associated technical effect.

[0023] According to one aspect of the invention, a data processing device for a motor vehicle is provided. The data processing device is configured to carry out the method described above. Optionally, the data processing device is configured to carry out a method step described as advantageous or optional and / or to implement a method feature in order to achieve an associated technical effect.

[0024] According to one aspect of the invention, a motor vehicle comprising the data processing device described above is provided. Optionally, the data processing device of the motor vehicle and / or the motor vehicle is configured to perform a method step described as advantageous or optional and / or to implement a method feature in order to achieve an associated technical effect.

[0025] In the following, one embodiment is described with reference to the figures. Fig. 1 schematically shows a motor vehicle according to one aspect of the disclosure; Fig. 2 schematically shows a flow diagram of a method according to one aspect of the disclosure; Fig. 3 schematically shows an alternative representation of a flow diagram of steps of a method according to an aspect of the disclosure; and Fig. 4 schematically shows an audio content and an audio content adapted by a method according to an aspect of the disclosure.

[0026] Fig. Figure 1 schematically shows a motor vehicle 50 according to one aspect of the disclosure. The motor vehicle 50 is a land vehicle, for example, a passenger car.

[0027] In the motor vehicle 50 according to Fig. 1, there is a user 90 or passenger, for example, a driver or passenger. In an embodiment not shown, the motor vehicle 50 can accommodate multiple users 90.

[0028] The motor vehicle 50 comprises a data processing device 51 and an audio output device 55. The data processing device 51 and the audio output device 55 are connected to one another so that the data processing device 51 can initiate the output of acoustic signals and / or audio content 60 perceptible to the user 90 by the audio output device 55. Alternatively, the data processing device 51 can be configured to transmit audio content 60 to a user terminal, for example, a smartphone, a wearable, headphones, etc.

[0029] The motor vehicle 50 and the data processing device 51 are configured to carry out the method 100 according to Fig. 2 and Fig. 3 to be carried out.

[0030] For this purpose, the data processing device 51 is Fig. 1 is configured to capture audio content 60 comprising speech content 61. The audio content 60 can, for example, be transmitted to the data processing device 51 via the user terminal, retrieved from a server external to the vehicle, and / or received via radio. The speech content 61 comprises portions of the audio content 60, wherein the speech content 61 comprises content spoken by one or more speakers as acoustically perceivable information.

[0031] The data processing device 51 is configured to convert the speech content 61 into text 62 and to perform a complexity analysis to determine a complexity of the text 62 (see description of the Fig. 2 and Fig. 3).

[0032] The data processing device 51 is configured to determine a competency profile 95 characterizing the competency of the user 90. For this purpose, the data processing device 51 can capture a user input 91 and / or be configured to identify the user 91 based on image data from an interior camera and thus determine the competency profile 95. The competency includes language competency and subject-matter competency. The language competency particularly includes the language level, which describes the general listening comprehension of the user 90 in a plurality of classes, for example, from beginner to native speaker. The subject-matter competency optionally describes the knowledge the user 90 has of the subject of the audio content 60 or the language content 62.

[0033] The data processing device 51 is configured to determine an output content 64 based on the complexity and the competency profile 95 and the text 62 (see description of the Fig. 2 and Fig. 3).

[0034] The data processing device 51 is configured to convert the output content 64 into an audio output content 65 and to output it as adapted audio content 60' that is acoustically perceptible to the user 90. For this purpose, the data processing device 51 can be configured to output adapted audio content 60' via the vehicle-mounted audio output device 55 and / or to transmit it to the user terminal for output by the user terminal.

[0035] If there are multiple users 90, a minimum competency profile 95 can be determined to adapt the audio content 60 and / or the audio content 60 can be adapted to each individual user and output differently for different users 90, for example, via different audio output devices 55. The adapted audio content 60 can comprise different audio tracks in parallel, which are played back simultaneously in the motor vehicle 50. Optionally, the adapted audio content 60 includes synchronization sections in which the adapted audio content 60 is identical for each user 90.

[0036] Fig. 2 schematically shows a flow diagram of a method 100 according to one aspect of the disclosure. The method 100 according to Fig. 2 is a method 100 for adapting an audio content 60 to a competency profile 95 of a user 90 of a motor vehicle 50. Such a motor vehicle 50 is described with reference to Fig. 1 described. Fig. 2 is with reference to Fig. 1 described.

[0037] The procedure 100 according to Fig. 2 comprises: capturing 110 an audio content 60 comprising a speech content 61 (see Fig. 1).

[0038] The method 100 comprises: determining 120 a competency profile 95 characterizing a competency of the user 90. The competency of the user 90 is determined (see also Fig. 1). This can be done in different ways. The competency profile 95 is captured by a user input 91 and / or determined by identifying the user 90 based on camera data and / or other sensor data. By capturing the user input 91, the language and / or subject-matter competence regarding the audio content 60 to be played is entered directly by the user 60. Furthermore, the competency profile 95 can be determined based on camera images in which the face of the user 90 and / or the faces of the users 90 are recognized. The user(s) 90 can be identified by classifying the image data, with either a stored competency profile 95 (both language and subject-matter competence) being available for each user 90 or, for unknown users 90, the age of the user 90 being estimated, for example, in order to be able to assess the language level; for example, the older the user 90, the higher the language level.The competency profile 95 thus comprises a classification of a language level of the user 90 and / or a technical level of the user 90 related to the language content 61 or an estimate thereof based on age analogous to the estimate of language competence.

[0039] The method 100 comprises: determining 130 an audio output content 65 taking into account the speech content 61 and the competency profile 95 (see Fig. 3). Determining 130 the audio output content 65 comprises: converting 131 the speech content 61 into text 62; determining 132 a complexity of the text 62 based on a complexity analysis; determining 133 an output content 64 based on the complexity and the competency profile 95 and the text 62; and converting 134 the output content 64 into the audio output content 65. The output content 65 is determined 133 taking into account a threshold value relating to the complexity and defined by the competency profile 95. The output content 64 is converted 134 into the audio output content 65 taking into account a speaker of the speech content 61.

[0040] In the aforementioned steps, the competency profile 95 and / or the audio output content 65 can be determined taking into account a driving situation. For this purpose, the driving situation can be determined and / or estimated (predicted) for the future based on, for example, environmental sensors, traffic information, route planning, and so on.

[0041] The method 100 comprises: outputting 140 an adapted audio content 60` based on the audio output content 65. In this case, the adapted audio content 60` comprising the audio output content 65 is output (see Fig. 1).

[0042] Optionally, feedback from user 90 on the adapted audio content 60` can be recorded. If user 90 provides feedback regarding which sections of the adapted audio content 60` were understood well and which sections of the adapted audio content 60` were understood poorly, this can be used to learn a personalized competency profile 95 of user 90. Facial expressions such as a furrowed brow and / or gestures while listening to the adapted audio content 60` can optionally be automatically recorded and used via the interior camera as feedback. The feedback can be used to define and / or adjust the threshold value that indicates whether a speech content 61 and / or text 62 is too complicated and therefore needs to be replaced.

[0043] The adaptation of the speech content 61 can be made in advance for the entire recording or the entire audio content 60 or the adaptation can be made on-the-fly during playback.

[0044] Fig. 3 schematically shows an alternative representation of a flowchart of steps of a method 100 according to one aspect of the disclosure. The method 100 is described with reference to Fig. 2 described. Fig. 3 is made with reference to Fig. 1 and Fig. 2. In particular, Fig. 3 steps to determine 130 an audio output content 65 taking into account the language content 61 and the competency profile 95.

[0045] First, according to Fig. 3, the audio content 60 is captured with the speech content 61. The speech content 61 or the voice recording is converted into text 62. The conversion 131 of the speech content 61 into the text 62 is performed by the data processing device 51 using a speech-to-text method.

[0046] A complexity of the text 62 is determined 132. For this purpose, the data processing device 51 performs a complexity analysis of the text 62. For example, individual sections such as word components, words, clauses, sentences, and / or sentence structures, including their grammatical structure, for example, through tenses, can be classified with regard to their respective complexity. For example, a word can be assessed with regard to its frequency of use; the less frequently the word is used, the more complex it is. Technical terms can also be identified that are more difficult to understand for a user 90 who is unfamiliar with a specific field or less.

[0047] A speaker determination 133a is performed by the data processing device 51. In this process, a speaker and / or various speakers of the audio content 60 are identified, for example, based on metadata of the audio content 60 and / or classification of the audio content 60 or the voice. Depending on the speaker and / or speakers, a voice training 133b is performed, with a voice model being trained for each speaker and / or, if available, an existing voice model being retrieved.

[0048] Output content 64 is determined 133 based on the complexity and the competency profile 95 and the text 62. If the complexity of a section of the text, for example a word, clause, and / or sentence, exceeds the competency profile 95 of the user 90, i.e., the complexity exceeds a threshold, adapted or alternative words, formulations, and explanations are determined as output content 64 that are suitable for the competency profile 95 of the user 90, but correspond as faithfully as possible to the original spoken content. If the complexity falls below the threshold, i.e., the text 62 is not too complicated and comprehensible for the user 90, the text 62 can be incorporated unchanged into the output content 64.

[0049] The output content 64 is converted 134 into the audio output content. The adapted sections of the text 62 are synthesized into speech using the voice model of the speaker who spoke the original content to be replaced.

[0050] The unaltered and adapted sections are merged. More complex words / phrases are replaced with the adapted, generated simplified language content.

[0051] Fig. 4 schematically shows an audio content 60 ( Fig. 4, above) and an audio content 60` adapted by a method 100 according to one aspect of the disclosure ( Fig. 4, below). The method 100 is described with reference to Fig. 2 and Fig. 3 described. Fig. 4 is made with reference to Fig. 1 to 3.

[0052] In Fig. 4 above, a portion of an audio recording is shown as an audio content 60 by way of example. The audio content 60 is divided into three sections or phases (1), (2), and (3). The audio content 60 comprises speech content 61, which in phases (1) and (3) includes word components, words, and / or sentences that do not exceed the competency defined by the competency profile 95 of the user 90. The complexity of the audio content 60 and / or speech content 61 in phases (1) and (3) thus falls below the threshold value in each case. Phase (2) comprises word components, words, and / or sentences that exceed the competency defined by the competency profile 95 of the user 90. The complexity of the audio content 60 and / or speech content 61 in phase (2) thus exceeds the threshold value.

[0053] According to procedure 100 according to Fig. 1 and Fig. 2, an alternative formulation for the audio content 60 and / or speech content 61 is determined in phase (2) and synthesized into the new section (2'). Subsequently, section (2) in the audio content 60 is Fig. 4, above by section (2') in the adapted audio content 60` according to Fig. 4, below. Sections (1) and (3) are transferred unchanged from audio content 60 to the adapted audio content 60`.

[0054] In this example, the adjusted section (2') lasts longer than the original section (2). Reference symbol (part of the description) 50 motor vehicles 51 Data processing device 55 Audio output device 60 audio content 60` adapted audio content 61 Language content 62 Text 64 Output content 65 Audio output content 90 users 91 User input 95 Competence profile 100 procedures 110 Capture 120 Determining a competency profile 130 Specifying audio output content 131 Convert into text 132 Determining a complexity 133a Determination of speaker 133b Language training 133 Determining an output content 134 Convert to audio output content 135 Merge 140 Issues QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2021 108 042 A1

[0004] US 9,405,741 B1

[0006] EP 1 172 804 B1

[0007]

Claims

[1] Method (100) for adapting an audio content (60) to a competency profile (95) of a user (90) of a motor vehicle (50), the method (100) comprising: - capturing (110) an audio content (60) comprising a speech content (61); - determining (120) a competency profile (95) characterizing a competency of the user (90); - determining (130) an audio output content (65) taking into account the speech content (61) and the competency profile (95); and - Outputting (140) an adapted audio content (60`) based on the audio output content (65). [2] Method (100) according to claim 1, wherein the competency profile (95) comprises a classification of a language level of the user (90) and / or a technical level of the user (90) related to the language content (61). [3] Method (100) according to claim 1 or 2, wherein the competency profile (95) is detected by a user input (91) and / or is determined by an identification of the user (90) based on camera data. [4] Method (100) according to one of the preceding claims, wherein determining (130) the audio output content (65) comprises: - Conversion (131) of the language content (61) into text (62); - determining (132) a complexity of the text (62) based on a complexity analysis; - determining (133) an output content (64) based on the complexity and the competency profile (95) and the text (62); and - Converting (134) the output content (64) into the audio output content (65). [5] Method (100) according to claim 4, wherein the determination (133) of the output content (65) takes place taking into account a threshold value relating to the complexity and defined by the competency profile (95). [6] Method (100) according to claim 4 or 5, wherein the conversion (134) of the output content (64) into the audio output content (65) takes place taking into account a speaker of the speech content (61). [7] Method (100) according to one of the preceding claims, wherein the competency profile (95) and / or the audio output content (65) is determined taking into account a driving situation. [8] Computer program and / or computer-readable medium comprising instructions which, when the program or instructions are executed by a data processing device (51), cause the device (51) to carry out the method (100) and / or the steps of the method (100) according to one of claims 1 to 7. [9] Data processing device (51) for a motor vehicle (50), wherein the data processing device (51) is configured to carry out the method (100) according to one of claims 1 to 7. [10] Motor vehicle (50) comprising the data processing device (51) according to claim 9.

Images