Vehicle-mounted sound control method and device

Abstract

The application relates to a vehicle-mounted sound control method and device, which comprises the following steps: identifying at least two sound requests; obtaining current driving scene information and determining the working cognitive load degree of a driver according to the current driving scene information; determining the attention resource demand corresponding to each sound and respectively assigning each sound according to the determined attention resource demand; determining the priority corresponding to each sound and determining the corresponding sound emission rule according to the working cognitive load degree, the assignment of each sound and the priority of each sound; and controlling the audio unit to emit sound according to the sound emission rule and the priority corresponding to each sound. The application dynamically controls the vehicle-mounted sound in different driving scenes, so that the different sound demands of high-load driving scenes and low-load driving scenes can be better met.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to an in-vehicle voice control method and device. Background Technology

[0002] With the development of automotive intelligence, in-vehicle active safety prompts are becoming more comprehensive, in-vehicle audio-visual entertainment functions are becoming richer, and the sounds that the vehicle system can play are becoming more diverse. How to avoid mutual interference between many in-vehicle sounds, and provide users with a more diverse sound experience while ensuring driving safety, has become an urgent problem to be solved.

[0003] In existing in-vehicle sound control technologies, mixing strategies are determined solely from the perspective of sound source category, ignoring the diverse sound requirements of different in-vehicle scenarios. Moreover, the multi-source playback strategy is too simplistic and fails to dynamically adapt to different driving scenarios. Consequently, it cannot better meet the different sound requirements of high-load and low-load driving scenarios, resulting in interference between multiple sounds in high-load scenarios or a lack of sound diversity in low-load scenarios, thus affecting driving safety. Summary of the Invention

[0004] One of the objectives of this invention is to provide an in-vehicle audio control method and apparatus to solve the problem that existing in-vehicle audio control technologies do not dynamically adapt to different driving scenarios, thus failing to better meet the different sound requirements of high-load and low-load driving scenarios.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A vehicle-mounted voice control method includes the following steps: At least two voice requests were identified; Obtain current driving scenario information and determine the driver's cognitive workload based on the current driving scenario information; Determine the attention resource requirements for each sound, and assign values ​​to each sound according to the determined attention resource requirements; Determine the priority of each sound, and determine the corresponding vocalization rules based on the cognitive load level, the value assigned to each sound, and the priority of each sound; The audio unit is controlled to emit sound according to the sound emission rules and the priority of each sound.

[0006] Based on the aforementioned technical means, since driving scenario information has been acquired, it can be analyzed to determine the current cognitive load of the driver, thereby mapping the driver's current attention resource capacity. Furthermore, based on the attention resource requirements and priorities corresponding to each sound, the sound rules in high-load or low-load scenarios can be adaptively adjusted. This allows for dynamic control of in-vehicle sounds in different driving scenarios, thereby better meeting the different sound requirements of high-load and low-load driving scenarios.

[0007] Furthermore, obtaining the current driving scenario information includes: It acquires information on driver proficiency, road conditions, traffic complexity, vehicle speed, and visibility.

[0008] Based on the aforementioned technical means, since information such as the driver's driving proficiency, road conditions, traffic complexity, vehicle speed, and road visibility are obtained in the current driving scenario, comprehensive analysis can be performed on this information to more accurately determine whether the current driving scenario is a high-load or low-load scenario.

[0009] Furthermore, determining the driver's cognitive workload based on the current driving scenario information includes: Based on the current driving scenario information, determine whether the current driver meets the conditions of the first load scenario or the second load scenario; If the conditions of the first load scenario are met, the current driving scenario is determined to be a high load scenario, and a first load value is assigned to the high load scenario. If the conditions of the second load scenario are met, the current driving scenario is determined to be a low load scenario, and a second load value is assigned to the low load scenario.

[0010] Based on the aforementioned technical means, since load values ​​are assigned for different load scenarios, the driver's attention resource capacity in different driving scenarios can be mapped according to the different assignments. This allows for the acquisition of vocalization rules corresponding to the driving scenario based on the relationship between the driver's attention resource capacity in different driving scenarios and the attention resource requirements of different sounds.

[0011] Furthermore, the conditions for the first load scenario are: It meets one or a combination of the following conditions: low driver proficiency, poor road conditions, high traffic complexity, high vehicle speed, and low visibility.

[0012] Based on the above technical means, since multiple judgment conditions under high-load scenarios are taken into account, the actual driving scenario information can be compared with the judgment conditions of high-load scenarios, so as to accurately determine the driving scenario in which the driver is currently located and match the vehicle sound emission rules that are suitable for the current driving scenario.

[0013] Furthermore, the conditions for the second load scenario are: It meets one or a combination of the following conditions: high driver proficiency, good road conditions, low traffic complexity, low vehicle speed, and high visibility.

[0014] Based on the above technical means, since multiple judgment conditions under low load scenarios are taken into account, the actual driving scenario information can be compared with the judgment conditions of low load scenarios, so as to accurately determine the driving scenario in which the driver is currently located and match the vehicle sound emission rules that are suitable for the current driving scenario.

[0015] Furthermore, determining the attention resource requirements for each sound and assigning values ​​to each sound based on the determined attention resource requirements includes: Obtain the audio source attention resource requirement assignment table; The sound source of each sound is determined, and the corresponding sound source assignment is found from the sound source attention resource requirement assignment table according to the determined sound source; Assign values ​​to the corresponding sounds based on the found sound sources.

[0016] Based on the aforementioned technical means, since each sound is assigned a value according to the value in the sound source attention resource requirement assignment table, the attention resource requirement of each sound can be accurately assigned, thereby determining the sound production rules based on the assignment of each sound and its corresponding priority in the current driving scenario.

[0017] Furthermore, the sound source attention resource requirement assignment table is a resource assignment table for the driver's required attention, which is obtained by comprehensively judging the driver's cognitive difficulty of sound information and the duration of sound.

[0018] Based on the aforementioned technical means, since the driver's required attention resource allocation table is obtained by comprehensively judging the driver's cognitive difficulty of sound information and the duration of sound, the attention resource requirements of each sound can be accurately assigned in actual driving scenarios, thereby more precisely controlling the sound unit to emit sounds that are adapted to the driving scenario.

[0019] Furthermore, determining the priority of each sound includes: Based on the current driving scenario and the source of each sound, determine the urgency level of each sound in the current driving scenario; The priority of each sound is determined based on its level of urgency.

[0020] Based on the aforementioned technical means, since the priority of each sound is determined according to its urgency in the current driving scenario, a mixing strategy can be determined according to the priority of different sounds, and the primary and secondary relationship of different sounds in playback can be determined, ensuring driver safety while also providing the driver with a diverse sound experience.

[0021] Furthermore, determining the corresponding vocalization rules based on the cognitive workload level, the assigned value of each sound, and the priority of each sound includes: Calculate the sum of the values ​​assigned to all sounds in the current driving scenario, and determine the driver's workload value in the current driving scenario based on the aforementioned cognitive workload level; The corresponding vocalization rules are determined based on the sum of the assigned values, the load value, and the priority of each sound.

[0022] Based on the above technical means, by establishing the summation relationship between the driver's attention resource capacity in different driving scenarios and the attention resource demand of different sounds, the rules for outputting sound when multiple sound requests occur are formulated, and the mixing strategy is determined according to the priority of different sounds, the primary and secondary relationship of different sounds in playback is determined, ensuring driver safety while also bringing drivers a diverse sound experience.

[0023] Furthermore, the vocalization rules include: In the high-load scenario, if the sum of the attention resource requirements of the high-priority sound and the low-priority sound is less than or equal to the first load value, the high-priority sound and the low-priority sound are mixed and played. In the high-load scenario, if the sum of the attention resource requirements of the high-priority sound and the low-priority sound is greater than the first load value, then the high-priority sound is played. In the low-load scenario, if the sum of the attention resource requirements of the high-priority sound and the low-priority sound is less than or equal to the second load value, the high-priority sound and the low-priority sound are mixed and played. In the low-load scenario, if the sum of the attention resource requirements of the high-priority sound and the low-priority sound is greater than the second load value, then the high-priority sound is played.

[0024] Based on the aforementioned technical means, since the summation relationship between attention resource capacity and attention resource demand of different sounds is set under different load scenarios, it is possible to formulate sound output rules for single tone or mixed tone when multiple sound requests occur, thereby better meeting the different sound needs of high-load driving scenarios and low-load driving scenarios.

[0025] Furthermore, controlling the audio unit to emit sound according to the sound emission rules and the priority corresponding to each sound includes: According to the sound production rules and the priority of each sound, the audio unit is controlled to play high-priority sounds; Alternatively, the audio unit can be controlled to play the mix according to the sound production rules, and the main and auxiliary tone types corresponding to each sound can be determined according to the priority of each sound during the mix playback. Based on the main and auxiliary tone types corresponding to each sound, volume adjustment and / or sound field avoidance and / or reverberation processing can be performed.

[0026] Based on the aforementioned technical means, by determining the mixing strategy according to the priority of different sounds, the primary and secondary relationships of different sounds during playback can be determined. This allows for dynamic control of in-vehicle sound in different driving scenarios, thereby better meeting the different sound requirements of high-load and low-load driving scenarios.

[0027] A vehicle-mounted voice control device, comprising: The voice recognition module is used to identify at least two voice requests; The cognitive load module is used to acquire current driving scenario information and determine the driver's cognitive load level based on the current driving scenario information. The attention resource requirement module is used to determine the attention resource requirement for each sound and assign values ​​to each sound according to the determined attention resource requirement. The vocalization rule module is used to determine the priority of each sound and to determine the corresponding vocalization rule based on the cognitive load level, the value assigned to each sound, and the priority of each sound. The sound control module is used to control the audio unit to emit sound according to the sound emission rules and the priority of each sound.

[0028] Furthermore, the cognitive load module includes: The scene information acquisition module is used to acquire information such as driver proficiency, road conditions, traffic complexity, vehicle speed, and road visibility.

[0029] Furthermore, the cognitive load module also includes: The load scenario determination module is used to determine whether the current driver meets the conditions of the first load scenario or the second load scenario based on the current driving scenario information. A high-load scenario determination module is used to determine that the current driving scenario is a high-load scenario when the conditions of the first load scenario are met, and to assign a first load value to the high-load scenario; The low-load scenario determination module is used to determine that the current driving scenario is a low-load scenario when the conditions of the second load scenario are met, and to assign a second load value to the low-load scenario.

[0030] Furthermore, the conditions for the first load scenario are: It meets one or a combination of the following conditions: low driver proficiency, poor road conditions, high traffic complexity, high vehicle speed, and low visibility.

[0031] Furthermore, the conditions for the second load scenario are: It meets one or a combination of the following conditions: high driver proficiency, good road conditions, low traffic complexity, low vehicle speed, and high visibility.

[0032] Furthermore, the attention resource requirement module includes: The assignment table retrieval module is used to retrieve the audio source attention resource requirement assignment table; The lookup module is used to determine the sound source of each sound and look up the corresponding sound source assignment from the sound source attention resource requirement assignment table according to the determined sound source. The assignment module is used to assign values ​​to the corresponding sounds based on the searched sound source.

[0033] Furthermore, the sound source attention resource requirement assignment table is a resource assignment table for the driver's required attention, which is obtained by comprehensively judging the driver's cognitive difficulty of sound information and the duration of sound.

[0034] Furthermore, the vocalization rule module includes: The urgency level determination module is used to determine the urgency level of each sound in the current driving scenario based on the current driving scenario and the sound source of each sound. The priority determination module is used to determine the priority of corresponding sounds based on the determined urgency level.

[0035] Furthermore, the vocalization rules module also includes: The assignment calculation module is used to calculate the sum of the assignments of all sounds in the current driving scenario, and to determine the driver's workload value in the current driving scenario based on the work cognitive load level. The vocalization rule determination module is used to determine the corresponding vocalization rule based on the sum of the assigned values, the load value, and the priority of each sound.

[0036] Furthermore, the vocalization rules include: In the high-load scenario, if the sum of the attention resource requirements of the high-priority sound and the low-priority sound is less than or equal to the first load value, the high-priority sound and the low-priority sound are mixed and played. In the high-load scenario, if the sum of the attention resource requirements of the high-priority sound and the low-priority sound is greater than the first load value, then the high-priority sound is played. In the low-load scenario, if the sum of the attention resource requirements of the high-priority sound and the low-priority sound is less than or equal to the second load value, the high-priority sound and the low-priority sound are mixed and played. In the low-load scenario, if the sum of the attention resource requirements of the high-priority sound and the low-priority sound is greater than the second load value, then the high-priority sound is played.

[0037] Furthermore, the sound control module includes: The high-priority sound control module is used to control the audio unit to play high-priority sounds according to the sound production rules and the priority of each sound. The mixing control module is used to control the audio unit to play the mix according to the sound production rules, and to determine the main and auxiliary tone types of each sound during the mix playback according to the priority of each sound, and to perform volume adjustment and / or sound field avoidance and / or reverberation processing based on the main and auxiliary tone types of each sound.

[0038] A vehicle includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the in-vehicle voice control method.

[0039] A computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the aforementioned vehicle voice control method.

[0040] The beneficial effects of this invention are: (1) The embodiments of the present invention can help drivers quickly obtain information under high-load driving conditions, avoid secondary sounds from interfering with the driver's driving tasks, and thus effectively ensure driving safety.

[0041] (2) The embodiments of the present invention can help drivers obtain different information in a timely manner under low-load driving conditions, and meet the diverse sound needs of drivers.

[0042] (3) The embodiments of the present invention can analyze the driving scenario information to determine the current cognitive load of the driver and thus map the driver's current attention resource capacity. Furthermore, based on the attention resource requirements and priorities corresponding to each sound, the sound rules under high load or low load scenarios can be adapted to dynamically control the in-vehicle sound under different driving scenarios, thereby better meeting the different sound requirements of high load driving scenarios and low load driving scenarios. Attached Figure Description

[0043] Figure 1 This is a flowchart of the vehicle-mounted sound control method of the present invention; Figure 2 This is a schematic diagram of the structure of the vehicle-mounted sound control device of the present invention; Figure 3 This is a structural schematic diagram of the vehicle provided by the present invention. Detailed Implementation

[0044] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0045] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0046] The following description, with reference to the accompanying drawings, illustrates an embodiment of an in-vehicle audio control method and apparatus according to the present invention. Addressing the problem mentioned in the background section that existing in-vehicle audio control technologies do not dynamically adapt to different driving scenarios, thus failing to better meet the different sound requirements of high-load and low-load driving scenarios, the present invention provides an in-vehicle audio control method. This method analyzes driving scenario information to determine the current cognitive load of the driver, thereby mapping the driver's current attention resource capacity. Furthermore, based on the attention resource requirements and priorities corresponding to each sound, it adaptively adjusts the sound rules for high-load or low-load scenarios, thereby outputting the required sound effect using sound rules adapted to the current driving scenario while ensuring safe driving. Therefore, the present invention can dynamically control in-vehicle audio in different driving scenarios, thereby better meeting the different sound requirements of high-load and low-load driving scenarios.

[0047] Specifically, Figure 1 This is a flowchart of an embodiment of the vehicle sound control method of the present invention.

[0048] like Figure 1As shown, the vehicle-mounted voice control method includes the following steps: In step S100, at least two voice requests are identified.

[0049] It is understood that the vehicle control system in this embodiment of the invention can control the output of various types of sounds; wherein, the sound sources of the output sounds include, but are not limited to: E / B-call (emergency call / one-click rescue service), telephone, TTS broadcast (text message broadcast), navigation broadcast, alarm prompt tone, voice dialogue, turn signal sound, system prompt tone, and multimedia audio sounds; this embodiment of the invention can, during the driver's driving process, analyze at least two sound playback requests from the above-mentioned sound sources issued by the vehicle control system based on driving scenario information to determine the current cognitive load of the driver, thereby mapping the driver's current attention resource capacity; furthermore, based on the attention resource requirements and priorities corresponding to each sound, the sound rules can be adaptively adjusted for high-load or low-load scenarios, thereby dynamically controlling the vehicle sounds in different driving scenarios to better meet the different sound requirements of high-load and low-load driving scenarios.

[0050] like Figure 1 As shown, the vehicle-mounted voice control method includes the following steps: In step S200, current driving scenario information is obtained, and the driver's cognitive workload is determined based on the current driving scenario information.

[0051] It is understood that in this embodiment of the invention, the current driving scenario information of the driver and the vehicle can be obtained through vehicle sensors and cameras, etc. The obtained current driving scenario information includes, but is not limited to: driver driving proficiency information, road condition information, traffic complexity information, vehicle speed status information, and road condition visibility information. Since this information in the current driving scenario is obtained, it can be analyzed to accurately determine the driver's cognitive load in the current driving scenario, avoiding the interference of multiple sounds in high-load scenarios that affect driving safety, and the lack of diverse sound experiences in low-load scenarios.

[0052] In one embodiment of the present invention, obtaining current driving scenario information includes: obtaining driver driving proficiency information, road condition information, traffic complexity information, vehicle speed status information, and road condition visibility information. It is understood that the above-mentioned driver driving proficiency information can be obtained by comprehensive analysis based on the driver's emergency operations (e.g., sudden braking, sudden acceleration, etc.) recorded in the vehicle system, the driver's facial emotions (e.g., tense facial expressions) captured by the camera, and the reaction time for emergency avoidance (e.g., the time difference between the moment the system detects the dangerous target and the moment the driver brakes suddenly), or it can be obtained through human-computer interaction, etc. The obtained driver driving proficiency is either high or low.

[0053] The aforementioned road condition information can be obtained by combining road surface video acquired by the camera with the driver's driving operations, and the system can detect and analyze information such as road surface smoothness, road surface slope, and road curvature. The resulting road condition information is categorized as either "good" or "poor." The aforementioned traffic complexity information can be obtained by combining road surface video acquired by the camera with real-time navigation information, and the system can detect and analyze this information. The resulting traffic complexity information is categorized as either "high" or "low." The aforementioned vehicle speed status information can be obtained by acquiring real-time vehicle speed through a vehicle speed sensor and analyzing it. The resulting vehicle speed status information is categorized as either "high" or "low." The aforementioned road visibility clarity information can be obtained by analyzing video of the scene directly ahead acquired by the camera. The resulting clarity information is categorized as "low clarity during the day," "high clarity during the day," "low clarity at night," "high clarity on sunny days," and "low clarity on rainy days."

[0054] In one embodiment of the present invention, determining the driver's cognitive workload based on current driving scenario information includes: determining whether the current driver meets the conditions of a first load scenario or a second load scenario based on the current driving scenario information; if the conditions of the first load scenario are met, the current driving scenario is determined to be a high load scenario, and a first load value is assigned to the high load scenario; if the conditions of the second load scenario are met, the current driving scenario is determined to be a low load scenario, and a second load value is assigned to the low load scenario.

[0055] It is understood that, in this embodiment of the invention, the driver's current cognitive workload is determined by judging information such as the driver's driving proficiency, road conditions, traffic complexity, vehicle speed, and road visibility. This determines the driver's current attentional resource capacity. The cognitive workload includes high workload and low workload.

[0056] Optionally, different driving scenarios can be assigned values ​​based on the level of cognitive workload. For high-load scenarios, in one embodiment, the high-load scenario can be assigned a value of 30 (i.e., the first load value); while for low-load scenarios, in one embodiment, the low-load scenario can be assigned a value of 40 (i.e., the second load value).

[0057] In one embodiment of the present invention, the determination conditions for the first load scenario, based on the driver's driving proficiency information, road condition information, traffic complexity information, vehicle speed status information, and road condition visibility clarity information obtained above, include, but are not limited to: low driver's driving proficiency, poor road conditions, high traffic complexity, high vehicle speed, and low road condition visibility clarity; the determination conditions for the second load scenario include, but are not limited to: high driver's driving proficiency, good road conditions, low traffic complexity, low vehicle speed, and high road condition visibility clarity.

[0058] For example, novice drivers (with low driving proficiency) face a high cognitive workload when driving on roads with complex conditions because their low proficiency and complex road conditions require them to concentrate fully on the driving task; while experienced drivers (with high driving proficiency) face a low cognitive workload when driving on roads with complex conditions and are more at ease in dealing with complex road conditions.

[0059] Based on the aforementioned technical means, by acquiring information such as the driver's driving proficiency, road conditions, traffic complexity, vehicle speed, and road visibility in the current driving scenario, comprehensive analysis can be performed on this information to more accurately determine whether the current driving scenario is a high-load or low-load scenario. Furthermore, by assigning load values ​​to different load scenarios, the driver's attention resource capacity in different driving scenarios can be mapped according to different assignments. This allows for the acquisition of vocalization rules corresponding to the driving scenario based on the relationship between the driver's attention resource capacity in different driving scenarios and the attention resource requirements of different sounds.

[0060] like Figure 1 As shown, the vehicle-mounted voice control method includes the following steps: In step S300, the attention resource requirements for each sound are determined, and each sound is assigned a value according to the determined attention resource requirements.

[0061] It is understood that, in this embodiment of the invention, each sound in the vehicle control system can be assigned a corresponding attention resource requirement value. In the process of assigning attention requirements, the driver's attention resource requirement for each sound information can be determined first, and then a value can be assigned to it. The driver's attention resource requirement for each sound is affected by various factors, including but not limited to the following factors: the duration of the sound, the driver's cognitive difficulty of the sound information, and the driver's familiarity with different sounds. Specifically, the sound source attention resource requirement assignment is shown in Table 1 below.

[0062] Table 1. Audio Source Attention Resource Requirements Assignment Table

[0063] Understandably, the values ​​assigned in Table 1 above indicate that telephone-type voice communication requires sustained driver attention and feedback, involves a high degree of driver involvement, and lasts for a relatively long period; therefore, the attention resource requirement for these sound sources is assigned a high value. Navigation broadcasts from unfamiliar road sections require careful listening and understanding from the driver, and also last for a relatively long period, thus requiring a high attention resource value. Conversely, familiar warning sounds (such as seatbelt unfastened or turn signal sounds) are easily recognized, resulting in a low attention resource value. The values ​​in Table 1 are for reference only.

[0064] It is worth noting that the audio sources and assignments in Table 1 are only examples and can be added or changed according to the in-vehicle audio sources. The assignment relationships can be given specific values ​​according to different judgment dimensions. This embodiment of the invention does not impose any restrictions.

[0065] In one embodiment of the present invention, the attention resource requirement corresponding to each sound is determined, and each sound is assigned a value according to the determined attention resource requirement, including: obtaining a sound source attention resource requirement assignment table; determining the sound source of each sound, and searching for the corresponding sound source assignment from the sound source attention resource requirement assignment table according to the determined sound source; and assigning a value to the corresponding sound according to the searched sound source assignment.

[0066] Based on the aforementioned technical means, since each sound is assigned a value according to the value in the sound source attention resource requirement assignment table, the attention resource requirement of each sound can be accurately assigned. Thus, the sound production rules can be determined according to the assignment of each sound and its corresponding priority in the current driving scenario. Moreover, since the driver's required attention resource assignment table is obtained by comprehensively judging the driver's cognitive difficulty of sound information and the duration of the sound, the attention resource requirement of each sound can be accurately assigned in the actual driving scenario, thereby more precisely controlling the sound unit to emit sounds that are adapted to the driving scenario.

[0067] like Figure 1As shown, the vehicle-mounted voice control method includes the following steps: In step S400, the priority of each sound is determined, and the corresponding vocalization rules are determined based on the cognitive load level, the value assigned to each sound, and the priority of each sound.

[0068] It is understood that in this embodiment of the invention, the priority of each sound can be determined according to the urgency of the current driving scenario, and then a mixing strategy can be determined according to the priority of different sounds to determine the primary and secondary relationship of different sounds in playback, thereby ensuring driver safety while also providing the driver with a diverse sound experience.

[0069] In one embodiment of the present invention, determining the priority of each sound includes: determining the urgency of each sound in the current driving scenario based on the current driving scenario and the sound source of each sound; and determining the priority of the corresponding sound based on the determined urgency.

[0070] Specifically, the priority of different sounds is determined based on their importance and urgency, including but not limited to the following scenarios: When driving at high speeds, navigation information is immediate and guiding. Drivers need to quickly obtain navigation information, make reasonable route plans, and perform related driving tasks. At this time, multimedia audio is a task unrelated to driving and has a lower level of importance and urgency. Therefore, when navigation broadcasts and multimedia are presented at the same time, navigation takes higher priority than multimedia.

[0071] When driving on complex road sections, if you suddenly encounter road construction or an accident ahead, and the driver needs to quickly change lanes or exit the highway, the urgency and importance of the feedback information from the voice control command will decrease. Therefore, when navigation broadcasts and voice dialogues occur simultaneously, navigation broadcasts take precedence over voice dialogues.

[0072] When driving at low speeds, if a phone call or multimedia device is available, the phone call is of higher importance and urgency to the driver, so the phone call takes priority over the multimedia device.

[0073] While driving at high speeds, both phone calls and multimedia sounds are irrelevant to driving tasks. Phone calls require the driver to maintain continuous attention, making them more disruptive to driving. In this situation, to ensure driving safety, multimedia sounds should have higher priority than phone calls, but it is also necessary to control the interference of both sounds on driving.

[0074] Based on the above scenario, the priority of two sound requests is determined by the urgency and importance of different sounds to the user, thereby determining the primary and secondary relationship in sound playback. The primary and secondary relationship includes, but is not limited to, the order of sound playback and the volume of the sounds.

[0075] In one embodiment of the present invention, the corresponding vocalization rules are determined based on the level of cognitive workload, the assigned value of each sound, and the priority of each sound, including: calculating the sum of the assigned values ​​of all sounds in the current driving scenario, and determining the driver's workload value in the current driving scenario based on the level of cognitive workload; and determining the corresponding vocalization rules based on the sum of assigned values, the workload value, and the priority of each sound.

[0076] Specifically, by establishing a comparative relationship between the driver's attention resource capacity and the attention resource requirements of different sounds in different driving scenarios, as well as the priority of different sounds, a sound output rule can be formulated to determine whether to output a single tone or a mixed tone when multiple sound requests occur.

[0077] The specific rules for vocalization are as follows: 1. High-load scenario: The total demand for attention resources from high-priority sound and low-priority sound is less than or equal to 30 (i.e., the first load value), so control the mixing and playback of the two sounds.

[0078] 2. High-load scenario: If the total attention resource requirement of high-priority sound and low-priority sound is greater than 30, only high-priority sound will be played.

[0079] 3. Low-load scenario: The total demand for attention resources from high-priority sound and low-priority sound is less than or equal to 40 (i.e., the second load value), and the two sounds are mixed and played.

[0080] 4. Low-load scenario: If the ratio of high-priority sound to low-priority sound is greater than 40, only the high-priority sound will be played.

[0081] Based on the above technical means, by establishing the summation relationship between the driver's attention resource capacity in different driving scenarios and the attention resource demand of different sounds, the rules for outputting sound when multiple sound requests occur are formulated, and the mixing strategy is determined according to the priority of different sounds, the primary and secondary relationship of different sounds in playback is determined, ensuring driver safety while also bringing drivers a diverse sound experience.

[0082] like Figure 1 As shown, the vehicle-mounted voice control method includes the following steps: In step S500, the audio unit is controlled to emit sound according to the sound emission rules and the priority of each sound.

[0083] It is understood that in the embodiments of the present invention, the sound playback effect is determined according to the sound rules and priorities, the mixing strategy can be determined according to the priorities of different sounds, and the primary and secondary relationships of different sounds in the playback can be determined. Thus, for different driving scenarios, the in-vehicle sound can be dynamically controlled, thereby better meeting the different sound requirements of high-load driving scenarios and low-load driving scenarios.

[0084] In one embodiment of the present invention, controlling the audio unit to emit sound according to the sound emission rules and the priority corresponding to each sound includes: controlling the audio unit to play high-priority sounds according to the sound emission rules and the priority corresponding to each sound; or controlling the audio unit to play a mix according to the sound emission rules, and determining the main and auxiliary tone types corresponding to each sound when playing the mix according to the priority corresponding to each sound, and performing volume adjustment and / or sound field avoidance and / or reverberation processing based on the main and auxiliary tone types corresponding to each sound.

[0085] Specifically, examples will be given of methods for controlling at least two sounds.

[0086] When traveling on a highway, a user needs to change lanes as they approach an exit ramp. They activate their turn signal (attention resource requirement assigned a value of 10), and the turn signal sound plays continuously. Simultaneously, the navigation system announces an exit ramp ahead (attention resource requirement assigned a value of 20). The navigation information is crucial for the user's quick comprehension and influences their next driving task. However, the turn signal sound is more familiar and requires less attention. The combined attention resource requirement of both sounds is less than the user's current attention resource capacity (currently in a high-load state, current attention resource capacity is 30; the sum of the attention resource requirements of the turn signal and navigation is 30, therefore, according to the rules, the two sounds are mixed). Therefore, the two sounds can be directly mixed and output, with the navigation information as the primary sound and the turn signal sound as the secondary sound. The processing methods for the primary and secondary sounds include, but are not limited to, using audio processing techniques such as volume adjustment, sound field avoidance, and reverb.

[0087] In assisted driving mode, the camera detects road construction ahead and an alarm TTS (attention resource requirement assigned 20, high priority) broadcasts an announcement about road construction ahead, requesting immediate vehicle control. The user needs to take control of the vehicle to avoid the construction area, controlling the steering wheel and speed to quickly change lanes. The user's attention requirement for this audio information is relatively high. At this moment, a phone call suddenly comes in (attention resource requirement assigned 20, low priority). The phone call requires the user's continuous attention and output, also consuming a significant portion of the user's attention resources. The total attention resource requirement of the two sounds exceeds the user's current total attention resource capacity (current state is high load, current attention resource capacity is 30, the sum of attention resource requirements of alarm TTS and phone call is 40, which is higher than the attention resource capacity of 30, therefore, according to the rules, only the high-priority sound is played). To ensure driving safety, only the highest priority sound, i.e., the alarm TTS, is played at this time.

[0088] During a daily commute, when encountering traffic jams and driving at low speeds, the navigation system (familiar route, attention resource requirement assigned 10) announces: "Road ahead is congested, please be aware of collision risk." Simultaneously, the user turns on music (attention resource requirement assigned 10). Because the vehicle is moving at low speeds, the user's attention resources are relatively abundant, and the multimedia music consumes a smaller proportion of the user's attention resources. The total attention resource required by the two sounds is lower than the user's total attention resource capacity under low-load conditions (current state is low-load, current attention resource capacity is 40, the sum of the attention resource requirements of the navigation and multimedia is 20, lower than the attention resource capacity of 40; therefore, according to the rules, the two sounds are mixed). Therefore, the two sounds can be mixed for output, but the priority relationship between the two sounds must be considered. In the mixed output, the navigation information is output as the main sound, and the music as the secondary sound. The processing methods for the main and secondary sounds include, but are not limited to, using the following audio processing techniques for sound playback: volume adjustment, sound field avoidance, reverb, etc.

[0089] During a daily commute, when encountering traffic jams and the vehicle is traveling at low speed, the navigation system is broadcasting (familiar route, attention resource requirement assigned 10, high priority) warning of road congestion ahead and potential collision risks. At this moment, a phone call comes in (attention resource requirement assigned 20, low priority). Although the phone call consumes a significant portion of attention resources, the user's attention resources are relatively abundant due to the low vehicle speed. Therefore, both audio can be output simultaneously (currently a low-load state, current attention resource capacity is 40, the sum of the attention resource requirements for navigation and the phone call is 30, lower than the attention resource capacity of 40; therefore, according to the rules, the two audios are mixed). Furthermore, the priority relationship between the two audios must be considered. Therefore, in the mixed playback, the navigation information is output as the main audio and the phone call as the secondary audio. The processing methods for the main and secondary audios include, but are not limited to, the following audio processing techniques: volume adjustment, sound field avoidance, and reverb. Sound field avoidance directs the navigation sound field towards the road ahead, while the phone call sound revolves around the user, reducing interference between the different audios.

[0090] This invention provides an in-vehicle sound control method. This method analyzes driving scenario information to determine the current cognitive load of the driver, thereby mapping the driver's current attentional resource capacity. Furthermore, based on the attentional resource requirements and priorities corresponding to each sound, it can adaptively adjust the sound rules for high-load or low-load scenarios. This allows for the output of the required sound effects while ensuring safe driving, utilizing sound rules adapted to the current driving scenario. Therefore, this invention can dynamically control in-vehicle sounds for different driving scenarios, better meeting the different sound requirements of high-load and low-load driving scenarios.

[0091] Secondly, the vehicle-mounted sound control device proposed in the embodiments of the present invention will be described with reference to the accompanying drawings.

[0092] Figure 2 This is a schematic diagram of the structure of the vehicle-mounted sound control device according to an embodiment of the present invention, as shown below. Figure 2 As shown, this embodiment also proposes an in-vehicle voice control device, including: The voice recognition module 100 is used to recognize at least two voice requests; The cognitive load module 200 is used to acquire current driving scenario information and determine the driver's cognitive load level based on the current driving scenario information. The attention resource requirement module 300 is used to determine the attention resource requirement for each sound and assign values ​​to each sound according to the determined attention resource requirement. The vocalization rule module 400 is used to determine the priority of each sound and to determine the corresponding vocalization rule based on the level of cognitive workload, the value assigned to each sound, and the priority of each sound. The sound control module 500 is used to control the sound output of the audio unit according to the sound output rules and the priority of each sound.

[0093] Optionally, the cognitive load module 200 includes: a scenario information acquisition module, a load scenario judgment module, a high load scenario judgment module, and a low load scenario judgment module.

[0094] The scene information acquisition module is used to acquire information such as driver proficiency, road conditions, traffic complexity, vehicle speed, and road visibility.

[0095] The load scenario judgment module is used to determine whether the current driver meets the conditions of the first load scenario or the second load scenario based on the current driving scenario information. The high-load scenario determination module is used to determine the current driving scenario as a high-load scenario when the conditions of the first load scenario are met, and to assign a first load value to the high-load scenario; The low-load scenario determination module is used to determine the current driving scenario as a low-load scenario when the conditions of the second load scenario are met, and to assign a second load value to the low-load scenario.

[0096] Optionally, the conditions for the first load scenario are: satisfying one or a combination of the following: low driver proficiency, poor road conditions, high traffic complexity, high vehicle speed, and low visibility of road conditions.

[0097] Optionally, the conditions for the second load scenario are: satisfying one or a combination of the following: high driver proficiency, good road conditions, low traffic complexity, low vehicle speed, and high visibility of the road conditions.

[0098] Optionally, the attention resource requirement module 300 includes: an assignment table acquisition module, a lookup module, and an assignment module.

[0099] The assignment table acquisition module is used to obtain the audio source attention resource requirement assignment table. The lookup module is used to determine the sound source of each sound, and to look up the corresponding sound source assignment from the sound source attention resource requirement assignment table based on the determined sound source. The assignment module is used to assign values ​​to the corresponding sounds based on the searched sound source.

[0100] Optionally, the sound source attention resource requirement assignment table is a resource assignment table for the driver's required attention, which is obtained by comprehensively judging the driver's cognitive difficulty of sound information and the duration of sound.

[0101] Optionally, the vocalization rule module 400 includes: an urgency determination module, a priority determination module, a value calculation module, and a vocalization rule determination module.

[0102] The urgency determination module is used to determine the urgency level of each sound in the current driving scenario based on the current driving scenario and the source of each sound. The priority determination module is used to determine the priority of corresponding sounds based on the determined urgency level; The assignment calculation module is used to calculate the sum of the assignments of all sounds in the current driving scenario and determine the driver's workload in the current driving scenario based on the level of cognitive workload. The sound production rule determination module is used to determine the corresponding sound production rule based on the sum of the assigned values, the load value, and the priority of each sound.

[0103] Optionally, the vocalization rules include: In high-load scenarios, if the total demand for attention resources from high-priority and low-priority sounds is less than or equal to the first load value, the high-priority and low-priority sounds are mixed and played. In high-load scenarios, if the sum of attention resource requirements for high-priority sounds and low-priority sounds exceeds the first load value, then the high-priority sound will be played. In low-load scenarios, if the sum of attention resource requirements of high-priority sound and low-priority sound is less than or equal to the second load value, control the mixing and playback of high-priority sound and low-priority sound. In low-load scenarios, if the combined demand for attentional resources from high-priority and low-priority sounds exceeds the second load value, then the high-priority sound will be played.

[0104] Optionally, the sound control module 500 includes a high-priority sound control module and a mixing control module.

[0105] Among them, the high-priority sound control module is used to control the audio unit to play high-priority sounds according to the sound production rules and the priority of each sound; The mixing control module is used to control the audio unit to play the mix according to the sound production rules, and to determine the main and auxiliary tone types of each sound during the mix playback according to the priority of each sound, and to perform volume adjustment and / or sound field avoidance and / or reverberation processing based on the main and auxiliary tone types of each sound.

[0106] It should be noted that the foregoing explanation of the vehicle voice control method embodiment also applies to the vehicle voice control device of this embodiment, and will not be repeated here.

[0107] This invention provides an in-vehicle sound control device. This device analyzes driving scenario information to determine the current cognitive load of the driver, thereby mapping the driver's current attentional resource capacity. Furthermore, based on the attentional resource requirements and priorities corresponding to each sound, it can adaptively adjust the sound output rules for high-load or low-load scenarios. Thus, while ensuring safe driving, it outputs the required sound effects using sound output rules adapted to the current driving scenario. Therefore, this invention can dynamically control in-vehicle sounds for different driving scenarios, thereby better meeting the different sound requirements of high-load and low-load driving scenarios.

[0108] Figure 3 This is a schematic diagram of a vehicle provided in an embodiment of the present invention. The vehicle may include: The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0109] When the processor 502 executes the program, it implements the vehicle voice control method provided in the above embodiments.

[0110] Furthermore, the vehicle also includes: Communication interface 503 is used for communication between memory 501 and processor 502.

[0111] The memory 501 is used to store computer programs that can run on the processor 502.

[0112] Memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0113] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0114] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0115] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0116] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described vehicle voice control method.

[0117] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0118] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0119] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0120] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0121] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0122] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0123] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

Claims

1. A vehicle-mounted sound control method, characterized in that, Includes the following steps: At least two voice requests were identified; Obtain current driving scenario information and determine the driver's cognitive workload based on the current driving scenario information; Determine the attention resource requirements for each sound, and assign values ​​to each sound according to the determined attention resource requirements; Determine the priority of each sound, and determine the corresponding vocalization rules based on the cognitive load level, the value assigned to each sound, and the priority of each sound; The sound generation rules include: in a high-load scenario, if the sum of attention resource requirements of high-priority sounds and low-priority sounds is less than or equal to a first load value, controlling the high-priority sounds and low-priority sounds to be mixed and played; in the same high-load scenario, if the sum of attention resource requirements of high-priority sounds and low-priority sounds is greater than the first load value, then playing the high-priority sounds; in a low-load scenario, if the sum of attention resource requirements of high-priority sounds and low-priority sounds is less than or equal to a second load value, controlling the high-priority sounds and low-priority sounds to be mixed and played; and in the same low-load scenario, if the sum of attention resource requirements of high-priority sounds and low-priority sounds is greater than the second load value, then playing the high-priority sounds. Controlling the audio unit to emit sound according to the sound emission rules and the priority of each sound includes: controlling the audio unit to play high-priority sounds according to the sound emission rules and the priority of each sound; or controlling the audio unit to play a mix according to the sound emission rules, and determining the primary and secondary tone types of each sound during the mix playback according to the priority of each sound, and performing volume adjustment and / or sound field avoidance and / or reverberation processing based on the primary and secondary tone types of each sound.

2. The method according to claim 1, characterized in that, The acquisition of current driving scenario information includes: It acquires information on driver proficiency, road conditions, traffic complexity, vehicle speed, and visibility.

3. The method according to claim 1, characterized in that, Determining the driver's cognitive workload based on current driving scenario information includes: Based on the current driving scenario information, determine whether the current driver meets the conditions of the first load scenario or the second load scenario; If the conditions of the first load scenario are met, the current driving scenario is determined to be the high load scenario, and the first load value is assigned to the high load scenario; If the conditions of the second load scenario are met, the current driving scenario is determined to be the low load scenario, and the second load value is assigned to the low load scenario.

4. The method according to claim 1, characterized in that, The process of determining the attention resource requirements for each sound and assigning values ​​to each sound based on the determined attention resource requirements includes: Obtain the audio source attention resource requirement assignment table; The sound source of each sound is determined, and the corresponding sound source assignment is found from the sound source attention resource requirement assignment table according to the determined sound source; Assign values ​​to the corresponding sounds based on the found sound sources.

5. The method according to claim 4, characterized in that, The sound source attention resource requirement assignment table is a resource assignment table for the driver's required attention, which is obtained by comprehensively judging the driver's cognitive difficulty of sound information and the duration of sound.

6. The method according to claim 1, characterized in that, Determining the priority of each sound includes: Based on the current driving scenario and the source of each sound, determine the urgency level of each sound in the current driving scenario; The priority of each sound is determined based on its level of urgency.

7. The method according to claim 1, characterized in that, The step of determining the corresponding vocalization rules based on the cognitive workload, the assigned value of each sound, and the priority of each sound includes: Calculate the sum of the values ​​assigned to all sounds in the current driving scenario, and determine the driver's workload value in the current driving scenario based on the aforementioned cognitive workload level; The corresponding vocalization rules are determined based on the sum of the assigned values, the load value, and the priority of each sound.

8. The method according to claim 3, characterized in that, The conditions for the first load scenario are: It meets one or a combination of the following conditions: low driver proficiency, poor road conditions, high traffic complexity, high vehicle speed, and low visibility.

9. The method according to claim 3, characterized in that, The conditions for the second load scenario are: It meets one or a combination of the following conditions: high driver proficiency, good road conditions, low traffic complexity, low vehicle speed, and high visibility.

10. A vehicle-mounted voice control device for implementing the vehicle-mounted voice control method as described in any one of claims 1-9, characterized in that, include: The voice recognition module is used to identify at least two voice requests; The cognitive load module is used to acquire current driving scenario information and determine the driver's cognitive load level based on the current driving scenario information. The attention resource requirement module is used to determine the attention resource requirement for each sound and assign values ​​to each sound according to the determined attention resource requirement. The vocalization rule module is used to determine the priority of each sound and to determine the corresponding vocalization rule based on the cognitive load level, the value assigned to each sound, and the priority of each sound. The sound control module is used to control the audio unit to emit sound according to the sound emission rules and the priority of each sound.

11. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the in-vehicle voice control method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the in-vehicle voice control method as described in any one of claims 1-9.

Images