Voice endpoint detection control method and device and vehicle

Abstract

The invention provides a voice endpoint detection control method and device and a vehicle, the method is applied to the vehicle, and the method comprises the steps: carrying out the sound source positioning according to voice information under the condition that the voice information output by a user is recognized, and obtaining the target position of the user in the vehicle; determining a target display area corresponding to the target position in the vehicle; within a preset voice endpoint detection duration, if a voice ending endpoint of the voice information is not detected, controlling the target display area to display according to a first display mode; wherein the first display mode is used for reminding the user to end the output of the voice information within the voice endpoint detection duration. According to the method, the user can be effectively reminded to speak out the voice information as soon as possible within the preset voice endpoint detection duration, so that a complete voice instruction is detected as soon as possible, voice control is realized as soon as possible, and the success rate of voice control is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicles, and more specifically, to a control method, device and vehicle for voice endpoint detection in the field of vehicles. Background Art

[0002] In the prior art, when electronic devices perform voice endpoint detection, the duration of voice endpoint detection is usually a default fixed duration. If the user does not speak a complete voice message within the fixed duration, the electronic device will not be able to recognize the complete voice command, resulting in voice control failure. Therefore, how to improve the success rate of voice control has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a control method, device and vehicle for voice endpoint detection, which can effectively remind the user to speak voice information in a timely manner within a preset voice endpoint detection time, so as to detect the complete voice command as soon as possible, so as to realize voice control as soon as possible and improve the success rate of voice control.

[0004] In a first aspect, a control method for voice endpoint detection is provided, the method comprising: when voice information output by a user is identified, locating the sound source according to the voice information to obtain a target position of the user in a vehicle; determining a target display area corresponding to the target position in the vehicle; within a preset voice endpoint detection duration, if a voice end endpoint of the voice information is not detected, controlling the target display area to display in a first display mode; wherein the first display mode is used to remind the user to end the output of the voice information within the voice endpoint detection duration.

[0005] The above technical solution accurately identifies the user's target position in the vehicle through sound source localization technology, determines the target display area corresponding to the target position based on the target position, and controls the target display area to be displayed in the first display mode, so that the user can more intuitively receive the visual feedback information of the target display area, especially in a complex in-vehicle environment, which greatly enhances the convenience and response intuitiveness of voice interaction, thereby improving the overall user experience; in addition, by displaying in the target display area through the first display mode, the user can effectively be reminded to end the input of voice information as soon as possible, so as to detect the complete voice command as soon as possible, and recognize it based on the complete voice command, thereby improving the success rate of voice control, and thus improving the overall voice interaction efficiency and experience effect.

[0006] In combination with the first aspect, in some possible implementations, the vehicle is provided with at least one dimming glass; determining the target display area corresponding to the target position in the vehicle includes: in the at least one dimming glass, determining the area in the dimming glass corresponding to the target position as the target display area.

[0007] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the vehicle is provided with a dimming glass located in the roof area, and the dimming glass is divided into multiple areas; among the at least one dimming glass, the area in the dimming glass corresponding to the target position is determined as the target display area, including: among the multiple areas of the dimming glass, the area corresponding to the target position is determined as the target display area.

[0008] The above technical solution, by dividing a piece of dimming glass into multiple areas, enables users sitting in different positions to view the visual feedback information of the target display area, thereby reminding the users to speak out voice information in time, while reducing the cost of use.

[0009] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, multiple dimming glasses are arranged at different positions on the vehicle; and in the at least one dimming glass, the area in the dimming glass corresponding to the target position is determined as the target display area, including: determining, among the multiple dimming glasses, the dimming glass that is closest to the target position; and determining the area in the dimming glass that is closest to the target position as the target display area.

[0010] The above technical solution determines the dimming glass closest to the target position as the target display area, which is convenient for users sitting at different target positions to view. It can ensure that users receive and view visual feedback information at the most comfortable viewing angle and the closest distance, thereby effectively reminding users at different target positions to complete the input of voice information as soon as possible so that the complete voice command can be detected as soon as possible.

[0011] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the method also includes: when a voice start endpoint is detected from the voice information, controlling the target display area to be displayed in a second display mode; wherein the second display mode is used to remind the user that the current moment is the moment when the voice start endpoint is detected; when a voice end endpoint is detected from the voice information, controlling the target display area to be displayed in a third display mode; wherein the third display mode is used to remind the user that the current moment is the moment when the voice end endpoint is detected.

[0012] The above technical solution can provide instant feedback on the real-time status of voice endpoint detection by controlling the target display area to display in different modes when the voice start endpoint and the voice end endpoint are detected, so that users can intuitively feel the real-time status of voice endpoint detection, thereby enabling users to finish outputting voice information as soon as possible and improving the intelligence level of the vehicle.

[0013] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the target display area is an area in the dimming glass; the first display mode is to display in the target display area with a first transparency; the second display mode is to display in the target display area with a second transparency; the third display mode is to display in the target display area with a third transparency; the first transparency, the second transparency and the third transparency are different from each other.

[0014] According to the above technical solution, the dimming glass realizes display of different transparencies, that is, it displays with the second transparency in the target display area at the speech starting endpoint, displays with the third transparency in the target display area at the speech ending endpoint, and displays with the first transparency in the target display area between the speech starting endpoint and the speech ending endpoint. It can visually create rich layering and dynamic effects according to the real-time status of speech endpoint detection, and realize flexible and changeable information display.

[0015] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, multiple display screens are arranged at different positions on the vehicle; determining the target display area corresponding to the target position in the vehicle includes: determining a display screen facing the user among the multiple display screens; and determining the area in the display screen facing the user as the target display area.

[0016] The above technical solution sets the target display area as the area in the display screen facing the user, so that the user can view the visual feedback information more conveniently without turning the head significantly or moving the line of sight to a farther position. The user can view it directly facing the display screen, thereby improving the efficiency and convenience of voice interaction.

[0017] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the first display mode is displayed with dynamically changing transparency, and the controlling of the target display area to be displayed according to the first display mode includes: according to the output duration of the voice information, controlling the target display area to be displayed with the dynamically changing transparency, and the transparency changes with the change of the output duration.

[0018] In the above technical solution, during the process of outputting voice information, the target display area is controlled to be displayed according to dynamically changing transparency, and the dynamically changing transparency is used to indicate the change in the output duration of the voice information, so as to further remind the user to finish outputting the voice information as soon as possible.

[0019] In a second aspect, a control device for voice endpoint detection is provided, the device comprising:

[0020] An acquisition module is used to locate the sound source according to the voice information output by the user to obtain the target position of the user in the vehicle when the voice information output by the user is recognized; a determination module is used to determine the target display area corresponding to the target position in the vehicle; a control module is used to control the target display area to display in a first display mode if the voice end endpoint of the voice information is not detected within a preset voice endpoint detection time; wherein the first display mode is used to remind the user to end the output of the voice information within the voice endpoint detection time.

[0021] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the determination module is specifically used for: at least one dimming glass is provided on the vehicle; the target display area corresponding to the target position is determined in the vehicle, including: in the at least one dimming glass, the area in the dimming glass corresponding to the target position is determined as the target display area.

[0022] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the determination module is specifically used to: the vehicle is provided with a dimming glass located in the roof area, and the dimming glass is divided into multiple areas; in the at least one dimming glass, the area in the dimming glass corresponding to the target position is determined as the target display area, including: among the multiple areas of the dimming glass, the area corresponding to the target position is determined as the target display area.

[0023] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the determination module is specifically used to: provide multiple dimming glasses at different positions on the vehicle; determine the area in the dimming glass corresponding to the target position in the at least one dimming glass as the target display area, including: determining the dimming glass closest to the target position among the multiple dimming glasses; and determining the area in the dimming glass closest to the target position as the target display area.

[0024] In combination with the second aspect and the above-mentioned implementation methods, in certain implementation methods of the second aspect, the control module is specifically used to: when a voice start endpoint is detected from the voice information, control the target display area to be displayed in a second display mode; wherein the second display mode is used to remind the user that the current moment is the moment when the voice start endpoint is detected; when a voice end endpoint is detected from the voice information, control the target display area to be displayed in a third display mode; wherein the third display mode is used to remind the user that the current moment is the moment when the voice end endpoint is detected.

[0025] In combination with the second aspect and the above-mentioned implementation manner, in some implementation manners of the second aspect, the device also includes: a display module for: the target display area is an area in the dimming glass; the first display mode is to display in the target display area with a first transparency; the second display mode is to display in the target display area with a second transparency; the third display mode is to display in the target display area with a third transparency; the first transparency, the second transparency and the third transparency are different from each other.

[0026] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the determination module is specifically used to: provide multiple display screens at different positions on the vehicle; determine the target display area corresponding to the target position in the vehicle, including: determining a display screen facing the user among the multiple display screens; and determining an area in the display screen facing the user as the target display area.

[0027] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the control module is specifically used to: the first display mode is displayed with dynamically changing transparency, and the control of the target display area to display according to the first display mode includes: according to the output duration of the voice information, controlling the target display area to display according to the dynamically changing transparency, and the transparency changes with the change of the output duration.

[0028] In a third aspect, a vehicle is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the method in the first aspect or any possible implementation of the first aspect.

[0029] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0030] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic flow chart of a control method for voice endpoint detection provided in an embodiment of the present application;

[0032] Figure 2 It is a schematic diagram of modules involved in processing voice information provided by an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of a sound pickup beam provided in an embodiment of the present application;

[0034] Figure 4 This is a schematic diagram of the area division of dimming glass on a six-passenger vehicle provided in an embodiment of the present application;

[0035] Figure 5 This is a schematic diagram of the area division of another dimming glass on a six-passenger vehicle provided in an embodiment of the present application;

[0036] Figure 6 This is a schematic diagram of the area division of dimming glass on a four-passenger vehicle provided in an embodiment of the present application;

[0037] Figure 7 This is a schematic diagram of the area division of the dimming glass on the third six-passenger vehicle provided in an embodiment of the present application;

[0038] Figure 8 This is a schematic diagram of the area division of another dimming glass on a four-passenger vehicle provided in an embodiment of the present application;

[0039] Fig. 9 This is a schematic diagram of the area division of a display screen on a six-passenger vehicle provided in an embodiment of the present application;

[0040] Fig.10 This is a schematic diagram of the area division of a display screen on a four-passenger vehicle provided in an embodiment of the present application;

[0041] Fig.11 is a schematic diagram of the transparency change of a dimming glass provided in an embodiment of the present application;

[0042] Fig.12 is a schematic diagram of another type of transparency change of the dimming glass provided in an embodiment of the present application;

[0043] Fig.13is a schematic diagram of a control method for implementing voice endpoint detection provided by an embodiment of the present application;

[0044] Fig.14 It is a structural schematic diagram of a control device for voice endpoint detection provided in an embodiment of the present application;

[0045] Fig.15 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0047] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0048] To facilitate understanding of the embodiments of the present application, the technical terms involved in the embodiments of the present application are first explained below:

[0049] Voice Activity Detection (VAD) is an important part of the speech recognition process. It is used to locate the speech start and end points from the noisy speech. During the speech recognition process, the electronic device locates the speech start and end points through VAD, and then uses the speech information between the speech start and end points as input for speech recognition processing, and implements the corresponding voice control function based on the speech recognition results.

[0050] In the prior art, when electronic devices perform voice endpoint detection, the time for voice endpoint detection is usually a default fixed duration. If the user does not speak a complete voice message within the fixed duration, the electronic device will not be able to recognize the complete voice command, resulting in voice control failure. Therefore, how to improve the success rate of voice control has become a technical problem that needs to be solved urgently.

[0051] Based on the problems existing in the above-mentioned related technologies, an embodiment of the present application provides a control method for voice endpoint detection, which can effectively remind the user to speak the voice information as soon as possible within the preset voice endpoint detection time, so as to detect the complete voice command as soon as possible, thereby improving the efficiency of detecting complete voice commands, improving the success rate of voice control, and enhancing the convenience of voice interaction.

[0052] Figure 1 It is a schematic flow chart of a control method for voice endpoint detection provided in an embodiment of the present application.

[0053] For example, Figure 1 The method 100 comprises:

[0054] 101. When the voice information output by the user is recognized, the sound source is located according to the voice information to obtain the target position of the user in the vehicle.

[0055] 102 , determining a target display area corresponding to a target position in a vehicle.

[0056] 103. If the voice end endpoint of the voice information is not detected within the preset voice endpoint detection time, the target display area is controlled to be displayed in a first display mode; wherein the first display mode is used to remind the user to end the output of the voice information within the voice endpoint detection time.

[0057] In an embodiment of the present application, the target position of the user in the vehicle is accurately identified by sound source localization technology, and the target display area corresponding to the target position is determined based on the above target position, and the target display area is controlled to be displayed in the first display mode, so that the user can more intuitively receive the visual feedback information of the target display area, especially in a complex in-vehicle environment, which greatly enhances the convenience and response intuitiveness of voice interaction, thereby improving the overall user experience; in addition, by displaying in the target display area in the first display mode, the user can be effectively reminded to end the input of voice information as soon as possible, so as to detect the complete voice command as soon as possible, and recognize it based on the complete voice command, thereby improving the overall voice interaction efficiency and experience effect.

[0058] Below Figure 1 The implementation method of each step in the embodiment shown is specifically described:

[0059] As for 101, it can be understood that the above voice information can be voice information collected when the user starts to speak under the premise that the user wakes up the voice command function. The above voice information is collected by the microphone array.

[0060] For example, Figure 2 As shown, Figure 2This is a schematic diagram of a module involved in processing voice information provided by an embodiment of the present application, and the modules involved include: a microphone array module 201, a DSP (Digital Signal Processing) module 202, a HUT (Head Unit Terminal, vehicle-mounted host) 203 and a voice front-end module 204. Among them, the microphone array module 201 is used as hardware for collecting voice information. After the microphone array module 201 collects all the voice information, the voice information is input into the DSP module 202, and the DSP module 202 performs analog signal conversion to digital signal, signal gain processing, and delay processing on the voice information. The DSP module 202 sends the processed voice information to the voice front-end module 204 through the HUT 203, and completes the operations of echo cancellation and sound source localization in the voice front-end module 204.

[0061] When locating the sound source based on the above voice information, the effective sound direction can be confirmed and the sound source position can be quickly located. When locating the sound source, narrow beam technology can also be used to form a sound pickup beam with a narrow beam. The above narrow beam can form a sound pickup beam at a certain angle in space, retain the voice information in the beam, suppress the voice information outside the beam, shield the noise outside the window, and improve the recognition rate.

[0062] For example, Figure 3 As shown, Figure 3 This is a schematic diagram of a sound pickup beam provided by an embodiment of the present application. Based on the use of narrow beam technology, if the user in the main driving seat is outputting voice information at this time, a sound pickup beam with an angle of θ1 can be formed in space, the voice information within the angle of θ1 is retained, and other voice information outside the angle of θ1 is shielded. If the user in the co-pilot seat is outputting voice information at this time, a sound pickup beam with an angle of θ3 can be formed in space, the voice information within the angle of θ3 is retained, and other voice information outside the angle of θ3 is shielded.

[0063] The above-mentioned sound source localization and narrow beam technology, as an effective means to achieve directional sound pickup and noise suppression, can achieve a noise suppression effect of more than 20 decibels, significantly improve recognition accuracy, effectively avoid interference and false wake-up, and thus ensure that users have a good experience.

[0064] Furthermore, the target position of the user in the vehicle can be determined based on the above-mentioned sound source position. The above-mentioned target position can be understood as: the seat position of the user in the vehicle, which seat position includes: the front driver's seat, the co-driver's seat and any one of the back seats.

[0065] For 102, the target display area may be a display area that the user can visually see at his seat position. For example, the display area closest to his seat position may be used as the target display area so that the user can visually see the target display area at his seat position, thereby capturing visual feedback information of the target display area.

[0066] The target display area is an area on the vehicle with a display function, such as a display area on a display screen on the vehicle, or an area in the dimming glass installed on the vehicle. It is understood that dimming glass is a type of intelligent high-end functional glass, also known as electrically controlled dimming glass. The dimming glass is a new type of special optoelectronic glass product with a sandwich structure in which a liquid crystal film is compounded between two layers of glass and bonded together at high temperature and high pressure to form an integral body. Since dimming glass can present visual effects of different transparencies, the area in the dimming glass can also be used as an area with a display function.

[0067] In a possible implementation, a correspondence relationship may be preset between the seat position and the display area, so that based on the correspondence relationship, when the seat position is known, the display area corresponding to the seat position may be obtained. In other words, based on the correspondence relationship and the target position, the display area corresponding to the target position may be determined as the target display area.

[0068] In some embodiments, at least one dimming glass is provided on the vehicle; the above-mentioned implementation method of determining the target display area corresponding to the target position in the vehicle may include: in at least one dimming glass, determining the area in the dimming glass corresponding to the target position as the target display area.

[0069] The number of dimming glasses installed on the vehicle may be one or more. The locations where dimming glasses can be installed on the vehicle may include: the roof area, the front windshield area, the window area, etc. Therefore, one or more dimming glasses may be installed at the locations where dimming glasses can be installed in the vehicle.

[0070] Specifically, when a vehicle is provided with one dimming glass, the target display area corresponding to the target position may be located in the area of ​​the dimming glass. When a vehicle is provided with multiple dimming glasses, the target display area corresponding to the target position may be: an area in one dimming glass corresponding to the target position among the multiple dimming glasses.

[0071] In some embodiments, the vehicle is provided with a dimming glass located in the roof area, and the dimming glass is divided into multiple areas; the implementation method of determining the area in the dimming glass corresponding to the target position as the target display area in at least one dimming glass may include: among the multiple areas of the dimming glass, determining the area corresponding to the target position as the target display area.

[0072] Taking into account the area on the roof of the vehicle, most of the passengers can see it by looking up. Based on the cost of use, a piece of dimming glass can be installed in the roof area.

[0073] Specifically, the dimming glass located in the roof area can be divided into a plurality of areas based on the number of seats or the number of seat rows in the vehicle, so that each seat position corresponds to an area in the dimming glass.

[0074] For example, taking a six-passenger vehicle as an example, Figure 4 As shown, Figure 4 : is a schematic diagram of the area division of dimming glass on a six-passenger vehicle provided in an embodiment of the present application, Figure 4 The seats in the vehicle include: a front row of seats 401 on the left, a front row of seats 402 on the right, a second row of seats 403 on the left, a second row of seats 404 on the right, a third row of seats 405 on the left, and a third row of seats 406 on the right. The vehicle is provided with a dimming glass 407 located in the roof area.

[0075] In a possible implementation, the switchable glass 407 may be divided into a plurality of areas based on the total number of seat positions in the vehicle, and in the plurality of areas, different seat positions correspond to different areas.

[0076] like Figure 4 As shown in (a) in the figure, the dimming glass 407 is divided into 6 areas according to the total number of seat positions in the vehicle (the total number is 6 at this time), and each seat position corresponds to a different area in the dimming glass 407.

[0077] For example, the front row of seats 401 on the left corresponds to area A in the dimming glass 407, the front row of seats 402 on the right corresponds to area B in the dimming glass 407, the second row of seats 403 on the left corresponds to area C in the dimming glass 407, the second row of seats 404 on the right corresponds to area D in the dimming glass 407, the third row of seats 405 on the left corresponds to area E in the dimming glass 407, and the third row of seats 406 on the right corresponds to area F in the dimming glass 407. In this case, when the target position is the front row of seats 401 on the left, the target display area is area A, when the target position is the front row of seats 402 on the right, the target display area is area B, when the target position is the second row of seats 403 on the left, the target display area is area C, when the target position is the second row of seats 404 on the right, the target display area is area D, when the target position is the third row of seats 405 on the left, the target display area is area E, and when the target position is the third row of seats 406 on the right, the target display area is area F.

[0078] In another possible implementation, the switchable glass 407 may be divided into a plurality of areas based on the number of rows of seat positions in the vehicle, wherein different rows of seat positions correspond to different areas, and the same row of seat positions correspond to the same area.

[0079] like Figure 4 As shown in (b) in the figure, the dimming glass 407 is divided according to the number of rows of seat positions in the vehicle (the number of rows is 3 in this case), and the dimming glass 407 is divided into 3 areas. The seat positions in the same row correspond to the same area in the dimming glass 407, and the seat positions in different rows correspond to different areas in the dimming glass 407.

[0080] For example, the seat positions of the first row (the front row seat 401 on the left and the front row seat 402 on the right) correspond to the area G in the dimming glass 407. The seat positions of the second row (the second row seat 403 on the left and the second row seat 404 on the right) correspond to the area H in the dimming glass 407. The seat positions of the third row (the third row seat 405 on the left and the third row seat 406 on the right) correspond to the area I in the dimming glass 407. In this case, when the target position is the front row seat 401 on the left or the front row seat 402 on the right, the target display area is the area G, when the target position is the second row seat 403 on the left or the second row seat 404 on the right, the target display area is the area H, and when the target position is the third row seat 405 on the left or the third row seat 406 on the right, the target display area is the area I.

[0081] In an optional implementation, the dimming glass 407 may not be divided into regions. Figure 4As shown in (c) in FIG. 8 , the dimming glass 407 is not divided, that is, each seat position in the vehicle corresponds to the entire area of ​​the dimming glass 407 .

[0082] For example, the left front row seat 401, the right front row seat 402, the left second row seat 403, the right second row seat 404, the left third row seat 405, and the right third row seat 406 all correspond to the area J in the dimming glass 407. In this case, when the target position is the left front row seat 401, the right front row seat 402, the left second row seat 403, the right second row seat 404, the left third row seat 405, or the right third row seat 406, the target display area is area J.

[0083] Taking into account that the dimming glass is located in the roof area, it may be inconvenient for users in the front row to view it. Therefore, when users in the front row output voice information, it is difficult for them to conveniently view feedback information through the dimming glass in the roof area. Therefore, the dimming glass in the roof area can also be divided based on the number of rear seats, and the rear seats are the remaining seats except the first row seats.

[0084] For example, Figure 5 As shown, Figure 5 2 is another schematic diagram of the area division of dimming glass on a six-passenger vehicle provided by an embodiment of the present application. The total number of rear seats of the six-passenger vehicle is 4, and the rear seats include two rows of seats 403 on the left, two rows of seats 404 on the right, three rows of seats 405 on the left, and three rows of seats 406 on the right. Figure 5 As shown in (a) in the figure, the dimming glass 407 is divided into 4 areas according to the total number of rear seats in the vehicle (the total number of rear seats is 4 at this time), and each rear seat position corresponds to a different area in the dimming glass 407.

[0085] For example, the left second row of seats 403 corresponds to area A1 in the dimming glass 407, the right second row of seats 404 corresponds to area B1 in the dimming glass 407, the left third row of seats 405 corresponds to area C1 in the dimming glass 407, and the right third row of seats 406 corresponds to area D1 in the dimming glass 407. In this case, when the target position is the left second row of seats 403, the target display area is area A1, when the target position is the right second row of seats 404, the target display area is area B1, when the target position is the left third row of seats 405, the target display area is area C1, and when the target position is the right third row of seats 406, the target display area is area D1.

[0086] In one possible implementation, the dimming glass 407 can be divided into multiple areas based on the number of rows of rear seat positions in the vehicle. The above-mentioned rear seat positions refer to seat positions other than the first row in the vehicle. Among the multiple areas, seat positions in different rows correspond to different areas, and seat positions in the same row correspond to the same area.

[0087] like Figure 5 As shown in (b) in the figure, the dimming glass 407 is divided into two areas according to the number of rows of rear seats in the vehicle (the number of rows is 2 in this case), and the rear seats in the same row correspond to the same area in the dimming glass 407, and the rear seats in different rows correspond to different areas in the dimming glass 407.

[0088] For example, the seat positions of the second row (the second row of seats 403 on the left and the second row of seats 404 on the right) correspond to the area E1 in the dimming glass 407. The seat positions of the third row (the third row of seats 405 on the left and the third row of seats 406 on the right) correspond to the area F1 in the dimming glass 407. In this case, when the target position is the second row of seats 403 on the left or the second row of seats 404 on the right, the target display area is the area E1, and when the target position is the third row of seats 405 on the left or the third row of seats 406 on the right, the target display area is the area F1.

[0089] In a possible implementation, the dimming glass 407 may not be divided into regions. Figure 5 As shown in (c) in FIG. 4 , the dimming glass 407 is not divided, that is, each rear seat in the vehicle corresponds to the entire area of ​​the dimming glass 407. For example, the left second row of seats 403, the right second row of seats 404, the left third row of seats 405, and the right third row of seats 406 correspond to the area G1 in the dimming glass 407. In this case, when the target position is the left second row of seats 403, the right second row of seats 404, the left third row of seats 405, or the right third row of seats 406, the target display area is the area G1.

[0090] For example, taking a four-passenger vehicle as an example, Figure 6 As shown, Figure 6 : is a schematic diagram of the area division of dimming glass on a four-passenger vehicle provided in an embodiment of the present application, Figure 6 The seats in the vehicle include: a front seat 601 on the left, a front seat 602 on the right, a rear seat 603 on the left, and a rear seat 604 on the right. The vehicle is provided with a dimming glass 605 located in the roof area.

[0091] In a possible implementation, the switchable glass 605 may be divided into a plurality of areas based on the number of seat positions in the vehicle, and in the plurality of areas, different seat positions correspond to different areas.

[0092] like Figure 6 As shown in (a), the dimming glass 605 is divided into 4 areas according to the total number of seat positions in the vehicle (the total number is 4 at this time), and each seat position corresponds to a different area in the dimming glass 605.

[0093] For example, the left front seat 601 corresponds to area a in the dimming glass 605, the right front seat 602 corresponds to area b in the dimming glass 605, the left rear seat 603 corresponds to area c in the dimming glass 605, and the right rear seat 604 corresponds to area d in the dimming glass 605. In this case, when the target position is the left front seat 601, the target display area is area a, when the target position is the right front seat 602, the target display area is area b, when the target position is the left rear seat 603, the target display area is area c, and when the target position is the right rear seat 604, the target display area is area d.

[0094] In a possible implementation, the switchable glass 605 can be divided into multiple areas based on the number of rows of seats in the vehicle, with different rows of seats corresponding to different areas and the same row of seats corresponding to the same area.

[0095] like Figure 6 As shown in (b) in FIG. 1 , the dimming glass 605 is divided into two areas according to the number of rows of seats in the vehicle (the number of rows is 2 at this time), the first row of seats (the front row seat 601 on the left and the front row seat 602 on the right) are corresponded to area e in the dimming glass 605, and the rear row of seats (the rear row seat 603 on the left and the rear row seat 604 on the right) are corresponded to area f in the dimming glass 605. In this case, when the target position is the front row seat 601 on the left or the front row seat 602 on the right, the target display area is area e, and when the target position is the rear row seat 603 on the left or the rear row seat 604 on the right, the target display area is area f.

[0096] In a possible implementation, the dimming glass 605 may not be divided into regions. Figure 6As shown in (c) in FIG. 1 , the dimming glass 605 is not divided, that is, each seat position in the vehicle corresponds to the entire area of ​​the dimming glass 605. For example, the left front seat 601, the right front seat 602, the left rear seat 603, and the right rear seat 604 correspond to area g in the dimming glass 605. In this case, when the target position is the left front seat 601, the right front seat 602, the left rear seat 603, or the right rear seat 604, the target display area is area g.

[0097] In some embodiments, multiple dimming glasses are arranged at different positions on the vehicle; in at least one dimming glass, an area in the dimming glass corresponding to the target position is determined as the target display area, including: determining the dimming glass closest to the target position among the multiple dimming glasses; determining the area in the dimming glass closest to the target position as the target display area.

[0098] It can be understood that different positions on the vehicle include: the position of the front windshield area, the position of the window area, and the position of the roof area.

[0099] Specifically, part of the glass on the front windshield is dimming glass, and the other part of the glass is still normal glass, and the above part of the glass refers to the area that does not block the driver's sight. In other words, part of the glass on the front windshield is dimming glass, and the other part of the glass is normal glass, that is, non-dimming glass.

[0100] For example, taking a six-passenger vehicle as an example, Figure 7 As shown, Figure 7 This is a schematic diagram of the area division of the dimming glass on the third six-passenger vehicle provided in the embodiment of the present application. Figure 7 The seats in the vehicle include: a front row seat 401 on the left, a front row seat 402 on the right, a second row seat 403 on the left, a second row seat 404 on the right, a third row seat 405 on the left, and a third row seat 406 on the right. The vehicle is provided with 6 dimming glasses located in the window area and 1 dimming glass 707 located in the front windshield area. The above 6 dimming glasses located in the window area include: a dimming glass 701 on the left side of the front row seat 401 on the left, a dimming glass 702 on the right side of the front row seat 402 on the right, a dimming glass 703 on the left side of the second row seat 403 on the left, a dimming glass 704 on the right side of the second row seat 404 on the right, a dimming glass 705 on the left side of the third row seat 405 on the left, and a dimming glass 706 on the right side of the third row seat 406 on the right.

[0101] In a possible implementation, if there is a dimming glass located in a window area in the vehicle, the dimming glass in the window area is determined as the dimming glass that is closest to the target position in the vehicle.

[0102] A possible implementation is Figure 7 As shown in (a) in FIG. 1 , if the vehicle is provided with 6 dimming glasses located in the window area, the area in the dimming glass closest to the target position among the 6 dimming glasses located in the window area is determined as the target display area. When the target position is the front seat 401 on the left, the dimming glass closest to the target position is the dimming glass 701, and the target display area is the area in the dimming glass 701. When the target position is the front seat 402 on the right, the dimming glass closest to the target position is the dimming glass 702, and the target display area is the area in the dimming glass 702. When the target position is the second row of seats 403 on the left, the dimming glass closest to the target position is the dimming glass 703, and the target display area is the area in the dimming glass 703. When the target position is the second row of seats 404 on the right, the dimming glass closest to the target position is the dimming glass 704, and the target display area is the area in the dimming glass 704. When the target position is the three rows of seats 405 on the left, the dimming glass closest to the target position is the dimming glass 705, and the target display area is the area in the dimming glass 705. When the target position is the three rows of seats 406 on the right, the dimming glass closest to the target position is the dimming glass 706, and the target display area is the area in the dimming glass 706.

[0103] In a possible implementation, on the premise that part of the glass on the front windshield is dimming glass, part of the glass on the front windshield is determined as the dimming glass that is closest to the front seats.

[0104] like Figure 7 As shown in (a), if the vehicle is provided with 6 dimming glasses located in the window area, 1 piece of partial glass 707 on the front windshield (i.e., dimming glass 707), and 1 piece of dimming glass 407 located in the roof area, then the area of ​​the dimming glass closest to the target position among the above 8 pieces of dimming glasses will be determined as the target display area.

[0105] Exemplarily, when the target position is the front row seat 401 on the left or the front row seat 402 on the right, the dimming glass closest to the target position is the dimming glass 707, and the target display area is the area in the dimming glass 707. When the target position is the second row seat 403 on the left, the dimming glass closest to the target position is the dimming glass 703, and the target display area is the area in the dimming glass 703. When the target position is the second row seat 404 on the right, the dimming glass closest to the target position is the dimming glass 704, and the target display area is the area in the dimming glass 704. When the target position is the third row seat 405 on the left, the dimming glass closest to the target position is the dimming glass 705, and the target display area is the area in the dimming glass 705. When the target position is the third row seat 406 on the right, the dimming glass closest to the target position is the dimming glass 706, and the target display area is the area in the dimming glass 706.

[0106] The dimming glass 707 can be located in an area on the front windshield, so that users sitting in the front seats (the left front seat 401 and the right front seat 402) can view this area together to obtain visual feedback information. In addition, there can be two areas of dimming glass on the front windshield, the two areas are respectively facing the two front seats, and users sitting in the front seats (the left front seat 401 and the right front seat 402) can view one of the areas closest to the seat to obtain visual feedback information.

[0107] One possible implementation is that if some six-passenger vehicles have three rows of seats without window glass, the dimming glass in the roof area can be used as the dimming glass closest to the three rows of seats.

[0108] like Figure 7 As shown in (b), the vehicle is provided with four dimming glasses located in the window area, a dimming glass 707 and a dimming glass 407 located in the roof area. The four dimming glasses located in the window area include: a dimming glass 701 on the left side of the front seat 401 on the left, a dimming glass 702 on the right side of the front seat 402 on the right, a dimming glass 703 on the left side of the second row of seats 403 on the left, and a dimming glass 704 on the right side of the second row of seats 404 on the right. Among the six dimming glasses in total, the area in the dimming glass closest to the target position is determined as the target display area.

[0109] Exemplarily, when the target position is the front row seat 401 on the left or the front row seat 402 on the right, the dimming glass closest to the target position is the dimming glass 707, and the target display area is the area in the dimming glass 707. When the target position is the second row seat 403 on the left, the dimming glass closest to the target position is the dimming glass 703, and the target display area is the area in the dimming glass 703. When the target position is the second row seat 404 on the right, the dimming glass closest to the target position is the dimming glass 704, and the target display area is the area in the dimming glass 704. When the target position is the third row seat 405 on the left or the third row seat 406 on the right, the dimming glass closest to the target position is the dimming glass 407, and the target display area is the area in the dimming glass 407.

[0110] At this time, the target display area of ​​the three rows of seats 405 on the left or the three rows of seats 406 on the right is the dimming glass 407. The dimming glass 407 can be divided into two areas according to the total number of three rows of seats (the total number is 2 at this time). The target display area of ​​the three rows of seats 405 on the left or the three rows of seats 406 on the right can be one of the areas respectively, or they can be not divided. The target display area of ​​the three rows of seats 405 on the left or the three rows of seats 406 on the right is the entire area of ​​the dimming glass 407.

[0111] For example, taking a four-passenger vehicle as an example, Figure 8 As shown, Figure 8 1 is a schematic diagram of the area division of another dimming glass on a four-passenger vehicle provided in an embodiment of the present application. Figure 8 The seats in the vehicle include: a front seat 601 on the left, a front seat 602 on the right, a rear seat 603 on the left and a rear seat 604 on the right. The vehicle is provided with 4 dimming glasses located in the window area and 1 partial glass 805 (i.e., dimming glass 805) located on the front windshield. The above 4 dimming glasses located in the window area include: a dimming glass 801 on the left side of the front seat 601 on the left, a dimming glass 802 on the right side of the front seat 602 on the right, a dimming glass 803 on the left side of the second row of seats 603 on the left and a dimming glass 804 on the right side of the second row of seats 604 on the right.

[0112] In a possible implementation, if there is a dimming glass located in a window area in the vehicle, the dimming glass in the window area is determined as the dimming glass closest to the target position in the vehicle.

[0113] like Figure 8 As shown, if the vehicle is provided with four dimming glasses located in the window area, the area in the dimming glass in the window area which is closest to the target position among the four dimming glasses located in the window area is determined as the target display area.

[0114] Exemplarily, when the target position is the front row seat 601 on the left, the dimming glass closest to the target position is the dimming glass 801, and the target display area is the area in the dimming glass 801. When the target position is the front row seat 602 on the right, the dimming glass closest to the target position is the dimming glass 802, and the target display area is the area in the dimming glass 802. When the target position is the second row seat 603 on the left, the dimming glass closest to the target position is the dimming glass 803, and the target display area is the area in the dimming glass 803. When the target position is the second row seat 604 on the right, the dimming glass closest to the target position is the dimming glass 804, and the target display area is the area in the dimming glass 804.

[0115] In a possible implementation, on the premise that part of the glass on the front windshield is dimming glass, part of the glass on the front windshield is determined as the dimming glass closest to the front seats.

[0116] like Figure 8 As shown, when the target position is the left front seat 601 or the right front seat 602, the dimming glass closest to the target position is the dimming glass 805, and the target display area is the area in the dimming glass 805. When the target position is the left second row seat 603, the dimming glass closest to the target position is the dimming glass 803, and the target display area is the area in the dimming glass 803. When the target position is the right second row seat 604, the dimming glass closest to the target position is the dimming glass 804, and the target display area is the area in the dimming glass 804.

[0117] The dimming glass 805 can be located in an area on the front windshield, so that users sitting in the front seats (the left front seat 601 and the right front seat 602) can view this area together to obtain visual feedback information. In addition, there can be two areas of dimming glass on the front windshield, the two areas are respectively facing the two front seats, and users sitting in the front seats (the left front seat 601 and the right front seat 602) can view one of the areas closest to the seat to obtain visual feedback information.

[0118] In some embodiments, a plurality of display screens are provided at different positions on a vehicle; a target display area corresponding to a target position is determined in the vehicle, including: determining a display screen facing a user among the plurality of display screens; and determining an area in the display screen facing the user as the target display area.

[0119] It is understood that the above display screen refers to an electronic screen installed inside the car for displaying text, images or video content; the area in the above display screen may be an area that does not affect the normal display content of the display screen. The multiple display screens installed on the vehicle may include: a central control screen, a seat back screen, and a ceiling screen.

[0120] Among the multiple display screens, if there is one display screen facing the user, the area in the display screen facing the user is determined as the target display area; if there are multiple display screens facing the user, the area in the display screen facing the user and closest to the target position is determined as the target display area.

[0121] In the above-mentioned vehicle, if the target position is the front row of seats, that is, the first row of seats in the vehicle, the display screen facing the user is the central control screen, and therefore the area in the central control screen is the target display area; if the target position is the back row of seats, that is, other seats in the vehicle except the first row, the display facing the user includes the central control screen, the seat back screen and the ceiling screen. At this time, it is necessary to determine the display screen closest to the target position, and determine the area in the display screen facing the user and closest to the target position as the target display area.

[0122] One possible implementation method is that when there are seatback screens, ceiling screens and central control screens at the same time, the display screen closest to the front seats in the vehicle and facing the user is the central control screen, and the display screen closest to the rear seats in the vehicle and facing the user is the seatback screen.

[0123] For example, taking a six-passenger vehicle as an example, Fig. 9 As shown, Fig. 9 This is a schematic diagram of the area division of display screens on a six-passenger vehicle provided in an embodiment of the present application. In the figure, the multiple display screens include: 4 seatback screens located behind the seats, 1 ceiling screen 905 located above the vehicle, and 1 central control screen 906 located on the center console of the vehicle. The above 4 seatback screens located behind the seats include: a seatback screen 901 located behind the front seat 401 located on the left, a seatback screen 902 located behind the front seat 402 located on the right, a seatback screen 903 located behind the second row of seats 403 located on the left, and a seatback screen 904 located behind the second row of seats 404 located on the right.

[0124] like Fig. 9As shown, if the vehicle is provided with a seatback screen, a ceiling screen 905 and a central control screen 906 at the same time, when the target position is the front row seat 401 on the left or the front row seat 402 on the right, the display screen facing the user and closest to the target position is the central control screen 906, and the target display area is the area in the central control screen 906. When the target position is the second row seat 403 on the left, the display screen facing the user and closest to the target position is the seatback screen 901, and the target display area is the area in the seatback screen 901. When the target position is the second row seat 404 on the right, the display screen facing the user and closest to the target position is the seatback screen 902, and the target display area is the area in the seatback screen 902. When the target position is the third row seat 405 on the left, the display screen facing the user and closest to the target position is the seatback screen 903, and the target display area is the area in the seatback screen 903. When the target position is the three rows of seats 406 on the right, the display screen facing the user and closest to the target position is the seatback screen 904 , and the target display area is the area in the seatback screen 904 .

[0125] One possible implementation method is that when there are both a ceiling screen and a central control screen, the display screen that is closest to the front seats in the vehicle and faces the user is the central control screen, and the display screen that is closest to the rear seats in the vehicle and faces the user is the ceiling screen. In this case, the central control screen or the ceiling screen is determined as the display screen facing the user and closest to the target position.

[0126] like Fig. 9 As shown, assuming that the vehicle is provided with both a ceiling screen 905 and a central control screen 906, when the target position is the left front seat 401 or the right front seat 402, the display screen facing the user and closest to the target position is the central control screen 906, and the target display area is the area in the central control screen 906. When the target position is the left second row of seats 403, the right second row of seats 404, the left third row of seats 405, or the right third row of seats 406, the display screen facing the user and closest to the target position is the ceiling screen 905, and the target display area is the area in the ceiling screen 905.

[0127] One possible implementation method is that when there are seatback screens, ceiling screens and central control screens at the same time, the display screen that is closest to the front seats in the vehicle and faces the user is the central control screen, and the display screen that is closest to the rear seats in the vehicle and faces the user is the seatback screen. In this case, the central control screen or seatback screen is determined as the display screen that is closest to the target position and faces the user.

[0128] For example, taking a four-passenger vehicle as an example, Fig.10 As shown, Fig.10 : is a schematic diagram of the area division of a display screen on a four-passenger vehicle provided in an embodiment of the present application, Fig.10The seats in the vehicle include: a front seat 601 on the left, a front seat 602 on the right, a rear seat 603 on the left and a rear seat 604 on the right. The vehicle is provided with two seatback screens behind the seats, a ceiling screen 1003 above the vehicle and a central control screen 1004 on the center console of the vehicle. The above two seatback screens behind the seats include: a seatback screen 1001 behind the front seat 601 on the left and a seatback screen 1002 behind the front seat 602 on the right.

[0129] If the vehicle is provided with a seatback screen, a ceiling screen 1003, and a central control screen 1004 at the same time, when the target position is the left front seat 601 or the right front seat 602, the display screen facing the user and closest to the target position is the central control screen 1004, and the target display area is the area in the central control screen 1004. When the target position is the left second row seat 603, the display screen facing the user and closest to the target position is the seatback screen 1001, and the target display area is the area in the seatback screen 1001. When the target position is the right second row seat 604, the display screen facing the user and closest to the target position is the seatback screen 1002, and the target display area is the area in the seatback screen 1002.

[0130] One possible implementation method is that when there are both a ceiling screen and a central control screen, the display screen that is closest to the front seats in the vehicle and faces the user is the central control screen, and the display screen that is closest to the rear seats in the vehicle and faces the user is the ceiling screen. In this case, the central control screen or the ceiling screen is determined as the display screen facing the user and closest to the target position.

[0131] like Fig.10 As shown, assuming that the vehicle is provided with both a ceiling screen 1003 and a central control screen 1004, when the target position is the left front seat 601 or the right front seat 602, the display screen facing the user and closest to the target position is the central control screen 1004, and the target display area is the area in the central control screen 1004. When the target position is the left second row of seats 603 or the right second row of seats 604, the display screen facing the user and closest to the target position is the ceiling screen 1003, and the target display area is the area in the ceiling screen 1003.

[0132] For 103, it can be understood that the above-mentioned preset voice endpoint detection duration can be a pre-set fixed duration, which is the maximum duration of voice endpoint detection. The user needs to output voice information within the fixed duration in order to recognize a complete voice command as soon as possible. The above-mentioned first display mode is used to remind and inform the user to end the output of voice information within the preset voice endpoint detection duration. The first display mode includes: text display mode, image display mode, animation display mode, color or transparency display mode, etc. The embodiment of the present application does not specifically limit the specific form of the first display mode.

[0133] In some embodiments, when a voice start endpoint is detected from voice information, the target display area is controlled to be displayed in a second display mode; wherein the second display mode is used to remind the user that the current moment is the moment when the voice start endpoint is detected; when a voice end endpoint is detected from voice information, the target display area is controlled to be displayed in a third display mode; wherein the third display mode is used to remind the user that the current moment is the moment when the voice end endpoint is detected.

[0134] It is understandable that the above three display modes are different from each other. The difference may be in the displayed content or in that each display mode is displayed in a different sub-region under the premise that the target display region is divided into a plurality of sub-regions.

[0135] In the case where the target display area is not divided, the three display modes mentioned above may be displayed in the target display area with three different transparencies.

[0136] In the case of dividing the target display area, the three display modes can be displayed in different sub-areas with the same transparency. Specifically, the target display area can be divided into three sub-areas, the first display mode is displayed in sub-area one, the second display mode is displayed in sub-area two, and the third display mode is displayed in sub-area three.

[0137] Optionally, in the case of dividing the target display area, the three display modes may also be displayed in different sub-areas with different transparencies.

[0138] In some embodiments, the target display area is an area in the dimming glass; the first display mode is to display in the target display area with a first transparency; the second display mode is to display in the target display area with a second transparency; the third display mode is to display in the target display area with a third transparency; the first transparency, the second transparency and the third transparency are different from each other.

[0139] It can be understood that the above-mentioned first display mode can be displayed in sub-area 1 of the target display area with a first transparency, the above-mentioned second display mode can be displayed in sub-area 2 of the target display area with a second transparency, and the above-mentioned third display mode can be displayed in sub-area 3 of the target display area with a third transparency.

[0140] For example, Fig.11 As shown, Fig.11 Schematic diagram of the transparency change of a dimming glass provided by an embodiment of the present application, wherein the first transparency is lower than the second transparency, and the third transparency is lower than the first transparency. Fig.11 As shown in (a) of FIG. 1 , the target display area can be divided into three sub-areas in the horizontal direction. Fig.11 As shown in (b) in FIG. 1 , the target display area can also be divided into three sub-areas in the vertical direction.

[0141] For example, Fig.12 As shown, Fig.12 Schematic diagram of another type of transparency change of the dimming glass provided in the embodiment of the present application, assuming that the first transparency is higher than the second transparency, and the third transparency is higher than the first transparency. Fig.12 As shown in (a) of FIG. 1 , the target display area can be divided into three sub-areas in the horizontal direction. Fig.12 As shown in (b) in FIG. 1 , the target display area can also be divided into three sub-areas in the vertical direction.

[0142] exist Fig.11 and Fig.12 In the first display mode, the first transparency is used to display in sub-area 1 of the target display area to prompt the user that the voice endpoint detection is currently in progress. The second display mode is used to display in sub-area 2 of the target display area with a second transparency to prompt the user that the voice endpoint detection has started. The third display mode is used to display in sub-area 3 of the target display area with a third transparency to prompt the user that the voice endpoint detection has ended.

[0143] It should be noted that the embodiments of the present application are only for the convenience of explanation. Fig.11 and 12 The exemplary description is made by taking the sub-region division method in as an example. In a specific implementation, other sub-region division methods can also be used to divide the target display area into three sub-regions according to actual needs. The embodiment of the present application does not make specific limitations on this.

[0144] In some embodiments, the first display mode is to display with dynamically changing transparency, and the target display area is controlled to be displayed according to the first display mode, including: according to the output duration of the voice information, the target display area is controlled to be displayed with dynamically changing transparency, and the transparency changes with the output duration.

[0145] Specifically, the first display mode can be displayed in the target display area with the first transparency, and the first transparency includes a plurality of different transparencies. As the output duration of the voice information changes, the target display area is controlled to dynamically change according to the plurality of different transparencies.

[0146] Exemplarily, when the first transparency is lower than the second transparency and the third transparency is lower than the first transparency, the first transparency includes multiple different transparencies, and the multiple different transparencies decrease in size successively. Therefore, when the user outputs voice information within the preset voice endpoint detection time, as the output time of the voice information increases, the target display area is dynamically displayed with the multiple decreasing transparencies mentioned above.

[0147] In the above process, when VAD endpoint detection starts, the transparency of the target display area is high. During the VAD endpoint detection process, the transparency will gradually change dynamically to indicate the processing progress until the VAD endpoint detection is completed, and the target display area will present a low transparency state.

[0148] Exemplarily, when the first transparency is higher than the second transparency and the third transparency is higher than the first transparency, the first transparency includes multiple different transparencies, and the multiple different transparencies increase in size successively. Therefore, when the user outputs voice information within the preset voice endpoint detection time, as the output time of the voice information increases, the target display area is dynamically displayed with the multiple increasing transparencies mentioned above.

[0149] In the above scheme, when the VAD endpoint detection starts, the transparency of the target display area is low. During the VAD endpoint detection process, the transparency will gradually increase dynamically to indicate the processing progress until the VAD endpoint detection is completed, and the target display area will present a high transparency state.

[0150] In some embodiments, if the target display area is an area in the dimming glass, a specific pattern or color can also be displayed in the area in the dimming glass to remind the user to end the output of voice information as soon as possible within the preset voice endpoint detection time.

[0151] In some embodiments, if the target display area is a display screen, when the user outputs voice information within the preset voice endpoint detection time, the target display area will be displayed in the form of text, images or animation to remind the user of the remaining time for outputting voice information. For example, the remaining time for the user to output voice information can be directly displayed in the target display area.

[0152] Fig.13This is a schematic diagram of a control method for implementing voice endpoint detection provided in an embodiment of the present application.

[0153] For example, Fig.13 As shown, when implementing the control method of the above-mentioned voice endpoint detection, multiple modules are involved, including: microphone array module 1301, HUT1305, display unit control module 1306 and display module 1307. Among them, HUT1305 includes: voice front-end module 1302, voice recognition module 1303 and voice main program 1304.

[0154] Specifically, the microphone array module 1301 is used to collect the original voice information of the user and transmit the original voice information of the user to the Figure 2 After being processed by the modules involved, it is input into the voice front-end module 1302.

[0155] The above-mentioned voice front-end module 1302 is used to Figure 2 The voice information processed by the modules involved in the above is subjected to front-end noise reduction, and then the voice information after noise reduction is localized. The above-mentioned narrow beam technology must be used in conjunction with sound source localization, which can not only completely retain the voice information in the beam direction, but also attenuate the sound outside the beam direction, and simultaneously perform directional sound pickup in the direction of the user. In summary, in a complex in-vehicle use environment, the original voice information of the user is converted into clean voice information through the voice front-end module 1302, and the specific location of the user is obtained through sound source localization technology.

[0156] The above-mentioned speech recognition module 1303 is used to convert the speech information in the above-mentioned beam direction into text, and at the same time perform semantic understanding, and send the semantics to the speech main program.

[0157] The above-mentioned voice main program 1304 is used to complete data synchronization with the voice recognition module 1302. The voice recognition module 1302 needs to synchronize the target data to the voice main program 1304. The target data includes the location of the sound source positioning, the preset voice endpoint detection duration, and the voice start endpoint and voice end endpoint of the voice information.

[0158] The HUT 1305 serves as an infotainment system in the vehicle system. It not only integrates the voice recognition module 1303 and the voice main program 1304 , but also can synchronize the above data to the dimming glass control module 1306 .

[0159] The display unit control module 1306 is used to control the target display area of ​​the display module 1307 to display in the first display mode / the second display mode / the third display mode.

[0160] The display module 1307 may be a dimming glass or a display screen, and correspondingly, the display unit control module 1306 may be a dimming glass control module or a display screen control module.

[0161] The above-mentioned dimming glass control module serves as the main control unit of the dimming glass and can control various setting items of the dimming glass, such as color and transparency, etc. The dimming glass control module transmits data with HUT1305. When HUT1305 obtains the above-mentioned data, it transmits the data to the dimming glass control module, and the dimming glass control module drives the dimming glass to perform the final display.

[0162] The above-mentioned display screen control module serves as the main control unit of the display screen and can control various setting items of the display screen. The display screen control module transmits data with HUT1305. When HUT1305 obtains data, it transmits the data to the display screen control module, and the display screen control module drives the display screen for final display.

[0163] Fig.14 It is a structural schematic diagram of a control device for voice endpoint detection provided in an embodiment of the present application.

[0164] For example, Fig.14 As shown, the device 1400 includes:

[0165] The acquisition module 1401 is used to locate the sound source according to the voice information when the voice information output by the user is recognized, so as to obtain the target position of the user in the vehicle.

[0166] The determination module 1402 is used to determine a target display area corresponding to the target position in the vehicle.

[0167] Control module 1403 is used to control the target display area to display in a first display mode if the voice end endpoint of the voice information is not detected within the preset voice endpoint detection time; wherein the first display mode is used to remind the user to end the output of the voice information within the voice endpoint detection time.

[0168] In one possible implementation, the determination module is specifically used to: provide at least one dimming glass on the vehicle; determine a target display area corresponding to a target position in the vehicle, including: in at least one dimming glass, determining an area in the dimming glass corresponding to the target position as the target display area.

[0169] In one possible implementation, the determination module is specifically used for: a vehicle is provided with a dimming glass located in the roof area, and the dimming glass is divided into multiple areas; in at least one dimming glass, the area in the dimming glass corresponding to the target position is determined as the target display area, including: among the multiple areas of the dimming glass, the area corresponding to the target position is determined as the target display area.

[0170] In one possible implementation, the determination module is specifically used to: provide multiple dimming glasses at different positions on the vehicle; in at least one dimming glass, determine the area in the dimming glass corresponding to the target position as the target display area, including: determining the dimming glass closest to the target position among the multiple dimming glasses; and determining the area in the dimming glass closest to the target position as the target display area.

[0171] In one possible implementation, the control module includes a first control unit and a second control unit. The first control unit is specifically used to: when a voice start endpoint is detected from the voice information, control the target display area to be displayed in a second display mode; wherein the second display mode is used to remind the user that the current moment is the moment when the voice start endpoint is detected; the second control unit is specifically used to: when a voice end endpoint is detected from the voice information, control the target display area to be displayed in a third display mode; wherein the third display mode is used to remind the user that the current moment is the moment when the voice end endpoint is detected.

[0172] In a possible implementation, the device also includes: a display module specifically used for: the target display area is an area in the dimming glass; the first display mode is to display in the target display area with a first transparency; the second display mode is to display in the target display area with a second transparency; the third display mode is to display in the target display area with a third transparency; the first transparency, the second transparency and the third transparency are different from each other.

[0173] In one possible implementation, the determination module is specifically used to: provide multiple display screens at different positions on the vehicle; determine a target display area corresponding to the target position in the vehicle, including: determining a display screen facing the user among the multiple display screens; and determining an area in the display screen facing the user as the target display area.

[0174] In one possible implementation, the control module is specifically used to: the first display mode is to display with dynamically changing transparency, and control the target display area to display according to the first display mode, including: according to the output duration of the voice information, controlling the target display area to display with dynamically changing transparency, and the transparency changes with the change of the output duration.

[0175] Fig.15 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.

[0176] For example, Fig.15 As shown, the vehicle 1500 includes: a memory 1501 and a processor 1502, wherein the memory 1501 stores an executable program code 15011, and the processor 1502 is used to call and execute the executable program code 15011 to perform a control method for voice endpoint detection.

[0177] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute a control method for voice endpoint detection provided in an embodiment of the present application.

[0178] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0179] In the case of dividing each functional module according to each function, the device may also include an acquisition module, a determination module, a control module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0180] It should be understood that the device provided in this embodiment is used to execute the above-mentioned control method of voice endpoint detection, and thus can achieve the same effect as the above-mentioned implementation method.

[0181] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related program codes, etc.

[0182] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0183] In addition, the device provided in the embodiments of the present application may specifically be a chip, component or module, and the chip may include a connected processor and memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a control method for voice endpoint detection provided in the above embodiments.

[0184] This embodiment also provides a computer-readable storage medium, which stores computer program code. When the computer program code runs on a computer, the computer executes the above-mentioned related method steps to implement a control method for voice endpoint detection provided in the above embodiment.

[0185] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a control method for voice endpoint detection provided by the above embodiment.

[0186] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0187] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0188] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0189] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A control method for voice endpoint detection, characterized in that: The method comprises: When the voice information output by the user is recognized, the sound source is localized according to the voice information to obtain the target position of the user in the vehicle; determining a target display area corresponding to the target position in the vehicle; Within the preset voice endpoint detection time, if the voice end endpoint of the voice information is not detected, the target display area is controlled to display in a first display mode; wherein, the first display mode is used to remind the user to end the output of the voice information within the voice endpoint detection time.

2. The method according to claim 1, characterized in that: The vehicle is provided with at least one dimming glass; The determining of a target display area corresponding to the target position in the vehicle includes: In the at least one piece of dimming glass, an area in the dimming glass corresponding to the target position is determined as the target display area.

3. The method according to claim 2, characterized in that The vehicle is provided with a dimming glass located in the roof area, and the dimming glass is divided into a plurality of areas; The step of determining, in the at least one piece of dimming glass, an area in the dimming glass corresponding to the target position as the target display area comprises: Among the multiple areas of the switchable glass, an area corresponding to the target position is determined as the target display area.

4. The method according to claim 2, characterized in that: The vehicle is provided with a plurality of dimming glasses at different positions; The step of determining, in the at least one piece of dimming glass, an area in the dimming glass corresponding to the target position as the target display area comprises: Determine, among the plurality of dimming glasses, a dimming glass closest to the target position; An area in the dimming glass that is closest to the target position is determined as the target display area.

5. The method according to claim 1, characterized in that The method further comprises: In the case where a speech start endpoint is detected from the speech information, controlling the target display area to be displayed in a second display mode; wherein the second display mode is used to remind the user that the current moment is the moment when the speech start endpoint is detected; In the case where a voice end point is detected from the voice information, the target display area is controlled to be displayed in a third display mode; wherein the third display mode is used to remind the user that the current moment is the moment when the voice end point is detected.

6. The method according to claim 5, characterized in that The target display area is an area in the dimming glass; The first display mode is to display in the target display area with a first transparency; The second display mode is to display in the target display area with a second transparency; The third display mode is to display in the target display area with a third transparency; the first transparency, the second transparency and the third transparency are different from each other.

7. The method according to claim 1, characterized in that The vehicle is provided with a plurality of display screens at different positions; The determining of a target display area corresponding to the target position in the vehicle includes: Determining a display screen facing the user among the multiple display screens; An area in a display screen facing the user is determined as the target display area.

8. The method according to claim 1, characterized in that The first display mode is displaying with dynamically changing transparency, and the controlling the target display area to display according to the first display mode includes: According to the output duration of the voice information, the target display area is controlled to be displayed according to the dynamically changing transparency, and the transparency changes with the change of the output duration.

9. A control device for voice endpoint detection, characterized in that: The device comprises: An acquisition module, configured to, when the voice information output by the user is recognized, perform sound source localization according to the voice information to obtain the target position of the user in the vehicle; a determination module, configured to determine a target display area corresponding to the target position in the vehicle; A control module is used to control the target display area to display in a first display mode if the voice end endpoint of the voice information is not detected within a preset voice endpoint detection time; wherein the first display mode is used to remind the user to end the output of the voice information within the voice endpoint detection time.

10. A vehicle, characterized in that: The vehicle comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the vehicle executes the method according to any one of claims 1 to 8.

Images