Out-of-vehicle speech interaction method, out-of-vehicle speech interaction apparatus, and vehicle

Abstract

An out-of-vehicle speech interaction method, an out-of-vehicle speech interaction apparatus, and a vehicle. The out-of-vehicle speech interaction method comprises: when a vehicle is in a sentry mode and changes from a standby state to a non-standby state, broadcasting a speech prompt according to a preset rule (S10); acquiring a speech input from a user outside the vehicle (S20); and executing a response operation on the basis of the speech input from the user outside the vehicle (S30). The capability of a conventional sentry mode is reused, and the conventional sentry mode is combined with an out-of-vehicle speech interaction capability, thereby realizing an out-of-vehicle speech interaction experience triggered by a state change of the sentry mode.

Description

A method for interacting with external voices, an apparatus for interacting with external voices, and a vehicle Technical Field

[0001] The embodiments of the present invention relate to, but are not limited to, the field of human-computer interaction technology, and more specifically, to an off-vehicle voice interaction method, an off-vehicle voice interaction device, and a vehicle. Background Art

[0002] With the development of intelligent vehicle technology, vehicle human-machine interaction (HMI) functions are becoming more intelligent. Voice interaction, as a common HMI method, has also been widely used in vehicles. Traditional vehicle voice interaction functions primarily focus on in-vehicle HMI scenarios, ignoring out-of-vehicle HMI scenarios, where users interact with the vehicle. This results in insufficient intelligence for HMI functions outside the vehicle, making it difficult to meet HMI needs. Summary of the Invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] An embodiment of the present invention discloses a method for external voice interaction, comprising:

[0005] When the vehicle is in sentry mode and changes from standby to non-standby, a voice reminder is played according to the preset rules;

[0006] Get voice input from users outside the vehicle;

[0007] Execute a response operation according to the voice input of the user outside the vehicle.

[0008] Optionally, performing a response operation according to the voice input of the user outside the vehicle includes:

[0009] Recognizing user intention based on the voice input of the user outside the vehicle;

[0010] When the target operation corresponding to the user intention is a communication operation, obtaining the number of executions of the target operation within a preset period, wherein the communication operation includes at least one of a text message operation, a voice call operation, and a voice message operation, and the preset period includes the current state switching period or the current engine shutdown period, and the current state switching period refers to the period during which the vehicle changes from the standby state to the non-standby state and then returns to the standby state;

[0011] Determining whether the number of executions of the target operation within the preset period is less than a first preset number;

[0012] If so, executing the target operation;

[0013] If not, output a prompt indicating that execution is rejected.

[0014] Optionally, the off-vehicle voice interaction method further includes:

[0015] When the sentinel state changes from the standby state to the non-standby state, obtaining statistical parameters during this state switching period, wherein the state switching period refers to the period during which the vehicle changes from the standby state to the non-standby state and then returns to the standby state;

[0016] When the statistical parameters meet the preset conditions, the off-vehicle voice interaction function during this state switching is turned off, wherein the preset conditions include at least one of the following: the duration of this state switching period is greater than the preset duration, and the number of interactions during this state switching period is greater than a second preset number.

[0017] Optionally, the off-vehicle voice interaction method further includes:

[0018] Get the number of external voice interactions during the current shutdown period;

[0019] When the number of external vehicle voice interactions is greater than a third preset number, the external vehicle voice interaction function during the current engine shutdown period is turned off.

[0020] Optionally, the broadcasting of a voice reminder according to a preset rule further includes:

[0021] A voice reminder is extracted from a preset voice reminder set for broadcast, wherein the non-standby state includes an alert state and an alarm state, the preset voice reminder set is a voice reminder set corresponding to the current state of the vehicle, or the preset voice reminder set is a voice reminder set corresponding to instructions pre-stored by the owner, or the preset voice reminder set is a voice reminder set corresponding to the current scene of the vehicle.

[0022] Optionally, performing a response operation according to the voice input of the user outside the vehicle includes:

[0023] Recognizing user intention based on the voice input of the user outside the vehicle;

[0024] When the target operation corresponding to the user intention is a communication operation, determining the target object according to the voice input of the user outside the vehicle, wherein the communication operation includes at least one of a text message operation, a voice call operation, and a voice message operation, and the target object includes the vehicle owner terminal or the vehicle manufacturer terminal;

[0025] Execute the target operation corresponding to the user intention on the target object.

[0026] Optionally, performing a response operation according to the voice input of the user outside the vehicle includes:

[0027] Recognizing user intention based on the voice input of the user outside the vehicle;

[0028] Determining whether the target operation corresponding to the user's intention is a preset prohibited operation;

[0029] If so, prohibiting execution of the target operation;

[0030] If not, the target operation is performed.

[0031] Optionally, broadcasting a voice reminder according to a preset rule includes:

[0032] Acquire an image of the vehicle's environment;

[0033] triggering the vehicle from the standby state to the non-standby state according to the image recognition, wherein the life form type includes at least two of children, animals, and adults;

[0034] Output a voice reminder corresponding to the life form type.

[0035] An embodiment of the present invention further discloses an out-of-vehicle voice interaction device, comprising a computer-readable storage medium storing a computer program and a processor. When the computer program is read and executed by the processor, the out-of-vehicle voice interaction method described above is implemented.

[0036] An embodiment of the present invention further discloses a vehicle, characterized in that it includes the above-mentioned off-vehicle voice interaction device.

[0037] Compared with the prior art, the vehicle-external voice interaction method, vehicle-external voice interaction device, and vehicle according to the embodiments of the present invention have at least the following beneficial effects:

[0038] When the vehicle is in Sentry Mode and changes from standby to non-standby, a voice reminder is broadcast according to preset rules, voice input from the user outside the vehicle is obtained, and a response operation is executed based on the voice input of the user outside the vehicle. This is to combine the traditional Sentry Mode with the external voice interaction capabilities, reusing the traditional Sentry Mode capabilities. The changes in the vehicle's operating status (reflected in the Sentry State) identified in Sentry Mode are used as the trigger conditions for external voice interaction. By broadcasting a voice reminder, the external user is alerted, and by obtaining the external user's voice input, a response operation is executed based on the external user's voice input, enabling voice interaction between the vehicle and the external user. This realizes the external voice interaction experience triggered by the change in Sentry Mode status, and improves the intelligence of external voice interaction. In addition, reusing the traditional Sentry Mode capabilities eliminates the need to repeatedly develop monitoring and image recognition functions, achieving an improved product experience while maintaining cost savings.

[0039] Still other aspects will become apparent upon reading and understanding the detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a flow chart of a foreign language interaction method according to an embodiment of the present invention;

[0041] FIG2 is a flow chart of another embodiment of the method for external voice interaction according to an embodiment of the present invention;

[0042] FIG3 is a schematic diagram of an embodiment of an implementation architecture of an off-vehicle voice interaction method according to an embodiment of the present invention;

[0043] FIG4 is a flow chart of another embodiment of the method for external voice interaction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned objects, features and advantages of the embodiments of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0045] It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. The terms "including" and their variations used in the present invention are open-ended, i.e., "including but not limited to"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; and the term "optionally" means "optional embodiments." Definitions of other terms are provided in the following description.

[0046] As shown in FIG1 , in one embodiment, the vehicle-external voice interaction method according to an embodiment of the present invention includes:

[0047] Step S10: When the vehicle is in sentry mode and changes from standby state to non-standby state, a voice reminder is broadcast according to a preset rule.

[0048] Sentry mode is usually activated after the vehicle is parked and the engine is turned off. In some embodiments, when there is no one in the driver's seat of the vehicle, the vehicle is in parking gear, the vehicle is in the off state, and the central control lock is locked, the vehicle enters sentry mode. A vehicle in sentry mode will use functions such as video monitoring, image recognition and / or vibration detection to detect abnormal conditions in the vehicle's environment and abnormal collisions suffered by the vehicle. Sentry mode includes multiple sentry states, and the sentry state changes with the detection results. In some embodiments, the sentry state includes a standby state and a non-standby state. The non-standby state includes an alert state and an alarm state. The alert state triggering conditions of the sentry mode include: a living being is within a preset distance range (such as 1 meter) around the vehicle for a period of time; the alarm state triggering conditions of the sentry mode include a collision or vibration of the vehicle. When sentry mode is enabled, the vehicle enters standby mode. If an alert state triggering condition is detected, such as someone leaning against the car, the sentry mode switches to an alert state. If an alarm state triggering condition is detected, such as someone hitting the car, the sentry mode switches to an alarm state.

[0049] When the vehicle transitions from standby to non-standby mode, a voice reminder is broadcast according to pre-set rules. This voice reminder can be a guiding phrase, directing the user outside the vehicle to engage in voice interaction with the vehicle. For example, "You're too close to me. Want to get to know me? You could say 'Tell me about yourself.'" This guiding phrase not only serves as a reminder without causing any discomfort or inconvenience to the user outside the vehicle, but also demonstrates the product's functionality.

[0050] In some embodiments, when the vehicle changes from a standby state to a non-standby state, a TTS voice announcement is performed to output a preset voice reminder.

[0051] In some embodiments, the broadcasting of the voice reminder according to the preset rules includes: randomly extracting a voice reminder from a preset voice reminder library for broadcasting. The preset voice reminder library stores multiple voice reminders, for example, "You are too close to me, please be careful", "You are too close to me, do you want to get to know me, you can say 'introduce yourself'", "You are too close to me, do you want to move the car? You can say 'I want to move the car'", "You are too close to me, do you want to leave a message for the owner? You can say 'I want to leave a message'", "You are too close to me, do you want to call the owner? You can say 'I want to make a phone call'", "You are too close to me, is there any danger around? You can say 'help me call the police'". By randomly extracting voice reminders from a database storing multiple voice reminders for broadcasting, the broadcasted voice reminders are not repeated, thereby improving the user experience.

[0052] In one embodiment, the broadcasting of voice reminders according to preset rules includes: extracting voice reminders from a preset voice reminder set for broadcasting, wherein the non-standby state includes an alert state and an alarm state, the preset voice reminder set is a voice reminder set corresponding to the current state of the vehicle, or the preset voice reminder set is a voice reminder set corresponding to instructions pre-stored by the owner, or the preset voice reminder set is a voice reminder set corresponding to the current scene of the vehicle.

[0053] In some embodiments, broadcasting a voice reminder according to a preset rule includes randomly selecting and broadcasting a voice reminder from a set of voice reminders corresponding to the vehicle's current state. The vehicle's current state includes an alert state and an alarm state, each corresponding to a separate voice reminder set. Because the alert state and the alarm state are triggered by different factors and pose different levels of threat to the vehicle, presetting different voice reminder sets and providing different voice reminders can better adapt to the vehicle's current state and enhance the intelligence of the voice reminders. For example, a gentle, humorous voice reminder may be used for the alert state, while a stern voice reminder may be used for the alarm state. Voice reminders associated with the alert state may include "You're too close, please be careful," "You're too close, would you like to get to know me? You could say 'Introduce yourself,'" and "You're too close, would you like to move? You could say 'I need to move.'" Voice reminders associated with the alarm state may include "Don't hit me, it hurts," and "Please don't hit me, I'm recording."

[0054] In some embodiments, broadcasting a voice reminder according to preset rules includes: determining whether the vehicle owner has pre-stored instructions, including but not limited to a move reminder instruction, a video recording instruction, and a chat reminder instruction; if so, obtaining and outputting a voice prompt corresponding to the vehicle owner's pre-stored instruction. For example, if the vehicle owner's vehicle is parked in a location that blocks other vehicles and there is a high probability of needing to move the vehicle in the next few hours, the vehicle owner may pre-store a move reminder instruction. When the sentry state changes from standby to non-standby, a corresponding move reminder voice message is broadcast, such as "You're too close to me. Do you want to move? You can say 'I want to move.'" For example, if the vehicle owner's vehicle is parked in a poor location and is at high risk of being vandalized, the vehicle owner may pre-store a video recording instruction. When the sentry state changes from standby to non-standby, a corresponding video recording reminder voice message is broadcast, such as "You're too close to me. I'm recording. Do you have anything to say to me?" For example, if a car owner wants to show off their vehicle, they can pre-store a chat prompt command. When the Sentry mode changes from standby to non-standby, the corresponding chat prompt voice message will be played, such as "You're too close to me. Do you want to get to know me? You can say 'Introduce yourself.'" This allows the vehicle's voice prompts to be more relevant to the vehicle's environment and the owner's preferences when the vehicle changes from standby to non-standby, improving the user experience.

[0055] If the owner's instructions are not pre-stored, a voice reminder is randomly extracted from the preset voice reminder library for broadcast, or a voice reminder is randomly extracted from the voice reminder set corresponding to the current state of the vehicle for broadcast. The relevant explanations are as described above and will not be repeated here.

[0056] In some embodiments, the broadcasting of the voice reminder according to the preset rules includes: obtaining the current scene of the vehicle, selecting and outputting a voice prompt from the voice prompt set corresponding to the scene of the vehicle, wherein multiple scenes can be preset, and a corresponding voice prompt set can be preset for each scene, and the scene types include but are not limited to home, company, shopping mall, downtown, etc. In one implementation scenario, the current scene of the vehicle is determined by obtaining the location of the vehicle. For example, if the vehicle is located at the owner's home, the current scene of the vehicle is a home scene; if the vehicle is located at a shopping mall, the current scene of the vehicle is a shopping mall scene. In another implementation scenario, the current scene of the vehicle is set by the owner. For example, the owner can set the current scene of the vehicle on a dedicated APP.

[0057] If the vehicle's current scene fails to be acquired, a voice reminder is randomly extracted from the preset voice reminder library for broadcast, or a voice reminder is randomly extracted from the voice reminder set associated with the current sentry state for broadcast. The relevant explanations are as described above and will not be repeated here.

[0058] In the implementation manner given above, the pre-stored voice reminder can be a factory default voice prompt, or a voice recorded by the vehicle owner, or a text set by the vehicle owner and then broadcasted through TTS.

[0059] The vehicle's sentry status (alarm status, alert status), voice reminder content, communication content, etc. are recorded in the log so that the owner can check and understand the vehicle's condition.

[0060] In some embodiments, when the vehicle changes from a standby state to a non-standby state, a pre-sound effect is broadcast first, and then a voice reminder is broadcast according to a preset rule. The pre-sound effect is a prompt sound effect, which is used to remind the vehicle that a voice reminder is about to be broadcast, for example, "ding dong" or "ding", which can avoid the discomfort caused to users outside the vehicle (including pedestrians) by directly and abruptly broadcasting voice reminders.

[0061] Step S20: obtaining voice input from a user outside the vehicle.

[0062] When the vehicle changes from a standby state to a non-standby state, a voice reminder is broadcast according to a preset rule, and after the voice reminder is broadcast according to the preset rule, or while the voice reminder is broadcast according to the preset rule, or before the voice reminder is broadcast according to the preset rule, a voice input from a user outside the vehicle is obtained.

[0063] In some embodiments, when the sentry state changes from standby state to non-standby state, a TTS voice reminder is broadcast according to preset rules to guide the user outside the vehicle to communicate with the vehicle, and at the same time wake up the outside vehicle voice function to obtain the voice input of the user outside the vehicle in real time.

[0064] Step S30: executing a response operation according to the voice input of the user outside the vehicle.

[0065] After noise reduction of the voice input from the user outside the vehicle, speech recognition is performed. Based on the speech recognition results, the user's intent can be determined through keyword and / or semantic understanding. The target action corresponding to the user's intent is further determined, and a response action is executed. Response actions include but are not limited to human-machine voice chat, information query and feedback (such as checking the weather, asking about the horoscope, and introducing the vehicle), sending text messages, and initiating voice calls.

[0066] In an embodiment of the present invention, when a vehicle is in sentry mode and changes from standby to non-standby, a voice reminder is broadcast according to preset rules, voice input from an external user is obtained, and a response operation is performed based on the voice input from the external user. This is to combine the traditional sentry mode with the external voice interaction capability, reuse the traditional sentry mode capability, and use the change in the vehicle operating state (reflected as the sentry state) identified in sentry mode as a trigger condition for external voice interaction. By broadcasting a voice reminder, the external user is reminded, and by obtaining the voice input from the external user, a response operation is performed based on the voice input from the external user, thereby achieving voice interaction between the vehicle and the external user, thereby realizing an external voice interaction experience triggered by a change in sentry mode state. In addition, by reusing the traditional sentry mode capability, there is no need to repeatedly develop monitoring and image recognition functions, thereby achieving an improved product experience while maintaining cost savings.

[0067] In addition, the embodiment of the present invention guides users outside the vehicle to communicate directly with the vehicle and its users through voice reminders outside the vehicle, which can relieve the emotions of users outside the vehicle and provide a more humane and advanced experience. In addition, through the intelligent voice interaction capability outside the vehicle, it can also satisfy the curiosity of users outside the vehicle about the vehicle and enhance the vehicle's brand image and reputation.

[0068] In some embodiments, as shown in FIG2 , step S30 includes:

[0069] Step S31: Identify the user's intention based on the voice input of the user outside the vehicle.

[0070] After noise reduction of the voice input of the user outside the vehicle, voice recognition is performed to identify keywords that represent the user's intention. Based on the identified keywords, they are matched with the preset operation mapping table to determine the corresponding target operation.

[0071] Step S32, when the target operation corresponding to the user's intention is a communication operation, obtain the number of executions of the target operation within a preset period, wherein the communication operation includes at least one of a text message operation, a voice call operation, and a voice message operation, and the preset period includes the current state switching period or the current engine shutdown period, and the current state switching period refers to the period when the vehicle changes from the standby state to the non-standby state and then returns to the standby state.

[0072] In order to prevent the car owner from being disturbed by communication operations such as text messages and phone calls, anti-gaming measures are set for this type of operation to obtain the number of executions of the target operation within the preset period. By limiting the number of executions of the target operation within the preset period, a large number of text messages or phone calls can be prevented from being sent to the car owner's terminal.

[0073] During this state switching period, since each triggering of the sentry mode will cause the sentry state to switch, the vehicle switches from the standby state to the non-standby state (such as the alert state or the alarm state). After the triggering event disappears for a period of time, the vehicle returns to the standby state from the non-standby state.

[0074] The current shutdown period refers to the period from when the vehicle is parked and shut down to when it is restarted.

[0075] Step S33 , determining whether the number of executions of the target operation within the preset period is less than a first preset number.

[0076] Step S34: If yes, execute the target operation.

[0077] Step S35: If not, output a prompt of execution refusal.

[0078] By limiting the number of communication operations during a state switch or an engine shutdown period, it is possible to avoid inconvenience to car owners caused by too many text messages, phone calls, voice messages, etc., thereby ensuring the car owner's product experience.

[0079] In some embodiments, the vehicle-external voice interaction method of the present invention further includes:

[0080] When the sentry state changes from standby state to non-standby state, the statistical parameters during this state switching are obtained; when the statistical parameters meet the preset conditions, the off-vehicle voice interaction function during this state switching is turned off, wherein the preset conditions include at least one of the following: the duration of this state switching period is greater than the preset duration, and the number of interactions during this state switching period is greater than a second preset number.

[0081] Before disabling the off-board voice interaction function during this state switch, a reminder message may be output to inform the user outside the vehicle that the off-board voice interaction function is about to terminate. For example, a countdown "beep beep beep" sound may be output, or a TTS broadcast of "I'm leaving, goodbye" may be broadcast.

[0082] In some implementations, the preset condition is that the duration of this state switch is greater than a preset duration. After Sentry Mode is activated, Sentry Mode enters a standby state. When an abnormality is detected in the vehicle's environment or the vehicle suffers an abnormal collision, Sentry Mode enters a non-standby state. After the abnormality / abnormal collision disappears for a period of time, Sentry Mode returns to a standby state from the non-standby state. When the abnormality / abnormal collision persists, Sentry Mode remains in a non-standby state. The duration of each state switch in this scenario is limited. When a state switch lasts longer than the preset duration, the vehicle's external voice interaction capability is forcibly interrupted.

[0083] Therefore, by limiting the duration of a single state switching period, the vehicle and the owner can be prevented from being maliciously harassed, thereby improving the owner's product experience.

[0084] In some other embodiments, the preset condition is that the number of interactions during the current state transition is greater than a second preset number. When the Sentry state changes from standby to non-standby, the number of interactions during the current state transition is counted. If the number of interactions exceeds the second preset number, it indicates that the number of interactions during the state transition is too high, and the vehicle-external voice interaction capability is forcibly interrupted, thereby preventing the vehicle and the owner from being maliciously harassed and improving the owner's product experience.

[0085] In other embodiments, the statistical parameter is determined to meet the preset condition when either the duration of the current state transition is greater than a preset duration or the number of interactions during the current state transition is greater than a second preset number. By limiting the duration of the state transition and the number of interactions, a better anti-gaming effect is achieved, improving the owner's product experience.

[0086] In some embodiments, the vehicle-external voice interaction method of the present invention further includes:

[0087] The number of external voice interactions during the current ignition-off period is obtained; if the number of external voice interactions exceeds a third preset number, the external voice interaction function is disabled during the current ignition-off period. The external voice interaction function stops obtaining external user voice input or stops responding to external user voice input. The third preset number is greater than the first preset number and greater than or equal to the second preset number.

[0088] By limiting the number of off-vehicle voice interactions during a period of vehicle shutdown, the car owner can be prevented from being disturbed by text messages and phone calls, and the vehicle can be prevented from being triggered too many times, resulting in excessive power loss.

[0089] In some embodiments, step S30 includes: identifying the user intention based on the voice input of the user outside the vehicle; when the target operation corresponding to the user intention is a communication operation, determining the target object based on the voice input of the user outside the vehicle, wherein the communication operation includes at least one of a text message operation, a voice call operation, and a voice message operation, and the target object includes the vehicle owner terminal or the vehicle manufacturer terminal; executing the target operation corresponding to the user intention on the target object.

[0090] Specifically, after noise reduction is performed on the voice input from the user outside the vehicle, voice recognition is performed to identify keywords that represent the user's intent. Based on the identified keywords, the corresponding target operation is determined by matching them with a preset operation mapping table. For example, if the keywords "move car" and "leave message" are identified from the voice input from the user outside the vehicle and matched with the preset operation mapping table, the corresponding target operation is determined to be a text message leaving operation. If the keywords "make a call" and "contact the owner" are identified from the voice input from the user outside the vehicle and matched with the preset operation mapping table, the corresponding target operation is determined to be executing a voice call operation.

[0091] When executing the SMS message operation, as shown in Figure 4, the SMS interface of the TSP cloud service message service can be called to send an SMS message to the target object.

[0092] When initiating a voice call, the vehicle's built-in network voice call application can be invoked to request a voice call with the target party. This can include requesting a voice call with the vehicle owner's terminal or requesting a voice call with the vehicle manufacturer's terminal through the vehicle manufacturer's hotline. As shown in Figure 4, in one embodiment, leveraging the vehicle's existing "E-CALL" capability, when a distress-sounding command such as "help" or "help" is recognized from the voice input of an external user, it is matched against a preset operation mapping table, and the corresponding target operation is determined to be a voice call operation. Based on this, the target party is determined to be the vehicle manufacturer, and the E-Call capability is automatically invoked (equivalent to clicking the "SOS" button in the vehicle), initiating a voice call request to the vehicle manufacturer's E-Call service platform. Using an external microphone and external speakers as the basic equipment for external user voice calls improves voice call services for external users. By leveraging the capabilities of existing network voice call applications or communication service hardware, when a voice call request is detected by an external user, the network voice call application is automatically invoked to initiate a voice call request to the vehicle user, providing an instant communication channel between the external user and the vehicle owner.

[0093] As shown in Figure 4, executing a target action on a target object can involve sending a text message to the vehicle owner's mobile device or a message to the vehicle owner's or vehicle manufacturer's dedicated app. When a vehicle owner's or vehicle manufacturer's terminal receives a text message, voice call request, or voice message, the user can edit the reply text. This text is then sent to the vehicle computer via the cloud service and announced via text-to-text transmission (TTS) outside the vehicle, allowing users outside the vehicle to receive a response more promptly and avoiding conflicts caused by parking.

[0094] By executing communication operations such as SMS message operations, voice call operations, and voice message operations, it provides channels for users outside the vehicle to communicate with car owners and car companies, avoiding conflicts caused by parking problems and improving communication efficiency and effectiveness.

[0095] In some embodiments, step S30 includes:

[0096] Based on the voice input of the user outside the vehicle, the user intention is identified; whether the target operation corresponding to the user intention belongs to the preset prohibited operation is determined; if so, the target operation is prohibited; if not, the target operation is executed.

[0097] The preset prohibited operation can be implemented as an operation instruction set, and whether the target operation is one of the preset prohibited operations can be determined by judging whether the target operation is one of the operation instruction set.

[0098] The preset prohibited operations can also be implemented as a skill domain of semantic understanding. By performing semantic understanding on the target operation, it is determined whether the corresponding skill belongs to the preset skill domain. If so, it is determined to be a preset prohibited operation.

[0099] Among them, the preset prohibited operations include but are not limited to vehicle control, navigation, system control, multimedia and other operations that change the vehicle status, involve vehicle safety and user privacy, such as waking up the in-car voice control, turning on the navigation, turning on the music, adjusting the volume, opening the windows, air conditioning, and trunk, etc., to prevent the vehicle from being controlled by illegal vehicle users, and to prevent the situation where the windows, air conditioning, and trunk are opened by users outside the vehicle, which endangers the vehicle safety, meet safety and compliance requirements, and avoid causing loss of product experience to car owners.

[0100] If the target operation is a preset prohibited operation, the target operation is prohibited from being executed. At the same time, a prohibition prompt can be output, such as "You do not have the operation permission."

[0101] If the target operation does not belong to the preset prohibited operations, the target operation is executed. For example, if the target operation is to check the weather, and this operation does not belong to the preset prohibited operations, the weather query operation is executed.

[0102] The embodiment of the present invention identifies the target operation corresponding to the voice input of the user outside the vehicle, presets prohibited operations, and intercepts the voice commands of the user outside the vehicle, thereby avoiding the execution of commands such as vehicle component control or system application control during voice interaction outside the vehicle, thereby ensuring vehicle safety and avoiding loss of product experience for the owner.

[0103] In some embodiments, the step S10 of broadcasting a voice reminder according to a preset rule includes:

[0104] Acquire an image of the vehicle's environment; identify the life type that triggers the vehicle from a standby state to a non-standby state based on the image, where the life type includes at least two of children, animals, and adults; and output a voice reminder corresponding to the life type.

[0105] When the life form that triggers the sentinel is a child or an animal, due to the special nature of children and animals, a voice reminder exclusive to children or animals is used. And because children and animals usually do not interact with the vehicle by voice and have no need to contact the owner, the off-vehicle voice interaction function and the voice monitoring function can be disabled, thereby reducing unnecessary loss of vehicle power.

[0106] In some embodiments, in step S10, when the vehicle is in sentry mode, when the sentry state changes from standby state to non-standby state, broadcasting a voice reminder according to a preset rule specifically includes:

[0107] The system obtains voice input from a user outside the vehicle and captures an image of the user outside the vehicle. The system then identifies the life form type that triggers the sentry based on the voice input and image. If the life form type is a child or an animal, the system outputs a voice alert corresponding to the child or animal. Steps S20 and subsequent steps may not be executed until the voice alert corresponding to the child or animal is output.

[0108] In one implementation scenario, as shown in FIG4 , the vehicle-external voice interaction method of the present invention includes:

[0109] When the owner turns on Sentry Mode and locks the car and leaves, Sentry Mode is in standby mode;

[0110] When an outside user approaches or taps the vehicle, the Sentry Mode switches from standby to non-standby (alert or alarm). The Sentry Mode application sends the Sentry Mode state switching information to the Active Interaction Module, which wakes up the outside TTS to broadcast a voice reminder and then obtains the outside user's voice input (the Active Interaction Module obtains the outside user's voice input through the voice engine).

[0111] The user outside the vehicle interacts with the vehicle through voice. During this process, the vehicle's voice engine recognizes the user's intention. When the user intends to leave a message, the vehicle's voice engine will announce "Please say your message" through the vehicle's TTS. When the user intends to move the vehicle, the vehicle's TTS will announce "OK, contacting the owner." When the user intends to make a phone call, the vehicle's TTS will announce "OK, calling the owner." When the user intends to ask for help, the vehicle's TTS will announce "OK, calling the emergency hotline." When the user intends to chat or provide information services, the vehicle's voice semantic processing is reused.

[0112] When the user intends to leave a message or move the car, the TSP message service is called to send a text message notification to the car owner. The car owner checks the text messages on his mobile phone or uses the mobile phone APP to view the message. The car owner can also reply on the mobile phone APP to upload the car owner's reply text to the cloud, and then send the car owner's reply text to the vehicle host, and the car owner's reply text is broadcast through the external voice TTS.

[0113] When the user intends to make a phone call or ask for help, the phone interface is called to call the car owner or the car manufacturer's hotline.

[0114] In some embodiments, as shown in FIG3 , a system architecture diagram for implementing the above-described method for external vehicle voice interaction is shown. The system includes: an active interaction module, a sentinel module, an external vehicle voice pickup device, and an external vehicle voice output device.

[0115] The Sentry Module is configured to adjust the Sentry state based on the vehicle environment and transmit changes in Sentry state to the Active Interaction Module. In one embodiment, the Sentry Module can configure Sentry mode and provide a Sentry application for the vehicle owner to enable or disable Sentry mode. The Sentry Module can connect to a camera, acquire images captured by the camera, identify the images using an image recognition algorithm model, and switch Sentry state based on the image recognition results. The Sentry Module can also connect to a vibration sensor, receiving vibration signals from the sensor and switching Sentry state based on the vibration signals.

[0116] The active interaction module is configured to receive sentry status change information from the sentry module and, when the sentry status changes from standby to non-standby, broadcast a voice reminder via an external voice output device according to preset rules. Optionally, before broadcasting the voice reminder via the output device, the active interaction module further determines whether preconditions for enabling external voice are met. Otherwise, the voice reminder is not broadcast. Preconditions may include: the external voice master switch is on, the vehicle is in park, the driver's seat is unoccupied, the central locking system is locked, and the vehicle key is outside the vehicle.

[0117] The vehicle's external voice pickup device is used to pick up the voice of the user outside the vehicle and send the voice of the user outside the vehicle to the active interaction module. The vehicle's external voice pickup device can be an external microphone.

[0118] The active interaction module is also used to obtain voice input from users outside the vehicle and perform response operations based on the voice input from users outside the vehicle.

[0119] The specific explanation of the execution steps of the active interaction module is as above and will not be repeated here.

[0120] In one embodiment, in the sentinel module, the image recognition algorithm module uses the panoramic camera to monitor the video stream around the vehicle, identifies the biological body approaching the vehicle, and sends the recognition result to the sentinel mode application. After the sentinel mode application receives the signal that the biological body is approaching the vehicle, it triggers the alert state and starts the video recording function. At the same time, the alert state trigger signal is transmitted to the active interaction module in the monitoring state. The active interaction module then controls the broadcast voice and wakes up the voice dialogue state, obtains the voice input of the user outside the vehicle, and performs a response operation based on the voice input of the user outside the vehicle.

[0121] Through the active interaction module to perceive the working condition and operating status of the entire vehicle, the existing sentry mode products, external voice interaction products and other product strengths are integrated to provide car owners with different external voice interaction product services.

[0122] In some embodiments, as shown in Figure 3, the outdoor voice interaction system also includes a voice engine, an indoor voice pickup device and an indoor voice output device; the voice engine includes a voice switching module, a voice arbitration module, an indoor voice recognition module and an outdoor voice recognition module, the indoor voice pickup device is connected to the indoor voice recognition module, the outdoor voice pickup device is connected to the outdoor voice recognition module, and the indoor voice recognition module and the outdoor voice recognition module are both connected to the voice arbitration module; the voice switching module is connected to the outdoor voice pickup device and the outdoor voice output device, the indoor voice pickup device and the indoor voice output device, and the indoor voice recognition module is connected to the outdoor voice recognition module, and is used to control the switching between the outdoor voice pickup device and the indoor voice pickup device, as well as the switching between the outdoor voice output device and the indoor voice output device, and the switching between the indoor voice recognition module and the outdoor voice recognition module.

[0123] Among them, after the voice switching module receives the in-vehicle and out-vehicle voice switching command sent by the active interaction engine, it switches between the outside-vehicle voice pickup device and the inside-vehicle voice pickup device, as well as between the outside-vehicle voice output device and the inside-vehicle voice output device, and between the inside-vehicle voice recognition module and the outside-vehicle voice recognition module.

[0124] The voice arbitration module is a foundational capability module for voice products. It includes a semantic understanding algorithm model for understanding the semantics of user voice input. Both the in-vehicle and out-of-vehicle voice recognition modules are connected to the voice arbitration module. This allows the reuse of in-vehicle voice capabilities when implementing some out-of-vehicle voice interaction functions, reducing development difficulty and cost. In one embodiment, when the out-of-vehicle user's voice input identifies the intended action as voice chat or information service, the in-vehicle voice capabilities are reused. The out-of-vehicle voice interaction shares the semantic understanding algorithm model with the in-vehicle voice interaction, making out-of-vehicle voice interaction more engaging and safer while remaining compliant.

[0125] The specific explanation of the execution steps of the active interaction module is as above and will not be repeated here.

[0126] The embodiment of the present invention takes the active interaction module product as the hub, and through the active interaction module's perception of the vehicle's working conditions and operating status, integrates the existing sentry mode products, off-vehicle voice interaction products, vehicle-system Internet phone products, vehicle networking products and other product strengths to provide car owners with different off-vehicle voice interaction product services.

[0127] In some embodiments, the vehicle-external voice interaction device of the present invention includes a computer-readable storage medium storing a computer program and a processor. When the computer program is read and executed by the processor, the vehicle-external voice interaction method described above is implemented. The vehicle-external voice interaction device can be implemented as a vehicle body domain controller. The advantages of the vehicle-external voice interaction device of the embodiments of the present invention compared to the prior art are substantially the same as those of the vehicle-external voice interaction method described above and are not further described here.

[0128] In some embodiments, the vehicle of the present invention includes the above-described vehicle-external voice interaction device. The beneficial effects of the vehicle of the present invention embodiment relative to the prior art are substantially the same as those of the above-described vehicle-external voice interaction method, and are not further described here.

[0129] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM). In the embodiments of the present invention, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present invention. In addition, the functional units in multiple embodiments of the present invention can be integrated into a single processing unit, multiple units can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or software functional units.

[0130] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the off-vehicle voice interaction method as described in any of the above embodiments.

[0131] Although the embodiments of the present invention are disclosed above, the protection scope of the embodiments of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention, and these changes and modifications will fall within the protection scope of the embodiments of the present invention.

Claims

1. A method for voice interaction outside a vehicle, comprising: When the vehicle is in sentry mode and changes from standby to non-standby, a voice reminder is broadcast according to the preset rules; Get voice input from users outside the vehicle; A response operation is performed according to the voice input of the user outside the vehicle.

2. The vehicle-external voice interaction method according to claim 1, wherein: The performing a response operation according to the voice input of the user outside the vehicle includes: Recognizing user intention according to the voice input of the user outside the vehicle; When the target operation corresponding to the user intention is a communication operation, the number of executions of the target operation within a preset period is obtained, wherein the communication operation includes at least one of a text message operation, a voice call operation, and a voice message operation, and the preset period includes the current state switching period or the current engine shutdown period, and the current state switching period refers to the period during which the vehicle changes from the standby state to the non-standby state and then returns to the standby state; Determining whether the number of executions of the target operation within the preset period is less than a first preset number; If so, executing the target operation; If not, a prompt indicating that execution is rejected is output.

3. The vehicle-external voice interaction method according to claim 1 or 2, further comprising: When the sentinel state changes from the standby state to the non-standby state, obtaining statistical parameters during this state switching period, wherein the state switching period refers to the period during which the vehicle changes from the standby state to the non-standby state and then returns to the standby state; When the statistical parameters meet the preset conditions, the off-vehicle voice interaction function during this state switching is turned off, wherein the preset conditions include at least one of the following: the duration of this state switching is greater than the preset duration, and the number of interactions during this state switching is greater than a second preset number.

4. The vehicle-external voice interaction method according to claim 1 or 2, further comprising: Get the number of vehicle-external voice interactions during this shutdown period; When the number of external vehicle voice interactions is greater than a third preset number, the external vehicle voice interaction function during the current engine shutdown period is turned off.

5. The vehicle-external voice interaction method according to claim 1 or 2, wherein: The broadcasting of voice reminders according to preset rules also includes: Voice reminders are extracted from a preset voice reminder set for broadcasting, wherein the non-standby state includes an alert state and an alarm state, the preset voice reminder set is a voice reminder set corresponding to the current state of the vehicle, or the preset voice reminder set is a voice reminder set corresponding to instructions pre-stored by the owner, or the preset voice reminder set is a voice reminder set corresponding to the current scene of the vehicle.

6. The vehicle-external voice interaction method according to claim 1 or 2, wherein: According to the voice input of the user outside the vehicle, executing a response operation includes: Recognizing user intention according to the voice input of the user outside the vehicle; When the target operation corresponding to the user intention is a communication operation, determining the target object according to the voice input of the user outside the vehicle, wherein the communication operation includes at least one of a text message operation, a voice call operation, and a voice message operation, and the target object includes a vehicle owner terminal or a vehicle manufacturer terminal; Perform a target operation corresponding to the user intention on the target object.

7. The vehicle-external voice interaction method according to claim 1 or 2, wherein: According to the voice input of the user outside the vehicle, executing a response operation includes: Recognizing user intention according to the voice input of the user outside the vehicle; Determining whether the target operation corresponding to the user's intention belongs to a preset prohibited operation; If so, prohibiting execution of the target operation; If not, the target operation is performed.

8. The vehicle-external voice interaction method according to claim 1 or 2, wherein: The voice reminder broadcast according to the preset rules includes: Acquire an image of the vehicle's environment; a life type that triggers the vehicle from the standby state to the non-standby state according to the image recognition, wherein the life type includes at least two of children, animals, and adults; Output a voice reminder corresponding to the life form type.

9. An external vehicle voice interaction device, comprising a computer-readable storage medium storing a computer program and a processor, wherein when the computer program is read and executed by the processor, the external vehicle voice interaction method according to any one of claims 1 to 8 is implemented.

10. A vehicle, comprising the off-vehicle voice interaction device according to claim 9.

Images