Terminal control method, device, medium, equipment, vehicle or robot

Abstract

The application relates to a terminal control method and device, a medium, equipment and a vehicle or robot, and relates to the technical field of interactive control, wherein the terminal control method comprises the following steps: receiving a voice instruction of a target user outside a cabin of the terminal; and controlling the terminal to move to a target position corresponding to the voice instruction based on the voice instruction. The application enriches the interactive control form of the terminal by controlling the terminal to move to a specified target position through a voice instruction, and the voice instruction control is more in line with the natural human-computer interaction demand, so that the convenience and flexibility of the terminal interactive control are improved.

Description

Technical Field

[0001] This application relates to the field of interactive control technology, specifically to a terminal control method, device, medium, equipment, and vehicle or robot. Background Technology

[0002] Currently, in interactive control scenarios of terminals such as vehicles and robots, most rely on operation buttons or interactive controls to control the terminal. These interaction methods are simple, lack convenience and flexibility, and are difficult to meet the user's needs. Summary of the Invention

[0003] In view of this, this application provides a terminal control method, device, medium, equipment, and vehicle or robot, the main purpose of which is to control the terminal to move to the target position corresponding to the voice command by receiving voice commands from the target user outside the terminal cockpit, thereby improving the convenience and flexibility of terminal interactive control and optimizing the user operation experience.

[0004] In a first aspect, this application provides a terminal control method, the method comprising:

[0005] Receives voice commands from the target user outside the cockpit of the terminal; Based on the voice command, the terminal is controlled to move towards the target location corresponding to the voice command.

[0006] Optionally, controlling the terminal to move towards the target location corresponding to the voice command includes: Based on the first voice command of the target user, control the terminal to move towards the first location corresponding to the target user; and / or, Based on the target user's second voice command, the terminal is controlled to move to the second position indicated by the second voice command.

[0007] Optionally, controlling the terminal to move towards the first location corresponding to the target user based on the target user's first voice command includes: In response to the target user being within the recognition range of the terminal, the terminal is controlled to move towards the first position based on the target user's first voice command.

[0008] Optionally, the method further includes: In response to the target user being outside the recognition range of the terminal, a first voice message is output and / or the terminal is controlled to move in the direction corresponding to the target user. The first voice message is used to prompt the target user that the target user is outside the recognition range and / or to prompt the target user to move.

[0009] Optionally, the method further includes: In response to the fact that the direction corresponding to the target user is the direction in which the terminal's movement is restricted, a second voice message is output; the second voice message is used to prompt the target user that the direction corresponding to the target user is the direction in which the terminal's movement is restricted and / or to prompt the target user to move.

[0010] Optionally, before controlling the terminal to move to the first location corresponding to the target user, the method further includes: A third voice message is output, which is used to prompt that the target user will be located within the recognition range.

[0011] Optionally, controlling the terminal to move towards the first location corresponding to the target user based on the target user's first voice command includes: The target user's action is determined based on the target user's first voice command; Based on the target user's actions, the location where the target user is located or the location indicated by the target user is determined as the first location, and the terminal is controlled to move towards the first location.

[0012] Optionally, the target user's actions include at least one of the target user's body movements, head movements, hand gestures, and facial movements.

[0013] Optionally, controlling the terminal to move to the second location indicated by the second voice command based on the target user's second voice command includes: Based on the address information corresponding to the second voice command, the terminal is controlled to move to the second location corresponding to the address information.

[0014] Optionally, controlling the terminal to move to the second location indicated by the second voice command based on the address information corresponding to the second voice command includes: Based on the address information, a fourth voice message is output, which is used to determine the terminal's movement route information; In response to the confirmation information corresponding to the movement route information, the terminal is controlled to move to the second location corresponding to the address information.

[0015] Optionally, controlling the terminal to move to the second location indicated by the second voice command based on the target user's second voice command includes: In response to the target identifier corresponding to the second voice command being within the recognition range of the terminal, the terminal is controlled to move to the second position indicated by the second voice command based on the target identifier.

[0016] Optionally, controlling the terminal to move towards the target location corresponding to the voice command based on the voice command of the target user outside the terminal cockpit includes: Based on the voice command of the target user outside the terminal cockpit, a fifth voice is output, which is used to perform a confirmation inquiry based on the voice command. Based on the response information from the fifth voice message, the terminal is controlled to move towards the target location corresponding to the response information.

[0017] Optionally, controlling the terminal to move towards the target location corresponding to the response information based on the fifth voice message includes: If the response information is a confirmation message from the fifth voice, the terminal is controlled to move towards the target location corresponding to the voice command. If the response information is an update of the target location, the terminal is controlled to move towards the target location corresponding to the response information based on the update information.

[0018] Optionally, the method further includes: When the terminal has sufficient space to move, a sixth voice message is output, which is used to prompt the terminal to move to the target location; If the terminal has insufficient space to move, a seventh voice message is output; the seventh voice message is used to indicate that the terminal cannot be moved and / or to indicate that the terminal needs to be taken over.

[0019] Optionally, the terminal may include a vehicle or a robot.

[0020] Secondly, this application provides a terminal control device, which includes a receiving module and a control module; The receiving module is configured to receive voice commands from a target user outside the cockpit of the terminal. The control module is configured to control the terminal to move towards the target location corresponding to the voice command based on the voice command.

[0021] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0022] Fourthly, this application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and running thereon, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0023] Fifthly, this application provides a vehicle or robot, including: the terminal control device described in the second aspect above, or the computer-readable storage medium described in the third aspect above, or the electronic device described in the fourth aspect above.

[0024] By employing the above technical solutions, this application provides a terminal control method, device, medium, equipment, and vehicle or robot. The terminal control method receives voice commands from a target user outside the terminal's cockpit and controls the terminal to move towards the target location corresponding to the voice command. This application enriches the interactive control methods of the terminal by controlling its movement to a designated target location via voice commands. Furthermore, voice command control aligns with the natural human-computer interaction needs, thus improving the convenience and flexibility of terminal interactive control.

[0025] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings: Figure 1 A schematic flowchart of a terminal control method provided in an embodiment of this disclosure is shown; Figure 2 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 3 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 4 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 5 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 6 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 7 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 8 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 9 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 10 This illustration shows a schematic diagram of an interactive scenario provided by an embodiment of the present disclosure; Figure 11 A schematic diagram of a control flow provided by an embodiment of this disclosure is shown; Figure 12 This illustration shows a query process diagram provided by an embodiment of the present disclosure; Figure 13 A schematic diagram of the structure of a terminal control device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0027] In the description of this disclosure, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this disclosure and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure.

[0028] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise expressly specified.

[0029] In this disclosure, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0030] The present disclosure will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0031] The following is combined with Figure 1 Some embodiments of the terminal control method disclosed herein are introduced.

[0032] This disclosure provides a terminal control method, such as... Figure 1 As shown, the control method includes the following steps S101~S102: S101, Receive voice commands from the target user outside the cockpit of the terminal.

[0033] The target user is, for example, the user of the terminal, or other personnel with terminal operation permissions. The voice command is, for example, a voice command containing the target user's mobile control intention, which may be natural speech output by the target user, or a digital voice signal obtained after analog-to-digital conversion of natural speech, or instruction information obtained after semantic parsing and instruction recognition processing of digital voice signal.

[0034] In one embodiment of this invention, natural speech output by the target user is acquired via an audio acquisition unit deployed outside the terminal cockpit. This audio acquisition unit includes a microphone array and an audio processing unit, which converts the acquired natural speech into a digital speech signal and transmits the digital speech signal to the terminal's controller via a data transmission bus. Upon receiving the digital speech signal, the controller performs semantic analysis and command recognition, extracting command information containing the user's intention to control the device.

[0035] It is understood that this embodiment can also receive the target user's natural speech through wireless communication devices such as mobile phones, smartwatches, smart bracelets, smart glasses, and smart Bluetooth keys. These wireless communication devices convert the natural speech into digital voice signals and send the digital voice signals to the terminal's controller through their wireless communication links with the terminal. The controller performs semantic analysis and command recognition on the received digital voice signals to extract command information containing mobile control intentions.

[0036] For example, when performing semantic parsing and command recognition on digital voice signals, the controller improves the accuracy and adaptability of voice command recognition through generalization processing and referential resolution processing, thereby accurately recognizing the user's mobile control intentions.

[0037] The generalization process is used to perform semantic generalization matching on the diverse spoken expressions contained in natural speech. For example, the controller has a built-in semantic generalization library that maps different spoken expressions to similar control commands, avoiding recognition failures due to differences in expression. For example, "Come here" and "Move over here" can be uniformly mapped to commands to move to the target user's location.

[0038] Referential resolution processing is used to determine referential words contained in natural speech. For example, the controller combines context, environment, or historical information to semantically resolve ambiguous referents such as "here," "there," and "this location" to determine the actual location or direction. For instance, "here" in "drive here" can be determined as the target user's current location.

[0039] S102. Based on voice commands, control the terminal to move to the target location corresponding to the voice command.

[0040] In one embodiment of this invention, the movement of the control terminal towards the target location corresponding to the voice command can be determined by the terminal's controller, which issues execution commands to control the terminal's actuators, such as power, steering, braking, or drive, to perform movement actions, thereby causing the terminal to move towards the target location. For example, the controller determines, based on the voice command, that the target user's movement control intention is to move the terminal to the target location. Subsequently, the controller generates an execution command based on this movement control intention to move the terminal from its current location to the target location and issues it to the terminal's actuators to drive the terminal to move towards the target location.

[0041] In some examples, when generating execution instructions, the controller can also combine the terminal's status information and surrounding environment information to comprehensively decide on the specific parameters and execution strategy of the instructions. The terminal's status information may include current battery level, speed, vehicle posture, and fault status. The surrounding environment information may include obstacle distribution, road conditions, movable areas, and positional constraints. The controller makes a comprehensive decision based on this information; if it detects that the terminal's current battery level is low or that there are obstacles at the target location, it can adjust the speed or detour accordingly.

[0042] In one embodiment of this invention, the controller may be a controller deployed locally on the terminal or a cloud controller deployed on a cloud server. The terminal-local controller described in this application may be a single controller or a combination of multiple domain controllers. Taking a vehicle as an example, the vehicle's intelligent driving domain controller, intelligent cockpit domain controller, etc., may be used individually or in combination as the controller described in this embodiment.

[0043] This disclosure enables mobile control of the terminal using voice commands, enriching the interactive control methods of the terminal, improving the convenience and flexibility of control, and optimizing the user's operating experience.

[0044] Optionally, the terminal described in this disclosure may include a vehicle or a robot.

[0045] The robots mentioned may include manned flying robots (such as personal aircraft with cockpits or air taxis), manned walking robots (such as large biomimetic walking vehicles equipped with seats or enclosed cabins, including large versions of robotic horses and robotic dogs), mobile intelligent service robots (such as automated mobile cafes, mobile karaoke rooms, and mobile clinics with integrated cockpits for user use), and mobile operating cabins in industrial / special operations robots (such as mobile robot platforms equipped with protective cockpits in mining and disaster relief scenarios, supporting operators to ride or work autonomously), etc. These vehicles or robots can receive voice commands from target users outside the cockpit to perform corresponding movement, service, or operational tasks.

[0046] For example, if the terminal is a vehicle, the controller can send corresponding execution commands to the vehicle's motor / engine, steering gear, brake pump and other actuators to control the vehicle's movement, steering and braking, so as to control the vehicle to move to the target position.

[0047] If the terminal is a robot, the controller can send execution commands to the robot's drive wheels, servo motors and other actuators to control the robot's movement and turning, thereby controlling the robot to move to the target position.

[0048] This disclosure also provides some embodiments for controlling the terminal to move to the target location corresponding to the voice command, so that the controller can control the terminal to move to the corresponding location and / or output corresponding voice information to interact with the target user based on different voice commands of the target user.

[0049] In an optional embodiment, based on a first voice command from the target user, the control terminal moves to a first location corresponding to the target user; and / or, based on a second voice command from the target user, the control terminal moves to a second location indicated by the second voice command.

[0050] The first voice command and the second voice command are both voice commands. The first voice command and the second voice command respectively indicate target locations of different directional types. For example, the first voice command points to the first location corresponding to the target user, and the second voice command points to the second location indicated by the voice command.

[0051] In one embodiment of this example, the controller receives a first voice command from the target user and controls the terminal to move to the first location corresponding to the target user.

[0052] For example, the controller parses the received voice command (semantic parsing and command recognition), identifies that the voice command contains a movement control intent pointing to a first location corresponding to the target user, and determines the voice command as the first voice command. Then, the controller generates an execution command to move to the first location based on the first voice command and sends it to the actuator of the terminal to control the terminal to move to the first location.

[0053] In one scenario, a target user in a parking lot issues a voice command to a vehicle saying, "Drive to my side." The vehicle then drives out of its parking space, moves towards the target user's location (the first location), and stops beside the target user.

[0054] In another scenario, the target user gives the voice command "drive to the pillar next to me" to the vehicle in the parking lot. The vehicle then moves to the designated location (second position) where the pillar next to the target user is located.

[0055] In one embodiment of this example, the controller receives a second voice command from the target user and controls the terminal to move to the second position indicated by the second voice command.

[0056] For example, the controller parses the received voice command, identifies that the voice command contains a direction identifier or address indicating a movement control intention, and determines that the voice command is a second voice command. Then, the controller generates an execution command to move to the second location based on the second voice command and sends it to the terminal's controller to control the terminal to move to the second location.

[0057] In one scenario, the target user gives the voice command "drive to address xxx" to the vehicle in the park, and then the vehicle moves from the current parking point to address xxx (the second location).

[0058] In another scenario, the target user gives the voice command "drive to the pillar in front" to the vehicle, and then the vehicle moves to the location of the pillar in front (the second position).

[0059] In one embodiment of this invention, the controller receives a first voice command and a second voice command from the target user, and controls the movement of the terminal using the following two execution methods: The first execution method is sequential execution, that is, the controller generates and sends (sends to the actuator) execution instructions to move to the first position and to the second position respectively according to the order in which the first voice instruction and the second voice instruction are received.

[0060] In some examples, when controlling the terminal to move, the controller may issue an execution command to move to the previous position (the position corresponding to the previously received voice command), and after the terminal moves to the previous position, issue an execution command to move to the next position (the position corresponding to the subsequently received voice command), thereby controlling the terminal to move to the next position. Alternatively, the controller may interrupt the terminal's movement while it is moving to the previous position and then control the terminal to move to the next position.

[0061] In some scenarios, such as when the target user first issues the voice command "Drive over here," the vehicle then moves to the target user's initial location. While the vehicle is moving, the target user issues a second voice command, "Park in the adjacent parking space." The vehicle then stops its current movement towards the initial location, reverses direction to execute the second voice command, and drives into the adjacent parking space.

[0062] In other scenarios, the target user first issues the first voice command, "Drive to my location," and the vehicle drives out of the parking space and moves towards the target user's location. While the vehicle is moving, the target user issues the second voice command, "Drive out of the parking lot." The vehicle does not interrupt its current movement and, after reaching the target user's location and waiting for the target user to get in, continues to execute the second voice command to drive out of the parking lot.

[0063] The second execution method is integrated execution. After the controller parses the first position corresponding to the first voice command and the second position corresponding to the second voice command, it combines the movement routes corresponding to the two commands to merge the routes and plan an integrated route that includes the first position and the second position. Then, it sends the execution command corresponding to the integrated route to the actuator to control the terminal to continuously complete the movement action to the first position and the second position along the integrated route.

[0064] It should be noted that the application scenarios of the second execution method are similar to those of the first execution method, with the only difference being the internal movement control logic. Therefore, its application scenarios will not be elaborated here.

[0065] In this embodiment, by distinguishing the location-pointing type of voice commands and adapting them to the corresponding mobile control logic, and supporting flexible methods such as independent execution of single commands, sequential execution of multiple commands, and integrated execution, the system accurately matches different mobile control needs of users, improves the adaptability and execution efficiency of the voice control terminal, makes human-computer voice interaction fit the actual use scenario, and further optimizes the user's operating experience.

[0066] Furthermore, in this disclosure, when the terminal is controlled to move towards the first position corresponding to the target user based on the first voice command of the target user, the controller, in response to the target user being within the recognition range of the terminal, controls the terminal to move towards the first position based on the first voice command of the target user.

[0067] The recognition range of the terminal refers, for example, to the visual recognition range of the visual acquisition components (such as cameras, visual sensors, etc.) mounted on the terminal. Alternatively, it refers to the point cloud recognition range of the lidar component mounted on the terminal. This recognition range is determined by the hardware parameters of the visual acquisition components or lidar components, the installation angle, and factors such as ambient light and obstacles. It is understood that the adaptation principle of the visual acquisition range and the point cloud acquisition range is the same in this disclosure, and the following description of the visual acquisition range applies to the point cloud recognition range, which will not be repeated here.

[0068] In one embodiment of this example, after receiving the first voice command, the controller first acquires image data of the surrounding environment through the vision acquisition component, and then performs feature recognition on the image data. If a target user is identified, an execution command is generated based on the target user's location (first location) and the movement control intention to the target user's corresponding location contained in the first voice command, and the execution command is sent to the executor to control the terminal to move to the first location corresponding to the target user.

[0069] In some scenarios, when the target user issues the first voice command "drive over here" in the area in front of the vehicle (within the recognition range), the vehicle will immediately move from its current parking space to the first location where the target user is located.

[0070] In other scenarios, when the target user is separated from the vehicle by another vehicle (outside the recognition range) and issues the first voice command "drive over here", the vehicle remains in its current state and does not perform any movement.

[0071] In this embodiment, the determination of the recognition range provides accurate location information for the terminal to move to the user's location, ensuring the accuracy of voice command execution and the safety of movement, and improving the reliability of voice control.

[0072] Optionally, in response to the target user being outside the recognition range of the terminal, the controller outputs a first voice and / or controls the terminal to move in the direction corresponding to the target user, wherein the first voice is used to prompt the target user that the target user is outside the recognition range and / or prompt the target user to move.

[0073] For example, the first voice may be used to inform the target user that the vehicle is currently out of the recognition range, guide the target user to move to the specified recognition range, or prompt the target user that the vehicle will actively drive in the direction it is in.

[0074] For example, the direction corresponding to the target user is the approximate direction of the target user relative to the terminal. This direction can be determined using wireless communication devices and by combining signal ranging and direction-finding positioning algorithms.

[0075] like Figure 2 The diagram illustrates an interactive scenario. The target user is located outside the visual recognition range but within the wireless signal recognition range. The terminal's wireless signal recognition range includes the left front area, right front area, and rear area. The controller, using wireless communication equipment and signal ranging and direction-finding positioning algorithms, determines that the target user is located in the right front area of ​​the terminal. Subsequently, the controller sends an execution command to the actuator to move to the right front, controlling the terminal to move to the right front. like Figure 3 The diagram illustrates an interactive scenario. The terminal is parked in a parking space, with adjacent vehicles obstructing its view on both sides. The target user, positioned to the left front of the terminal and outside its visual recognition range, issues a voice command saying "Drive over here." Upon receiving the voice command and determining from image data acquired by the visual acquisition unit that the target user is outside the visual recognition range, the controller calculates the approximate direction of the target user using wireless positioning signals and built-in signal ranging and direction-finding algorithms, confirming that the target user is located to the left front of the vehicle. Subsequently, the controller issues an execution command to the actuator to move to the left front, controlling the terminal to safely drive out of the obstruction blind spot until the target user is within its visual recognition range or has completely left the parking space.

[0076] In one embodiment of this example, in response to the target user being outside the visual recognition range of the terminal, the controller sends an execution command to the speaker to play voice, thereby controlling the speaker to output the first voice.

[0077] like Figure 4 The diagram illustrates an interactive scenario where a target user is blocked from the vehicle by a parking pillar, placing the user outside the visual recognition range, and issues the first voice command, "Drive over here." The vehicle then plays the first voice message: "You are not recognized. Please move to the front of the vehicle and try again."

[0078] In one embodiment of this example, in response to the target user being outside the recognition range and determining the direction corresponding to the target user, the controller issues an execution command to the actuator to move in the direction corresponding to the target user, thereby controlling the terminal to move in the direction corresponding to the target user.

[0079] In some scenarios, when the target user and the vehicle are obstructed by adjacent parked vehicles, causing the target user to be outside the terminal's recognition range, and the first voice command "drive over here" is issued, the vehicle moves from its current position toward the general direction of the target user.

[0080] In one embodiment of this example, in response to the target user being outside the recognition range of the terminal and determining the direction corresponding to the target user, the controller controls the speaker component to output the first voice and controls the terminal to move in the direction corresponding to the target user.

[0081] In some scenarios, when the target user is obstructed by a parking space pillar and is outside the vehicle's recognition range, and issues the initial voice command "Drive over here," the vehicle first announces, "You are not detected. I will move slightly out of the blind spot to identify you." Then, it slowly moves from its current position towards the approximate direction of the target user. Once the front of the vehicle has rounded the parking space pillar and detected the target user, the vehicle moves towards the target user's location and stops next to them.

[0082] In this embodiment, the robustness, security, and user interaction experience of the voice interaction control terminal are improved by using visual recognition blind spot determination, multi-form voice prompts, and wireless communication positioning assistance to determine the general direction.

[0083] Furthermore, in response to the fact that the direction corresponding to the target user is the direction in which the terminal's movement is restricted, the controller outputs a second voice message; the second voice message is used to prompt the target user that the direction corresponding to the target user is the direction in which the terminal's movement is restricted and / or to prompt the target user to move.

[0084] For example, the restricted movement direction of the terminal includes directions where movement is prohibited due to chassis movement constraints, parking boundaries, or obstacles. This restricted movement direction is not limited to directly behind the terminal; it can also be any safe, prohibited direction such as the side adjacent to an obstacle, a blind spot with a very small turning radius, etc.

[0085] In some embodiments of this example, the controller determines the direction corresponding to the target user, and then performs a safe passage assessment based on the direction corresponding to the target user. If it is confirmed that the direction corresponding to the target user falls into a restricted movement direction such as a reversing no-entry zone, a very small turning blind spot, or a rear obstacle-enclosed area, the controller controls the speaker to output a second voice.

[0086] like Figure 5 The diagram illustrates an interactive scenario where, when a vehicle is perpendicularly parked close to the rear window, and the target user issues the first voice command, "Drive over here," from the blind spot directly behind the vehicle, the vehicle broadcasts a second voice message: "You are in a restricted area behind the vehicle and cannot reverse to approach. Please move to the left or right front of the vehicle to try again."

[0087] In this embodiment, by outputting voice prompts in a timely manner when the user's corresponding direction is a direction with restricted movement, the risk of collision caused by the vehicle being moved forcibly is avoided, and the user is guided to a safe and passable area, thereby improving the safety and user experience in the voice-controlled parking scenario.

[0088] In an optional embodiment, before the terminal is moved to the first location corresponding to the target user, a third voice message is output to indicate that the target user will be located within the recognition range.

[0089] In one embodiment of this invention, before issuing an execution command to the actuator, the controller issues an execution command to the speaker to output a third voice message, thereby controlling the speaker to play the third voice message to prompt the target user terminal to search for the target user within the recognition range.

[0090] like Figure 6 The diagram illustrates an interactive scenario where, after the target user issues the first voice command, "Drive over here," the vehicle's speaker immediately plays a third voice message: "Let me find you."

[0091] In this embodiment, by playing a search prompt voice before the movement is executed, the target user can know the current recognition and execution status of the vehicle in real time, thereby improving the clarity of interaction and user experience.

[0092] Furthermore, in this disclosure, when controlling the terminal to move towards the first location corresponding to the target user based on the first voice command of the target user, the controller determines the target user's action based on the first voice command of the target user; according to the target user's action, the location where the target user is located or the location indicated by the target user is determined as the first location, and the terminal is controlled to move towards the first location.

[0093] The location of the target user refers to the location where the target user is located, and the location indicated by the target user refers to the location that the target user points to through an action.

[0094] In one embodiment of this invention, the controller receives a first voice command, identifies the target user's actions through image data acquired by the visual acquisition component, matches the target user's actions with a preset action template to determine the target user's movement intention, and then determines the first position corresponding to the target user based on the movement intention. Subsequently, the controller issues an execution command to the actuator to control the terminal to move to the first position.

[0095] For example, the controller receives a first voice command from the target user, "Come here," and determines that there is a user within the recognition range making a "waving" gesture. The controller identifies the user making the "waving" gesture as the target user and determines that the target user's intention to move is to stop next to the target user. Subsequently, the controller uses the target user's location as the first location and controls the vehicle to move towards that first location.

[0096] like Figure 7 The diagram illustrates an interactive scenario where the target user issues the initial voice command "Come here" and then waves. The vehicle moves among multiple users to the target user who has made the waving gesture.

[0097] In another example, the controller receives the first voice command "Stop over there" from the target user and recognizes the target user's gesture indicating "point to the left parking space." The controller determines that the target user's intention is to park the vehicle in the parking space on the left. Subsequently, the controller uses this parking space location as the first position and controls the vehicle to move to and park in the parking space.

[0098] In this embodiment, by combining voice commands and user actions, the target user and their actual movement intentions can be accurately distinguished. This can identify the target user who initiates the voice command in a multi-person scenario, and also accurately identify the specific parking space indicated by the target user. This makes the terminal's voice control fit the actual usage scenario, improving positioning accuracy and interaction reliability.

[0099] In an optional embodiment, the target user's actions include at least one of the target user's limb movements, head movements, gestures, and facial movements.

[0100] For example, the body movements include, but are not limited to: waving, waving, leaning forward, turning around, and taking steps to indicate the direction.

[0101] The head movements include, but are not limited to: nodding, shaking the head, turning the head to a designated location, and raising the head to indicate something.

[0102] The gestures include, but are not limited to: pointing to a location, reaching out to beckon, gesturing with the palm to stop, and indicating a parking space area.

[0103] The facial movements include, but are not limited to: looking at the vehicle, turning the face toward the target, etc.

[0104] In some embodiments of this example, these actions are bound to preset action templates and movement operation intentions. After recognizing the target user's actions, the controller determines the target user's movement operation intention through this binding relationship. In one scenario, the target user is looking at the vehicle while waving and nodding in its direction. The controller matches this action with a preset action template, recognizes the intention of the movement as driving towards the user's location and stops, and controls the vehicle to perform the corresponding movement.

[0105] When the terminal is controlled to move to the second location indicated by the second voice command based on the second voice command of the target user, the controller controls the terminal to move to the second location corresponding to the address information based on the address information corresponding to the second voice command.

[0106] In one embodiment of this invention, the controller performs semantic parsing on the voice command. When it parses out that the voice command contains address-like content such as a location name and location description, it determines that the second voice command contains address information. Subsequently, the controller matches the address information with the built-in map or positioning data to obtain the corresponding second location coordinates, plans a movement route from the current location to the second location, and issues an execution command to the actuator according to the movement route, controlling the terminal to move to the second location corresponding to the address information.

[0107] In some scenarios, the target user issues a second voice command to the vehicle in the parking lot of the xxxx residential community, "Drive the car to the parking lot of the xxxx office building," and the vehicle then automatically drives to the parking lot of the xxxx office building.

[0108] In this embodiment, by semantically parsing the voice commands to extract address information and combining it with map matching and route planning, the terminal can directly drive to the specified address location based on the user's voice, without the need for manual operation or on-site guidance, thus improving the convenience and practicality of voice control in remote and non-line-of-sight scenarios.

[0109] In an optional embodiment, the controller outputs a fourth voice message based on the address information, the fourth voice message being used to determine the terminal's movement route information. In response to a confirmation message corresponding to the movement route information, the controller controls the terminal to move to the second location corresponding to the address information.

[0110] For example, the movement route information is navigation information from the current location to the target address, which includes, but is not limited to, movement route, estimated travel time, road conditions along the way, and parking space distribution at the destination.

[0111] In one embodiment of this invention, after the controller plans the movement route based on the address information, it controls the speaker to output a fourth voice message. This fourth voice message includes the movement route information. Upon receiving a confirmation voice message from the user, the controller sends an execution command to the actuator to control the terminal to move to the second position according to the movement route.

[0112] like Figure 8 The diagram illustrates an interactive scenario where the target user issues a second voice command: "Drive the car to the parking lot of the xxxx residential complex." The vehicle's speaker then plays a fourth voice message: "A route has been planned for you to the parking lot of the xxxx residential complex. The journey is approximately 1.5 kilometers and is expected to take 10 minutes. Do you confirm departure?" After the target user replies "Confirm," the vehicle automatically drives towards the parking lot of the xxxx residential complex according to the planned route.

[0113] In this embodiment, by pre-planning the movement route and confirming it via voice broadcast before executing the movement interaction, the safety and usage risks caused by direct execution due to address resolution errors or unreasonable routes are avoided. This allows the target user to check and confirm the movement information in advance, improving the reliability of voice-controlled navigation and the user's sense of security.

[0114] In this disclosure, when the terminal is controlled to move to the second position indicated by the second voice command based on the second voice command of the target user, the controller responds to the target identifier corresponding to the second voice command being within the recognition range of the terminal, and controls the terminal to move to the second position corresponding to the target identifier based on the target identifier.

[0115] For example, the target identifier is a reference object with a fixed location and significant visual features, including but not limited to reference objects such as walls, pillars, and signs, as well as information such as text, numbers, symbols, QR codes, parking space numbers, area identifiers, and garage floor identifiers on the reference objects.

[0116] In some embodiments of this example, the controller performs semantic parsing on the second voice command. When the parsed content includes object names, location markers, text numbers, symbol features, color features, etc., the controller determines that the second voice command contains a target identifier. Subsequently, the controller identifies the surrounding environment using image data acquired by the visual acquisition unit. After the controller identifies a reference target matching the target identifier and determines the location information of the reference target, it determines the second location corresponding to the target identifier and plans a movement route from the current location to the second location. Then, the controller issues an execution command to the actuator to move the terminal to the second location.

[0117] like Figure 9 The diagram illustrates an interactive scenario where a user in a parking lot issues a second voice command: "Drive to the pillar next to Zone D ahead." The vehicle identifies and locates the pillar marked "Zone D" using image data collected by the vision acquisition component, then moves to the pillar and parks.

[0118] In some scenarios, the target user issues a second voice command, "Drive to the blue road sign ahead," and the vehicle identifies the blue road sign with the corresponding mark by image data collected by the vision acquisition component, locates it, and then moves to the road sign and stops.

[0119] In this embodiment, by combining voice commands with visual identifier recognition, the terminal can locate the target location based on the features such as objects, text, and numbers described by the user, thereby improving the flexibility and practicality of voice control in complex indoor environments.

[0120] This disclosure provides an embodiment where, when executing a voice command from a target user outside the terminal's cockpit to control the terminal to move towards the target location corresponding to the voice command, the target location is first determined through voice inquiry before moving to the target location. In this embodiment, the controller outputs a fifth voice command based on the voice command from the target user outside the terminal's cockpit; and controls the terminal to move towards the target location corresponding to the response information of the fifth voice command. The fifth voice command is used to confirm the voice command.

[0121] In one embodiment of this invention, after receiving a voice command from a target user outside the cockpit, the controller first parses the voice command to generate a fifth voice message containing the information to be confirmed, and then controls the speaker to play the fifth voice message. Subsequently, the controller obtains the target user's response information, parses the response information to obtain the confirmed target location, and then issues an execution command to the actuator based on the confirmed target location to control the terminal to move towards the confirmed target location. The information to be confirmed refers to target user characteristics, target identification characteristics, address characteristics, etc., obtained from the voice command.

[0122] In some scenarios, the target user issues a voice command from outside the vehicle saying "Come here," and the vehicle plays a fifth voice prompt: "Okay, are you wearing a red shirt today?" After the target user replies "Yes," the vehicle identifies the user wearing red as the target user and moves towards their location.

[0123] In some scenarios, the target user issues a voice command to "drive to parking space 03 in zone B". The vehicle plays the fifth voice prompt: "We are about to proceed to parking space 03 in zone B. Do you confirm?" After the target user replies "yes", the vehicle moves to the corresponding parking space based on the address information.

[0124] In some scenarios, the target user issues a voice command to "drive to the yellow pillar ahead". The vehicle plays the fifth voice prompt: "The target location is next to the yellow pillar ahead. Do you confirm?" After the target user replies "yes", the vehicle recognizes the yellow pillar and moves to its location.

[0125] In this embodiment, by using voice inquiry for confirmation, combined with verification of target user characteristics, address information, target identifier characteristics and other confirmation items, the target user or target identifier can be accurately located, avoiding misidentification and misexecution, and improving the safety and reliability of external voice control of vehicle movement.

[0126] Furthermore, if the response information is a confirmation message of the fifth voice, the controller controls the terminal to move towards the target location corresponding to the voice command; if the response information is an update message of the target location, the controller controls the terminal to move towards the target location corresponding to the update message based on the update message.

[0127] In one embodiment of this invention, the controller performs semantic parsing on the received response information. If the parsed response information contains expressions such as affirmation, agreement, or confirmation, the controller determines that the response information is confirmation information for the fifth voice command. Subsequently, the controller controls the terminal to move towards the target location obtained from the previous parsing (i.e., the target location corresponding to the voice command).

[0128] In one embodiment of this example, the controller performs semantic parsing on the received response information. If the parsed response information contains expressions of correction, replacement, or adjustment, the controller determines that the response information is the updated information of the target location. Subsequently, the controller reconfirms the target location (that is, the target location corresponding to the response information) based on the updated location information and controls the terminal to move.

[0129] In some examples, through multiple rounds of questioning, the controller can interact with the target user multiple times to correct the target user characteristics or target location information until the target location is confirmed before controlling the terminal to move.

[0130] like Figure 10 The diagram illustrates an interactive scenario where a user outside the vehicle issues a voice command, "Come here." The vehicle then plays a fifth voice prompt: "Okay, are you wearing a red shirt?" The user replies, "No, I'm wearing a yellow shirt." After confirming the user's yellow shirt, the vehicle moves towards the user's location.

[0131] In some scenarios, the user issues a voice command to "drive to the yellow pillar ahead." The vehicle then plays a fifth voice prompt: "The target location is next to the second yellow pillar. Confirm?" The user replies, "No, it's the third." After confirming that the target location is the third yellow pillar, the vehicle moves towards that location.

[0132] In an optional embodiment, the controller outputs a sixth voice prompt when the terminal has sufficient space to move, and outputs a seventh voice prompt when the terminal has insufficient space to move. The sixth voice prompt is used to prompt the terminal to move to the target location, and the seventh voice prompt is used to prompt that the terminal cannot be moved and / or to prompt that the terminal needs to be taken over.

[0133] In one embodiment of this invention, the controller identifies and detects obstacles and passable areas around the terminal and along the planned movement route based on image data acquired by the vision acquisition component. If the route is unobstructed, without significant obstacles, and the passability meets the requirements for safe vehicle movement, the controller determines that the terminal has sufficient space to move. Subsequently, the controller sends an execution command to the speaker component to play the sixth voice message to alert the user that movement is imminent.

[0134] In some scenarios, the vehicle uses images captured by a camera to identify that there are no obstacles or narrow areas on its path to the target location, and then plays a sixth voice message: "Approaching the target location."

[0135] In one embodiment of this invention, the controller identifies and detects obstacles and passable areas around the terminal and on the planned movement route based on image data acquired by the vision acquisition component. If obstacles are detected on the planned movement route, the passage width is insufficient, or a fixed obstacle prevents safe passage, the controller determines that the terminal has insufficient movement space. Subsequently, the controller sends an execution command to the speaker component to play the seventh voice message to inform the user that movement is not permitted.

[0136] In some scenarios, if there are obstacles or narrow passageways on the vehicle's path as it moves to the target location based on images captured by the camera, the seventh voice prompt will play: "Insufficient space for movement; unable to move automatically. Please manually take over the vehicle."

[0137] In this embodiment, by real-time detection of the vehicle's movement space and outputting different prompt voices, the user is informed to move normally when the route is safe, and is promptly reminded that movement is impossible and manual intervention is suggested when space is insufficient. This effectively improves the safety and reliability of external voice control of vehicle movement and avoids movement risks caused by space constraints.

[0138] Combining the above embodiments and Figure 11 The diagram illustrates a control flow of a terminal control method provided in this disclosure. This control flow includes two scenarios: the vehicle is in a parking space and the vehicle is not in a parking space. The controller described in the above embodiments of this disclosure includes... Figure 11The document describes three modules: Electronic Identity (EID), Autonomous Driving (AD), and Visual Language Navigation (VLN). The AD module includes a Planning (PnC) submodule and an Agent. The VLN module can be deployed locally or in the cloud. Cloud-deployed VLN modules interact with local systems via a communication connection.

[0139] After receiving the voice command from the target user, the Agent triggers the following control flow: The S1.EID module confirms the source of the voice command.

[0140] If it is confirmed that the voice command originated from inside the cabin, then execute S2.

[0141] If it is confirmed that the voice command comes from outside the cabin, then execute S3.

[0142] The S2.EID module confirms the vehicle's driving mode.

[0143] If it is confirmed that the vehicle is currently in manual driving mode, the EID module will output a prompt voice message: "Please try driving assistance mode first."

[0144] If it is confirmed that the vehicle is currently in assisted driving mode, the Agent will output a voice prompt: "Such commands are not supported when the driver is inside the vehicle."

[0145] The S3.EID module confirms the source of the external voice commands.

[0146] If it is confirmed that the external voice command comes from the external audio acquisition unit, then execute S4.

[0147] If it is confirmed that the external voice command comes from a wireless communication device (such as smart glasses, smartphones, smartwatches, etc.), then execute S6.

[0148] The S4.AD module attempts to obtain the direction corresponding to the target user.

[0149] The AD module sends a request to the Bluetooth key to obtain the direction corresponding to the target user. If the direction corresponding to the target user is not obtained, the Agent outputs a voice prompt: "It seems we can't find where you are. Try again diagonally in front of the vehicle." If the direction corresponding to the target user is obtained, then S5 is executed.

[0150] The S5.AD module confirms the vehicle's current location.

[0151] If it is confirmed that the vehicle's current location is within the parking space, then execute S7.

[0152] If the vehicle's current position is outside the parking space, then execute S15.

[0153] The S6.AD module confirms whether the Bluetooth key is connected.

[0154] If the Bluetooth key is confirmed to be connected, proceed to step S4.

[0155] If it is confirmed that the Bluetooth key is not connected, the Agent will output a prompt voice message: "Please approach the vehicle to connect the Bluetooth key."

[0156] The S7.AD module confirms the target user's orientation relative to the vehicle.

[0157] If the target user is confirmed to be behind the vehicle, the Agent will output a voice prompt: "Please try again from diagonally in front of the vehicle."

[0158] If the target user is confirmed to be located on the left or right side of the vehicle, then execute S8.

[0159] The S8.AD module confirms whether the berthing conditions are met.

[0160] The parking conditions are the conditions that the vehicle must meet to activate the parking assist mode, including but not limited to: no system abnormalities in the vehicle, the parking slope meets the requirements, the gear shift mechanism is in the P gear and can be released, the vehicle is stationary, the vehicle is in a legal parking space, the rearview mirrors are fully extended, the doors / front hood / rear hood are fully closed, the target user's identity is verified as valid, and there is an executable parking route.

[0161] The parking assist mode refers to the autonomous driving control action performed by the vehicle after the parking assist function is activated, allowing it to park out of the parking space from the current parking space in a preset direction.

[0162] If it is confirmed that the vehicle meets the parking exit conditions, then execute S9.

[0163] If it is confirmed that the vehicle does not meet the parking exit conditions, the Agent will output a voice prompt: "The conditions for parking exit assist mode are not met at present, and the operation cannot be performed."

[0164] The S9.AD module confirms whether there is sufficient space for movement.

[0165] For example, determining whether the movement space is sufficient may include: verifying whether the vehicle's vertical space meets the preset width redundancy (e.g., 0.5 meters), the horizontal space meets the preset length redundancy (e.g., 0.7 meters), and whether there are no nearby obstacles obstructing the movement route.

[0166] If sufficient space is confirmed, then execute S10 and S12 respectively.

[0167] If it is confirmed that there is insufficient mobile space, the Agent will output a voice prompt: "Mobile space seems to be insufficient. Please try manual operation."

[0168] The S10.AD module notifies the PnC submodule to initiate docking, and the Agent outputs a voice prompt: "Okay, starting to move to your location."

[0169] The S11.PnC submodule controls vehicle parking.

[0170] The S12.VLN module performs the task of finding the target user.

[0171] If the target user is found, proceed to step S13.

[0172] If the target user is not found, the process will continue in a loop until the target user is found or a timeout occurs.

[0173] The S13.PnC submodule confirms whether the mode switching conditions are met.

[0174] The mode switching conditions are preset conditions that the vehicle must meet to switch to the target mode. The target mode includes either a roaming parking assist mode or a parking space-finding assist mode. The roaming parking assist mode refers to the vehicle autonomously planning its movement route based on the target user's location information and moving to the target user's location. The parking space-finding assist mode refers to the vehicle autonomously searching for available parking spaces within the parking lot and planning its parking route to complete the parking maneuver.

[0175] For example, the mode switching conditions include, but are not limited to, the vehicle having left the parking space, the vehicle being in a drivable state, and the target location being within the recognition range.

[0176] If the mode switching conditions are confirmed to be met, then execute S14.

[0177] If the mode switching condition is not met, the process continues in a loop until the vehicle is completely parked out of the parking space.

[0178] The S14.PnC submodule confirms whether the target location is reachable.

[0179] If the target location is reachable, then execute S16.

[0180] If the target location is unreachable, execute S11 to continue parking until the vehicle is completely out of the parking space.

[0181] The S15.VLN module performs the task of finding the target user.

[0182] If the target user is found, proceed to step S16.

[0183] If the target user is not found, the process will continue in a loop until the target user is found or a timeout occurs.

[0184] S16.AD module triggers query process.

[0185] The AD module generates query text based on the target user characteristics and query template identified by the VLN module, and outputs the query voice through the voice assistant and speaker.

[0186] If the user replies with confirmation or does not reply within a preset time, the target location is confirmed, and S17 is executed.

[0187] If the user replies with a negative response and clarification, the target location is updated based on the clarification, and S18 is executed.

[0188] The S17.PnC submodule controls vehicle movement based on the target location.

[0189] The S18.PnC submodule controls vehicle movement based on the updated target location.

[0190] Furthermore, such as Figure 12 The diagram shown illustrates an inquiry process, which includes: The S161.AD module confirms the characteristics of the target user.

[0191] For example, the AD module confirms the characteristics of the target user based on the visual features identified by the VLN module. For instance, it confirms that the target user is wearing a red shirt.

[0192] S162.Agent assembles the query voice text and triggers the first round of queries.

[0193] For example, the agent generates query text based on the target user's characteristics and query template, and controls the voice assistant and speaker to output the query voice to the target user. For example, the first round of queries might be triggered: "You're wearing a red top, right?"

[0194] S163.Agent attempts to obtain user response information.

[0195] If a user reply is received within the preset time, then execute S164.

[0196] If no user response is received within the preset time, proceed to step S169.

[0197] S164.Agent parses user reply information.

[0198] If the user's reply contains affirmative semantics (such as "yes" or "correct"), then the target user's characteristics are confirmed to be valid, and step S169 is executed.

[0199] If the user's reply contains negative and clarifying semantics (such as "No, it's yellow"), then execute S165.

[0200] If the user's reply contains negative semantics and is not accompanied by clarifying information (such as "incorrect" or "no"), then execute S166.

[0201] S165.Agent assembles the clarification confirmation voice text and triggers the clarification confirmation query.

[0202] For example, the Agent updates the target user's characteristics based on the clarifying semantics in the user's reply and generates corresponding clarifying voice confirmation text. For instance, if the user replies "No, it's yellow," the Agent updates the target user's characteristics to wearing a yellow shirt and outputs the query: "Okay, you're wearing a yellow shirt."

[0203] S166.Agent assembles the query voice text and triggers the next round of queries.

[0204] For example, the Agent regenerates the query text based on the user's response containing negative semantics without accompanying clarification, and controls the voice assistant and speaker to output the query text to the target user. For instance, it might ask again, "Tell me what color clothes you're wearing?"

[0205] S167.Agent attempts to retrieve the user's response information again.

[0206] If a user reply is received within the preset time, then execute S168.

[0207] If no user response is received within the preset time, proceed to step S169.

[0208] S168.Agent parses user reply information.

[0209] If the user's reply contains affirmative semantics (such as "yes") or declarative semantics (such as "yellow"), then execute S165.

[0210] If the user's reply contains negative semantics (such as "not correct"), then execute S166.

[0211] If the user's reply contains other content (such as "Please turn on the air conditioner for me"), then execute S169.

[0212] S169. End the inquiry process.

[0213] For example, if the inquiry process ends because the user's reply contains other content, the Agent parses the irrelevant content and distributes it to the corresponding in-vehicle function module to perform the response operation. For instance, when the user replies "Turn on the air conditioning for me," the Agent distributes the air conditioning control command to the in-vehicle air conditioning control module, which then performs the air conditioning turn-on action.

[0214] In some examples, embodiments of this application may include, but are not limited to, the following: Example 1: A terminal control method includes: receiving a voice command from a target user outside the cockpit of the terminal; and controlling the terminal to move towards the target location corresponding to the voice command based on the voice command.

[0215] Example 2: Controlling the terminal to move to the target location corresponding to the voice command includes: controlling the terminal to move to a first location corresponding to the target user based on a first voice command from the target user; and / or, controlling the terminal to move to a second location indicated by the second voice command based on a second voice command from the target user.

[0216] Example 3: Controlling the terminal to move to the first location corresponding to the target user based on the first voice command of the target user includes: responding to the target user being within the recognition range of the terminal, controlling the terminal to move to the first location based on the first voice command of the target user.

[0217] Example 4: The method further includes: in response to the target user being outside the recognition range of the terminal, outputting a first voice and / or controlling the terminal to move in the direction corresponding to the target user, wherein the first voice is used to prompt the target user to be outside the recognition range and / or prompt the target user to move.

[0218] Example 5: The method further includes: in response to the direction corresponding to the target user being a direction where the terminal's movement is restricted, outputting a second voice message; the second voice message is used to prompt the target user that the direction corresponding to the target user is a direction where the terminal's movement is restricted and / or to prompt the target user to move.

[0219] Example 6: Before controlling the terminal to move to the first location corresponding to the target user, the method further includes: outputting a third voice, the third voice being used to prompt that the target user will be searched within the recognition range.

[0220] Example 7: Controlling the terminal to move to the first location corresponding to the target user based on the first voice command of the target user includes: determining the target user's action based on the first voice command of the target user; determining the location where the target user is located or the location indicated by the target user as the first location based on the target user's action, and controlling the terminal to move to the first location.

[0221] Example 8: The target user's actions include at least one of the target user's limb movements, head movements, hand gestures, and facial movements.

[0222] Example 9: Controlling the terminal to move to the second location indicated by the second voice command based on the second voice command of the target user includes: controlling the terminal to move to the second location corresponding to the address information based on the address information corresponding to the second voice command.

[0223] Example 10: Controlling the terminal to move to the second location corresponding to the address information based on the address information corresponding to the second voice command includes: outputting a fourth voice based on the address information, the fourth voice being used to determine the terminal's movement route information; and controlling the terminal to move to the second location corresponding to the address information in response to the confirmation information corresponding to the movement route information.

[0224] Example 11: Controlling the terminal to move to the second position indicated by the second voice command based on the second voice command of the target user includes: responding to the target identifier corresponding to the second voice command being within the recognition range of the terminal, and controlling the terminal to move to the second position corresponding to the target identifier based on the target identifier.

[0225] Example 12: Controlling the terminal to move to the target location corresponding to the voice command based on the voice command of the target user outside the terminal cockpit includes: outputting a fifth voice command based on the voice command of the target user outside the terminal cockpit, the fifth voice command being used to confirm the inquiry based on the voice command; and controlling the terminal to move to the target location corresponding to the response information based on the response information of the fifth voice command.

[0226] Example 13: Controlling the terminal to move to the target location corresponding to the response information of the fifth voice message includes: when the response information is a confirmation message of the fifth voice message, controlling the terminal to move to the target location corresponding to the voice command; when the response information is an update message of the target location, controlling the terminal to move to the target location corresponding to the update message based on the update message.

[0227] Example 14: The method further includes: when the terminal has sufficient space to move, outputting a sixth voice message, the sixth voice message being used to prompt the terminal to move to the target location; when the terminal has insufficient space to move, outputting a seventh voice message, the seventh voice message being used to prompt that the terminal cannot be moved and / or to prompt that the terminal needs to be taken over.

[0228] Example 15: The terminal includes a vehicle or a robot.

[0229] It should be noted that the terminal control method involved in the above embodiments may include at least one of Embodiments 1 to 15. For example, Embodiment 1 can be implemented as an independent embodiment, Embodiment 1+2 can be implemented as an independent embodiment, Embodiment 1+2+3 can be implemented as an independent embodiment, Embodiment 1+2+4 can be implemented as an independent embodiment, Embodiment 1+2+4+5 can be implemented as an independent embodiment, Embodiment 1+2+6 can be implemented as an independent embodiment, Embodiment 1+2+3+6 can be implemented as an independent embodiment, Embodiment 1+2+4+6 can be implemented as an independent embodiment, Embodiment 1+2+4+5+6 can be implemented as an independent embodiment, Embodiment 1+2+7 can be implemented as an independent embodiment, and Embodiment 1+2+3+7 can be implemented as an independent embodiment. The following embodiments can be implemented as independent embodiments: Embodiment 1+2+4+7, Embodiment 1+2+4+5+7, Embodiment 1+2+4+5+6+7, Embodiment 1+2+7+8, Embodiment 1+2+3+7+8, Embodiment 1+2+4+7+8, Embodiment 1+2+4+5+7+8, Embodiment 1+2+4+5+6+7+8, Embodiment 1+2+9, Embodiment 1+2+3. +9 can be implemented as an independent embodiment; Embodiment 1+2+4+9 can be implemented as an independent embodiment; Embodiment 1+2+4+5+9 can be implemented as an independent embodiment; Embodiment 1+2+4+5+6+9 can be implemented as an independent embodiment; Embodiment 1+2+4+5+6+7+9 can be implemented as an independent embodiment; Embodiment 1+2+4+5+6+7+8+9 can be implemented as an independent embodiment; Embodiment 1+2+9+10 can be implemented as an independent embodiment; Embodiment 1+2+3+9+10 can be implemented as an independent embodiment; Embodiment 1+2+4+9+10 can be implemented as an independent embodiment; Embodiment 1+2+4+5+ Examples 9+10, 1+2+4+5+6+9+10, 1+2+4+5+6+7+9+10, 1+2+4+5+6+7+8+9+10, 1+2+11, 1+2+3+11, 1+2+4+11, 1+2+4+5+11, 1+2+4+5+11, and 1+2+4+5+6+11 can all be implemented as independent embodiments.Examples 1+2+4+5+6+7+11 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+11 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+9+11 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+9+10+11 can be implemented as independent embodiments; Examples 1+12 can be implemented as independent embodiments; Examples 1+2+3+12 can be implemented as independent embodiments; Examples 1+2+4+12 can be implemented as independent embodiments; Examples 1+2+4+5+12 can be implemented as independent embodiments. Examples can be implemented as independent embodiments; Examples 1+2+4+5+6+12 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+12 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+12 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+9+12 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+9+10+11+12 can be implemented as independent embodiments; Examples 1+12+13 can be implemented as independent embodiments; Examples 1+2+12+13 can... Examples 1+2+3+12+13, 1+2+4+12+13, 1+2+4+5+12+13, 1+2+4+5+6+12+13, 1+2+4+5+6+7+12+13, 1+2+4+5+6+7+8+12+13, 1+2+4+5+6+7+8+9+12+13 can be implemented as independent embodiments, and examples 1+2+4+5+6+12+13 can be implemented as independent embodiments. Examples 7+8+9+10+12+13 can be implemented as independent embodiments; examples 1+2+4+5+6+7+8+9+10+11+12+13 can be implemented as independent embodiments; examples 1+14 can be implemented as independent embodiments; examples 1+2+14 can be implemented as independent embodiments; examples 1+2+3+14 can be implemented as independent embodiments; examples 1+2+4+14 can be implemented as independent embodiments; examples 1+2+4+5+14 can be implemented as independent embodiments; examples 1+2+4+5+6+14 can be implemented as independent embodiments; examples 1+2+4+5+6+7+14 can be implemented as independent embodiments.Examples 1+2+4+5+6+7+8+14 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+9+14 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+9+10+11+14 can be implemented as independent embodiments; Examples 1+2+4+5+6+7+8+9+10+11+12+14 can be implemented as independent embodiments. Examples 1+2+4+5+6+7+8+9+10+11+12+13+14 can be implemented as independent embodiments, Examples 1+15 can be implemented as independent embodiments, Examples 1+2+15 can be implemented as independent embodiments, Examples 1+2+3+15 can be implemented as independent embodiments, Examples 1+2+4+15 can be implemented as independent embodiments, Examples 1+2+4+5+15 can be implemented as independent embodiments, Example 1 +2+4+5+6+15 can be implemented as an independent embodiment; Embodiment 1+2+4+5+6+7+15 can be implemented as an independent embodiment; Embodiment 1+2+4+5+6+7+8+15 can be implemented as an independent embodiment; Embodiment 1+2+4+5+6+7+8+9+15 can be implemented as an independent embodiment; Embodiment 1+2+4+5+6+7+8+9+10+15 can be implemented as an independent embodiment; Embodiment 1+2+4+5+6+7+8+9+ Examples 10+11+15 can be implemented as independent embodiments; examples 1+2+4+5+6+7+8+9+10+11+12+15 can be implemented as independent embodiments; examples 1+2+4+5+6+7+8+9+10+11+12+13+15 can be implemented as independent embodiments; examples 1+2+4+5+6+7+8+9+10+11+12+13+14+15 can be implemented as independent embodiments, and so on, but are not limited to these, and will not be listed one by one here.

[0230] Compared with related technologies, this embodiment enriches the interactive control methods of the terminal by controlling the terminal to move to a designated target location through voice commands. Furthermore, voice command control aligns with the natural human-computer interaction needs, thus improving the convenience and flexibility of terminal interactive control.

[0231] Based on this understanding, this disclosure also provides a terminal control device, such as... Figure 13 The diagram shows the structure of a terminal control device 130, which includes a receiving module 131 and a control module 132. The receiving module 131 is configured to receive voice commands from a target user outside the cockpit of the terminal. The control module 132 is configured to control the terminal to move to the target location corresponding to the voice command based on the voice command.

[0232] Optionally, the control module 132 is configured to control the terminal to move to a first position corresponding to the target user based on a first voice command from the target user; and / or, to control the terminal to move to a second position indicated by the second voice command based on a second voice command from the target user.

[0233] Optionally, the control module 132 is configured to control the terminal to move toward the first position based on the first voice command of the target user, in response to the target user being within the recognition range of the terminal.

[0234] Optionally, the control module 132 is configured to output a first voice and / or control the terminal to move in the direction corresponding to the target user in response to the target user being outside the recognition range of the terminal, wherein the first voice is used to prompt the target user to move outside the recognition range and / or prompt the target user to move.

[0235] Optionally, the control module 132 is configured to output a second voice message in response to the direction corresponding to the target user being a direction that restricts the movement of the terminal; the second voice message is used to prompt the target user that the direction corresponding to the target user is a direction that restricts the movement of the terminal and / or to prompt the target user to move.

[0236] Optionally, the control module 132 is configured to, before controlling the terminal to move toward the first location corresponding to the target user, further include: outputting a third voice message to prompt that the target user will be located within the recognition range.

[0237] Optionally, the control module 132 is configured to determine the target user's action based on the target user's first voice command; determine the target user's location or the location indicated by the target user as the first location based on the target user's action, and control the terminal to move towards the first location.

[0238] Optionally, the target user's actions include at least one of the target user's body movements, head movements, hand gestures, and facial movements.

[0239] Optionally, the control module 132 is configured to control the terminal to move to the second location corresponding to the address information based on the address information corresponding to the second voice command.

[0240] Optionally, the control module 132 is configured to output a fourth voice based on the address information, the fourth voice being used to determine the terminal's movement route information; and in response to a confirmation message corresponding to the movement route information, to control the terminal to move to the second location corresponding to the address information.

[0241] Optionally, the control module 132 is configured to, in response to the target identifier corresponding to the second voice command being located within the recognition range of the terminal, control the terminal to move to the second position corresponding to the target identifier based on the target identifier.

[0242] Optionally, the control module 132 is configured to output a fifth voice based on the voice command of the target user outside the terminal cockpit, the fifth voice being used to confirm the inquiry based on the voice command; and to control the terminal to move to the target location corresponding to the response information based on the response information of the fifth voice.

[0243] Optionally, the control module 132 is configured to, when the reply information is confirmation information of the fifth voice, control the terminal to move to the target location corresponding to the voice command; and when the reply information is update information of the target location, control the terminal to move to the target location corresponding to the update information based on the update information.

[0244] Optionally, the control module 132 is configured to output a sixth voice prompt when the terminal has sufficient space to move, the sixth voice prompt being used to prompt the terminal to move to the target location; and to output a seventh voice prompt when the terminal has insufficient space to move, the seventh voice prompt being used to prompt that the terminal cannot be moved and / or to prompt that the terminal needs to be taken over.

[0245] Optionally, the terminal may include a vehicle or a robot.

[0246] It should be noted that other corresponding descriptions of the functional units involved in the terminal control device provided in this embodiment can be found in [reference]. Figure 1 The corresponding description in [the document] will not be repeated here.

[0247] Based on the above, Figure 1 Correspondingly, this embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned terminal control method. Figure 1 The terminal control method shown.

[0248] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods of various implementation scenarios of this application.

[0249] Based on the above, Figure 1 The terminal control method shown, and Figure 13 To achieve the above objectives, this application also provides an electronic device, which can be configured on a vehicle or robot, etc., as shown in the virtual device embodiment. The device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 1 The terminal control method shown.

[0250] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0251] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0252] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0253] Furthermore, based on the above... Figure 1 The terminal control method shown, and Figure 13 The virtual device embodiment shown, as well as the above-described electronic device embodiment, this embodiment also provides a vehicle or robot that may include the above-described electronic device, thereby enabling it to perform actions such as Figure 1 The terminal control method shown.

[0254] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the solution of this embodiment, compared with the current related technologies, this application enriches the interactive control forms of the terminal by controlling the terminal to move to a designated target location through voice commands, and the voice command control conforms to the natural human-computer interaction needs, thus improving the convenience and flexibility of terminal interactive control.

[0255] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0256] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

[0257] It should be noted that the technical solutions in this disclosure are not limited to user terminals, but can also be extended to related applications of the same type that require control. All of these should fall within the protection scope of this disclosure, and no specific limitations are made here regarding the related applications that require control.

[0258] All articles and references disclosed above, including patent applications and publications, are incorporated herein by reference for various purposes. The term “substantially constitutes…” used to describe a combination should include the identified elements, components, parts, or steps, as well as other elements, components, parts, or steps that do not substantially affect the essential novelty of the combination. The use of the terms “comprising” or “including” to describe combinations of elements, components, parts, or steps herein also contemplates embodiments substantially constituted by such elements, components, parts, or steps. The use of the term “may” herein is intended to indicate that any described attribute included by “may” is optional.

[0259] Multiple elements, components, parts, or steps can be provided by a single integrated element, component, part, or step. Alternatively, a single integrated element, component, part, or step can be divided into multiple separate elements, components, parts, or steps. The use of "a" or "an" to describe an element, component, part, or step does not imply the exclusion of other elements, components, parts, or steps.

[0260] It should be understood that the above description is for illustrative purposes and not for limitation. Many embodiments and applications beyond the provided examples will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this teaching should not be determined by reference to the above description, but rather by reference to the foregoing claims and the full scope of their equivalents. For purposes of completeness, all articles and references, including patent applications and published disclosures, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the foregoing claims is not intended as a waiver of that subject matter, nor should it be considered as a failure of the applicant to consider that subject matter as part of the disclosed subject matter. It will be apparent to those skilled in the art that various modifications and variations can be made to this disclosure without departing from its spirit and scope. Thus, this disclosure is also intended to include such modifications and variations if they fall within the scope of the claims of this disclosure and their equivalents.

Claims

1. A terminal control method, characterized in that, include: Receives voice commands from the target user outside the cockpit of the terminal; Based on the voice command, the terminal is controlled to move towards the target location corresponding to the voice command.

2. The method according to claim 1, characterized in that, Controlling the terminal to move to the target location corresponding to the voice command includes: Based on the first voice command of the target user, control the terminal to move towards the first location corresponding to the target user; and / or, Based on the target user's second voice command, the terminal is controlled to move to the second position indicated by the second voice command.

3. The method according to claim 2, characterized in that, The step of controlling the terminal to move towards the first location corresponding to the target user based on the first voice command of the target user includes: In response to the target user being within the recognition range of the terminal, the terminal is controlled to move towards the first position based on the target user's first voice command.

4. The method according to claim 2 or 3, characterized in that, The method further includes: In response to the target user being outside the recognition range of the terminal, a first voice message is output and / or the terminal is controlled to move in the direction corresponding to the target user. The first voice message is used to prompt the target user that the target user is outside the recognition range and / or to prompt the target user to move.

5. The method according to claim 4, characterized in that, The method further includes: In response to the fact that the direction corresponding to the target user is the direction in which the terminal's movement is restricted, a second voice message is output; the second voice message is used to prompt the target user that the direction corresponding to the target user is the direction in which the terminal's movement is restricted and / or to prompt the target user to move.

6. The method according to any one of claims 2-5, characterized in that, Before controlling the terminal to move to the first location corresponding to the target user, the method further includes: A third voice message is output, which is used to prompt that the target user will be located within the recognition range.

7. The method according to any one of claims 2-6, characterized in that, The step of controlling the terminal to move towards the first location corresponding to the target user based on the first voice command of the target user includes: The target user's action is determined based on the target user's first voice command; Based on the target user's actions, the location where the target user is located or the location indicated by the target user is determined as the first location, and the terminal is controlled to move towards the first location.

8. The method according to claim 7, characterized in that, The target user's actions include at least one of the target user's body movements, head movements, hand gestures, and facial movements.

9. The method according to any one of claims 2-8, characterized in that, The step of controlling the terminal to move to the second location indicated by the second voice command based on the target user's second voice command includes: Based on the address information corresponding to the second voice command, the terminal is controlled to move to the second location corresponding to the address information.

10. The method according to claim 9, characterized in that, The step of controlling the terminal to move to the second location corresponding to the address information based on the address information corresponding to the second voice command includes: Based on the address information, a fourth voice message is output, which is used to determine the terminal's movement route information; In response to the confirmation information corresponding to the movement route information, the terminal is controlled to move to the second location corresponding to the address information.

11. The method according to any one of claims 2-10, characterized in that, The step of controlling the terminal to move to the second location indicated by the second voice command based on the target user's second voice command includes: In response to the target identifier corresponding to the second voice command being within the recognition range of the terminal, the terminal is controlled to move to the second position corresponding to the target identifier based on the target identifier.

12. The method according to any one of claims 1-11, characterized in that, The method of controlling the terminal to move towards the target location corresponding to the voice command based on the voice command of the target user outside the terminal cockpit includes: Based on the voice command of the target user outside the terminal cockpit, a fifth voice is output, which is used to perform a confirmation inquiry based on the voice command. Based on the response information from the fifth voice message, the terminal is controlled to move towards the target location corresponding to the response information.

13. The method according to claim 12, characterized in that, The step of controlling the terminal to move towards the target location corresponding to the response information based on the fifth voice message includes: If the response information is a confirmation message from the fifth voice, the terminal is controlled to move towards the target location corresponding to the voice command. If the response information is an update of the target location, the terminal is controlled to move towards the target location corresponding to the update information.

14. The method according to any one of claims 1-13, characterized in that, The method further includes: When the terminal has sufficient space to move, a sixth voice message is output, which is used to prompt the terminal to move to the target location; If the terminal has insufficient space to move, a seventh voice message is output; the seventh voice message is used to indicate that the terminal cannot be moved and / or to indicate that the terminal needs to be taken over.

15. The method according to any one of claims 1-14, characterized in that, The terminal may include a vehicle or a robot.

16. A terminal control device, characterized in that, The terminal control device includes a receiving module and a control module; The receiving module is configured to receive voice commands from a target user outside the cockpit of the terminal. The control module is configured to control the terminal to move towards the target location corresponding to the voice command based on the voice command.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 15.

18. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 15.

19. A vehicle or robot, characterized in that, include: The terminal control device as claimed in claim 16, or the computer-readable storage medium as claimed in claim 17, or the electronic device as claimed in claim 18.

Images