Interaction method and system of vehicle-mounted robot

Abstract

The present application discloses an interactive method and system for a vehicle-mounted robot, which relates to the field of intelligent automobile technology and can realize the real-time interactive function of the vehicle-mounted robot, thereby improving the intelligence of the vehicle cockpit. The specific scheme includes: after the vehicle controller receives the target instruction, it detects whether the vehicle controller includes the target control algorithm corresponding to the target instruction. If it does, it sends a target data request instruction to the vehicle-mounted robot, and the target data request instruction is used to request the real-time position of the target object required by the target control algorithm; after receiving the target data request instruction, the vehicle-mounted robot obtains the real-time position of the target object based on the image recognition method, and sends the real-time position of the target object to the vehicle controller; the vehicle controller generates a first control instruction according to the real-time position of the target object and the target control algorithm, and the first control instruction is used to control the mechanical arm of the vehicle-mounted robot to perform the operation corresponding to the target instruction.

Description

Technical Field

[0001] The present application relates to the field of intelligent automobile technology, and in particular to an interaction method and system for a vehicle-mounted robot. Background Art

[0002] As vehicle functions increase, the composition of the vehicle's intelligent cockpit system has also been upgraded from single-function electrical components to a complex electronic equipment system. At the same time, people's requirements for the intelligence of vehicles are getting higher and higher. Therefore, installing an on-board robot in the vehicle can improve the intelligence of the vehicle.

[0003] Currently, traditional industrial robots are mainly developed to perform precise tasks in highly controlled manufacturing environments. Time-consuming registration and calibration work is usually required before the robots can perform tasks. Such robots do not have the real-time interactive capabilities required in the vehicle's smart cockpit. Summary of the invention

[0004] The present application provides an interaction method and system for a vehicle-mounted robot, which can realize the real-time interaction function of the vehicle-mounted robot, thereby improving the intelligence of the vehicle cabin.

[0005] In a first aspect of an embodiment of the present application, a method for interacting with a vehicle-mounted robot is provided, the method comprising:

[0006] After receiving the target instruction, the vehicle controller detects whether the vehicle controller includes the target control algorithm corresponding to the target instruction. If it does, it sends a target data request instruction to the on-board robot, where the target data request instruction is used to request the real-time position of the target object required by the target control algorithm;

[0007] After receiving the target data request instruction, the vehicle-mounted robot obtains the real-time position of the target object based on the image recognition method, and sends the real-time position of the target object to the vehicle controller;

[0008] After receiving the real-time position of the target object, the vehicle controller generates a first control instruction according to the real-time position of the target object and the target control algorithm. The first control instruction is used to control the mechanical arm of the vehicle-mounted robot to perform the operation corresponding to the target instruction.

[0009] In one embodiment, before the vehicle controller receives the target instruction, the method further includes:

[0010] The cloud server pre-trains a plurality of control algorithms corresponding to preset instructions, and sends the trained plurality of control algorithms to the vehicle controller;

[0011] Detecting whether the vehicle controller includes a target control algorithm corresponding to the target instruction includes: detecting whether a plurality of trained control algorithms include the target control algorithm corresponding to the target instruction.

[0012] In one embodiment, the cloud server pre-trains a plurality of control algorithms corresponding to preset instructions, and sends the trained plurality of control algorithms to the vehicle controller, including:

[0013] The cloud server deploys the control algorithms corresponding to the multiple preset instructions in multiple fog servers;

[0014] Each fog server trains the control algorithm of the corresponding preset instruction and sends the trained multiple control algorithms to the vehicle controller.

[0015] In one embodiment, after detecting whether the vehicle controller includes a target control algorithm corresponding to the target instruction, the method further includes:

[0016] If the vehicle controller does not include the target control algorithm corresponding to the target instruction, a cockpit environment image acquisition instruction is sent to the on-board robot, and the cockpit environment image returned by the on-board robot is received;

[0017] The vehicle controller sends the cabin environment image and target instructions to multiple fog servers;

[0018] The target fog server extracts the real-time position of the target object required by the target control algorithm from the cockpit environment image, generates a second control instruction according to the real-time position of the target object and the target control algorithm, and sends the second control instruction to the vehicle controller. The target fog server is a fog server including the target control algorithm.

[0019] The on-board controller controls the mechanical arm of the on-board robot to execute the operation corresponding to the target instruction according to the second control instruction.

[0020] In one embodiment, after generating the control instruction according to the real-time position of the target object and the target control algorithm, the method further includes:

[0021] When the vehicle-mounted robot executes the operation corresponding to the target instruction, the position, speed and torque of the robotic arm are collected in real time, and the position, speed and torque of the robotic arm collected in real time are sent to the vehicle controller;

[0022] The vehicle controller modifies the target control algorithm based on the position of the robot arm, the speed of the robot arm, and the torque of the robot arm.

[0023] In one embodiment, the vehicle controller receiving the target instruction includes:

[0024] Receive target control instructions input by a user in the vehicle cabin, or receive target control instructions sent by a terminal device, or receive target control instructions sent by a user based on a VR device.

[0025] In one embodiment, the vehicle-mounted robot obtains the real-time position of the target object based on an image recognition method, including:

[0026] The on-board robot uses an image recognition algorithm based on visual positioning markers to determine the real-time location of the target object.

[0027] In a second aspect of an embodiment of the present application, there is provided a vehicle-mounted robot interaction system, the system comprising: a vehicle controller and a vehicle-mounted robot;

[0028] The vehicle controller is used to detect whether the vehicle controller includes a target control algorithm corresponding to the target instruction after receiving the target instruction, and if so, send a target data request instruction to the vehicle-mounted robot, the target data request instruction is used to request the real-time position of the target object required by the target control algorithm;

[0029] The vehicle-mounted robot is used to obtain the real-time position of the target object based on the image recognition method after receiving the target data request instruction, and send the real-time position of the target object to the vehicle controller;

[0030] The vehicle controller is also used to generate a first control instruction according to the real-time position of the target object and the target control algorithm after receiving the real-time position of the target object. The first control instruction is used to control the mechanical arm of the vehicle-mounted robot to perform the operation corresponding to the target instruction.

[0031] In one embodiment, the system further comprises: a cloud server;

[0032] A cloud server, used for pre-training control algorithms corresponding to a plurality of preset instructions, and sending the trained control algorithms to a vehicle controller;

[0033] The vehicle controller is specifically used to detect whether the trained multiple control algorithms include a target control algorithm corresponding to the target instruction.

[0034] In one embodiment, the system further comprises: a plurality of fog servers;

[0035] The cloud server is also used to deploy control algorithms corresponding to multiple preset instructions in multiple fog servers;

[0036] Each fog server is used to train the control algorithm of the corresponding preset instruction and send the trained multiple control algorithms to the vehicle controller.

[0037] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0038] The embodiment of the present application provides an interactive method for a vehicle-mounted robot. After receiving a target instruction through a vehicle controller, it is detected whether the vehicle controller includes a target control algorithm corresponding to the target instruction. If it is included, a target data request instruction is sent to the vehicle-mounted robot, wherein the target data request instruction is used to request the real-time position of the target object required by the target control algorithm. Then, after receiving the target data request instruction, the vehicle-mounted robot obtains the real-time position of the target object based on an image recognition method, and sends the real-time position of the target object to the vehicle controller. Finally, after receiving the real-time position of the target object, the vehicle controller generates a first control instruction according to the real-time position of the target object and the target control algorithm, and the first control instruction is used to control the mechanical arm of the vehicle-mounted robot to perform the operation corresponding to the target instruction. The interactive process of the vehicle-mounted robot in the present application adopts image-based visual control, and does not require the robot to perform time-consuming registration and calibration work, and can realize the real-time interactive function required in the vehicle cockpit. At the same time, by deploying the control process of the vehicle-mounted robot in the vehicle controller, the vehicle-mounted robot does not need to have high computing power requirements, which can reduce the equipment requirements of the vehicle-mounted robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A process of an interactive method of a vehicle-mounted robot provided in an embodiment of the present application Figure 1 ;

[0040] Figure 2 A process of an interactive method of a vehicle-mounted robot provided in an embodiment of the present application Figure 2 ;

[0041] Figure 3 A structural diagram of an interactive system for a vehicle-mounted robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0044] Additionally, the use of “based on” or “according to” is meant to be open and inclusive, as a process, step, calculation, or other action “based on” or “according to” one or more conditions or values ​​may, in practice, be based on additional conditions or beyond values.

[0045] As vehicle functions increase, the composition of the vehicle's intelligent cockpit system has also been upgraded from single-function electrical components to a complex electronic equipment system. At the same time, people's requirements for the intelligence of vehicles are getting higher and higher. Therefore, installing an on-board robot in the vehicle can improve the intelligence of the vehicle.

[0046] Currently, traditional industrial robots are mainly developed to perform precise tasks in highly controlled manufacturing environments. Time-consuming registration and calibration work is usually required before the robots can perform tasks. Such robots do not have the real-time interactive capabilities required in the vehicle's smart cockpit.

[0047] In order to solve the above problems, the embodiment of the present application provides an interactive method for a vehicle-mounted robot. The interactive process of the vehicle-mounted robot in the present application adopts image-based visual control, which does not require time-consuming registration and calibration of the robot, and can realize the real-time interactive function required in the vehicle cockpit. At the same time, by deploying the control process of the vehicle-mounted robot in the vehicle controller, the vehicle-mounted robot does not need to have high computing power requirements, which can reduce the equipment requirements of the vehicle-mounted robot.

[0048] The vehicle-mounted robot interaction method provided in the embodiment of the present application is applied to the vehicle-mounted robot interaction system, which includes: a cloud server, a vehicle controller and a vehicle-mounted robot, wherein the cloud server and the vehicle-mounted robot are respectively connected to the vehicle controller for communication. The vehicle-mounted robot includes a retractable mechanical arm, and the vehicle-mounted robot includes a plurality of control algorithms for controlling the vehicle-mounted robot, and these control algorithms are trained control algorithms, wherein one type of control instruction can correspond to one control algorithm. The cloud server will train a plurality of control algorithms in advance, and deploy the trained control algorithms in the vehicle controller.

[0049] like Figure 1 As shown, an embodiment of the present application provides an interaction method of a vehicle-mounted robot, the method comprising the following steps:

[0050] Step 101: After receiving the target instruction, the vehicle controller detects whether the vehicle controller includes the target control algorithm corresponding to the target instruction. If so, a target data request instruction is sent to the vehicle-mounted robot.

[0051] Among them, the vehicle controller can be an on-board smart cockpit domain controller, and the on-board smart cockpit domain controller can be an RCU controller.

[0052] The vehicle-mounted robot includes arms and fingers with multiple degrees of freedom in terms of hardware structure. The head position and front end position of each arm of the vehicle-mounted robot include image acquisition devices, which can be RGB / TOF cameras. In addition, the finger ends of the robotic arms are equipped with touch sensors. The vehicle-mounted robot also includes a processing device for processing processes such as determining the position of the target object.

[0053] The target data request instruction is used to request the real-time position of the target object required by the target control algorithm. For example, if the target instruction is "I want to drink water", the target control algorithm corresponding to the target instruction is to control the robot arm of the vehicle robot to perform the operations of picking up the cup, unscrewing the cover, and handing the cup to the user. The position of the target object includes the position of the cup and the position of the user.

[0054] It can be understood that the target instruction here is an instruction that needs to be executed by the on-board robot. Since the vehicle controller needs to control the entire smart cockpit, after receiving the user's instruction, it is necessary to first determine whether the instruction is a target instruction. If the instruction is a target instruction, the subsequent process is executed.

[0055] Optionally, the process of the vehicle controller receiving the command may be: identifying the voice command of the user in the vehicle cabin, and if the voice command is a command that needs to be executed by the vehicle robot, then the voice command is the target command. It is also possible to detect whether the command needs to be executed by the vehicle robot by identifying the gesture command or action command of the user in the vehicle cabin; it is also possible to receive the command sent by the mobile terminal and detect whether the command is the target command; it is also possible to receive the command sent by the user based on the VR device and detect whether the command is the target command.

[0056] Step 102 : After receiving the target data request instruction, the vehicle-mounted robot obtains the real-time position of the target object based on an image recognition method, and sends the real-time position of the target object to the vehicle controller.

[0057] In the actual execution process, after receiving the target data request instruction, the vehicle-mounted robot collects the environment image in the vehicle cabin, and recognizes and detects the environment image in the vehicle cabin, thereby determining the real-time position of the target object. Alternatively, after receiving the target data request instruction, the vehicle-mounted robot collects the image of the target area in the vehicle cabin, and recognizes and detects the image of the target area, thereby determining the real-time position of the target object.

[0058] Among them, the target area is the conventional placement area of ​​the target object. For example, if the target instruction is "I want to drink water", then the target area is the conventional placement area of ​​the "cup". These areas can be personalized in advance, and the embodiments of the present application do not make specific limitations on this.

[0059] For example, the in-vehicle robot can also perform actions such as "handing things over", "unscrewing bottle caps", "massaging drivers and passengers", and "patting children to sleep".

[0060] Optionally, the above process of identifying and detecting the environment image in the vehicle cabin, or identifying and detecting the image of the target area may be:

[0061] The vehicle-mounted robot can use an image recognition algorithm based on visual positioning markers to determine the real-time position of the target object, or the vehicle-mounted robot can use a CNN image recognition algorithm to determine the real-time position of the target.

[0062] The image recognition algorithm based on the visual positioning marker may be an Apriltag recognition algorithm. As a visual positioning marker, Apriltag is similar to a QR code or a barcode, and can achieve high-precision and high-robust positioning through a specific algorithm, and uniquely identify through a specific encoding and decoding system.

[0063] Step 103: After receiving the real-time position of the target object, the vehicle controller generates a first control instruction according to the real-time position of the target object and the target control algorithm. The first control instruction is used to control the mechanical arm of the vehicle-mounted robot to perform the operation corresponding to the target instruction.

[0064] The first control instruction includes a control instruction for a motion path of the robot arm, a control instruction for a moving speed of the robot arm, and a control instruction for a grasping torque of the robot arm.

[0065] It can be understood that after receiving the real-time position of the target object, the vehicle controller inputs the unreplaceable real-time position of the target into the target control algorithm, obtains the first control instruction, and sends the first control instruction to the on-board robot. The on-board robot controls the motion path of the robotic arm, the moving speed of the robotic arm and the grasping torque of the robotic arm to make the robotic arm perform the operation corresponding to the target instruction.

[0066] The embodiment of the present application provides an interactive method for a vehicle-mounted robot. After receiving a target instruction through a vehicle controller, it is detected whether the vehicle controller includes a target control algorithm corresponding to the target instruction. If it is included, a target data request instruction is sent to the vehicle-mounted robot, wherein the target data request instruction is used to request the real-time position of the target object required by the target control algorithm. Then, after receiving the target data request instruction, the vehicle-mounted robot obtains the real-time position of the target object based on an image recognition method, and sends the real-time position of the target object to the vehicle controller. Finally, after receiving the real-time position of the target object, the vehicle controller generates a first control instruction according to the real-time position of the target object and the target control algorithm, and the first control instruction is used to control the mechanical arm of the vehicle-mounted robot to perform the operation corresponding to the target instruction. The interactive process of the vehicle-mounted robot in the present application adopts image-based visual control, and does not require the robot to perform time-consuming registration and calibration work, and can realize the real-time interactive function required in the vehicle cockpit. At the same time, by deploying the control process of the vehicle-mounted robot in the vehicle controller, the vehicle-mounted robot does not need to have high computing power requirements, which can reduce the equipment requirements of the vehicle-mounted robot.

[0067] Optionally, before the vehicle controller receives the target instruction in the above step 101, the method further includes:

[0068] The cloud server pre-trains a plurality of control algorithms corresponding to preset instructions, and sends the trained plurality of control algorithms to the vehicle controller;

[0069] Correspondingly, the process of detecting whether the vehicle controller includes the target control algorithm corresponding to the target instruction in the above step 101 may be: detecting whether the trained multiple control algorithms include the target control algorithm corresponding to the target instruction.

[0070] It is understandable that when the on-board robot performs operations corresponding to target instructions, it is necessary to first train multiple control algorithms, where one type of instruction corresponds to one control algorithm. Therefore, before interaction, multiple control algorithms can be pre-trained in the cloud server, and then the trained multiple control algorithms can be sent down and deployed in the vehicle controller. This not only can utilize the large amount of databases and processing resources in the cloud server to improve the algorithm training efficiency, but also can realize data sharing and collective learning. At the same time, the control algorithm is sent down and deployed in the vehicle controller. In this way, during the control process of the on-board robot, the control delay problem can be avoided and the agility of the on-board robot can be improved.

[0071] It should be noted that the vehicle controller and the onboard robot can act as WiFi and mobile 4G / 5G gateways to the cloud server. Before the vehicle controller connects to the cloud server, it needs to be authenticated. If the authentication is passed, the vehicle controller can connect to the cloud server, which can improve the security of communication.

[0072] Optionally, the process of the cloud server pre-training the control algorithms corresponding to multiple preset instructions can also be: the cloud server deploys the control algorithms corresponding to the multiple preset instructions in multiple fog servers; each fog server trains the control algorithm corresponding to the preset instructions, and sends the trained multiple control algorithms to the vehicle controller.

[0073] That is to say, when the cloud server trains the control algorithm, it can train it directly in the cloud server, or send multiple control algorithms to multiple fog servers respectively, use the fog server to train the corresponding control algorithm, and the fog server sends the trained control algorithm to the vehicle controller. This can reduce costs and reduce the delay rate of data interaction.

[0074] like Figure 2 As shown, after the above step 101 detects whether the vehicle controller includes a target control algorithm corresponding to the target instruction, the method further includes:

[0075] Step 201: If the vehicle controller does not include a target control algorithm corresponding to the target instruction, a cockpit environment image acquisition instruction is sent to the vehicle-mounted robot, and the cockpit environment image returned by the vehicle-mounted robot is received.

[0076] Step 202: The vehicle controller sends the cabin environment image and target instructions to multiple fog servers.

[0077] Step 203: The target fog server extracts the real-time position of the target object required by the target control algorithm from the cockpit environment image, generates a second control instruction according to the real-time position of the target object and the target control algorithm, and sends the second control instruction to the vehicle controller. The target fog server is a fog server including the target control algorithm.

[0078] Step 204: The vehicle-mounted controller controls the mechanical arm of the vehicle-mounted robot to execute the operation corresponding to the target instruction according to the second control instruction.

[0079] That is to say, if the vehicle controller does not include the target control algorithm corresponding to the target instruction, it means that the target instruction is a new instruction, not some preset instructions in the vehicle controller, and there is no control algorithm corresponding to the target instruction in the vehicle controller. At this time, the cockpit environment image of the vehicle collected by the on-board robot and the target instruction are sent to multiple fog servers. Each fog server determines whether it includes the control algorithm corresponding to the target instruction. If it does, the fog server is the target fog server. The target fog server can identify the target object from the cockpit environment image of the vehicle and determine the position information of the target object according to the target object required by the control algorithm. Then, the determined position information of the target object is input into the control algorithm corresponding to the target instruction to obtain the second control instruction.

[0080] The second control instruction is similar to the first control instruction, and also includes a control instruction for the motion path of the robot arm, a control instruction for the movement speed of the robot arm, and a control instruction for the gripping torque of the robot arm.

[0081] In addition, image recognition algorithms and intelligent grasping control algorithms are deployed in both cloud servers and fog servers, which can guide static and dynamic robots to perform actions such as picking up objects, writing, hugging, etc. that meet human requirements.

[0082] It is understandable that since the user's target instructions are diverse, when it is detected that there is no control algorithm corresponding to the target instruction in the vehicle controller, it is detected whether there is a control algorithm corresponding to the control instruction in the cloud server or fog server, and then the control algorithm in the cloud server or fog server is used to control the robot, which can improve the comprehensiveness and intelligence of the vehicle-mounted robot service.

[0083] Furthermore, the communication between the vehicle controller and the fog server, and the communication between the vehicle controller and the cloud server can use the asynchronous network protocol UDP, but in the process of using UDP communication, data packets may be lost or out of order during transmission. The "heartbeat" mechanism can be used to solve these problems. The "heartbeat" is a switch signal that turns on when the first control signal arrives at the vehicle controller. It will remain on for a period of time and will only turn off if no packet of the selected command is received during this period. We can think of the "heartbeat" as performing a "convolution" on the signal received from the vehicle controller with a moving window. Finally, a ramp function is added to the beginning and end of the "heartbeat" signal to ensure smooth and stable start and stop transitions, which can solve the problem that data packets may be lost or out of order during transmission.

[0084] Optionally, after generating the control instruction according to the real-time position of the target object and the target control algorithm in the above step 103, the method further includes:

[0085] When the on-board robot executes the operation corresponding to the target instruction, it collects the position, speed and torque of the robotic arm in real time, and sends the real-time collected position, speed and torque of the robotic arm to the vehicle controller; the vehicle controller corrects the target control algorithm according to the position, speed and torque of the robotic arm.

[0086] That is to say, when the on-board robot executes the operation corresponding to the target instruction, it will also collect the position, speed and torque of the robotic arm fed back during the operation in real time, and use the feedback position, speed and torque of the robotic arm to further adjust and correct the control algorithm, thereby improving the accuracy of the on-board robot control.

[0087] like Figure 3 As shown, an embodiment of the present application provides a vehicle-mounted robot interaction system, the system comprising a vehicle controller 11 and a vehicle-mounted robot 12;

[0088] The vehicle controller 11 is used to detect whether the vehicle controller 11 includes a target control algorithm corresponding to the target instruction after receiving the target instruction, and if so, send a target data request instruction to the vehicle-mounted robot 12, where the target data request instruction is used to request the real-time position of the target object required by the target control algorithm;

[0089] The vehicle-mounted robot 12 is used to obtain the real-time position of the target object based on the image recognition method after receiving the target data request instruction, and send the real-time position of the target object to the vehicle controller 11;

[0090] The vehicle controller 11 is also used to generate a first control instruction according to the real-time position of the target object and the target control algorithm after receiving the real-time position of the target object. The first control instruction is used to control the mechanical arm of the vehicle-mounted robot 12 to perform the operation corresponding to the target instruction.

[0091] like Figure 3 As shown, the system further includes: a cloud server 13;

[0092] The cloud server 13 is used to pre-train control algorithms corresponding to a plurality of preset instructions, and send the trained control algorithms to the vehicle controller 11;

[0093] The vehicle controller 11 is specifically used to detect whether the trained multiple control algorithms include a target control algorithm corresponding to the target instruction.

[0094] like Figure 3 As shown, the system further includes: a plurality of fog servers 14;

[0095] The cloud server 13 is also used to deploy control algorithms corresponding to multiple preset instructions in multiple fog servers 14;

[0096] Each fog server 14 is used to train a control algorithm corresponding to a preset instruction and send the trained control algorithms to the vehicle controller 11 .

[0097] In one embodiment, the vehicle controller 11 is further configured to:

[0098] If the vehicle controller 11 does not include the target control algorithm corresponding to the target instruction, it sends a cockpit environment image acquisition instruction to the vehicle-mounted robot 12, and receives the cockpit environment image returned by the vehicle-mounted robot 12;

[0099] The vehicle controller 11 sends the cabin environment image and target instructions to multiple fog servers 14;

[0100] The target fog server 14 extracts the real-time position of the target object required by the target control algorithm from the cabin environment image, generates a second control instruction according to the real-time position of the target object and the target control algorithm, and sends the second control instruction to the vehicle controller. The target fog server 14 is a fog server 14 including the target control algorithm.

[0101] The on-board controller controls the mechanical arm of the on-board robot 12 to perform the operation corresponding to the target instruction according to the second control instruction.

[0102] In one embodiment, the vehicle-mounted robot 12 is further used to collect the position of the robotic arm, the speed of the robotic arm and the torque of the robotic arm in real time when executing the operation corresponding to the target instruction, and send the real-time collected position of the robotic arm, the speed of the robotic arm and the torque of the robotic arm to the vehicle controller 11;

[0103] The vehicle controller 11 is also used to modify the target control algorithm according to the position of the robot arm, the speed of the robot arm and the torque of the robot arm.

[0104] In one embodiment, the vehicle controller 11 is specifically configured to:

[0105] Receive target control instructions input by a user in the vehicle cabin, or receive target control instructions sent by a terminal device, or receive target control instructions sent by a user based on a VR device.

[0106] In one embodiment, the vehicle-mounted robot 12 is specifically used for:

[0107] The vehicle-mounted robot 12 determines the real-time position of the target object based on an image recognition algorithm of the visual positioning marker.

[0108] The vehicle-mounted robot interaction system provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0109] The specific definition of the vehicle-mounted robot interaction system can be found in the definition of the vehicle-mounted robot interaction method above, which will not be repeated here. Each module in the above vehicle-mounted robot interaction system can be fully or partially implemented by software, hardware or a combination thereof.

[0110] In another embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle-mounted robot interaction method in the embodiment of the present application are implemented.

[0111] In another embodiment of the present application, a computer program product is also provided. The computer program product includes computer instructions. When the computer instructions are executed on the vehicle-mounted robot interaction system, the vehicle-mounted robot interaction system executes each step of the vehicle-mounted robot interaction method in the method flow shown in the above method embodiment.

[0112] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer execution instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more servers that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0113] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A vehicle-mounted robot interaction method, characterized in that: The method comprises: The cloud server deploys the control algorithms corresponding to the multiple preset instructions in multiple fog servers; Each of the fog servers trains a control algorithm corresponding to a preset instruction, and sends the trained multiple control algorithms to the vehicle controller; After receiving the target instruction, the vehicle controller detects whether the trained multiple control algorithms include the target control algorithm corresponding to the target instruction, and if so, sends a target data request instruction to the vehicle-mounted robot, wherein the target data request instruction is used to request the real-time position of the target object required by the target control algorithm; After receiving the target data request instruction, the vehicle-mounted robot obtains the real-time position of the target object based on an image recognition method, and sends the real-time position of the target object to the vehicle controller; After receiving the real-time position of the target object, the vehicle controller generates a first control instruction according to the real-time position of the target object and the target control algorithm, wherein the first control instruction is used to control the mechanical arm of the vehicle-mounted robot to perform an operation corresponding to the target instruction; If the vehicle controller does not include the target control algorithm corresponding to the target instruction, sending a cockpit environment image acquisition instruction to the onboard robot, and receiving the cockpit environment image returned by the onboard robot; The vehicle controller sends the cabin environment image and the target instruction to a plurality of fog servers; The target fog server extracts the real-time position of the target object required by the target control algorithm from the cockpit environment image, generates a second control instruction according to the real-time position of the target object and the target control algorithm, and sends the second control instruction to the vehicle controller, wherein the target fog server is a fog server including the target control algorithm; The vehicle controller controls the mechanical arm of the vehicle-mounted robot to perform the operation corresponding to the target instruction according to the second control instruction.

2. The method according to claim 1, characterized in that After generating the first control instruction according to the real-time position of the target object and the target control algorithm, the method further includes: When the vehicle-mounted robot executes the operation corresponding to the target instruction, the position of the mechanical arm, the speed of the mechanical arm and the torque of the mechanical arm are collected in real time, and the position of the mechanical arm, the speed of the mechanical arm and the torque of the mechanical arm collected in real time are sent to the vehicle controller; The vehicle controller modifies the target control algorithm based on the position of the robotic arm, the speed of the robotic arm, and the torque of the robotic arm.

3. The method according to claim 1, characterized in that The vehicle controller receiving the target instruction includes: Receive a target control instruction input by a user in a vehicle cabin, or receive a target control instruction sent by a terminal device, or receive a target control instruction sent by the user based on a VR device.

4. The method according to claim 1, characterized in that: The method of obtaining the real-time position of the target object based on the image recognition method includes: The vehicle-mounted robot determines the real-time position of the target object based on an image recognition algorithm of a visual positioning marker.

5. An interactive system for a vehicle-mounted robot, characterized in that: The system includes: a cloud server, a plurality of fog servers, a vehicle controller and a vehicle-mounted robot; The cloud server is used to deploy control algorithms corresponding to multiple preset instructions in multiple fog servers; Each of the fog servers is used to train a control algorithm corresponding to a preset instruction and send the trained multiple control algorithms to the vehicle controller; The vehicle controller is used to detect whether the trained multiple control algorithms include the target control algorithm corresponding to the target instruction after receiving the target instruction, and if so, send a target data request instruction to the vehicle-mounted robot, wherein the target data request instruction is used to request the real-time position of the target object required by the target control algorithm; The vehicle-mounted robot is used to obtain the real-time position of the target object based on an image recognition method after receiving the target data request instruction, and send the real-time position of the target object to the vehicle controller; The vehicle controller is used to generate a first control instruction according to the real-time position of the target object and the target control algorithm after receiving the real-time position of the target object, wherein the first control instruction is used to control the mechanical arm of the vehicle-mounted robot to perform an operation corresponding to the target instruction; If the vehicle controller does not include the target control algorithm corresponding to the target instruction, sending a cockpit environment image acquisition instruction to the onboard robot, and receiving the cockpit environment image returned by the onboard robot; The vehicle controller is used to send the cabin environment image and the target instruction to multiple fog servers; A target fog server is used to extract the real-time position of the target object required by the target control algorithm from the cockpit environment image, generate a second control instruction according to the real-time position of the target object and the target control algorithm, and send the second control instruction to the vehicle controller, wherein the target fog server is a fog server including the target control algorithm; The vehicle controller is used to control the mechanical arm of the vehicle-mounted robot to perform the operation corresponding to the target instruction according to the second control instruction.

Images