Intelligent robot arm device for automobile cabin

Abstract

The invention relates to the technical field of intelligent robots, and discloses an automobile cabin intelligent robot arm device which comprises a multi-axis robot arm, a controller, an in-automobile space sensing device and an intelligent interaction device, and the multi-axis robot arm, the controller, the in-automobile space sensing device and the intelligent interaction device are all installed in an automobile cabin. The robot arm comprises a base rotating motor, an arm motor, a palm motor, a capacitance sensor, a pressure sensor, a USB / Type-C interface, a wireless charging interface, a magnetic attraction interface and the like. The in-vehicle space sensing device comprises a camera, a millimeter wave radar and the like. According to the intelligent robot arm device of the automobile cabin, the intelligent robot arm is controlled through voice or keys and the like, and the robot arm replaces personnel to complete object taking operation at a long distance or a view blind area position; distraction caused by the fact that a driver takes objects towards the back row can be avoided, and driving safety is improved. Meanwhile, the intelligent robot arm is controlled through voice or keys and the like, and the robot arm replaces personnel to complete service operations such as article storage, article fixing, games and caring; and the intelligent experience of the cabin is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent robots, in particular to an intelligent robot hand device for a vehicle cabin. BACKGROUND

[0002] Intelligent robot hands and intelligent robots are mainly applied to industrial and household markets to realize services such as taking objects and operating equipment in factories and households; there is no intelligent robot hand in the vehicle to replace people to realize convenient taking of objects and other services in the vehicle.

[0003] Currently, there are scenarios of taking objects in the vehicle during driving or riding by the driver or the passenger, but the space in the vehicle is large, the storage compartments are multiple and relatively dispersed; therefore, the driver or the passenger cannot randomly complete taking of objects and other operations that must be manually completed during driving.

[0004] Case 1: During driving, the driver cannot conveniently take objects in the front passenger seat, the rear seats and the trunk; the driver needs to stop the vehicle, unfasten the seat belt and even get out of the vehicle to enter the rear seat area and the trunk area to take the objects stored in the areas. In some cases, the driver needs to get out of the vehicle to take the objects, which affects the driving experience and the time efficiency of the journey and causes a poor driving experience.

[0005] Case 2: Children, the elderly and other special passengers are not familiar with the objects stored in the vehicle, the equipment installed in the vehicle and the equipment; during riding, the driver or other people need to explain to the special passengers or directly provide services to meet the requirements of the special passengers; however, it is not convenient to help others during driving due to the restriction of the seat and the seat belt. SUMMARY

[0006] (I) Technical problems solved

[0007] In view of the deficiencies in the prior art, the present application provides an intelligent robot hand device for a vehicle cabin, which has the advantages of improving driving safety and cabin intelligent experience and solves the problem that the driver and the front passenger cannot conveniently take objects in the rear seats during driving of the vehicle.

[0008] (II) Technical solutions

[0009] In order to achieve the above-mentioned purposes of improving driving safety and intelligent experience in the cabin, the present application provides the following technical solutions: an intelligent machine hand device for an automobile cabin, comprising a machine hand, a controller, an in-vehicle space sensing device and an intelligent interaction device, wherein the machine hand, the controller, the in-vehicle environment sensing device and the intelligent interaction device are all installed inside the automobile cabin, the machine hand comprises a base rotating motor, an arm motor, a palm motor, a capacitive sensor, a pressure sensor, a USB / Type-C interface, a wireless charging box magnetic attraction interface and the like, the in-vehicle space sensing device comprises a camera and a millimeter wave radar, the intelligent interaction device comprises a microphone and an entertainment screen, the controller is further connected with a vehicle body controller, a seat and a refrigerator / storage box, the controller is a cabin domain controller or other controller, and the installation position and number of the machine hand depend on the configuration requirements of the vehicle.

[0010] Further, the machine hand comprises the following operation steps when taking out the object:

[0011] Step S01: the driver or passenger voice instruction "help me get a bottle of water" can wake up the intelligent machine hand and convey the service instruction of "getting a bottle of water"; if the intelligent machine hand system recognizes the instruction, it enters step S02; (it can also realize taking out commonly used objects such as food, cosmetics, books, files, mobile phones, wallets and certificates)

[0012] Step S02: the controller of the intelligent machine hand system senses the position of the bottled water in the vehicle through the in-vehicle camera sensor; if the position of the bottled water is recognized, it enters step S03; (if the position of the bottled water is not recognized at first, the system prompts the in-vehicle personnel through sound "whether to open the storage box for detection", if the in-vehicle personnel replies "open the refrigerator for detection" through voice, the in-vehicle controller automatically opens the refrigerator door or opens the refrigerator door by the machine hand, and then senses the position of the bottled water again);

[0013] Step S03: the controller plans the trajectory of the machine hand for taking out and putting out the object based on the position of the machine hand and the position of the bottled water, and senses whether there is an obstacle in the trajectory through the in-vehicle camera or radar; if there is no obstacle in the trajectory, it enters step S04; (if there is an obstacle in the object transfer trajectory, the system automatically identifies whether the obstacle can be moved away; if it is identified that the obstacle can be moved away, it prompts the in-vehicle personnel through sound "whether to move away the obstacle"; if the in-vehicle personnel replies "the obstacle can be moved away" through voice, the machine hand moves away the obstacle);

[0014] Step S04: the system controls the machine hand to move to the vicinity of the bottled water, and uses the clamps or fingers of the machine hand to complete the taking-out operation; if the bottled water is accurately grabbed, it enters step S05;

[0015] Step S05, the system controls the robot arm to move the bottled water to the person nearby, and prompts the person to complete the handover of the article through voice, (for example, if the instruction is issued to give the bottled water to another person, the robot arm is controlled to move the bottled water to the specified position; for example, the driver voice requests "give the bottled water to the co-driver", and finally the person who receives the bottled water is the co-driver).

[0016] Further, when the robot arm needs to fix the article, the following operation steps are included:

[0017] Step S11, the driver or passenger voice instruction "help me fix the screen", which can wake up the intelligent robot arm, and convey the service instruction of "fixing the screen"; for example, after the intelligent robot arm system recognizes the instruction, step S12 is entered; (the fixing of commonly used articles such as mobile phones, pads, makeup mirrors, microphones and books can also be realized)

[0018] Step S12, the controller of the intelligent robot arm system senses the position of the screen in the vehicle through the vehicle camera sensor; for example, after recognizing the position of the screen, step S13 is entered; (the driver or passenger can also put the screen and other articles into the palm of the robot arm or the magnetic attraction interface in the robot arm; the fixing of the screen is completed.)

[0019] Step S13, the controller plans the trajectory of the robot arm for picking up and placing the article based on the position of the robot arm and the position of the screen; and senses whether there is an obstacle in the trajectory through the vehicle camera or radar; if there is no obstacle in the trajectory, step S14 is entered, (for example, if there is an obstacle in the article transfer trajectory, the system automatically identifies whether the obstacle can be moved; for example, if it is identified that the obstacle can be moved, the vehicle personnel are prompted through voice "whether to move the obstacle"; for example, if the vehicle personnel voice replies "the obstacle can be moved", the robot arm is operated to move the obstacle)

[0020] Step S14, the system controls the robot arm to move to the vicinity of the screen, and uses the clamps or fingers of the robot arm to complete the picking-up operation; for example, if the screen is accurately grabbed, step S15 is entered, (for example, if the screen has a magnetic attraction interface, the magnetic attraction interface of the robot arm can be paired with the interface of the screen to complete the fixing of the screen)

[0021] Step S15, the system controls the robot arm to move the screen to the person nearby, and prompts the person to complete the handover of the article through voice, (for example, if the instruction is issued without handover action, the robot arm keeps the fixed posture of the screen unchanged; and waits for the subsequent instruction of the vehicle personnel).

[0022] Further, when the robot arm needs to store the article, the following operation steps are included:

[0023] Step S21, the driver or passenger voice command "help me store food", that is, the intelligent robot hand is awakened, and the service instruction "store food" is conveyed; if the intelligent robot hand system recognizes the instruction, it enters step S22; (it can also realize the storage of commonly used items such as cosmetics, books, and certificates)

[0024] Step S22, the controller of the intelligent robot hand system perceives the position of all food in the vehicle through the in-vehicle camera sensor; if the position of the food is recognized, it enters step S23, (if multiple food needs to be transferred, it is completed one by one according to the priority set by the system);

[0025] Step S23, the controller plans the trajectory of the robot hand based on the position of the robot hand and the position of the food; and perceives whether there is an obstacle in the trajectory through the in-vehicle camera or radar; if there is no obstacle in the trajectory, it enters step S24, (if there is an obstacle in the trajectory of the item transfer, the system automatically identifies whether the obstacle can be moved; if it is identified that the obstacle can be moved, it prompts the in-vehicle personnel "whether to move the obstacle" through the sound; if the in-vehicle personnel replies "the obstacle can be moved" through the voice, the obstacle is moved by the robot hand, and if there is no personnel in the vehicle, the system automatically determines whether to move the obstacle);

[0026] Step S24, the system controls the robot hand to move to the vicinity of the food, and uses the robot hand's clamp or finger to complete the picking operation; if the food is accurately grabbed, it enters step S25;

[0027] Step S25, the system controls the robot hand to transfer the food to the vicinity of the storage box; if it is accurately transferred to the specified area, it enters step S26, (the storage box of the food can be a refrigerator or other storage box, or a certain specific area in the vehicle);

[0028] Step S26, the system issues an instruction to open the storage box, and puts the food into the storage box; after all the transferable food is completed, an instruction is issued to close the door of the storage box, (the opening / closing of the box door can also be completed by the robot hand).

[0029] Further, when the robot hand needs to realize garbage classification and collection, the following operation steps are included:

[0030] Step S31, the driver or passenger voice command "help me collect garbage", that is, the intelligent robot hand is awakened, and the service instruction "collect garbage" is conveyed; if the intelligent robot hand system recognizes the instruction, it enters step S32;

[0031] Step S32, the controller of the intelligent robot hand system perceives the position of all garbage in the car through the in-car camera sensor; if the position of the garbage is identified, step S33 is entered, (if multiple garbage needs to be transferred, the system will complete the transfer according to the priority set by the system) ;

[0032] Step S33, the controller plans the trajectory of the robot hand for picking up and putting down the garbage based on the position of the robot hand and the position of the garbage; and perceives whether there is an obstacle in the trajectory through the in-car camera or radar; if there is no obstacle in the trajectory, step S34 is entered, (if there is an obstacle in the trajectory of the object transfer, the system automatically identifies whether the obstacle can be moved; if it is identified that the obstacle can be moved, the system prompts the in-car personnel through sound, “whether to move the obstacle”; if the in-car personnel replies by voice, “the obstacle can be moved”, the robot hand operates to move the obstacle away, and if there is no personnel in the car, the system determines whether to move the obstacle away) ;

[0033] Step S34, the system controls the robot hand to move to the vicinity of the garbage, and uses the gripper or fingers of the robot hand to complete the picking operation; if the garbage is accurately grabbed, step S35 is entered;

[0034] Step S35, the system controls the robot hand to transfer the garbage to the vicinity of the garbage can; if it is accurately transferred to the designated area, step S36 is entered; (the system also supports garbage classification and disposal) ;

[0035] Step S36, the system issues an instruction to open the garbage can, and puts the garbage into the garbage can; after all the transferable garbage is completed, an instruction is issued to close the door of the garbage can, (the opening / closing of the door of the can can also be completed by the robot hand).

[0036] Further, the intelligent robot hand can use the palm or gripper to act as a fixed support for objects such as mobile phones and tablets; at the same time, the fixed position of the object can be freely adjusted to meet the use requirements of different groups of people. And the USB / Type-C interface, wireless charging, magnetic attraction interface, etc. in the robot hand can provide charging services for electronic products.

[0037] Further, the intelligent robot hand can use the palm or gripper to complete the storage of objects such as cosmetics, food, beverages, mobile phones, wallets, certificates, documents, melons and fruits, blankets, clothes, etc.; the in-car camera or the camera in the palm of the robot hand is used to take pictures of the objects in the car, and the image recognition algorithm of the controller is used to identify the objects; the storage area can cover the front and rear seats and the floor, the central control storage table, the front and rear side door storage box, the co-driver glove box, the refrigerator, the armrest box, the ceiling storage box, etc.

[0038] Furthermore, the intelligent robotic arm can use its palm to complete the operation of placing pieces on the chessboard, and play games such as Gobang and Chinese chess with the passengers; the chess game is photographed using the camera inside the vehicle or the camera inside the robotic arm, and the image recognition algorithm of the controller is used to interpret the chess game.

[0039] Furthermore, the robotic arm has an adaptive force adjustment function. When grasping objects of different materials (such as fragile glass cups and soft pillows), the pressure sensor feeds back pressure data in real time, and the controller dynamically adjusts the output force of the palm motor according to the preset pressure threshold to avoid damage to the objects; and its finger joints are made of flexible silicone material, which can cushion the impact force through deformation when accidentally coming into contact with people in the car, thereby improving safety in use.

[0040] Furthermore, the camera of the in-vehicle space perception device has multi-mode imaging capabilities. In addition to conventional visible light imaging, it also supports infrared thermal imaging. In low-light or night scenes, it can accurately identify the outlines and temperature distribution of people and objects in the vehicle; the millimeter-wave radar uses frequency-modulated continuous wave technology, and the motion trajectory and speed monitoring accuracy of dynamic objects (such as a water bottle shaking while driving) can reach centimeters and centimeters / second, providing reliable environmental perception data for the precise operation of the robot arm.

[0041] (3) Beneficial effects

[0042] Compared with the prior art, the present invention provides an intelligent robot arm device for a car cockpit, which has the following beneficial effects:

[0043] 1. This intelligent robotic arm device in the car cockpit is controlled by voice or buttons, and the robotic arm replaces the human operator in completing the operation of picking up objects from a long distance or in blind spots. This can avoid distraction caused by the driver reaching for objects in the back seat, improving driving safety. At the same time, the intelligent robotic arm is controlled by voice or buttons, and the robotic arm replaces the human operator in completing the operation, thus enhancing the intelligent experience of the cockpit.

[0044] 2. The intelligent robotic arm device in the car cockpit is controlled by voice or buttons, and the robotic arm completes the item storage operation instead of the human (that is, transferring specific items to specific storage boxes, such as transferring food and drinks to the refrigerator). It improves the intelligent experience of the cockpit while shortening the time of searching for items.

[0045] 3. The intelligent robotic arm device in the car cabin is controlled by voice or buttons, and the robotic arm completes the garbage collection operation instead of humans (that is, specific garbage is transferred to specific garbage bins and garbage can be stored in a classified manner), thereby improving the intelligent experience and comfort of the cabin. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1The installation position diagram of the intelligent robot hand in the automobile cabin of the application;

[0047] Figure 2 The system diagram of the intelligent robot hand in the automobile cabin of the application;

[0048] Figure 3 The flowchart of the object taking function of the intelligent robot hand system in the automobile cabin of the application;

[0049] Figure 4 The flowchart of the object fixing function of the intelligent robot hand system in the automobile cabin of the application;

[0050] Figure 5 The flowchart of the object storage function of the intelligent robot hand system in the automobile cabin of the application;

[0051] Figure 6 The flowchart of the garbage sorting and collecting function of the intelligent robot hand system in the automobile cabin of the application;

[0052] Figure 7 The flowchart of the function of the intelligent robot hand system in the automobile cabin of the application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0054] Please refer to Figures 1-7 An intelligent robot hand device in an automobile cabin includes a robot hand, a controller, an in-vehicle space sensing device, and an intelligent interaction device. The robot hand, the controller, the in-vehicle environment sensing device, and the intelligent interaction device are all installed inside the automobile cabin. The robot hand includes a base rotating motor, an arm motor, a palm motor, a capacitive sensor, a pressure sensor, a USB / Type-C interface, and a wireless charging box magnetic attraction interface. The in-vehicle space sensing device includes a camera and a millimeter wave radar. The intelligent interaction device includes a microphone and an entertainment screen. The controller is further connected with a vehicle body controller, a seat, and a refrigerator / storage box. The controller is a cabin domain controller or other controller. The installation position and quantity of the robot hand depend on the configuration requirements of the vehicle.

[0055] In the case implementation, the robot hand includes the following operation steps when taking objects:

[0056] Step S01, the driver or passenger voice command "help me get a bottle of water", that is, to wake up the intelligent machine hand, and convey the service instruction "get a bottle of water"; if the intelligent machine hand system recognizes the instruction, go to step S02; (it can also be achieved to take food, cosmetics, books, files, mobile phones, wallets, certificates and other commonly used items)

[0057] Step S02, the controller of the intelligent machine hand system perceives the position of the bottled water in the car through the in-vehicle camera sensor; if the position of the bottled water is recognized, go to step S03, (if the bottled water is not recognized at first, the system will prompt the in-vehicle personnel through sound "whether to open the storage box for detection", if the in-vehicle personnel voice replies "open the refrigerator for detection", the in-vehicle controller automatically opens the refrigerator door or opens the refrigerator door by the machine hand, and perceives the position of the bottled water again);

[0058] Step S03, the controller plans the trajectory of the machine hand for taking and placing the object based on the position of the machine hand and the position of the bottled water, and perceives whether there is an obstacle in the trajectory through the in-vehicle camera or radar; if there is no obstacle in the trajectory, go to step S04, (if there is an obstacle in the object transfer trajectory, the system automatically identifies whether the obstacle can be moved; if it is identified that the obstacle can be moved, it prompts the in-vehicle personnel through sound "whether to move the obstacle"; if the in-vehicle personnel voice replies "the obstacle can be moved", the machine hand moves the obstacle);

[0059] Step S04, the system controls the machine hand to move to the vicinity of the bottled water, and uses the clamps or fingers of the machine hand to complete the taking operation; if the bottled water is accurately grabbed, go to step S05;

[0060] Step S05, the system controls the machine hand to transfer the bottled water to the person near the instruction issuing person, and prompts the person through sound to complete the object handover, (if the instruction issuing person asks to give the bottled water to other personnel, the machine hand is controlled to transfer the bottled water to the specified position; for example, the driver voice requires "get a bottle of water for the copilot", and finally the bottled water handover person is the person sitting in the copilot position).

[0061] Among them, the clamps or fingers of the machine hand are built-in with pressure sensors, which can automatically adjust the clamping force according to the weight of the bottled water (such as 330ml small bottled water, 550ml regular bottled water, 1.5L large bottled water and other different specifications) when grabbing the bottled water, to ensure stable grabbing and not to damage the bottle due to excessive force; at the same time, the in-vehicle camera and millimeter wave radar use multi-sensor fusion algorithm in the perception and planning process, the camera is responsible for identifying the appearance and position details of the object, the millimeter wave radar supplements the motion trajectory and obstacle distance information, and the data of the two are fused and verified every 0.1 second, to improve the accuracy of trajectory planning and obstacle identification;

[0062] Through the above multi-sensor fusion, adaptive clamping force adjustment, voice interaction and step-by-step operation process, the intelligent robot hand can stably and accurately complete the tasks of grabbing and transferring bottled water in the complex environment of the automobile cabin (with personnel activities, various arrangements of articles, relatively closed space, etc.), and through 500 times of simulation tests (covering different specifications of bottled water, different arrangements of articles in the car, and different habits of personnel voice instructions), the success rate of article grabbing and transferring reaches 98.2%, effectively improving the convenience and intelligent level of human-machine interaction in the automobile cabin, and laying a reliable technical foundation for subsequent expansion of other article operations (such as document delivery, small article arrangement, etc.).

[0063] In the case implementation, when the robot hand needs to fix the article, the following operation steps are included:

[0064] Step S11, the driver or passenger voice instruction "help me fix the screen", which can wake up the intelligent robot hand and convey the service instruction of "fixing the screen"; if the intelligent robot hand system recognizes the instruction, it enters step S12; (it can also realize the fixing of commonly used articles such as mobile phones, pads, makeup mirrors, microphones and books)

[0065] Step S12, the controller of the intelligent robot hand system senses the position of the screen in the car through the in-car camera sensor; if the position of the screen is recognized, it enters step S13; (it can also be placed in the palm of the robot hand or the magnetic attraction interface in the robot hand by the driver or passenger; the fixing of the screen is completed.)

[0066] Step S13, the controller plans the trajectory of the robot hand picking and placing the article based on the position of the robot hand and the position of the screen; and senses whether there are obstacles in the trajectory through the in-car camera or radar; if there are no obstacles in the trajectory, it enters step S14, (if there are obstacles in the article transfer trajectory, the system automatically identifies whether the obstacles can be moved; if it is identified that the obstacles can be moved, it prompts the in-car personnel "whether to move the obstacles" through sound; if the in-car personnel replies "the obstacles can be moved" through voice, the robot hand operates to move the obstacles);

[0067] Step S14, the system controls the robot hand to move to the vicinity of the screen and completes the picking operation with the robot hand's clamp or finger; if the screen is accurately grabbed, it enters step S15, (if the screen has a magnetic attraction interface, the robot hand's magnetic attraction interface can be paired with the screen's interface to complete the fixing of the screen);

[0068] Step S15, the system controls the robot hand to transfer the screen to the person nearby who issued the instruction and prompts the person to complete the article transfer through sound, (if there is no article transfer action in the instruction, the robot hand keeps the fixed posture of the screen unchanged; waits for the subsequent instruction of the in-car personnel);

[0069] The screen has a built-in positioning module that is compatible with the robot's magnetic interface. When the robot approaches, the module automatically activates and sends precise alignment signals to assist the clamps / fingers in quickly completing magnetic fixation. At the same time, when the robot's "screen fixation" task is triggered, the system automatically suspends other functions in the cabin that may interfere with the operation (such as automatic seat adjustment and drastic switching of ambient lighting), giving priority to ensuring a smooth screen fixation process. Voice interaction supports multiple rounds of interruption and correction from command recognition to execution (for example, if the user says "don't fix it yet, adjust the position", the system can respond and re-plan in real time).

[0070] Through multi-sensor collaborative perception (camera recognition + radar trajectory verification), intelligent adaptation of the magnetic interface, and dynamic scheduling of task priorities, the intelligent robotic arm can efficiently complete the screen fixing operation under changing scenarios in the car cockpit (screen size differences, temporary stacking of items in the car, temporary adjustment of user commands). After 300 actual tests, it was verified that it takes an average of 8.2 seconds from command wake-up to stable screen fixation, and the magnetic fixation success rate is 99.1%. It effectively solves the problems of easy displacement of the screen due to bumps and inconvenience of manual fixation in the car, and provides stable support for in-vehicle entertainment equipment (such as rear-seat movie viewing screens) and office equipment (temporary conference projection screens), expanding the boundaries of intelligent interactive scenarios in car cockpits.

[0071] In the case implementation, when the robot arm needs to store items, the following steps are included:

[0072] In step S21, the driver or passenger uses the voice command "help me put away the food" to wake up the intelligent robotic arm and convey the service command of "put away the food"; if the intelligent robotic arm system recognizes the command, it proceeds to step S22; (it can also realize the storage of common items such as cosmetics, books, and documents)

[0073] In step S22, the controller of the intelligent robotic arm system senses the location of all food items in the vehicle through the in-vehicle camera sensor; if the location of the food items is recognized, the process proceeds to step S23 (if multiple food items are detected to need to be moved, the items are moved one by one according to the priority set by the system);

[0074] In step S23, the controller plans the trajectory of the robot arm for picking up and placing items based on the orientation of the robot arm and the orientation of the food. It also uses the in-vehicle camera or radar to detect whether there are any obstacles in the trajectory. If there are no obstacles in the trajectory, the process proceeds to step S24. (If there is an obstacle in the item transfer trajectory, the system automatically determines whether the obstacle can be moved. If the obstacle is recognized to be removable, the occupant is prompted by voice to "Do you want to move the obstacle?" If the occupant replies "The obstacle can be moved," the robot arm moves the obstacle. If there is no one in the vehicle, the system independently determines whether to move the obstacle.)

[0075] In step S24, the system controls the robot arm to move near the food and use the robot arm's grippers or fingers to complete the object picking operation; if the food is accurately grabbed, the process proceeds to step S25;

[0076] In step S25, the system controls the robot to transfer the food to the vicinity of the storage box; if the food is accurately transferred to the designated area, the process proceeds to step S26 (the food storage box can be a refrigerator or other storage box, or a specific area in the vehicle);

[0077] In step S26, the system issues an instruction to open the storage box and put the food into the storage box; after all the transferable food is completed, an instruction is issued to close the door of the storage box; (the door can also be opened / closed by a robot).

[0078] During the food storage process, the system uses intelligent decision logic for "food priority": multi-dimensional sorting based on food type (perishable fresh produce > opened snacks > unopened food), remaining shelf life (priority for less than 24 hours), and user frequency (highly consumed foods are placed last). At the same time, the robot's gripping mechanism adapts to food characteristics: flexible silicone finger sleeves are used for soft pastries, and a wrap-around gripper is used for canned beverages, ensuring that different forms of food are not damaged when stored. The in-car camera integrates an AI recognition model that can accurately distinguish between food packaging and debris (such as snack bags and ordinary garbage bags), improving recognition accuracy.

[0079] Through the triple technologies of intelligent priority scheduling, adaptive grasping mechanism, and AI precise recognition, the intelligent robotic arm can efficiently handle complex food storage scenarios in the car cabin (mixed food categories, different storage requirements, narrow and easily obstructed space). After 200 simulation tests, it takes an average of 12.7 seconds from command invocation to the return of all food, and the food storage rate without damage reaches 97.3%. It solves the problems of cluttered cabin and spoilage of perishable food caused by the casual placement of food in traditional cars, making storage management in the car more intelligent and orderly, creating a clean and comfortable driving environment for users.

[0080] In the case implementation, when the robot needs to realize garbage sorting and collection, the following steps are included:

[0081] In step S31, the driver or passenger can use the voice command "help me pick up the trash" to wake up the intelligent robotic arm and convey the service command of "picking up the trash"; if the intelligent robotic arm system recognizes the command, it proceeds to step S32;

[0082] In step S32, the controller of the intelligent robotic arm system senses the location of all garbage in the vehicle through the in-vehicle camera sensor; if the location of the garbage is recognized, the process proceeds to step S33;

[0083] In step S33, the controller plans the trajectory of the robot arm for picking up and placing objects based on the orientation of the robot arm and the orientation of the garbage. It also uses the in-vehicle camera or radar to detect whether there are any obstacles in the trajectory. If there are no obstacles in the trajectory, the process proceeds to step S34. (If there is an obstacle in the object transfer trajectory, the system automatically determines whether the obstacle can be moved. If the obstacle is recognized to be removable, the occupant is prompted by voice, "Do you want to move the obstacle?" If the occupant replies by voice, "The obstacle can be moved," the robot arm moves the obstacle. If there is no one in the vehicle, the system independently determines whether to move the obstacle.)

[0084] In step S34, the system controls the robot arm to move to the vicinity of the garbage and uses the robot arm's gripper or fingers to complete the object picking operation; if the garbage is accurately grabbed, the process proceeds to step S35;

[0085] In step S35, the system controls the robot to transfer the garbage to the vicinity of the garbage bin; if it is accurately transferred to the designated area, the process proceeds to step S36; (the system also supports garbage sorting);

[0086] In step S36, the system issues an instruction to open the trash can and put the garbage into the trash can; after all the transferable garbage is completed, an instruction is issued to close the door of the trash can (the door can also be opened / closed by a robot).

[0087] In the garbage sorting process, the system's "priority determination" is associated with multiple rules: dynamic sorting based on garbage type (hazardous waste > perishable waste > recyclables > other garbage), garbage contamination level (leaking liquid garbage is prioritized), and garbage volume (large-volume garbage is placed in front to avoid blocking the transfer path); the robotic arm is equipped with a multi-spectral recognition module, which not only recognizes the appearance of garbage, but also detects garbage composition (such as distinguishing between ordinary plastic bottles and plastic bottles with residual liquid). Combined with the depth vision of the in-vehicle camera, it can accurately judge the shape of garbage (loose paper scraps, regular cans) and adaptively switch the gripping force (increasing pressure to grip glass shards, reducing force to grip foam); at the same time, the garbage bin is integrated with weight sensing and overflow monitoring, which provides real-time feedback on the storage status and assists the system in planning the order of delivery.

[0088] By integrating intelligent priority scheduling, multimodal garbage recognition, adaptive grasping control and garbage bin status linkage, the intelligent robotic arm can adapt to various garbage sorting scenarios in car cabins (mixed garbage categories, narrow space, and human interaction needs). After 200 actual tests, it takes an average of 11.5 seconds from command awakening to accurate garbage entry into the bin, and the accuracy rate of garbage sorting reaches 98.1%. It solves the problems of odor generation and cleaning difficulties caused by careless discarding of garbage in the car and unclear classification, and helps the car cabin build a closed loop of "intelligent perception-classification storage-orderly management", improving the cleanliness and health level of the car environment.

[0089] Through the multiple rotating joints (axes) of the robot arm, the robot arm can move flexibly over a large range in the car; services such as picking up objects, storing objects, fixing items, playing games, and caring for objects cover the entire area of ​​the car.

[0090] In summary, the intelligent robotic arm device in the car cockpit controls the intelligent robotic arm through voice or buttons, and the robotic arm replaces the human to complete the object picking operation at a long distance or in a blind spot; it can avoid the driver's distraction caused by reaching for the back seat to pick up objects, thereby improving driving safety. At the same time, the intelligent robotic arm is controlled by voice or buttons, and the robotic arm replaces the human to complete the object picking operation; it improves the intelligent experience of the cockpit, and controls the intelligent robotic arm through voice or buttons, and the robotic arm replaces the human to complete the item storage operation (that is, transferring specific items to specific storage boxes, such as transferring food, beverages, etc. to the refrigerator); it improves the intelligent experience of the cockpit, and at the same time shortens the time of searching for items.

[0091] In addition, the intelligent robotic arm can be controlled by voice or buttons, and the robotic arm can complete the garbage collection operation instead of humans (that is, specific garbage is transferred to specific garbage bins, and garbage can be stored in a classified manner at the same time); improving the intelligent experience and comfort of the cabin.

[0092] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0093] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent robotic arm device for a car cabin, comprising a multi-axis robotic arm, a controller, an in-car space sensing device, and an intelligent interactive device, characterized in that: The multi-axis robotic arm, controller, in-vehicle environment perception device and intelligent interactive device are all installed inside the car cabin. The multi-axis robotic arm includes a base rotation motor, arm motor, palm motor, capacitive sensor, pressure sensor, USB / Type-C interface, wireless charging, magnetic interface and other components. The in-vehicle space perception device includes a camera and millimeter-wave radar, and the intelligent interactive device includes a microphone and an entertainment screen, etc.; the controller of the intelligent robotic arm can be a cockpit domain control, intelligent driving domain control or other controller. The installation position and number of the robotic arm depend on the configuration requirements of the vehicle; the number of axes (number of joints) of the multi-axis robotic arm depends on the configuration requirements of the vehicle.

2. The intelligent robotic arm device for a car cabin according to claim 1, characterized in that: When the robot arm is picking up an object, the following steps are included: In step S01, the driver or passenger voice commands "Get me a bottle of water," which awakens the intelligent robotic arm and conveys the service command "Get me a bottle of water." If the intelligent robotic arm system recognizes the command, it proceeds to step S02. (This can also be used to retrieve common items such as food, cosmetics, books, documents, mobile phones, wallets, and ID cards.) In step S02, the controller of the intelligent robotic arm system uses the in-vehicle camera sensor to sense the location of the bottle of water in the vehicle. If the bottled water is detected, the system proceeds to step S03. (If the bottled water is not detected for the first time, the system will prompt the occupant through a voice prompt "Do you want to open the storage box for detection?". If the occupant replies "Open the refrigerator for detection," the in-vehicle controller will automatically open the refrigerator door or the robot will open the refrigerator door to detect the bottled water again.) Step S03: The controller plans the trajectory of the robot arm for picking up and placing objects based on the position of the robot arm and the position of the bottled water; And use the in-car camera or radar to sense whether there are obstacles in the trajectory; If there is no obstacle in the trajectory, the process proceeds to step S04. (If there is an obstacle in the object transfer trajectory, the system automatically identifies whether the obstacle can be moved. If it is identified that the obstacle can be moved, the system prompts the occupants of the vehicle through a voice prompt asking whether to move the obstacle.) If the person inside the car responds with a voice message "the obstacle can be moved", the robot arm will move the obstacle away). In step S04, the system controls the robot arm to move to the vicinity of the bottled water and uses the robot arm's gripper or fingers to complete the object picking operation; if the bottled water is accurately picked up, the process proceeds to step S05; In step S05, the system controls the robot arm to transfer the bottled water to the vicinity of the person who issued the instruction, and prompts the person to complete the handover through voice. (If the instruction issued suggests giving the bottled water to another person, the robot arm will be controlled to transfer the bottled water to another designated location; for example, if the driver voice requests "give the bottle of water to the passenger seat", the person who will ultimately hand over the bottled water is the person sitting in the passenger seat).

3. The vehicle cockpit intelligent robotic arm device according to claim 1, characterized in that: When the robot arm needs to complete the object fixing, the following steps are included: In step S11, the driver or passenger uses the voice command "Help me fix the screen" to wake up the intelligent robotic arm and convey the service command "Fix the screen". If the intelligent robotic arm system recognizes the command, it proceeds to step S12. (It can also be used to fix common items such as mobile phones, PADs, makeup mirrors, microphones, books, etc.) In step S12, the controller of the intelligent robotic arm system senses the position of the screen in the car through the in-car camera sensor; if the position of the screen is recognized, the process proceeds to step S13; (The driver or passenger can also place the screen or other items into the palm of the robot or into the magnetic interface in the robot hand to complete the screen fixation) Step S13: The controller plans the trajectory of the robot arm for picking up and placing objects based on the orientation of the robot arm and the orientation of the screen; And use the in-car camera or radar to sense whether there are obstacles in the trajectory; If there is no obstacle in the trajectory, the process proceeds to step S14. (If there is an obstacle in the object transfer trajectory, the system automatically identifies whether the obstacle can be moved; if it is identified that the obstacle can be moved, the system prompts the occupants of the vehicle to "move the obstacle" through a voice prompt.) If the person inside the car responds with a voice message "the obstacle can be moved", the robot arm will move the obstacle away). In step S14, the system controls the robot arm to move near the screen and use its grippers or fingers to complete the object-grabbing operation. If the screen is accurately grasped, the system proceeds to step S15. (If the screen has a magnetic interface, the robot arm's magnetic interface can be paired with the screen's interface to complete the screen fixation.) In step S15, the system controls the robot arm to move the screen to the vicinity of the person who issued the instruction, and prompts the person to complete the item handover through sound (if the instruction is issued without a handover action, the robot arm will keep the screen in a fixed position and wait for subsequent instructions from the person in the car).

4. The intelligent robotic arm device for a car cockpit according to claim 1, characterized in that: When the robot arm needs to store items, the following steps are included: In step S21, the driver or passenger uses the voice command "help me put away the food" to wake up the intelligent robotic arm and convey the service command of "put away the food". If the intelligent robotic arm system recognizes the command, it proceeds to step S22. (It can also realize the storage of common items such as cosmetics, books, and documents) In step S22, the controller of the intelligent robotic arm system senses the location of all food items in the vehicle through the in-vehicle camera sensor; if the location of the food items is recognized, the process proceeds to step S23 (if multiple food items are detected to need to be moved, the items are moved one by one according to the priority set by the system); Step S23: The controller plans the trajectory of the robot arm for picking up and placing the food based on the position of the robot arm and the position of the food; The system detects whether there are obstacles in the trajectory through the in-car camera or radar; if there are no obstacles in the trajectory, it proceeds to step S24. (If there are obstacles in the object transfer trajectory, the system automatically identifies whether the obstacles can be moved; if it is identified that the obstacles can be moved, it prompts the occupants of the vehicle through a voice prompt "Do you want to move the obstacles?") If the person in the car replies "the obstacle can be moved", the robot arm will move the obstacle. If there is no one in the car, the system will independently determine whether to move the obstacle). In step S24, the system controls the robot arm to move near the food and use the robot arm's grippers or fingers to complete the object picking operation; if the food is accurately grabbed, the process proceeds to step S25; In step S25, the system controls the robot to transfer the food to the vicinity of the storage box; if the food is accurately transferred to the designated area, the process proceeds to step S26 (the food storage box can be a refrigerator or other storage box, or a specific area in the vehicle); Step S26: The system issues an instruction to open the storage box and put the food into the storage box; After all the transferable food is completed, an instruction is issued to close the storage box door (the door can also be opened / closed by a robot).

5. The vehicle cockpit intelligent robotic arm device according to claim 1, characterized in that: When the robot needs to realize garbage sorting and collection, the following operation steps are included: In step S31, the driver or passenger can use the voice command "help me pick up the trash" to wake up the intelligent robotic arm and convey the service command of "picking up the trash"; if the intelligent robotic arm system recognizes the command, it proceeds to step S32; In step S32, the controller of the intelligent robotic arm system senses the location of all garbage in the vehicle through the camera sensor inside the vehicle; if the location of the garbage is recognized, the process proceeds to step S33 (if it senses that multiple garbage, food, etc. need to be transferred, the transfers are completed one by one according to the priority set by the system); Step S33: The controller plans the trajectory of the robot arm for picking up and placing objects based on the position of the robot arm and the position of the garbage; And use the in-car camera or radar to sense whether there are obstacles in the trajectory; If there is no obstacle in the trajectory, the process proceeds to step S34. (If there is an obstacle in the object transfer trajectory, the system automatically identifies whether the obstacle can be moved. If it is identified that the obstacle can be moved, the system prompts the occupants through a voice prompt, "Do you want to move the obstacle?") If the person in the car replies "the obstacle can be moved", the robot arm will move the obstacle. If there is no one in the car, the system will independently determine whether to move the obstacle). In step S34, the system controls the robot arm to move to the vicinity of the garbage and uses the robot arm's gripper or fingers to complete the object picking operation; if the garbage is accurately grabbed, the process proceeds to step S35; In step S35, the system controls the robot to move the garbage to the vicinity of the garbage bin; if the garbage is accurately transferred to the designated area, the process proceeds to step S36; (The system also supports garbage classification); In step S36, the system issues an instruction to open the trash can and put the garbage into the trash can; after all the transferable garbage is completed, an instruction is issued to close the door of the trash can (the door can also be opened / closed by a robot).

6. The vehicle cockpit intelligent robotic arm device according to claim 1, characterized in that: The intelligent robotic arm can use its palm or clamp to act as a holder for items like phones and tablets. The position of the items can be freely adjusted to meet the needs of different users. Furthermore, the robotic arm can provide charging services for electronic products using USB / Type-C ports, wireless charging ports, and magnetic ports.

7. The vehicle cockpit intelligent robotic arm device according to claim 1, characterized in that: The intelligent robotic arm can use its palm or clamps to store items such as cosmetics, food, beverages, mobile phones, wallets, certificates, documents, fruits, blankets, clothes, etc.; the in-car camera or the camera in the robotic arm is used to take pictures of the items in the car, and the image recognition algorithm of the controller is used to identify the items; the storage area can cover the front and rear seats and floors, the central control storage table, the front and rear side door storage boxes, the passenger glove box, the refrigerator, the armrest box, the ceiling storage box and other areas.

8. The vehicle cockpit intelligent robotic arm device according to claim 1, characterized in that: The intelligent robotic arm can use its palm to complete the operation of placing pieces on the chessboard, and play games such as Gobang and Chinese chess with the passengers; the chess game is photographed using a camera inside the vehicle or a camera inside the robotic arm, and the image recognition algorithm of the controller is used to interpret the chess game.

9. The vehicle cockpit intelligent robotic arm device according to claim 1, characterized in that: The robotic arm has an adaptive force adjustment function. When grasping objects of different materials (such as fragile glass cups and soft pillows), the pressure sensor feeds back pressure data in real time, and the controller dynamically adjusts the output force of the palm motor according to the preset pressure threshold to avoid damage to the objects. Its finger joints are made of flexible silicone material, which can cushion the impact force by deformation when accidentally coming into contact with people in the car, thereby improving safety in use.

10. The vehicle cockpit intelligent robotic arm device according to claim 1, characterized in that: The camera of the in-vehicle space perception device has multi-mode imaging capabilities. In addition to conventional visible light imaging, it also supports infrared thermal imaging. In low-light or night scenes, it can accurately identify the outlines and temperature distribution of people and objects in the car; the millimeter-wave radar uses frequency-modulated continuous wave technology, and the motion trajectory and speed monitoring accuracy of dynamic objects (such as a water bottle shaking while driving) can reach centimeters and centimeters / second, providing reliable environmental perception data for the precise operation of the robot arm.

Images