Community vegetable buying unmanned vehicle intelligent operation method and system

Abstract

The application discloses a community vegetable buying unmanned vehicle intelligent operation method and system, and belongs to the field of community intelligent service and remote control of unmanned carrying equipment. The application builds a 5G bidirectional communication link of the unmanned vehicle and a vegetable buying APP, constructs a closed loop operation system in four stages of preparation, driving, vegetable buying and returning, configures full remote control, autonomous driving and remote control vegetable buying double working modes, realizes remote inspection of dishes, OCR label identification, dish grabbing, voice intercom and code scanning settlement by relying on a vehicle-mounted video, a dexterous arm, a navigation and a voice module, and is provided with autonomous and remote control double returning control mechanisms. The application solves the defects that people with difficulty in movement are limited in buying vegetables and lack of on-site selection and purchase experience in online shopping, realizes remote unmanned operation of the whole process of buying vegetables, and is convenient for community landing and popularization.

Description

Technical Field

[0001] This invention belongs to the field of remote control of unmanned transportation equipment and intelligent community services, specifically relating to an intelligent method and system for remote grocery shopping based on unmanned community vehicles. Background Technology

[0002] Pursuing fresh and safe ingredients and scientifically balanced diets are basic needs for residents to improve their quality of life. In addition to the traditional online shopping and home delivery model, going to supermarkets and farmers' markets near the community to buy groceries has become a mainstream lifestyle. When buying groceries at supermarkets, residents can directly compare the types, prices, and quality of similar ingredients and check the production dates. When purchasing groceries at farmers' markets, residents can promptly select seasonal fresh produce and also communicate and negotiate prices with vendors face-to-face.

[0003] However, this type of offline on-site grocery shopping model requires users to personally go to the store to select and purchase groceries, which has obvious limitations for people with mobility difficulties, the elderly, busy mothers and working people. Traveling is time-consuming and laborious, travel is inconvenient in bad weather, and long-distance purchasing is costly.

[0004] Current online delivery models mostly rely on platform-wide unified order fulfillment, lacking the immersive experience and autonomy of on-site selection, detailed verification of ingredients, and real-time interaction with merchants. Existing unmanned delivery equipment can only achieve point-to-point transport, lacking the capabilities for a closed-loop operation encompassing remote visual selection, remote-controlled mechanical grasping, remote intercom communication, online self-checkout, and intelligent autonomous return. This results in a limited model, poor adaptability to various operating conditions, and a lack of human-machine interaction. Therefore, providing home-based users with the convenience, immersive experience, and real-time interaction of grocery shopping near their communities, enabling remote self-selection, detailed verification of ingredients, and scientific purchasing, while replacing manual labor in the entire grocery shopping process, is a pressing technological need that requires immediate attention. Summary of the Invention

[0005] This invention addresses the technical problems of limited travel for community residents to buy groceries offline, lack of immersive interaction in remote grocery shopping, and inability to independently and precisely select groceries. It provides a smart operation method and system for community grocery shopping based on unmanned vehicles. Technical solution

[0006] In a first aspect, embodiments of the present invention provide an intelligent operation method for unmanned vehicles used for grocery shopping in communities, the method comprising a preparation stage, a driving stage, a grocery shopping stage, and a return stage: (1) Preparation stage: The grocery shopping unmanned vehicle establishes a two-way communication link with the grocery shopping unmanned vehicle application APP. The unmanned vehicle performs a power-on self-test and reports the device status to the APP. The APP configures the full remote control mode or autonomous driving + remote grocery shopping mode according to the user's selection. In autonomous driving mode, the APP generates or loads the navigation path and sends it to the unmanned vehicle. If the path is abnormal, it automatically switches to remote control mode. In remote control mode, it supports recording and storing the driving path. (2) Driving stage: The unmanned vehicle performs driving operations according to the selected mode. In remote control mode, the APP sends driving instructions to achieve closed-loop control and synchronously transmits video, vehicle speed and location information. In autonomous mode, it drives autonomously according to the navigation path. After arriving at the grocery shopping location, it sends an arrival prompt to the APP and automatically switches to remote grocery shopping mode. (3) Grocery shopping stage: The APP controls the unmanned vehicle to extend its dexterous arm and turn on the camera to form a video monitoring of the vegetables; the user adjusts the posture of the dexterous arm and zooms the camera through the APP to view the details of the vegetables; the APP performs OCR recognition on the vegetable labels and displays them, and supports two-way voice communication with the merchant; after confirming the vegetables, the APP controls the gripper to complete the grabbing and one-click basket filling; after the purchase is completed, the unmanned vehicle is controlled to enter the checkout area, and payment is completed by remotely scanning the code through the APP. (4) Return stage: The APP sends a one-click collection command to control the dexterous arm and camera to reset; the user selects remote return or autonomous navigation return mode; the unmanned vehicle travels to the starting point in return mode, and after arriving, it shuts down and cuts off the power, completing the closed loop of grocery shopping operation.

[0007] Secondly, embodiments of the present invention provide a community grocery delivery unmanned vehicle intelligent operation system, the system including a grocery delivery unmanned vehicle and a grocery delivery unmanned vehicle application APP, the two realizing bidirectional data interaction through a 5G communication module: The unmanned grocery shopping vehicle includes a small wheeled chassis, a dexterous arm, grippers, a control unit, an audio-visual unit, and a 5G communication module. The small wheeled chassis integrates a computing module, a positioning and navigation module, an environmental perception module, and a power supply module. The audio-visual unit includes an audio-visual encoding / decoding module, a driving video acquisition module, a dexterous arm grocery shopping video acquisition module, and a speaker. The unmanned grocery shopping vehicle application (APP) includes a vehicle status monitoring module, a vehicle driving remote control module, an audio interaction module, a food monitoring video module, a dexterous arm activity control module, a gripper activity control module, and a tag information recognition module. The control unit receives and parses APP commands, scheduling the various modules to collaboratively complete driving, video transmission, voice communication, food grabbing, path execution, and reset / return operations. Beneficial effects

[0008] Compared with existing technologies, this invention has significant technological advancements and engineering application advantages, as detailed below: First, this invention targets the daily fresh food purchasing scenarios of community residents, and builds a fully intelligent unmanned grocery shopping system to realize unmanned and remote grocery shopping operations, greatly expanding the applicable scenarios.

[0009] This invention innovatively constructs a standardized four-stage operation process: preparation, driving, grocery shopping, and return. Relying on a two-way communication and interaction mechanism between the unmanned vehicle and a mobile app, it achieves closed-loop control of the entire grocery shopping task, from initiation, driving and purchasing, on-site selection and settlement, to return and reset. Compared to the traditional manual offline purchasing model, this invention completely avoids the limitations of manual travel and on-site duty for purchasing, effectively reducing labor time costs and travel risks. It can be adapted to various restricted scenarios such as inclement weather, busy commutes, and people with mobility impairments, greatly improving the versatility of intelligent grocery shopping operations.

[0010] Secondly, this invention is designed for the daily fresh food purchasing scenarios of community residents, and adopts a compatible architecture with dual driving mode and dual return mode to improve the system's adaptability to operating conditions and overall driving reliability.

[0011] This invention features a dual-operation mechanism: a fully remote-controlled driving mode and an autonomous navigation driving mode. It also supports both remote-controlled return and one-click autonomous return strategies. For complex road conditions and unexpected obstacles, manual remote-controlled closed-loop control ensures driving accuracy and safety. For regular road sections, autonomous intelligent cruising is achieved using path planning algorithms and SLAM positioning modules. This invention, through a dual-mode dynamic switching mechanism, balances controllability in complex scenarios with operational efficiency in regular scenarios, effectively improving the fault tolerance and environmental adaptability of the unmanned vehicle driving system.

[0012] Third, this invention addresses the daily fresh food purchasing scenarios of community residents by building a visualized remote sensing and human-computer interaction system to solve the technical shortcomings of insufficient precision in remote grocery shopping operations.

[0013] This invention utilizes an in-vehicle video acquisition module, a zoom adjustment mechanism, OCR food label recognition technology, and a two-way voice intercom module to construct a fully visualized remote shopping operation chain. Users can obtain high-definition detailed images of food items in real time via a mobile app, automatically identify food specifications, and communicate remotely with merchants through two-way voice interaction. This solves the technical shortcomings of existing unmanned grocery shopping devices, such as low visualization of the shopping process, lack of detailed recognition, broken human-computer interaction links, and poor product selection accuracy, significantly improving the accuracy and intelligence of remote grocery shopping.

[0014] Fourth, this invention is designed for the daily fresh food purchasing scenarios of community residents, integrating multi-module collaborative self-inspection and intelligent execution mechanisms to improve the overall operational stability and integrity of the system.

[0015] This invention incorporates hardware self-checking, communication link verification, and navigation status detection mechanisms in the pre-operation phase. These mechanisms can proactively identify potential risks such as communication anomalies, module failures, and insufficient power, ensuring reliable operation startup. Simultaneously, it integrates intelligent execution functions such as precise grasping by a dexterous arm, automatic basket loading, remote barcode scanning and settlement, path recording and storage, and one-click equipment reset. The entire process requires no manual on-site intervention, resulting in a highly automated and seamless workflow. This effectively reduces operational error rates and equipment failure rates, significantly improving the overall system's operational safety and robustness.

[0016] Fifth, this invention is geared towards the daily fresh food purchasing scenarios of community residents. The method has a simple architecture, strong portability, low engineering implementation cost, and good prospects for industrial application.

[0017] The method proposed in this invention can be implemented using a conventional unmanned vehicle drive-by-wire chassis, positioning and navigation module, video acquisition equipment, and mobile software system. It does not require dedicated customized hardware or high-cost modification equipment, making it easy to deploy and highly replicable. The overall control logic is clearly layered and the steps are highly modular, facilitating engineering transfer, iterative upgrades, and mass deployment, thus possessing excellent engineering practice value and market prospects. Attached Figure Description

[0018] Figure 1 is a block diagram of a community grocery delivery unmanned vehicle according to an embodiment of the present invention.

[0019] Figure 2 is a block diagram of the functional modules of the grocery shopping unmanned vehicle APP according to an embodiment of the present invention.

[0020] Figure 3 is a timing diagram of the interaction during the preparation stage according to an embodiment of the present invention.

[0021] Figure 4 is a timing diagram of the driving phase interaction according to an embodiment of the present invention.

[0022] Figure 5 is a timing diagram of the interaction during the grocery shopping stage according to an embodiment of the present invention.

[0023] Figure 6 is a timing diagram of the return phase interaction according to an embodiment of the present invention.

[0024] Figure 7 is a flowchart of the overall business process of the community grocery delivery unmanned vehicle intelligent operation described in this invention.

[0025] The attached diagram is briefly described as follows: Figure 1 This is a block diagram of the community grocery delivery unmanned vehicle of the present invention, showing the main hardware modules that make up the grocery delivery unmanned vehicle; Figure 2 This is a block diagram of the functional modules of the grocery shopping unmanned vehicle APP of the present invention, showing the functional modules included in the APP; Figure 3Figure 4 is a timing diagram of the interaction between the APP and the unmanned grocery shopping vehicle during the preparation stage of the present invention, showing the interaction logic of communication establishment, mode selection, path configuration and mode switching; Figure 5 is a timing diagram of the interaction between the APP and the unmanned grocery shopping vehicle during the driving stage of the present invention, showing the command transmission and information feedback process of remote control and autonomous driving modes. Figure 5 This is a sequence diagram of the interaction between the APP and the unmanned grocery shopping vehicle in the grocery shopping stage of an embodiment of the present invention, which shows the intelligent grocery shopping interaction process of remote video inspection, vegetable recognition, voice intercom, vegetable grabbing and online settlement; Figure 6 This is a timing diagram of the interaction between the APP and the unmanned grocery delivery vehicle during the return phase of an embodiment of the present invention, illustrating the closed-loop return interaction of device reset, remote control return and autonomous return dual-path control, and arrival stop. Figure 7 This is a flowchart illustrating the overall business process of the community grocery delivery unmanned vehicle intelligent operation described in this embodiment of the invention. It is used to explain the complete operation process from equipment startup, mode selection, driving and purchasing, grocery selection, online settlement to returning and shutting down. Detailed Implementation

[0026] Referring to Figures 1 and 2, the system described in this invention mainly consists of two parts: an unmanned grocery shopping vehicle and an unmanned grocery shopping vehicle application APP. It is adapted to the local fresh food procurement scenario in the community and realizes an intelligent closed-loop operation for home users to remotely and independently purchase groceries.

[0027] 1.1 Composition of the unmanned grocery delivery vehicle The components of the unmanned grocery delivery vehicle are shown in Figure 1, and mainly include: (1) Small wheeled unmanned vehicle chassis: integrates computing module, positioning and navigation module, environmental perception module and power supply module, providing unmanned vehicle with driving, environmental perception, positioning and navigation and energy supply capabilities, and is the motion control carrier of unmanned vehicle; (2) Dexterous arm: a controlled actuator that can adjust its posture under the control of APP commands, and cooperate with the camera to achieve close-range inspection and grabbing of vegetables; this invention is aimed at the fresh food purchase scenario of community home users, and completes intelligent grocery shopping operation by relying on the short-distance driving environment in the community.

[0028] (3) Gripper: An end effector installed at the end of the dexterous arm, used to grab the target food and put it into the basket; (4) Control Unit: The main controller of the system, responsible for receiving and parsing APP instructions and controlling the coordinated operation of each module; (5) Audio and video unit: including audio and video encoding and decoding module, driving video acquisition module, dexterous arm grocery shopping video acquisition module and speaker, used to transmit driving / grocery shopping images and realize voice intercom; (6) 5G communication module: As a communication interface, it realizes bidirectional data transmission with the grocery shopping APP, and transmits control commands, status data and audio and video streams.

[0029] All modules work collaboratively under the unified scheduling of the control unit: the small wheeled unmanned vehicle chassis travels according to the positioning and navigation module and path planning instructions, the environmental perception module collects road information to assist in obstacle avoidance, the dexterous arm and gripper complete the food grabbing according to the APP instructions, the audio and video unit transmits images back and realizes voice interaction, and the 5G communication module ensures the stability of the data link.

[0030] 1.2 Composition of the Grocery Shopping Unmanned Vehicle Application App The self-service grocery delivery app, as shown in Figure 2, mainly includes: (1) Unmanned vehicle status monitoring module: used to receive and display the real-time status information of the grocery delivery unmanned vehicle, including battery level, location, driving status, etc.; (2) Unmanned vehicle driving remote control module: sends driving control commands to the grocery delivery unmanned vehicle to realize forward, backward, left, right and speed control, and supports path recording and storage; (3) Audio interaction module: used to receive and decode the audio data transmitted back by the grocery delivery vehicle, and at the same time encode the user's voice and send it to the vehicle to realize two-way voice communication; (4) Vegetable monitoring video module: Receives and displays the driving video and vegetable buying video transmitted back by the unmanned vegetable buying vehicle and the smart arm, providing users with a visual selection screen; (5) Dexterous arm activity control module: sends dexterous arm posture adjustment commands to the grocery delivery unmanned vehicle to control the tilt, rotation and zoom of the dexterous arm; (6) Gripper Activity Control Module: Sends gripper control commands to the unmanned grocery delivery vehicle to control the opening and closing of the gripper, thereby enabling the grabbing and release of vegetables; (7) Tag information recognition module: Perform OCR recognition on the video footage of the vegetables transmitted back by the unmanned grocery shopping vehicle, extract and display the vegetable tag information.

[0031] As a user interaction terminal, the APP enables full-process control and status monitoring of the unmanned grocery delivery vehicle through the aforementioned modules. Users can complete mode selection, route planning, grocery selection, and payment operations through the APP interface.

[0032] 2. Specific methods When the system described in this invention is working, the unmanned grocery shopping vehicle and the grocery shopping app establish a connection through a 5G communication link, and sequentially perform collaborative operations in the preparation, driving, grocery shopping, and return phases: the app sends control commands, the unmanned grocery shopping vehicle receives and executes them, and simultaneously transmits status data and audio / video streams back; the app adjusts the control commands based on the feedback, forming a closed-loop control; during the driving process, the unmanned grocery shopping vehicle achieves path navigation and environmental perception; in the grocery shopping phase, it uses a dexterous arm and grippers to grasp the produce; in the return phase, it automatically resets and returns to its starting point, ultimately completing the remote grocery shopping task. The specific embodiments of this invention will be described in detail below with reference to Figures 3-6.

[0033] 2.1 Preparation Phase This phase involves the startup initialization, status self-check, working mode configuration, and route preparation of the unmanned grocery delivery vehicle and its app, establishing a stable communication link and operational foundation for the subsequent driving phases. The interaction sequence diagram between the unmanned grocery delivery vehicle and its app in this phase is shown below. Figure 3 As shown. Specifically, it includes the following steps: Step 101: System Startup and Communication Link Establishment The user starts the unmanned grocery delivery vehicle and its application app. After powering on, the unmanned vehicle automatically performs system initialization and hardware self-test, including initialization and functional verification of the power module, communication module, navigation module, video acquisition module, and driver module. After the app starts, it automatically establishes a two-way communication link with the unmanned grocery delivery vehicle and sends status check command messages to the vehicle.

[0034] Step 102: Device Status Interaction and Self-Test Feedback After receiving a status check command from the app, the self-driving grocery delivery vehicle returns current device status information to the app, including remaining battery power, navigation module readiness status, communication link quality, and video module operational status. The app then verifies the device status based on the returned information. If the self-test status is abnormal (such as communication link interruption, video module failure, insufficient power, etc.), the APP generates a device status alarm signal to prompt the user to troubleshoot and maintain the problem; the grocery delivery unmanned vehicle enters standby mode in abnormal status, and the APP re-initiates the status check process after the user completes the maintenance. If the self-test is normal, the APP will enter the work mode selection interface, where users can select the work mode.

[0035] Step 103: Configure and confirm the working mode Users select a working mode via the APP. The working modes include two types: full remote control mode and autonomous driving + remote control grocery shopping mode. The APP sends a working mode setting instruction message to the grocery shopping unmanned vehicle. After receiving the instruction, the grocery shopping unmanned vehicle completes the working mode configuration and returns a current working mode confirmation message to the APP.

[0036] Step 104: Route preparation for autonomous driving + remote grocery shopping mode When a user selects the autonomous driving + remote grocery shopping mode, the app uses its built-in route planning algorithm to automatically generate a navigation route data package based on the user's chosen intended grocery shopping location. If the navigation path is successfully generated, the APP will send the generated navigation path data packet to the grocery delivery vehicle; after receiving the path data, the grocery delivery vehicle will complete the autonomous navigation path loading, parsing and verification, and return a path loading status confirmation message to the APP; after receiving the feedback that the loading is successful, the APP will send a start autonomous departure command message to the grocery delivery vehicle, and after receiving the command, the grocery delivery vehicle will start autonomous driving. If navigation path generation fails, the app automatically reads the preset path dataset stored locally. If an available preset path exists, the app selects the preset path and sends it to the grocery delivery vehicle. The grocery delivery vehicle loads the path and executes subsequent steps. If no preset path exists, the app generates a path generation failure alarm signal and forcibly switches to full remote control mode.

[0037] Step 105: Path preparation for full remote control mode When the user selects the full remote control mode, the app enters the remote control mode interface, where the user can choose whether to record the driving route: If the user selects a recording path, the APP will start the path recording function and flash the prompt button to indicate the recording status to the user; while the grocery delivery vehicle is driving, it will upload GNSS positioning data to the APP in real time, and the APP will store the location data as a trajectory record file; the user can end the recording at any time through the button in the APP, and enter the location name to save the trajectory as a preset path; If the user does not record the route, the APP will directly enter remote control mode. The user can send speed and direction control commands through the APP, and the unmanned grocery delivery vehicle will receive the commands and execute the driving actions through the 5G communication module. At the same time, it will send back road environment video stream and location data to the APP in real time. The APP will determine whether the unmanned grocery delivery vehicle has reached the user's intended location based on the location data.

[0038] 2.2 Driving Phase This phase enables remote driving control, status monitoring, and arrival confirmation of the unmanned grocery shopping vehicle from its starting point to the target shopping location, providing a reliable location foundation for subsequent grocery shopping stages. The interaction sequence diagram between the unmanned grocery shopping vehicle app and the unmanned grocery shopping vehicle in this phase is shown below. Figure 4 As shown. Specifically, it includes the following steps: Step 201: Mode Branch Execution. Based on the working mode selected in the preparation phase, proceed to the corresponding driving process: If it is in full remote control driving mode, proceed with steps 202-203; If it is in autonomous driving mode, proceed to steps 204-205.

[0039] Step 202: In the closed-loop control mode of full-process remote driving, the APP sends a set of driving control commands to the grocery delivery unmanned vehicle, including forward, backward, left, and right directional control signals and speed control signals. After receiving the remote control commands, the grocery delivery unmanned vehicle parses and drives the drive-by-wire chassis actuators to adjust its driving posture, achieving controlled driving. During driving, the grocery delivery unmanned vehicle transmits road environment video streams, vehicle speed information, and GNSS positioning data back to the APP in real time through the onboard video acquisition module and positioning module. The APP receives and decodes the data, rendering the driving video, vehicle speed, and location information on the display interface. Users can adjust control commands based on real-time feedback, forming a closed-loop remote control link. If the user enables the path recording function, the APP will synchronously save the received location information to the path file. When the grocery delivery unmanned vehicle remotely drives to the designated location, the APP ends path recording and saves the path file, completing the driving control.

[0040] Step 203: Arrival Confirmation for Remote Driving. Based on the GNSS positioning data transmitted back by the unmanned grocery shopping vehicle, the app determines whether the vehicle has arrived at the user's intended location. Upon arrival, the app generates an arrival notification signal to inform the user that the trip has ended, and the user can switch to remote grocery shopping mode through the app.

[0041] Step 204: Path Execution and Status Feedback in Autonomous Driving Mode. The grocery-buying unmanned vehicle navigates autonomously based on the navigation path data package sent by the APP, using the SLAM positioning module and path planning algorithm. During the journey, the unmanned vehicle sends real-time road environment video streams, vehicle speed data, and location data to the APP via a video acquisition module, vehicle speed sensor, and positioning module. The APP receives and displays the driving video and speed data, monitoring the driving status throughout the process. If the unmanned vehicle has not reached the designated location, it continues to drive along the navigation path and continuously transmits status data; if it has reached the designated location, the unmanned vehicle stops driving and sends an arrival confirmation message to the APP.

[0042] Step 205: Arrival Confirmation and Mode Switching for Autonomous Driving After receiving the arrival confirmation message from the unmanned grocery shopping vehicle, the APP displays a prompt message indicating that it has arrived at the designated location and emits a prompt sound; then the APP takes over the unmanned grocery shopping vehicle and automatically switches to remote grocery shopping mode. Users can control the unmanned vehicle to move to the designated grocery shopping area through the remote control commands of the APP and enter the grocery shopping stage.

[0043] 2.3 Grocery Shopping Stage This phase utilizes two-way communication between the unmanned grocery delivery vehicle and the grocery delivery app to achieve a closed-loop operation encompassing remote viewing of produce, precise identification, accurate retrieval, remote communication, and online payment. The sequence diagram of the interaction between the unmanned grocery delivery vehicle and the app in this phase is shown below. Figure 5 As shown. Specifically, it includes the following steps: Step 301: Dexterous arm initialization and video capture of the food area Users send control signals to the self-driving grocery delivery vehicle via a grocery shopping app, requesting the extension of its dexterous arm and the activation of its camera. Upon receiving the commands, the vehicle extends its arm and activates its camera module, creating a real-time video feed of the produce. Users can further send motion control commands to the dexterous arm via the app, adjusting its pitch, horizontal, and orientation (up, down, left, right, forward, and backward) to obtain a panoramic video view of the produce area. The self-driving vehicle receives these commands, controls the dexterous arm's movement, and transmits the video stream of the produce area back to the app in real time.

[0044] Step 302: Detailed inspection of dishes and identification of label information Users view the "vegetable video" on the app. After finding a dish of interest, they send a remote control command to the unmanned grocery delivery vehicle, guiding it to the corresponding dish area. The user then sends a camera zoom control signal to the vehicle via the app. Upon receiving the command, the vehicle controls the camera to zoom in and out, allowing for a closer look at the dish's details. The vehicle's camera module automatically recognizes the dish label information and transmits the video footage containing the label text back to the app. The app then uses OCR to extract and display the label text from the video footage. Users can also enable the voice playback function to listen to the label information as needed.

[0045] Step 303: Remote intercom interaction (optional step) To communicate with the salesperson, users can activate the voice intercom function through the app and send a speaker activation control signal to the self-service grocery delivery vehicle. Upon receiving the command, the vehicle turns on its speaker and simultaneously captures the salesperson's voice through a microphone, encodes it, and transmits it to the app. The app decodes the voice signal and plays it back to the user. The user's voice signal is also encoded by the app and transmitted to the self-service grocery delivery vehicle, where it is played back through the speaker, enabling two-way voice communication. After the communication ends, the user sends a speaker deactivation control signal to the self-service grocery delivery vehicle through the app, and the vehicle deactivates its speaker upon receiving the command.

[0046] Step 304: Picking and Placing the Vegetables into the Basket After the user confirms the desired item, they send a control message to the automated grocery delivery vehicle (AGV) via the app, remotely controlling the arm to approach the item. Then, the user sends a gripper control message via the app, instructing the gripper to open and grab the item. Once grabbed, the user sends a one-click input box control message to the AGV via the app. Upon receiving the command, the AGV moves its arm to above the delivery basket according to preset coordinates, opens the gripper to release the item, and after it falls into the basket, the arm automatically returns to its initial position. The AGV then sends a confirmation message to the app confirming the grabbing completion.

[0047] Step 305: Control of Menu Selection and Settlement Process Users can decide whether to continue buying groceries based on the completed information provided by the app: If the user continues to buy groceries, they can repeat steps 301 to 304 to view, identify, and retrieve the next item. If the user no longer needs to buy groceries, they can send remote driving information to the unmanned grocery shopping vehicle via the app to control the vehicle to drive to the checkout area. Once the unmanned grocery shopping vehicle arrives at the checkout area, it controls the dexterous arm camera to face the cashier.

[0048] Step 306: Remote QR code payment The sales clerk tallies the menu items and shows the payment code to the camera on the unmanned grocery delivery vehicle; the camera on the unmanned vehicle captures the video of the payment code and transmits it back to the app; after receiving the video, the app extracts the payment code information, and the user completes the payment by scanning the code on the app; after the payment is completed, the user sends remote driving information to the unmanned grocery delivery vehicle through the app, controlling the vehicle to leave the payment area and enter the return phase.

[0049] 2.4 Return Phase This phase involves the automated grocery delivery vehicle returning from the checkout area to its starting point, including device reset, mode configuration, closed-loop driving control, arrival confirmation, and task completion. The interaction sequence diagram between the grocery delivery vehicle app and the automated grocery delivery vehicle in this phase is shown below. Figure 6As shown. Specifically, it includes the following steps: Step 401: Device Reset and Driving Status Reconstruction. The user triggers the "One-Click Favorite" command through the grocery shopping APP. The APP sends a dexterous arm reset and camera shutdown control signal to the grocery shopping unmanned vehicle through the communication link. After receiving the command, the grocery shopping unmanned vehicle performs the dexterous arm posture return to zero and the camera module power-off operation, retracts the dexterous arm into the vehicle body storage slot, restores the unmanned vehicle to the initial configuration in the driving state, and returns a "Favorite Complete" status confirmation message to the APP.

[0050] Step 402: Return Route Mode Configuration and Route Preparation. The app provides users with a return route mode selection interface, where users can choose between remote control return route mode or autonomous navigation return route mode. If the user selects the remote return mode, proceed to step 403; If the user selects the autonomous navigation return mode, proceed to steps 404-406.

[0051] Step 403: Closed-Loop Driving Control in Remote Control Return Mode. The user sends a set of driving control commands to the unmanned grocery delivery vehicle via the app, including forward, backward, left, and right directional control signals and speed control signals. Upon receiving the remote control commands, the unmanned grocery delivery vehicle parses and drives the drive-by-wire chassis actuators to adjust its driving posture, achieving controlled driving. During driving, the unmanned grocery delivery vehicle transmits real-time road environment video streams, vehicle speed information, and GNSS positioning data back to the app via its onboard video acquisition and positioning modules. The app receives and decodes the data, rendering the driving video, vehicle speed, and location information on the display interface. The user can adjust control commands based on real-time feedback, forming a closed-loop remote control link.

[0052] Step 404: Path Configuration and Verification in Autonomous Navigation Return Mode. After the user selects autonomous navigation return mode, the APP reads the locally stored return path dataset. If an available path exists, the APP sends a mode switching command and navigation path data packet to the unmanned grocery delivery vehicle via the communication link. After receiving the data, the unmanned grocery delivery vehicle completes the working mode switch, loads the navigation path into the autonomous navigation controller, completes path parsing and verification, and then returns a confirmation message to the APP stating "Mode switching completed, path loaded successfully." If no available navigation path exists, the APP generates a "path missing alarm signal," prompting the user to switch to remote control return mode.

[0053] Step 405: Autonomous Driving and Status Monitoring in Autonomous Navigation Return Mode. After receiving feedback that the path has been successfully loaded, the APP sends a "Start Autonomous Return" control command to the grocery delivery driverless vehicle. Upon receiving the command, the grocery delivery driverless vehicle autonomously drives along the navigation path based on the SLAM positioning module and path planning algorithm. During the journey, the grocery delivery driverless vehicle sends real-time road environment video streams, vehicle speed data, and positioning data to the APP through the video acquisition module, vehicle speed sensor, and positioning module. The APP receives and displays the data, providing full monitoring of the driverless vehicle's driving status.

[0054] Step 406: Arrival Confirmation and Task Completion. After the unmanned grocery shopping vehicle reaches the preset destination, the drive-by-wire chassis actuator stops driving and sends an arrival confirmation message to the APP via the communication module. After receiving the message, the APP generates an arrival prompt message on the interface and triggers a prompt sound. After the user confirms arrival, they send a shutdown command to the unmanned grocery shopping vehicle via the APP. After receiving the command, the unmanned grocery shopping vehicle performs system shutdown and power-off operations. The user takes the groceries from the unmanned grocery shopping vehicle's basket, completing the grocery shopping task.

Claims

1. A community grocery shopping unmanned vehicle intelligent operation method, characterized in that, Includes the following steps: S1: Preparation phase: The grocery shopping unmanned vehicle is powered on, performs a self-test, and establishes a communication connection with the grocery shopping unmanned vehicle APP. The APP configures the working mode according to the user's instructions, including full remote control mode or autonomous driving + remote grocery shopping dual mode; if the autonomous driving mode is selected, the APP generates or loads the preset navigation route and sends it to the unmanned vehicle. If path generation fails or there is no preset path, the APP will force a switch to remote control mode. S2: Driving Phase: The driverless car drives according to the configuration mode. In remote control mode, the APP sends driving commands to control the movement of the driverless car, and can also record the driving path and save it as a preset path. In autonomous mode, the driverless car drives autonomously according to the issued navigation path, and sends an arrival prompt to the APP after arriving at the designated grocery store location. S3: Grocery Shopping Stage: The APP sends commands to the autonomous vehicle to extend the dexterous arm and activate the camera, creating a video monitoring the produce. Users can adjust the dexterous arm's posture and control the camera's zoom via the APP to view the details of the produce and extract the product label information through OCR recognition. If users need to communicate with merchants, they can enable the voice intercom function via the APP. After selecting the produce, the APP sends a grabbing command to control the gripper to grab the produce and place it into the vehicle's basket. S4: Settlement Phase: The unmanned vehicle travels to the settlement area, and the APP receives the payment code video transmitted by the unmanned vehicle. The user scans the code through the APP to complete the payment. S5: Return Phase: The APP sends a one-click save command to control the dexterous arm to reset. The user selects remote return or one-click autonomous return mode. The unmanned vehicle returns to the starting point according to the selected mode. After arriving, it performs a power-off operation, and the user takes out the food to complete the task.

2. The method of claim 1, wherein, In step S1, the self-test of the unmanned vehicle after powering on includes: power status, navigation module status, audio and video module status, dexterous arm status, and gripper status. After the self-test is completed, the device status information is returned to the APP.

3. The method of claim 1, wherein, In step S2, the specific process of path recording in remote control mode is as follows: the APP starts the recording function, saves the real-time location information of the unmanned vehicle to the path file, and ends the recording after reaching the designated location and saves it as a preset path. The user can customize the path name.

4. The method of claim 1, wherein, In step S3, the specific process for identifying dish label information is as follows: the APP takes a screenshot of the dish video footage transmitted back by the unmanned vehicle, automatically identifies and extracts the label text and displays it, and the user can choose to read the label information aloud.

5. The method of claim 1, wherein, In step S3, the voice intercom function is implemented as follows: the APP sends a speaker turn-on command to control the unmanned vehicle to turn on the sound pickup and speaker equipment; the user's voice is encoded by the APP and transmitted to the unmanned vehicle for playback; the merchant's voice is encoded by the unmanned vehicle and transmitted to the APP for decoding and playback; after the communication ends, the APP sends a turn-off command to control the unmanned vehicle to turn off the speaker.

6. The method according to claim 1, characterized in that, In step S3, the specific process of food grabbing is as follows: the APP sends a dexterous arm control command to adjust the dexterous arm above the food, and then sends a gripper control command to control the gripper to open and close to grab the food. After grabbing, the APP sends a one-click box command to control the dexterous arm to move above the basket and release the food, and then the dexterous arm resets.

7. A community vegetable buying unmanned vehicle intelligent operation system for implementing the method of claim 1. The system includes a self-driving grocery-buying vehicle and its accompanying app, which communicate bidirectionally via a 5G communication module. The self-driving grocery-buying vehicle comprises: a small wheeled chassis integrating a computing module, a positioning and navigation module, an environmental perception module, and a power supply module for driving, positioning, navigation, and environmental perception; a dexterous arm and grippers for adjusting posture and grasping produce, respectively; a control unit for receiving and parsing commands from the app and controlling the coordinated operation of the modules; an audio-visual unit including an audio-visual encoding / decoding module, a driving video acquisition module, a dexterous arm grocery-buying video acquisition module, and a speaker for transmitting video feeds and enabling voice communication; and a 5G communication module for data transmission with the app. The self-driving grocery-buying grocery-buying vehicle app... It includes: an unmanned vehicle status monitoring module, used to display the unmanned vehicle's battery level, location, and driving status information; an unmanned vehicle driving remote control module, used to send driving control commands, record and save driving paths; an audio interaction module, used to enable two-way voice communication between users and merchants; a food monitoring video module, used to display driving videos and food videos transmitted by the unmanned vehicle; a dexterous arm activity control module, used to send dexterous arm posture adjustment commands; a gripper activity control module, used to send gripper opening and closing control commands; and a label information recognition module, used to recognize and extract text information from food labels.

8. The system of claim 7, wherein, The positioning and navigation module of the unmanned grocery shopping vehicle supports autonomous path planning and preset path loading, and can drive autonomously according to the navigation path issued by the APP.

9. The system of claim 7, wherein, The dexterous arm video acquisition module of the unmanned grocery shopping vehicle can adjust the focal length, supporting magnified viewing of vegetable details.

Images