Collaborative fork truck system and fork truck control method

Abstract

A collaborative forklift system is disclosed, comprising: a user terminal configured to send forklift control commands to a server in response to receiving forklift control commands input by a user; a server configured to process the received forklift control commands, generate target handling tasks corresponding to at least one forklift, and send the corresponding target handling tasks to the forklifts; and a forklift configured to, upon receiving a target handling task, decompose the target handling task into a task sequence and execute the task sequence sequentially to complete the target handling task. This disclosure achieves semi-automatic control of the forklifts by efficiently understanding and decomposing the received user commands through the server and sending the decomposed tasks to the forklifts. Furthermore, by constructing diverse human-machine collaborative relationships, the system possesses sufficient flexibility and adaptability to various logistics scenarios, reducing costs while improving system operating efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent warehousing technology, and in particular to a collaborative forklift system and forklift control method. Background Technology

[0002] With the development of the smart warehousing industry, unmanned forklifts, due to their high degree of automation, can serve as a replacement for manual forklifts. However, unmanned forklifts have not rapidly captured the market, and their market penetration rate remains low. The main reasons are that existing unmanned forklifts and their systems are expensive and have low task execution efficiency. Furthermore, there is a contradiction between the automation design goals of unmanned forklifts and the high flexibility and high economic efficiency pursued by the warehousing and logistics industry, making it difficult for unmanned forklifts to adapt to the complex and ever-changing actual needs of warehousing operations. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a collaborative forklift system and forklift control method to solve the issues of high cost and low flexibility of current unmanned forklifts.

[0004] A first aspect of this disclosure provides a collaborative forklift system, comprising: a user terminal configured to send a forklift control command to a server in response to receiving a forklift control command input by a user; the server configured to process the received forklift control command based on the forklift control command, generate a target handling task corresponding to at least one forklift, and send the corresponding target handling task to the forklift; and the at least one forklift configured to, upon receiving the target handling task, decompose the target handling task into a task sequence, and execute the task sequence sequentially to complete the target handling task.

[0005] A second aspect of this disclosure provides a forklift control method, comprising: a user terminal, in response to receiving a forklift control command input by a user, sending the forklift control command to a server; the server, based on the received forklift control command, processing the forklift control command to generate a target handling task corresponding to at least one forklift, and sending the corresponding target handling task to the forklift; and the at least one forklift, after receiving the target handling task, decomposing the target handling task into a task sequence, and sequentially executing the task sequence to complete the target handling task.

[0006] A third aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the forklift control method provided in the second aspect above.

[0007] A fourth aspect of this disclosure provides an electronic device comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the forklift control method provided in the second aspect above.

[0008] The fifth aspect of this disclosure provides a computer program product that, when instructions in the computer program product are executed by a processor, performs the forklift control method provided in the second aspect above.

[0009] The collaborative forklift system disclosed herein receives forklift control commands input by the user through a user terminal. The server converts these commands into forklift-level tasks and sends them to the forklifts. Upon receiving the forklift-level tasks, the forklifts break them down into a task sequence and execute them sequentially to complete the handling task corresponding to the forklift control command. This disclosure employs a system architecture consisting of a user terminal, a server, and a forklift, enabling human-machine collaboration. The server can efficiently understand and break down the received user commands in real time and send the decomposed tasks to the forklifts, achieving semi-automatic control of the forklifts. Furthermore, the architecture of this disclosure can include multiple forklifts, allowing users to control multiple forklifts simultaneously through the user terminal, achieving diversified human-machine collaboration. This reduces labor costs in warehousing and logistics, and provides the entire system with sufficient flexibility and adaptability to various logistics scenarios. In particular, it can reduce or even eliminate the need for logistics site modifications, lower deployment costs, and improve the handling efficiency of logistics through human-machine collaboration. Attached Figure Description

[0010] Figure 1 This is a structural diagram of a collaborative forklift system provided in an exemplary embodiment of this disclosure.

[0011] Figure 2 This is a schematic diagram of the software architecture of a collaborative forklift system provided in an exemplary embodiment of this disclosure.

[0012] Figure 3 This is a schematic diagram of the control flow of a collaborative forklift system provided in an exemplary embodiment of this disclosure.

[0013] Figure 4 This is a schematic diagram of the control flow of a collaborative forklift system provided in another exemplary embodiment of this disclosure.

[0014] Figure 5 This is a flowchart illustrating a forklift control method provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0015] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0016] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0017] Application Overview In the field of smart warehousing, forklifts are widely used as a material handling equipment in warehousing systems. Depending on the degree of automation, forklifts can be divided into manual forklifts and unmanned forklifts. Manual forklifts require an operator to drive the forklift to handle goods, while unmanned forklifts can handle goods without an operator.

[0018] As the level of intelligence in the warehousing sector increases, more and more companies are investing heavily in the research and development and promotion of unmanned forklifts. However, the market penetration rate of unmanned forklifts remains low. This is due, in part, to the high cost of unmanned forklifts and their supporting systems, and in part, to the conflict between the automation design goals of unmanned forklifts and the high flexibility and economic efficiency pursued by the warehousing and logistics industry. This results in unmanned forklifts being unable to adapt to the complex and ever-changing actual needs of warehousing operations.

[0019] To address the issues of high cost and low flexibility of current unmanned forklifts, this disclosure provides a collaborative forklift system. The system receives forklift control commands from the user via a user terminal. A server converts these commands into forklift-level tasks and sends them to the forklifts. Upon receiving the forklift-level tasks, the forklifts break them down into a sequence and execute them sequentially to complete the transport task. Therefore, this disclosure employs a system architecture that enables human-machine collaboration, consisting of a user terminal, server, and forklift. The server efficiently understands and breaks down the received user commands in real time, sending the decomposed tasks to the forklifts for semi-automatic control. Furthermore, this architecture can include multiple forklifts, allowing users to control multiple forklifts simultaneously via the user terminal, achieving diversified human-machine collaboration. This reduces labor costs in warehousing and logistics, provides the system with high flexibility and adaptability to various logistics scenarios, and significantly reduces or eliminates the need for logistics site modifications, lowering deployment costs while improving transport efficiency through human-machine collaboration.

[0020] Exemplary System Figure 1 This is a schematic diagram of the architecture of a collaborative forklift system provided in an embodiment of this disclosure, such as... Figure 1As shown, the collaborative forklift system includes a user terminal 10, a server 20, and at least one forklift 30. Information between the user terminal 10 and the forklift 30 needs to be transmitted through the server 20.

[0021] For example, user terminal 10 can communicate with server 20 via the Internet, and server 20 can communicate with at least one forklift 30 via a local area network (LAN), wireless local area network (WLAN), or other network. This disclosure does not limit the number of forklifts included in the collaborative forklift system. When the collaborative forklift system includes multiple forklifts, multiple forklifts can be controlled by one user terminal 10, or multiple user terminals 10 can control multiple forklifts separately.

[0022] User terminal 10 can provide a human-machine interface to realize functions such as user authentication (e.g., operator authentication), task parameter configuration, control command issuance, equipment monitoring, and operation data visualization. User terminal 10 may include at least one of personal computers, laptops, smartphones, tablets, portable wearable devices, and dedicated input / output devices. This disclosure does not limit the specific type of user terminal; any device with functions such as interface display, command input, calculation processing, information storage, and mobile communication can be used as user terminal 10 in this disclosure embodiment.

[0023] In some examples, the user terminal 10 may include a display for showing a human-machine interface (HMI). Users can view relevant information (including forklift status information, user information (such as operator information), status information of each storage location in the inventory area, parameter information of the goods, statistical information such as task execution status) through the HMI of the user terminal 10. Users can also publish tasks, manage fleets, switch forklift working / running modes, adjust tasks, and issue any instructions.

[0024] For example, users can monitor real-time fleet data, real-time vehicle data, statistical data, etc. through user terminal 10, and make control task adjustment decisions based on the monitoring data, such as speed adjustment, cargo location adjustment, route adjustment, work / operation mode switching, task pause, task resumption, etc.

[0025] The fleet's real-time data includes the fleet's operational information, including but not limited to: the forklift number and corresponding status, task information, estimated completion time, battery level, and location information of each forklift in the fleet. This real-time fleet data can be displayed on the user interface of the user terminal 10 for easy viewing and management.

[0026] Real-time vehicle data includes the real-time operation status of at least one forklift in the fleet. Real-time vehicle data includes, but is not limited to: forklift number and corresponding status, task information, estimated completion time, battery information, sensor status, and sensor detection information.

[0027] The statistical data includes various types of information from user and fleet interactions, including but not limited to: user waiting time in a period before the current moment, forklift idle time, average number of tasks completed per forklift per hour, percentage of time spent in automatic mode and percentage of time spent in manual mode, etc.

[0028] Server 20 may include a standalone server or a server cluster consisting of multiple servers; this embodiment of the present disclosure does not limit this. Server 20 is used to implement functions such as forklift operation information collection, task scheduling and adjustment, traffic control and navigation path planning, and data storage and analysis. The server can communicate with multiple forklifts in the collaborative forklift system via a local area network.

[0029] Forklift 30 is used to perform various handling tasks, including inbound, picking, outbound, sorting, replenishment, and navigation tasks. This disclosure does not limit the type of handling task. During the handling process, forklift 30 can detect its status information and the surrounding environment information through sensors. The forklift 30's status information includes its operating status and the execution status of the corresponding handling task. The surrounding environment information includes information on obstacles (including other forklifts in the collaborative forklift system, operators, and racks), information on the goods to be handled, and information on the storage location of the goods to be picked up or placed.

[0030] For example, to ensure that the forklift can perform handling tasks efficiently and safely, the forklift 30 is equipped with various types of sensors. These sensors include not only sensors for sensing the forklift's surrounding environment and cargo-related information, such as lidar, cameras, and depth cameras, but also sensors for detecting the forklift's own motion state, such as inertial measurement units (IMUs), wheel speed sensors, draw rope sensors, and gravity sensors. This disclosure does not limit the types of sensors installed on the forklift 30; this is merely an illustrative example.

[0031] When forklifts are used to move goods, they can move the goods directly or move the pallets, cages, boxes, and original boxes on which the goods are placed. This disclosure does not limit this.

[0032] In some examples, the forklift 30 may also include a display for showing a human-machine interface. Through the human-machine interface on the forklift 30, users can view system-related information (including the forklift's working mode, user information, status information of each storage location in the inventory area, parameter information of the goods, statistical information such as task execution status), and can also issue tasks, switch modes, manage fleets, adjust tasks, and issue any instructions.

[0033] During task execution, forklift 30 can upload detected environmental perception information or forklift status information to server 20 at a preset frequency. After determining the target status information based on the environmental perception information and / or forklift status information, server 20 can upload the target status information to user terminal 10 at a preset frequency. After receiving the target status information sent by server 20, user terminal 10 can display the target status information through user interface, so that the user can decide whether to intervene in the task execution of forklift 30 based on the target status information.

[0034] In response to a user-input adjustment command received by user terminal 10, which may be used to adjust / modify task information or pause task execution, user terminal 10 sends the adjustment command to server 20 via the Internet after the user confirms the adjustment. Upon receiving the adjustment command, server 20 adjusts the forklift's handling task based on the system's status monitoring results and sends the adjusted handling task to the corresponding forklift 30 via the local area network, whereby forklift 30 executes the adjusted handling task.

[0035] The forklift 30 provided in this embodiment can operate in both automatic and manual modes under the control of the server receiving instructions triggered by the user terminal 10. It can also freely switch between automatic and manual modes depending on changes in the work scenario or the integration of manual mode. When the forklift 30 is operating in automatic mode, the user can send task information through the user terminal 10 to control the forklift 30, allowing it to automatically execute handling tasks as an unmanned forklift. In automatic mode, the user can simultaneously control multiple forklifts through the user terminal 10, enabling diversified human-machine collaboration. This reduces the per-person labor cost in warehousing and logistics operations, lowering labor costs and providing the entire system with sufficient flexibility and adaptability to various logistics scenarios.

[0036] When forklift 30 is unable to complete the handling task in automatic mode, in response to a command triggered by user terminal 10 or forklift 30 to switch the working mode from automatic to manual, forklift 30 switches from automatic mode to manual mode. When forklift 30 is working in manual mode, forklift 30 can perform handling tasks as a manually operated forklift under the driver's control, which can improve the success rate and efficiency of task execution.

[0037] This disclosed collaborative forklift system can be integrated with various application scenarios in warehousing systems and meet the needs of each scenario. When it is determined that the forklift's operating mode needs to be switched (e.g., manual intervention is required when the forklift is operating in automatic mode, or switching from manual mode to automatic mode), the user can input a switching command on the user terminal 10 or the forklift 30. Responding to the user's input switching command, the user terminal 10 or the forklift 30 can switch the forklift 30's operating mode, either from automatic to manual mode or vice versa. By freely switching the forklift's operating mode according to the operational needs of different scenarios, the safety, efficiency, and success rate of task execution can be improved.

[0038] Figure 2 This is a schematic diagram of the software architecture of a collaborative forklift system provided in an embodiment of the present disclosure, such as... Figure 2 As shown, the software modules deployed on user terminal 10 include a user login module 101, a fleet management module 102, an information monitoring module 103, and a task setting module 104. Through these software modules, users can log in, create fleets, and monitor information. They can also set tasks or display relevant information about the warehousing system through the user terminal 10's human-machine interface. The software modules deployed on user terminal 10 can run on hardware devices with processing capabilities, such as the processor of user terminal 10.

[0039] The software modules deployed on server 20 include an authentication module 201, a vehicle management module 202, a data statistics module 203, a task breakdown module 204, a scheduling optimization module 205, and a traffic control module 206. These software modules on server 20 can authenticate user identities and manage fleets of multiple forklifts in a unified manner, such as forklift information collection and storage, forklift task management and publishing, forklift scheduling, and traffic control. The software modules deployed on server 20 can run on the server 20's processor (such as a central processing unit (CPU) and / or a graphics processing unit (GPU)).

[0040] The software modules deployed on the forklift 30 include a system management module 301, a task management module 302, an environmental perception module 303, a behavior planning module 304, a motion control module 305, and basic software function modules for related sensors and actuators in the equipment driver layer 306. These software modules enable functions such as task management and behavior decision-making, environmental perception, navigation, and fork control. The software modules on the forklift 30 can run on the forklift 30's main controller, which may include hardware devices such as a processor, graphics card, and communication interface.

[0041] like Figure 2 As shown, before issuing tasks to the forklift 30 or viewing relevant information in the warehousing system through the user terminal 10, the user must first log in to the user terminal 10 to verify their identity. First, the user terminal 10 receives login information entered by the user through the user login module 101. This login information can be an account password or login information corresponding to other biometric identification technologies, such as facial recognition or fingerprints. After receiving the login information, the user login module 101 sends it to the server 20, where the authorization module 201 performs authorization authentication on the login information based on the database, and then sends the verification result back to the user terminal 10. By having the user terminal 10 perform authorization authentication on the login information during the user login phase, system security can be improved.

[0042] After the user terminal 10 confirms successful login, it can perform fleet formation and management functions. For example, the fleet management module 102 can display information about available forklifts in the warehousing system on the human-machine interface of the user terminal 10 based on the relevant operational information of the warehousing system stored in the information monitoring module 103. After receiving a fleet formation instruction from the user, the fleet management module 102 of the user terminal 10 can form at least one available forklift in the system into a work fleet. It should be noted that an available forklift refers to a forklift that is in normal condition and does not belong to another user's work fleet. The fleet management module 102 and the information monitoring module 103 can provide relevant information about the warehousing system, enabling users to form and manage work fleets or forklifts within fleets based on this information.

[0043] After the fleet is assembled, the user terminal 10 can respond to the user's input of the task selection instruction and select the relevant information of the task type of the work fleet to complete the selection and display of the task type. The task type includes, but is not limited to, outbound picking, inbound placement, picking replenishment, warehouse location exchange, and cargo transportation. The task type selected by the user terminal 10 can be used as the task type of the work fleet in the current work cycle.

[0044] After receiving a forklift control command input by the user, the task setting module 104 of the user terminal 10 can send the forklift control command to the server 20. The server 20 parses and decomposes the received forklift control command through the task decomposition module 204, generates a handling task corresponding to each forklift 30, and sends the corresponding handling task to the forklift. After receiving the handling task, the forklift 30 can decompose the equipment-level handling task into an action-level task sequence through the task management module 302, and perform scheduling optimization through the scheduling optimization module 205. The behavior planning module 304 makes a behavior plan based on the environmental perception data of the environmental perception module 303 and sends it to the motion control module 305. The motion control module 305 drives the forklift to move according to the behavior plan through the device drive layer 306. In addition, the behavior planning module 304 communicates with the traffic control module 206 in the server 20 and receives navigation information output by the traffic control module 206 during the movement of the forklift 30, thereby avoiding the forklift 30 from blocking or deadlocking.

[0045] During the handling process of forklift 30, environmental perception module 303 senses environmental perception information and forklift status information detected by sensors, and system management module 301 collects environmental perception information and forklift status information, and sends the environmental perception information and forklift status information to server 20 according to preset rules (such as according to preset frequency). Vehicle management module 202 in server 20 can manage forklifts based on the received environmental perception information and forklift status information sent by each forklift, while data statistics module 203 can store relevant information in the database according to preset rules. When a user needs to view the vehicle information of the work fleet, they can send a request to server 20 through user terminal 10 and receive the vehicle information of the work fleet from server 20.

[0046] Figure 3 This is a schematic diagram of the control flow of a collaborative forklift system provided in an embodiment of the present disclosure, such as... Figure 3 As shown, the collaborative forklift system includes: a user terminal 10, a server 20, and at least one forklift 30.

[0047] User terminal 10 is configured to send forklift control commands to server 20 in response to receiving forklift control commands input by the user.

[0048] For example, user terminal 10 can receive forklift control commands input by a user (such as an operator). After receiving the forklift control commands input by the user, user terminal 10 can send the forklift control commands to server 20 so that the server can parse and process the forklift control commands.

[0049] Forklift control commands are used to instruct at least one forklift to perform a target task. This at least one forklift can be a forklift in a fleet of vehicles managed by a user logged into user terminal 10. The forklift control commands can be issued to a task assigned to the entire fleet or to a specific forklift within the fleet; this disclosure does not limit the scope of the command.

[0050] The forklift control command may include information about at least one forklift (such as forklift number and initial position) and task information of the target task. The task information includes at least one of the following: task type, cargo information (such as cargo number, cargo position, cargo size, cargo weight, cargo stability and other physical characteristic parameters), running speed, task start point, task end point and operating parameters (such as walking speed, lifting height and other kinematic parameters).

[0051] The forklift control instructions received by the user terminal 10 can be natural language instructions, text instructions, operation instructions, or voice instructions, etc. The tasks corresponding to the forklift control instructions can include goods picking tasks, goods storage tasks, goods replacement tasks, goods outbound tasks, and goods inbound tasks, etc.

[0052] In some examples, after user terminal 10 confirms that the user's login information has been successfully logged in, user terminal 10 can receive a fleet assembly instruction input by the user. In response to the fleet assembly instruction, user terminal 10 assembles at least one available forklift in the system into a work fleet. After the fleet is assembled, user terminal 10 responds to the forklift control instruction input by the user and issues a handling task to at least one forklift in the work fleet through server 20.

[0053] For example, a user can issue a handling task to the forklifts in the fleet of work vehicles under their management through the user terminal 10. Since the data between the user terminal 10 and the forklift 30 needs to be transmitted through the server 20, the user terminal 10 can send the forklift control command to the server 20 after receiving the forklift control command issued by the user.

[0054] For example, after receiving the forklift control command input by the user, the user terminal 10 can also display a prompt message to prompt the user to confirm the corresponding task information. After receiving the confirmation operation input by the user, the forklift control command is then sent to the server 20.

[0055] Server 20 is configured to process the received forklift control commands, generate a target handling task corresponding to at least one forklift 30, and send the corresponding target handling task to the forklift 30.

[0056] For example, after receiving the forklift control command sent by the user terminal 10, the server 20 can parse and process the forklift control command to generate a target handling task corresponding to at least one forklift. This target handling task can be a task that the forklift can understand and execute (forklift-level task). In other words, by processing the forklift control command through the server 20, the user-level task can be broken down into a forklift-level task.

[0057] Target handling tasks refer to the various types of tasks that forklifts can perform in logistics scenarios, such as placing goods, picking goods, moving, and stopping.

[0058] For example, depending on the input method of the forklift control command, the forklift control command can be an operation command or a natural language command. For instance, the forklift control command received by the user terminal 10 can be an operation command input by the user through the human-machine interface of the user terminal 10. The forklift control command received by the user terminal 10 can also be a voice command input by the user.

[0059] When the forklift control command received by the server 20 is an operation command, the server can use a rule engine model to reorganize and parse the task information in the command, and split the handling task based on the preset task splitting rules to obtain the target handling task at the forklift level.

[0060] For example, when the task type in the forklift control command is a goods exchange task, server 20 can use a rule engine model to split the forklift control command into picking tasks and placing tasks according to time sequence. When the task information in the forklift control command includes returning to the initial position, a movement task can be added to the picking and placing tasks, with the destination of this movement sub-task being the initial position of the forklift. When the forklift control command includes a cluster task involving multiple forklifts, server 20 can use a task scheduling model to schedule and allocate the split tasks, assigning each task to the corresponding forklift 30 for execution based on the current state, position, and load capacity of each forklift 30.

[0061] When the forklift control command received by server 20 is a natural language command, server 20 can use a large language model (LLM) to perform semantic parsing and intent recognition on the natural language command to obtain the target handling task at the forklift level.

[0062] At least one forklift 30 is configured to receive a target handling task, break down the target handling task into a task sequence, and execute the task sequence sequentially to complete the target handling task.

[0063] After receiving the forklift-level target handling task sent by the server 20, the forklift 30 can use the task decomposition model to decompose the target handling task into action-level tasks, thereby obtaining a task sequence.

[0064] Action-level tasks refer to tasks that the forklift's controller can handle, such as trajectory tracking, fork lifting, fork extension, docking, navigation, acceleration, or deceleration. Action-level tasks are the smallest unit in the entire task flow system. After breaking down the target handling task into a sequence of action-level tasks, the forklift 30 can execute the sequence of tasks sequentially to complete the target handling task.

[0065] For example, the task decomposition model can be a rule-based deterministic workflow orchestration model. This model can decompose forklift-level handling tasks into ordered sequences of tasks, where each sequence includes multiple action-level tasks.

[0066] For example, taking a pickup task as an example, after receiving the pickup task, the forklift 30 can use the task decomposition model to decompose the pickup task into a task sequence. The task sequence includes: retrieving the target location, adjusting the fork arm to the travel height, moving to the vicinity of the target location, rotating to align with the pallet direction, lowering the fork arm to the pickup height, detecting the pallet position, correcting the position for precise docking, extending the fork arm to the bottom of the pallet, raising the fork arm and lifting the pallet, retracting the fork arm and moving backward. Each task in the above task sequence is an action-level task.

[0067] The collaborative forklift system provided in this disclosure adopts a system architecture consisting of a user terminal, a server, and forklifts, enabling human-machine collaboration. The server can efficiently understand and break down received user commands in real time, and then send the broken-down tasks to the forklifts to achieve semi-automatic control. Furthermore, the architecture of this disclosure can include multiple forklifts, allowing users to control multiple forklifts simultaneously through the user terminal, achieving diversified human-machine collaboration. This reduces labor costs in warehousing and logistics, and provides the entire system with sufficient flexibility and adaptability to various logistics scenarios. In particular, it can reduce or even eliminate the need for logistics site modifications, lower deployment costs, and improve handling efficiency through human-machine collaboration.

[0068] In some embodiments of this disclosure, the forklift control commands are voice commands or natural language commands, and the server 20 is further configured to: The forklift control commands are parsed and processed using a natural language model to determine the handling commands. Based on the number of forklifts and forklift information, the handling commands are broken down to generate target handling tasks for each forklift.

[0069] The number of forklifts and forklift information are pre-stored information on server 20, or the forklift control instructions include the number of forklifts and forklift information.

[0070] For example, the forklift control command received by the user terminal 10 can be a voice command or a natural language command. Therefore, after receiving the forklift control command, the server 20 needs to parse it to obtain the handling instruction. For example, the server 20 can parse the forklift control command, extract the key task information in the forklift control command, and generate a clear and executable handling instruction based on the key task information.

[0071] In some examples, the natural language model can be a Large Language Model (LLM). The LLM performs semantic understanding of the natural language text, extracting key entity information such as task start point, task end point, and operation object. Server 20 can generate structured field information based on the key entity information and send the target handling task to the forklift based on this structured field information. If some field information cannot be extracted from the natural language instructions, a completion strategy can be used to complete it.

[0072] For example, if the destination location information is not extracted from the natural language command, the server 20 can interact with the warehouse management system (WMS) to obtain the location information of the target shelf. If the pallet number, cargo attributes, or other information is not extracted from the natural language command, the forklift 30's sensors (such as vision sensors and weight sensors) can acquire and transmit this information in real time during the handling task. If important information affecting the safety or correctness of the task is not extracted from the natural language command, the server 20 can generate a confirmation request and send it to the user terminal 10 for manual confirmation and completion by the user.

[0073] For example, taking the natural language instruction "Please move 50 pallets from inventory area A to target location B for outbound processing" received by user terminal 10, server 20 first uses a natural language model to perform semantic understanding and intent recognition on the instruction, extracting key information such as the number of pallets to be moved, the starting point being inventory area A, and the target location being target location B. Based on this key information, the handling instruction can be determined. Next, server 20 queries pre-stored forklift-related information, including the number of forklifts, the type of each forklift, its current location, and its working status (such as whether it is idle, whether it is performing other tasks, and its battery status). Alternatively, if the forklift control instruction issued by the user includes the number of forklifts and forklift information (e.g., the operator's instruction is "Please have idle forklifts 1 to 3 move 50 pallets from inventory area A to target location B for outbound processing"), server 20 can also parse the number of forklifts and forklift information from the forklift control instruction. Server 20 then decomposes the task based on the number of forklifts, forklift information, and the extracted key elements of the handling task. During the dismantling process, the server comprehensively considers factors such as the load capacity of each forklift, the distance from its current location to inventory area A, and the optimal path from inventory area A to the target location B, generating an executable target handling task for each forklift 30. For example, server 20 assigns a handling task of moving 20 pallets to forklift 1, 15 pallets to forklift 2, and 15 pallets to forklift 3, and specifies the picking order, travel path, and unloading position for each forklift, thereby ensuring that each forklift can work collaboratively and efficiently complete the handling task.

[0074] The target handling tasks decomposed by server 20 can be tasks that forklifts can understand and execute (forklift-level tasks). For example, the target handling task corresponding to forklift 1 includes: forklift 1 moves to the corresponding position in inventory area A where pallet A1 is stored, picks up pallet A1 at the corresponding position of pallet A1, and then moves pallet A1 to the target position B for outbound processing; then moves from the target position B to the corresponding position in inventory area A where the next pallet A2 is stored, picks up pallet A2 at the corresponding position of pallet A2, and then moves pallet A2 to the target position B for outbound processing; and so on, decomposed to obtain the handling tasks corresponding to forklift 1 handling 20 pallets.

[0075] In some examples, user terminal 10 receives forklift control commands issued to forklifts in the fleet currently managed by user terminal 10. Since the task issued by the forklift control command is a user-level task, server 20 needs to parse and decompose the forklift control command after receiving it to break down the user-level task into tasks that the forklifts can understand and execute, thereby obtaining the forklift-level target handling task for each forklift in the fleet. After obtaining the forklift-level task, server 20 can send the corresponding target handling task to each forklift in the fleet via wireless local area network.

[0076] This disclosure receives forklift control commands via a server, efficiently understands and decomposes these commands into task breakdowns, resulting in understandable and executable target handling tasks for the forklift. These task tasks are then sent to the forklift, achieving semi-automatic control. Furthermore, the architecture of this disclosure allows for the management of multiple forklifts through a user terminal, enabling diverse human-machine collaboration. This reduces labor costs in warehousing and logistics, while also providing the entire system with high flexibility and adaptability to various logistics scenarios.

[0077] Figure 4 This is a schematic diagram of the control flow of another collaborative forklift system provided in an embodiment of the present disclosure, such as... Figure 4 As shown, any forklift 30 is further configured as follows: The system uses sensor-collected environmental information to perceive the environment around the forklift, executes a sequence of tasks based on the perceived environmental information, and feeds back detection information to the server according to preset rules. The detection information includes the forklift's status information and the perceived environmental information.

[0078] During the handling process, the forklift 30 can collect environmental information around it using sensors (such as LiDAR, cameras, and ultrasonic sensors) to perceive the environment and obtain environmental perception information. This environmental perception information may include the position and size of obstacles, the operating status of other forklifts, ground markings, shelf positions, pallet status, and the physical parameters of the goods to be handled. Based on this environmental perception information, the forklift 30 sequentially performs operations such as moving, picking up, lifting, transporting, and lowering, thereby completing the tasks in the task sequence.

[0079] During the handling process, in addition to collecting environmental information around the forklift, the system can also collect forklift status information, such as operating parameters and task execution status. Operating parameters include forklift position, travel speed, steering angle, fork lifting height and tilt angle, remaining battery power, and drive motor operating parameters. Task execution status includes the current task progress (e.g., completed pick-up operation, in transit, or about to be lowered) and whether any abnormalities occur during task execution (e.g., fork positioning deviation, cargo tilting, pick-up failure).

[0080] After obtaining detection information (such as forklift status information and environmental perception information), forklift 30 can report the detection information to server 20 according to preset rules. For example, forklift 30 can send its status information and / or environmental perception information to server 20 at a preset frequency or when triggered by specific events (such as reaching a designated location, completing a picking action, detecting an abnormal situation, etc.), so that server 20 can monitor the operating status and working environment of each forklift in real time, thereby effectively monitoring and scheduling the entire fleet of work vehicles.

[0081] The forklift 30 can adjust its movement trajectory and operating actions in real time based on the detection information to ensure safe and accurate driving and operation in complex environments. For example, when the forklift detects an obstacle ahead, it can calculate and execute operations such as deceleration, avoidance, or stopping based on the distance to the obstacle and its own speed. The forklift 30 can also report the detection information to the server 20. After receiving the detection information sent by the forklift 30, the server 20 can remotely monitor and manage the operation of the forklift, and can also provide feedback on the status information of the forklift to the user terminal 10 to issue alarms or prompts to the user, thereby ensuring the safety and efficiency of forklift operation.

[0082] Server 20 is also configured to receive detection information from each forklift, adjust the target handling task of each forklift based on the detection information, and send the adjusted target handling task to each forklift.

[0083] After receiving the forklift's status information and / or environmental awareness information from the forklift, the server 20 can adjust the forklift's target handling task based on the forklift's status information and / or environmental awareness information, and send the adjusted target handling task to the forklift.

[0084] The collaborative forklift system provided in this disclosure enhances the perception capabilities of existing forklifts, enabling them to sense and obtain environmental information. The forklift can then transmit this environmental information and its status information to a server, allowing the server to adjust its handling tasks based on these parameters. This server-based adjustment of forklift handling tasks provides the system with high flexibility and adaptability to various logistics scenarios, thereby improving logistics handling efficiency.

[0085] like Figure 4 As shown, in some embodiments of this disclosure, the server 20 adjusts the target handling tasks of each forklift based on the detection information, and is further configured as follows: Based on the detection information, the target status information is determined; the target status information is sent to the user terminal 10; based on the adjustment instructions fed back by the user terminal 10, the target handling tasks of each forklift are adjusted.

[0086] User terminal 10 is further configured to: receive target status information and display the target status information; and in response to receiving an adjustment instruction triggered by the user, send the adjustment instruction to server 20.

[0087] After detecting environmental perception information or forklift status information, forklift 30 can feed this information back to server 20 in real time. Server 20 can process and filter the detection information reported by forklift 30 to obtain target status information. When server 20 determines that manual intervention is required based on the target status information, server 20 can send the target status information to user terminal 10 so that the user can adjust the handling task through user terminal 10. After determining the target status information, server 20 can also directly send the target status information to user terminal 10. In this case, the user can determine whether to adjust the target handling task through user terminal 10. If it is determined that the handling task needs to be adjusted, user terminal 10 responds to the adjustment command triggered by the user and sends the adjustment command to server 20 to adjust the target handling task.

[0088] For example, server 20 can determine target status information based on the status information of forklift 30. The target status information may include any one of the following: forklift handling task completion information, forklift handling task execution failure information, forklift emergency braking information, and forklift resumption of operation status.

[0089] Adjustment commands are used to modify task-related information for the target handling task, including but not limited to adjusting forklift operating parameters, adjusting forklift working mode, adjusting forklift handling of goods, adjusting goods storage location, and changing forklift tasks.

[0090] For example, adjustment instructions include a first switching instruction to switch from automatic mode to manual mode or a second switching instruction to switch from manual mode to automatic mode.

[0091] When generating a target handling task based on forklift control commands, server 20 can determine the type of the target handling task according to the forklift's capabilities (such as the forklift's mechanical structure and level of intelligence). The types of target handling tasks include first-class tasks (such as standard tasks) that can be completed by the forklift in automatic mode, and second-class tasks (such as non-standard tasks) that require manual intervention. Server 20 can specify the type of target handling task when issuing the task to the forklift.

[0092] The first type of task can be a task that the forklift 30 can complete autonomously. The operation process of the first type of task does not require human intervention and can achieve a certain success rate (such as 99%). For example, the unmanned forklift can automatically navigate to the corresponding location of the target goods, pick up the pallet storing the target goods, and then transport the pallet to the designated location in the shipping area.

[0093] The second category of tasks can be those that cannot be completed autonomously by the forklift and require manual intervention. These include tasks that require human intervention at key points to complete. Alternatively, the second category can also include tasks where the forklift 30 cannot guarantee a minimum success rate when completed autonomously. For example, when performing high-bay pallet retrieval tasks, the forklift 30 can automatically navigate to the target goods' location. However, if the pallet containing the target goods is located at a high bay, and the retrieval of goods at higher levels exceeds the forklift 30's automated operation design range (e.g., the forklift 30 lacks sufficient sensing capabilities or corresponding mechanical structures), then the forklift 30 may be unable to complete the pallet retrieval action or may have a low success rate. Therefore, for this second category of tasks, necessary human intervention at key points is required to ensure the safety and efficiency of task execution.

[0094] Forklift 30 has two operating modes: automatic and manual. In automatic mode, the forklift is highly intelligent and can automatically complete the material handling task. In manual mode, the forklift is less intelligent and requires manual intervention, with the forklift handling the material under the operator's control. For the first type of task, forklift 30 can complete the handling task in automatic mode. For the second type of task, forklift 30 needs to switch between automatic and manual modes to complete the task.

[0095] Taking a forklift 30 performing a transport task that includes a first type of task and a second type of task as an example, the forklift 30 can execute the first type of task in automatic mode. After the first type of task is completed, the forklift 30 can send task completion information to the server indicating that the first type of task has been completed. Based on this task completion information, the server 20 determines that the first type of task has been completed. Since the next task in the second type of task requires manual intervention, the server 20 sends the target status information indicating that the first type of task has been completed to the user terminal 10. Alternatively, after the first type of task is completed, the forklift 30 can continue to execute the second type of task. If the second type of task fails, the forklift 30 sends task execution information indicating that the second type of task has failed to the server, and the server 20 sends this information to the user terminal 10.

[0096] After receiving target status information indicating the completion of the first type of task or the failure of the second type of task, user terminal 10 can display the target status information on the user interface to prompt the user that the first type of task has been completed, or that the second type of task has failed, or whether the user wants to switch the forklift's working mode. Upon receiving an adjustment command triggered by the user (such as a first switch command to switch from automatic to manual mode), user terminal 10 sends the first switch command to server 20. Based on the first switch command fed back by user terminal 10, server 20 adjusts the target handling tasks of each forklift to control the forklift 30 to switch its working mode from automatic to manual to execute the second type of task.

[0097] After the forklift 30 switches its working mode from automatic to manual, the operator can drive the forklift 30 to perform the second type of task. In this mode, the forklift 30 can function as a manual forklift, completing the handling task under the operator's manual control. Once the forklift 30 has completed the second type of task, it can send task completion information to the server 20, indicating that the second type of task has been completed. Based on this task completion information, the server 20 confirms that the second type of task has been completed. Since the forklift 30 is currently operating in manual mode, the server 20 can send the target status information indicating the completion of the second type of task to the user terminal 10.

[0098] After receiving the target status information indicating the completion of the second type of task, user terminal 10 can display the target status information on the user interface to indicate to the user that the second type of task is complete, or to prompt the user whether to switch the forklift's working mode. Upon receiving an adjustment command triggered by the user (such as a second switching command to switch from manual mode to automatic mode), user terminal 10 sends the second switching command to server 20. Based on the second switching command fed back by user terminal 10, server 20 adjusts the target handling tasks of each forklift to control the forklift 30 to switch its working mode from manual mode to automatic mode, so as to execute subsequent handling tasks in automatic mode.

[0099] For example, in the event of an emergency braking incident involving forklift 30 (such as an emergency braking incident caused by a collision or falling goods), forklift 30 can report the emergency braking situation to server 20. Server 20, based on the emergency braking situation, sends target status information instructing the forklift to brake suddenly to user terminal 10. Upon receiving the target status information instructing the forklift to brake suddenly, user terminal 10 can display the target status information on the user interface to prompt the user that the forklift has braked suddenly, or to prompt the user whether to switch the forklift's operating mode. Upon receiving an adjustment command triggered by the user (such as a first switching command to switch from automatic to manual mode), user terminal 10 sends the first switching command to server 20. Based on the first switching command fed back by user terminal 10, server 20 adjusts the target handling tasks of each forklift to control forklift 30 to switch its operating mode from automatic to manual mode, so as to perform the handling tasks in manual mode.

[0100] When forklift 30 resumes operation, it can report this status to server 20. Server 20, based on this status, sends target status information to user terminal 10, instructing the forklift to resume operation. Upon receiving this target status information, user terminal 10 can display it on the user interface to prompt the user to resume operation or to ask if the user wants to switch the forklift's operating mode. Upon receiving a user-triggered adjustment command (such as a second switching command to switch from manual to automatic mode), user terminal 10 sends the second switching command to server 20. Based on the second switching command from user terminal 10, server 20 adjusts the target handling tasks of each forklift to control forklift 30 to switch its operating mode from manual to automatic, enabling it to perform handling tasks in automatic mode.

[0101] In some examples, when the server 20 determines that the target handling task includes a first type of task and a second type of task, it may only issue the first type of task to the forklift 30 and not issue the second type of task to the forklift 30. After the server 20 confirms that the first type of task has been completed, it will prompt the user terminal 10 to intervene and switch the forklift 30 to manual mode.

[0102] In other examples, when server 20 determines that the target handling task includes both a first type of task and a second type of task, it can issue either a first type of task or a second type of task to forklift 30. This second type of task instructs the forklift to remain stationary until switched to manual mode, thus preventing forklift failure. After server 20 confirms the completion of the first type of task, it prompts user terminal 10 to switch the operating mode to manual mode, allowing the second type of task to be completed under operator control, ensuring both success rate and safety.

[0103] The collaborative forklift system provided in this disclosure categorizes tasks into standard and non-standard types based on the difficulty of automated operation, with the former being completed automatically by the forklift and the latter manually. By controlling the forklift to operate in different working modes under different scenarios, task execution efficiency and success rate can be improved. Furthermore, real-time information transmission allows users to promptly understand the operational status of warehouse logistics, making more rational decisions and targeted operations, thereby achieving a highly efficient and flexible work design and improving the system's operational efficiency.

[0104] In some embodiments of this disclosure, server 20 determines target state information based on various detection information, and is further configured as follows: Based on environmental perception information, the matching degree between the target storage location corresponding to the target handling task and the target cargo corresponding to the target handling task is determined; based on the matching degree, target status information is determined to characterize the matching status between the target storage location and the target cargo.

[0105] For example, the environmental perception information includes any one of the following: the vacancy status of the target storage location, the physical parameters of the target storage location, or the physical parameters of the target goods. Taking the target handling task as picking up or placing target goods at the target storage location, the forklift 30 can detect physical parameters such as whether the target storage location is vacant, the size and location of the target storage location, and the size, weight, stability, and fragility of the target goods through sensors during the execution of the target handling task, and report the above parameters to the server 20.

[0106] For example, the matching degree is used to characterize the matching degree between the target storage location and the target goods, that is, whether the target goods can be placed in the target storage location, or whether the target goods can be retrieved from the target storage location. For example, whether the target storage location has enough space to place the target goods, or whether the target storage location is free enough to place the target goods in the target storage location, or whether the target storage location has the target goods placed there.

[0107] The target status information is used to characterize the matching status between the target storage location and the target goods. For example, if the target storage location has enough space to put the target goods, the target status information is that the target storage location and the target goods are matched. On the other hand, if the target storage location is not idle, or if the target storage location does not have enough space to put the target goods, the target status information is that the target storage location and the target goods are not matched.

[0108] Server 20 determines the matching degree between the target storage location corresponding to the target handling task and the target goods corresponding to the target handling task based on environmental perception information. It is configured to: determine the matching degree between target storage locations based on the idle status information of the target storage location; or, determine the matching degree between target storage locations based on the physical parameter information of the target storage location and the physical parameter information of the target goods.

[0109] For example, server 20 can determine the matching degree between the target storage location and the target goods by using the idle status of the target storage location in the environmental awareness information. For instance, taking a handling task of placing the target goods in the target storage location as an example, if the target storage location is empty and no goods are placed there, it is determined that the target storage location and the target goods match, and the target goods can be placed in the target storage location. If the target storage location has goods placed there and is not idle, it is determined that the target storage location and the target goods do not match, and the target goods cannot be placed in the target storage location.

[0110] For example, taking a handling task as retrieving target goods from a target storage location, if the target storage location contains the target goods, it is determined that the target storage location and the target goods are matched, and the target goods can be retrieved from the target storage location. If the target storage location does not contain any goods or contains other goods, it is determined that the target storage location and the target goods are not matched, and the target goods cannot be retrieved from the target storage location.

[0111] For example, taking the task of placing target goods in a target storage location as an example, the server 20 can also determine whether the size of the target storage location matches the size of the target goods by using the physical parameters of the target goods and the physical parameters of the target storage location in the environmental perception information, thereby determining whether there is enough space in the target storage location to place the target goods. For example, when the size of the target goods (such as width, height, and depth) is larger than the size of the target storage location, the target storage location does not have enough space to place the target goods, and it can be determined that the target storage location and the target goods do not match. When the size of the target goods is smaller than the size of the target storage location, the target storage location has enough space to place the target goods, and it can be determined that the target storage location and the target goods match.

[0112] Based on the matching degree, if the server 20 determines that the target storage location and the target goods do not match, it sends target status information indicating the mismatch between the target storage location and the target goods to the user terminal 10. Upon receiving the target status information indicating the mismatch, the user terminal 10 can display the target status information on the user interface to prompt the user that the storage location is mismatched, or to prompt the user whether to reassign the storage location. The user can issue a storage location adjustment command based on the target status information. After receiving the storage location adjustment command triggered by the user, the user terminal 10 sends the command to the server 20. Based on the storage location adjustment command fed back by the user terminal 10, the server 20 adjusts the target handling tasks of each forklift to control the forklift 30 to move the target goods to the newly selected storage location. It should be noted that the newly selected storage location can be an available storage location selected by the user on the user terminal 10, or a storage location reselected by the server from the available storage locations provided by the Warehouse Management System (WMS) after scheduling the interface based on the storage location adjustment command. This embodiment does not limit this.

[0113] In some examples, after detecting environmental perception information, the forklift 30 can also directly determine the matching degree between the target storage location and the target goods based on the environmental perception information, and feed the matching degree information back to the server 20.

[0114] For example, after detecting the physical parameters (such as width, height, and depth) of the target goods, the forklift 30 can compare these parameters with the physical parameters of the target storage location. If it is determined that the target storage location does not have enough space to place the target goods, a mismatch between the target storage location and the target goods can be identified, and the forklift 30 can report this mismatch information to the server 20. The server 20 can then use the matching information to call the WMS interface to reselect a storage location that matches the target goods; alternatively, the server 20 can also send a prompt to the user terminal 10 based on the matching information, allowing the user terminal 10 to reselect a storage location that matches the target goods.

[0115] For example, if the forklift 30 detects that other goods are placed in the target storage location (in a non-idle state), meaning the target goods cannot be placed in the target storage location, it can be determined that the target storage location and the target goods are mismatched. The forklift 30 can report this mismatch information to the server 20. The server 20 can then call the WMS interface based on the matching information to reselect a storage location that matches the target goods; alternatively, the server 20 can also send a prompt to the user terminal 10 based on the matching information, allowing the user terminal 10 to reselect a storage location that matches the target goods.

[0116] The collaborative forklift system provided in this embodiment can determine the matching degree between the storage location and the goods based on the detection information of the forklift, and feed back the matching status to the user terminal. This allows the user to understand the matching status between the goods and the storage location in a timely manner and make more reasonable storage location adjustment decisions, thereby achieving a highly efficient and flexible work design and improving the system's operating efficiency.

[0117] In some embodiments of this disclosure, server 20 adjusts the target handling tasks of each forklift based on the detection information, and is further configured as follows: Based on environmental perception information, target operating parameters are determined; wherein, environmental perception information includes physical parameter information of the target cargo corresponding to the target handling task or the distance between the forklift and obstacles; the target operating parameters are sent to user terminal 10; based on the adjustment instructions fed back by user terminal 10, the target handling tasks of each forklift are adjusted.

[0118] For example, server 20 can determine target operating parameters based on the physical parameters of the target goods reported by forklift 30 or the distance between the forklift and obstacles. These target operating parameters include, but are not limited to, the forklift's operating speed, direction of travel, and path. For instance, taking operating speed as the target operating parameter, server 20 can determine a recommended operating speed based on the stability of the target goods to ensure that the goods do not fall if they tilt. As another example, server 20 can determine the target operating speed based on the distance between the forklift and obstacles to ensure that the forklift does not collide with the obstacles, thus ensuring safe operation.

[0119] After determining the target operating parameters, server 20 can send these parameters to user terminal 10 to prompt the user whether to adjust the operating speed of forklift 30. Once the user confirms the adjustment, they can input an adjustment command on user terminal 10, which will then send the command to server 20. Server 20 will adjust the operating parameters of forklift 30 based on the command. The adjustment command sent by the user may include the operating speed, which can be determined based on the target operating parameters fed back by server 20.

[0120] In some embodiments of this disclosure, the forklift 30 executes a sequence of tasks sequentially based on environmental perception information, and is further configured as follows: Based on environmental perception information, the target operating parameters are determined; the environmental perception information includes the physical parameters of the target cargo corresponding to the target handling task or the distance between the forklift and obstacles; the vehicle travels and executes the task sequence according to the target operating parameters.

[0121] For example, after detecting the physical parameters of the target cargo or the distance between the forklift and an obstacle, the forklift can directly adjust its operating parameters based on these parameters. Simultaneously, the forklift can send speed adjustment information (such as deceleration and obstacle avoidance) and obstacle information to the server.

[0122] In some examples, the target operating parameter is determined as the first operating parameter when the distance between the forklift and the obstacle is greater than a first distance threshold; the target operating parameter is determined as the second operating parameter when the distance between the forklift and the obstacle is greater than a second distance threshold but less than the first distance threshold; and the target operating parameter is determined as the third operating parameter when the distance between the forklift and the obstacle is less than the second distance threshold.

[0123] For example, when the distance between the forklift and the obstacle is greater than the first distance threshold, it indicates that the forklift is far from the obstacle, and a higher operating speed can be maintained at this distance. When the distance between the forklift and the obstacle is greater than the second distance threshold but less than the first distance threshold, it indicates that the distance between the forklift and the obstacle is in the medium distance range, and the forklift can reduce its speed. When the distance between the forklift and the obstacle is less than the second distance threshold, it indicates that the distance between the forklift and the obstacle is relatively short, and the forklift can reduce its speed to the minimum or stop to avoid a collision.

[0124] The collaborative forklift system provided in this disclosure allows the server to determine the distance between the forklift and obstacles based on environmental perception information fed back by the forklift. The server then determines operating parameters based on this distance and sends them to the user terminal. This allows the user to promptly understand the forklift's operating information and make more reasonable speed adjustment decisions. This achieves both high efficiency and flexibility in its work design, while also preventing collisions between the forklift and obstacles, thus improving forklift operation safety. Alternatively, the forklift can directly determine the obstacle distance based on environmental perception information and control its speed accordingly. Direct speed control by the forklift further enhances safety.

[0125] In some embodiments of this disclosure, the forklift 30 is also configured to: In response to receiving a first switching command from the user, the forklift's operating mode is switched from automatic mode to manual mode; or, in response to receiving a second switching command from the user, the forklift's operating mode is switched from manual mode to automatic mode based on a preset time.

[0126] For example, the forklift 30 may include a display for showing a human-machine interface, through which a user can switch the operating mode of the forklift 30. For instance, after receiving a mode switching command triggered by the user on the display of the forklift 30, the processor of the forklift 30 can switch the operating mode of the forklift for assisted driving control.

[0127] For example, as the operator approaches the forklift 30, the forklift, upon detecting the operator, can decelerate based on the distance between the forklift 30 and the operator, and remain stationary when the operator is detected to be within close proximity or inside the forklift. Upon receiving the first switching command from the operator, the forklift 30 can switch its operating mode from automatic to manual, allowing the operator to take over the forklift and perform handling tasks. When the forklift 30 is in manual mode, its display shows that the forklift is in manual mode and displays necessary functions and information, such as camera access.

[0128] For example, after the operator completes a handling task requiring manual intervention, they can input a second switching command on the forklift 30. In response to this command, the forklift 30 switches its operating mode from manual to automatic for assisted driving control. The forklift 30 then switches back to automatic mode only after a preset time has elapsed since receiving the second switching command. This mode switching only occurs after the operator has left the forklift 30, ensuring operator safety.

[0129] When the preset time expires and no operator is detected still within or near the forklift, the forklift 30 automatically switches modes and displays the automatic mode and related status information, such as current task progress and planned path, on the human-machine interface. If the operator is detected still within or near the forklift within the preset time, the forklift 30 can pause the mode switching and issue a prompt, such as an audible alarm or a pop-up window, and resume the switching process only after the operator leaves, further ensuring operational safety.

[0130] When the forklift 30 is in automatic mode, its human-machine interface can display either the current task interface or the takeover interface. In automatic mode, the forklift 30's human-machine interface includes both the current task interface and the takeover interface. By inputting user commands into the forklift 30's human-machine interface, users can switch from the current task interface to the takeover interface.

[0131] The current task interface displays forklift operation information, including but not limited to: the task being performed by the forklift and its progress, the fleet information to which it belongs, the objects and targets currently being handled, the user information managing the forklift, and the operating mode. The user interface of forklift 30 can jump from the current task interface to the takeover interface. The takeover interface is the operation interface for users to request takeover of the forklift's work. The takeover interface includes the user login screen, takeover task information, and takeover request controls.

[0132] The collaborative forklift system provided in this disclosure allows the forklift to switch its operating mode upon receiving a mode switching command triggered by a user, thereby achieving flexible control of the forklift in different scenarios. Since the forklift in this disclosure can operate in both manual and automatic modes, only algorithmic improvements to existing manual forklifts are needed to enable it to work effectively in various environments, thus reducing or even eliminating the need for logistics site modifications and lowering deployment costs. Furthermore, the forklift operating mode switching mechanism not only meets the diverse needs of human-machine collaboration but also improves the handling efficiency of logistics through human-machine collaboration.

[0133] Exemplary methods Figure 5 This is a flowchart illustrating a forklift control method provided in an exemplary embodiment of this disclosure. This forklift control method is applied to the cooperative forklift system described in the above embodiment. Figure 5 As shown, the forklift control method includes the following steps: Step 501: Upon receiving the forklift control command input by the user, the user terminal sends the forklift control command to the server.

[0134] Step 502: Based on the received forklift control instructions, the server processes the forklift control instructions, generates a target handling task corresponding to at least one forklift, and sends the corresponding target handling task to the forklift.

[0135] For example, step 502 includes: the server using a natural language model to parse and process the forklift control instructions to determine the handling instructions; based on the number of forklifts and forklift information, the handling instructions are broken down to generate target handling tasks corresponding to each forklift; wherein the number of forklifts and forklift information are information pre-stored by the server, or the forklift control instructions include the number of forklifts and forklift information.

[0136] Step 503: After receiving the target handling task, at least one forklift breaks down the target handling task to obtain a task sequence, and executes the task sequence in sequence to complete the target handling task.

[0137] For example, the forklift can perceive the environment around it based on the environmental information collected by sensors, execute a sequence of tasks based on the environmental perception information, and feed back the detection information to the server according to preset rules; wherein, the detection information includes the forklift's status information and the environmental perception information.

[0138] The server receives the detection information from each forklift, adjusts the target handling task of each forklift based on the detection information, and sends the adjusted target handling task to each forklift.

[0139] In some embodiments, the server can determine target status information based on various detection information; send the target status information to the user terminal; the user terminal can receive the target status information sent by the server and display the target status information; in response to receiving an adjustment command triggered by the user, the user terminal sends the adjustment command to the server. The server adjusts the target handling tasks of each forklift based on the adjustment command fed back by the user terminal.

[0140] For example, the target status information includes any one of the following: forklift handling task completion information, forklift handling task execution failure information, forklift emergency braking information, or forklift resumption of operation status; the adjustment instruction includes a first switching instruction to switch from automatic mode to manual mode or a second switching instruction to switch from manual mode to automatic mode.

[0141] In some embodiments, the server determines target status information based on various detection information, including: determining the matching degree between the target storage location corresponding to the target handling task and the target goods corresponding to the target handling task based on environmental perception information; and determining target status information to characterize the matching status between the target storage location and the target goods based on the matching degree.

[0142] For example, the environmental perception information includes any one of the following: the idle status information of the target storage location, the physical parameter information of the target storage location, or the physical parameter information of the target cargo. Based on the environmental perception information, the server determines the matching degree between the target storage location corresponding to the target handling task and the target cargo corresponding to the target handling task, including: the server determining the matching degree between the target storage locations based on the idle status information of the target storage locations; or, the server determining the matching degree between the target storage locations based on the physical parameter information of the target storage locations and the physical parameter information of the target cargo.

[0143] In some embodiments, the server adjusts the target handling task of each forklift based on the detection information, including: the server determining the target operating parameters based on environmental perception information; wherein the environmental perception information includes the physical parameter information of the target cargo corresponding to the target handling task or the distance between the forklift and the obstacle; sending the target operating parameters to the user terminal; and adjusting the target handling task of each forklift based on the adjustment instructions fed back by the user terminal.

[0144] In some embodiments, the forklift executes a task sequence sequentially based on environmental perception information, including: the forklift determining target operating parameters based on environmental perception information; wherein, the environmental perception information includes physical parameter information of the target cargo corresponding to the target handling task or the distance between the forklift and obstacles; and driving and executing the task sequence according to the target operating parameters.

[0145] In some embodiments, the forklift or server determines target operating parameters based on environmental perception information, including: determining the target operating parameter as a first operating parameter when the distance between the forklift and the obstacle is greater than a first distance threshold; determining the target operating parameter as a second operating parameter when the distance between the forklift and the obstacle is greater than a second distance threshold and less than the first distance threshold; and determining the target operating parameter as a third operating parameter when the distance between the forklift and the obstacle is less than the second distance threshold.

[0146] In some embodiments, the forklift may also switch its operating mode from automatic mode to manual mode in response to receiving a first switching command input by the user; or, in response to receiving a second switching command input by the user, switch its operating mode from manual mode to automatic mode based on a preset duration.

[0147] The implementation methods of each step in the forklift control method provided in this embodiment can be referred to the implementation methods of the corresponding steps in the aforementioned cooperative forklift system, and will not be repeated here.

[0148] The forklift control method provided in this disclosure receives forklift control commands input by a user through a user terminal. A server converts these commands into forklift-level tasks and sends them to the forklift. Upon receiving the forklift-level tasks, the forklift breaks them down into a task sequence and executes them sequentially to complete the handling task corresponding to the forklift control command. This disclosure employs a system architecture that enables human-machine collaboration, consisting of a user terminal, server, and forklift. The server can efficiently understand and break down received user commands in real time and send the decomposed tasks to the forklift, achieving semi-automatic control of the forklift. Furthermore, the architecture can include multiple forklifts, allowing users to control multiple forklifts simultaneously through the user terminal, enabling diversified human-machine collaboration. This reduces labor costs in warehousing and logistics, provides the system with high flexibility and adaptability to various logistics scenarios, and significantly reduces or eliminates the need for logistics site modifications, lowering deployment costs. Simultaneously, human-machine collaboration improves the handling efficiency of logistics.

[0149] Exemplary device This disclosure provides a user terminal, which includes at least one processor and a memory. The processor is configured to: In response to receiving a forklift control command input by the user, the system sends the forklift control command to the server; receives and displays the target status information; and in response to receiving an adjustment command triggered by the user, sends the adjustment command to the server.

[0150] The implementation methods of the user terminal executing each step provided in this embodiment can be referred to the implementation methods of the user terminal executing corresponding steps in the aforementioned collaborative forklift system, and will not be repeated here. It should be noted that the user terminal can also be configured to execute other steps in the aforementioned collaborative forklift system where the user terminal is the execution subject.

[0151] This disclosure provides a server including at least one processor and memory. The processor is configured to: Based on the received forklift control commands, the forklift control commands are processed to generate a target handling task corresponding to at least one forklift, and the corresponding target handling task is sent to the forklift.

[0152] The implementation method of the server executing each step provided in this embodiment can be referred to the implementation method of the server executing the corresponding step in the aforementioned collaborative forklift system, and will not be repeated here. It should be noted that the server can also be configured to execute other steps in the aforementioned collaborative forklift system where the server is the execution subject.

[0153] This disclosure provides a forklift that includes at least one processor and a memory. The processor is configured to: Receive the target transportation task, break down the target transportation task into a task sequence, and execute the task sequence in sequence to complete the target transportation task.

[0154] The implementation methods of the forklift performing each step provided in this embodiment can be referenced from the implementation methods of the forklift performing corresponding steps in the aforementioned collaborative forklift system, and will not be repeated here. It should be noted that the forklift can also be configured to perform other steps in the aforementioned collaborative forklift system where the forklift is the executing entity.

[0155] Exemplary computer program products and computer-readable storage media In addition to the methods and devices described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform steps in the forklift control methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above, steps performed by a user terminal, and / or steps performed by a server, and / or steps performed by a forklift.

[0156] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0157] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the forklift control methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0158] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0159] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0160] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A collaborative forklift system, comprising: The user terminal is configured to send the forklift control command to the server in response to receiving the forklift control command input by the user. The server is configured to process the forklift control commands received, generate a target handling task corresponding to at least one forklift, and send the corresponding target handling task to the forklift. The at least one forklift is configured to, upon receiving the target handling task, break down the target handling task into a task sequence, and execute the task sequence sequentially to complete the target handling task.

2. The collaborative forklift system according to claim 1, wherein, The forklift control commands are voice commands or natural language commands; the server is further configured to: The forklift control commands are parsed and processed using a natural language model to determine the handling instructions; Based on the number of forklifts and forklift information, the handling instructions are broken down to generate target handling tasks corresponding to each forklift. The number of forklifts and the forklift information are information pre-stored by the server, or the forklift control instructions include the number of forklifts and the forklift information.

3. The cooperative forklift system according to claim 2, wherein, Any of the forklifts is further configured to: perceive the environment around the forklift based on environmental information collected by sensors, execute the task sequence sequentially based on the environmental perception information, and feed back detection information to the server according to preset rules; wherein, the detection information includes the status information of the forklift and the environmental perception information; The server is also configured to receive the detection information fed back by each of the forklifts, adjust the target handling task of each forklift based on the detection information, and send the adjusted target handling task to each forklift.

4. The collaborative forklift system according to claim 3, wherein, Based on the detection information, the server adjusts the target handling task of each forklift, and is further configured as follows: Based on the detection information, the target state information is determined; The target status information is sent to the user terminal; Based on the adjustment instructions fed back from the user terminal, the target handling tasks of each forklift are adjusted.

5. The cooperative forklift system according to claim 4, wherein, The user terminal is further configured as follows: Receive the target status information and display the target status information; In response to receiving an adjustment command triggered by a user, the adjustment command is sent to the server.

6. The collaborative forklift system according to claim 4 or 5, wherein, The target status information includes any one of the following: forklift handling task completion information, forklift handling task execution failure information, forklift emergency braking information, or forklift resumption of operation status. The adjustment instructions include a first switching instruction to switch from automatic mode to manual mode or a second switching instruction to switch from manual mode to automatic mode.

7. The cooperative forklift system according to claim 4, wherein, Based on the detection information, the server determines the target state information and is further configured as follows: Based on the environmental perception information, the matching degree between the target storage location corresponding to the target handling task and the target goods corresponding to the target handling task is determined; Based on the matching degree, the target status information used to characterize the matching status between the target cargo location and the target cargo is determined.

8. The cooperative forklift system according to claim 7, wherein, The environmental perception information includes any one of the following: the vacancy status information of the target cargo location, the physical parameter information of the target cargo location, or the physical parameter information of the target cargo. The server, based on the environmental awareness information, determines the matching degree between the target storage location corresponding to the target handling task and the target goods corresponding to the target handling task, and is configured as follows: Based on the availability information of the target storage locations, determine the matching degree between the target storage locations; or, Based on the physical parameter information of the target cargo location and the physical parameter information of the target cargo, the matching degree between the target cargo location and the target cargo location is determined.

9. The cooperative forklift system according to claim 3, wherein, Based on the detection information, the server adjusts the target handling task of each forklift, and is further configured as follows: Based on the environmental perception information, target operating parameters are determined; wherein, the environmental perception information includes physical parameter information of the target cargo corresponding to the target handling task or the distance between the forklift and obstacles; The target operating parameters are sent to the user terminal; Based on the adjustment instructions fed back from the user terminal, the target handling tasks of each forklift are adjusted.

10. The cooperative forklift system according to claim 3, wherein, The forklift executes the task sequence sequentially based on environmental perception information, and is further configured as follows: Based on the environmental perception information, target operating parameters are determined; wherein, the environmental perception information includes physical parameter information of the target cargo corresponding to the target handling task or the distance between the forklift and obstacles; Drive and execute the task sequence according to the target operating parameters.

11. The collaborative forklift system according to claim 9 or 10, wherein, The determination of target operating parameters based on the environmental perception information includes: If the distance between the forklift and the obstacle is greater than a first distance threshold, the target operating parameter is determined as the first operating parameter; If the distance between the forklift and the obstacle is greater than a second distance threshold and less than the first distance threshold, the target operating parameter is determined to be the second operating parameter. If the distance between the forklift and the obstacle is less than the second distance threshold, the target operating parameter is determined as the third operating parameter.

12. The cooperative forklift system according to any one of claims 1-5, wherein, The forklift is also configured to: In response to receiving a first switching command from the user, the operating mode of the forklift is switched from automatic mode to manual mode; or, In response to receiving a second switching command from the user, the forklift's operating mode is switched from manual mode to automatic mode based on a preset duration.

13. A forklift control method, applied to the collaborative forklift system according to any one of claims 1-12, comprising: In response to receiving a forklift control command input by the user, the user terminal sends the forklift control command to the server. The server processes the received forklift control commands, generates a target handling task for at least one forklift, and sends the target handling task to the forklift. After receiving the target handling task, at least one forklift breaks down the target handling task into a task sequence and executes the task sequence in sequence to complete the target handling task.

Images