Interactive control method and system of multiple cleaning robots based on large language model

Abstract

The present invention discloses an interactive control method and system for multiple tank cleaning robots based on a large language model, and belongs to the technical field of port intelligent equipment. The method comprises: establishing a tank cleaning data set by constructing a tank cleaning instruction paradigm template, and training a tank cleaning operation large language model, and converting the tank cleaning voice instruction into a tank cleaning instruction paradigm executable by the tank cleaning robot through the large language model; establishing a real-time map in the bulk carrier hold, and performing path planning for each tank cleaning robot based on the tank cleaning instruction paradigm; and performing multi-cabin and multi-tank cleaning robot collaborative operations according to the path planning results corresponding to the tank cleaning instruction paradigm. The system comprises: a multi-machine remote simulation driver, a multi-cabin visual display system, an intelligent tank cleaning interactive system, and a multi-robot remote control system. The various interactive control methods of the present invention can realize unmanned tank cleaning operations in multiple cargo holds in a bulk carrier at the same time, effectively improving the safety and efficiency of tank cleaning operations.

Description

Technical Field

[0001] The present invention belongs to the technical field of port intelligent equipment, and in particular relates to an interactive control method and system for multiple tank cleaning robots based on a large language model. Background Art

[0002] The construction of a new generation of shipping systems has put forward higher requirements for the level of intelligence in the water transportation process. In this process, the intelligent and automated operation technology of bulk cargo ports has attracted much attention. Although the automation level of gantry cranes in bulk carrier ports is relatively high today, the overall automated unloading has not yet been fully realized due to the limitation of manual tank cleaning. Ports in a small number of developed areas have realized simple remote-controlled tank cleaning operations, but there are still problems such as poor coordination between the mixed operation of shore-side gantry cranes / unloading equipment and tank cleaning robots in the cabin, high risk factor of mixed operations, and low intelligence of remote operations. As the last obstacle to achieving fully automatic unloading of bulk carriers, tank cleaning has always affected the automation level and work efficiency of the unloading process.

[0003] Limited by the complex tank cleaning process, it is difficult for the current in-cabin cleaning equipment to achieve unmanned and fully autonomous tank cleaning operations, while one-to-one manual remote control of the in-cabin cleaning equipment cannot reduce labor costs. At the same time, the efficiency of the tank cleaning operation is not high, which affects the overall degree of automation and work efficiency.

[0004] Therefore, it is necessary to design a new control method for the tank cleaning robot to improve the efficiency of the tank cleaning operation. Summary of the invention

[0005] In view of this, the present invention proposes a method and system for interactively controlling multiple tank cleaning robots based on a large language model, which are used to solve the problem of low tank cleaning operation efficiency of bulk carrier tank cleaning robots.

[0006] In a first aspect, the present invention discloses an interactive control method for multiple tank cleaning robots based on a large language model, the method comprising:

[0007] Construct a tank cleaning instruction paradigm template, convert the tank cleaning voice instructions into a tank cleaning instruction paradigm executable by the tank cleaning robot through the tank cleaning instruction paradigm template, and establish a tank cleaning data set;

[0008] The large language model is trained by using the cabin cleaning dataset to obtain the cabin cleaning language understanding model.

[0009] Obtain the tank cleaning voice instructions of the bulk cargo hold to be processed, and generate the tank cleaning instruction paradigm according to the tank cleaning language understanding model;

[0010] Establish a real-time map of the bulk cargo ship's hold and plan the paths of each cleaning robot based on the cleaning command paradigm;

[0011] The multi-cabin and multi-cabin cleaning robots work collaboratively according to the path planning results corresponding to the cabin cleaning instruction paradigm.

[0012] On the basis of the above technical solution, preferably, the tank cleaning instruction paradigm template includes operation sequence, starting point, end point, action keywords, tank cleaning operation effect, cycle judgment, pause operation and auxiliary information;

[0013] The operation sequence is the movement operation sequence of the cleaning robot between different operation positions;

[0014] The starting point and the end point are the starting point and the end point of a single operation;

[0015] The action keywords are the cabin clearing action between two positions in a single operation; the action keywords include shoveling, pushing, piling, scraping and unloading;

[0016] The tank cleaning operation effect is set according to the requirements of the tank cleaning operation;

[0017] The cycle judgment is used to judge whether a single operation needs to be repeated according to the cycle judgment condition;

[0018] The pause operation is used to record the time when the action needs to be temporarily stopped;

[0019] The auxiliary information is used for function expansion.

[0020] On the basis of the above technical solution, preferably, the establishment of the tank clearing data set specifically includes:

[0021] The sample data is constructed with the tank cleaning voice command as the sample attribute and the corresponding tank cleaning command paradigm as the sample label to form a tank cleaning data set;

[0022] Build a standard tank cleaning action library to store the action keywords of tank cleaning operations;

[0023] The tank cleaning dataset is expanded from two perspectives: language habits and action keywords.

[0024] On the basis of the above technical solution, preferably, the path planning of each tank cleaning robot in combination with the tank cleaning instruction paradigm specifically includes:

[0025] Determine the position of the tank cleaning robot and the position of the stockpile based on the real-time map inside the bulk cargo ship's hold;

[0026] Determine the starting and ending points of the tank cleaning operation according to the tank cleaning instruction paradigm;

[0027] The optimal path for the cleaning robot is planned by the adaptively improved A-Star algorithm; the evaluation function of the adaptively improved A-Star algorithm is:

[0028] ;

[0029] in, i Indicates the current node. i =1,2,…, n , n Indicates the total number of nodes, is the cost from the starting point through the current node to the end point, It is a comprehensive cost function, which is used to calculate the cost from the starting point to the current node; is a heuristic function used to calculate the estimated value from the current node to the end point;

[0030] The comprehensive cost function The expression is as follows:

[0031]

[0032] in, They represent the material cost function, the tank cleaning action cost function, the tank cleaning operation effect cost function, and the tank cleaning corner smoothing cost function, respectively. are the corresponding weight coefficients respectively;

[0033] The heuristic function The expression is as follows:

[0034]

[0035] in, is the starting point coordinate, is the end point coordinate.

[0036] Based on the above technical solution, preferably, it is characterized in that the material cost function The expression is:

[0037]

[0038] in, Indicates The material height of each node, Indicates the material height when the material cost is the largest. Indicates the degree to which the material cost varies with the material height. Indicates the height of the material at the critical state, and All are constant coefficients.

[0039] On the basis of the above technical solution, preferably, it is characterized in that the tank clearing action cost function The expression is as follows:

[0040]

[0041] in, They are the cost of shoveling, pushing, piling, scraping and unloading for the cleaning actions. are the weight coefficients of the cost values ​​of each cleaning action;

[0042] The cost function of the cleaning operation effect The expression is as follows:

[0043]

[0044] in, They are the cost values ​​of four types of tank cleaning operations. are the weight coefficients of the cost value of each cleaning operation effect.

[0045] On the basis of the above technical solution, preferably, the tank cleaning corner smoothing cost function The mathematical expression is:

[0046]

[0047] in, For the known m The objective function obtained by linear regression fitting of corner path sample data is: For the t The input value of the sample data, For the t The fitted output value of the sample data, For the t The actual output value of the sample data, m Represents the total amount of sample data, t =1,2,…, m .

[0048] In a second aspect, the present invention discloses an interactive control system for multiple cleaning robots based on a large language model, the system comprising:

[0049] Multi-machine remote simulation driver: used to remotely control multiple cleaning robots through the human-machine interaction interface, joystick and remote switch button;

[0050] Multi-cabin visual display system: used to display the working status of each cabin cleaning robot and the real-time point cloud in the cabin;

[0051] Intelligent cabin cleaning interactive system: used to establish interactive multi-cabin cleaning operation modes, control the multi-robot remote control system according to the corresponding operation modes and path planning results to perform multi-cabin and multi-cabin cleaning robot collaborative operations, and automatically switch the operation mode of each cabin cleaning robot according to the cleaning progress;

[0052] Multi-robot remote control system: used to establish communication between each tank cleaning robot and the multi-machine remote simulation driver and the intelligent tank cleaning interactive system, and receive control instructions from the multi-robot remote control system and the intelligent tank cleaning interactive system to control each tank cleaning robot.

[0053] On the basis of the above technical solutions, preferably, the establishment of an interactive multi-cabin cleaning operation mode and the coordinated operation of multiple cabins and multiple cabin cleaning robots according to the corresponding operation modes and path planning results specifically include:

[0054] The tank cleaning process is divided into three parts: robot autonomous tank cleaning, voice interactive tank cleaning and manual remote control tank cleaning.

[0055] Design corresponding autonomous cleaning operation mode, voice interactive cleaning operation mode and manual remote control cleaning operation mode for the three links respectively;

[0056] Among them, the autonomous cabin cleaning operation mode means that when there are a large number of materials in the cabin, the cabin cleaning robot can realize autonomous navigation, obstacle avoidance and path planning in the cabin and perform the cabin cleaning task;

[0057] The voice interaction tank cleaning operation mode is to convert the tank cleaning voice command into a tank cleaning command paradigm executable by the tank cleaning robot by building a tank cleaning command paradigm template, and establish a tank cleaning data set; train the large language model through the tank cleaning data set to obtain a tank cleaning language understanding model; when there are scattered materials left in the cabin, the operator inputs the tank cleaning voice command, and the tank cleaning language understanding model generates a tank cleaning command paradigm according to the tank cleaning voice command, establishes a real-time map of the bulk carrier cabin, and combines the tank cleaning command paradigm to perform path planning for each tank cleaning robot; the tank cleaning robot performs the tank cleaning task in the cabin according to the path planning result corresponding to the tank cleaning command paradigm;

[0058] Manual remote control tank cleaning operation mode means that the operator remotely controls the tank cleaning robot through a multi-machine remote simulation driver to perform tank cleaning tasks in dangerous areas prone to collision;

[0059] The cabin cleaning robots of multiple cabins perform cabin cleaning operations in the corresponding cabins simultaneously according to the corresponding operation modes.

[0060] On the basis of the above technical solution, preferably, the automatic switching of the operation modes of each tank cleaning robot according to the tank cleaning progress specifically includes:

[0061] Establish evaluation indicators for adaptive switching of the operation mode of the cabin cleaning robot, including shoveling operation time, completion of shoveling operation, distance from the cabin side wall, degree of obstacles around the robot, tortuosity of the planned path, and repetition between the operation path and the planned path; evaluate the task completion level of each cabin cleaning robot through the operation mode adaptive switching evaluation system;

[0062] A multi-level fuzzy comprehensive evaluation method is used to construct an evaluation system for adaptive switching of operation modes, and the weights of various evaluation indicators are calculated;

[0063] The need to switch the operating mode is determined based on the task completion level of each cleaning robot, and the operating mode is automatically switched.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] 1) The present invention constructs a tank cleaning data set by constructing a tank cleaning instruction paradigm template and training a large language model, thereby realizing the conversion of the operator's tank cleaning voice instruction to the tank cleaning instruction paradigm executable by the tank cleaning robot. Based on voice interactive control, refined tank cleaning operations can be performed on specific areas, thereby enhancing the unmanned operation capability of tank cleaning, and realizing unmanned tank cleaning operations for multiple cargo holds in bulk carriers at the same time, without the need for personnel to operate across / inside the holds, thereby reducing the labor cost and equipment standby time of the tank cleaning operation, and effectively improving the safety and efficiency of the tank cleaning operation.

[0066] 2) The present invention constructs a standard tank cleaning action library, fully considers the influence of multiple languages, multiple dialects, language expression logic and expression fluency, and expands the tank cleaning data set from the two perspectives of language habits and action keywords. The data set expansion method enriches the tank cleaning data set. On the one hand, it can help the tank cleaning language understanding model to better handle language variants and subtle differences in dialects, improve the understanding, translation and fault tolerance of specific contexts, greatly reduce the difficulty of operators in voice interaction, and improve the accuracy and performance in multi-language scenarios; on the other hand, it enriches the tank cleaning action keywords, so that the tank cleaning language understanding model generates a more accurate tank cleaning instruction paradigm, greatly reducing the difficulty of operators in voice interaction, thereby improving the accuracy of tank cleaning operations.

[0067] 3) The present invention combines the tank cleaning instruction paradigm, and uses the adaptively improved A-Star algorithm to plan the path of each tank cleaning robot, and optimizes the heuristic function of the A-Star algorithm. Compared with the common Euclidean distance, Manhattan distance and other methods, the optimized heuristic function of the present invention visits fewer nodes and the path is closer to the actual nodes; in addition, the present invention considers the comprehensive cost function of the tank cleaning robot when performing operations from four levels: material cost, tank cleaning action, tank cleaning operation effect, and smoothness of tank cleaning corners, and plans the optimal path for the tank cleaning robot, which can effectively improve the efficiency, quality and energy saving level of voice interactive tank cleaning operations.

[0068] 4) The present invention designs an interactive control system for multiple tank cleaning robots based on a large language model, which divides the tank cleaning process into three parts: autonomous tank cleaning by tank cleaning robots, voice interactive tank cleaning, and manual remote control tank cleaning. A corresponding tank cleaning operation mode is designed for each part, so that one operator can control multiple tank cleaning robots to operate in multiple cabins at the same time, realizing a "single person to multiple machines" tank cleaning operation, achieving cost reduction and efficiency improvement on the basis of improving the level of automation, and improving operating efficiency.

[0069] 5) The present invention uses a multi-level fuzzy comprehensive evaluation method to establish an operating mode adaptive switching evaluation system to evaluate the task completion level of each cleaning robot. The operating mode is switched autonomously according to the task completion level of each cleaning robot, and the real-time dynamic adjustment of the three cleaning operation modes of robot autonomous cleaning, voice interactive cleaning and manual remote control cleaning is realized. It can optimize resource allocation, quickly adapt to changing working environments and requirements, and improve cleaning speed and flexibility.

[0070] 6) The interactive control system of multiple tank cleaning robots based on a large language model of the present invention can effectively reduce the danger of the tank cleaning process and improve the cleaning efficiency, which is beneficial to the improvement of the equipment technology level in the bulk terminal industry and the overall operation efficiency of the port. It has great application potential and broad promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0072] Figure 1 A flow chart of an interactive control method of multiple tank cleaning robots based on a large language model of the present invention;

[0073] Figure 2This is an architecture diagram of an interactive control system for multiple cabin cleaning robots based on a large language model of the present invention;

[0074] Figure 3 A layout diagram of a multi-machine remote control simulation driving device of the present invention;

[0075] Figure 4 is a schematic diagram of a multi-cabin visual display system of the present invention;

[0076] Figure 5 The present invention is a flowchart of the multi-cabin and multi-machine full-process cabin clearing operation. DETAILED DESCRIPTION

[0077] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] See also Figure 1 The present invention proposes an interactive control method for multiple cabin cleaning robots based on a large language model, the method comprising:

[0079] S1. Construct a tank cleaning instruction paradigm template, convert the tank cleaning voice instructions into a tank cleaning instruction paradigm executable by the tank cleaning robot through the tank cleaning instruction paradigm template, and establish a tank cleaning data set.

[0080] The present invention develops a tank cleaning instruction paradigm so that any tank cleaning voice instruction can be converted into a tank cleaning instruction paradigm executable by a tank cleaning robot, and then produces a tank cleaning data set, and develops a filtering strategy for erroneous instructions and a tank cleaning data set self-expansion rule to enrich the tank cleaning data set.

[0081] Step S1 specifically includes the following sub-steps:

[0082] S11. Construct a template for the cabin clearance instruction paradigm.

[0083] The tank clearing instruction paradigm template of the present invention is used to convert the operator's operating voice instructions into a tank clearing instruction paradigm, and the tank clearing instruction paradigm template includes "operation sequence + starting point + end point + action keywords + tank clearing operation effect + cycle judgment + pause operation + auxiliary information" and other contents.

[0084] Among them, the operation sequence is the sequence of moving operations between different operation positions, and the number of times is related to the number of single operations decomposed into the input command and the number of cycles; movement within the same operation position is a single operation; the starting point and the end point are the starting point and the end point in a single operation, and the basis for setting the starting point and the end point is to manually or automatically generate this different operation area through SLAM positioning and mapping, and then provide corresponding coordinate information for the starting point and the end point; the action keywords are the cabin clearing actions such as shoveling, pushing, piling, scraping and unloading between two positions in a single operation; the cabin clearing operation effect has four options: precise, safe, stable and efficient, which are set according to the needs of the cabin clearing operation; the loop judgment is used to judge whether a single operation needs to be repeated according to the loop judgment condition; for example, the loop judgment condition is that the material at a certain position is shoveled clean, then before the material at a certain position is shoveled clean, new shoveling tasks will continue to be inserted into the operation sequence after the shoveling action in the operation sequence is completed; pausing the operation refers to whether the action needs to be temporarily stopped, such as avoiding moving targets such as grabs; auxiliary information is used to store other useful information for easy function expansion.

[0085] S12, converting the tank cleaning voice command into a tank cleaning command paradigm executable by the tank cleaning robot.

[0086] For the same action, different operators will give different types of clearing voice instructions, for example:

[0087] Operator A: Quickly shovel the material from the left to the right, then pause for 5 seconds and move forward at a constant speed to the middle point;

[0088] Operator B: Quickly shovel the material from the left to the right, then pause for 5 seconds and move to the middle point;

[0089] Operator C: The material is shoveled from the left to the right at high speed, paused for 5 seconds and then moved to the middle point;

[0090] Operator D: Clean the material, shovel from left to right, finish quickly, stop for 5 seconds and go to the middle point;

[0091] Operator E: Move the material from the right to the middle point. Before that, shovel the material first, quickly shovel from the left to the right, and pause for 5 seconds after completing the shoveling.

[0092] In order to obtain accurate tank clearing instructions, the present invention converts the above tank clearing voice instructions into a unified tank clearing instruction paradigm through a tank clearing instruction paradigm template. Table 1 is an example of the use of the tank clearing instruction paradigm.

[0093] Table 1 Example of using the clearing instruction pattern

[0094]

[0095] S13: Build a tank cleaning dataset and expand the dataset.

[0096] S131, constructing sample data with the tank clearing voice command as the sample attribute and the corresponding tank clearing command paradigm as the sample label to form a tank clearing data set.

[0097] S132: construct a standard tank cleaning action library for storing action keywords for tank cleaning operations.

[0098] The tank cleaning robot described in the present invention is an excavator, loader and other equipment. According to the needs of the actual tank cleaning project, a standard tank cleaning action library that can describe the overall tank cleaning action is established through a large amount of research. All tank cleaning actions can only come from the standard tank cleaning action library. Specifically, a questionnaire survey can be issued to summarize the type keywords of tank cleaning operation actions from talents in the industry or field; review and screen the tank cleaning operation action keywords to determine the tank cleaning operation action keywords that can be used; manually enter the determined tank cleaning operation action keywords; update and modify the action keywords to determine the final standard paradigm tank cleaning action keywords; update the entered standard paradigm action keyword dialect data set library.

[0099] The summarized action keyword instruction information table of the cleaning robot's cleaning operation is shown in Table 2.

[0100] Table 2 Keywords of cabin cleaning robot actions

[0101]

[0102] S133. Offline expansion of the cabin cleaning dataset from the perspectives of language habits and action keywords.

[0103] Since the collected tank clearing voice instructions and the corresponding tank clearing instruction paradigms are limited, the present invention combines the actual situation of tank clearing operations and expands the tank clearing dataset from two perspectives: language habits and action keywords. The expansion of the tank clearing dataset based on language habits includes the expansion of multiple languages, multiple dialects, language expression logic, and expression fluency.

[0104] (1) Design expansion rules that take into account multiple languages ​​and dialects to expand the cabin cleaning dataset.

[0105] When designing a tank cleaning robot, since operators come from different regions, their language habits may be inconsistent, including not speaking Mandarin, stuttering, or mixing dialects and Mandarin, etc., which will cause the tank cleaning robot to be unable to accurately understand the command information to perform tank cleaning work. To solve this problem, the present invention first gives priority to the Mandarin model, and sets up autonomous tank cleaning through fixed Mandarin in the industry-related tank cleaning instructions and paradigms. Then, a large language model is used to expand the tank cleaning data set, and the dialect languages ​​related to the tank cleaning instruction information in various places are manually entered. By continuously learning and optimizing the local data set, the language and dialect instruction types of the tank cleaning data set are enriched.

[0106] (2) Design expansion rules that conform to the language expression logic to expand the cleaning data set.

[0107] When the operator inputs the voice command for clearing the tank, the whole sentence of the voice command for clearing the tank often includes the elements of subject, predicate, object, attributive, adverbial, and complement, among which the subject, predicate, and object must be included. Different operators have different ways of expression and order of expression, including the expression of the same sentence may have different understandings due to different order placement. Considering the different logical order of language expression, the present invention sets the fixed position of the subject, predicate, object, attributive, adverbial, and complement words in a paragraph. No matter how the operator changes the position of the modifier in the whole paragraph, the tank clearing language understanding model can convert the tank clearing voice command into a standard tank clearing command paradigm in a timely manner according to the set language expression logic after receiving the tank clearing voice command input by the operator, and notify the tank clearing robot to perform the tank clearing operation.

[0108] (3) Design expansion rules that combine action logic and expression fluency to expand the cabin cleaning dataset.

[0109] Since operators may be nervous, stumble in speech, or other factors when inputting the voice command for clearing the cargo hold, which may cause a certain keyword or action to appear repeatedly in a whole sentence, in order to achieve the combination of action logic and expression fluency, the present invention deletes the repeated words when a certain keyword or action word appears repeatedly. For example, if an operator says, "Car, shovel the material, shovel, shovel to the right", the words that do not conform to the action logic and language expression fluency can be deleted by capturing the keywords and action keywords of the language expression logic.

[0110] (4) Expansion of the cabin cleaning dataset based on action keywords.

[0111] Based on the standard tank cleaning action library in step S132, the action keywords of the tank cleaning robot can be summarized, and the operator may encounter new tank cleaning operation action keywords in the tank cleaning operation process. Through the feedback function, new action keywords can be added in real time to enrich the tank cleaning data set library, and the local action keyword data set library can be continuously optimized and improved, so that the tank cleaning operation can be carried out more safely, efficiently and accurately next time.

[0112] Through the above methods, the tank cleaning data set can be expanded offline to improve the effect of generating the tank cleaning instruction paradigm. For example, the operator inputs the voice instruction information "Push the material from the left to the right" and changes it to "Please quickly push the material on the left to the right until the material on the left is cleared", that is, in the expansion of the tank cleaning data set "position + action + effect + number of navigation target points + number of cycles", the corresponding tank cleaning instruction paradigms in the whole sentence structure are: left, right, push, quickly, until there is no material on the left, which expands the operation effect of the action keyword, and until there is no material, enriching the language expression ability.

[0113] S2. Train a large language model using the cabin cleaning dataset to obtain a cabin cleaning language understanding model.

[0114] The large language model is trained by the above tank clearing dataset to realize the transformation from tank clearing voice instructions to tank clearing instruction paradigm.

[0115] The present invention adopts GPT, a large language model, for training to obtain a clearing language understanding model, which is named Clearing-GPT.

[0116] In addition to the above offline expansion method of the tank cleaning dataset, the tank cleaning dataset can also be expanded online during the tank cleaning operation to update the tank cleaning language understanding model.

[0117] During the training of the large language model, the cleaning robot will perform the cleaning operation according to the cleaning instruction paradigm generated by the cleaning language understanding model. The operator will input a data level related to the cleaning effect (divided into four levels):

[0118] Failed: The cleaning task failed and the overall framework of cleaning was not fully implemented;

[0119] Pass: The task of cleaning the tank can be basically completed, but the materials in the area are not completely cleaned and have not been completely moved and cleaned;

[0120] Good: The cleaning task is completed, and the corners and specific areas are cleaned. However, there is a certain delay error in the cleaning process because the input command is not responded to in time, which leads to a longer cleaning operation time;

[0121] Excellent: The cleaning task is completed and the operator's voice cleaning instructions are well understood and the corresponding cleaning actions are performed in a short time. The area and corners are cleaned and moved completely, and there is no material visible to the naked eye.

[0122] Based on the above evaluation criteria, higher training weights are given to excellent tank-clearing voice commands and corresponding tank-clearing command paradigms, making it easier for Clearing-GPT to generate excellent tank-clearing command paradigms.

[0123] S3. Obtain the tank clearing voice instruction of the bulk cargo hold to be processed, and generate a tank clearing instruction paradigm according to the tank clearing language understanding model.

[0124] After the tank cleaning language understanding model is trained in step S2, it can be used to generate a tank cleaning instruction paradigm for the bulk cargo tank to be processed.

[0125] The operator's tank cleaning voice command is obtained and input into the tank cleaning language understanding model, and the corresponding tank cleaning command paradigm is output. The tank cleaning command paradigm is the tank cleaning command that can be executed by the tank cleaning robot.

[0126] In practical applications, errors are inevitable when converting tank cleaning voice commands into tank cleaning operation paradigms. The present invention designs an intelligent tank cleaning interactive error command filtering strategy to filter or correct some commands that are misidentified. The intelligent tank cleaning interactive error command filtering strategy includes:

[0127] a. Filtering rules for wrong cabin cleaning instructions based on operation rules. Operation rules include operation logic, operation space, robot motion constraints, etc. For example, if the starting point and end point of the instruction are beyond the range of the cabin, then the cabin cleaning operation paradigm must be wrong.

[0128] b. Filtering rules for wrong tank cleaning instructions based on observation information. The starting point, end point, and action are considered in combination with observation information. For example, if the instruction is to shovel material from point a to point b, and if there is no material at points a and b in the observation information, then the tank cleaning operation paradigm is likely to be wrong. After identifying the wrong instruction, the operator is reminded through voice broadcast and asked whether to continue to execute the tank cleaning instruction paradigm.

[0129] S4. Establish a real-time map of the bulk carrier hold and plan the paths of each hold cleaning robot in combination with the hold cleaning instruction paradigm.

[0130] After establishing a real-time map of the bulk carrier hold, the starting and ending points of the tank cleaning operation are determined according to the tank cleaning instruction paradigm. Based on the position of the tank cleaning robot and the position of the stockpile, the most efficient path for the tank cleaning operation is planned for the tank cleaning robot through the adaptively improved A-Star algorithm.

[0131] The evaluation function of the adaptive improved A-Star algorithm adopted in the present invention is:

[0132] ;

[0133] in, i Indicates the current node. i =1,2,…, n , n Indicates the total number of nodes, is the cost from the starting point through the current node to the end point, It is a comprehensive cost function, which is used to calculate the cost from the starting point to the current node; is a heuristic function used to calculate the estimated value from the current node to the end point;

[0134] The comprehensive cost function The expression is as follows:

[0135]

[0136] in, They represent the material cost function, the tank cleaning action cost function, the tank cleaning operation effect cost function, and the tank cleaning corner smoothing cost function, respectively. are the corresponding weight coefficients respectively.

[0137] ① Material cost: When the cleaning robot is performing the cleaning operation, the real-time map is converted into a grid map, and the amount of remaining cargo in each grid map is different. The larger the cargo volume, the more difficult it is to push and shovel, and the higher the damage to the cleaning robot, so the material cost is greater. At this time, when the cleaning robot bucket is 80% full, this is the optimal bucket transfer volume. d , when the bucket volume reaches the saturation of the bucket, the material cost function at this time is , the expression is:

[0138]

[0139] in, Indicates The material height of each node, Indicates the material height when the material cost is the largest. Indicates the degree to which the material cost varies with the material height. Indicates the height of the material at the critical state, and All are constant coefficients.

[0140] ② Tank cleaning action: When performing tank cleaning operations, the tank cleaning robot will perform tank cleaning actions such as shoveling, pushing, piling, scraping, and unloading. In particular, different tank cleaning actions will be adjusted and changed during the shoveling operation. The more actions involved in the tank cleaning operation, the greater the difficulty of the shoveling operation, and the greater the damage to the tank cleaning robot. Therefore, the present invention designs a cost function for the tank cleaning action , when the cleaning action needs to be switched continuously during the cleaning operation, increase the cost function The weight of each cleaning action is set to adjust the proportion of the action in the entire cleaning operation. The cleaning action cost function The expression is as follows:

[0141]

[0142] in, They are the cost of shoveling, pushing, piling, scraping and unloading for the cleaning actions. are the weight coefficients of the cost values ​​of each cleaning action;

[0143] ③ Tank cleaning effect: When the tank cleaning robot performs tank cleaning, the operator may have different requirements and standards for the operation effect according to the placement of materials in the bulk carrier warehouse, environmental conditions and operation objectives. The tank cleaning effect set by the present invention is safety, efficiency, accuracy and stability. The present invention designs a cost function for the tank cleaning effect. , in order to select the appropriate cleaning operation effect according to different operation conditions, the cleaning operation effect cost function of the present invention is The expression is as follows:

[0144]

[0145] in, They are the cost values ​​of four types of tank cleaning operations. are the weight coefficients of the cost value of each cleaning operation effect.

[0146] ④ Smoothing of cleaning corners: When performing cleaning operations, the cleaning robot needs to consume a lot of energy, such as kinetic energy for pushing materials, mechanical energy for moving, and energy for overcoming opposing forces such as friction. This results in a large amount of fuel cost during the cleaning operation, and also a great loss to the robot itself. Therefore, the present invention sets a cost function for the smoothing of cleaning corners. , reducing redundant points and achieving smoothing of the corners of the cleaning path to reduce the cleaning cost of the cleaning robot. The mathematical expression of the cleaning corner smoothing cost function of the present invention is:

[0147]

[0148] in, is the objective function obtained by linear regression fitting of the known m corner path sample data, is the input value of the tth sample data, is the fitting output value of the tth sample data, is the true output value of the tth sample data, m represents the total amount of sample data, t=1,2,…,m.

[0149] For example, in the search path, you may encounter three or more consecutive nodes on the same line. This means that there are fewer materials that cannot be shoveled in this section of the path, and the access and calculation time of the redundant nodes in the middle can be deleted. If three nodes are on the same line, the redundant nodes in the middle can be deleted.

[0150] The present invention adopts entropy method to determine the comprehensive cost function The comprehensive weight of each indicator Similarly, for the cleaning action cost function, the entropy method is used to determine the cost value of each cleaning action And the corresponding weight coefficient ; For the cost function of the cleaning operation effect, the entropy method is used to determine the cost value of each cleaning operation effect And the corresponding weight coefficient Based on the needs of actual engineering and tank cleaning operations, the principal component analysis method can be used when necessary to increase the weight of a certain cost value so as to complete the tank cleaning operation more efficiently, safely and quickly.

[0151] The heuristic function plays a key role in optimal path planning. It guides the search direction of the A-Star algorithm. When the estimated cost value of the heuristic function is less than the true value, the search range of the A-Star algorithm becomes larger and the number of search nodes becomes more; when the estimated cost value of the heuristic function is greater than the true cost value, the search range will become smaller and the number of search nodes will also become smaller, but the generation of the optimal path cannot be guaranteed. In order to make the planned path more realistic, the present invention uses an optimized heuristic function. , the expression is as follows:

[0152]

[0153] in, is the starting point coordinate, is the end point coordinate.

[0154] Compared with the common Euclidean distance, Manhattan distance and other methods, the optimized heuristic function of the present invention visits fewer nodes and the path is closer to the actual nodes.

[0155] S5. Perform multi-cabin and multi-cabin cleaning robot collaborative operations according to the path planning results corresponding to the cabin cleaning instruction paradigm.

[0156] Step S3 completes the conversion of the tank-clearing voice command to the tank-clearing command paradigm, and step S4 completes the path planning based on the tank-clearing command paradigm. The tank-clearing robot can perform the tank-clearing operation according to the path planning result.

[0157] The method embodiment of the above steps S1 to S4 realizes control of the tank cleaning robot based on voice interaction, and the voice interaction control method is a voice interaction tank cleaning operation mode.

[0158] On the basis of the above method embodiments, the present invention also proposes a multi-tank cleaning robot interactive control system based on a large language model. The main difference between the interactive control system and the above method embodiments is that an autonomous tank cleaning operation mode and a manual remote control tank cleaning operation mode are newly added, and autonomous switching of the three operation modes can be realized.

[0159] like Figure 2 The figure shows an architecture diagram of a multi-tank cleaning robot interactive control system based on a large language model of the present invention, wherein the system includes a multi-machine remote simulation driver, a multi-cabin visual display system, an intelligent tank cleaning interactive system and a multi-robot remote control system.

[0160] Multi-machine remote simulation driver: used to remotely control multiple cleaning robots through the human-machine interaction interface, joystick and remote switch button.

[0161] like Figure 3 The layout diagram of the multi-machine remote simulation driver of the present invention is shown. In addition to the normal steering wheel, brake, throttle, display and other basic operating equipment, a shoveling mode switching lever is also set to adjust the shoveling mode of the cleaning robot; on the left side of the operating table, a remote control operation lever of the cleaning robot is set to operate the shoveling, bucket rotation and material dropping of the cleaning robot; the cleaning robot switching button and the cleaning robot voice control button are installed in the middle of the steering wheel, and the three cleaning robots can be freely switched and controlled. For example, when the operating table reminds the operator to operate the cleaning robot A, the operator presses the switching button on the operating table and switches to the A robot. At the same time, the real-time control status and surrounding environment of the cleaning robot A are also displayed on the display, and the auxiliary screen displays the working status of the remaining two cleaning robots. In addition, a sensing device is set at the bottom of the driving position, which can establish remote sensing with the cleaning robot. When the cleaning robot encounters vibration, the remote control cab also has the same sensing. A running button for controlling the autonomous operation of the cleaning robot is set on one side of the operating table.

[0162] Multi-cabin visual display system: used to display the working status of each cabin cleaning robot and the real-time point cloud in the cabin.

[0163] like Figure 4The schematic diagram of the multi-cabin visual display system of the present invention is shown. It uses a high-definition and high-refresh rate display, Intel CPU, selects an industrial computer as the vehicle-mounted main control, installs the Ubuntu system, builds the ROS environment, and writes the robot control driver and sensor data acquisition and return program. Build a cloud control center, and the multi-machine remote simulation driver also installs the Ubuntu system, builds the ROS environment, and writes the robot data display interface. The design layout of the multi-cabin visual display system is completed by a large main screen, two auxiliary small screens and a prompt board display. The main screen mainly displays the cleaning robot being controlled by the driver, and the auxiliary screen displays the other two intelligent cleaning robots operating autonomously. The main screen displays the working status of the main cleaning robot and its point cloud map in the cargo hold from multiple dimensions. The two auxiliary sub-screens mainly display the working status of the other two cleaning robots, and are equipped with prompt boards for reminders and business overviews, thereby realizing multi-cabin and multi-cleaning robot collaborative control, and completing the cleaning operation efficiently and safely.

[0164] Intelligent cabin cleaning interactive system: used to establish interactive multi-cabin cleaning operation modes, control the multi-robot remote control system according to the corresponding operation modes to perform multi-cabin and multi-cabin cleaning robot collaborative operations, and automatically switch the operation mode of each cabin cleaning robot according to the cleaning progress.

[0165] The intelligent tank cleaning interactive system of the present invention designs interactive multiple tank cleaning operation modes, including an autonomous tank cleaning operation mode, a voice interactive tank cleaning operation mode and a manual remote control tank cleaning operation mode.

[0166] The present invention divides the tank cleaning process into three links: autonomous tank cleaning by robot, voice interactive tank cleaning and manual remote control tank cleaning. Among them, autonomous tank cleaning by robot means that when there are a large amount of materials in the cabin, the tank cleaning robot collects environmental data through its own on-board intelligent perception system, performs path planning in the cabin to achieve autonomous navigation, obstacle avoidance, and performs tank cleaning tasks; voice interactive tank cleaning operation means that when there are scattered materials left in the cabin, the operator inputs a tank cleaning voice command, the tank cleaning language understanding model generates a tank cleaning command paradigm according to the tank cleaning voice command, and the tank cleaning robot performs the tank cleaning task in the cabin according to the path planning result corresponding to the tank cleaning command paradigm; manual remote control tank cleaning means that the operator controls the tank cleaning robot to perform tank cleaning tasks in dangerous areas prone to collisions through a remote control system. Then, an interactive multi-tank cleaning operation mode is designed based on the above three links. Autonomous cabin cleaning operation mode: Corresponding to the robot's autonomous cabin cleaning link, the cabin cleaning robot uses the on-board intelligent perception system (such as lidar, cameras and other sensors for environmental perception), positioning and mapping technology (using the SLAM algorithm to build an environmental map in real time and determine the robot's position), path planning and navigation technology (using the planning algorithm to generate obstacle avoidance paths and achieve autonomous movement), decision-making and control technology (the robot makes decisions and controls movement based on perception information and task requirements), and then achieves autonomous navigation, obstacle avoidance and completes the cabin cleaning task.

[0167] Voice interaction tank cleaning operation mode: corresponds to the voice interaction tank cleaning link, and is implemented based on Clearing-GPT. The operator turns on the tank cleaning robot, turns on the intelligent tank cleaning interactive system to link to each tank cleaning robot, and switches to the voice interaction tank cleaning operation mode by inputting voice commands on the Web-based multi-machine remote control interactive interface; when the tank cleaning robot needs to be controlled again to perform other tasks, only voice input "Hello, A / B / C" is needed to wake up the tank cleaning robot, without the need for the operator to remotely control the robot, and then the operator can input the tank cleaning voice command to perform the tank cleaning operation without manual operation.

[0168] In the voice interaction tank cleaning operation mode, the intelligent tank cleaning interactive system converts the tank cleaning voice instructions into a tank cleaning instruction paradigm executable by the tank cleaning robot by constructing a tank cleaning instruction paradigm template, and establishes a tank cleaning data set; trains the large language model through the tank cleaning data set to obtain a tank cleaning language understanding model; obtains the tank cleaning voice instructions of the bulk cargo hold to be processed, and generates a tank cleaning instruction paradigm according to the tank cleaning language understanding model; establishes a real-time map inside the bulk cargo hold, and performs path planning for each tank cleaning robot in combination with the tank cleaning instruction paradigm; each tank cleaning robot performs tank cleaning operations according to the path planning results. For the specific implementation process of the voice interaction tank cleaning operation mode, please refer to the previous embodiment of a multi-tank cleaning robot interactive control method based on a large language model, which will not be repeated here.

[0169] Manual remote control tank cleaning operation mode: Before starting the tank cleaning operation, the staff needs to have a comprehensive understanding of the situation in the cabin, including the type, quantity, distribution and other information of the cargo in the cabin. At the same time, it is also necessary to check the performance of the tank cleaning equipment to ensure that the equipment is in good working condition. According to the situation in the cabin, a reasonable tank cleaning operation plan is formulated, including the operation sequence, operation area, operation time, etc. A tank cleaning robot is selected for manual control using the multi-machine remote simulation driver and the Web-based multi-machine remote control interactive interface. The operator performs the tank cleaning work on the tank cleaning robot according to the cabin environment information displayed in real time in the Web interface. Through the remote control system, the staff can monitor the operation of the tank cleaning equipment in real time and adjust the operation plan in time. When the tank cleaning operation is completed, the staff needs to check the operation results to confirm that all the cargo has been cleared. Then, the tank cleaning equipment is turned off and the operation is ended. Multi-robot remote control system: used to establish communication between each tank cleaning robot and the multi-machine remote simulation driver and the intelligent tank cleaning interactive system, and receive control instructions from the multi-robot remote control system and the intelligent tank cleaning interactive system to control each tank cleaning robot.

[0170] The communication mechanism between the multi-machine remote simulator and the cleaning robots in the multi-robot remote control system uses the most advanced 5G network communication technology. The operator uses the web-based multi-machine remote control interactive interface on the multi-machine remote simulator to observe the operation of multiple cleaning robots in the cabin in real time.

[0171] like Figure 5 The flowchart of the multi-cabin multi-machine full-process tank cleaning operation of the present invention is shown. The full-process tank cleaning operation of "one person to multiple machines" of the present invention can be divided into three steps: the robot autonomously cleans the cargo hold when there are a large amount of materials in the cargo hold; the second tank cleaning based on voice interaction is performed when there are scattered materials left in the hold; and finally, dangerous areas prone to collisions such as the corners of the cargo hold are manually remotely controlled to clean the cargo hold. In different cabins, different tank cleaning robots can perform tank cleaning operations independently without interfering with each other. According to the placement of the material piles in the bulk carrier warehouse, different tank cleaning robots perform different modes of tank cleaning operations, and the same tank cleaning robot can perform multiple cleaning modes in the cargo hold, and three tank cleaning robots can be operated by one operator to clean multiple cabins at the same time, so as to perform operations efficiently and safely.

[0172] For example, three areas to be processed are divided, and three cleaning robots perform cleaning operations at the same time. Autonomous cleaning operations are realized according to the set parameters and given input instructions. The intelligent cleaning system cloud platform can simultaneously control three loaders to synchronously perform the same cleaning tasks and actions, that is, set the parameters on the cloud platform and enter the relevant cleaning instruction information. If you want the three robots to work together to clean the cabin at the same time and perform different cleaning operations, you only need to switch the target loader and enter the cleaning operation instructions. The cleaning robots in the three cabins do not interfere with each other. When the autonomous cleaning operation mode of any cabin ends, the staff can input voice cleaning instructions and switch to the voice interactive cleaning operation mode to control the cleaning robot operation. When a cleaning robot appears in any warehouse to clean dangerous areas such as the corners of the cargo hold that are prone to collision, the operator switches to the manual remote control cleaning operation mode and remotely controls the cleaning robot to perform the operation.

[0173] At the same time, the intelligent tank cleaning interactive system can be used to achieve autonomous switching between autonomous tank cleaning operation mode, voice interactive tank cleaning operation mode and manual remote control tank cleaning operation mode.

[0174] The present invention uses a multi-level fuzzy comprehensive evaluation method to perform a feasibility analysis on the working process of the cleaning robot. The cleaning robot will transmit the analysis results to the console interface, and the operator can choose whether to switch the working steps. While controlling the cleaning robot, other cleaning robots are still performing local operations.

[0175] First, the evaluation index of the adaptive switching of the tank cleaning robot operation mode is established. The present invention quantifies the tank cleaning operation index, which is mainly divided into the following indexes:

[0176] Loading operation time: refers to the time taken by the cargo clearing robot to perform loading operations on the prescribed path planning. The longer the time, the more difficult the loading is and the more residual cargo there is.

[0177] Completion degree of shoveling operation: refers to the amount of cargo left in the cabin after the cabin cleaning robot has completed the shoveling operation. The completion degree of shoveling cargo is judged based on the image of the residual cargo in the laser cloud point window, which can also be divided into good, better, general, poor, and bad.

[0178] Distance from the cabin side wall: refers to the straight-line distance between the cabin cleaning robot and the cabin side wall. When the straight-line distance is too short, it indicates that the probability of collision between the cabin cleaning robot and the cabin is greater. Similarly, the farther away from the cabin side wall, the lower the probability of collision with the cabin side wall.

[0179] The degree of obstacles around the robot: the complexity of the terrain around the cleaning robot, such as the density of the remaining cargo around it, and the obstacles nearby. The higher the obstacle degree, the greater the probability that the cleaning robot cannot avoid obstacles, and vice versa. Therefore, the degree of obstacles around the robot can also be divided into five levels: good, better, average, poor, and worst.

[0180] The tortuosity of the planned path refers to the degree of curvature of the path planned by the system during local path planning. The greater the curvature of the path, the greater the difficulty of the tank cleaning robot's operation. The smaller the curvature of the tank cleaning robot, the easier it is to complete the tank cleaning operation. The tortuosity is divided into five levels based on the system's own judgment: good, better, average, poor, and bad.

[0181] The repetition degree between the operation path and the planned path refers to the overlap between the trajectory of the cleaning robot after completing the cleaning operation and the trajectory of the planned path. The higher the overlap degree, the higher the efficiency of the cleaning operation, and the lower the repetition degree, the lower the efficiency of the cleaning operation.

[0182] Then, a multi-level fuzzy comprehensive evaluation method is used to construct an evaluation system for adaptive switching of operation modes.

[0183] The adaptive switching evaluation system of the operation mode includes three levels, namely the target level, the criterion level and the indicator level. The criterion level includes effectiveness, risk and planning. The effectiveness focuses on the efficiency of the cleaning robot to see whether the residues are shoveled clean. When there are a lot of residues to be shoveled, the probability of the residues being shoveled clean is very low, and it is necessary to evaluate whether the expected effect is achieved. The risk is that when the cleaning robot reaches the edge of the cabin or encounters obstacles, the cleaning robot will be in danger. At this time, a risk assessment is conducted. Planning is when the cleaning robot is running according to the optimal path during path planning. When encountering certain difficulties or the optimal path is very complicated, a planning evaluation is conducted. The indicator level includes the aforementioned shoveling operation time, the completion of the shoveling operation, the distance from the cabin side wall, the degree of obstacles around the robot, the tortuosity of the planned path, and the repetition of the operation path and the planned path.

[0184] Since the materials in the warehouse are placed in different positions, their weights and unevenness are inconsistent, the operating conditions and progress of each cleaning robot are inconsistent. The present invention is based on the above-mentioned operating mode adaptive switching evaluation system, calculates the weights of each evaluation index, and calculates the completion of each cleaning robot's task. Since the operating areas are divided, each cleaning robot performs cleaning operations in its own divided material area, and the overall weight and cleaning difficulty of the materials in each material area are inconsistent, and there is a difference in the cleaning operation time. Therefore, when one of the cleaning robots completes the cleaning operation, the computer will convert it into a completion rate based on the completion of the operation by the cleaning robot, and use the completion rate as the evaluation result. According to the calculation results obtained, five task completion levels are divided, namely excellent, good, medium, acceptable, and poor.

[0185] Finally, the adaptive operation mode is switched. The operator decides whether to switch to the voice interactive cleaning operation mode or the manual remote control cleaning operation mode according to the task completion level obtained by the cleaning robot that completes the cleaning operation first. If the task completion level is good or excellent, we believe that the autonomous cleaning operation mode of the cleaning robot is efficient and safe, and autonomous cleaning operations can be performed, and the cleaning operation algorithm does not need to be changed; if the task completion level is below good, it means that the cleaning robot has low efficiency and low safety, and the operator needs to perform remote operation and voice interactive cleaning to clean the remaining materials, and the cleaning operation algorithm will be continuously updated according to the cleaning situation, such as updating the Clearing-GPT model, so that the cleaning operation effect of the cleaning robot is continuously improved; when the cleaning robot continuously updates the cleaning operation algorithm, the tasks requiring human-machine interactive cleaning will also be continuously reduced as the algorithm is improved, greatly reducing the cleaning operation pressure of the staff.

[0186] In addition, the tank cleaning robot of the present invention also has voice broadcast and automatic charging functions. When the input command is wrong, the tank cleaning robot will give timely feedback and voice broadcast, and update the database in time. The operator can update the data set online to improve and expand the tank cleaning data set; when there is no control, the tank cleaning robot will temporarily stop working; and when any of the tank cleaning operations is completed, the robot that has completed the tank cleaning operation will voice broadcast and feedback to the on-site staff, so that they can input the command in time and let the loader go to the charging pile to charge autonomously in time.

[0187] The interactive control system of multiple tank cleaning robots based on a large language model of the present invention has the following advantages over the prior art:

[0188] The present invention proposes an unmanned tank cleaning method in a cabin that integrates the tank cleaning instruction paradigm technology, and replaces the current tank cleaning equipment with a tank cleaning robot, so that the tank cleaning personnel can be transferred out of the cabin to perform remote tank cleaning, thereby realizing unmanned operation in the cabin, improving the working efficiency of the tank cleaning machinery, avoiding personnel boarding the ship and entering the cabin, reducing unsafe factors such as human-machine intersection and machine-machine intersection, and ensuring the safety of personnel and equipment during the tank cleaning process.

[0189] The present invention has trained a large language model dedicated to port unloading and tank clearing - Clearing-GPT. Taking advantage of the advantages of large language models such as deep understanding, language conversion optimization, autonomous generation, and rapid response, a large language model in the field of port unloading and tank clearing is trained, and a tank clearing data set for bulk carrier tank clearing is produced through field investigation of port bulk carrier unloading and language system collection. The difficulty of voice interaction for operators is greatly reduced, avoiding the robot from being unable to obtain correct output results, and simplifying the process of remote interaction with the tank clearing robot to a certain extent.

[0190] The present invention improves on the traditional A-Star algorithm, combines the tank cleaning instruction paradigm to carry out path planning of the tank cleaning robot, comprehensively considers the cost value factors of each tank cleaning instruction paradigm, and improves operation safety and operation efficiency.

[0191] Taking into account the situation that the complex environment inside the cabin cannot support the fully autonomous operation of the tank cleaning robot and the staff needs to remotely control the robot to perform the tank cleaning operation, the present invention designs a multi-cabin and multi-machine full-process tank cleaning operation method for different tank cleaning stages, so that one operator can control three tank cleaning robots to operate in three cabins at the same time, realizing "single person to multiple machines" tank cleaning operation, achieving cost reduction and efficiency improvement on the basis of improving the level of automation, reducing resource consumption, and reducing 70% of operators, thereby improving operating efficiency.

[0192] The present invention constructs an adaptive switching evaluation system for operation modes, and the cleaning robot can switch autonomously among autonomous cleaning operation mode, voice interactive cleaning operation mode and manual remote control cleaning operation mode in a timely manner according to the completion of the operation, so as to further improve the cleaning efficiency.

[0193] The interactive control system of multiple tank cleaning robots based on a large language model of the present invention can effectively reduce the danger of the tank cleaning process and solve the problem of difficulty in recruiting workers. At the same time, the improvement of the degree of intelligence can improve the efficiency of tank cleaning work, which is 45-50% higher than the traditional tank cleaning method. The present invention is conducive to the improvement of the equipment technology level in the bulk cargo terminal industry and the overall operation efficiency of the port. With the advantages of 5G communication technology, it has great application potential and broad promotion prospects.

[0194] The above system embodiments are implemented based on the method embodiments. Please refer to the method embodiments for a brief description of the system embodiments.

[0195] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.

[0196] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to implement all or part of the steps of the method described in the embodiment of the present invention. The storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0197] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An interactive control method for multiple tank cleaning robots based on a large language model, characterized in that: The method comprises: Construct a tank cleaning instruction paradigm template, convert the tank cleaning voice instructions into a tank cleaning instruction paradigm executable by the tank cleaning robot through the tank cleaning instruction paradigm template, and establish a tank cleaning data set; The large language model is trained by using the cabin cleaning dataset to obtain the cabin cleaning language understanding model. Obtain the tank cleaning voice instructions of the bulk cargo hold to be processed, and generate the tank cleaning instruction paradigm according to the tank cleaning language understanding model; Establish a real-time map of the bulk cargo ship's hold and plan the paths of each cleaning robot based on the cleaning command paradigm; The multi-cabin and multi-cabin cleaning robots are operated collaboratively according to the path planning results corresponding to the cabin cleaning instruction paradigm; The tank cleaning instruction paradigm template includes operation sequence, starting point, end point, action keywords, tank cleaning operation effect, cycle judgment, pause operation and auxiliary information; The operation sequence is the movement operation sequence of the cleaning robot between different operation positions; The starting point and the end point are the starting point and the end point of a single operation; The action keywords are the cabin clearing action between two positions in a single operation; the action keywords include shoveling, pushing, piling, scraping and unloading; The tank cleaning operation effect is set according to the requirements of the tank cleaning operation; The cycle judgment is used to judge whether a single operation needs to be repeated according to the cycle judgment condition; The pause operation is used to record the time when the action needs to be temporarily stopped; The auxiliary information is used for function expansion; The path planning of each tank cleaning robot in combination with the tank cleaning instruction paradigm specifically includes: Determine the position of the tank cleaning robot and the position of the stockpile based on the real-time map inside the bulk cargo ship's hold; Determine the starting and ending points of the tank cleaning operation according to the tank cleaning instruction paradigm; The optimal path for the cleaning robot is planned by the adaptively improved A-Star algorithm; the evaluation function of the adaptively improved A-Star algorithm is: ； in, i Indicates the current node. i =1,2,…, n , n Indicates the total number of nodes, is the cost from the starting point through the current node to the end point, It is a comprehensive cost function, which is used to calculate the cost from the starting point to the current node; is a heuristic function used to calculate the estimated value from the current node to the end point; The comprehensive cost function The expression is as follows: ； in, They represent the material cost function, the tank cleaning action cost function, the tank cleaning operation effect cost function, and the tank cleaning corner smoothing cost function, respectively. are the corresponding weight coefficients respectively; The heuristic function The expression is as follows: ； in, is the starting point coordinate, is the end point coordinate; The material cost function The expression is: ； in, Indicates The material height of each node, Indicates the material height when the material cost is the largest. Indicates the degree to which the material cost varies with the material height. Indicates the height of the material at the critical state, and All are constant coefficients; The cost function of the clearing action The expression is as follows: ； in, They are the cost of shoveling, pushing, piling, scraping and unloading for the cleaning actions. are the weight coefficients of the cost values ​​of each cleaning action; The cost function of the cleaning operation effect The expression is as follows: ； in, They are the cost values ​​of four types of tank cleaning operations. are the weight coefficients of the cost values ​​of each cleaning operation effect; The clearing corner smoothing cost function The mathematical expression is: ； in, s (∙) is the known m The objective function obtained by linear regression fitting of corner path sample data is: For the t The input value of the sample data, For the t The fitted output value of the sample data, For the t The actual output value of the sample data, m Represents the total amount of sample data, t =1,2,…, m .

2. The interactive control method of multiple tank cleaning robots based on a large language model according to claim 1 is characterized in that: The establishment of the tank clearing data set specifically includes: The sample data is constructed with the tank cleaning voice command as the sample attribute and the corresponding tank cleaning command paradigm as the sample label to form a tank cleaning data set; Build a standard tank cleaning action library to store the action keywords of tank cleaning operations; The tank cleaning dataset is expanded from two perspectives: language habits and action keywords.

3. An interactive control system of multiple cleaning robots based on a large language model using the method of claim 1, characterized in that: The system comprises: Multi-machine remote simulation driver: used to remotely control multiple cleaning robots through the human-machine interaction interface, joystick and remote switch button; Multi-cabin visual display system: used to display the working status of each cabin cleaning robot and the real-time point cloud in the cabin; Intelligent cabin cleaning interactive system: used to establish interactive multi-cabin cleaning operation modes, control the multi-robot remote control system according to the corresponding operation modes and path planning results to perform multi-cabin and multi-cabin cleaning robot collaborative operations, and automatically switch the operation mode of each cabin cleaning robot according to the cleaning progress; Multi-robot remote control system: used to establish communication between each tank cleaning robot and the multi-machine remote simulation driver and the intelligent tank cleaning interactive system, and receive control instructions from the multi-robot remote control system and the intelligent tank cleaning interactive system to control each tank cleaning robot.

4. The interactive control system of multiple cleaning robots based on a large language model according to claim 3 is characterized in that: The establishment of the interactive multi-cabin cleaning operation mode and the coordinated operation of the multi-cabin and multi-cabin cleaning robots according to the corresponding operation modes and path planning results specifically include: The tank cleaning process is divided into three parts: robot autonomous tank cleaning, voice interactive tank cleaning and manual remote control tank cleaning. Design corresponding autonomous cleaning operation mode, voice interactive cleaning operation mode and manual remote control cleaning operation mode for the three links respectively; Among them, the autonomous cabin cleaning operation mode means that when there are a large number of materials in the cabin, the cabin cleaning robot can realize autonomous navigation, obstacle avoidance and path planning in the cabin and perform the cabin cleaning task; The voice interaction tank cleaning operation mode is to convert the tank cleaning voice command into a tank cleaning command paradigm executable by the tank cleaning robot by building a tank cleaning command paradigm template, and establish a tank cleaning data set; train the large language model through the tank cleaning data set to obtain a tank cleaning language understanding model; when there are scattered materials left in the cabin, the operator inputs the tank cleaning voice command, and the tank cleaning language understanding model generates a tank cleaning command paradigm according to the tank cleaning voice command, establishes a real-time map of the bulk carrier cabin, and combines the tank cleaning command paradigm to perform path planning for each tank cleaning robot; the tank cleaning robot performs the tank cleaning task in the cabin according to the path planning result corresponding to the tank cleaning command paradigm; Manual remote control tank cleaning operation mode means that the operator remotely controls the tank cleaning robot through a multi-machine remote simulation driver to perform tank cleaning tasks in dangerous areas prone to collision; The cabin cleaning robots of multiple cabins perform cabin cleaning operations in the corresponding cabins simultaneously according to the corresponding operation modes.

5. The interactive control system of multiple cleaning robots based on a large language model according to claim 3 is characterized in that: The automatic switching of the operation modes of each tank cleaning robot according to the tank cleaning progress specifically includes: Establish evaluation indicators for adaptive switching of the operation mode of the cabin cleaning robot, including shoveling operation time, completion of shoveling operation, distance from the cabin side wall, degree of obstacles around the robot, tortuosity of the planned path, and repetition between the operation path and the planned path; evaluate the task completion level of each cabin cleaning robot through the operation mode adaptive switching evaluation system; A multi-level fuzzy comprehensive evaluation method is used to construct an evaluation system for adaptive switching of operation modes, and the weights of various evaluation indicators are calculated; The need to switch the operating mode is determined based on the task completion level of each cleaning robot, and the operating mode is automatically switched.

Images