Automatic Control Method and System for Unmanned Vehicle Formation Based on Large Language Model

Abstract

The present application discloses an automatic control method and system for unmanned vehicle formations based on large language models. Among them, the method includes: analyzing the obtained task instructions to obtain task semantics, where the task semantics includes the number of unmanned vehicles, task objectives, and font styles; extracting the task formation in the task objectives and determining the task level according to the task formation; determining the target external tool based on the task level; obtaining the corresponding control strategy based on the target external tool; and performing automatic control of the unmanned vehicle formation according to the control strategy. This method can accurately extract key information such as the number of unmanned vehicles and task objectives, can dynamically adjust the control strategy according to the complexity and importance of the task, enabling the formation control to flexibly adapt to the task requirements of various different types. The intelligent strategy generation mechanism ensures the coordination and efficiency of the unmanned vehicle formation when performing tasks, effectively improving the operation efficiency, reliability, and formation accuracy.

Description

Technical Field

[0001] The present disclosure relates to the technical field of coordinated control of unmanned vehicle systems, and particularly to an automatic control method and system for unmanned vehicle formation based on a large language model. Background Art

[0002] In the rapidly developing autonomous systems, coordinating multiple unmanned vehicles to form a cohesive intelligent formation is a highly challenging task. The main task of formation control is to control the states of the agents in the formation to track or maintain a specific geometric formation shape. The traditional formation control strategy design mode is usually preset and static, relying heavily on predefined algorithms. Its inherent design mode limits the number and configuration of formations. Expanding the number scale and diversity of vehicle formations requires a fundamental reorganization or modification of the existing algorithms, lacking flexibility and generality. In complex, changeable, and unpredictable scenarios, it is difficult to flexibly adapt to different task requirements and the formation change requirements of unmanned vehicle fleets of different scales. Summary of the Invention

[0003] In view of this, the embodiments of the present disclosure provide an automatic control method and system for unmanned vehicle formation based on a large language model, which can solve the problems existing in the prior art, such as single solution, lack of flexibility, generality, and inability to meet the formation requirements of different scenarios.

[0004] In a first aspect, the embodiments of the present disclosure provide an automatic control method for unmanned vehicle formation based on a large language model, specifically including the following:

[0005] Analyze the obtained task instructions to obtain task semantics, where the task semantics include the number of unmanned vehicles, task objectives, and font styles;

[0006] Extract the task formation in the task objectives and determine the task level according to the task formation;

[0007] Based on the task level, determine the target external tool;

[0008] Obtain the corresponding control strategy based on the target external tool, where the control strategy includes unmanned vehicle formation position information;

[0009] Perform automatic control of the unmanned vehicle formation according to the control strategy.

[0010] Optionally, when the task formation belongs to the first formation library, the task level is the first level;

[0011] The first formation library includes letters and Chinese characters without curves and without cross-collision points;

[0012] When the task formation belongs to the second formation library, the task level is the second level;

[0013] The second formation library includes letters and Chinese characters containing curves and / or cross collision points.

[0014] Optionally, obtaining the corresponding control strategy based on the target external tool includes:

[0015] When the task level is the first level, the target external tool is the first type of external tool, and the corresponding control strategy includes arranging all unmanned vehicles according to a preset inter-vehicle distance and task formation;

[0016] When the task level is the second level, the target external tool is the second type of external tool, and the control strategy is obtained according to the second type of external tool and a preset strategy.

[0017] Optionally, obtaining the control strategy according to the second type of external tool and a preset strategy includes:

[0018] Obtaining the text information of the task formation and its font style, where the text information includes text content, text width, and text height;

[0019] Based on the text information, obtaining a target text image;

[0020] Converting the target text image into an array and performing binarization processing on the array to generate a binary image;

[0021] Extracting an initial skeleton image from the binary image, performing line width processing on the initial skeleton image to obtain a target skeleton image, where the line width in the target skeleton image is all one pixel wide;

[0022] Analyzing the target skeleton image based on a preset analysis strategy to obtain a control strategy.

[0023] Optionally, analyzing the target skeleton image based on a preset analysis strategy to obtain a control strategy includes:

[0024] Analyzing the target skeleton image to obtain all points with pixel values less than a preset threshold and their corresponding coordinate information;

[0025] Based on the coordinate information, determining uniform coordinate point information consistent with the number of unmanned vehicles.

[0026] Optionally, determining uniform coordinate point information consistent with the number of unmanned vehicles based on the coordinate information includes:

[0027] A100, recording all points with pixel values less than a preset threshold and their corresponding coordinate information in a first point set;

[0028] A200, randomly select a point from the first set of points as the target point, and move the target point to the second set of points;

[0029] A300, determine whether the number of points in the second set of points is the same as the number of driverless vehicles. If not, calculate the distances from all points in the first set of points to all points in the second set of points, obtain the point in the first set of points with the minimum distance to the second set of points, and denote it as the target point pair;

[0030] Record all the obtained target point pairs as the minimum distance point set;

[0031] A400, select the point with the maximum distance from the minimum distance point set and denote it as the target point, and move the target point to the second set of points;

[0032] Return to execute A300. When the number of points in the second set of points is the same as the number of driverless vehicles, use all the point information in the second set of points as the uniform coordinate point information.

[0033] Optionally, the automatic control method for driverless vehicle formation based on the large language model further includes:

[0034] Obtain the expected formation information based on the preset inter-vehicle distance and the uniform coordinate point information;

[0035] If the expected formation information conflicts with the actual arrangement, dynamically adjust the preset inter-vehicle distance and the uniform coordinate point information.

[0036] In a second aspect, the present application discloses an automatic control system for driverless vehicle formation based on the large language model, including:

[0037] An interaction module, including system-level prompts, constraint-level prompts, and user-level prompts. The system-level prompts are used to set roles and goals, the constraint-level prompts are used to ensure output standardization and accuracy, and the user-level prompts are used for human-computer interaction and customized control;

[0038] A tool-enhanced large language model, communicatively connected to the interaction module, for outputting a control strategy for automatic control of driverless vehicle formation according to the task instructions received by the interaction module and the automatic control method for driverless vehicle formation based on the large language model;

[0039] A driverless vehicle control module, communicatively connected to the tool-enhanced large language model, for automatically scheduling driverless vehicles for formation according to the control strategy.

[0040] Optionally, the automatic control system for driverless vehicle formation based on the large language model further includes a verification module, which is used to verify the control strategy. After the verification passes, the tool-enhanced large language model issues the control strategy to the driverless vehicle control module.

[0041] Optionally, if the control strategy fails the verification, the verification module sends an information indicating the verification failure to the tool-enhanced large language model, and the tool-enhanced large language model re-obtains the control strategy.

[0042] In a third aspect, embodiments of the present disclosure further provide a computer device, adopting the following technical solution:

[0043] The computer device includes:

[0044] At least one processor; and,

[0045] A memory communicatively connected to the at least one processor; wherein,

[0046] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute any one of the above-mentioned automatic control methods for unmanned vehicle formations based on a large language model.

[0047] In a fourth aspect, embodiments of the present disclosure further provide a computer-readable storage medium, which stores computer instructions for causing a computer to execute any one of the above-mentioned automatic control methods for unmanned vehicle formations based on a large language model.

[0048] In a fifth aspect, embodiments of the present disclosure further provide a computer program product, including a computer program / instructions, and when the computer program / instructions are executed by a processor, the steps of any one of the above-mentioned methods are implemented.

[0049] The automatic control method for unmanned vehicle formations based on a large language model disclosed in this application can accurately extract key information such as the number of unmanned vehicles, task objectives, and font styles by deeply analyzing task instructions. This powerful parsing ability ensures the precise execution of tasks and can effectively reduce errors; by extracting the task formation in the task objective and determining the task level according to the formation, the control strategy can be dynamically adjusted according to the complexity and importance of the task. This flexibility enables formation control to adapt to various different types of task requirements; generating corresponding control strategies according to the task level, including the position information of the unmanned vehicle formation. This intelligent strategy generation mechanism ensures the coordination and efficiency of the unmanned vehicle formation when executing tasks; finally, automatically controlling the unmanned vehicle formation according to the generated control strategy. This automatic control can effectively reduce the need for human intervention and improve the efficiency, reliability, and formation accuracy of operations.

[0050] The above description is only an overview of the technical solution of the present disclosure. In order to understand the technical means of the present disclosure more clearly, it can be implemented according to the content of the specification. In order to make the above and other purposes, features, and advantages of the present disclosure more obvious and understandable, the following preferred embodiments are specifically given and described in detail in conjunction with the accompanying drawings as follows. Description of the Drawings

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings required for use in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0052] Figure 1 It is a schematic flowchart of the automatic control method for unmanned vehicle formation based on a large language model provided by the embodiments of the present disclosure.

[0053] Figure 2 It is a schematic flowchart of the method for obtaining a control strategy according to a second type of external tool and a preset strategy provided by the embodiments of the present disclosure.

[0054] Figure 3 It is a schematic flowchart of the method for analyzing a target skeleton image based on a preset analysis strategy to obtain a control strategy provided by the embodiments of the present disclosure.

[0055] Figure 4 It is a schematic flowchart of the method for obtaining uniform coordinate point information provided by the embodiments of the present disclosure.

[0056] Figure 5 For Figure 4 a schematic flowchart of a specific embodiment of

[0057] Figure 6 It is a schematic flowchart of the inspection method for the safety inspection link provided by the embodiments of the present disclosure.

[0058] Figure 7 It is a schematic framework diagram of the automatic control system for unmanned vehicle formation based on a large language model provided by the embodiments of the present disclosure.

[0059] Figure 8 It is a schematic structural diagram of a computer device provided by the embodiments of the present disclosure. Detailed Embodiments

[0060] The embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings.

[0061] It should be clear that the following uses specific specific examples to illustrate the implementation manners of the present disclosure, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by the present disclosure.

[0062] It should be noted that the following describes various aspects of embodiments within the scope of the appended claims. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.

[0063] It also should be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present disclosure in a schematic manner. The components shown in the drawings only show the components related to the present disclosure, rather than being drawn according to the number, shape and size of the components in actual implementation. The types, quantities and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0064] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0065] Refer to Figure 1 , this application discloses an automatic control method for unmanned vehicle formation based on a large language model. The method specifically includes:

[0066] S100, obtain a task instruction.

[0067] Specifically, the task instruction can be a pre - programmed script or a real - time instruction issued by the user; the task instruction can exist in the form of text, voice or image, etc.; the multi - source task reception improves the flexibility and adaptability of the system, and can handle both fixed programming and dynamic instructions.

[0068] For example, the task instruction may be "Use 20 vehicles to form a convoy, adjust the convoy to form the letter B, and use Songti font style."

[0069] S200, analyzing the task instruction to obtain task semantics, where the task semantics includes the number of unmanned vehicles, task objectives, and font style.

[0070] Specifically, task instructions can be analyzed based on a large language model to obtain task semantics, which can effectively improve the accuracy and efficiency of instruction parsing and reduce the errors and time costs of manual parsing. The refinement of task semantics means higher task execution quality and efficiency.

[0071] For example, the extracted task semantics include: the number of unmanned vehicles is 20, the task objective is to adjust the fleet to form the formation of the letter B, and the font style is Songti.

[0072] S300, extracting a task formation in the task target, and determining a task level according to the task formation.

[0073] When the task formation belongs to the first formation library, the task level is the first level; wherein the first formation library includes letters and Chinese characters that do not contain curves and do not contain cross collision points.

[0074] When the task formation belongs to the second formation library, the task level is the second level; wherein the second formation library includes letters and Chinese characters containing curves and / or intersection collision points.

[0075] Furthermore, the first formation library includes one-line formation, two-line formation, three-line formation, L formation, T formation, A formation, E formation, F formation, H formation, I formation, K formation, M formation, N formation, V formation, W formation, X formation, Y formation, Z formation and other formations with simple strokes.

[0076] Furthermore, the second formation library includes B formation, C formation, D formation, G formation, J formation, O formation, P formation, Q formation, R formation, S formation, U formation and Chinese characters containing collision points; taking "中" as an example, there are two cross-collision points, namely the intersection of the vertical line and the flat mouth.

[0077] Determining the task level helps to allocate different resources and control strategies to different tasks, ensuring that each task can be properly handled, while improving the flexibility and adaptability of the system to handle tasks of different complexity.

[0078] For example, when the extracted task formation is B and it is determined that the task formation belongs to the second formation library, the corresponding task level is the second level.

[0079] S400. Determine the target external tool based on the task level;

[0080] Obtain the corresponding control strategy based on the target external tool.

[0081] Specifically, when the task level is the first level, the target external tool is the first type of external tool, and the corresponding control strategy includes: arranging all unmanned vehicles according to the preset inter-vehicle distance and task formation.

[0082] When the task level is the second level, the target external tool is the second type of external tool, and obtain the control strategy according to the second type of external tool and the preset strategy.

[0083] Through this step, ensure that each task can obtain the most suitable control strategy, improve the success rate and efficiency of task execution, and at the same time can reduce unnecessary high-cost operations, and the resource utilization is more efficient.

[0084] S500. Automatically control the formation of unmanned vehicles according to the control strategy.

[0085] Specifically, automatically regulate the movement of all unmanned vehicles according to the obtained control strategy to present an arrangement plan matching the task instructions, which can significantly reduce the need for manual operations, improve the automation level and efficiency of task execution, effectively enhance the reliability and stability of the system, and reduce the possibility of human errors.

[0086] The method for automatically controlling the formation of unmanned vehicles based on the large language model disclosed in this application can accurately extract key information such as the number of unmanned vehicles, task objectives, and font styles by deeply analyzing the task instructions through its powerful parsing ability, which ensures the accurate execution of tasks and can effectively reduce errors; by extracting the task formation in the task objective and determining the task level according to the formation, it can dynamically adjust the control strategy according to the complexity and importance of the task. This flexibility enables the formation control to adapt to various different types of task requirements; generate the corresponding control strategy according to the task level, including the position information of the unmanned vehicle formation. This intelligent strategy generation mechanism ensures the coordination and efficiency of the unmanned vehicle formation when performing tasks; finally, automatically control the formation of unmanned vehicles according to the generated control strategy. This automatic control can effectively reduce the need for human intervention, improve the efficiency, reliability, and formation accuracy of operations.

[0087] Refer to Figure 2 ., the method for obtaining the control strategy according to the second type of external tool and the preset strategy specifically includes the following steps:

[0088] S421. Obtain the text information of the task formation and its font style, where the text information includes the text content, text width, and text height.

[0089] Specifically, the text information of the task formation and its font style can be obtained based on the draw.textbbox method and drawn at the center of the image, using black as the drawing color to ensure that the image can be recognized as a binary image in subsequent steps.

[0090] In this embodiment, the recognition and processing of different task formations and font styles are supported, effectively improving the flexibility and adaptability of the system.

[0091] S422. Obtain the target text image based on the text information.

[0092] Specifically, the target text image can be drawn in a preset drawing software based on the text information.

[0093] For example, according to the extracted text information, such as "V-shaped" and "bold", the corresponding text image is generated; for example, the generated image shows a bold "V" shape with specific width and height parameters.

[0094] Converting the text information into an image facilitates the intuitive understanding of the task objectives. Especially in complex or abstract task instructions, the image processing helps to clearly display the task content. The image-based text information is convenient for subsequent image processing steps and improves the processing efficiency.

[0095] S423. Convert the target text image into an array and perform binarization processing on the array to generate a binary image.

[0096] Among them, in this binary image, the foreground object, that is, the target text, is highlighted in white, and the background is black.

[0097] Specifically, the generated text image can be converted into an array format, and each element of the array corresponds to a pixel point of the image; perform binarization processing on the array to convert each pixel point in the image into 0 or 1, where 0 represents the background and 1 represents the text content.

[0098] Through binarization processing, the complexity of the image can be effectively simplified, facilitating subsequent skeleton extraction and analysis. At the same time, the binary image reduces the data volume and can speed up the calculation speed of subsequent image processing steps.

[0099] S424. Extract the initial skeleton image from the binary image, perform line width processing on the initial skeleton image to obtain the target skeleton image, and the line widths in the target skeleton image are all one pixel wide.

[0100] Among them, the extracted initial skeleton image is a refined image.

[0101] Specifically, pixels on the boundary of the foreground object in the binary image can be identified (a pixel is considered a boundary pixel if only one or two of its neighboring pixels are foreground pixels). Through a lookup table (which determines which pixels can be removed based on the pattern of the 8 neighboring pixels of the pixel), the boundary pixels are removed while ensuring that the connectivity of the foreground object is not disrupted; this process of repeatedly identifying and removing the boundary is iterated until no more boundary pixels can be removed, until the initial skeleton image can be extracted.

[0102] In this step, the skeletonization process can accurately extract the main framework of the text, ensuring the accurate recognition of the task formation; the one-pixel-wide line processing standardizes the image format, facilitating subsequent analysis and control strategy generation.

[0103] S425. Analyze the target skeleton image based on a preset analysis strategy to obtain a control strategy.

[0104] Based on the analysis of the skeleton image, an intelligent control strategy can be generated to ensure the efficient cooperative operation of the unmanned vehicle formation; at the same time, by analyzing the skeleton image, the task formation can be accurately recognized, and the corresponding control strategy can be generated to ensure the accuracy and efficiency of task execution.

[0105] The method for obtaining a control strategy according to the first preset analysis strategy disclosed in this embodiment, through binarization processing and skeletonization processing of the text image, can reduce noise and redundant information in the image, making the final skeleton image clearer and more accurate, which helps to extract more accurate information in the subsequent analysis process, thereby obtaining a more accurate control strategy; converting the text image into an array and generating a binary image through binarization processing makes the image data more compact and the processing speed faster, which is particularly important for scenarios that require real-time processing of a large amount of data; by performing line width processing on the initial skeleton image, it is ensured that the line width in the final target skeleton image is all one pixel wide, which can avoid analysis errors caused by changes in image resolution or line width; analyzing the target skeleton image based on a preset analysis strategy can customize the analysis strategy according to different requirements and scenarios, so as to obtain a control strategy that better meets the actual needs, enhancing adaptability and flexibility.

[0106] Refer to Figure 3 , the method for analyzing the target skeleton image based on a preset analysis strategy to obtain a control strategy specifically includes the following steps:

[0107] S4251. Analyze the target skeleton image to obtain all points with pixel values less than the preset threshold and their corresponding coordinate information.

[0108] Specifically, an image processing library (such as OpenCV) can be used to read the target skeleton image first; traverse each pixel point in the image to determine whether its pixel value is less than a preset threshold (for example, the pixel value of the black skeleton in the white background is relatively low); record the qualified pixel points and their coordinate information (x, y) in a list.

[0109] Furthermore, pixel values less than 128 (black) can be set to True, and others to False.

[0110] Through the preset threshold, key points in the skeleton image can be accurately extracted, avoiding the interference of redundant information; traversing the image for pixel value analysis is an efficient algorithm that can quickly process a large amount of data.

[0111] S4252, based on the coordinate information, determine the uniform coordinate point information consistent with the number of unmanned vehicles.

[0112] Ensure that the distribution of unmanned vehicles in the image is balanced, avoiding the over-concentration or absence of unmanned vehicles in certain areas; through uniform distribution, the task allocation of unmanned vehicles can be optimized to ensure that each unmanned vehicle can effectively execute tasks.

[0113] The method for analyzing the target skeleton image based on the preset analysis strategy to obtain the control strategy disclosed in this embodiment, by accurately extracting the key points in the skeleton image and evenly distributing the coordinates of unmanned vehicles, ensures that each unmanned vehicle can efficiently execute tasks, effectively avoiding waste of resources; through the preset threshold and uniform distribution algorithm, the influence of environmental noise and image distortion on the system can be reduced; accurate coordinate information and uniform distribution strategy can ensure the accuracy and consistency of unmanned vehicles when executing tasks; decomposing complex image analysis tasks into simple pixel value analysis and coordinate processing can simplify the implementation and maintenance of the system; by adjusting the preset threshold and the number of unmanned vehicles, different task formations and font styles can be adapted, enhancing the flexibility and adaptability of the system.

[0114] Refer to Figure 4 , the method for determining the uniform coordinate point information consistent with the number of unmanned vehicles based on the coordinate information specifically includes the following steps:

[0115] A100, record all points with pixel values less than the preset threshold and their corresponding coordinate information in the first point set.

[0116] A200, randomly select a point from the first point set as the target point and move the target point to the second point set.

[0117] Wherein, after moving the target point to the second point set, the target point no longer exists in the original first point set, that is, each time a point is moved into the second point set, the corresponding point in the first point set is reduced.

[0118] Randomly select the initial points to ensure a certain degree of randomness in the starting points of the distribution, laying a foundation for subsequent uniform distribution; through random selection, the process of selecting initial points is simplified, avoiding complex calculations.

[0119] A300, determine whether the number of points in the second point set is the same as the number of unmanned vehicles. If not, calculate the distances from all points in the first point set to all points in the second point set, obtain the point in the first point set with the minimum distance to the second point set, and record it as the target point pair. Record all the obtained target point pairs as the minimum distance point set.

[0120] A400, select the point with the maximum distance from the minimum distance point set and record it as the target point, and move the target point to the second point set.

[0121] Select the point with the maximum distance, which helps to ensure that the newly selected point is as far away from the currently selected points as possible, guarantee the uniform distribution among unmanned vehicles, and at the same time can maximize the coverage of key areas in the image, ensuring that each unmanned vehicle can cover as many key points as possible; by maximizing the distance, the distribution of points can be gradually balanced, avoiding over-concentration of points.

[0122] Return to execute A300, and each time judge the consistency between the number of points in the second point set and the number of unmanned vehicles.

[0123] A500, when the number of points in the second point set is the same as the number of unmanned vehicles, use all the point information in the second point set as the uniform coordinate point information.

[0124] Ensure that the finally obtained number of points is the same as the number of unmanned vehicles, avoiding the problem of too many or too few points; through iterative optimization, the uniform distribution of points can be efficiently achieved, ensuring that each unmanned vehicle can effectively execute tasks.

[0125] The method disclosed in this embodiment for determining uniform coordinate point information consistent with the number of unmanned vehicles based on coordinate information can ensure that each target unmanned vehicle can efficiently execute tasks and effectively avoid waste of resources by accurately extracting key points in the skeleton image and gradually optimizing the distribution of points; through preset thresholds and distance calculations, the influence of environmental noise and image distortion on the system is effectively reduced, improving the robustness of the system; accurate coordinate information and uniform distribution strategies can ensure the accuracy and consistency of unmanned vehicles when executing tasks; decomposing complex image analysis tasks into simple pixel value analysis and distance calculations can simplify the implementation and maintenance of the system; by adjusting the preset thresholds and the number of unmanned vehicles, different task formations and font styles can be adapted, effectively enhancing the flexibility and adaptability of the system.

[0126] Further, referring to Figure 5, for example, in the initial state, the first point set contains N points and the second point set contains 0 points.

[0127] Randomly select a point from the first point set (N points) as the first point and move the first point to the second point set; at this time, the first point set contains N - 1 points and the second point set contains 1 point.

[0128] Secondly, calculate the distances from all points in the first point set to the first point, mark the point with the maximum distance as the second point, and move the second point to the second point set; at this time, the first point set contains N - 2 points and the second point set contains 2 points.

[0129] Then, calculate the distances from all points (N - 2 points) in the first point set to all points (2 points) in the second point set, obtain the point with the minimum distance between any point in the first point set and the second point set, and mark it as the target point pair. For example, first determine the point with the minimum distance between the first point in the first point set and the second point set to get the first target point pair; then determine the point with the minimum distance between the second point in the first point set and the second point set to get the second target point pair, and so on, determine the point with the minimum distance between the Qth point in the first point set and the second point set to get the Qth target point pair, and judge whether Q is equal to M, where M is the number of points currently contained in the first point set. If Q is less than M, recalculate until all points in the current first point set are traversed; when Q is equal to M, that is, after completing the distance calculation of all points in the current first point set to the second point set, the corresponding minimum distance point set can be obtained.

[0130] In this step, M = N - 2, so N - 2 pairs of points with the minimum distance are obtained. Denote all the obtained target point pairs as the minimum distance point set, that is, the minimum distance point set contains N - 2 pairs of target point pairs.

[0131] Select the point with the maximum distance from the minimum distance point set and mark it as the target point, and move the target point to the second point set; at this time, the first point set contains N - 3 points and the second point set contains 3 points; specifically, select the point with the maximum distance from the N - 2 pairs of points with the minimum distance as the next point to be moved to the second point set.

[0132] If the number of points in the second point set is not consistent with the number of unmanned vehicles, that is, it is still less than the number of unmanned vehicles, calculate the distances from all points (N - 3 points) in the first point set to all points in the second point set, obtain the point with the minimum distance between any point in the first point set and the second point set, and mark it as the target point pair. Denote all the obtained target point pairs as the minimum distance point set, and then move the point with the maximum distance in the minimum distance point set to the second point set. Repeat this process until the number of points in the second point set is consistent with the number of unmanned vehicles. Finally, use all the point information in the second point set as the uniform coordinate point information.

[0133] Refer to Figure 6, Further, the automatic control method for unmanned vehicle formation based on large language models disclosed in this application further includes a safety inspection link, and the inspection method of this safety inspection link specifically includes:

[0134] B100, Based on the preset inter-vehicle distance and uniform coordinate point information, obtain the expected formation information.

[0135] Specifically, the position information of each vehicle at the actual scale can be obtained according to the same method as the ratio of the inter-vehicle distance in the uniform coordinate point information to the preset inter-vehicle distance, that is, the expected formation information is obtained.

[0136] B200, If the expected formation information conflicts with the actual arrangement, dynamically adjust the preset inter-vehicle distance and uniform coordinate point information.

[0137] Specifically, compare the expected formation information with information such as the actual site to determine whether there is a conflict (for example, the expected positions of two or more unmanned vehicles overlap). If a conflict is found, dynamically adjust the preset inter-vehicle distance and uniform coordinate point information. The specific practices can include: 1) Adjust the inter-vehicle distance: increase or decrease the preset inter-vehicle distance to avoid overlap between unmanned vehicles; 2) Reallocate coordinates: According to the new inter-vehicle distance, recalculate the uniform coordinate point information to ensure that the positions of unmanned vehicles no longer conflict.

[0138] By dynamically adjusting the inter-vehicle distance and coordinate information, it can adapt to changes in the actual arrangement in real time, avoiding conflicts caused by environmental changes or adjustments in the number of unmanned vehicles; through the safety inspection link, it ensures that the positions of unmanned vehicles in the formation are reasonable, avoiding collisions and mutual interference, and improving the safety of the formation.

[0139] Refer to Figure 7 , Second, this application discloses an automatic control system for unmanned vehicle formation based on large language models, including:

[0140] An interaction module, including an interactive dialogue window, system-level prompts, constraint-level prompts, and user-level prompts. The system-level prompts are used to set roles and goals, the constraint-level prompts are used to ensure output standardization and accuracy, and the user-level prompts are used for human-computer interaction and customized control;

[0141] A tool-enhanced large language model, communicatively connected to the interaction module, and used to output a control strategy for automatic control of unmanned vehicle formation according to the task instructions received by the interaction module and the automatic control method for unmanned vehicle formation based on large language models. Specifically, the tool-enhanced large language model includes a large language model and a tool collaboration module. The large language model can be used to analyze task instructions, and the tool collaboration module is used to call corresponding target external tools according to the task level to obtain corresponding control strategies.

[0142] The unmanned vehicle control module is communicatively connected to the tool-enhanced large language model and is used to automatically dispatch unmanned vehicles for formation according to the control strategy.

[0143] Among them, the interaction module adopts a multi-level structure design, provides overall guidelines for the large model through system-level prompts, defines the output of the large model using constraint-level prompts to ensure its relevance and accuracy, and serves as a channel for human-computer interaction through user-level prompts, allowing users to manipulate the behavior of the large model through intuitive commands.

[0144] The set system-level prompts are mainly responsible for defining the role of the large language model when performing tasks and its basic task framework. It is the core mechanism for the model to understand the task background and provides macro guidance for the processing of task instructions. For example, a "formation commander" role can be defined for the large model first. This role assigns clear responsibilities and task scopes to the model, making it more focused on the specific scenario of unmanned vehicle formation, better understanding the behavior and dialogue of the role, rather than dealing with overly broad or irrelevant requests, thus generating more context-appropriate outputs. The system-level prompts also include task guidelines for the formation task, such as commanding the vehicle fleet to form a specific formation. Such high-level task guidance enables clear task goals when dealing with complex formations, ensuring that the model understands the user's intent and thus generating outputs that better meet actual needs.

[0145] The set constraint-level prompts can standardize and normalize the output content generated by the large model to ensure the relevance and accuracy of its output. The constraint-level prompts require the content generated by the model to be presented in a specific form (for example, coordinate data in JSON format) by setting a strict output format. This method not only ensures that the output can be directly read and executed by the unmanned vehicle control system but also improves the efficiency of system integration and reduces the time cost of manual intervention and data conversion. Compared with the traditional free output structure, the standardized output structure significantly improves the automation and real-time performance of the system. At the same time, precise numerical standards can be set for the output through constraint-level prompts. For example, the model must ensure requirements such as the minimum distance between unmanned vehicles. This technical improvement directly enhances the reliability of the system in dealing with complex environments.

[0146] The set user-level prompts allow users to manipulate the behavior of the final output formation plan through intuitive commands in an interactive dialogue window, achieving precise task control, directly reflecting the user's intent and needs, and being able to perform corresponding formation control operations according to the user's instructions.

[0147] Through user-level prompts, users can directly input the specific requirements of the formation task in natural language or structured command form. The flexible design of this module allows users to define multiple formation parameters, such as 1) Number of unmanned vehicles: users can specify the number of unmanned vehicles participating in the formation through instructions. For example, users can enter the command "Use 20 unmanned vehicles to form a formation" or "Automatically adjust the number of vehicles according to mission requirements", and the system will generate a suitable formation plan based on this. 2) Formation arrangement: users can require the formation to form a specific shape or character, such as "arrange into the letter B" or "form a square array". The system will generate an accurate formation that meets the requirements based on user instructions, with the cooperation of font libraries and coordinate calculation tools. 3) Formation style and style: users can choose different formation styles or styles according to visual or functional requirements. For example, by entering "Use Song style" or "Select modern style formation", the system can call different font libraries to display a variety of formation effects. This highly customized function meets the diverse needs in different scenarios.

[0148] The tool-enhanced large language model includes a tool collaboration module and a large language model. The tool collaboration module can call an external font library and the skeleton positioning of a coordinate calculation tool to implement the execution of Chinese and English character formation tasks in various styles.

[0149] Specifically, when the user inputs instructions through the interactive module, the tool-enhanced large language model can extract the name of the function call, the parameters of the function call, the text used to generate the target string skeleton, the number of unmanned vehicles (car_number), and the font style used when generating the skeleton from the received user input; secondly, create a white background image, set the color mode to grayscale mode, and load the specified font file from the external font library. Create a zero-size image, use the draw.textbbox method to calculate the width and height of the target string text, and draw it in the center of the image, and the drawing color is black; then convert the image to an array and binarize it to generate a binary image. Pixel values ​​less than 128 (black) are set to True, and others are False; next, use the skeletonize function of the skimage library to extract the skeleton of the image and generate a skeleton image with a width of only one pixel; finally, find all points in the skeleton image with True pixel values, obtain their coordinates, and evenly select car_number coordinate points from the skeleton.

[0150] Furthermore, the unmanned vehicle formation automatic control system based on the large language model also includes a safety module that can check and verify the output of the large model.

[0151] Specifically, the safety control module can parse the output and check whether it conforms to the expected JSON format. If the output content is not in the standard JSON format, the regular expression r'\[([\d\.]+), ([\d\.]+)\]' will be used to extract possible coordinate points from the text.

[0152] Furthermore, it can also check whether the generated coordinates match the number of vehicle fleets, specifically including: extracting the number of generated coordinate points and comparing it with the number of unmanned vehicle parameters specified by the user. If the number of generated coordinate points is less than or more than the number specified by the user, it is regarded as an anomaly; the safety control module will provide feedback "The number of generated coordinates is more / less than the number specified by the user. Please regenerate the output containing car_number coordinate points."

[0153] Furthermore, coordinate verification and physical feasibility checks can be performed, specifically including: calculating the distances between all unmanned vehicles to ensure that the distances between vehicles are greater than the safety threshold. If the distances between vehicles are too small (less than the set safety threshold), the safety control module will give feedback again, requiring adjustment of the distribution of coordinate points to increase the vehicle spacing; then, based on the feedback from the safety control module, re-understand the user's requirements and readjust the output results.

[0154] If the generated coordinates do not match the number of vehicle fleets, increase or decrease the generation of coordinate points and ensure that all generated points match the specified number of vehicles; if the distances between vehicles are too small, adjust the distribution of coordinate points to increase the vehicle spacing.

[0155] The automatic control system for unmanned vehicle formation based on the large language model disclosed in this application ensures the consistency and goal orientation of the system in task processing by setting roles and goals, thereby improving the efficiency of task processing; at the same time, it can ensure the standardization and accuracy of the output, reduce errors and deviations, and enhance the reliability and accuracy of the system; it supports human-computer interaction and customized control, enabling the system to be adjusted according to the specific needs of users and providing personalized control solutions.

[0156] By communicating and connecting with the interaction module, the tool-enhanced large language model can output suitable control strategies based on the received task instructions and the automatic control method for unmanned vehicle formation based on the large language model, enhancing the intelligence and adaptability of the system.

[0157] Each module of the system, such as the interaction module, the tool-enhanced large language model, the unmanned vehicle control module, and the verification module, can operate and be upgraded independently, providing good flexibility and scalability, facilitating the functional expansion and technological upgrade of the system in the future; user-level prompts enable the system to perform customized control according to the needs of specific users, enhancing the applicability and user experience of the system; the tool-enhanced large language model relies on a large amount of data and algorithms, can perform complex decision-making analysis, and generate efficient and intelligent unmanned vehicle formation control strategies, enabling the system to have a powerful data-driven decision-making ability.

[0158] In summary, this large language model-based unmanned vehicle formation automatic control system provides a comprehensive solution for the automatic control of unmanned vehicle formations through its high efficiency, accuracy, safety, reliability, flexibility, and intelligent decision-making ability, and has broad application prospects and important practical significance.

[0159] Furthermore, the large language model-based unmanned vehicle formation automatic control system further includes a verification module, which is used to verify the control strategy. After the verification passes, the tool-enhanced large language model sends the control strategy to the unmanned vehicle control module.

[0160] If the control strategy fails the verification, the verification module sends a verification failure message to the tool-enhanced large language model, and the tool-enhanced large language model re-obtains the control strategy.

[0161] Specifically, check whether the format of the uniform coordinate points is in json format. If not, use the regular expression r'\[([\d\.]+), ([\d\.]+)\]' to extract possible coordinate points from the text to be in json format.

[0162] In this embodiment, the verification module verifies the control strategy to ensure the security and effectiveness of the strategy; when the strategy passes the verification, it is directly sent to the unmanned vehicle control module, reducing human intervention and errors, and improving the automation level of the system.

[0163] The existence of the verification module makes it necessary for the control strategy to pass strict verification before execution, avoiding the implementation of inappropriate or dangerous strategies, enhancing the security and reliability of the system. If the control strategy fails the verification, the system will automatically re-obtain a new strategy, and this iterative optimization process helps the system to continuously improve, ensuring that the finally implemented strategy is optimal.

[0164] A computer device according to an embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0165] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In an embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device executes all or part of the steps of the automatic control method for unmanned vehicle formation based on the large language model in the foregoing embodiments of the present disclosure.

[0166] Those skilled in the art should understand that, in order to solve the technical problem of how to obtain a good user experience effect, known structures such as communication buses and interfaces may also be included in this embodiment, and these known structures should also be included in the protection scope of the present disclosure.

[0167] As Figure 8 FIG. is a schematic structural diagram of a computer device provided by an embodiment of the present disclosure. It shows a schematic structural diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 8 The shown computer device is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present disclosure.

[0168] As Figure 8 As shown, the computer device may include a processor (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). In RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0169] Generally, the following devices may be connected to the I / O interface: input devices including, for example, sensors or visual information acquisition devices; output devices including, for example, display screens; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. The communication device may allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or wiredly to exchange data. Although Figure 8A computer device with various devices is shown, but it should be understood that it is not necessary to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0170] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the method for automatically controlling the formation of large language model-based unmanned vehicles according to the embodiments of the present disclosure are executed.

[0171] For a detailed description of this embodiment, reference can be made to the corresponding descriptions in the foregoing embodiments, and details will not be repeated here.

[0172] A computer-readable storage medium according to an embodiment of the present disclosure stores non-temporary computer-readable instructions. When the non-temporary computer-readable instructions are run by a processor, all or part of the steps of the method for automatically controlling the formation of large language model-based unmanned vehicles according to the foregoing embodiments of the present disclosure are executed.

[0173] The above-mentioned computer-readable storage medium includes but is not limited to: optical storage media (such as CD-ROMs and DVDs), magneto-optical storage media (such as MOs), magnetic storage media (such as magnetic tapes or external hard drives), media with built-in rewritable non-volatile memories (such as memory cards), and media with built-in ROMs (such as ROM cartridges).

[0174] For a detailed description of this embodiment, reference can be made to the corresponding descriptions in the foregoing embodiments, and details will not be repeated here.

[0175] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-mentioned specific details are only for illustrative and easy-to-understand purposes, rather than limitations, and the above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0176] In this disclosure, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended terms, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the term "and / or", and can be used interchangeably with it, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0177] In addition, as used herein, the "or" used in the listing of items starting with "at least one" indicates a disjunctive listing, so that for example, the listing of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the term "exemplary" does not mean that the examples described are preferred or better than other examples.

[0178] It should also be noted that in the systems and methods of this disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of this disclosure.

[0179] Various changes, substitutions, and alterations to the technologies described herein can be made without departing from the teachings defined by the appended claims. In addition, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and acts described above. Current or later-developed processes, machines, manufactures, compositions of events, means, methods, or acts that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or acts within their scope.

[0180] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

[0181] The foregoing description has been presented for purposes of illustration and description. In addition, this description is not intended to limit embodiments of the present disclosure to the form disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some of their variations, modifications, alterations, additions, and subcombinations.

Claims

1. An automatic control method for unmanned vehicle formation based on large language models, characterized in that, Including: Analyze the obtained task instructions to obtain task semantics, where the task semantics include the number of unmanned vehicles, task objectives, and font styles; Extract the task formation in the task objectives, and determine the task level according to the task formation; Based on the task level, determine the target external tool; Obtain the corresponding control strategy based on the target external tool, where the control strategy includes unmanned vehicle formation position information; Automatically control the unmanned vehicle formation according to the control strategy; When the task formation belongs to the first formation library, the task level is the first level; the first formation library includes letters and Chinese characters without curves and without intersection collision points; When the task formation belongs to the second formation library, the task level is the second level; the second formation library includes letters and Chinese characters with curves and / or with intersection collision points; The obtaining the corresponding control strategy based on the target external tool includes: When the task level is the first level, the target external tool is the first type of external tool, and the corresponding control strategy includes: arranging all unmanned vehicles according to a preset inter-vehicle distance and task formation; When the task level is the second level, the target external tool is the second type of external tool, and obtain the control strategy according to the second type of external tool and a preset strategy.

2. The automatic control method for unmanned vehicle formation based on large language model according to claim 1, wherein, The obtaining the control strategy according to the second type of external tool and a preset strategy includes: Obtain the text information of the task formation and its font style, where the text information includes text content, text width, and text height; Based on the text information, obtain a target text image; Convert the target text image into an array, and perform binarization processing on the array to generate a binary image; Extract an initial skeleton image from the binary image, perform line width processing on the initial skeleton image to obtain a target skeleton image, where the line widths in the target skeleton image are all one pixel wide; Analyze the target skeleton image based on a preset analysis strategy to obtain a control strategy.

3. The automatic control method for unmanned vehicle formation based on large language model according to claim 2, characterized in that, The analyzing the target skeleton image based on a preset analysis strategy to obtain a control strategy includes: Analyze the target skeleton image to obtain all points with pixel values less than a preset threshold and their corresponding coordinate information; Based on the coordinate information, determine uniform coordinate point information consistent with the number of unmanned vehicles.

4. The automatic control method for unmanned vehicle formation based on large language model according to claim 3, wherein The based on the coordinate information, determining uniform coordinate point information consistent with the number of unmanned vehicles includes: A100, record all points with pixel values less than a preset threshold and their corresponding coordinate information in a first point set; A200, randomly select a point from the first point set as a target point, and move the target point to a second point set; A300, determine whether the number of points in the second point set is consistent with the number of unmanned vehicles. If not, calculate the distances from all points in the first point set to all points in the second point set, obtain the point in the first point set with the minimum distance to the second point set, and record it as a target point pair; Record all the obtained target point pairs as a minimum distance point set; For A400, select the point with the largest distance from the set of minimum distance points and denote it as the target point, and move the target point to the second point set; Return to execute A300. When the number of points in the second point set is the same as the number of unmanned vehicles, use all the point information in the second point set as the uniform coordinate point information.

5. The method for automatically controlling the formation of driverless vehicles based on a large language model according to claim 4, wherein, It further includes: Based on the preset inter-vehicle distance and the uniform coordinate point information, obtain the expected formation information; If the expected formation information conflicts with the actual arrangement, dynamically adjust the preset inter-vehicle distance and the uniform coordinate point information.

6. An automatic control system for unmanned vehicle formations based on large language models, characterized in that, It includes: An interaction module, including system-level prompts, constraint-level prompts, and user-level prompts. The system-level prompts are used to set roles and goals, the constraint-level prompts are used to ensure the normalization and accuracy of the output, and the user-level prompts are used for human-computer interaction and customized control; A tool-enhanced large language model, communicatively connected to the interaction module, and used to output a control strategy for automatically controlling the formation of unmanned vehicles according to the task instructions received by the interaction module and the method for automatically controlling the formation of unmanned vehicles based on the large language model according to any one of claims 1-5; An unmanned vehicle control module, communicatively connected to the tool-enhanced large language model, and used to automatically dispatch unmanned vehicles for formation according to the control strategy.

7. The automatic control system for unmanned vehicle formation based on large language model according to claim 6, wherein, It further includes a verification module, which is used to verify the control strategy. After the verification passes, the tool-enhanced large language model will send the control strategy to the unmanned vehicle control module.

8. The automatic control system for unmanned vehicle formation based on large language model according to claim 7, characterized in that, If the control strategy fails to pass the verification, the verification module sends an information indicating that the verification fails to the tool-enhanced large language model, and the tool-enhanced large language model re-obtains the control strategy.

Images