Multi-agent arrangement control method and system and vehicle-mounted equipment

Abstract

The invention provides a multi-agent arrangement control method and system and vehicle-mounted equipment, and the method comprises the steps: responding to an arrangement control instruction, obtaining an initial position of an agent and an environment image of a position where the agent is located, and enabling the arrangement control instruction to have a target arrangement position of the agent; identifying an environment image by using a semantic segmentation model to obtain a drivable area, and planning a driving path in combination with the initial position and the target arrangement position; controlling the intelligent body to run to the target arrangement position according to the driving path; and if a pose adjustment instruction is received, the target agent is controlled to adjust the pose, so that the agents are arranged in a preset arrangement array, and the pose adjustment instruction is generated according to the difference between the current arrangement array and the preset arrangement array. According to the method, multi-agent arrangement is dynamically optimized by utilizing environment perception and path planning capabilities of the agents and combining formation monitoring of a global view angle, so that global coordination during multi-agent formation arrangement is ensured, and the accuracy and efficiency of multi-agent arrangement are improved.

Description

Technical Field

[0001] The present application relates to the technical field of multi-agent coordinated control, and particularly to a multi-agent arrangement control method, system and vehicle-mounted device. Background Art

[0002] With the continuous development of artificial intelligence, multi-agent collaborative operations are widely used in scenarios such as industrial production, disaster search and rescue, park patrol, and performance display. In multi-agent collaborative operations, multi-agent arrangement control is one of the key technologies. Multi-agent arrangement is achieved by the coordinated cooperation between agents to accurately locate their respective positions for formation arrangement. In multi-agent arrangement, it is necessary to maintain movement in the desired formation and reasonably and efficiently avoid obstacles. Therefore, how to achieve efficient and accurate multi-agent arrangement control is the current key technical requirement.

[0003] In related technologies, multi-agent arrangement control mainly relies on key technologies such as environmental perception, trajectory planning, and cooperative control. Among them, environmental perception is usually achieved by identifying obstacle areas and drivable areas based on semantic segmentation technology, trajectory planning is usually achieved based on global path planning technology, and cooperative control usually includes centralized and distributed control. However, there are still significant deficiencies in the efficiency, accuracy, etc. of multi-agent collaborative control, especially showing great limitations in complex arrangement tasks in large-scale scenarios, and it is difficult to meet the actual needs of complex scenarios and large-scale agents, resulting in unsatisfactory effects in terms of accuracy and efficiency of multi-agent arrangement. Summary of the Invention

[0004] In view of the above shortcomings, the present application discloses a multi-agent arrangement control method, system and vehicle-mounted device for solving the technical problem of poor accuracy and efficiency of multi-agent arrangement control.

[0005] In a first aspect, the present application provides a multi-agent arrangement control method, the method comprising: in response to a received arrangement control instruction, acquiring an initial position of an agent and an environmental image of the position where the agent is located, the arrangement control instruction carrying a target arrangement position of the agent; using a semantic segmentation model to identify the environmental image to obtain a drivable area, and planning a driving path according to the drivable area, the initial position and the target arrangement position; controlling the agent to drive to the target arrangement position according to the driving path; if a pose adjustment instruction is received, controlling the target agent to adjust its pose according to the pose adjustment instruction so that each agent is arranged in a preset arrangement formation, the pose adjustment instruction being generated according to the difference between the current arrangement formation and the preset arrangement formation.

[0006] In an embodiment of the present application, the architecture of the semantic segmentation model includes an image input layer, a data encoder, a data decoder, and an image output layer; the data encoder sequentially includes an atrous convolution array, a feature stacking layer, a multi-branch feature extraction structure, and a first convolutional layer; the data decoder sequentially includes a second convolutional layer, a first upsampling layer, a feature concatenation layer, a third convolutional layer, and a second upsampling layer.

[0007] In an embodiment of the present application, the atrous convolution array includes a depthwise separable convolutional layer, a pyramid atrous rate array containing the depthwise separable convolutional layer at a preset multiple interval, and a global pooling layer; the multi-branch feature extraction structure includes an extended convolutional layer, a feature extraction parallel layer, a transposed convolutional layer, a channel attention mechanism layer, and a pointwise convolutional layer. The feature extraction parallel layer includes multiple feature extraction channels, and the feature extraction channels include convolutional units and batch normalization units.

[0008] In an embodiment of the present application, using the semantic segmentation model to identify the environmental image to obtain a drivable area includes: extracting multi-scale features of the environmental image through the atrous convolution array, stacking the multi-scale features through the feature stacking layer to obtain a multi-channel feature map, extracting multi-branch features of the multi-channel feature map through the multi-branch feature extraction structure, and convolving the multi-branch features through the first convolutional layer to obtain a fused feature map; convolving the low-level features in the multi-scale features through the second convolutional layer to obtain a low-level feature map, upsampling the fused feature map through the first upsampling layer to obtain an upsampled feature map, concatenating the low-level feature map and the upsampled feature map through the concatenation layer to obtain a concatenated feature map, and sequentially convolving and upsampling the concatenated feature map through the third convolutional layer and the second upsampling layer to obtain a segmentation image, where the position information of the drivable area is marked in the segmentation image; converting the position information of the drivable area from the pixel coordinate system to the world coordinate system to obtain the drivable area.

[0009] In an embodiment of the present application, the method for planning the driving path includes: planning an initial path according to the drivable area, the initial position, and the target arrangement position; if a dynamic obstacle is detected in the driving environment, then tracking the position, movement speed, and movement direction of the obstacle, and predicting the movement trajectory of the dynamic obstacle; if there is an intersection point between the movement trajectory and the initial path, then calculating the first time for the agent to reach the intersection point and calculating the second time for the dynamic obstacle to reach the intersection point; if the first time is equal to the second time, then determining the intersection point as a collision risk area and locally adjusting the initial path at the collision risk area to obtain the driving path.

[0010] In an embodiment of the present application, controlling the intelligent agent to travel to the target arrangement position according to the travel path includes: obtaining the intelligent agent information and the path information of the travel path, where the intelligent agent information includes the current position, travel direction, travel speed, and body length of the intelligent agent, and the path information includes the current path curvature; determining a preview distance according to the travel speed and the current path curvature, and searching for a preview target point on the travel path in front of the intelligent agent that is at a distance of the preview distance from the current position; calculating the deviation angle between the travel direction and the direction of the preview target point, and calculating the target steering angle of the intelligent agent according to the deviation angle, the body length, and the preview distance, and determining the target speed according to the lateral deviation between the intelligent agent and the preview target point; controlling the intelligent agent to travel to the preview target point according to the target steering angle and the target speed, and continuing to determine the next preview target point, the next target steering angle, and the next target speed to control the intelligent agent to travel until the intelligent agent reaches the target arrangement position.

[0011] In an embodiment of the present application, the generation method of the arrangement control instruction includes: dividing the arrangement area into multiple sub-areas according to the preset arrangement formation, each sub-area corresponding to an arrangement sub-task, and the arrangement sub-task includes the number of intelligent agents arranged in the corresponding sub-area and multiple arrangement positions; allocating the target arrangement position for the intelligent agent from the multiple arrangement positions, and allocating a unique code for the intelligent agent, where the unique code has a one-to-one correspondence with the target arrangement position; generating the arrangement control instruction according to the target arrangement position and the unique code.

[0012] In an embodiment of the present application, the generation method of the pose adjustment instruction includes: if there is a difference between the current arrangement formation and the preset arrangement formation, determining the intelligent agent with the difference as the target intelligent agent; detecting the current pose of the target intelligent agent, and determining the adjustment strategy of the target intelligent agent according to the preset arrangement formation and the current pose; generating the pose adjustment instruction of the target intelligent agent according to the adjustment strategy and the unique code corresponding to the target intelligent agent.

[0013] In a second aspect, the present application provides a multi-agent arrangement control system, the system comprising: an acquisition module, configured to obtain an initial position of an agent and an environmental image of the position where the agent is located in response to a received arrangement control instruction, wherein the target arrangement position of the agent is carried in the arrangement control instruction; a planning module, configured to identify the environmental image by using a semantic segmentation model to obtain a drivable area, and plan a driving path according to the drivable area, the initial position and the target arrangement position; a control module, configured to control the agent to drive to the target arrangement position according to the driving path; and an adjustment module, configured to, if a pose adjustment instruction is received, control a target agent to adjust its pose according to the pose adjustment instruction, so that the agents are arranged in a preset arrangement formation, and the pose adjustment instruction is generated according to the difference between the current arrangement formation and the preset arrangement formation.

[0014] In a third aspect, the present application provides a vehicle-mounted device, comprising: one or more processors; a storage device, configured to store one or more programs, which, when executed by the one or more processors, cause the vehicle-mounted device to implement the multi-agent arrangement control method described in the first aspect.

[0015] As described above, a multi-agent arrangement control method, system and vehicle-mounted device provided by the embodiments of the present application have the following beneficial effects:

[0016] First, in response to a received arrangement control instruction, obtain the initial position of the agent and the environmental image of the position where the agent is located, then identify the environmental image by using a semantic segmentation model to obtain a drivable area, and plan a driving path according to the drivable area, the initial position and the target arrangement position, wherein the target arrangement position of the agent is carried in the arrangement control instruction, then control the agent to drive to the target arrangement position according to the driving path, and in the case of receiving a pose adjustment instruction, control the target agent to adjust its pose, so that the agents are arranged in a preset arrangement formation, and the pose adjustment instruction is generated according to the difference between the current arrangement formation and the preset arrangement formation. Based on global arrangement planning, according to the target arrangement position corresponding to the agent itself, use environmental perception and path planning technologies to plan an optimal path, and combine with formation monitoring from a global perspective to adjust the agents that need fine-tuning, so as to realize the dynamic optimization of multi-agent arrangement, ensure the global coordination during multi-agent formation arrangement, and improve the accuracy and efficiency of multi-agent arrangement.

[0017] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0019] Figure 1 is a schematic diagram of the implementation environment of a multi-agent arrangement control system shown in an exemplary embodiment of the present application;

[0020] Figure 2 is a flowchart of a multi-agent arrangement control method shown in an exemplary embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a preset arrangement formation shown in an exemplary embodiment of the present application;

[0022] Figure 4 is a structural diagram of a semantic segmentation model shown in an exemplary embodiment of the present application;

[0023] Figure 5 is a structural diagram of a multi-branch feature extraction structure shown in an exemplary embodiment of the present application;

[0024] Figure 6 is a schematic diagram of a local arrangement area shown in an exemplary embodiment of the present application;

[0025] Figure 7 is a flowchart of intelligent body pose adjustment shown in an exemplary embodiment of the present application;

[0026] Figure 8 is a flowchart of a specific multi-agent arrangement control method shown in an exemplary embodiment of the present application;

[0027] Figure 9 is a block diagram of a multi-agent arrangement control system shown in an exemplary embodiment of the present application;

[0028] Figure 10 is a schematic structural diagram of a vehicle-mounted device provided in an embodiment of the present application. Detailed implementation manners

[0029] The embodiments of the present application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, rather than for limiting the protection scope of the present application.

[0030] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner. Therefore, only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The form, quantity, and proportion of each component in actual implementation can be arbitrarily changed, and the layout form of its components may also be more complex.

[0031] In the following description, a large number of details are explored to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0032] In multi-agent arrangement control, it mainly relies on key technologies such as environmental perception, trajectory planning, and cooperative control. Among them, environmental perception is usually achieved by identifying obstacle areas and drivable areas based on semantic segmentation technology, trajectory planning is usually achieved based on global path planning technology, and cooperative control usually includes centralized and distributed control. However, through the research of the inventors of the present application, it is found that there are still significant deficiencies in aspects such as the efficiency and accuracy of multi-agent cooperative control, especially showing great limitations in complex arrangement tasks in large-scale scenarios, and it is difficult to meet the actual needs of complex scenarios and large-scale agents, resulting in unsatisfactory results in terms of precision and efficiency of multi-agent arrangement. Moreover, there are also deficiencies in the applicability of semantic segmentation technology and the dynamic adaptability of trajectory planning, further affecting the accuracy and efficiency of multi-agent arrangement.

[0033] For example, in terms of environmental perception, current semantic segmentation algorithms identify drivable areas, but their recognition ability for complex terrains is limited. The accuracy significantly decreases in irregular terrains (such as slopes, sandy lands, and occluded areas), making it difficult to provide stable recognition results for drivable areas. Moreover, the extraction of multi-scale features is insufficient, resulting in the loss or misjudgment of perception information. In terms of trajectory planning, the path planning algorithms adopted have insufficient real-time obstacle avoidance ability and path optimization effect under the interference of dynamic environments, making it difficult to dynamically adapt to complex scenarios and leading to a reduction in arrangement efficiency. In terms of global optimization and multi-agent collaboration, they are mainly divided into two categories: centralized and distributed. Centralized methods are limited by communication bandwidth and latency issues and are difficult to coordinate a large number of agents in real time. Distributed methods lack a global perspective and are difficult to achieve the overall arrangement goal, affecting the arrangement accuracy and efficiency.

[0034] Therefore, please refer to Figure 1 , Figure 1 which is a schematic diagram of the implementation environment of a multi-agent arrangement control system shown in an exemplary embodiment of the present application. As Figure 1 shown, this implementation environment includes a cloud server 110, a global monitoring device 120, and agents 130. In multi-agent arrangement, there are multiple agents 130. Among them, the cloud server 110 is respectively communicatively connected to the global monitoring device 120 and the agents 130, and uses 5G (Fifth Generation Mobile Communication Technology) to achieve high-speed and low-latency data transmission, status synchronization, and task instruction issuance, etc. The global monitoring device 120 is used to achieve global monitoring of multi-agent arrangement through aerial work. The cloud server 110 is used to issue arrangement control instructions and pose adjustment instructions. In addition, the agents 130 can be vehicles (including driverless vehicles and manned vehicles), robots, drones, and transportation equipment with driving ability, environmental perception ability, path planning ability, and pose adjustment ability, etc. The global monitoring device 120 can be an aerial work drone and a rotating camera, etc.

[0035] The multi-agent arrangement control system can be integrated into the agent 130. Based on the global arrangement plan, according to the target arrangement position corresponding to the agent itself, it uses environmental perception and path planning technologies to plan an optimal path, and combines the formation monitoring from a global perspective to adjust the agents that need fine-tuning, realizing the dynamic optimization of multi-agent arrangement, ensuring the global coordination during multi-agent formation arrangement, and improving the accuracy and efficiency of multi-agent arrangement.

[0036] Please refer to Figure 2 , Figure 2 which is a flowchart of a multi-agent arrangement control method shown in an exemplary embodiment of the present application. This method can be applied to Figure 1For the described implementation environment, it should be understood that this method can also be applied to other exemplary implementation environments, and this embodiment does not limit the implementation environment applicable to this method.

[0037] As Figure 2 shown, in an exemplary embodiment, the multi-agent arrangement control method at least includes steps S210 to S240, which are introduced in detail as follows:

[0038] Step S210, in response to the received arrangement control instruction, obtain the initial position of the agent and the environmental image of the position where the agent is located. The arrangement control instruction carries the target arrangement position of the agent.

[0039] Step S220, use a semantic segmentation model to identify the environmental image to obtain the drivable area, and plan a driving path according to the drivable area, the initial position, and the target arrangement position.

[0040] Step S230, control the agent to drive to the target arrangement position according to the driving path.

[0041] Step S240, if a pose adjustment instruction is received, then control the target agent to adjust its pose according to the pose adjustment instruction so that the agents are arranged in a preset arrangement formation. The pose adjustment instruction is generated according to the difference between the current arrangement formation and the preset arrangement formation.

[0042] Among them, the arrangement control instruction and the pose adjustment instruction are generated and issued by the cloud server. The cloud server will issue an arrangement control instruction for each agent and issue a pose adjustment instruction for the target agent that needs to adjust its pose. In addition, the preset arrangement formation can be various arrangement formations in various scenarios. The scenarios include but are not limited to industrial production, disaster search and rescue, park patrol, and performance display, etc. The arrangement formations include but are not limited to linear arrangement formations, matrix arrangement formations, honeycomb arrangement formations, and circular arrangement formations, etc.

[0043] In step S210, after receiving the arrangement control instruction, obtain the initial position and the environmental image of the current location of the agent. Among them, the initial position can be obtained based on the lidar installed on the agent, and the environmental image can be obtained based on the camera installed on the agent. In addition, the arrangement control instruction will also be parsed to obtain the target arrangement position of the agent.

[0044] In step S220, after identifying obstacles and the drivable range in the environmental image through the deployed semantic segmentation model to obtain the drivable area, the driving path of the agent can be planned according to the drivable area, the initial position, and the target arrangement position. This path is a reliable path to avoid obstacles.

[0045] In step S230, the intelligent agent is controlled to drive according to the driving path until the intelligent agent reaches its corresponding target arrangement position.

[0046] In step S240, if a pose adjustment instruction is received, indicating that the corresponding target intelligent agent needs to adjust its pose, that is, there is a difference between the pose of the target intelligent agent in the current arrangement formation and the pose of the target intelligent agent in the preset arrangement formation, then in response to the pose adjustment instruction, the target intelligent agent is controlled to adjust its pose so that finally all intelligent agents are arranged in the preset arrangement formation.

[0047] In this embodiment, based on the global arrangement planning, according to the target arrangement position corresponding to the intelligent agent itself, using the environmental perception and path planning technologies, an optimal path is planned, and combined with the formation monitoring from the global perspective, the intelligent agents that need fine-tuning are adjusted, realizing the dynamic optimization of the multi-intelligent agent arrangement, ensuring the global coordination when the multi-intelligent agent formation is arranged, and improving the accuracy and efficiency of the multi-intelligent agent arrangement.

[0048] Please refer to Figure 3 , Figure 3 which is a schematic diagram of a preset arrangement formation shown in an exemplary embodiment of the present application. As Figure 3 shown, exemplarily, it is a preset arrangement formation for the vehicle group arrangement in a performance display scenario. This arrangement formation is a circular arrangement formation, and the vehicle group arranges around the stage, with a fixed number of circles and the total number of vehicles. For example, a total of 11 circles are arranged, and the number of vehicles in each circle is a fixed quantity, and each circle expands by 5.5 meters starting from the first circle. After the cloud server sends the arrangement control instruction to each vehicle, the vehicle goes to the corresponding target arrangement position. At this time, a global monitoring device such as a drone flies to a preset height (such as 200 meters above the center of the stage), starts the global perception module, conducts target detection, captures the overall distribution state of the vehicle group in real time, and synchronizes it to the cloud server through the 5G network to adjust the pose of the vehicle and complete the vehicle group arrangement.

[0049] In a possible embodiment, in the preset arrangement formation, the positions and postures of each intelligent agent, the relative positions between intelligent agents, and the relative positions between each intelligent agent and other objects in the arrangement scenario are shown.

[0050] In this way, based on the relative positions between intelligent agents and the relative positions between each intelligent agent and other objects in the arrangement scenario, an adjustment strategy can be more precisely formulated for the target intelligent agent that needs pose adjustment.

[0051] In a possible embodiment, the formation information of the preset arrangement formation includes the sorting method of intelligent agents.

[0052] For example, taking the Figure 3 preset arrangement formation as an example, the vehicles are arranged starting from the innermost circle at the three o'clock direction of the stage center and arranged in sequence from the inner circle to the outer circle.

[0053] In one embodiment, the architecture of the semantic segmentation model includes an image input layer, a data encoder, a data decoder, and an image output layer; the data encoder sequentially includes an atrous convolution array, a feature stacking layer, a multi-branch feature extraction structure, and a first convolution layer; the data decoder sequentially includes a second convolution layer, a first upsampling layer, a feature concatenation layer, a third convolution layer, and a second upsampling layer.

[0054] Exemplarily, the semantic segmentation model is a model based on CSNet (Compact Semantic Segmentation Network).

[0055] In this embodiment, considering that image segmentation incurs a large amount of computational cost and parameters when expanding the receptive field or using self-attention, and ignores the crucial multi-scale information in dense regions, therefore, considering the model effect and processing efficiency, a multi-branch feature extraction structure is introduced into the semantic segmentation model. This multi-branch feature extraction structure is a lightweight network structure. In this way, it is possible to effectively extract multi-scale features in the picture while reducing the consumption of computing resources and time, improving the accuracy of the model, and achieving accurate recognition of the drivable area.

[0056] In one embodiment, the atrous convolution array includes a depthwise separable convolution layer, a pyramid atrous rate array with depthwise separable convolution layers at preset multiples as intervals, and a global pooling layer; the multi-branch feature extraction structure includes an extended convolution layer, a feature extraction parallel layer, a transposed convolution layer, a channel attention mechanism layer, and a pointwise convolution layer. The feature extraction parallel layer includes multiple feature extraction channels, and the feature extraction channels include convolution units and batch normalization units.

[0057] In this embodiment, considering that there are still two problems in the atrous spatial pyramid pooling module of the conventional atrous convolution algorithm: one is that the original atrous rate interval of the atrous convolution is relatively large, which is not conducive to extracting low-resolution feature maps containing high-level semantic information, and the other is that the number of parameters of the standard convolution is large, reducing the training efficiency of the network model. Therefore, an atrous convolution array is proposed. The atrous convolution array includes a depthwise separable convolution layer, a pyramid atrous rate array with depthwise separable convolution layers at preset multiples as intervals, and a global pooling layer. Among them, the depthwise separable convolution layer decomposes the standard convolution into two steps: depth convolution and point convolution, significantly reducing the number of model parameters and computational complexity, while maintaining a high feature extraction ability. And a small value is set for the preset multiple. Using a small preset multiple as the interval to set the atrous rate can effectively capture information of different scales of low-resolution feature maps containing high-level semantic information.

[0058] In addition, in this embodiment, considering that the segmentation of drivable areas usually requires the use of multi-scale features to process targets of different sizes, but conventional segmentation models cannot effectively utilize these features, and thus cannot accurately segment targets of different scales. Therefore, a multi-branch feature extraction structure is introduced. Among them, the transposed convolution layer serves as a shortcut connection to ensure the consistency of feature channels and promote the optimization of gradient backpropagation, thereby simplifying network training. The parallel feature extraction layer is composed of parallel convolutions with different kernel sizes, that is, it contains multiple feature extraction channels, each channel consists of a convolutional unit and a batch normalization unit. Different channels can use convolutional kernels of different sizes or types to capture features of various scales and patterns in the input data, ensuring the comprehensiveness and accuracy of feature extraction.

[0059] In this way, through the improved dilated convolution array and multi-branch feature extraction structure, the model can accurately identify the drivable areas in complex scenes, improving the accuracy of the semantic segmentation model.

[0060] In one embodiment, the training process of the semantic segmentation model includes but is not limited to data preprocessing, model construction, model training, and model evaluation. Among them, data preprocessing is to perform quality enhancement and data augmentation on the pictures in the sample image set; model construction is to construct an initial model and initialize the model weights; model training includes, in sequence, defining the loss function of semantic segmentation, setting training parameters, forward propagation, calculating the loss, backpropagation, iterative training, and model verification; model evaluation is to use test pictures to analyze model problems and improve and optimize the problems. In this way, the high accuracy and robustness of the semantic segmentation model are ensured.

[0061] In a possible embodiment, different arrangement scenarios are used to train different semantic segmentation models and deploy them. The images in the sample image set used for training are the sample images under the corresponding arrangement scenarios, so as to efficiently and accurately implement semantic segmentation tasks in different complex environments, provide high-quality environmental perception data, and provide accurate scene information support for multi-agent arrangement.

[0062] In one embodiment, a semantic segmentation model is used to identify an environmental image to obtain a drivable area, including: extracting multi-scale features of the environmental image through an atrous convolution array, stacking the multi-scale features through a feature stacking layer to obtain a multi-channel feature map, extracting multi-branch features of the multi-channel feature map through a multi-branch feature extraction structure, and convolving the multi-branch features through a first convolutional layer to obtain a fused feature map; convolving low-level features in the multi-scale features through a second convolutional layer to obtain a low-level feature map, upsampling the fused feature map through a first upsampling layer to obtain an upsampled feature map, splicing the low-level feature map and the upsampled feature map through a splicing layer to obtain a spliced feature map, and sequentially convolving and upsampling the spliced feature map through a third convolutional layer and a second upsampling layer to obtain a segmentation image, where the position information of the drivable area is marked in the segmentation image; converting the position information of the drivable area from the pixel coordinate system to the world coordinate system to obtain the drivable area.

[0063] In this embodiment, based on the improved atrous convolution array and multi-branch feature extraction structure in the semantic segmentation model, the drivable area is identified, which ensures the accuracy of the obtained drivable area and provides high-quality input for the subsequent path planning of the intelligent agent.

[0064] Please refer to Figure 4 , Figure 4 which is a structural diagram of a semantic segmentation model shown in an exemplary embodiment of the present application. As Figure 4 shown, exemplarily, the atrous convolution array Atrous Conv uses a DCNN (Deep Convolutional Neural Network), mainly including a 1×1 depthwise separable convolutional layer, a pyramid atrous rate array with a depthwise separable convolutional layer at intervals of multiples of 2 (such as atrous convolutions with atrous rates of 2, 4, 8, 12, 16), and a global pooling layer.

[0065] Please refer to Figure 5 , Figure 5 which is a structural diagram of a multi-branch feature extraction structure shown in an exemplary embodiment of the present application. As Figure 5As shown, exemplarily, the expansion conv layer is a structure of 1x1 convolution + ReLu (Rectified Linear Unit) activation function. In the four parallel feature extraction layers, each channel has a convolution unit and a batch normalization unit, namely BN (Batch Normalization). The kernel sizes of these convolution units are 3×1, 1×3, 3×3, and 1×1 respectively. The channel attention mechanism layer includes Add (Attention-based deep model) and SE, that is, SENet (Squeeze-and-Excitation Networks). The kernel size of the transposed convolution layer is 1×1.

[0066] Exemplarily, please continue to refer to Figure 4 , use the semantic segmentation model to identify the environmental image to obtain the drivable area. The detailed steps are as follows: Input the environmental image with a size of 640x640 through the image input layer. Extract the multi-scale features feature1.1 to feature1.7 of the environmental image through the Atrous Conv array. Stack the multi-scale features feature1.1 to feature1.7 through the Stacked layer to obtain the multi-channel feature feature2 and form a multi-channel feature map. Extract the multi-branch features feature3 of the multi-channel feature map through the Multi-Branch Module structure, and perform convolution on the multi-branch feature feature3 through the first 1×1DConv layer to obtain the fused feature feature4 and form a fused feature map; Perform convolution on the low-level feature feature1.1 in the multi-scale features through the second 1×1DConv layer to obtain the feature feature5 and form a low-level feature map. Upsample the fused feature map through the first Upsample by 4 layer to obtain the upsampled feature feature6 and form an upsampled feature map. Concatenate feature5 and feature6 through the Concat layer to obtain the concatenated feature feature7 and form a concatenated feature map, and perform convolution and upsampling on the concatenated feature map in sequence through the third 3×3DConv layer and the second Upsample by 4 layer, and finally output a single-channel segmentation image of 640x640x1 from the image output layer.

[0067] In one embodiment, the method for planning a driving path includes: planning an initial path according to the drivable area, the initial position, and the target arrangement position; if a dynamic obstacle is detected in the driving environment, tracking the position, movement speed, and movement direction of the obstacle, and predicting the movement trajectory of the dynamic obstacle; if there is an intersection point between the movement trajectory and the initial path, calculating the first time for the agent to reach the intersection point and the second time for the dynamic obstacle to reach the intersection point; if the first time is equal to the second time, determining the intersection point as a collision risk area, and locally adjusting the initial path at the collision risk area to obtain the driving path.

[0068] In this embodiment, after obtaining the initial path based on global path planning, during the driving process of the agent along the initial path, environmental data is collected in real time, dynamic obstacle detection and prediction of the movement trajectory of the dynamic obstacle are performed, and when there is an intersection point between the movement trajectory and the initial path and the agent and the dynamic obstacle reach the intersection point simultaneously, it is determined that there is a safety hazard in the driving path, so a safe path is re-searched at the intersection point and the initial path is locally adjusted. In this way, by combining the path planning algorithm and the real-time obstacle avoidance mechanism, it is ensured that the path is always in a safe state, significantly improving the path planning efficiency and safety of the agent in a dynamic environment, and thus ensuring the efficiency of multi-agent arrangement.

[0069] Exemplarily, to calculate the first time for the agent to reach the intersection point, that is, obtaining the first distance from the agent's position to the intersection point and the agent's speed, and calculating the first distance and the agent's speed to obtain the first time; to calculate the second time for the dynamic obstacle to reach the intersection point, that is, obtaining the second distance from the obstacle position of the dynamic obstacle to the intersection point, and calculating the second distance and the movement speed of the dynamic obstacle to obtain the second time.

[0070] Exemplarily, if a static obstacle is detected on the initial path, the position of the static obstacle is used as the collision risk area, and the initial path is locally adjusted at the collision risk area to obtain the driving path.

[0071] Exemplarily, the monitoring of dynamic obstacles and static obstacles is realized based on sensors (such as lidar, camera, or millimeter-wave radar, etc.) loaded on the agent.

[0072] In a possible embodiment, local adjustment is performed on the initial path at the collision risk area to obtain a driving path, including: constructing a local map at the collision risk area, where the local map includes environmental information in the collision risk area; searching and generating multiple candidate paths that bypass dynamic or static obstacles according to the local map; calculating the comprehensive cost of each candidate path, and determining the target candidate path according to the comprehensive cost. The comprehensive cost includes path length cost, path smoothness cost, and obstacle collision cost, and the obstacle collision cost includes obstacle existence cost and obstacle distance cost; performing smoothing processing on the target candidate path and connecting it with the initial path to obtain a driving path.

[0073] Among them, performing smoothing processing on the target candidate path can avoid sharp turns or discontinuities in the path and improve the maneuverability of the agent.

[0074] Exemplarily, the global path search method used for global path planning can be the hybrid Astar search algorithm, and the local planning algorithm used for local path planning can be the dynamic A star search algorithm, RRT (Rapidly-exploring Random Tree), or the artificial potential field-based method, etc. Of course, the embodiments of the present application do not limit the specific global path search method and local planning algorithm.

[0075] In an embodiment, controlling the agent to drive to the target arrangement position according to the driving path includes: obtaining the agent information and the path information of the driving path. The agent information includes the current position, driving direction, driving speed, and body length of the agent, and the path information includes the current path curvature; determining the preview distance according to the driving speed and the current path curvature, and searching for a preview target point on the driving path in front of the agent that is at a distance of the preview distance from the current position; calculating the deviation angle between the driving direction and the direction of the preview target point, and calculating the target steering angle of the agent according to the deviation angle, body length, and preview distance, and determining the target speed according to the lateral deviation between the agent and the preview target point; controlling the agent to drive to the preview target point according to the target steering angle and the target speed, and continuing to determine the next preview target point, the next target steering angle, and the next target speed to control the agent to drive until the agent reaches the target arrangement position.

[0076] In this embodiment, the preview distance is determined according to the driving speed and the current path curvature. Among them, the preview distance is proportional to the driving speed and inversely proportional to the current path curvature. That is, when the driving speed is large, the agent needs more time to complete the steering adjustment, and a larger preview distance can allow the agent to perceive the change of the path in advance, so as to have enough time for smooth steering. In addition, the greater the current path curvature, the smaller the preview distance should be set to ensure that the agent can adjust the direction in time and avoid deviating from the path.

[0077] In this embodiment, the calculation formula for the target steering angle is as follows:

[0078] δ = arctan(2L·sin(α) / d)

[0079] Wherein, δ represents the target steering angle, α represents the deviation angle, L represents the body length, and d represents the preview distance.

[0080] In this embodiment, the target speed is determined according to the lateral deviation between the agent and the preview target point. When the lateral deviation is large, the target speed is small; when the lateral deviation is small, the target speed is large.

[0081] In this way, based on the pure tracking control of the agent, by continuously setting preview points on the driving path, the speed and direction of the agent are adjusted in real time to ensure that the agent can accurately drive along the predetermined path to the target arrangement position, achieving smooth and accurate path following, thereby improving the efficiency and accuracy of multi-agent arrangement.

[0082] Exemplarily, the current position of the agent can be monitored and obtained based on the sensors installed on the agent, such as GPS (Global Positioning System); the driving direction and speed can be monitored and obtained based on the sensors installed on the agent, such as IMU (Inertial Measurement Unit).

[0083] Exemplarily, if the agent is a vehicle, the value of the body length is the value of the vehicle wheelbase.

[0084] In a possible embodiment, considering that the driving path is usually represented by a series of discrete coordinate points, if none of the path points in the driving path is exactly located at the position of the preview distance, the preview target point is determined by linear interpolation; in addition, to improve the control smoothness of the agent's steering, the target steering angle is smoothed and compensated by combining feedforward control and low-pass filtering to reduce the instability or oscillation of the agent caused by the jump of the preview target point.

[0085] In a possible embodiment, the preview distance is determined according to the driving speed and the current path curvature, that is, the preview distance is determined according to a preset first mapping relationship, and the first mapping relationship is the corresponding relationship between various driving speeds, path curvatures and the corresponding preview distances; in addition, the target speed is determined according to the lateral deviation between the agent and the preview target point, that is, the target speed is determined according to a preset second mapping relationship, and the second mapping relationship is the corresponding relationship between various lateral deviations and the corresponding target speeds.

[0086] In one embodiment, the method for generating the arrangement control instruction includes: dividing the arrangement area into multiple sub-areas according to a preset arrangement formation, where each sub-area corresponds to an arrangement sub-task, and the arrangement sub-task includes the number of agents to be arranged and multiple arrangement positions in the corresponding sub-area; allocating a target arrangement position for the agent from the multiple arrangement positions, and allocating a unique code for the agent, where there is a one-to-one correspondence between the unique code and the target arrangement position; generating the arrangement control instruction according to the target arrangement position and the unique code.

[0087] Among them, for the division of the sub-areas, please refer to Figure 6 , Figure 6 which is a schematic diagram of a part of the arrangement area shown in an exemplary embodiment of the present application. As Figure 6 shown, this part can be regarded as the layout of one of the sub-areas.

[0088] In this embodiment, the cloud server uses a distributed task allocation algorithm to break down the overall arrangement task into several sub-tasks, that is, divides the arrangement area into several grid areas. The agents to be arranged in each area are handled by the corresponding sub-tasks, and each sub-task is assigned to each agent node. While allocating the target arrangement position, a unique code is also assigned to the agent. There is a one-to-one correspondence between the unique code and the target arrangement position, which can ensure that the positions and codes obtained by each agent are neither repeated nor omitted. In this way, the unique code and accurate positioning of each agent in the entire arrangement task are beneficial to tracking each agent and its corresponding arrangement task. In addition, by decomposing the overall arrangement task into multiple small tasks for parallel processing, the processing requirements for large-scale arrangements can be met, and the efficiency and accuracy of multi-agent collaborative arrangement are improved.

[0089] Exemplarily, algorithms such as load balancing, consistent hashing, or distributed consensus can be used to divide the number of agents arranged in multiple sub-areas.

[0090] Exemplarily, a unique code is assigned to the agent according to a preset rule, where the preset rule includes but is not limited to generating the unique code using a timestamp, an agent identifier, or other random factors.

[0091] In a possible embodiment, in order to prevent the situation where the target arrangement configuration corresponding to the agent in the global scope and the unique code are repeated, therefore, before generating the arrangement control instruction, a global consistency check is performed on the target arrangement position and the unique code corresponding to each agent to ensure the reliability of task allocation.

[0092] In a possible embodiment, in the global monitoring of multi-agent arrangement by the global monitoring device, each arrangement sub-task corresponds to a detection sub-task. In this way, by decomposing the overall monitoring task into multiple small tasks for parallel processing, the monitoring accuracy can be improved, thereby ensuring the efficiency and accuracy of multi-agent collaborative arrangement.

[0093] In one embodiment, the method for generating the pose adjustment instruction includes: if there is a difference between the current arrangement formation and the preset arrangement formation, determining the agents with differences as target agents; detecting the current poses of the target agents, and determining the adjustment strategies of the target agents according to the preset arrangement formation and the current poses; and generating the pose adjustment instructions for the target agents according to the adjustment strategies and the unique codes corresponding to the target agents.

[0094] Please continue to refer to Figure 3 , in the preset arrangement formation, in addition to planning the target arrangement positions of each agent, it also includes the postures of the agents, the relative positions between the agents, and the relative positions between each agent and other objects in the arrangement scene.

[0095] Therefore, in this embodiment, after the cloud server obtains the current arrangement formation (including the current poses of each agent) by using the global perspective and real-time perception ability of the global monitoring device, it can compare the current arrangement formation with the preset arrangement formation (including the preset poses of each agent), determine the adjustment strategies of the target agents, and control the target agents to make adjustments. In this way, by combining the high-altitude global perspective with the real-time adjustment of the agents, efficient and accurate arrangement of multiple agents can be achieved under multi-agent arrangement tasks of different scales.

[0096] Please refer to Figure 7 , Figure 7 is a flowchart of an intelligent agent pose adjustment shown in an exemplary embodiment of the present application. As Figure 7 shown, taking the agent as a vehicle and the global monitoring device as a drone as an example, after the vehicle group has initially completed the arrangement, the steps of intelligent agent pose adjustment at least include step S710 to step S760, which are described in detail as follows:

[0097] Step S710, the drone takes pictures of the arranged vehicle group to obtain vehicle group arrangement pictures;

[0098] Step S720, the drone uses the target detection algorithm to identify the vehicle group arrangement image, obtains the current arrangement formation of the vehicle group, and transmits it to the cloud server;

[0099] Step S730, after receiving the current arrangement formation, the cloud server determines whether there is a difference between the current arrangement formation and the preset arrangement formation. If not, it indicates that the vehicle group arrangement is completed. If there is a difference, it enters step S740;

[0100] Step S740, the cloud server determines the target vehicle to be adjusted and the adjustment strategy of the target vehicle;

[0101] Step S750, the cloud server generates pose adjustment instructions based on the adjustment strategy and sends them to the corresponding target vehicle;

[0102] In step S760, the target vehicle responds to the received pose adjustment instruction, adjusts its pose, and then returns to step S710. The drone continuously acquires and identifies the vehicle group arrangement image and transmits it to the cloud server for formation difference judgment, and controls the target vehicle to make adjustments until the vehicle group arrangement is completed.

[0103] Please refer to Figure 8 , Figure 8 which is a flowchart of a specific multi-agent arrangement control method shown in an exemplary embodiment of the present application. As Figure 8 shown, continuing to take the agent as a vehicle and the global monitoring device as a drone as an example, this specific multi-agent arrangement control method at least includes steps S810 to S890, which are described in detail as follows:

[0104] In step S810, the cloud server establishes communication connections with the drone and each vehicle respectively;

[0105] In step S820, start the vehicle group arrangement task. The drone flies above the arrangement area according to the preset height, and the cloud server issues arrangement control instructions to each vehicle;

[0106] In step S830, the vehicle terminal acquires the environmental image and the initial position, and uses a semantic segmentation model to identify the environmental image to obtain the drivable area;

[0107] In step S840, the vehicle terminal uses the hybrid A-star search algorithm to dynamically plan the driving path according to the drivable area, the initial position, and the target arrangement position in the arrangement control instruction;

[0108] In step S850, the vehicle terminal uses a pure tracking control algorithm to control the vehicle to drive to the target arrangement position according to the driving path;

[0109] In step S860, the drone takes pictures of the arranged vehicle group, obtains the vehicle group arrangement picture, and uses an object detection algorithm to identify the vehicle group arrangement image to obtain the current arrangement formation of the vehicle group, and transmits it to the cloud server;

[0110] In step S870, the cloud server determines whether there is a difference between the current arrangement formation and the preset arrangement formation. If not, it indicates that the vehicle group arrangement is completed. If there is a difference, it enters step S880;

[0111] In step S880, the cloud server issues a pose adjustment instruction to the target vehicle that needs to be adjusted;

[0112] Step S890: The target vehicle adjusts its pose in response to the received pose adjustment instruction, and then returns to Step S810. The drone continuously acquires and identifies the vehicle group arrangement image and transmits it to the cloud server for formation difference judgment, and controls the target vehicle to make adjustments until the vehicle group arrangement is completed.

[0113] In this way, the high-altitude perspective of the global monitoring device is fully utilized to assist the global perception and dynamic adjustment of multi-agent arrangement, and combined with semantic segmentation, trajectory planning, pure tracking control and target detection, the multi-intelligent collaborative arrangement task is completed, significantly improving the efficiency, robustness and accuracy of vehicle group arrangement.

[0114] The above multi-agent arrangement control method first responds to the received arrangement control instruction, acquires the initial position of the agent and the environmental image of the position where the agent is located, then uses a semantic segmentation model to identify the environmental image to obtain the drivable area, and plans the driving path according to the drivable area, the initial position and the target arrangement position. The target arrangement position of the agent is carried in the arrangement control instruction. Then, the agent is controlled to drive to the target arrangement position according to the driving path. In the case of receiving a pose adjustment instruction, the target agent is controlled to adjust its pose so that each agent is arranged in a preset arrangement formation. The pose adjustment instruction is generated according to the difference between the current arrangement formation and the preset arrangement formation. Based on the global arrangement plan, according to the target arrangement position corresponding to the agent itself, environmental perception and path planning technologies are used to plan an optimal path, and combined with the formation monitoring from a global perspective, the agents that need fine-tuning are adjusted to achieve the dynamic optimization of multi-agent arrangement, ensuring the global coordination during multi-agent formation arrangement and improving the accuracy and efficiency of multi-agent arrangement.

[0115] Please refer to Figure 9 , Figure 9 which is a block diagram of a multi-agent arrangement control system shown in an exemplary embodiment of the present application. This system can be applied to Figure 1 the implementation environment shown. It should be understood that this system can also be applicable to other exemplary implementation environments, and this embodiment does not limit the implementation environment applicable to this system.

[0116] As Figure 9 shown, in an exemplary embodiment, the multi-agent arrangement control system 900 at least includes an acquisition module 910, a planning module 920, a control module 930 and an adjustment module 940, which are introduced in detail as follows:

[0117] The acquisition module 910 is configured to acquire the initial position of the agent and the environmental image of the position where the agent is located in response to the received arrangement control instruction, and the arrangement control instruction carries the target arrangement position of the agent;

[0118] A planning module 920, configured to use a semantic segmentation model to identify an environmental image, obtain a drivable area, and plan a driving path based on the drivable area, the initial position, and the target arrangement position;

[0119] A control module 930, configured to control the agent to drive to the target arrangement position according to the driving path;

[0120] An adjustment module 940, configured to, if a pose adjustment instruction is received, control the target agent to adjust its pose according to the pose adjustment instruction, so that the agents are arranged in a preset arrangement formation, and the pose adjustment instruction is generated according to the difference between the current arrangement formation and the preset arrangement formation.

[0121] It should be noted that the in-process communication system provided in the above embodiment and the in-process communication method provided in the above embodiment belong to the same concept. The content of the operations performed by each module has been described in detail in the method embodiment, and will not be repeated here.

[0122] Please refer to Figure 10 , Figure 10 which is a schematic structural diagram of a vehicle-mounted device provided by an embodiment of the present application. Figure 10 shows a schematic structural diagram of a computer system of a vehicle-mounted device suitable for implementing the embodiments of the present application. It should be noted that Figure 10 the shown computer system 1000 of the vehicle-mounted device is only an example, and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0123] As Figure 10 shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage section 1008 into the random access memory (RAM) 1003, such as executing the method in the above embodiment. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0124] The following components are connected to the I / O interface 1005: an input part 1006 including a keyboard, a mouse, etc.; an output part 1007 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 1008 including a hard disk, etc.; and a communication part 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part 1009 performs communication processing via a network such as the Internet. The drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 1010 as needed so that a computer program read from it can be installed into the storage part 1008 as needed.

[0125] Specifically, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by a central processing unit (CPU) 1001, various functions defined in the system of the present application are executed.

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0127] The units involved in the embodiments described in this application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not, in some cases, constitute a limitation on the units themselves.

[0128] The above embodiments are only used to exemplarily illustrate the principles and effects of this application, rather than to limit this application. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed in this application should still be covered by the claims of this application.

Claims

1. A multi-agent arrangement control method, characterized in that: The method comprises: In response to the received arrangement control instruction, acquiring the initial position of the agent and the environment image where the agent is located, wherein the arrangement control instruction carries the target arrangement position of the agent; Using a semantic segmentation model to identify the environment image, obtain a drivable area, and plan a driving path according to the drivable area, the initial position, and the target arrangement position; Controlling the intelligent body to travel to the target arrangement position according to the driving path; If a posture adjustment instruction is received, the target agent is controlled to adjust its posture according to the posture adjustment instruction so that each agent is arranged into a preset formation. The posture adjustment instruction is generated according to the difference between the current formation and the preset formation.

2. The multi-agent arrangement control method according to claim 1, characterized in that: The architecture of the semantic segmentation model includes an image input layer, a data encoder, a data decoder and an image output layer; The data encoder includes a hole convolution array, a feature stacking layer, a multi-branch feature extraction structure and a first convolution layer in sequence; The data decoder sequentially comprises a second convolutional layer, a first upsampling layer, a feature concatenation layer, a third convolutional layer and a second upsampling layer.

3. The multi-agent arrangement control method according to claim 2, characterized in that: The atrous convolution array includes a depthwise separable convolution layer, a pyramid atrous rate array containing the depthwise separable convolution layer and spaced at preset multiples, and a global pooling layer; The multi-branch feature extraction structure includes an extended convolution layer, a feature extraction parallel layer, a deconvolution layer, a channel attention mechanism layer and a point-by-point convolution layer. The feature extraction parallel layer includes multiple feature extraction channels, and the feature extraction channel includes a convolution unit and a batch normalization unit.

4. The multi-agent arrangement control method according to claim 2, characterized in that: The method of using a semantic segmentation model to identify the environment image and obtain a drivable area includes: Extracting multi-scale features of the environment image through the atrous convolution array, stacking the multi-scale features through the feature stacking layer to obtain a multi-channel feature map, extracting multi-branch features of the multi-channel feature map through the multi-branch feature extraction structure, and convolving the multi-branch features through the first convolution layer to obtain a fused feature map; Convolving low-level features in the multi-scale features through the second convolution layer to obtain a low-level feature map, upsampling the fused feature map through the first upsampling layer to obtain an upsampled feature map, splicing the low-level feature map and the upsampled feature map through the splicing layer to obtain a spliced ​​feature map, and sequentially convolving and upsampling the spliced ​​feature map through the third convolution layer and the second upsampling layer to obtain a segmented image, wherein the segmented image is annotated with drivable area location information; The position information of the drivable area is converted from a pixel coordinate system to a world coordinate system to obtain the drivable area.

5. The multi-agent arrangement control method according to claim 1, characterized in that: The planning method of the driving path includes: Planning an initial path according to the drivable area, the initial position and the target arrangement position; If a dynamic obstacle is detected in the driving environment, the position, speed and direction of the obstacle are tracked to predict the movement trajectory of the dynamic obstacle; If the motion trajectory and the initial path have an intersection, then calculating a first time for the agent to reach the intersection, and calculating a second time for the dynamic obstacle to reach the intersection; If the first time is equal to the second time, the intersection is determined as a collision risk area, and the initial path is partially adjusted in the collision risk area to obtain the driving path.

6. The multi-agent arrangement control method according to claim 1, characterized in that: The step of controlling the intelligent body to travel to the target arrangement position according to the travel path includes: Acquire agent information and path information of the driving path, wherein the agent information includes the current position, driving direction, driving speed and body length of the agent, and the path information includes the curvature of the current path; Determine a preview distance according to the driving speed and the curvature of the current path, and search for a preview target point at a distance from the current position by the preview distance on the driving path in front of the intelligent body; Calculating the deviation angle between the driving direction and the preview target point direction, and calculating the target turning angle of the intelligent body according to the deviation angle, the body length and the preview distance, and determining the target speed according to the lateral deviation between the intelligent body and the preview target point; The intelligent body is controlled to travel to the preview target point according to the target steering angle and the target speed, and the next preview target point, the next target steering angle and the next target speed are continuously determined to control the intelligent body to travel until the intelligent body reaches the target arrangement position.

7. The multi-agent arrangement control method according to any one of claims 1 to 6, characterized in that: The generation method of the arrangement control instruction includes: Divide the arrangement area into a plurality of sub-areas according to the preset arrangement formation, each sub-area corresponds to an arrangement sub-task, and the arrangement sub-task includes the number of agents arranged in the corresponding sub-area and a plurality of arrangement positions; Assigning the target arrangement position to the agent from a plurality of arrangement positions, and assigning a unique code to the agent, wherein the unique code has a one-to-one correspondence with the target arrangement position; The arrangement control instruction is generated according to the target arrangement position and the unique code.

8. The multi-agent arrangement control method according to claim 7, characterized in that: The method for generating the posture adjustment instruction includes: If the current arrangement formation is different from the preset arrangement formation, the agent with the difference is determined as the target agent; Detecting the current position and posture of the target intelligent body, and determining an adjustment strategy of the target intelligent body according to the preset arrangement formation and the current position and posture; The posture adjustment instruction of the target intelligent body is generated according to the unique code corresponding to the adjustment strategy and the target intelligent body.

9. A multi-agent arrangement control system, characterized in that: The system comprises: A collection module, used for acquiring the initial position of the agent and the environment image of the agent's position in response to the received arrangement control instruction, wherein the arrangement control instruction carries the target arrangement position of the agent; A planning module, used to identify the environment image using a semantic segmentation model, obtain a drivable area, and plan a driving path according to the drivable area, the initial position and the target arrangement position; A control module, used for controlling the intelligent body to travel to the target arrangement position according to the travel path; The adjustment module is used to control the target intelligent agent to adjust its posture according to the posture adjustment instruction if it receives the posture adjustment instruction, so that each intelligent agent is arranged into a preset arrangement formation, and the posture adjustment instruction is generated according to the difference between the current arrangement formation and the preset arrangement formation.

10. A vehicle-mounted device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, enables the vehicle-mounted device to implement the multi-agent arrangement control method as described in any one of claims 1 to 6.

Images