Multi-agent perception fusion method based on machine learning

Abstract

The invention discloses a multi-agent perception fusion method based on machine learning, and the method comprises the steps: obtaining all agents in a current road region and corresponding agent information, and generating a primary and secondary agent adaptation group of each agent through a preset primary and secondary agent adaptation mechanism; based on each agent, a road image information sequence transmitted by each agent in the main and secondary agent adaptation group is obtained, a road image set is formed, the road image set comprises at least one road image information sequence, and the road image information sequence comprises a plurality of road images; and based on each intelligent agent, transmitting the collected road image set to a vehicle-mounted calculation module of the intelligent agent to detect a road target object, and outputting road perception information of the intelligent agent. Therefore, while the sensing range of the intelligent agent is expanded, the resource utilization of a built-in computing chip and a communication network is optimized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent agent control technology, and in particular to a multi-agent perception fusion method based on machine learning. Background Art

[0002] In recent years, with the gradual development and maturity of machine learning technology, deep learning-based computer vision technology has achieved improved performance in a range of tasks, such as object detection and image classification. Computer vision technology is also increasingly being applied in various industrial fields, such as pedestrian recognition, fingerprint recognition, facial recognition, and autonomous driving. A multi-agent perception fusion system is a system that acquires information about the surrounding environment of multiple agents by collecting images captured by their cameras. Existing agent perception systems generally include the following approaches, based on visual perception, radar perception, and wireless communication.

[0003] The Chinese invention patent application number 202210483252.9 discloses a multi-agent perception fusion system based on machine learning and its implementation method. The photos collected by multiple cameras installed by the multi-agent are transformed and calculated by the model to fuse the features into the same vector space, and then the fused features are predicted. Finally, the position and size information of the agents, pedestrians, traffic signs, traffic lights, etc. in the surrounding environment, as well as the drivable area, lane lines, curbs and other features are output; the communication module is used to communicate with all environmental agents, share perception information, increase the perception range of the agent, and reduce the perception blind spot. The output of the algorithm can be used as the input of the trajectory prediction and planning control module.

[0004] However, using communication modules to share perception information of all agents in the road area without differentiation may cause the computing chip in each agent to process a large amount of information that is irrelevant to the current task or redundant, resulting in a waste of resources. Currently, there is no reasonable relationship network for multi-agent sharing and integration of environmental information to adapt to the complex and changing road environment. Summary of the Invention

[0005] This application provides a multi-agent perception fusion method based on machine learning, which expands the perception range of the agent while optimizing the resource utilization of its built-in computing chip and communication network.

[0006] This application provides a multi-agent perception fusion method based on machine learning, including: S101, obtaining all agents and their corresponding agent information in the current road area, where the agent information of each agent includes location information, task information, and field of view information; S102, based on the agent information of each agent, using a pre-set primary and secondary agent adaptation mechanism, generating a primary and secondary agent adaptation group for each agent, the primary and secondary agent adaptation group including a primary agent and at least one secondary agent; S103, based on each agent, obtaining a road image information sequence transmitted by each agent in its primary and secondary agent adaptation group to form a road image set, wherein the road image set includes at least one road image information sequence, and the road image information sequence includes a plurality of road images; S104, based on each intelligent agent, the collected road image set is transmitted to its on-board computing module to detect road target objects, and the road perception information of the intelligent agent is output.

[0007] Preferably, the position information includes the position coordinates of the agent, the task information includes the task set of the agent, the field of view information includes the sum of the acquisition ranges of multiple vehicle-mounted cameras of the agent, and the field of view information is recorded as a scanning area.

[0008] Preferably, the pre-set primary and secondary agent adaptation mechanism specifically includes: S201, based on each agent, determine itself as the master agent, calculate the fitness between the master agent and each other agent, and obtain the fitness of several agent pairs; S202, based on the fitness of all agent pairs, other agents in the agent pair whose fitness is greater than the preset fitness threshold are determined as secondary agents, and a binding relationship is established between them and the corresponding main agent, so that the main agent can receive the road image information collected by all secondary agents through wireless communication.

[0009] Preferably, the degree of adaptation is calculated according to the following formula:

[0010] in, is the fitness between agent A and agent B, is the distance adaptation value between agent A and agent B, is the task adaptation value between agent A and agent B, is the field of view adaptation value of agent A and agent B, 、 and are pre-set weight factors, which are used to indicate the influence of distance, task and field of view on the fitness.

[0011] Preferably, the distance adaptation value is calculated based on the position information of the two agents, specifically: , is the Euclidean distance between two agents; The task adaptation value is calculated based on the task set of the two agents, specifically: , is the task set of agent A, which includes at least one task content. is the task set of agent B; The field of view adaptation value is calculated based on the scanning area of ​​the two agents, specifically: , is the scanning area of ​​agent A, is the scanning area of ​​agent B Zhongyu The difference between the areas, is the overlapping area of ​​the scanning areas of agent A and agent B, m and n are pre-set weight factors, which are used to indicate the influence of the distinguishing area and the overlapping area on the field of view adaptation value.

[0012] Preferably, each of the intelligent bodies is provided with an onboard computing module, which includes an image feature extraction module, a multi-feature fusion module and a multi-class target object detection module; The vehicle-mounted computing module is specifically used for: The received road image set is input into the image feature extraction module, and the pre-set feature extraction network is used to extract the features of each road image, output the road feature vector set, and transmit it to the multi-feature fusion module; The multi-feature fusion module is used to: use a pre-set spatial feature fusion network to convert the received road feature vector set into a unified three-dimensional vector space, fuse the road feature vectors from different on-board cameras and different intelligent agents in the three-dimensional vector space, output the fused feature vector, and transmit it to the multi-class target object detection module; The multi-class target object detection module is used to: use a pre-set multi-head prediction network to perform multi-class target object detection tasks on the received fusion feature vectors, and output target objects and their location information.

[0013] Preferably, the method for setting the weight factors corresponding to the difference areas and the overlap areas in the field of view adaptation value specifically includes: S301, obtaining the current position information and task content of the two agents, and obtaining the correlation coefficient between the two agents based on the position information and task content of the two agents; S302, dynamically adjust the pre-set basic weight factor based on the correlation coefficient and , and obtain the weight factors m and n corresponding to the distinguished area and the overlapping area respectively.

[0014] Preferably, the weight factors m and n corresponding to the distinguishing area and the overlapping area, respectively, are calculated according to the following formulas:

[0015]

[0016] Among them, m is the weight factor of the distinguished area, n is the weight factor of the overlapping area, is the basic weight factor for distinguishing regions, and r is the correlation coefficient.

[0017] Preferably, the S301 specifically includes: S401: Generate an agent-association feature vector based on the current position information and task content of the two agents. The agent-association feature vector includes the distance between the two agents, the deviation angle of the secondary agent's position relative to the current direction of the primary agent, and the similarity of the agents' current task content. S402: Input the agent-related feature vector into a pre-trained agent-related prediction model, and output the correlation coefficient of the two agents.

[0018] Preferably, the method further comprises: S105, periodically executing steps S101 to S104 at preset time intervals to achieve dynamic update of the primary and secondary agent adaptation groups of each agent.

[0019] One or more technical solutions provided in this application have at least the following technical effects or advantages: By establishing a primary-secondary agent adaptation mechanism, the primary agent only shares information with necessary secondary agents and receives only information from other agents that is highly relevant to its task, location, and field of view. This reduces unnecessary information transmission and processing, thereby optimizing resource utilization of computing chips and communication networks. Through multi-dimensional feature fusion, multi-view, multi-agent, and multi-location road image information is integrated to obtain a more comprehensive and accurate description of environmental information, taking into account multiple factors such as the agent's task, location, and field of view, and can adapt to complex and changing road environments. The primary and secondary agent adaptation groups are established based on the positional relationship between the agents, their field of view, and the current task requirements. This makes information sharing more targeted and effective, and the agents can more quickly obtain the most valuable environmental information for their current tasks. By reducing unnecessary information sharing, the burden on the communication network is reduced and the network transmission efficiency and stability are improved; the establishment of primary and secondary agent adaptation groups makes the multi-agent system more flexible and efficient, able to adapt to different environments and task requirements, and improves the overall performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the process of multi-agent perception fusion method based on machine learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] To facilitate understanding of the present invention, the present application will be described more comprehensively below with reference to the relevant drawings; the drawings show preferred embodiments of the present invention, but the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to enable a more thorough and comprehensive understanding of the disclosed content of the present invention.

[0022] It should be noted that the terms “vertical”, “horizontal”, “up”, “down”, “left”, “right” and similar expressions used in this document are for illustrative purposes only and do not represent the only implementation method.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains; the terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more of the associated listed items.

[0024] Example 1: Figure 1 It is a flowchart of a multi-agent perception fusion method based on machine learning in an embodiment of the present invention.

[0025] like Figure 1 As shown in FIG, a multi-agent perception fusion method based on machine learning includes the following steps: S101, obtaining all agents in the current road area and their corresponding agent information, where the agent information of each agent includes location information, task information, and field of view information.

[0026] Among them, the position information includes the position coordinates of the intelligent agent (which can be obtained from the vehicle-mounted positioning device), the task information includes the task set of the intelligent agent (the task set is pre-loaded into the intelligent agent, and the intelligent agent executes tasks in sequence based on the task set to complete the work in the road area), and the field of view information includes the sum of the acquisition ranges of the intelligent agent's multiple vehicle-mounted cameras, that is, the scanning area.

[0027] Specifically, the agent's scanning area is defined as follows: the location of each onboard camera and its corresponding camera parameters (such as mounting position, mounting angle, and focal length) determine the camera's field of view based on its location. The sum of the fields of view of multiple onboard cameras (deduplicated) constitutes the agent's scanning area. Let agent A's scanning area be VA, and agent B's scanning area be VB. The scanning areas can be represented within the current road area.

[0028] S102, based on the agent information of each agent, using the pre-set primary and secondary agent adaptation mechanism, generate a primary and secondary agent adaptation group for each agent, the primary and secondary agent adaptation group including a primary agent and at least one secondary agent.

[0029] In some embodiments, the pre-set primary and secondary agent adaptation mechanism specifically includes: S201, based on each intelligent agent, determine itself as the main intelligent agent, calculate the fitness between the main intelligent agent and each other intelligent agent respectively, and obtain the fitness of several pairs of intelligent agents.

[0030] Specifically, the fitness between the main agent and each other agent is calculated according to the following formula:

[0031] in, is the fitness between agent A and agent B, is the distance adaptation value between agent A and agent B, is the task adaptation value between agent A and agent B, is the field of view adaptation value of agent A and agent B, 、 and are pre-set weight factors, which are used to indicate the influence of distance, task and field of view on the fitness.

[0032] The distance adaptation value is calculated based on the position information of the two agents, specifically: , is the Euclidean distance between two agents, calculated using their position coordinates. The larger the value of , the closer the spatial proximity between agents A and B is, and thus the higher the distance adaptation value.

[0033] The task adaptation value is calculated based on the task set of the two agents, specifically: , is the task set of agent A, which includes at least one task content. is the task set of agent B. Therefore, the more similar the tasks are, The larger the value of , the more similar agents A and B are in executing tasks. Similar tasks must have different viewing directions set on different agents, which can make up for each other's blind spots.

[0034] The field of view adaptation value is calculated based on the scanning area of ​​the two agents, specifically: , is the scanning area of ​​agent A (main agent), is the scanning area of ​​agent B Zhongyu The difference between the areas, is the overlapping area of ​​the scanning areas of agent A and agent B, m and n are pre-set weight factors, which are used to indicate the influence of the distinguishing area and the overlapping area on the field of view adaptation value.

[0035] It should be noted that m is greater than n, and the sum of m and n is 1. Since the secondary agent's purpose is to supplement the primary agent's blind spot vision, the weight factor of the distinguishing area needs to be larger, indicating that the distinguishing area between the secondary agent and the primary agent has a greater impact on the decision-making degree of the two agents' field of vision adaptation. This is because the larger the distinguishing area, the greater the possibility that the secondary agent can effectively compensate for the primary agent's blind spot. Furthermore, if the two agents require a high field of vision adaptation value, the overlap area between their fields of vision cannot be too small. This is because a larger overlap indicates that the road environment information around the two agents is more similar, which can avoid generating too much invalid road environment information relative to the primary agent. A larger overlap indicates a greater field of vision adaptation value for both agents. Therefore, through the coordinated judgment of the overlapping and distinguishing areas, their mutual influence and control can more accurately and reasonably evaluate the field of vision adaptation value of the two agents.

[0036] S202, based on the fitness of all agent pairs, other agents in the agent pair whose fitness is greater than the preset fitness threshold are determined as secondary agents, and a binding relationship is established between them and the corresponding main agent, so that the main agent can receive the road image information collected by all secondary agents through wireless communication.

[0037] S103, based on each agent, obtain the road image information sequence transmitted by each agent in its primary and secondary agent adaptation group to form a road image set, the road image set includes at least one road image information sequence, and the road image information sequence includes several road images.

[0038] Among them, each intelligent body is equipped with multiple on-board cameras to synchronously collect road images. The road image information sequence transmitted by each intelligent body is composed of road images collected by multiple cameras installed on the intelligent body, so that environmental information from different perspectives and different positions can be obtained.

[0039] S104, based on each intelligent agent, the collected road image set is transmitted to its on-board computing module to detect road target objects, and the road perception information of the intelligent agent is output.

[0040] In some embodiments, each intelligent body is provided with an on-board computing module, which includes an image feature extraction module, a multi-feature fusion module and a multi-class target object detection module.

[0041] Specifically, the on-board computing module is used to: The received road image set is input into the image feature extraction module, and a feature extraction network (such as CSPDarkNet) is used to extract features from each road image. A road feature vector set is output and transmitted to the multi-feature fusion module. The feature extraction network is used to extract features from the road image. Through deep learning technology, key features such as edges, textures, and shapes can be automatically extracted from the road image. The extracted features are represented as high-dimensional vectors, which contain key information in the image. The road feature vectors extracted from all road images form a road feature vector set. The multi-feature fusion module is used to: use a spatial feature fusion network (such as a Transformer network model) to convert the received road feature vector set into a unified three-dimensional vector space, fuse the road feature vectors from different on-board cameras and different agents in the three-dimensional vector space, output the fused feature vector, and transmit it to the multi-class target object detection module; thereby, the road image information from different perspectives and different positions is fused to obtain a more comprehensive and accurate description of the environmental information; as an example, combined with the camera parameters, the received road feature vector set is converted into a unified three-dimensional vector space and divided into grids. For each grid, the corresponding projection position of each grid in the multiple cameras of the agent is calculated according to the camera parameters, and then the features of the corresponding positions of the road images collected by the multiple cameras of the multi-agent in the feature extraction network are fused as the features of the grid in the vector space. The output of the spatial feature fusion is the feature of the entire vector space at a single moment; The multi-class target object detection module is used to: use a multi-head prediction network to perform multi-class target object detection tasks on the received fused feature vectors, and output target objects and their location information; wherein, multi-class target object detection tasks include target detection, target segmentation, image classification, etc. As an example, the multi-head prediction network is a collection of multiple prediction networks, including a target detection network, an image segmentation network, and an image classification network. It uses the received fused feature vectors as shared features, and performs different categories of prediction tasks through prediction networks with different functions. The target detection network is used to predict the shared features in the vector space and output the size, location, and number of target objects such as intelligent agents, pedestrians, traffic lights, and traffic signs. The image segmentation network is used to output information such as the range of lane lines, drivable areas, and road edges. The image classification network outputs information such as the current weather, lighting, and whether the road surface is wet, rainy, or snowy.

[0042] Specifically, the process of obtaining the above-mentioned feature extraction network, spatial feature fusion network, and multi-head prediction network includes: A1. Collect a large number of road images, corresponding camera parameters (such as installation location, installation angle, focal length, and pixel density), and the corresponding agent's position and orientation. This includes road images under different weather conditions, time periods, and traffic conditions. The camera parameters are used in the subsequent spatial feature fusion network. The collected road images are annotated, such as the location and category of objects such as vehicles, pedestrians, and traffic signs, to obtain a training set. A2. Feature extraction network training: Select a suitable feature extraction network architecture, such as CSPDarkNet, to calculate features of road images. Optimize network parameters using the backpropagation algorithm to accurately extract key features from images and obtain road feature vectors. A3. Spatial feature fusion network training: Design a suitable spatial feature fusion network architecture, such as the Transformer network model. Combined with camera parameters, use the training set to train the spatial feature fusion network so that it can accurately fuse features from different cameras into a unified 3D vector space. A4. Multi-head prediction network training: Design a suitable multi-head prediction network architecture that can simultaneously perform multiple categories of prediction tasks. Prepare training data containing the fused fusion feature vector and target object information annotations. Use the training data to train the multi-head prediction network. Optimize the network parameters through the backpropagation algorithm so that it can accurately output information and location information of target objects such as vehicles, pedestrians, and traffic signs.

[0043] It should be noted that the detailed working principles of the feature extraction network, spatial feature fusion network, and multi-head prediction network can refer to the description of the relevant existing technology, and the present invention will not elaborate on this.

[0044] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By establishing a primary-secondary agent adaptation mechanism, the primary agent only shares information with necessary secondary agents and receives only information from other agents that is highly relevant to its task, location, and field of view. This reduces unnecessary information transmission and processing, thereby optimizing resource utilization of computing chips and communication networks. Through multi-dimensional feature fusion, multi-view, multi-agent, and multi-location road image information is integrated to obtain a more comprehensive and accurate description of environmental information, taking into account multiple factors such as the agent's task, location, and field of view, and can adapt to complex and changing road environments. The primary and secondary agent adaptation groups are established based on the positional relationship between the agents, their field of view, and the current task requirements. This makes information sharing more targeted and effective, and the agents can more quickly obtain the most valuable environmental information for their current tasks. By reducing unnecessary information sharing, the burden on the communication network is reduced and the network transmission efficiency and stability are improved; the establishment of primary and secondary agent adaptation groups makes the multi-agent system more flexible and efficient, able to adapt to different environments and task requirements, and improves the overall performance of the system.

[0045] Example 2: In Example 1, the calculation of the field of view adaptation value depends on the weight factors (m and n) of the overlapping area and the distinguishing area, and these weight factors are pre-set fixed values. Through the collaborative judgment of the overlapping area and the distinguishing area, they influence and control each other. However, there may be some areas in the distinguishing area between the secondary intelligent agent and the main intelligent agent that are not related to the main intelligent agent, so the setting of the corresponding weight factors is extremely critical. Due to the complex and changeable road environment, the use of fixed weight values ​​has great limitations and lacks adaptability and reliability.

[0046] Therefore, the embodiments of the present application are optimized based on the above embodiments.

[0047] In some embodiments, a method for setting weight factors corresponding to distinct areas and overlapping areas in a field of view adaptation value specifically includes: S301, obtaining the current position information and task content of the two agents, and obtaining the correlation coefficient r between the two agents based on the position information and task content of the two agents.

[0048] Among them, the correlation coefficient r is used to represent the difference between regions The relevance of r to the current task content of the main agent. The value range of r is between [0, 1]. The larger the value, the higher the relevance.

[0049] S302, dynamically adjust the pre-set basic weight factor based on the correlation coefficient and ( Greater than ), obtaining the weight factors m and n corresponding to the distinguished and overlapping areas, respectively. Therefore, the larger the correlation coefficient, the greater the impact of the ratio of the distinguished area to the main agent's scanning area on the field of view adaptation value, focusing more on the distinguished area and avoiding omissions in the field of view.

[0050] Specifically, the weight factors m and n corresponding to the distinguished area and the overlapping area are calculated according to the following formulas:

[0051]

[0052] Among them, m is the weight factor of the distinguished area, n is the weight factor of the overlapping area, is the basic weight factor for distinguishing regions, and r is the correlation coefficient.

[0053] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By introducing the correlation coefficient r and dynamically adjusting the weight factors m and n based on r, the calculation of the field of view adaptation value is made more adaptive and reliable, better adapting to the complex and variable road environment and the diversity of tasks, and improving the collaborative safety effect and task execution efficiency between intelligent agents. By dynamically adjusting the weight factor, when the secondary agent has a high correlation with the main agent's task content, the weight factor m of the distinguishing area increases, thereby enhancing the focus on the distinguishing area, helping to avoid omissions in the field of vision, improving the agent's perception of the surrounding environment and decision-making accuracy, and making the calculation of the field of vision adaptation value more flexible and accurate.

[0054] Example 3: Further limit the correlation coefficient between the two agents in Example 2.

[0055] Therefore, the embodiments of the present application are optimized based on the above embodiments.

[0056] In some embodiments, step S301 specifically includes: S401, based on the current position information and task content of the two agents, generate an agent-associated feature vector, which includes the distance value of the two agents, the deviation angle value of the secondary agent's position relative to the current forward direction of the main agent (the angle formed by the line connecting the two agent's position points and the ray of the current forward direction of the main agent) and the similarity of the agent's current task content.

[0057] S402: Input the agent-related feature vector into a pre-trained agent-related prediction model, and output the correlation coefficient of the two agents.

[0058] Specifically, the methods for obtaining pre-trained agent-related prediction models include: Collect agent-association feature vectors between primary and secondary agents at a large number of historical moments in the road area, and label each agent-association feature vector to obtain a training dataset. The labeling content is set as the correlation coefficient between the two agents, which is set based on the matching of the actual distinguishing area at the historical moment and the field of view required by the task content of the primary agent at the historical moment (which can be set based on the hit probability). This can be obtained through manual labeling or automatic calculation. The pre-selected neural network structure is trained using the training data set, the model parameters are optimized, and the final agent-related prediction model is obtained.

[0059] In some embodiments, the method further comprises: S105, periodically execute steps S101 to S104 at preset time intervals to achieve dynamic update of the primary and secondary agent adaptation groups of each agent to adapt to the complex and changeable dynamic environment in the road area, improve the field of view of each agent, and avoid resource loss of the built-in computing chip.

[0060] Specifically, the preset time interval can be dynamically adjusted based on actual conditions, set based on the motion trajectories and motion parameters of all agents within the road area to adapt to the complex and changing dynamic environment within the road area. For example, if the real-time monitoring detects a significant change in the current motion trajectory or motion parameters (acceleration, direction) of an agent relative to the previous moment (the change is greater than a preset threshold), a periodic update is triggered, and a new cycle of updated monitoring begins.

[0061] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By considering multiple factors such as the distance between the primary and secondary agents, the direction deviation angle, and the similarity of the task content, an agent-related feature vector is generated and input into the pre-trained agent-related prediction model to more accurately calculate the correlation coefficient between agents; Dynamically update the primary and secondary agent adaptation groups, enabling agents to adjust their field of view in a timely manner according to changes in the road environment, improving their environmental perception and decision-making efficiency; By periodically updating and dynamically adjusting the time interval, we can avoid resource loss of the built-in computing chip while ensuring the field of view of the intelligent agent, thereby improving the overall performance of the system.

[0062] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A multi-agent perception fusion method based on machine learning, characterized in that: include: S101, obtaining all agents and their corresponding agent information in the current road area, where the agent information of each agent includes location information, task information, and field of view information; S102, based on the agent information of each agent, using a pre-set primary and secondary agent adaptation mechanism, generating a primary and secondary agent adaptation group for each agent, the primary and secondary agent adaptation group including a primary agent and at least one secondary agent; S103, based on each agent, obtaining a road image information sequence transmitted by each agent in its primary and secondary agent adaptation group to form a road image set, wherein the road image set includes at least one road image information sequence, and the road image information sequence includes a plurality of road images; S104, based on each intelligent agent, the collected road image set is transmitted to its on-board computing module to detect road target objects, and the road perception information of the intelligent agent is output.

2. The multi-agent perception fusion method based on machine learning according to claim 1, characterized in that: The position information includes the position coordinates of the intelligent agent, the task information includes the task set of the intelligent agent, the field of view information includes the sum of the acquisition ranges of multiple vehicle-mounted cameras of the intelligent agent, and the field of view information is recorded as a scanning area.

3. The multi-agent perception fusion method based on machine learning according to claim 2, characterized in that: The pre-set primary and secondary agent adaptation mechanism specifically includes: S201, based on each agent, determine itself as the master agent, calculate the fitness between the master agent and each other agent, and obtain the fitness of several agent pairs; S202, based on the fitness of all agent pairs, other agents in the agent pair whose fitness is greater than the preset fitness threshold are determined as secondary agents, and a binding relationship is established between them and the corresponding main agent, so that the main agent can receive the road image information collected by all secondary agents through wireless communication.

4. The multi-agent perception fusion method based on machine learning according to claim 3, characterized in that: The degree of fitness is calculated according to the following formula: in, is the fitness between agent A and agent B, is the distance adaptation value between agent A and agent B, is the task adaptation value between agent A and agent B, is the field of view adaptation value of agent A and agent B, 、 and are pre-set weight factors, which are used to indicate the influence of distance, task and field of view on the fitness.

5. The multi-agent perception fusion method based on machine learning according to claim 4, characterized in that: The distance adaptation value is calculated based on the position information of the two agents, specifically: , is the Euclidean distance between two agents; The task adaptation value is calculated based on the task set of the two agents, specifically: , is the task set of agent A, which includes at least one task content. is the task set of agent B; The field of view adaptation value is calculated based on the scanning area of ​​the two agents, specifically: , is the scanning area of ​​agent A, is the scanning area of ​​agent B Zhongyu The difference between the areas, is the overlapping area of ​​the scanning areas of agent A and agent B, m and n are pre-set weight factors, which are used to indicate the influence of the distinguishing area and the overlapping area on the field of view adaptation value.

6. The multi-agent perception fusion method based on machine learning according to claim 1, characterized in that: Each of the intelligent bodies is provided with an on-board computing module, which includes an image feature extraction module, a multi-feature fusion module and a multi-class target object detection module; The vehicle-mounted computing module is specifically used for: The received road image set is input into the image feature extraction module, and the pre-set feature extraction network is used to extract the features of each road image, output the road feature vector set, and transmit it to the multi-feature fusion module; The multi-feature fusion module is used to: use a pre-set spatial feature fusion network to convert the received road feature vector set into a unified three-dimensional vector space, fuse the road feature vectors from different on-board cameras and different intelligent agents in the three-dimensional vector space, output the fused feature vector, and transmit it to the multi-class target object detection module; The multi-class target object detection module is used to: use a pre-set multi-head prediction network to perform multi-class target object detection tasks on the received fusion feature vectors, and output target objects and their location information.

7. The multi-agent perception fusion method based on machine learning according to claim 5, characterized in that: The method for setting the weight factors corresponding to the difference area and the overlap area in the field of view adaptation value specifically includes: S301, obtaining the current position information and task content of the two agents, and obtaining the correlation coefficient between the two agents based on the position information and task content of the two agents; S302, dynamically adjust the pre-set basic weight factor based on the correlation coefficient and , and obtain the weight factors m and n corresponding to the distinguished area and the overlapping area respectively.

8. The multi-agent perception fusion method based on machine learning according to claim 7, characterized in that: The weight factors m and n corresponding to the distinguished area and the overlapping area, respectively, are calculated according to the following formula: Among them, m is the weight factor of the distinguished area, n is the weight factor of the overlapping area, is the basic weight factor for distinguishing regions, and r is the correlation coefficient.

9. The multi-agent perception fusion method based on machine learning according to claim 7, characterized in that: The S301 specifically includes: S401: Generate an agent-association feature vector based on the current position information and task content of the two agents. The agent-association feature vector includes the distance between the two agents, the deviation angle of the secondary agent's position relative to the current direction of the primary agent, and the similarity of the agents' current task content. S402: Input the agent-related feature vector into a pre-trained agent-related prediction model, and output the correlation coefficient of the two agents.

10. The multi-agent perception fusion method based on machine learning according to claim 1, characterized in that: The method further comprises: S105, periodically executing steps S101 to S104 at preset time intervals to achieve dynamic update of the primary and secondary agent adaptation groups of each agent.

Images