Point element annotation capping conflict processing method and device based on deep reinforcement learning

Abstract

The invention belongs to the technical field of data processing, and provides a point element note capping conflict processing method and device based on deep reinforcement learning, and the method comprises the steps: determining the conflict existence, actual capping area, position priority and semantic importance level of each point element note in a target region; calculating to obtain an importance weight score of the element annotation of each point; determining an agent selection strategy by combining the importance weight score and a pre-obtained conflict number corresponding to each capping conflict event, and dividing target point element notes with capping conflicts into a dynamic conflict data set and a static conflict data set; and based on the dynamic and static conflict data sets, by means of a pre-established MADDPG model and through a displacement constraint reward strategy, controlling the target point element annotation with the capping conflict to execute a displacement action until the displacement action reaches an optimal position. According to the scheme, the processing efficiency and quality of large-scale conflict scenes can be considered.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for handling point feature annotation overlay conflicts based on deep reinforcement learning. Background Technology

[0002] In the fields of geographic information systems, cartography, and data visualization, point feature annotations, as the core carriers for identifying the names and attributes of geographic entities, are a key medium for users to obtain spatial information. With the development of technologies such as the Internet of Things, autonomous driving, and high-precision maps, geographic data is growing exponentially, and high-density point feature distribution scenarios such as urban core areas and emergency command maps are becoming increasingly common, leading to a more prominent problem of spatial overlap conflicts between point feature annotations.

[0003] Currently, traditional methods for handling annotation overlap conflicts mainly fall into two categories: one is manual optimization based on geometric rules, such as ArcGIS's MapLex intelligent annotation system, which adjusts annotation positions by pre-setting rules such as text avoidance cache range and overlap tolerance. However, this type of method relies on manually defined rules, and when faced with dynamic scaling and multi-source heterogeneous data scenarios, it is prone to local optima and global suboptimal problems, and the overlap problem is difficult to be fully resolved after adjustment. The other is automatic processing methods based on heuristic algorithms, such as genetic algorithms, simulated annealing algorithms, and ant colony algorithms. Although they can achieve a certain degree of automation, they have drawbacks such as low computational efficiency, easy to get trapped in local optima, and difficulty in taking into account multiple constraints in high-density data or dynamic environments. Furthermore, they cannot guarantee the positional stability of highly important annotations, leading to an imbalance in the transmission of map information.

[0004] It is evident that the current point feature annotation overlay conflict handling method suffers from technical problems such as low automation, poor adaptability to complex scenarios, and insufficient multi-agent collaborative optimization capabilities. Summary of the Invention

[0005] This invention provides a method and apparatus for handling point feature annotation overlay conflicts based on deep reinforcement learning, which addresses the shortcomings of current point feature annotation overlay conflict handling methods, such as low automation, poor adaptability to complex scenarios, and insufficient multi-agent collaborative optimization capabilities.

[0006] On one hand, this invention provides a method for handling point feature annotation overlay conflicts based on deep reinforcement learning, comprising: determining the existence of conflict, actual overlay area, positional priority, and semantic importance level of each point feature annotation within a target area, and calculating the importance weight score of each point feature annotation; combining the importance weight score and the number of conflicts corresponding to each overlay conflict event obtained in advance, determining an agent selection strategy, and dividing the target point feature annotations with overlay conflicts into a dynamic conflict dataset and a static conflict dataset; based on the dynamic conflict dataset and the static conflict dataset, using a pre-established MADDPG model, and through a shift constraint reward strategy, controlling the target point feature annotations with overlay conflicts to perform shift actions until the optimal position.

[0007] On the other hand, the present invention also provides a point feature annotation overlay conflict handling device based on deep reinforcement learning, comprising: a calculation module, used to determine the conflict existence, actual overlay area, position priority, and semantic importance level of each point feature annotation within a target area, and to calculate the importance weight score of each point feature annotation; a partitioning module, used to combine the importance weight score and the number of conflicts corresponding to each overlay conflict event obtained in advance to determine the agent selection strategy, and to partition the target point feature annotations with overlay conflicts into a dynamic conflict dataset and a static conflict dataset; and a processing module, used to control the target point feature annotations with overlay conflicts to perform a shift action until the optimal position based on the dynamic conflict dataset and the static conflict dataset, using a pre-established MADDPG model and a shift constraint reward strategy.

[0008] The present invention provides a method and apparatus for handling point feature annotation overlay conflicts based on deep reinforcement learning. By integrating conflict existence, actual overlay area, location priority, and semantic importance level to construct a multi-dimensional importance weight scoring system, it achieves accurate screening of high-priority conflict annotations, ensuring the priority presentation of key geographic information. By determining specific agent selection strategies and dividing target point feature annotations with overlay conflicts into static and dynamic conflict datasets, it reduces model training complexity and improves processing efficiency through a static environment, while adapting to complex multi-annotation conflict scenarios through a dynamic environment. Relying on the multi-agent collaborative optimization capability and multi-constraint shift reward strategy of the MADDPG model, it effectively reduces annotation overlay conflicts and secondary conflicts while meeting cartographic specifications, significantly improving the rationality and readability of annotation layout, and balancing processing efficiency and configuration quality in large-scale conflict scenarios. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating the point feature annotation overlay conflict handling method based on deep reinforcement learning provided in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the principle of handling point feature annotation overlap conflicts using the MADDPG model; Figure 3 This is a layout diagram showing how to score the overlay position based on its priority. Figure 4 This is a schematic diagram illustrating the selection principle of point element annotations; Figure 5 This is a schematic diagram illustrating the principle of the agent selection strategy; Figure 6 This is a schematic diagram illustrating the training principle of the MADDPG model; Figure 7 This is a schematic diagram illustrating the setting principle of the buffer area; Figure 8 This is a schematic diagram illustrating the changes in the reward function for the static environment group; Figure 9 This is a schematic diagram illustrating the changes in the reward function of the dynamic environment group; Figure 10 This is a schematic diagram showing the changes in the loss functions of the Actor network and Critic network in the static environment group; Figure 11 This is a schematic diagram illustrating the changes in the loss functions of the Actor network and Critic network in the dynamic environment group; Figure 12 This is a schematic diagram illustrating the changes in the reward function; Figure 13 This is a schematic diagram showing the changes in the loss function of the Actor network and the Critic network; Figure 14 This is a schematic diagram of the structure of the point feature annotation overlay conflict handling device based on deep reinforcement learning provided in an embodiment of the present invention. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0012] The following is combined Figures 1 to 14 This invention describes in detail the method and apparatus for handling point feature annotation overlay conflicts based on deep reinforcement learning, as provided in the embodiments of the present invention.

[0013] like Figure 1 As shown in the figure, the point feature annotation overlay conflict handling method based on deep reinforcement learning provided in this embodiment of the invention mainly includes the following steps: Step 110: Determine the conflict existence, actual coverage area, location priority, and semantic importance level of each feature annotation within the target area, and calculate the importance weight score of each feature annotation.

[0014] This embodiment uses importance weight scores to detect spatial overlap conflicts between point feature annotations and quantifies the severity of overlap conflicts between different point feature annotations, providing a theoretical basis for prioritizing subsequent conflict resolution. In the context of map point annotations, the calculation of importance weight scores requires comprehensive consideration of multiple influencing factors. This embodiment uses four factors—conflict existence, actual overlap area, location priority, and semantic importance level—as the core influencing elements of the importance weight score. Through the synergistic effect of multiple factors, it objectively reflects the differences in the importance of overlap conflicts between different point feature annotations.

[0015] Step 120: Combine the importance weight score and the number of conflicts corresponding to each overlay conflict event obtained in advance to determine the agent selection strategy, and divide the target point feature annotations with overlay conflicts into dynamic conflict datasets and static conflict datasets.

[0016] This embodiment, through the construction of dynamic and static conflict datasets, can provide accurate data support for subsequent model training and actual conflict handling.

[0017] Step 130: Based on the dynamic conflict dataset and the static conflict dataset, and with the help of the pre-established MADDPG model, and through the shift constraint reward strategy, control the target point feature annotations with overlapping conflicts to perform shift actions until the optimal position is reached.

[0018] It is understandable that by using two types of datasets, leveraging the MADDPG (Multi-Agent Deep Deterministic Policy Gradient) model and employing a shift-constrained reward policy, point element overlap conflicts can be effectively and efficiently handled. MADDPG is a multi-agent reinforcement learning algorithm based on DDPG (Deep Deterministic Policy Gradient), suitable for multi-agent collaborative or adversarial scenarios.

[0019] This embodiment utilizes the MADDPG model to detect point annotation elements with spatial overlap conflicts and determine their conflict priorities. It then employs a targeted agent selection strategy, constructs a relatively static agent operating environment, and establishes corresponding shift-constraint reward strategies to constrain and guide point annotations. Next, leveraging the action guidance of the actor network and the evaluation of the actor network's actions by the commentator network within the MADDPG model, along with the target actor network and target commentator network, the network parameters are continuously updated through experience replay and loss function calculation to achieve a better fit with the actual situation, thus enabling prediction and guidance of point annotation actions. Finally, by determining whether the actions meet the constraints, a reasonable point annotation layout scheme is output to resolve spatial overlap conflicts between point annotations. The specific process is as follows: Figure 2 As shown.

[0020] In one embodiment, combined with Figure 2 The importance weight score of each feature annotation is calculated, specifically including: On the one hand, based on the existence of conflict, the overlap conflict score of each element annotation is determined.

[0021] In the process of representing point feature annotations, overlapping between annotations is inevitable. Overlapping conflicts significantly impact the clarity of map information. Detecting these overlapping conflicts is a fundamental requirement. For cases of overlapping conflicts, a score of 1 can be assigned to the conflict, representing the occurrence of a conflict; conversely, 0 represents the absence of a conflict. The specific formula is: (1) In the formula, Points are awarded for the cover collision.

[0022] On the other hand, the coverage area score for each element annotation is determined based on the actual coverage area.

[0023] Understandably, the size of the area of ​​the overlaid feature directly reflects the severity of the overlay; a larger overlay area has a greater impact on the clear transmission of map information. Generally, when determining the score for the overlay area, the overall layout and visual effect of the map should be considered. If the overlay feature area is large, it means that the point feature annotation covers it to a high degree, which will seriously obscure the information carried by the point feature itself. In this case, a higher score should be assigned to it, so that it is more sensitive to large-area overlay when evaluating overlay conflicts, prompting the algorithm to prioritize handling such cases when resolving conflicts, and minimizing the occurrence of large-area overlay. Conversely, for cases where the overlay feature area is small, its damage to information transmission is relatively limited, and the overlay area score can be appropriately reduced. Therefore, in this embodiment, the value of the overlay area score is not fixed, but dynamically set according to the actual overlay area, as follows: (2) In the formula, Score based on the area of ​​the covering. To broaden the factor and avoid excessively small values, it is manually specified based on the actual situation. This represents the actual coverage area.

[0024] On the other hand, based on position priority, the overlay position score of each element annotation is determined.

[0025] It is understandable that the positional priority of point feature annotations directly affects the rationality and readability of annotation configuration. Specifically, positional priority is usually determined based on a combination of factors, including the distribution characteristics of map features, visual perception patterns, and cartographic standards. For point features, the basic principle is generally to the right first, then above, then to the left, and finally below. Therefore, this embodiment assigns different overlay position scores to different locations of point feature annotations according to positional priority, as detailed below... Figure 3 The subgraphs A1 and B1 are shown in the diagram.

[0026] On the other hand, based on the semantic importance level, the semantic score of each element annotation is determined.

[0027] It is understandable that point feature annotations on maps, as semantic identifiers of geographic features, exhibit significant differences in importance. For example, point feature annotations such as city names and key scenic spots have a significantly higher priority than those of less important features. Therefore, it is necessary to assign differentiated semantic scores to point feature annotations based on their semantic importance level, ensuring that highly semantically important point feature annotations are identified first. In practical applications, the determination of semantic importance levels follows these principles: First, based on the differentiated assignment of point feature annotations according to their level attributes, high-level attribute annotations, such as provincial administrative centers and national nature reserves, are assigned higher scores than low-level annotations, such as ordinary villages and rural roads, to reflect the level difference. Second, non-high-level annotations, such as companies, apartments, and buildings, are assigned lower scores than high-level annotations. Then, annotations with non-high-level attributes but special value, such as museums and scenic spots, are assigned the second-highest score to ensure their priority in conflict resolution. Finally, ordinary level annotations, such as detached houses without attribute markings and unnamed paths, are assigned the lowest score and are given the highest priority for concessions in conflict resolution.

[0028] This scoring system constructs a priority logic for resolving annotation conflicts through hierarchical semantic importance identification, ensuring that the primary and secondary information in map expression is clear and cognitively efficient. Specific examples of annotation semantic scoring are shown in Table 1.

[0029] Table 1 Examples of assigning semantic scores to annotations

[0030] Finally, the scores for overlapping conflict, overlapping area, overlapping location, and annotation semantics are weighted and summed to calculate the importance weight score for each element annotation.

[0031] By independently judging each of the four parts of a single point feature annotation, and then weighting and summing the scores calculated from the four parts, the total importance weight score of the single point feature annotation can be obtained. In this embodiment, weight values ​​of 0.3, 0.25, 0.25, and 0.2 are assigned to the overlay conflict score, overlay area score, overlay position score, and annotation semantic score, respectively, to distinguish their different priorities and thus determine the processing priority level of the current feature. The specific formula is as follows: (3) In the formula, Assigning importance weights to scores, To score for the cover collision, Score based on the area of ​​the covering. Score for the pressure point position. The semantic score is assigned to the annotation.

[0032] By performing the above calculations on the annotation data, the identification can be based primarily on whether the annotation causes overlay, secondarily on the area of ​​overlay and the location of the annotation, and thirdly on the semantics of the annotation. This enables a comprehensive identification of annotation data with serious spatial overlay conflicts and strong semantic levels.

[0033] In one embodiment, the agent selection strategy is determined by combining the number of conflicts and the importance weight score corresponding to each capping conflict event, specifically including: First, determine the number of conflicts in each capping conflict event.

[0034] In one scenario, if the number of conflicts is 2, then the conflict type is determined to be binary.

[0035] In another scenario, if the number of conflicts is 3, then the conflict type is determined to be ternary.

[0036] In another scenario, if the number of conflicts is greater than 3, the conflict type is determined to be complex.

[0037] Finally, based on the conflict type and the importance weight score, the agent selection strategy is determined.

[0038] In the conflict classification stage, based on the number of conflicts involved in a single overlay conflict event, all overlay conflict events are divided into three categories to provide a classification basis for subsequent differentiated processing. The binary type consists of pairwise overlay conflicts between two point feature annotations. The ternary type consists of small-scale overlay conflicts between three point feature annotations. The complex type consists of large-scale overlay conflicts between four or more annotations.

[0039] In practical applications, for point feature annotations that have spatial overlap conflicts, the integrity of the annotation as a whole should be maintained when moving them to avoid situations such as content loss during the movement. For example... Figure 4 As shown in subgraph A2, this ensures the accuracy of the annotation content. Therefore, in this embodiment, the rectangular area covered by the point feature annotation itself is treated as an agent to participate in the action of deep reinforcement learning, such as... Figure 4 As shown in subplot B2, the final position of the rectangular area serves as the distribution location for the point feature annotations. Furthermore, according to the feature annotation positions specified in cartographic standards, the distribution location of point feature annotations should be directly to the right of the point feature at a predetermined distance d, as shown below. Figure 4 As shown in the C2 subgraph, in this embodiment, this distribution location will be used as the initial position for the agent's movement in subsequent actions.

[0040] In one embodiment, the agent selection strategy is determined based on the conflict type and importance weight score, specifically including: In one scenario, if the conflict type is complex, the agent selection strategy is to annotate all point elements involved in the current overlay conflict event as agents.

[0041] In another scenario, if the conflict type is binary or ternary, the agent selection strategy is to combine the importance weight scores of each point element annotation involved in the current overlay conflict event and select some point element annotations as agents.

[0042] Furthermore, the target point feature annotations with overlapping conflicts are divided into dynamic conflict datasets and static conflict datasets, specifically including: If the conflict type is complex, the corresponding target point feature annotations with overlapping conflicts will be assigned to the dynamic conflict dataset.

[0043] If the conflict type is binary or ternary, the feasibility of the agent selection strategy is determined. If the agent selection strategy is feasible, or feasible after adjustment, the corresponding target point element annotations with overlapping conflicts are assigned to the static conflict dataset. If the agent selection strategy is infeasible and still infeasible after adjustment, the corresponding target point element annotations with overlapping conflicts are assigned to the dynamic conflict dataset.

[0044] In this embodiment, for complex types of overlay conflict events, due to the large number of conflicts and complex interaction relationships involved, even if some agents are selected to participate in the calculation, the dynamic adjustment of the remaining annotations may still lead to frequent changes in the environmental state, making it difficult to construct a static environment. Therefore, the annotations of target point elements with overlay conflicts can be directly included in the dynamic conflict dataset, and the model can be trained according to the dynamic environment logic.

[0045] For the two types of overlay conflict events classified as binary and ternary, these events are relatively small in scale and have relatively simple interaction relationships, providing a basis for constructing a static environment. The processing flow consists of two steps: determining the agent selection strategy and assessing the feasibility of conflict resolution during the execution of the agent selection strategy. For details, please refer to [link to relevant documentation]. Figure 5 .

[0046] like Figure 5 As shown, the selection of agents is divided into two steps. The first step is the initial screening of agents. This involves combining the importance weight scores of point feature annotations with the spatial distribution of conflicts, such as the density of annotations and the size of surrounding blank areas. Priority is given to selecting point feature annotations with lower importance weight scores (i.e., lower importance levels) as agents to participate in model training. The core purpose is that high-level annotations usually carry more critical geographical information, and their positional stability is crucial to map readability. Avoiding high-level annotations from participating in movement adjustments ensures the maximum possible accuracy of their original positions.

[0047] The second step involves assessing the feasibility of conflict resolution and dividing the dataset. Low-level annotations selected are used as agents. The feasibility of resolving the overlapping conflict through action adjustments is first assessed. If the low-level annotation can resolve the conflict independently, the target point element annotations involved in the overlapping conflict event are included in the static conflict dataset. If the low-level annotation cannot resolve the conflict, other annotations are selected as agents based on the spatial distribution characteristics of the conflict. For example, other high-level annotations or point element annotations with greater adjustment space can be used as new agents, and the feasibility of conflict resolution is reassessed. If the new agent can resolve the conflict, the target point element annotations involved in the overlapping conflict event are still included in the static conflict dataset. If, after two rounds of selection, the conflict still cannot be resolved by a single or a few agents, all point element annotations under the overlapping conflict event are used as agents, and the target point element annotations involved in the overlapping conflict event are included in the dynamic conflict dataset. The model then explores resolution strategies through multi-agent collaborative interaction.

[0048] Understandably, the MADDPG model drives each agent to autonomously generate action decisions, with its action strategy limited to moving and adjusting within eight pre-defined candidate positions around the annotation. Through continuous iteration to filter for the optimal landing point, and using an algorithm to determine whether the annotation overlaps with the target position, if the model can ultimately match each annotation to a target position that satisfies the no-overlap constraint, then the agent's action strategy of resolving overlap conflicts through action adjustment is feasible; otherwise, it is not feasible.

[0049] Combination Figure 2 As can be seen, this embodiment adopts an action policy framework based on deterministic policy gradients. In a multi-agent environment, the decision-making policies of each agent are modeled through deep neural networks. Each agent's policy output depends not only on its current state but also on the behavioral information of other agents. To enhance policy exploration capabilities, the MADDPG model introduces a time-dependent noise mechanism. This mechanism allows the agent's action selection to expand the exploration space by superimposing perturbation terms related to historical noise, rather than being limited to the current policy mapping. This time-dependent noise characteristic effectively guides the agent to discover better decision paths while ensuring policy continuity.

[0050] In the MADDPG model, the actual action of each agent is composed of the action prediction information output by its actor network plus a noise term, as shown in the following formula: (4) In the formula, It is an intelligent agent i In time t The final action chosen, It is an intelligent agent iDeterministic actions generated by the actor network based on current state information, i.e., action prediction information. It is the first The agent in the th... The state of the step, It is the first The parameters of the network related to each agent, It is a noise term used to enhance the exploratory aspect.

[0051] In one embodiment, the MADDPG model specifically includes: An actor network is used to acquire the local state information of each agent at the current moment, and output the action prediction information at the current moment based on the local state information at the current moment.

[0052] The commentator network is used to obtain global action information of all agents at the current moment, and output the current reward value based on the global action information and the action prediction information at the current moment. It is used to evaluate the quality of the actions performed by the agent network, specifically to evaluate the quality of the actions performed by the agent network.

[0053] The target actor network is configured in a one-to-one correspondence with the actor network. The target actor network is used to obtain the local state information of each agent at the next moment, output the action prediction information at the next moment based on the local state information at the next moment, and follow the network parameters of the actor network through a soft update mechanism.

[0054] The target commentator network is used to obtain global action information of all agents in the next time step, and output the target reward value based on the global action information and the action prediction information in the next time step.

[0055] Even better, during the model application phase, the MADDPG model guides the agent's independent decision-making solely based on the agent's own state information and the action prediction information output by the actor network and the target actor network.

[0056] Combination Figure 2 and Figure 6 As shown, the MADDPG model is based on the Actor-Critic (policy-value) architecture and constructs a network system for multi-agent scenarios. It configures an independent Actor network and Critic network for each agent, and introduces a target network, experience playback, and centralized training and decentralized execution mechanisms to achieve policy optimization and environmental adaptation.

[0057] In terms of network operation logic, the actor network outputs the action prediction information for the current moment based on the agent's current state information. The action generation strategy can be represented as follows: That is, in the states t Select action a t The strategy involves the following steps: After the action prediction information is input into the commentator network, it calculates the reward value for the current action by combining it with global action information from other agents. The commentator network then evaluates the action chosen by the actor network based on this current reward value, determining its merits. The actor network then optimizes its network parameters based on the commentator network's evaluation. Simultaneously, the target actor network (i.e., the target actor network) calculates the action prediction information for the agent at the next time step. The target critic network (i.e., the target commentator network) further calculates the target reward value using the result from the target actor network. The details are as follows: (5) In the formula, It is the current moment. t The current reward value, The discount factor determines the importance of future rewards. The target actor network in the next moment t +1 to the target reward value, Represents state-based Calculated expected value, Indicates the first Step's environmental state, express The actions to be performed step by step.

[0058] The critic network optimizes its parameters using gradient descent based on the difference between the current reward value and the target reward value, achieving an accurate estimate of the policy value. This process ultimately updates the parameters of both the target actor network and the critic network. Specifically, during training, the actor network updates its parameters based on the critic network's evaluation results. In this process, the actor network's goal is to update its parameters based on the reward value. and benchmark value V critic ( s t Maximize the advantage function A ( s t ,a t ), and through the soft update mechanism of the target Actor network (i.e. θ actor = τθ online + (1 -τ ) θ actor Stable training, among which, θonline This represents the online network parameters of the Actor network currently being trained. θ actor This represents the old parameters of the target Actor network. τ This indicates the rate of soft updates. The Critic network calculates the target Q-value using the target Critic network, and then updates the parameters by calculating the loss.

[0059] To ensure training stability and efficiency, the target actor network and the target commentator network are both delayed-update copies of the current network. Their parameters are updated through a soft-update mechanism, as follows: (6) In the formula, These are the parameters of the current actor network. These are the parameters of the target actor network. This is the soft update ratio, typically a value less than 1. The parameters of the target actor network and the target commentator network are gradually updated to approximate the current network through soft updates, thus avoiding instability during training.

[0060] Meanwhile, this embodiment introduces an experience replay mechanism, where the agent stores the experience of each step in a buffer. During training, data is randomly sampled to update the network, breaking the temporal correlation of experience and improving data diversity. A centralized training and distributed execution mechanism is adopted. During the training phase, the commentator network integrates the state and action information of all agents for joint evaluation, ensuring the global nature of policy optimization. During the execution phase, each agent makes independent decisions based only on its own observations, adapting to multi-agent collaborative scenarios.

[0061] In summary, MADDPG, through the synergistic effects of alternating Actor-Critic optimization, soft updates of the target network, experience replay, and centralized training combined with distributed execution mechanisms, can achieve stable and efficient policy learning in multi-agent environments, driving the system towards overall optimal decision-making. The entire training process is as follows: Figure 6 As shown.

[0062] In one embodiment, the process of establishing a shift-constrained reward strategy specifically includes: On the one hand, during the execution of the shifting action, if the target point feature annotation does not move beyond the preset map boundary, a non-negative value will be set as the reward value; if the target point feature annotation moves beyond the map boundary, a negative value will be set as the penalty value, thus obtaining the map constraint reward strategy.

[0063] It is understandable that the map sheet, as the carrier of map data, is the foundation of all map data. Therefore, the movement range of point feature annotations must be within the map sheet frame to avoid exceeding the map sheet boundaries and causing data loss, thereby affecting the accuracy of the map data. Therefore, this embodiment uses map sheet boundary constraints as the most basic constraint condition. If a point feature annotation moves beyond the map sheet boundary, a penalty value of -5 is assigned; conversely, a reward value of 0 is assigned. This guides the agent to avoid exceeding the map sheet boundary. The reward value in the map sheet constraint reward strategy... The specific formula is: (7) On the other hand, during the execution of the shifting action, if there is no secondary overlap conflict between the target point feature annotation and other point feature annotations, a non-negative value will be set as the reward value. If there is a secondary overlap conflict between the target point feature annotation and other point feature annotations, the intersection area of ​​the secondary overlap conflict will be determined, and the negative number of the intersection area will be used as the penalty value, thus obtaining the first conflict constraint reward strategy.

[0064] During the execution of actions by an agent, it is inevitable that it will overlap with other point feature annotations. This indicates that the agent has not achieved an ideal distribution state, and the occurrence of overlap conflict is the most serious situation. Therefore, this embodiment constructs a secondary conflict constraint condition and treats this constraint as the most stringent constraint to avoid the occurrence of secondary conflicts. If a point feature annotation causes a secondary overlap conflict with other point feature annotations, the intersection area A between the agent and the conflicting annotation is calculated, and -A is used as the penalty value; otherwise, 0 is used as the reward value. The reward value in the primary conflict constraint reward strategy is... The calculation formula is as follows: (8) On the other hand, during the execution of the shifting action, if there is no secondary overlap conflict between the target point feature annotation and other point features, a non-negative value will be set as the reward value; if there is a secondary overlap conflict between the target point feature annotation and other point features, a negative value will be set as the penalty value, thus obtaining the secondary conflict constraint reward strategy.

[0065] During the execution of actions, intelligent agents inevitably overlap with other point features. This can affect the representation of map data, making point feature annotations ambiguous and prone to errors in information transmission. Therefore, this embodiment constructs a secondary conflict constraint condition between point feature annotations and other point features to avoid such secondary conflicts. If a point feature annotation causes a secondary overlap conflict with other point features, a penalty value of -2 is assigned; otherwise, 0 is used as a reward value. The reward value in the secondary conflict constraint reward strategy... The calculation formula is as follows: (9) On the other hand, during the execution of the shifting action, if the target point feature annotation does not move beyond the pre-set buffer area, a non-negative value will be set as the reward value; if the target point feature annotation moves beyond the buffer area, a negative value will be set as the penalty value, thus obtaining the shifting buffer constraint reward strategy.

[0066] In a specific implementation, the buffer region is determined through the following process: First, determine the set distance between the point feature annotation and the corresponding point feature.

[0067] Then, using the point features as the center and the set distance as the radius, the initial area of ​​the circle is determined.

[0068] Subsequently, based on the initial region, multiple rectangular regions are constructed in multiple defined directions starting from the point features.

[0069] Finally, multiple rectangular areas are used as buffer areas.

[0070] According to cartographic standards, point feature annotations surrounding point features should be distributed within a defined distance range. This defined distance d can be determined according to cartographic standards, and the final distribution of the point feature annotations must also fall within this range. Specifically, as shown below... Figure 7 As shown in sub-figure A3. As a preferred example, this embodiment constructs eight rectangular regions in eight directions around the point feature as buffer zones, specifically as follows... Figure 7 As shown in subplot B3, the size of the buffer area and the set distance d from the point feature are determined by the annotation content size and cartographic specifications. In practical applications, the number of rectangular areas can be reasonably set according to actual needs. This embodiment only uses 8 as a preferred example for illustration and is not intended to impose a specific limitation.

[0071] The agent's movement is restricted to a buffer zone to ensure the agent's distribution conforms to mapping specifications. If the agent's final action exceeds the buffer zone, a penalty of -6 is applied; otherwise, a reward of 0 is given. This is the reward value in the movement buffer constraint reward policy. The calculation formula is as follows: (10) Finally, the map sheet constraint reward strategy, the first conflict constraint reward strategy, the second conflict constraint reward strategy, and the movement buffer constraint reward strategy are used as the shift constraint reward strategy.

[0072] In practical applications, an agent can only be considered to have reached the target state if all its actions in the current state strictly conform to the positive reward conditions in the aforementioned constraint-reward strategies. In this embodiment, the reward values ​​from each of the above constraint-reward strategies are sequentially accumulated, serving as a key basis for evaluating the agent's learning effect, guiding its subsequent learning direction, and optimizing overall learning performance. The total reward value... The specific formula is: (11) If the total reward value If the value reaches 0, it indicates that the agent's movement satisfies all constraints, and its distribution scheme is output; otherwise, training continues and actions are executed.

[0073] To verify the effectiveness of the method provided in this embodiment, an example is given below to illustrate the specific implementation process. The target area in this example contains 142 point feature annotations. First, spatial overlap conflicts are determined by calculating importance weight scores, and the importance of these conflicts is assessed. The spatial overlap conflicts in the point feature annotations within the target area are then ranked by importance. Detection reveals that there are 40 point feature conflicts with spatial overlap in the target area. These 40 point feature annotations are ranked according to their importance weight scores, and the information for the top 10 point feature annotations is shown in Table 2.

[0074] Table 2. Annotation information for the first 10 points.

[0075] Then, the annotation data within the target area is processed step-by-step according to the logic of basic extraction, conflict analysis, and effect verification. First, point elements and point element annotations are extracted from the annotation data of the target area to clarify the spatial distribution of point element annotations. Next, for the extracted point element annotations, the rectangular area corresponding to each point element annotation is further extracted, and the spatial range of the point element annotations is transformed into quantifiable geometric units, which serves as the core basis for judging whether there is spatial overlap conflict between point element annotations. Subsequently, conflicting annotations are identified through geometric overlay analysis, and the rectangular areas of conflicting annotations are highlighted to visually present the conflict distribution characteristics. Finally, the spatial position adjustment of the annotation content is replaced by the movement of the annotation rectangular areas. By observing the spatial distribution of the adjusted annotation rectangular areas, the model's effect on resolving annotation conflicts is evaluated, thereby judging the overall performance of the model.

[0076] By extracting conflicts and determining their importance from the annotation data of the target area, the spatial overlapping conflicts between annotations are grouped according to their distribution. Based on whether the conflicts within a group can construct a static operating environment, they are divided into dynamic environment group and static environment group, namely dynamic conflict dataset and static conflict dataset. According to the corresponding agent selection strategy, some point feature annotations are selected as agents to participate in the later experiments.

[0077] To adapt the MADDPG model to the scale and complexity of the number of agents and their actions within the target region, a parameter system needs to be designed based on the algorithm's characteristics and research requirements. This system should ensure training stability while addressing annotation conflicts. The specific parameter settings, design basis, and training rules are as follows: First, for the experience data management stage, an experience pool with a capacity of 100,000 is set. A rule is set to initiate network parameter updates once the experience pool is full, ensuring that a sufficiently diverse range of interaction samples can be accumulated during the initial training phase. This capacity is based on the number of experience tuples generated by a single agent interaction. This ensures that sufficient historical experience can be stored to break the temporal correlation of data and avoid training oscillations, while also preventing excessive memory consumption due to excessive capacity. This provides a reliable data foundation for optimizing the Actor and Critic networks.

[0078] Specifically, the sample size in each batch is 64, which aligns with the common configuration of mini-batch stochastic gradient descent in deep reinforcement learning. Secondly, regarding the network learning rate design, the Actor network has a learning rate of 0.0005, and the Critic network also has a learning rate of 0.005. This is because the Critic network is responsible for evaluating action value and needs to respond more quickly to changes in value in empirical data to provide accurate optimization signals, hence the relatively higher learning rate. The Actor network is responsible for generating action policies and needs to iterate slowly to avoid excessive policy fluctuations, hence the lower learning rate. The discount factor is set to 0.99, a value close to 1, indicating that the model focuses more on long-term cumulative rewards rather than short-term local rewards, which aligns with the task characteristic of annotation conflict optimization requiring multi-agent collaborative adjustment.

[0079] To balance the agent's exploration and utilization, a noise mechanism is adopted, with an initial noise value of 0.9, a decay rate of 0.91, and a minimum value of 0.1. In the early stages of training, the noise value is relatively high, and the agent mainly focuses on exploration to avoid getting trapped in local optima. As training progresses, the noise value is gradually reduced at a decay rate of 0.91 until it stabilizes at 0.1. At this point, the agent shifts to utilizing existing optimal strategies and focuses on optimizing the accuracy of annotation conflict resolution. This noise decay logic conforms to the training pattern of multi-agent tasks from scenario exploration to strategy optimization.

[0080] Regarding training rounds and action rules, considering the complexity of annotation conflicts in the study area, the maximum number of actions per round for the agent was set to 20, ensuring that the agent could optimize local conflicts within a single round. Simultaneously, the total number of training rounds was set to 600. Through iterative cycles of 10 actions per round, the agent continuously updates its strategy based on interaction experience, gradually approaching the optimal solution for annotation conflict resolution. Finally, based on the above parameter configuration, the MADDPG model was used to calculate the data in the study area to optimize and resolve the annotation conflict problem within the study area. The conflict handling results for the static and dynamic environment groups are shown in Table 3.

[0081] Table 3. Conflict resolution results for the static environment group and the dynamic environment group.

[0082] As shown in the precise data in Table 3, the first value in the last row of the totals is the subtotal for the dynamic scenario, and the second value is the sum of the two totals, used to visually present the overall processing effect. In handling conflict issues, the MADDPG model demonstrated significant processing efficiency, with a dramatic change in the number of conflicts before and after processing. The number of conflicts decreased dramatically from 40 to 12, reflecting the overall effectiveness of the model in handling conflicts. In the specific scenario of the static environment group, all ten existing conflicts were properly resolved after processing. This result fully demonstrates the model's powerful ability and significant effectiveness in resolving conflicts in static scenarios.

[0083] In the dynamic environment group, after processing by the MADDPG model, the number of conflicts decreased from the initial 17 to 12. Although some conflicts remain, the overall processing trend is positive. Even in the face of complex conflict scenarios, the model still optimizes and improves the position distribution of agents as much as possible with its advanced algorithm architecture and flexible adjustment mechanism. Compared with before processing, the rationality of the distribution of agents has been significantly improved, and the space utilization has also been optimized to a certain extent.

[0084] The overall processing results of the four dynamic environment groups show that the conflict situation was optimized to varying degrees after model processing, and the distribution of agents was significantly improved. This indicates that although the model cannot completely eliminate all conflicts when facing extreme cases with dynamic and high-density distributions, due to the limitations of current algorithm complexity and the dynamic nature of the environment, it can still effectively optimize the layout of agents and improve the overall stability and operating efficiency of the system.

[0085] In the model system constructed in this study, after a series of complex and precise calculations, the changes in the reward function under different variables or conditions can be seen in [reference needed]. Figure 8 and Figure 9 .in, Figure 8 The changes in the reward function for the static environment group are shown. Figure 9 The changes in the reward function for the dynamic environment group are shown, where α is the smoothing coefficient, a parameter between 0 and 1. The changes in the reward functions of both groups show that although there are relatively frequent fluctuations, both groups exhibit a clear upward trend in the early stages of training. This fully demonstrates that in the early stages of training, through continuous learning and exploration, the model's adaptability to the environment and task rapidly improves, gradually moving towards the optimal strategy, resulting in significant training effectiveness.

[0086] Overall, the rewards remain relatively stable within a certain range. Further comparison of the reward function changes between the two groups reveals that the reward function fluctuations in the static environment group are significantly weaker than those in the dynamic environment group. The reward function value of the static environment group ultimately remains between -1400 and -600, while the reward function value of the dynamic environment group ultimately stabilizes between -8500 and -5000. The reward function value of the static environment group is significantly lower than that of the dynamic environment group. The time loss under the two environments can be seen in Table 4.

[0087] Table 4 Time Consumption in Two Environments

[0088] Table 4 shows that, considering the time consumption of the models in the static and dynamic environments, the time consumption is 1510.76 seconds and 3238.77 seconds, respectively, with average running times per round of 2.5179 seconds and 5.3980 seconds, respectively. This indicates that the model's running efficiency is significantly better in the static environment than in the dynamic environment, demonstrating that constructing a static environment greatly improves the convergence of the reward function and the optimizability of the algorithm.

[0089] Figure 10 In the figure, subplot A4 shows the change in the loss function of the Actor network in the static environment group, and subplot B4 shows the change in the loss function of the Critic network in the static environment group. Figure 11 In the diagram, subplot A5 shows the changes in the loss function of the Actor network in the dynamic environment group, and subplot B5 shows the changes in the loss function of the Critic network in the dynamic environment group.

[0090] from Figure 10 and Figure 11Looking at the changes in the loss function, on the one hand, the fluctuation range of the loss value in the static environment group is lower than that in the dynamic environment group, indicating that the training difficulty of the static environment is lower than that of the dynamic environment. The construction of the static environment group has contributed to reducing the training difficulty in the overall processing of the study. On the other hand, from its fluctuation, it can be seen that although the loss in both environments shows similar changes, the fluctuation of the static environment group is significantly lower than that of the dynamic environment. However, the fluctuation of the loss function in both environments eventually reaches a stable trend. The loss function shows a trend of gradual stabilization from the initial fluctuation to the later stage, indicating that the MADDPG model converges during the training process. The Actor network and Critic network can cooperate with each other and continuously learn to gradually optimize the strategy and value evaluation, proving the strong applicability of deep reinforcement learning to solve the overlay conflict between annotations of point elements.

[0091] In summary, by constructing a static operating environment for the model, the MADDPG model demonstrates significant advantages in handling conflicts between point feature annotations. The convergence results of the reward and loss functions show that the model can effectively adapt to the task scenario through continuous learning. In terms of actual conflict handling performance, the number of conflicts decreased significantly from 40 to 12, and all conflicts in the static environment group were resolved after model processing. Even in the complex situation of some unsolvable conflicts in the dynamic environment group due to excessively high agent distribution density, the model can optimize the agent position distribution as much as possible. Therefore, the MADDPG model not only improves the efficiency of conflict handling between point feature annotations but also provides effective support for the rational layout of point feature annotations.

[0092] The advantages of the method provided in this embodiment are further verified through comparative experiments below. This experiment compares the MapLex annotation configuration based on ArcGIS 10.5 with the reinforcement learning (DRL) method provided in this embodiment. Although MapLex intelligent annotation has the function of supporting the priority of annotation areas, it is impossible to count the annotation direction of the configuration results. Therefore, this experiment only uses the number of conflict-free annotations as the evaluation criterion. The specific comparison results can be found in Table 5.

[0093] Table 5 Comparison results of the two algorithms

[0094] As shown in Table 5, the DRL algorithm produces more conflict-free annotations than the Maplex intelligent annotation system. Judging from the number of point feature annotations located at the positions with the highest weight, the new algorithm achieves higher configuration quality. In this embodiment, the configuration of point feature annotations considers conflicts between symbols and annotations, and between annotations themselves, thus maintaining the aesthetic appeal of the map.

[0095] For the MADDPG model without a static environment and without defined agent selection rules, this embodiment treats all annotations with spatial overlap conflicts as agents and uses the MADDPG model for unified processing and computation. This approach aims to compare the advantages of the improved model in resolving annotation conflicts. During the experiment, the number of training iterations and related parameters of this model remained consistent with the MADDPG model in the improved environment. However, due to the lack of established agent selection rules, the number of agents that this model needs to process increased significantly, from 30 to 40. The specific processing results are detailed in Table 6 below.

[0096] Table 6 Statistical data on conflict resolution results

[0097] As shown in Table 6, the first value in the last row of the total is the subtotal result under the dynamic scenario, and the second value is the sum of the two total values, used to visually present the overall processing effect. After processing by this model, in the context of a globally dynamic operating environment, the process of resolving the spatial overlap problem between annotations triggered a large number of secondary conflicts. In particular, for the high-density distribution of annotations groups 2, 4, 12, and 14, while the model resolved the original conflicts, it also caused new secondary overlap conflicts between these annotations and other annotations, thus the total number of conflicts in these groups was not effectively reduced. From an overall perspective, the total number of conflicts was reduced from 40 to 21 after model processing, but the success rate was only 47.5%, which is far lower than the 72.5% success rate achieved by the method provided in this embodiment. This comparative result indirectly confirms the effectiveness and feasibility of constructing a static operating environment in this embodiment for improving the success rate of annotation conflict resolution.

[0098] On the other hand, a detailed analysis of the time consumption aspect is conducted. The time consumption of the method and the global dynamic scheme provided by this invention is specifically shown in Table 7.

[0099] Table 7 Time Consumption of the Method of the Present Invention and the Global Dynamic Scheme

[0100] As shown in Table 7, under the complex framework of the global dynamic operating environment, the total time consumed by the model operation is extremely considerable, reaching 60042.61 seconds, averaging 100.071 seconds per round. Compared with the method provided by this invention, it is clear that the running time in the global dynamic operating environment far exceeds the time required by the method provided by this invention. Specifically, the total time consumption of the method provided by this invention accounts for only 7.9% of the total time consumption in the global dynamic operating environment, a significant difference. This result fully demonstrates that the static operating environment constructed in this embodiment has significant feasibility and superiority in improving computational efficiency. By fixing some environmental factors, the static operating environment reduces the complex interactions between agents caused by dynamic changes in environmental states, reduces the complexity and uncertainty of system operation, thereby effectively shortening the computation time and improving the overall computational efficiency.

[0101] from Figure 12 The dynamic trend of the reward function clearly shows that, during the same 600 iterations, the model's reward function exhibits a significant convergence. However, there is no ideal positive correlation between the convergence effect of the reward function and the model's processing results. This clearly indicates that the current model's computational results still have considerable room for optimization and have not yet achieved the desired processing effect. In stark contrast, under the same 600-round computation scenario, the method provided by this invention also achieves convergence of its reward function after undergoing the same computational process, and the processing result is superior. This comparison demonstrates that, in a globally dynamic operating environment, existing methods are insufficient to meet the practical requirements of efficient and accurate conflict resolution when dealing with annotation conflict problems, and further highlights the advantages and value of the method provided by this invention in improving processing results.

[0102] Figure 13 In the diagram, subplot A6 illustrates the Actor network, and subplot B6 shows the changes in the loss function of the Critic network. During the continuous iterative computation, the loss rates of both the Actor and Critic networks exhibit a gradual decrease and eventually stabilize. This trend indicates that as the number of computation rounds increases, both networks gradually converge towards a stable operating state by continuously adjusting their internal parameters. From a theoretical deduction perspective, this suggests that the model is evolving towards optimization and approaching the expected ideal performance. However, although the networks have entered a stable operating phase, the conflict problem between annotations has not been ideally resolved. This phenomenon reveals the inherent limitations of the global dynamic environment in handling the conflict problem between annotations.

[0103] In summary, under the same computational rounds, this embodiment demonstrates better processing results and performance advantages in handling annotation problems with spatial overlap conflicts by constructing a MADDPG model with a relatively static operating environment, and also better maintains the positional priority of elements.

[0104] Based on the same general inventive concept, this invention also protects a point feature annotation overlay conflict handling device based on deep reinforcement learning. The point feature annotation overlay conflict handling device based on deep reinforcement learning provided by this invention will be described below. The point feature annotation overlay conflict handling device based on deep reinforcement learning described below can be referred to in correspondence with the point feature annotation overlay conflict handling method based on deep reinforcement learning described above.

[0105] like Figure 14 As shown, the point feature annotation overlay conflict handling device based on deep reinforcement learning provided in this embodiment of the invention specifically includes: The calculation module 210 is used to determine the conflict existence, actual coverage area, positional priority and semantic importance level of each feature annotation in the target area, and to calculate the importance weight score of each feature annotation.

[0106] The partitioning module 220 is used to combine the importance weight score and the number of conflicts corresponding to each overlay conflict event obtained in advance to determine the agent selection strategy, and to partition the target point feature annotations with overlay conflicts into the dynamic conflict dataset and the static conflict dataset.

[0107] Processing module 230 is used to control the target point feature annotations with overlapping conflicts to perform shifting actions until the optimal position based on dynamic conflict datasets and static conflict datasets, with the help of a pre-established MADDPG model and a shift constraint reward strategy.

[0108] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments of the relevant methods, and will not be elaborated further here.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for handling point feature annotation overlay conflicts based on deep reinforcement learning, characterized in that, include: Determine the conflict existence, actual coverage area, location priority, and semantic importance level of each feature annotation within the target area, and calculate the importance weight score of each feature annotation. Combining the importance weight score and the number of conflicts corresponding to each overlay conflict event obtained in advance, the agent selection strategy is determined, and the target point element annotations with overlay conflicts are divided into dynamic conflict datasets and static conflict datasets; Based on the dynamic conflict dataset and the static conflict dataset, and with the help of the pre-established MADDPG model, and through a shift constraint reward strategy, the target point feature annotations with overlapping conflicts are controlled to perform shift actions until the optimal position is reached.

2. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 1, characterized in that, The importance weight score of each feature annotation is calculated, including: Based on the existence of the conflict, determine the overlay conflict score for each element annotation; Based on the actual coverage area, determine the coverage area score for each element annotation; Based on the aforementioned position priority, the overlay position score for each element annotation is determined; Based on the semantic importance level, determine the annotation semantic score for each element annotation; The importance weight score of each element annotation is calculated by weighting and summing the overlay conflict score, the overlay area score, the overlay position score, and the annotation semantic score.

3. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 1, characterized in that, By combining the number of conflicts and importance weight scores corresponding to each overlay conflict event, the agent selection strategy is determined, including: Determine the number of conflicts in each capping conflict event; If the number of conflicts is 2, then the conflict type is determined to be binary. If the number of conflicts is 3, then the conflict type is determined to be a ternary type; If the number of conflicts is greater than 3, then the conflict type is determined to be complex. Based on the conflict type and the importance weight score, an agent selection strategy is determined.

4. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 3, characterized in that, Based on the conflict type and the importance weight score, an agent selection strategy is determined, including: If the conflict type is complex, the agent selection strategy is to use all point elements involved in the overlay conflict in the current overlay conflict event as agents. If the conflict type is binary or ternary, the agent selection strategy is to combine the importance weight score of each point element annotation involved in the current overlay conflict event and select some point element annotations as agents.

5. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 3, characterized in that, The annotations of target point features with overlapping conflicts are divided into dynamic conflict datasets and static conflict datasets, including: If the conflict type is complex, the corresponding target point feature annotations with overlapping conflicts will be assigned to the dynamic conflict dataset. If the conflict type is binary or ternary, the feasibility of the agent selection strategy is determined. If the agent selection strategy is feasible, or feasible after adjustment, the corresponding target point element annotations with overlapping conflicts are assigned to the static conflict dataset. If the agent selection strategy is infeasible and still infeasible after adjustment, the corresponding target point element annotations with overlapping conflicts are assigned to the dynamic conflict dataset.

6. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 1, characterized in that, The MADDPG model includes: An actor network is used to acquire the local state information of each agent at the current moment, and output the action prediction information at the current moment based on the local state information at the current moment. The commentator network is used to obtain global action information of all agents at the current moment, and output the current reward value based on the global action information and the action prediction information at the current moment, so as to evaluate the quality of the actions performed by the agent network. A target actor network is configured to correspond one-to-one with the actor network. The target actor network is used to obtain the local state information of each agent at the next moment, output the action prediction information at the next moment based on the local state information at the next moment, and follow the network parameters of the actor network through a soft update mechanism. The target commentator network is used to obtain global action information of all agents in the next time step, and output the target reward value based on the global action information and the action prediction information in the next time step.

7. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 6, characterized in that, During the model application phase, the MADDPG model guides the agent's independent decision-making solely based on the agent's own state information and the action prediction information output by the actor network and the target actor network.

8. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 1, characterized in that, The process of establishing the shift-constrained reward strategy includes: During the shifting process, if the target point feature annotation does not move beyond the preset map boundary, a non-negative value will be set as the reward value; if the target point feature annotation moves beyond the map boundary, a negative value will be set as the penalty value, thus obtaining the map constraint reward strategy. During the shifting action, if there is no secondary overlap conflict between the target point feature annotation and other point feature annotations, a non-negative value will be set as the reward value. If there is a secondary overlap conflict between the target point feature annotation and other point feature annotations, the intersection area of ​​the secondary overlap conflict will be determined, and the negative number of the intersection area will be used as the penalty value, thus obtaining the first conflict constraint reward strategy. During the shifting action, if there is no secondary overlap conflict between the target point feature annotation and other point features, a non-negative value will be set as the reward value; if there is a secondary overlap conflict between the target point feature annotation and other point features, a negative value will be set as the penalty value, resulting in a secondary conflict constraint reward strategy. During the shifting action, if the target point feature annotation does not move beyond the pre-set buffer area, a non-negative value will be set as the reward value; if the target point feature annotation moves beyond the buffer area, a negative value will be set as the penalty value, thus obtaining the shifting buffer constraint reward strategy. The map sheet constraint reward strategy, the first conflict generation constraint reward strategy, the second conflict generation constraint reward strategy, and the movement buffer constraint reward strategy are used as the shift constraint reward strategy.

9. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 8, characterized in that, The buffer zone is determined through the following process: Determine the set distance between the point feature annotation and the corresponding point feature; Using the point-like feature as the center and the set distance as the radius, determine the initial area of ​​the circle; Based on the initial region, multiple rectangular regions are constructed in multiple predetermined directions starting from the point features; The plurality of rectangular regions are used as buffer areas.

10. A point feature annotation overlay conflict handling device based on deep reinforcement learning, characterized in that, include: The calculation module is used to determine the conflict existence, actual coverage area, location priority and semantic importance level of each feature annotation in the target area, and to calculate the importance weight score of each feature annotation. The partitioning module is used to combine the importance weight score and the number of conflicts corresponding to each overlay conflict event obtained in advance to determine the agent selection strategy, and to partition the target point feature annotations with overlay conflicts into the dynamic conflict dataset and the static conflict dataset. The processing module is used to control the target point feature annotations with overlapping conflicts to perform shifting actions until the optimal position, based on the dynamic conflict dataset and the static conflict dataset, with the help of a pre-established MADDPG model and a shift constraint reward strategy.

Images