Point feature annotation cap conflict processing method and device based on deep reinforcement learning

Abstract

The application belongs to the technical field of data processing, and provides a point element annotation grommet conflict processing method and device based on deep reinforcement learning. The method comprises the following steps: determining the conflict existence, actual grommet area, position priority and semantic importance level of each point element annotation in a target area, and calculating the importance weight score of each point element annotation; combining the importance weight score and the conflict quantity corresponding to each grommet conflict event obtained in advance, determining the agent selection strategy, and dividing the target point element annotation with grommet conflict into dynamic and static conflict data sets; based on the dynamic and static conflict data sets, with the aid of the pre-established MADDPG model, and through the displacement constraint reward strategy, controlling the target point element annotation with grommet conflict to perform displacement action until the optimal position. The scheme can balance the processing efficiency and quality of large-scale conflict scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a point feature annotation cap conflict processing method and device based on deep reinforcement learning. BACKGROUND

[0002] In the field of geographic information systems, cartography and data visualization, point feature annotations, as the core carrier for identifying geographic entity names and attributes, are the key medium for users to obtain spatial information. With the development of technologies such as the Internet of Things and automatic driving high-precision maps, geographic data is growing exponentially, and high-density point feature distribution scenarios such as urban core areas and emergency command maps are becoming increasingly common, leading to increasingly prominent spatial cap conflict problems between point feature annotations.

[0003] Currently, traditional annotation cap conflict processing methods mainly fall into two categories: one is a manual optimization method based on geometric rules, such as the MapLex intelligent labeling system of ArcGIS, which adjusts annotation positions by presetting text avoidance buffer ranges, cap tolerance and other rules, but this method relies on manual rule definition, and in the face of dynamic scaling and multi-source heterogeneous data scenarios, it is prone to local optimization and global suboptimization, and the adjusted cap problem is difficult to fully solve; the other is an automatic processing method based on heuristic algorithms, such as genetic algorithms, simulated annealing algorithms and ant colony algorithms, which can achieve a certain degree of automation, but in high-density data or dynamic environments, there are problems such as low computational efficiency, easy to fall into local optimization, difficult to balance multiple constraints, and inability to guarantee the position stability of high-importance annotations, leading to an imbalance between primary and secondary information in map information transmission.

[0004] Therefore, the current point feature annotation cap conflict processing method has the technical problems of low automation, poor adaptability to complex scenarios and insufficient multi-agent collaborative optimization capability. SUMMARY

[0005] The present application provides a point feature annotation cap conflict processing method and device based on deep reinforcement learning, which solves the defects of the current point feature annotation cap conflict processing method, such as low automation, poor adaptability to complex scenarios and insufficient multi-agent collaborative optimization capability.

[0006] In one aspect, the present application provides a point feature annotation cover conflict processing method based on deep reinforcement learning, comprising: determining the conflict existence, actual cover area, position priority and semantic importance level of each point feature annotation in a target area, and calculating the importance weight score of each point feature annotation; combining the importance weight score and the conflict quantity corresponding to each cover conflict event obtained in advance, determining the agent selection strategy, and dividing the target point feature annotation with cover conflict into a dynamic conflict dataset and a static conflict dataset; based on the dynamic conflict dataset and the static conflict dataset, by means of a pre-established MADDPG model and through a displacement constraint reward strategy, controlling the target point feature annotation with cover conflict to perform displacement action until the optimal position.

[0007] In another aspect, the present application also provides a point feature annotation cover conflict processing device based on deep reinforcement learning, comprising: a calculation module for determining the conflict existence, actual cover area, position priority and semantic importance level of each point feature annotation in a target area, and calculating the importance weight score of each point feature annotation; a division module for combining the importance weight score and the conflict quantity corresponding to each cover conflict event obtained in advance, determining the agent selection strategy, and dividing the target point feature annotation with cover conflict into a dynamic conflict dataset and a static conflict dataset; and a processing module for controlling the target point feature annotation with cover conflict to perform displacement action until the optimal position based on the dynamic conflict dataset and the static conflict dataset, by means of a pre-established MADDPG model and through a displacement constraint reward strategy.

[0008] The point feature annotation cover conflict processing method and device based on deep reinforcement learning provided by the present application realize accurate screening of high-priority conflict annotations by constructing a multi-dimensional importance weight scoring system by fusing conflict existence, actual cover area, position priority and semantic importance level, and guarantee the priority presentation of key geographic information; by determining a specific agent selection strategy and dividing the target point feature annotation with cover conflict into static and dynamic conflict datasets, the model training complexity is reduced and the processing efficiency is improved through the static environment, and the complex multi-annotation conflict scene is adapted through the dynamic environment; relying on the multi-agent collaborative optimization capability of the MADDPG model and the multi-constraint displacement reward strategy, the annotation cover conflict and secondary conflict are effectively reduced under the premise of meeting the mapping specification, the rationality and readability of annotation layout are significantly improved, and the processing efficiency and configuration quality of large-scale conflict scenes can be considered. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to make the technical solutions in the present application or the prior art clearer, the accompanying drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and other accompanying drawings can be obtained by those skilled in the art without any creative effort on the basis of these accompanying drawings.

[0010] Figure 1 is a flowchart of a point element annotation cover conflict processing method based on deep reinforcement learning provided by an embodiment of the present application;

[0011] Figure 2 is a schematic diagram of the principle of processing point element annotation cover conflict by means of a MADDPG model;

[0012] Figure 3 is a layout diagram of setting a cover position score according to a position priority;

[0013] Figure 4 is a schematic diagram of the principle of selecting a point element annotation;

[0014] Figure 5 is a schematic diagram of the principle of selecting a policy by an agent;

[0015] Figure 6 is a schematic diagram of the training principle of a MADDPG model;

[0016] Figure 7 is a schematic diagram of the setting principle of a buffer area;

[0017] Figure 8 is a schematic diagram of the change of a static environment group reward function;

[0018] Figure 9 is a schematic diagram of the change of a dynamic environment group reward function;

[0019] Figure 10 is a schematic diagram of the change of a loss function of an Actor network and a Critic network in a static environment group;

[0020] Figure 11 is a schematic diagram of the change of a loss function of an Actor network and a Critic network in a dynamic environment group;

[0021] Figure 12 is a schematic diagram of the change of a reward function;

[0022] Figure 13 is a schematic diagram of the change of a loss function of an Actor network and a Critic network;

[0023] Figure 14This is a schematic diagram of the structure of the point feature annotation overlay conflict handling device based on deep reinforcement learning provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The following is combined Figures 1 to 14 This invention describes in detail the method and apparatus for handling point feature annotation overlay conflicts based on deep reinforcement learning, as provided in the embodiments of the present invention.

[0026] like Figure 1 As shown in the figure, the point feature annotation overlay conflict handling method based on deep reinforcement learning provided in this embodiment of the invention mainly includes the following steps:

[0027] Step 110: Determine the conflict existence, actual coverage area, location priority, and semantic importance level of each feature annotation within the target area, and calculate the importance weight score of each feature annotation.

[0028] This embodiment uses importance weight scores to detect spatial overlap conflicts between point feature annotations and quantifies the severity of overlap conflicts between different point feature annotations, providing a theoretical basis for prioritizing subsequent conflict resolution. In the context of map point annotations, the calculation of importance weight scores requires comprehensive consideration of multiple influencing factors. This embodiment uses four factors—conflict existence, actual overlap area, location priority, and semantic importance level—as the core influencing elements of the importance weight score. Through the synergistic effect of multiple factors, it objectively reflects the differences in the importance of overlap conflicts between different point feature annotations.

[0029] Step 120: Combine the importance weight score and the number of conflicts corresponding to each overlay conflict event obtained in advance to determine the agent selection strategy, and divide the target point feature annotations with overlay conflicts into dynamic conflict datasets and static conflict datasets.

[0030] This embodiment, through the construction of dynamic and static conflict datasets, can provide accurate data support for subsequent model training and actual conflict handling.

[0031] Step 130: Based on the dynamic conflict data set and the static conflict data set, by means of the pre-established MADDPG model and through the displacement constraint reward strategy, the target point element annotation with existing cap conflict is controlled to perform displacement action until the optimal position.

[0032] It can be understood that by means of the two types of data sets, the MADDPG (Multi-Agent Deep Deterministic Policy Gradient) model and the displacement constraint reward strategy, the point element cap conflict can be efficiently and accurately processed. MADDPG is a multi-agent reinforcement learning algorithm based on DDPG (Deep Deterministic Policy Gradient), which is suitable for multi-agent cooperation or confrontation scenarios.

[0033] In this embodiment, by means of the MADDPG model, the point annotation elements with spatial cap conflict are detected and the priority of the conflict is judged, and the corresponding agent selection strategy is developed to build a relatively static agent operating environment and a corresponding displacement constraint reward strategy to constrain and guide the point element annotation. Then by means of the action guidance of the actor network in the MADDPG model and the evaluation of the actor network action by the critic network, as well as the target actor network and the target critic network, the network parameters are continuously updated through experience replay and loss function calculation to achieve continuous fitting with the actual situation, realize the prediction and guidance of the point element annotation action. Finally, by judging whether the action meets the constraint condition or not, a reasonable point element annotation layout scheme is output, and the spatial cap conflict between point element annotations is solved. The specific process is shown in Figure 2 .

[0034] In an embodiment, in combination with Figure 2 , the importance weight score of each point element annotation is calculated, which specifically includes:

[0035] On the one hand, according to the conflict existence, the cap conflict score of each point element annotation is determined.

[0036] In the expression process of point element annotations, it is inevitable that the annotations will produce mutual cap. The occurrence of cap conflict has a great impact on the clear expression of map information. Whether the cap conflict between point element annotations can be detected is also a basic requirement. For the situation that there is cap conflict between point element annotations, the cap conflict score can be given as 1, representing the occurrence of conflict, and 0 representing no conflict. The specific formula is:

[0037] (1)

[0038] In the formula, is the score of the cover conflict.

[0039] On the other hand, according to the actual cover area, the cover area score of each point element annotation is determined.

[0040] It can be understood that the size of the annotation cover element area intuitively reflects the severity of the cover, and the larger the cover area, the greater the impact on the clear transmission of map information. Generally speaking, when determining the cover area score, the overall layout and visual effect of the map should be considered comprehensively. If the cover element area is large, it means that the point element annotation covers a high degree, which will seriously block the information carried by the point element itself. At this time, a higher score value should be given to make the algorithm more sensitive to large area cover when evaluating the cover conflict, so as to prioritize processing such cases when solving conflicts and minimize the occurrence of large area cover; on the contrary, for the case of small cover element area, the damage to information transmission is relatively limited, and the cover area score can be appropriately reduced. Therefore, the value of the cover area score in this embodiment is not fixed, but is dynamically set according to the actual cover area, as follows:

[0041] (2)

[0042] In the formula, is the cover area score, is the expansion coefficient to avoid the appearance of too small value, which is specified by artificial according to the actual situation, is the actual cover area.

[0043] In another aspect, according to the position priority, the cover position score of each point element annotation is determined.

[0044] It can be understood that the position priority of the point element annotation is directly related to the rationality and readability of the annotation configuration. Specifically, the position priority is usually determined based on the distribution characteristics of map elements, visual cognitive rules, and mapping standards and other factors. For point elements, the basic principle of right first, up second, left third, and down last is generally followed. Therefore, according to the position priority, different cover position scores are set for different positions of the point element annotation, as shown in A1 and B1 subgraphs in the formula. Figure 3

[0045] In another aspect, according to the semantic importance level, the annotation semantic score of each point element annotation is determined.

[0046] ​It can be understood that the point element annotation in the map is the semantic identification of the geographical element, and the importance difference is significant. For example, the priority of point element annotation such as city name, core scenic spot and the like is significantly higher than that of secondary ground annotation. Therefore, the annotation semantic score needs to be differentiated according to the semantic importance level of the point element annotation, so that the point element annotation with high semantic importance can be identified preferentially. In practical application, the determination of the semantic importance level follows the following principles: first, based on the differentiated assignment of the level attribute of the point element annotation, for high-level attribute annotations such as provincial administrative centers and national nature reserves, compared with low-level annotations such as ordinary villages and rural roads, a higher score is assigned to reflect the level difference. Secondly, for non-high-level annotations, such as companies, apartments and buildings, a score lower than that of high-level annotations is assigned. Then, for non-high-level annotations with special value, such as museums and scenic spots, a second highest score value is assigned to ensure their priority in conflict resolution. Finally, ordinary level annotations, such as independent houses without attribute annotations and unnamed roads, are assigned the lowest score value, which is the highest priority for concessions in conflict processing.

[0047] The score assignment system builds the priority logic of annotation conflict resolution through hierarchical semantic importance recognition, ensuring the clear primary and secondary of map information expression and cognitive efficiency. The specific annotation semantic score assignment example is shown in Table 1.

[0048] Table 1 Annotation Semantic Score Assignment Example

[0049]

[0050] Finally, the overlay conflict score, the overlay area score, the overlay position score and the annotation semantic score are weighted and summed to calculate the importance weight score of each point element annotation.

[0051] By independently judging the above four parts of a single point element annotation, the scores calculated by the four parts are weighted and summed to obtain the total importance weight score of the single point element annotation. In this embodiment, the overlay conflict score, the overlay area score, the overlay position score and the annotation semantic score are respectively assigned a weight value of 0.3, 0.25, 0.25 and 0.2 to distinguish their different priority levels, so as to determine the processing priority level of the current element. The specific formula is:

[0052] (3)

[0053] In the formula, is the importance weight score, is the overlay conflict score, is the overlay area score, is the overlay position score, To score the note semantics.

[0054] By performing the above calculation on the note data, the importance degree of whether the note produces a cap, the note cap area, and the note semantics are identified as the main, auxiliary, and secondary importance, respectively, so that the note data with a serious spatial cap conflict and strong semantic level can be comprehensively identified.

[0055] In an embodiment, the agent selection strategy is determined in combination with the conflict number and importance weight score corresponding to each cap conflict event, specifically including:

[0056] First, the number of conflicts in each cap conflict event is determined.

[0057] In one case, if the number of conflicts is 2, the conflict type is determined to be a binary type.

[0058] In another case, if the number of conflicts is 3, the conflict type is determined to be a ternary type.

[0059] In another case, if the number of conflicts is greater than 3, the conflict type is determined to be a complex type.

[0060] Finally, the agent selection strategy is determined according to the conflict type and in combination with the importance weight score.

[0061] In the conflict classification link, all cap conflict events are divided into three categories according to the number of conflicts involved in a single cap conflict event, providing classification basis for subsequent differentiated processing. The binary type is a two-point element note two-two cap conflict. The ternary type is a small-scale cap conflict among three point element notes. The complex type is a large-scale cap conflict among four or more notes.

[0062] In practical applications, for point element notes with spatial cap conflicts, the integrity of the note as a whole should be ensured when moving to avoid content loss and other situations during movement, such as Figure 4 as shown in the A2 subgraph, to ensure the correctness of the note content, therefore, the rectangular area covered by the point element note itself is used as an agent to participate in the action in deep reinforcement learning, such as Figure 4 as shown in the B2 subgraph, and the final moving position of the rectangular area is used as the distribution position of the point element note. At the same time, according to the element note position specified in the mapping specification, the distribution position of the point element note should be located at the positive right side of the point-like element within a certain distance d, such as Figure 4 as shown in the C2 subgraph, the distribution position is used as the initial position of the agent to participate in subsequent actions.

[0063] In an embodiment, the agent selection strategy is determined according to the conflict type and in combination with the importance weight score, specifically including:

[0064] In one case, if the conflict type is a complex type, the agent selection strategy is to select all point feature annotations involved in the current gapping conflict event as the agent.

[0065] In another case, if the conflict type is a binary type or a ternary type, the agent selection strategy is to combine the importance weight scores of each point feature annotation involved in the current gapping conflict event to select part of the point feature annotations as the agent.

[0066] Further, the target point feature annotations with gapping conflicts are divided into dynamic conflict data sets and static conflict data sets, specifically including:

[0067] If the conflict type is a complex type, the target point feature annotations with gapping conflicts are divided into dynamic conflict data sets.

[0068] If the conflict type is a binary type or a ternary type, the feasibility of the agent selection strategy is determined. If the agent selection strategy is feasible or the adjusted agent selection strategy is feasible, the target point feature annotations with gapping conflicts are divided into static conflict data sets. If the agent selection strategy is not feasible and the adjusted agent selection strategy is still not feasible, the target point feature annotations with gapping conflicts are divided into dynamic conflict data sets.

[0069] In this embodiment, for gapping conflict events of complex type, since such conflicts involve a large number of conflicts and complex interaction relationships, even if part of the agents are selected for calculation, the dynamic adjustment of the remaining annotations can still cause frequent changes in the environment state, making it difficult to build a static environment. Therefore, the target point feature annotations with gapping conflicts of this type can be directly classified into dynamic conflict data sets, and the model can be trained according to the dynamic environment logic.

[0070] For the two types of gapping conflict events divided into binary type and ternary type, such gapping conflict events have small size and relatively simple interaction relationships, and have the basis for building a static environment. The processing flow is divided into two steps of determining the agent selection strategy and judging the feasibility of conflict resolution in the execution of the agent selection strategy. For specific processes, please refer to Figure 5 .

[0071] As Figure 5As shown, the selection of the agent is divided into two steps. The first step is the preliminary screening of the agent. The point element annotation is screened in combination with the importance weight score of the point element annotation and the spatial distribution of the conflict, such as the density of the annotation and the size of the surrounding blank area. The point element annotation with a low importance weight score, i.e., a low importance level, is preferentially selected as the agent to participate in the model training. The core purpose is to avoid the high-level annotation from participating in the mobile adjustment, thereby maximizing the correctness of the original position of the high-level annotation.

[0072] The second step is the conflict resolution feasibility judgment and data set division. The low-level annotation screened is used as the agent to judge the feasibility of solving the overlay conflict through action adjustment. If it is judged that the low-level annotation can independently solve the conflict, the target point element annotation with the overlay conflict in the overlay conflict event is classified into the static conflict data set. If it is judged that the low-level annotation cannot solve the conflict, other annotations are selected as agents according to the spatial distribution characteristics of the conflict, such as high-level annotations or point element annotations with a larger position adjustment space. The conflict resolution feasibility is evaluated again. If the new agent can solve the conflict, the target point element annotation with the overlay conflict in the overlay conflict event is still classified into the static conflict data set. If the conflict cannot be solved by a single or a small number of agents after two screenings, all point element annotations in the overlay conflict event are used as agents, and the target point element annotation with the overlay conflict in the overlay conflict event is classified into the dynamic conflict data set. The model solves the strategy through the interaction of multiple agents.

[0073] It can be understood that the MADDPG model drives each agent to autonomously generate an action decision, and the action strategy is limited to mobile adjustment in the eight candidate positions around the annotation. The optimal landing point is selected through continuous iteration, and the algorithm is used to judge whether the annotation is overlaid. If the model can finally match each annotation to a target position that meets the no-overlay constraint, it means that the action strategy of the agent to solve the overlay conflict through action adjustment is feasible, otherwise, it means that it is not feasible.

[0074] In combination Figure 2 It can be seen that the embodiment adopts an action strategy framework based on deterministic policy gradient. In the multi-agent environment, the decision strategy of each agent is realized through deep neural network modeling. The strategy output of each agent not only depends on the current state, but also needs to perceive the behavior information of other agents. In order to enhance the strategy exploration ability, the MADDPG model introduces a time-dependent noise mechanism. This mechanism makes the action selection of the agent not limited to the current policy mapping, but expands the exploration space by superimposing a disturbance term related to historical noise. This time-dependent noise characteristic can effectively guide the agent to find a better decision path while ensuring the continuity of the strategy.

[0075] In the MADDPG model, the actual action of each agent is composed of the action prediction information output by the actor network of the agent and a noise term, and the specific formula is:

[0076] (4)

[0077] In the formula, is the state information of the agent i at time t t, is the state information of the agent i at time t+1, is the deterministic action generated by the actor network according to the current state information, i.e., the action prediction information, is the state of the agent at step t+1, is the parameter of the agent-related network,

[0078] In an embodiment, the MADDPG model specifically comprises:

[0079] An actor network, configured to acquire the local state information of each agent at the current time, and output the action prediction information at the current time according to the local state information at the current time.

[0080] A critic network, configured to acquire the global action information of all agents at the current time, and output the current reward value according to the global action information at the current time and the action prediction information at the current time, the current reward value being used to evaluate the quality of the action performed by the actor network, and specifically, the quality of the action performed by the actor network.

[0081] A target actor network, which is one-to-one corresponding to the actor network, and is configured to acquire the local state information of each agent at the next time, output the action prediction information at the next time according to the local state information at the next time, and follow the network parameters of the actor network through a soft update mechanism.

[0082] A target critic network, configured to acquire the global action information of all agents at the next time, and output the target reward value according to the global action information at the next time and the action prediction information at the next time.

[0083] More preferably, in the model application stage, the MADDPG model only uses the state information of the agent itself, and uses the action prediction information output by the actor network and the target actor network to guide the independent decision of the agent.

[0084] In combination with Figure 2 andFigure 6 As shown, the MADDPG model is based on the Actor-Critic (policy-value) architecture and constructs a network system for multi-agent scenarios. It configures an independent Actor network and Critic network for each agent, and introduces a target network, experience playback, and centralized training and decentralized execution mechanisms to achieve policy optimization and environmental adaptation.

[0085] In terms of network operation logic, the actor network outputs the action prediction information for the current moment based on the agent's current state information. The action generation strategy can be represented as follows: That is, in the state s t Select action a t The strategy involves the following steps: After the action prediction information is input into the commentator network, it calculates the reward value for the current action by combining it with global action information from other agents. The commentator network then evaluates the action chosen by the actor network based on this current reward value, determining its merits. The actor network then optimizes its network parameters based on the commentator network's evaluation. Simultaneously, the target actor network (i.e., the target actor network) calculates the action prediction information for the agent at the next time step. The target critic network (i.e., the target commentator network) further calculates the target reward value using the result from the target actor network. The details are as follows:

[0086] (5)

[0087] In the formula, It is the current moment. t The current reward value, The discount factor determines the importance of future rewards. The target actor network in the next moment t +1 to the target reward value, Represents state-based Calculated expected value, Indicates the first Step's environmental state, express The actions to be performed step by step.

[0088] The critic network optimizes its parameters using gradient descent based on the difference between the current reward value and the target reward value, achieving an accurate estimate of the policy value. This process ultimately updates the parameters of both the target actor network and the critic network. Specifically, during training, the actor network updates its parameters based on the critic network's evaluation results. In this process, the actor network's goal is to update its parameters based on the reward value. and the baseline value V critic ( s t )maximizing the advantage function A ( s t ,a t ), and stabilizing training through a soft update mechanism of the target actor network (i.e. θ actor = τθ online + (1 -τ ) θ actor )wherein, θ online represents the online network parameters of the actor network being currently trained, θ actor represents the old parameters of the target actor network, τ represents the ratio of soft update. The critic network calculates the target Q value through the target critic network, and then updates the parameters by calculating the loss.

[0089] To ensure the stability and efficiency of training, the target actor network and the target critic network are delayed update copies of the current network, and the update of the parameters of the two is realized through a soft update mechanism, as follows:

[0090] (6)

[0091] wherein, is the parameter of the current actor network, is the parameter of the target actor network, is the ratio of soft update, usually taking a value less than 1. The parameters of the target actor network and the target critic network slowly approach the current network through soft update, thereby avoiding instability in the training process.

[0092] Meanwhile, the embodiment introduces an experience replay mechanism, the agent stores each step experience in a buffer, and randomly samples data to update the network during training, breaking the time correlation of experience and improving data diversity. A centralized training and decentralized execution mechanism is adopted, the critic network integrates the state and action information of all agents for joint evaluation in the training phase, ensuring the globality of policy optimization, and each agent only makes independent decisions based on its own observation in the execution phase, adapting to the multi-agent collaborative scenario.

[0093] In summary, MADDPG can realize stable and efficient policy learning in multi-agent environment through the synergistic effect of Actor-Critic alternating optimization, target network soft update, experience replay and centralized training and decentralized execution mechanism, and promote the evolution of the system to the overall optimal decision. The whole training process is as shown in Figure 6 .

[0094] In an embodiment, the establishment process of the displacement constraint reward strategy specifically includes:

[0095] On the one hand, during the execution of the displacement action, if the movement of the target point feature label does not exceed the pre-set map boundary, a non-negative value is set as the reward value, and if the movement of the target point feature label exceeds the map boundary, a negative value is set as the penalty value, to obtain the map constraint reward strategy.

[0096] It can be understood that the map, as the carrier of map data, is the basis of all map data, and therefore the movement range of the point feature label needs to be within the framework of the map to avoid data loss caused by exceeding the map boundary, thereby affecting the correctness of the map data. Therefore, the map boundary constraint is taken as the most basic constraint condition in this embodiment, and if the point feature label moves beyond the map boundary, a penalty value of -5 is given to punish it, and otherwise, a reward value of 0 is given to guide the agent to avoid exceeding the map boundary. The specific formula of the reward value in the map constraint reward strategy is as follows:

[0097] (7)

[0098] On the other hand, during the execution of the displacement action, if there is no secondary overlay conflict between the target point feature label and other point feature labels, a non-negative value is set as the reward value, and if there is a secondary overlay conflict between the target point feature label and other point feature labels, the intersection area of the secondary overlay conflict is determined, and the negative number of the intersection area is set as the penalty value, to obtain the first secondary conflict constraint reward strategy.

[0099] During the execution of the action of the agent, the agent will inevitably produce overlay with other point feature labels, which indicates that the agent has not reached the ideal distribution state, and the occurrence of the overlay conflict is the most serious case. Therefore, the secondary conflict constraint condition is constructed in this embodiment, and this constraint is taken as the most strict constraint to avoid the occurrence of the secondary conflict. If the point feature label produces a secondary overlay conflict with other point feature labels, the intersection area A between the agent and the conflict label is calculated, and -A is taken as the penalty value, and otherwise, 0 is taken as the reward value. The calculation formula of the reward value in the first secondary conflict constraint reward strategy is as follows:

[0100] (8)

[0101] In another aspect, if there is no secondary overlay conflict between the target point feature label and other point features during the execution of the displacement action, a non-negative value is set as the reward value, and if there is a secondary overlay conflict between the target point feature label and other point features, a negative value is set as the penalty value, to obtain a second secondary conflict constraint reward strategy.

[0102] During the execution of the action by the agent, the agent inevitably produces an overlay with other point features, which affects the expression of the map data, making the point feature label unclear and prone to errors in information transmission. Therefore, the embodiment constructs a secondary conflict constraint condition between the point feature label and the point feature to avoid such secondary conflicts. If the point feature label produces a secondary overlay conflict with other point features, a penalty value of -2 is assigned to punish, and otherwise, 0 is set as the reward value. The calculation formula of the reward value in the second secondary conflict constraint reward strategy is as follows:

[0103] (9)

[0104] In another aspect, if the target point feature label does not move beyond the pre-set buffer area during the execution of the displacement action, a non-negative value is set as the reward value, and if the target point feature label moves beyond the buffer area, a negative value is set as the penalty value, to obtain a movement buffer constraint reward strategy.

[0105] In one specific implementation, the buffer area is determined by the following process:

[0106] First, a set distance between the point feature label and the corresponding point feature is determined.

[0107] Then, the initial area of the circle is determined with the point feature as the center and the set distance as the radius.

[0108] Subsequently, a plurality of rectangular areas are constructed in a plurality of set directions with the point feature as the starting point based on the initial area.

[0109] Finally, the plurality of rectangular areas are set as the buffer area.

[0110] According to the mapping specification, the point feature label around the point feature is to be distributed within a set distance range around it. The set distance d can be determined according to the mapping specification, and the final distribution position of the point feature label is also located in this range, as shown in A3 subgraph in Figure 7 As a preferred example, the embodiment constructs eight rectangular areas as buffer areas in eight directions around the point feature, as shown in Figure 7The size of the buffer area and the set distance d from the point feature are determined by the size of the note content and the mapping specification, as shown in the B3 subgraph. In practical applications, the number of rectangular areas can be reasonably set according to actual needs. In this embodiment, 8 is taken as a relatively optimal example for illustration, but is not limited in a specific manner.

[0111] The movement of the agent is limited within the buffer area, so as to achieve the purpose that the distribution of the agent conforms to the mapping specification. If the final action of the agent exceeds the buffer area, -6 is given as a punishment, otherwise, 0 is taken as a reward value. The reward value in the movement buffer constraint reward strategy The calculation formula is as follows:

[0112] (10)

[0113] Finally, the sheet constraint reward strategy, the first conflict constraint reward strategy, the second conflict constraint reward strategy, and the movement buffer constraint reward strategy are taken as the displacement constraint reward strategy.

[0114] In practical applications, only when all behaviors of the agent in the current state strictly meet the positive reward conditions in the above constraint reward strategies can it be determined that it has reached the target state. In this embodiment, the reward values in the above constraint reward strategies are sequentially accumulated, which is taken as a key basis for evaluating the learning effect of the agent, guiding the subsequent learning direction, and optimizing the overall learning performance. The specific formula of the total reward value is as follows:

[0115] (11)

[0116] If the total reward value reaches 0, it indicates that the movement of the agent meets all the constraint conditions, and the distribution scheme is output, otherwise the training and action are continued.

[0117] In order to verify the effect that the above method provided in this embodiment can achieve, the following example illustrates the specific implementation process of the method. The corresponding target area in this example covers 142 point feature notes. First, the spatial overlay conflicts are determined by calculating the importance weight score, the importance of the spatial overlay conflicts between the point feature notes is judged, and the importance of the spatial overlay conflicts between the point feature notes in the target area is sorted. Through detection, it is found that the number of point feature conflicts with spatial overlay conflicts in the target area is 40. The first 10 point feature note information is sorted according to the importance weight score, as shown in Table 2.

[0118] Table 2: First 10 point feature note information

[0119]

[0120] Then the target area of the note data by the foundation extraction, conflict analysis, effect verification logical step processing, first from the target area of the note data extraction point element and point element note, clear point element note spatial distribution state; Then for the extraction of point element note, further extraction of each point element note corresponding rectangular region, point element note space range into quantifiable geometric unit, as the core basis for judging whether there is spatial overlay conflict between point element notes; Subsequently, through the geometric superposition analysis to identify the conflict notes, and highlight the rectangular region of the note with conflict, intuitive presentation of conflict distribution characteristics; Finally, with the note rectangular region of the movement instead of the spatial position adjustment of the note content, through the observation of the spatial distribution state of the adjusted note rectangular region, complete the evaluation of the note conflict resolution effect of the model, and then judge the overall operation effect of the model.

[0121] By extracting the conflict and judging the importance of the note data in the target area, the spatial overlay conflicts between the notes are grouped according to their distribution, and the conflicts within the group are divided into dynamic environment group and static environment group according to whether they can construct static running environment, that is, dynamic conflict data set and static conflict data set, and according to the corresponding agent selection strategy, some point element notes are selected as agents to participate in the later experiment.

[0122] In order to adapt the MADDPG model to the number of agents in the target area and the complexity of action, the parameter system needs to be designed according to the algorithm characteristics and research needs, which can not only ensure the stability of training, but also solve the note conflict problem specifically. The specific parameter setting, design basis and training rules are as follows: First, for the experience data management link, set the experience pool with a capacity of 100000, and set the network parameter update rule when the experience pool is full, to ensure that enough diverse interaction samples can be accumulated in the initial training stage. The capacity of this capacity refers to the number of experience tuples generated by a single interaction of the agent, which can store enough historical experience to break the data time correlation and avoid training shock, and also will not occupy too much memory resources due to the large capacity, providing a reliable data basis for the optimization of Actor network and Critic network.

[0123] Specifically, the sample batch size is 64, which meets the common configuration of small batch stochastic gradient descent in deep reinforcement learning. Secondly, in the network learning rate design, the Actor network learning rate is 0.0005, and the Critic network learning rate is 0.005. Because the Critic network is responsible for evaluating the value of actions, it needs to respond to the value changes in the experience data more quickly to provide accurate optimization signals, so a relatively higher learning rate is set. The Actor network is responsible for generating action strategies, and needs to be iterated slowly to avoid excessive strategy fluctuations, so a lower learning rate is used. The discount factor is set to 0.99, which is close to 1, meaning that the model pays more attention to long-term cumulative rewards rather than short-term local rewards, which is consistent with the task characteristics of multi-agent collaborative adjustment required by the annotation conflict optimization.

[0124] For the balance between exploration and utilization of agents, a noise mechanism is used, with the initial noise value set to 0.9, the decay rate set to 0.91, and the minimum value set to 0.1. In the early stage of training, the noise value is relatively high, and the agent mainly explores to avoid falling into a local optimal solution. As the training progresses, the noise value gradually decreases at a decay rate of 0.91 until it stabilizes at 0.1. At this point, the agent shifts to utilizing the existing optimal strategy and focuses on optimizing the precision of annotation conflict resolution. This noise decay logic is consistent with the training law of multi-agent tasks from exploration scenarios to optimization strategies.

[0125] In terms of training rounds and action rules, considering the complexity of annotation conflicts in the study area, the maximum number of actions per round for an agent is set to 20 to ensure that the agent can complete the optimization of local conflicts within a single round. At the same time, the total number of training rounds is set to 600, and through the iterative cycle of 10 actions per round, the agent continuously updates the strategy from interactive experience and gradually approaches the optimal solution for annotation conflict resolution. Finally, based on the above parameter configuration, the MADDPG model is used to calculate the data in the study area to optimize and solve the conflict between annotations in the study area. The conflict processing results of the static environment group and the dynamic environment group can be seen in Table 3.

[0126] Table 3 Conflict processing results of static environment group and dynamic environment group

[0127]

[0128] From the precise data presented in Table 3, it can be seen that in the last row of Table 3, the first value in the total is the subtotal result under the dynamic scenario, and the second value is the sum of the two total values, which is used to intuitively present the overall processing effect. In the process of processing the conflict problem, the MADDPG model shows significant processing efficiency, and the number of conflicts changes significantly before and after processing, from 40 before processing to 12, which reflects the effectiveness of the model in processing conflicts. In this specific scenario of the static environment group, the original ten conflict problems have been properly solved after processing. This achievement fully demonstrates the strong ability and significant effect of the model in resolving conflicts in a static scenario.

[0129] In the dynamic environment group, after processing by the MADDPG model, the number of conflicts decreased from 17 to 12, although some conflicts remained, but the overall processing trend was good. Even in the face of complex conflict scenarios, the model still relied on its advanced algorithm architecture and flexible adjustment mechanism to optimize and improve the position distribution of the agents as much as possible. Compared with before processing, the rationality of the agent distribution was significantly improved, and the space utilization rate was also optimized to a certain extent.

[0130] From the overall processing results of the four dynamic environment groups, after processing by the model, the conflict situation has been optimized to varying degrees, and the agent distribution has been significantly improved. This shows that although the model cannot completely eliminate all conflicts when faced with dynamic and high-density distribution of extreme situations due to limitations of current algorithm complexity and environmental dynamic change characteristics, it can still effectively optimize the layout of agents and improve the overall stability and efficiency of the system.

[0131] In the model system constructed in this study, after a series of complex and precise calculation processes, the changes in the reward function with different variables or conditions can be seen in Figure 8 and Figure 9 . Among them, Figure 8 shows the changes in the reward function of the static environment group, Figure 9 shows the changes in the reward function of the dynamic environment group, where α is the smoothing coefficient, which is a parameter between 0 and 1. From the changes in the reward function of the two groups, it can be seen that although the reward functions of the two groups have frequent fluctuations, they both show a clear upward trend in the early training period. This fully shows that in the early training period, the model is continuously learning and exploring, and the adaptability to the environment and task is rapidly improving, and is gradually moving towards the optimal strategy, and the training effect is significant.

[0132] From the overall trend, the reward remains in a relatively stable and certain height interval. At the same time, the further comparison of the reward function changes of the two groups found that the reward function fluctuation of the static environment group was significantly weaker than that of the dynamic environment group, and the reward function value of the static environment group was finally maintained between-1400 and-600, while the reward function value of the dynamic environment group was finally stabilized between-8500 and-5000, and the reward function value of the static environment group was significantly lower than that of the dynamic environment group. The time loss in the two environments can be seen in Table 4.

[0133] Table 4 Time loss in two environments

[0134]

[0135] From Table 4, for the time loss of the two environments, the time loss of the static environment group and the dynamic environment group is 1510.76 seconds and 3238.77 seconds, respectively, and the average running time per round is 2.5179 seconds and 5.3980 seconds, respectively. It shows that for the static running environment, the running efficiency of the model is significantly better than that of the dynamic running environment, which shows that the construction of the static environment greatly improves the convergence of the reward function and the optimization of the algorithm.

[0136] Figure 10 In A4, the subgraph shows the loss function change of the Actor network in the static environment group, and B4 shows the loss function change of the Critic network in the static environment group; Figure 11 In A5, the subgraph shows the loss function change of the Actor network in the dynamic environment group, and B5 shows the loss function change of the Critic network in the dynamic environment group.

[0137] From Figure 10 and Figure 11 the loss function change, on the one hand, the loss value fluctuation range of the loss function of the static environment group is lower than that of the dynamic environment group, which shows that the training difficulty of the static environment is lower than that of the dynamic environment, and the construction of the static environment group contributes to reducing the training difficulty in the overall processing. On the other hand, from the fluctuation, although the loss of the two environments shows similar changes, the fluctuation of the static environment group is significantly lower than that of the dynamic environment. But the loss function fluctuation of the two environments finally realizes the stable trend. The loss function presents the trend of fluctuating in the early stage and gradually stabilizing in the later stage, which shows that the MADDPG model is convergent in the training process, and the Actor network and the Critic network can gradually optimize the strategy and value evaluation through mutual cooperation and continuous learning, which proves the strong applicability of deep reinforcement learning to solve the gage conflict between point elements.

[0138] In summary, by constructing a static operating environment for the model, the MADDPG model has shown significant advantages in dealing with the conflict between point feature annotations. From the final convergence results of the reward function and the loss function, the model can effectively adapt to the task scenario through continuous learning. In terms of actual conflict processing results, the number of conflicts has decreased significantly from 40 before processing to 12, and all conflicts in the static environment group have been resolved after model processing. Even in the face of complex situations where some conflicts have no solutions due to the high density of agent distribution in the dynamic environment group, the model can optimize the agent position distribution as much as possible. Therefore, the MADDPG model not only improves the efficiency of conflict processing between point feature annotations, but also provides effective support for the rational layout of point feature annotations.

[0139] The advantages of the above method provided by the embodiment are further verified by a comparative experiment. In this experiment, the MapLex annotation configuration based on ArcGIS 10.5 version is compared with the reinforcement learning method (Deep Reinforcement Learning, DRL) provided by the embodiment. Although the MapLex intelligent annotation has the function of supporting the priority of the annotation area, it cannot count the annotation direction of the configuration result, so this experiment only uses the number of conflict-free annotations as the evaluation standard. The specific comparison results can be seen in Table 5.

[0140] Table 5 Comparison results of two algorithms

[0141]

[0142] As can be seen from Table 5, the number of conflict-free annotations obtained by the DRL algorithm is more than that of the Maplex intelligent annotation system, and the configuration quality of the new algorithm is higher in terms of the number of point feature annotations in the position with the highest weight. When configuring point feature annotations in this embodiment, the conflicts between symbols and annotations, annotations and annotations are considered, and the aesthetics of the map is maintained.

[0143] For the MADDPG model without a static environment and without setting the agent selection rule, this embodiment considers all annotations with spatial overlap conflicts as agents and uses the MADDPG model for unified processing and calculation. Through this way, the advantages of the improved model in solving annotation conflict problems are compared. In the experiment, the number of training iterations and related parameters of this model are consistent with those of the MADDPG model after the environment is improved. However, due to the lack of agent selection rules, the number of agents that this model needs to process has increased significantly from 30 to 40. The specific processing results are shown in Table 6 below.

[0144] Table 6 Statistical data of conflict processing results

[0145]

[0146] As can be seen from Table 6, in the last row of Table 6, the first value in the total column is the subtotal result under the dynamic scenario, and the second value is the sum of the two total values, which is used to intuitively present the overall processing effect. After processing by the model, in the background of the global dynamic running environment, a large number of secondary conflicts are triggered in the solving process of the space overlay problem between the inscriptions. Especially for the high-density distributed 2nd, 4th, 12th and 14th groups of inscriptions, the model not only resolves the original conflicts, but also causes new secondary overlay conflicts between these inscriptions and other inscriptions, thereby making the total amount of conflicts in these groups not effectively reduced. From the overall level, the total number of conflicts is reduced from 40 to 21 after processing by the model, but the success rate of processing is only 47.5%, which is far lower than the success rate of 72.5% achieved by the method provided in the embodiment. This comparison result indirectly proves the effectiveness and feasibility of the static running environment constructed in the embodiment for improving the success rate of inscription conflict processing.

[0147] On the other hand, in terms of time loss, the method provided in the present application and the global dynamic scheme have the time loss as shown in Table 7.

[0148] Table 7 Time loss of the method of the present application and the global dynamic scheme

[0149]

[0150] As can be seen from Table 7, in the complex framework of the global dynamic running environment, the total time consumed by the model running is very considerable, reaching 60042.61 seconds, and the time consumed by each round reaches 100.071 seconds. Compared with the method provided in the present application, it can be obviously found that the running time under the global dynamic running environment is far more than the time required by the method provided in the present application. Specifically, the total time loss of the method provided in the present application only accounts for 7.9% of the total time loss under the global dynamic running environment, and the difference between them is large. This result fully embodies the significant feasibility and superiority of the static running environment constructed in the embodiment for improving the calculation efficiency. The static running environment fixes part of the environmental factors, reduces the complex interaction between agents due to the dynamic change of the environmental state, reduces the complexity and uncertainty of system running, thereby effectively shortening the calculation time and improving the overall calculation efficiency.

[0151] From Figure 12The dynamic change trend of the reward function can clearly show that the reward function of the model presents an obvious convergence trend in the same 600 iteration calculation process. However, the convergence effect of the reward function does not form an ideal positive correlation with the processing result of the model. This fully shows that the calculation result obtained by the current model still has a large optimization space and has not yet reached the ideal processing effect. In sharp contrast, under the same 600 round calculation condition, the reward function of the method provided by the application also realizes convergence after experiencing the same calculation processing process, and the processing result is better. This comparison result shows that under the global dynamic running environment, the processing effect of the existing method in processing the annotation conflict problem is difficult to meet the actual demand of efficiently and accurately solving the conflict, which also further highlights the advantages and values of the method provided by the application in improving the processing effect.

[0152] Figure 13 In the middle, the A6 subgraph shows the case of the Actor network, and the B6 subgraph shows the loss function change of the Critic network. In the continuous iteration calculation process, the loss rates corresponding to the Actor network and the Critic network both present a gradual decline and finally tend to be stable. This change trend shows that with the continuous increase of the calculation rounds, the two networks gradually converge to a stable running state through continuous adjustment of internal parameters, which indicates from the theoretical derivation that the model is evolving towards the optimization direction and approaching the expected ideal performance. However, although the network has entered the stable running stage, the conflict problem between the annotations has not been ideally eliminated, which reveals from the side that the global dynamic environment has inherent limitations in processing the conflict problem between the annotations.

[0153] In summary, under the same calculation round, the MADDPG model of the embodiment constructed based on the relatively static running environment has better processing results and performance advantages for processing the annotation problem with spatial cover conflict, and better maintains the position priority of the elements.

[0154] Based on the same overall inventive concept, the application also protects a point element annotation cover conflict processing device based on deep reinforcement learning. The point element annotation cover conflict processing device based on deep reinforcement learning provided by the application is described as follows, and the point element annotation cover conflict processing device based on deep reinforcement learning described below can be mutually corresponding and referred to the point element annotation cover conflict processing method based on deep reinforcement learning described above.

[0155] As shown in Figure 14 , the point element annotation cover conflict processing device based on deep reinforcement learning provided by the embodiment of the application specifically comprises:

[0156] The computing module 210 is configured to determine the existence of conflicts of each point feature annotation in the target area, the actual cap area, the position priority, and the semantic importance level, and to calculate the importance weight score of each point feature annotation.

[0157] The dividing module 220 is configured to determine the agent selection strategy in combination with the importance weight score and the conflict quantity corresponding to each cap conflict event obtained in advance, and to divide the target point feature annotation with cap conflict into the dynamic conflict dataset and the static conflict dataset.

[0158] The processing module 230 is configured to control the target point feature annotation with cap conflict to perform a shift action until the optimal position based on the dynamic conflict dataset and the static conflict dataset, by means of the pre-established MADDPG model and through the shift constraint reward strategy.

[0159] As to the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments of the method, and will not be described in detail here.

[0160] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for handling point feature annotation overlay conflicts based on deep reinforcement learning, characterized in that, include: Determine the conflict existence, actual coverage area, location priority, and semantic importance level of each feature annotation within the target area. Based on the conflict existence, determine the coverage conflict score of each feature annotation. Based on the actual coverage area, determine the coverage area score for each element annotation; Based on the aforementioned position priority, the overlay position score for each element annotation is determined; Based on the semantic importance level, determine the annotation semantic score for each element annotation; The importance weight score of each element annotation is calculated by weighting and summing the scores of the overlay conflict, overlay area, overlay location, and annotation semantics. Combining the importance weight scores and the pre-obtained number of conflicts corresponding to each capping conflict event, an agent selection strategy is determined, including: determining the number of conflicts in each capping conflict event; if the number of conflicts is 2, the conflict type is determined to be binary; if the number of conflicts is 3, the conflict type is determined to be ternary; if the number of conflicts is greater than 3, the conflict type is determined to be complex; if the conflict type is complex, the agent selection strategy is to use all point element annotations involved in the capping conflict in the current capping conflict event as agents; if the conflict type is binary or ternary, the agent selection strategy is to combine each point element annotation involved in the capping conflict in the current capping conflict event. The importance weight score is used to assign partial point feature annotations as agents; and target point feature annotations with overlapping conflicts are divided into dynamic conflict datasets and static conflict datasets, including: if the conflict type is complex, the corresponding target point feature annotations with overlapping conflicts are divided into dynamic conflict datasets; if the conflict type is binary or ternary, the feasibility of the agent selection strategy is judged. If the agent selection strategy is feasible, or feasible after adjustment, the corresponding target point feature annotations with overlapping conflicts are divided into static conflict datasets; if the agent selection strategy is infeasible and still infeasible after adjustment, the corresponding target point feature annotations with overlapping conflicts are divided into dynamic conflict datasets. Based on the dynamic conflict dataset and the static conflict dataset, and with the help of the pre-established MADDPG model, and through a shift constraint reward strategy, the target point feature annotations with overlapping conflicts are controlled to perform shift actions until the optimal position is reached.

2. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 1, characterized in that, The MADDPG model includes: An actor network is used to acquire the local state information of each agent at the current moment, and output the action prediction information at the current moment based on the local state information at the current moment. The commentator network is used to obtain global action information of all agents at the current moment, and output the current reward value based on the global action information and the action prediction information at the current moment, so as to evaluate the quality of the actions performed by the agent network. A target actor network is configured to correspond one-to-one with the actor network. The target actor network is used to obtain the local state information of each agent at the next moment, output the action prediction information at the next moment based on the local state information at the next moment, and follow the network parameters of the actor network through a soft update mechanism. The target commentator network is used to obtain global action information of all agents in the next time step, and output the target reward value based on the global action information and the action prediction information in the next time step.

3. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 2, characterized in that, During the model application phase, the MADDPG model guides the agent's independent decision-making solely based on the agent's own state information and the action prediction information output by the actor network and the target actor network.

4. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 1, characterized in that, The process of establishing the shift-constrained reward strategy includes: During the shifting process, if the target point feature annotation does not move beyond the preset map boundary, a non-negative value will be set as the reward value; if the target point feature annotation moves beyond the map boundary, a negative value will be set as the penalty value, thus obtaining the map constraint reward strategy. During the shifting action, if there is no secondary overlap conflict between the target point feature annotation and other point feature annotations, a non-negative value will be set as the reward value. If there is a secondary overlap conflict between the target point feature annotation and other point feature annotations, the intersection area of ​​the secondary overlap conflict will be determined, and the negative number of the intersection area will be used as the penalty value, thus obtaining the first conflict constraint reward strategy. During the shifting action, if there is no secondary overlap conflict between the target point feature annotation and other point features, a non-negative value will be set as the reward value; if there is a secondary overlap conflict between the target point feature annotation and other point features, a negative value will be set as the penalty value, resulting in a secondary conflict constraint reward strategy. During the shifting action, if the target point feature annotation does not move beyond the pre-set buffer area, a non-negative value will be set as the reward value; if the target point feature annotation moves beyond the buffer area, a negative value will be set as the penalty value, thus obtaining the shifting buffer constraint reward strategy. The map sheet constraint reward strategy, the first conflict generation constraint reward strategy, the second conflict generation constraint reward strategy, and the movement buffer constraint reward strategy are used as the shift constraint reward strategy.

5. The point feature annotation overlay conflict handling method based on deep reinforcement learning according to claim 4, characterized in that, The buffer zone is determined through the following process: Determine the set distance between the point feature annotation and the corresponding point feature; Using the point-like feature as the center and the set distance as the radius, determine the initial area of ​​the circle; Based on the initial region, multiple rectangular regions are constructed in multiple predetermined directions starting from the point features; The plurality of rectangular regions are used as buffer areas.

6. A point feature annotation overlay conflict handling device based on deep reinforcement learning, characterized in that, include: The calculation module is used to determine the conflict existence, actual coverage area, position priority and semantic importance level of each feature annotation in the target area, and to determine the coverage conflict score of each feature annotation based on the conflict existence. Based on the actual coverage area, determine the coverage area score for each element annotation; Based on the aforementioned position priority, the overlay position score for each element annotation is determined; Based on the semantic importance level, determine the annotation semantic score for each element annotation; calculate the importance weight score for each element annotation by weighting and summing the overlay conflict score, overlay area score, overlay position score, and annotation semantic score. The segmentation module is used to determine the agent selection strategy by combining the importance weight score and the number of conflicts corresponding to each overlay conflict event obtained in advance. This includes: determining the number of conflicts in each overlay conflict event; if the number of conflicts is 2, determining the conflict type as binary; if the number of conflicts is 3, determining the conflict type as ternary; if the number of conflicts is greater than 3, determining the conflict type as complex; if the conflict type is complex, the agent selection strategy is to use all point element annotations involving overlay conflicts in the current overlay conflict event as agents; if the conflict type is binary or ternary, the agent selection strategy is to combine the annotations of each point element involved in overlay conflicts in the current overlay conflict event... The importance weight score is recorded, and some point element annotations are used as agents; target point element annotations with overlapping conflicts are divided into dynamic conflict datasets and static conflict datasets, including: if the conflict type is complex, the corresponding target point element annotations with overlapping conflicts are divided into dynamic conflict datasets; if the conflict type is binary or ternary, the feasibility of the agent selection strategy is judged. If the agent selection strategy is feasible, or feasible after adjustment, the corresponding target point element annotations with overlapping conflicts are divided into static conflict datasets; if the agent selection strategy is infeasible and still infeasible after adjustment, the corresponding target point element annotations with overlapping conflicts are divided into dynamic conflict datasets. The processing module is used to control the target point feature annotations with overlapping conflicts to perform shifting actions until the optimal position, based on the dynamic conflict dataset and the static conflict dataset, with the help of a pre-established MADDPG model and a shift constraint reward strategy.

Images