Underwater detector cluster adaptive detection method and system based on distributed reinforcement learning

Abstract

The present invention discloses an adaptive detection method and system for underwater detector clusters based on distributed reinforcement learning, the method comprising acquiring detector state data and environmental parameter data, obtaining a standardized state matrix through adaptive standardization processing, constructing a group topological feature matrix based on spatial relationship quantification, and generating a final feature representation; constructing an action prototype matrix and an action coupling tensor, optimizing the action space based on constraint conditions, and obtaining a set of feasible actions through multi-layer evaluation screening; constructing a comprehensive reward function based on a multi-dimensional reward mechanism, performing strategy optimization and experience optimization, and obtaining a final optimization strategy; establishing a multi-layer decision-making evaluation system and a real-time monitoring mechanism, constructing an experience knowledge base and continuously optimizing. The present invention improves the collaborative performance and environmental adaptability of underwater detector clusters through multi-level feature extraction, dynamic action optimization, adaptive strategy learning, and knowledge accumulation.

Description

Technical Field

[0001] The present invention belongs to the field of underwater detection, and in particular to an underwater detector cluster adaptive detection method and system based on distributed reinforcement learning. Background Art

[0002] Underwater detector cluster systems have broad application prospects in the fields of marine resource exploration, underwater target search, and marine environment monitoring. Compared with a single detector, a detector cluster can carry out detection tasks simultaneously in a larger range, and has stronger environmental adaptability and task execution efficiency. Especially in complex underwater environments, cluster systems can effectively overcome problems such as limited acoustic channel bandwidth, large communication delays, and low positioning accuracy through multi-point collaborative detection, and significantly improve the reliability and integrity of detection data. Therefore, studying the adaptive detection method of underwater detector clusters based on reinforcement learning has important practical significance for improving underwater detection capabilities and promoting marine development.

[0003] At present, the control of underwater detector clusters mainly adopts rule-based deterministic control methods and behavior-based distributed control methods. The rule-based method guides the movement of the detector through preset control rules, such as the artificial potential field method and the virtual structure method. This type of method is simple to implement but lacks environmental adaptability. The behavior-based method decomposes complex cluster tasks into basic behavior units, such as obstacle avoidance, following, and aggregation, and realizes group control through behavior coordination. It has a certain ability to adapt to the environment but it is difficult to achieve the optimal execution of complex tasks. With the development of reinforcement learning technology, some studies have begun to apply deep reinforcement learning to the control of underwater detectors, learning the optimal control strategy by interacting with the environment, but these methods are mainly aimed at single detectors, and research on group collaborative control is still relatively limited.

[0004] The existing technology has the following specific problems in the control of underwater detector clusters: First, in terms of state representation, the existing methods often use simple state splicing or fixed weight feature combination, which is difficult to accurately characterize the complex spatial relationship and interaction mode between detectors, resulting in insufficient group coordination performance; second, in terms of action space design, most of them use discretized fixed action space, which cannot dynamically adjust the action accuracy and range according to task requirements and environmental changes, limiting the flexibility and accuracy of control; third, in terms of reward design, the existing methods usually use a simple weighted combination method to construct the reward function, which is difficult to balance multiple goals such as detection efficiency, safety and coordination, and the choice of weights is too dependent on experience; finally, in terms of knowledge accumulation and transfer, there is a lack of effective experience extraction and optimization mechanism, and it is impossible to make full use of the experience knowledge accumulated in historical tasks to improve the learning efficiency of new tasks, resulting in the need to re-learn each task, and low adaptability and efficiency. These problems seriously restrict the performance of underwater detector cluster systems in practical applications. Summary of the invention

[0005] The purpose of the invention is to provide an underwater detector cluster adaptive detection method and system based on distributed reinforcement learning to solve the above-mentioned problems existing in the prior art.

[0006] The technical solution is an adaptive detection method for underwater detector clusters based on distributed reinforcement learning, including the following steps:

[0007] S1. Obtain detector feature data, and obtain a standardized state matrix through adaptive standardization processing; construct a spatial relationship quantization matrix based on the standardized state matrix; generate a group topology feature matrix based on the spatial relationship quantization matrix; convert the group topology feature matrix into a probability distribution matrix and calculate the entropy feature vector; perform feature fusion on the standardized state matrix, the group topology feature matrix and the entropy feature vector to obtain the final feature representation; wherein the detector feature data includes the detector basic state data, environmental parameter data and the detector group position data;

[0008] S2. Based on the final feature representation, construct the action prototype matrix and action coupling tensor; based on the action prototype matrix and action coupling tensor, generate a set of constraints; based on the set of constraints, optimize the action coupling tensor to obtain an optimized action space; based on the optimized action space, construct the action evaluation matrix, and obtain a set of feasible actions through a screening algorithm; perform multi-level encoding on the set of feasible actions to generate an encoded action space;

[0009] S3. Based on the final feature representation, the group topology feature matrix and the encoded action space, a basic reward tensor including efficiency reward, safety reward, collaboration reward and information acquisition reward is constructed; based on the basic reward tensor, a dynamic weight adjustment matrix is ​​constructed to generate a comprehensive reward function; based on the comprehensive reward function, strategy optimization is performed to obtain optimized strategy parameters; based on the optimized strategy parameters, a hierarchical experience buffer is constructed and experience optimization is performed to obtain an optimized experience pool; based on the optimized experience pool, multi-strategy integration is performed to obtain the final optimized strategy;

[0010] S4. Based on the final optimization strategy, a decision evaluation tensor including the immediate decision layer, tactical decision layer and strategic decision layer is constructed, and a fusion decision matrix is ​​obtained through multi-level fusion; based on the fusion decision matrix, a real-time monitoring matrix is ​​constructed to perform anomaly detection and processing to obtain adjusted execution parameters; based on the adjusted execution parameters, a multi-dimensional evaluation index system is constructed, and evaluation weights and performance indicators are obtained through an adaptive weight algorithm; based on the adjusted execution parameters, evaluation weights and performance indicators, execution data is obtained, an experience knowledge base is constructed, and an optimized knowledge base and update rules are obtained through a knowledge extraction algorithm.

[0011] The underwater detector cluster adaptive detection system based on distributed reinforcement learning includes:

[0012] at least one processor; and,

[0013] a memory communicatively connected to at least one of the processors; wherein,

[0014] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the underwater detector cluster adaptive detection method based on distributed reinforcement learning.

[0015] Beneficial effects: The present invention realizes accurate characterization of group status and improves the synergistic effect of group behavior; realizes continuous optimization of group behavior and ensures the reliability and efficiency of system operation; realizes intelligent control of the entire process from state perception, action optimization, strategy learning to decision execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flow chart of the method of the present invention.

[0017] Figure 2 This is a flow chart of step S1 of the present invention.

[0018] Figure 3 This is a flow chart of step S2 of the present invention.

[0019] Figure 4 This is a flow chart of step S3 of the present invention.

[0020] Figure 5 This is a flow chart of step S4 of the present invention. DETAILED DESCRIPTION

[0021] like Figure 1 As shown, the present application proposes an underwater detector cluster adaptive detection method based on distributed reinforcement learning, comprising the following steps:

[0022] S1. Obtain detector feature data, and obtain a standardized state matrix through adaptive standardization processing; construct a spatial relationship quantization matrix based on the standardized state matrix; generate a group topology feature matrix based on the spatial relationship quantization matrix; convert the group topology feature matrix into a probability distribution matrix and calculate the entropy feature vector; perform feature fusion on the standardized state matrix, the group topology feature matrix and the entropy feature vector to obtain the final feature representation; wherein the detector feature data includes the detector basic state data, environmental parameter data and the detector group position data;

[0023] S2. Based on the final feature representation, construct the action prototype matrix and action coupling tensor; based on the action prototype matrix and action coupling tensor, generate a set of constraints; based on the set of constraints, optimize the action coupling tensor to obtain an optimized action space; based on the optimized action space, construct the action evaluation matrix, and obtain a set of feasible actions through a screening algorithm; perform multi-level encoding on the set of feasible actions to generate an encoded action space;

[0024] S3. Based on the final feature representation, the group topology feature matrix and the encoded action space, a basic reward tensor including efficiency reward, safety reward, collaboration reward and information acquisition reward is constructed; based on the basic reward tensor, a dynamic weight adjustment matrix is ​​constructed to generate a comprehensive reward function; based on the comprehensive reward function, strategy optimization is performed to obtain optimized strategy parameters; based on the optimized strategy parameters, a hierarchical experience buffer is constructed and experience optimization is performed to obtain an optimized experience pool; based on the optimized experience pool, multi-strategy integration is performed to obtain the final optimized strategy;

[0025] S4. Based on the final optimization strategy, a decision evaluation tensor including the immediate decision layer, tactical decision layer and strategic decision layer is constructed, and a fusion decision matrix is ​​obtained through multi-level fusion; based on the fusion decision matrix, a real-time monitoring matrix is ​​constructed to perform anomaly detection and processing to obtain adjusted execution parameters; based on the adjusted execution parameters, a multi-dimensional evaluation index system is constructed, and evaluation weights and performance indicators are obtained through an adaptive weight algorithm; based on the adjusted execution parameters, evaluation weights and performance indicators, execution data is obtained, an experience knowledge base is constructed, and an optimized knowledge base and update rules are obtained through a knowledge extraction algorithm.

[0026] like Figure 2 As shown, according to one aspect of the present application, step S1 is further:

[0027] S11, obtaining basic state data and environmental parameter data of the detector, and performing normalization processing by calculating the dynamic mean and dynamic standard deviation and combining with a preset exponential decay smoothing factor to obtain a normalized state matrix; wherein the basic state data of the detector includes the three-dimensional spatial coordinates, velocity scalar, horizontal plane deflection angle, vertical plane deflection angle and residual energy level of the detector; and the environmental parameter data includes water pressure, temperature, conductivity and water flow velocity vector;

[0028] S12, obtaining the position data of the detector group, constructing a position matrix and a spatial distance measurement matrix; performing a weighted combination of the spatial distance measurement matrix with the pre-configured angle correlation matrix and the energy potential difference matrix to obtain a spatial relationship quantization matrix; based on the spatial relationship quantization matrix and the position matrix, generating a group topology feature matrix through matrix operations adjusted by dynamic coefficients; wherein the spatial distance measurement matrix is ​​obtained by calculating the spatial distance, velocity difference and energy difference between the detectors;

[0029] S13, inputting the group topology feature matrix into the probability distribution conversion unit, and obtaining the probability distribution matrix through exponential normalization operation; performing weighted calculation on the probability distribution matrix and the pre-configured local environment weight matrix to generate a conditional information entropy matrix; extracting and combining row vectors of the conditional information entropy matrix to form an entropy feature vector;

[0030] S14. Based on the standardized state matrix, the group topology feature matrix and the entropy feature vector, a multidimensional feature tensor is generated through a tensor stacking operation; a compression algorithm is applied to the multidimensional feature tensor to obtain a compressed feature tensor; and a matrix multiplication operation is performed on the compressed feature tensor and a preconfigured adaptive feature selection matrix to obtain a final feature representation.

[0031] In Example 1 of the present application, the original state vector R_i=(x_i, y_i, z_i, vi_i, θ_i, φ_i, e_i) of each detector i (i=1, 2, ..., n) is obtained; wherein x_i, y_i, z_i are three-dimensional space coordinates; vi_i is a velocity scalar; θ_i is a horizontal plane deflection angle; φ_i is a vertical plane deflection angle; e_i is a residual energy level. The local environment parameter vector E_i =(p_i, t_i, c_i, f_i) is obtained; wherein p_i is water pressure; t_i is temperature; c_i is conductivity; and f_i is a water flow velocity vector. Adopt the adaptive extreme value standardization method X'_i=(X_iμ_d) / (σ_d+ε); where X_i is the original data; μ_d is the dynamic mean, calculated by exponential sliding; σ_d is the dynamic standard deviation; ε is the adaptive smoothing factor, ε=k*exp(-|σ_d|), k is the adjustment coefficient, the default value is 0.01; output standardized state matrix S = [S_1, S_2, ..., S_n] T .

[0032] Construct the Spatial Relationship Quantification Matrix (SRQM), SRQM_{ij}=w_d*D_{ij} + w_a * A_{ij} + w_e * E_{ij}; where D_{ij} is the spatial distance metric, D_{ij} = [(x_i x_j) 2 + (y_i y_j) 2 + (z_i z_j) 2 ] 1 / 3* exp(-λ|v_i v_j|); A_{ij} is the angle correlation, A_{ij}=cos(θ_i θ_j)*cos(φ_i φ_j)*(1 |v_i v_j| / v_max); E_{ij} is the energy potential difference; E_{ij}= (e_i + e_j) / (2e_max) * exp(-|e_i e_j| / e_max); w_d, w_a, w_e are adaptive weight coefficients. Construct the Group Topology Feature Matrix (GTFM), GTFM =α* SRQM*P +β*(SRQM*P) 2 ; Where P is the position matrix; α, β are dynamic adjustment coefficients; α=1 / (1 + exp(-||SRQM|| F )); β = 1·α; ||·|| F is the Frobenius norm; output group topology feature matrix GTFM.

[0033] Convert the group topology feature matrix GTFM into a probability distribution matrix P(GTFM)_{ij}= exp(GTFM_{ij}) / ∑∑exp(GTFM_{kl}); calculate the conditional information entropy H(i|j)=-∑P(GTFM)_{ij}*log(P(GTFM)_{ij})*W_{ij}; where W_{ij} is the local environment weight; W_{ij}=exp(-||E_i E_j||2 / τ); τ is the temperature parameter that controls the weight decay speed. Construct the entropy feature vector H_vec=[H(1|·), H(2|·), ..., H(n|·)] and output the entropy feature vector H_vec.

[0034] Construct a multidimensional feature tensor; F = Stack(S, GTFM, H_vec); Apply the tensor compression algorithm F' = TCP(F); TCP is the tensor compression projection, defined as F' = ∑(λ_k * U_k⊕ V_k⊕W_k), k=1,...,r; λ_k is the eigenvalue; U_k, V_k, W_k are basis vectors; r is the compression rank; ⊕ represents the tensor product. Construct the final feature representation Φ=Reshape(F')*Γ; Γ is the adaptive feature selection matrix; Γ_{ij}=sigmoid(ρ*|F'_{ij}|); ρ is the sensitivity parameter; Output the final feature representation Φ.

[0035] This embodiment achieves high-precision characterization of the state of a group of underwater detectors through multi-level feature extraction and fusion strategies. The use of adaptive standardization processing effectively solves the problem of inconsistent data scales of different sensors. Through dynamically adjusted standardization parameters, the processed data has good statistical characteristics; in terms of spatial relationship quantification, the measurement indicators of the three dimensions of distance, angle and energy are combined, and through a dynamic weight adjustment mechanism, the quantification results can accurately reflect the actual interaction relationship between detectors; in the topological feature extraction link, a graph embedding algorithm is used to retain the overall structural characteristics of the group and reflect the local connection pattern; through the introduction of entropy feature vectors, the uncertainty and dynamic change characteristics of group behavior are captured. This embodiment not only improves the accuracy and robustness of state representation, but also provides a reliable data basis for subsequent decision optimization, so that the system can better adapt to the complex and changeable underwater environment.

[0036] According to one aspect of the present application, step S11 is further:

[0037] S111, acquiring basic state data of the detector, and arranging them in time sequence to form a time series array; based on the time series array, calculating the mean deviation of each data through a sliding window to obtain the data fluctuation range; based on the data fluctuation range, using an exponential weighting method to calculate the data reliability weight to obtain a state data reliability matrix; performing a weighted combination of the basic state data of the detector and the state data reliability matrix to generate preprocessed state data;

[0038] S112, obtaining environmental parameter data, removing high-frequency noise by wavelet transform, and obtaining noise-reduced environmental data; applying local mean decomposition to the noise-reduced environmental data, extracting the change trend of the environmental parameters, and obtaining environmental trend data; comparing the noise-reduced environmental data with the environmental trend data, calculating the mutation degree of the environmental parameters, and obtaining an environmental stability matrix;

[0039] S113, based on the pre-processed state data and the environmental stability matrix, a dynamic mean matrix is ​​obtained by exponential sliding calculation; based on the dynamic mean matrix, a fluctuation variance is calculated; the fluctuation variance and the environmental stability matrix are weighted to obtain a dynamic variance matrix; the dynamic mean matrix and the dynamic variance matrix are input into an adaptive parameter generation unit, and a standardized parameter matrix is ​​calculated by recursive optimization;

[0040] S114. Based on the preprocessed state data and the standardized parameter matrix, a mapping function of a dynamic interval is constructed to perform preliminary standardization to obtain initial standardized data; the deviation between the initial standardized data and the pre-stored historical standardized results is calculated, and based on the deviation, the mapping function parameters are dynamically adjusted to obtain an adjustment coefficient matrix; the initial standardized data and the adjustment coefficient matrix are adaptively corrected to obtain a standardized state matrix.

[0041] This embodiment achieves high-quality preprocessing and standardization of detector status data, and improves the reliability of subsequent processing. Through the data reliability assessment mechanism, anomalies and noise in sensor data are effectively identified and processed, improving the quality of data; through trend analysis and mutation detection of environmental parameters, environmental change characteristics are accurately captured, providing an important reference for status assessment; through adaptive calculation of dynamic mean and variance, accurate characterization of data statistical characteristics is achieved, ensuring the adaptability of standardized processing; through adaptive correction mechanism, the stability and reliability of standardized results are ensured. This embodiment not only improves data quality, but also enables the system to better adapt to dynamic changes in data characteristics through multi-level adaptive mechanisms.

[0042] According to one aspect of the present application, step S12 is further:

[0043] S121, obtaining the position data of the detector group and converting it into three-dimensional coordinate data; projecting the three-dimensional coordinate data onto the unit sphere through a spherical projection algorithm to obtain spherical projection data; calculating the great circle distance between the detectors based on the spherical projection data to form an initial distance matrix; combining the initial distance matrix with a preset dynamic weight function to generate a weighted distance matrix according to the distance attenuation characteristic; applying a local sensitive hashing algorithm to the weighted distance matrix to construct a fast distance index to obtain a distance metric matrix;

[0044] S122, obtaining the horizontal deflection angle data and the vertical deflection angle data of the detector, applying the quaternion interpolation algorithm, calculating the angle change rate, and obtaining the angular velocity matrix; obtaining the velocity data of the detector, combining the angular velocity matrix with the velocity data, extracting the motion features through tensor decomposition, and obtaining the motion feature tensor; based on the motion feature tensor, calculating the motion consistency between the detectors, using the cosine similarity algorithm, generating the motion correlation matrix; segmenting the motion correlation matrix through an adaptive threshold, and obtaining the angle correlation matrix;

[0045] S123, obtaining the residual energy data of the detector, predicting the energy consumption trend by piecewise linear regression, and obtaining an energy trend matrix; based on the energy trend matrix, calculating the energy difference between the detectors, and combining with the preconfigured potential field model, generating an initial potential difference matrix; applying an energy flow analysis algorithm to the initial potential difference matrix, calculating the energy transfer efficiency, and obtaining an energy flow matrix; performing weighted fusion of the energy flow matrix and the initial potential difference matrix, and generating an energy potential difference matrix;

[0046] S124, extracting main features through a spectral clustering algorithm based on the distance metric matrix, the angle correlation matrix and the energy potential difference matrix to obtain a feature component matrix; applying a dynamic weight allocation algorithm to the feature component matrix to obtain a weight allocation matrix; performing a tensor product operation on the feature component matrix and the weight allocation matrix to generate a spatial relationship quantization matrix;

[0047] S125. Based on the position data of the detector group, a position matrix is ​​constructed; based on the spatial relationship quantization matrix and the position matrix, a graph embedding algorithm is applied to construct a topological structure to obtain an initial topological map; the edge weights of the initial topological map are optimized, and the key connection relationships are extracted through a recursive minimum spanning tree algorithm to obtain an optimized topological map; the optimized topological map is converted into an adjacency matrix representation, and structural features are extracted through graph convolution operations to obtain a topological feature matrix; the topological feature matrix is ​​feature-fused with the spatial relationship quantization matrix to generate a group topological feature matrix.

[0048] This embodiment achieves accurate characterization of the topological structure of the detector group. Through the spherical projection algorithm, efficient calculation of the three-dimensional spatial position relationship is achieved; through the analysis of angular velocity and motion characteristics, the motion correlation of the detector is accurately characterized; through energy flow analysis and potential field model, accurate evaluation of energy transfer efficiency is achieved; through multi-feature fusion and graph structure optimization, a representation that accurately reflects the topological characteristics of the group is constructed. This embodiment not only improves the calculation efficiency, but also achieves accurate characterization of the group topological structure through comprehensive consideration of multi-dimensional features, providing a reliable basis for subsequent decision optimization.

[0049] like Figure 3 As shown, according to one aspect of the present application, step S2 is further:

[0050] S21, based on the final feature representation, construct an action prototype matrix including a moving operation set and a detecting operation set; based on the action prototype matrix and the pre-stored distance data between detectors, calculate the action conflict detection matrix through the action coupling function; combine the action conflict detection matrix with the distance data between detectors to generate an action coupling tensor;

[0051] S22. Based on the group topology feature matrix, calculate the energy constraint, distance constraint, angle constraint and speed constraint to generate a set of constraint conditions; based on the set of constraint conditions, construct an optimization weight matrix; based on the optimization weight matrix and the action coupling tensor, perform matrix operations to obtain an action optimization matrix; apply an action space contraction algorithm to the action optimization matrix to generate an optimized action space;

[0052] S23. Based on the optimized action space, calculate the efficiency evaluation value, safety evaluation value and coordination evaluation value to generate an action evaluation matrix; input the action evaluation matrix into the action screening unit, calculate the action score and perform probability adjustment, and select the action combination with the highest action score as the feasible action set; based on the feasible action set, perform multi-level encoding to generate the encoded action space.

[0053] In Example 2 of the present application, the final feature representation Φ is obtained, and the action prototype matrix A_base is constructed, A_base={A_m | m∈M}; wherein, M = {movement operation set, detection operation set}; movement operation set = {forward, backward, rise, fall, turn left, turn right}; detection operation set = {sampling, data collection, sensor switch}. Each action prototype is represented as A_m = (Δ_x, Δ_y, Δ_z, Δ_v, Δ_θ, Δ_φ, Δ_e); Δ_* is the change amount of the corresponding state. Construct the action coupling tensor (ACT), ACT_{ijk}=ψ(A_i, A_j, d_k), where ψ is the action coupling function, ψ(A_i, A_j, d_k)=exp(-||A_i A_j||2*d_k / σ)*C_{ij}; where d_k is the distance between detectors; σ is the scale parameter; C_{ij} is the conflict detection matrix; output the action coupling tensor ACT.

[0054] Based on the group topology feature matrix GTFM, constraints are constructed: Constraints={Energy: g_e(A)=e_currente_required(A)≥0; Distance: g_d(A)=d_safe d_min(A)≥0; Angle: g_θ(A)= |θ_change|≤θ_max; Velocity: g_v(A)=|v_change|≤v_max;}, where e_required(A) is the energy required for action A; d_min(A) is the minimum group distance after performing action A; θ_change is the angle change; v_change is the velocity change. Construct the action optimization matrix (AOM), AM =Ω*(Iλ*L)*ACT; where Ω is the optimization weight matrix Ω_ij= exp(-μ * ||Φ_i Φ_j||2); L is the Laplace operator, L = DW; W_ij= exp(-||GTFM_i GTFM_j||_F / η); D is the degree matrix; λ is the smoothing coefficient. Apply the action space shrinkage algorithm A'=Shrink(AOM, Constraints); the shrink operation is defined as A'_i=A_i*(1∏(1 satisfy(g_k))); satisfy(g) is the constraint satisfaction function; output the optimized action space A'.

[0055] Construct the action evaluation matrix (AEM), AEM_ij =ξ_1*E_eff(i, j)+ξ_2*E_safe(i, j)+ξ_3*E_coord(i, j); where E_eff is the efficiency evaluation function, E_eff(i, j)=exp(-T_ij / T_max); E_safe is the safety evaluation function, E_safe(i, j)=1 exp(-d_ij / d_safe); E_coord is the collaborative evaluation function, E_coord(i, j)=cos(v_i, v_j) * min(||v_i||, ||v_j||) / v_max; ξ_* is the dynamic weight coefficient. Apply the action screening algorithm A_feasible = Filter(A', AEM); the Filter operation is defined as: calculate the action score; Score(a) = ∑(AEM_ij*I(a_i, a_j)), I(a_i, a_j) is the action compatibility indicator function; select the k action combinations with the highest scores; apply probability adjustment, P(a) = softmax(Score(a) / τ); output the feasible action set A_feasible.

[0056] Construct a hierarchical action code E(a)=Concat(E_base(a), E_context(a)); where E_base is the base coding function, E_base(a) = MDS(a, A_feasible); MDS is the multidimensional scaling transform; E_context is the context coding function, E_context(a)=Hash(GTFM * a); generate the final action representation A_final={(a, E(a)) |a∈A_feasible}; output the encoded action space A_final.

[0057] This embodiment improves the collaborative performance of the detector group. Through action prototype generation and action coupling analysis, a complete action space including basic actions and combined actions is constructed; through the dynamic generation and propagation of constraints, the adaptive contraction of the action space is achieved, effectively reducing the interference of invalid actions; a multi-level action evaluation mechanism is adopted to comprehensively evaluate the actions from the three dimensions of efficiency, safety and coordination, ensuring the feasibility and effectiveness of the selected actions; through the coding optimization of the action space, the compactness and interpretability of the action representation are improved. This embodiment can dynamically adjust the structure and scope of the action space according to the current environmental status and task requirements. At the same time, through a multi-dimensional evaluation mechanism, it ensures that the selected actions can not only meet the motion constraints of the single detector, but also promote the collaborative detection effect of the group, thereby improving the overall performance of the system.

[0058] According to one aspect of the present application, step S21 is further:

[0059] S211, based on the final feature representation, a basic motion element set is obtained through an action decomposition algorithm; a motion feature extraction algorithm is applied to the basic motion element set, and key motion features are extracted through principal component analysis to generate a motion feature matrix; the motion feature matrix is ​​matched and analyzed with preset physical constraints, and an action feasibility vector is obtained through a feasibility verification algorithm; based on the action feasibility vector, an atomic action set is constructed to generate an action prototype matrix;

[0060] S212, obtaining the relative position data and motion state data of the detectors, applying a dynamic correlation analysis algorithm, calculating the motion coupling relationship between the detectors, and obtaining a motion coupling matrix; performing spectral analysis on the motion coupling matrix, and extracting the coupling mode through a characteristic decomposition algorithm to generate a coupling mode vector; performing spatiotemporal mapping of the coupling mode vector and the preset environmental constraints, and optimizing the coupling relationship through a dynamic programming algorithm to obtain an optimized coupling matrix; constructing an action interaction model based on the optimized coupling matrix, and generating an action interaction tensor;

[0061] S213, based on the action prototype matrix and the action interaction tensor, extract the correlation features between the actions through the tensor decomposition algorithm to obtain the correlation feature matrix; apply the hierarchical clustering algorithm to the correlation feature matrix, and identify the action combination pattern through the similarity calculation method to generate the combination pattern matrix; adaptively match the combination pattern matrix with the preset task execution constraints, and obtain the adaptive scoring vector through the multi-objective optimization algorithm; based on the adaptive scoring vector, screen and optimize to generate the action combination set;

[0062] S214, based on the action combination set, applying the constraint propagation algorithm, analyzing the feasibility of the action sequence, and obtaining a feasibility evaluation matrix; performing probability modeling on the feasibility evaluation matrix, and calculating the state transition probability through the Markov decision process, and generating a transition probability matrix; based on the transition probability matrix, performing a comprehensive evaluation, and obtaining an evaluation result; based on the evaluation result, obtaining an execution cost vector through a dynamic programming algorithm; generating an action constraint matrix based on the execution cost vector and the preset feasibility constraint;

[0063] S215. Based on the action combination set and the action constraint matrix, the action space topology structure is constructed through the tensor network algorithm to obtain the action topology map; the graph embedding algorithm is applied to the action topology map, and the action space features are extracted through the dimensionality reduction method to generate the space feature matrix; the space feature matrix is ​​correlated with the preset action execution conditions to obtain the analysis results; based on the analysis results, the weight allocation vector is obtained through the adaptive weight algorithm; the weight allocation vector, the action combination set and the action constraint matrix are integrated to generate the action coupling tensor.

[0064] This embodiment improves the coordination and feasibility of group actions. Through the action decomposition algorithm, complex actions are accurately decomposed into basic motion units, and the feasibility of atomic actions is ensured through physical constraint verification; through dynamic correlation analysis and coupling pattern extraction, the motion coupling relationship between detectors is accurately identified, and action conflicts are effectively avoided; through multi-level action combination optimization, efficient generation and screening of action sequences are achieved, ensuring the effectiveness of combined actions; through the construction and optimization of the action space topological structure, a complete action coupling representation is formed. This embodiment not only improves the efficiency and quality of action generation, but also effectively solves the conflict problem in the coordinated motion of multiple detectors through multi-dimensional coupling analysis, laying the foundation for achieving efficient group collaborative control.

[0065] According to one aspect of the present application, step S23 is further:

[0066] S231, obtaining each action sequence in the optimized action space, predicting the action execution trajectory through the space-time mapping algorithm, and obtaining a set of predicted trajectories; based on the set of predicted trajectories, calculating the task completion time and energy consumption, and generating a resource consumption matrix using a multi-objective trade-off method; comparing the resource consumption matrix with the preset task time limit and energy constraints, and obtaining an efficiency evaluation matrix through piecewise linear optimization; applying adaptive normalization processing to the efficiency evaluation matrix to generate a standardized efficiency index;

[0067] S232. Obtain obstacle data in the environment, calculate the minimum safety distance based on the obstacle data and the predicted trajectory set through a collision prediction algorithm, and obtain a safety distance matrix; construct a probabilistic risk field based on the safety distance matrix, use a Monte Carlo tree search method to evaluate the collision risk, and generate a risk assessment matrix; combine the risk assessment matrix with the environmental stability matrix, calculate the comprehensive safety factor through Bayesian reasoning, and obtain a safety factor matrix; apply a nonlinear mapping function to the safety factor matrix to generate a standardized safety index;

[0068] S233, obtaining the relative motion data of the detectors in the predicted trajectory set, calculating the motion synchronization through the group consistency evaluation algorithm, and obtaining the synchronization matrix; based on the synchronization matrix and the communication topology between the detectors, using the information entropy analysis method to evaluate the information exchange efficiency, and generating a communication efficiency matrix; inputting the synchronization matrix and the communication efficiency matrix into the pre-configured synergy evaluation model, calculating the synergy effect through deep kernel regression, and obtaining a synergy evaluation matrix; applying adaptive threshold segmentation to the synergy evaluation matrix to generate a standardized synergy index;

[0069] S234. Based on the standardized efficiency index, the standardized safety index and the standardized coordination index, a multi-dimensional feature fusion algorithm is applied to construct an evaluation feature space to obtain a feature space matrix; principal component analysis is applied to the feature space matrix to extract key evaluation dimensions and generate an evaluation dimension matrix; based on the evaluation dimension matrix, a recursive least squares method is used to calculate the weight of each dimension to obtain a weight distribution vector; based on the evaluation dimension matrix and the weight distribution vector, a weighted combination is performed to generate an action evaluation matrix;

[0070] S235. Based on the action evaluation matrix, a set of candidate actions is selected through a random priority sampling algorithm to obtain a candidate action matrix; a feasibility verification algorithm is applied to the actions in the candidate action matrix to generate a feasibility scoring vector; the feasibility scoring vector is combined with the action evaluation matrix, and screened through a multi-objective optimization algorithm to obtain a preferred action set; an action smoothing algorithm is applied to the preferred action set to generate a feasible action set; based on the feasible action set, multi-level encoding is performed to generate an encoded action space.

[0071] This embodiment achieves accurate evaluation and optimal selection of group actions. Through the spatiotemporal mapping algorithm, the action execution trajectory is accurately predicted, and resource utilization is optimized through multi-objective trade-offs; through probabilistic risk field construction and Monte Carlo tree search, accurate evaluation of action safety is achieved; through group consistency evaluation and information entropy analysis, the synergistic effect of actions is accurately quantified; through multi-dimensional feature fusion and adaptive weight adjustment, comprehensive optimization of action evaluation results is achieved. This embodiment not only improves the accuracy of action selection, but also ensures the balance of selected actions in terms of efficiency, safety, and synergy through a multi-level evaluation mechanism, thereby improving the overall effect of group control.

[0072] like Figure 4 As shown, according to one aspect of the present application, step S3 is further:

[0073] S31. Based on the final feature representation, the group topology feature matrix and the encoded action space, the ratio of the position change vector to the energy consumption is calculated to obtain an efficiency reward, the distance between the detector and the obstacle is calculated to obtain a safety reward, the ratio of the trace of the group topology feature matrix to the maximum eigenvalue is calculated to obtain a synergy reward, and the information entropy gain is calculated to obtain an information acquisition reward; the efficiency reward, safety reward, synergy reward and information acquisition reward are combined to form a basic reward tensor; based on the final feature representation, a basic weight matrix is ​​constructed, and a dynamic weight adjustment matrix is ​​calculated; the dynamic weight adjustment matrix and the basic reward tensor are tensor-contracted to obtain a comprehensive reward function;

[0074] S32. Based on the comprehensive reward function, the action value function and the state value function are calculated to construct a policy gradient estimator; the policy gradient estimator is input into the adaptive update unit, and new policy parameters are obtained through normalization and scaling; the policy constraint optimization algorithm is applied to the new policy parameters to obtain optimized policy parameters;

[0075] S33, based on the optimized strategy parameters, construct a local experience buffer, including state, action, reward and next state transfer data; based on the constructed local experience buffer, generate a global experience buffer through experience importance calculation and importance sampling; apply feature extraction, cluster analysis, representative experience selection and knowledge compression algorithm to the global experience buffer to obtain an optimized experience pool;

[0076] S34. Based on the optimized experience pool, a predetermined number of policy networks are constructed; the performance evaluation value and policy entropy of each policy network are calculated to obtain a policy weight matrix; based on the policy weight matrix, the output of each policy network is weightedly combined to obtain an integrated strategy; the KL divergence between the policy networks is calculated, and based on the KL divergence, the weight of the integrated strategy is updated to obtain an updated integrated strategy; the updated integrated strategy is normalized to obtain the final optimized strategy.

[0077] In Example 3 of the present application, the final feature representation Φ, the group topology feature matrix GTFM and the final encoded action space A_final are obtained; the basic reward tensor R_base is constructed, R_base = {R_eff, R_safe, R_coord, R_info}; where R_eff is the efficiency reward, R_eff = exp(-||Δx|| / d_max) * (e_remain / e_init) α ; Δx is the position change vector, d_max is the maximum reference distance, α is the energy consumption weight; R_safe is the safety reward, R_safe=∏(sigmoid((d_i d_safe) / κ)), d_i is the distance to the obstacle; κ is the safety threshold adjustment parameter; R_coord is the collaborative reward, R_coord = tr(GTFM * GTFM T) / (n * λ_max(GTFM)), n is the number of detectors, λ_max is the maximum eigenvalue; R_info is the information acquisition reward, R_info = 1 exp(-H_gain / H_max), H_gain is the information entropy gain, and H_max is the maximum entropy gain reference value. Construct a dynamic weight adjustment matrix (DWAM), DWAM = Θ*(I +μ* G)*Ψ; where Θ is the basic weight matrix; Θ_ij= softmax(||Φ_i Φ_j||2); G is the gradient influence matrix, G = ▽_Φ(R_base); Ψ is the task correlation matrix, Ψ_ij = cos(task_i, task_j); μ is the gradient weight factor; generate a comprehensive reward function R_total=DWAM○R_base; where ○ represents the tensor contraction operation; R_total=∑_ijk(DWAM_ij * R_base_ijk); output the comprehensive reward function R_total.

[0078] Construct a policy gradient estimator (PGE), PGE(θ)=E[▽_θ logπ_θ(a|s) * (Q(s,a) b(s))]; where Q(s,a) is the action value function; Q(s,a)=R_total+γ*max_{a'}(Q(s',a')); b(s) is the baseline function, b(s)=μ_Q(s)+σ_Q(s)*tanh(V(s) / V_max), μ_Q is the mean Q value, σ_Q is the standard deviation of the Q value, and V(s) is the state value function. Apply the adaptive policy update algorithm θ_new = θ+η*Normalize(PGE(θ)); where the Normalize operation is defined as: Normalize(x)= (x μ_x) / (σ_x +ε) * scale; scale is the adaptive scaling factor, scale=min(1, sqrt(Δ / ||x||2)), Δ is the target update amplitude. Construct a policy constraint optimizer (PCO), PCO(θ_new)=argmin_θ||θ θ_new||2; subjectto:KL(π_θ_old || π_θ)≤ε_KL; |E_π_θ[R_total] E_π_θ_old[R_total]|≤ε_R; where KL is the KL divergence constraint; ε_KL is the KL divergence threshold; ε_R is the reward change threshold; output the optimized policy parameters θ_optimal.

[0079] Construct a hierarchical experience buffer (HEB), HEB = {B_local, B_global}; where B_local is the local experience buffer, B_local = {(s_i, a_i, r_i, s'_i)}; B_global is the global experience buffer, B_global = Aggregate(B_local_1, ..., B_local_n); the Aggregate operation is defined as: calculate the experience importance w_i = exp(R_total_i / τ); perform importance sampling P(sample_i) = w_i / ∑w_j. The Experience Distillation Algorithm (EDA) is applied, EDA(HEB) = {feature extraction F=ExtractFeatures(HEB); cluster analysis C= Cluster(F, k); representative experience selection E_rep=SelectRepresentative(C); knowledge compression K=Compress(E_rep)}, where ExtractFeatures is feature extraction based on principal component analysis; Cluster is a density clustering algorithm; SelectRepresentative is representative selection based on information entropy; Compress is a compression algorithm based on maximum information retention; and the output is the optimized experience pool HEB_optimized.

[0080] Construct a multi-policy integrator (MPI), π_ensemble(a|s)=∑w_i*π_i(a|s); where w_i is the policy weight, w_i = softmax(Performance(π_i)); Performance is the policy performance evaluation function, Performance(π) = E[R_total|π]+λ* H(π), H(π) is the policy entropy. Apply policy adaptive adjustment Adjust(π_ensemble) = {calculate policy difference Dij = KL(π_i || π_j); update ensemble weight w_new = w*exp(-η * D); normalize w_final=w_new / ∑w_new}, and output the final optimized policy π_final.

[0081] This embodiment achieves precise guidance and continuous optimization of the behavior of the detector group by constructing a multi-level reward mechanism and strategy optimization framework. In terms of reward design, it integrates the four dimensions of efficiency reward, safety reward, collaborative reward and information acquisition reward, and ensures that the reward signal accurately reflects the current task goal through dynamic weight adjustment; in terms of strategy optimization, a strategy update mechanism based on KL divergence constraints is adopted, which not only ensures the stability of strategy updates, but also maintains sufficient exploration space; in terms of experience management, by constructing a hierarchical experience buffer and combining importance sampling technology, the efficiency and quality of experience playback are improved; through a multi-strategy integration mechanism, the advantages of different strategies are complementary, and the adaptability and robustness of the system are improved. This embodiment not only speeds up the strategy convergence speed, but also effectively avoids local optimality, so that the detector group can maintain efficient and stable detection performance in complex and changeable underwater environments.

[0082] In Example 4 of the present application, in the underwater detector cluster detection, an environment and state space are constructed, which can fully describe all situations that the detector may encounter in a variable underwater environment. The state space (S) contains all underwater environmental parameters that can be sensed by the detector group, such as water temperature, water flow speed, depth, surrounding biological ecology and geological characteristics, etc. For each detector, the state also includes its mechanical state such as position, speed and direction. The state space can be expressed as s t ∈S, where t is the time step.

[0083] Construct the action space A, including direction adjustment {forward, backward, ascend, descend, turn left, turn right}; speed adjustment {accelerate, decelerate}; operation execution {sampling, data collection, turn on / off sensor}. Each action is implemented by a corresponding control command, which can be discrete or continuous, depending on the specific execution capability of the detector. The action can be expressed as a t ∈A.

[0084] Construct a reward function, the reward function R(s t , a t ,s t+1 ) can carefully map the immediate rewards and long-term effects of action execution. The formula is as follows:

[0085] R(s t , a t ,s t+1 )=r data (s t+1 )+r safety (s t )-r cost (a t )+r completion (s t+1 );

[0086] In the formula, r data To distribute rewards according to the quality and importance of collected data, r safety Assign rewards for behaviors that avoid dangerous areas or injuries, r cost Assign penalty points to the cost of using energy and time, r completion Provides additional rewards for mission completion.

[0087] The strategy is trained using DQN and the Actor-Critic method, where DQN uses a neural network to approximate the Q function Q(s, a, θ). Where s is the state, a is the action, and θ is the network parameter. The Q value update formula is as follows:

[0088] Q target =r+γ max a' Q(s', a', θ − );

[0089] Q(s,a,θ)←Q(s,a,θ)+α(Q target -Q(s,a,θ));

[0090] Where r is the reward, γ is the discount factor, usually between 0 and 1, which is used to measure the current value of future rewards, α is the learning rate, and θ − Represents the parameters of the target network, which periodically copies parameters from the master network to increase learning stability.

[0091] The Actor-Critic method combines the policy π(s, π) and the value function V(s, θ v ) is parameterized. The strategy (Actor) here directly outputs the probability distribution of actions, while the value function (Critic) evaluates the expected return of taking an action in a certain state. The Actor's policy gradient update formula is as follows:

[0092] ▽ θπ J(θ π )=E[▽ θπ logπ(a|s,θ π )Q π (s, a)];

[0093] The Critic's temporal difference (TD) update formula is as follows:

[0094] V (s, θ v )←V(s,θ v )+β(r+γV(s',θ v )-V(s,θ v ));

[0095] Where β is the learning rate of Critic.

[0096] According to one aspect of the present application, step S31 is further:

[0097] S311. Based on the final feature representation and encoded action space, obtain the position change data and energy consumption data of the detector, calculate the deviation between the actual path and the ideal path through the trajectory matching algorithm, and obtain the path deviation matrix; input the path deviation matrix and energy consumption data into the multi-objective evaluation model, and generate the efficiency evaluation vector through Pareto optimal analysis; apply adaptive threshold segmentation to the efficiency evaluation vector, combine it with the task completion calculation, and obtain the efficiency reward component; construct a time series correlation analysis based on the efficiency reward component to generate a basic efficiency reward.

[0098] S312. Obtain the distance data and relative speed data between the detector and the environmental obstacles, apply the risk field construction algorithm to generate a dynamic safety area map, and obtain the safety area matrix; perform a hierarchical analysis on the safety area matrix, evaluate the safety level through the fuzzy reasoning system, and generate a safety evaluation matrix; perform a time series comparison between the safety evaluation matrix and the historical safety status data, calculate the safety trend through the dynamic programming algorithm, and obtain the safety trend vector; based on the safety trend vector and the current safety status, apply the nonlinear mapping function to generate the safety reward component.

[0099] S313. Obtain the group topology feature matrix, extract the group structure features through the feature decomposition algorithm, and obtain the structure feature vector; apply information entropy analysis to the structure feature vector, calculate the order of the group configuration, and generate a configuration evaluation matrix; perform matching analysis on the configuration evaluation matrix and the task requirements, and obtain the configuration matching vector through the similarity calculation algorithm; generate the collaborative reward component based on the configuration matching vector and the group collaborative effect evaluation.

[0100] S314. Obtain the data collection records and information quality assessment data of the detector, calculate the validity and completeness of the data through the information value assessment algorithm, and obtain the information quality matrix; apply spatial correlation analysis to the information quality matrix, evaluate the spatial coverage effect of the data, and generate a coverage assessment vector; calculate the matching degree between the coverage assessment vector and the task goal, and obtain the information acquisition score through the weighted integration method; based on the information acquisition score and information timeliness analysis, generate the information reward component.

[0101] S315. Obtain basic efficiency rewards, safety reward components, collaborative reward components and information reward components, construct a multidimensional reward space through a tensor fusion algorithm, and obtain a reward feature tensor; apply principal component analysis to the reward feature tensor to extract key reward dimensions and generate a reward dimension matrix; input the reward dimension matrix into a dynamic weight generator, calculate the weight of each dimension through a recursive optimization algorithm, and obtain a reward weight vector; finally, perform a weighted combination of the reward dimension matrix and the reward weight vector to generate a comprehensive reward function.

[0102] This embodiment improves the training effect of reinforcement learning. Through trajectory matching and multi-target evaluation, the efficiency reward is accurately calculated, accurately reflecting the movement efficiency and energy utilization efficiency; through dynamic risk field construction and fuzzy reasoning, the adaptive calculation of safety rewards is realized, effectively avoiding dangerous states; through group structure feature analysis and information entropy evaluation, the synergy effect and information acquisition effect are accurately quantified; through tensor fusion and dynamic weight optimization, the effective integration of multi-dimensional rewards is achieved. This embodiment not only improves the accuracy and timeliness of the reward signal, but also through the adaptive weight adjustment mechanism, the reward signal can better guide the detector group to develop in the desired behavior direction, accelerating the convergence speed of strategy optimization.

[0103] like Figure 5 As shown, according to one aspect of the present application, step S4 is further:

[0104] S41. Based on the final optimization strategy, calculate the immediate action value to obtain the immediate decision data, calculate the expected reward of multiple time steps to obtain the tactical decision data, calculate the long-term goal completion and resource value to obtain the strategic decision data; combine the immediate decision data, tactical decision data and strategic decision data to form a decision evaluation tensor; apply the attention mechanism to the decision evaluation tensor to calculate the hierarchical weight; perform decision fusion based on the hierarchical weight, and obtain the fusion decision matrix through consistency test;

[0105] S42. Based on the fusion decision matrix, calculate the performance monitoring value, the default detection value and the energy efficiency monitoring value, and generate a real-time monitoring matrix in combination with the distance data between detectors; apply the local outlier factor algorithm to the real-time monitoring matrix to perform anomaly detection and obtain anomaly scores; based on the anomaly scores, use the spectral clustering algorithm to locate the fault, and obtain the adjusted execution parameters through parameter update, task reallocation and communication strategy adjustment;

[0106] S43. Based on the adjusted execution parameters, the execution process data is obtained; based on the execution process data, the task completion time ratio and the energy consumption ratio are calculated to obtain the efficiency index, the failure rate and the mean time between failures are calculated to obtain the robustness index, and the collaborative success rate and the communication graph entropy are calculated to obtain the collaborative index; the efficiency index, the robustness index and the collaborative index are combined to form a multi-dimensional evaluation index system; based on the multi-dimensional evaluation index system, the performance gradient and the index sensitivity are calculated, and the evaluation weight is obtained through gradient updating and normalization processing; based on the evaluation weight, the performance index is calculated;

[0107] S44. Based on the adjusted execution parameters, evaluation weights and performance indicators, obtain the data of successful cases, failed cases and adaptive cases in the execution process to build an experience knowledge base; evaluate the usage frequency and success rate of the data in the experience knowledge base to obtain the knowledge value evaluation results; based on the knowledge value evaluation results, apply maximum entropy coding to compress the knowledge to obtain compressed knowledge data; integrate the compressed knowledge data through a hierarchical knowledge graph, and generate update rules based on association rule mining to finally obtain an optimized knowledge base.

[0108] In Example 5 of the present application, the final feature representation Φ, the final encoded action space A_final, and the final optimization strategy π_final are obtained to construct a decision evaluation tensor (Decision Evaluation Tensor, DET), DET = {D_instant, D_tactical, D_strategic}; wherein D_instant is the instant decision layer, D_instant=π_final(a|s)* Q_immediate(s, a), Q_immediate is the immediate action value evaluation; D_tactical is the tactical decision layer, D_tactical =∑_t(γ t * E[R_total|π_t]), t is the tactical planning time step, γ is the time discount factor; D_strategic is the strategic decision layer, D_strategic =Γ* M *V, Γ is the long-term goal matrix, M is the task completion matrix, and V is the resource value matrix. Construct a multi-level decision fusion algorithm (Multi-level Decision FusionAlgorithm, MDFA), MDFA(DET) = {level weight calculation w_level = SoftAttention(DET), SoftAttention(x) = softmax(V * tanh(Wx + b)); decision fusion D_fusion=∑(w_level_i *DET_i); consistency check C_score = ConsistencyCheck(D_fusion), if C_score < threshold: trigger decision re-evaluation}; output fusion decision matrix D_fusion.

[0109] Construct a real-time monitoring matrix (RMM), RMM = Ξ* (P + V +E), where Ξ is the monitoring weight matrix, Ξ_ij = exp(-d_ij / d_ref) * corr(i, j); P is the performance monitoring matrix, P_ij = Performance(i, j) / Performance_max; V is the breach detection matrix, Vi_ij = 1·Violation(i, j) / Violation_max; E is the energy efficiency monitoring matrix, E_ij = (e_remain_i * e_remain_j) / (e_init_i * e_init_j).

[0110] Apply the anomaly detection and handling (ADH) algorithm, ADH(RMM) = {anomaly detection A_score = DetectAnomaly(RMM), DetectAnomaly is based on the local outlier factor LOF'(p)=avg_k(reach_dist'(p, o) / reach_dist'(o, o')); fault location F_loc=LocalizeFault(A_score), LocalizeFault uses the spectral clustering algorithm; adaptive adjustment Adjust(F_loc) = {update execution parameters, reallocate tasks, adjust communication strategies}}, and output the adjusted execution parameters Params_adjusted.

[0111] Construct a multi-dimensional evaluation metrics system (MEM), MEM={M_efficiency, M_robustness, M_coordination}, where M_efficiency is the efficiency indicator, M_efficiency = (T_actual / T_expected)*(E_consumed / E_budget); M_robustness is the robustness indicator, M_robustness = 1 (N_failures / N_total)*exp(-MTBF / MTBF_ref), MTBF is the mean time between failures, and M_coordination is the coordination indicator, M_coordination=(N_successful_coord / N_total_coord) *(1 entropy(communication_graph)). Construct an adaptive evaluation weights algorithm (AEWA), AEWA(MEM) = {calculate performance gradient ▽P = ΨPerformance / Ψw; estimate sensitivity S = Sensitivity(▽P), Sensitivity(x) = σ(x) / μ(x); update weight w_new = w_old+η*(S*▽P); normalize w_final= Normalize(w_new)}, where Ψ is the partial derivative, and output the evaluation weight w_final and performance indicator scores.

[0112] Construct an experience knowledge base (EKB), EKB = {K_success, K_failure, K_adaptation}; K_success is the successful experience, K_success = {(context, action, outcome, value)}; K_failure is the failed experience, K_failure = {(context, action, error_type, solution)}; K_adaptation is the adaptive experience, K_adaptation = {(context, change, adaptation, effect)}. Apply the knowledge distillation and update algorithm (KDU), KDU(EKB) = {knowledge value assessment V_knowledge = AssessValue(EKB), AssessValue is based on usage frequency and success rate; knowledge compression K_compressed = CompressKnowledge(EKB, V_knowledge), using maximum entropy coding; knowledge integration K_integrated = IntegrateKnowledge(K_compressed), using hierarchical knowledge graph; update rule generation Rules = GenerateRules(K_integrated), based on association rule mining}, and output the optimized knowledge base EKB_optimized and update rules Rules.

[0113] This embodiment improves the task completion quality and reliability of the detector group. Through the multi-level decision-making and evaluation mechanism, comprehensive coverage from immediate decision-making to strategic decision-making is achieved, ensuring the timeliness and foresight of decision-making; through real-time monitoring and exception handling mechanisms, system anomalies can be quickly detected and responded to, improving the system's fault tolerance and stability; by building a multi-dimensional evaluation index system, a comprehensive quantitative evaluation of system performance is achieved, providing a reliable basis for performance optimization; through the construction and optimization of the experience knowledge base, the effective accumulation and utilization of system experience is achieved, promoting the continuous improvement of system performance. This embodiment not only improves the reliability and efficiency of the system, but also realizes the continuous evolution and improvement of system performance through knowledge accumulation and optimization mechanisms.

[0114] In Example 6 of the present application, an exploration and exploitation strategy is implemented. In DQN, the balance between exploration and exploitation is often achieved through the ε-greedy strategy. Under this strategy, the detector selects the best action currently estimated with a probability of 1-ε, and randomly selects actions with a probability of ε. The ε value gradually decreases from a high value (e.g., 0.9) to a low value (e.g., 0.1) to facilitate the transition from extensive exploration in the early stage to focused exploitation in the later stage. If rand()<ε, a random action is selected; if rand()≥ε, argmax a Q(s t ,a,θ), where rand() represents a function that generates random numbers.

[0115] In distributed reinforcement learning, each detector shares its learning experience, such as gradients or parameters, through the network. This can be achieved through a parameter server learning model or federated learning. Among them, the parameter averaging method can be expressed as follows:

[0116] θ global =1 / N∑ i=1 N θ i ;

[0117] where θi is the local model parameter of the i-th detector and N is the total number of detectors involved in learning.

[0118] After the training is completed, each underwater detector will use the trained converged strategy network to obtain its own actions according to its own local observation state, and execute them in a distributed manner, ultimately achieving adaptive detection of the underwater detector cluster.

[0119] According to one aspect of the present application, step S42 is further:

[0120] S421. Based on the fusion decision matrix, the real-time execution status data and expected execution trajectory of the detector are obtained, and the state deviation is calculated through the trajectory deviation analysis algorithm to obtain the state deviation matrix; the time series correlation analysis is applied to the state deviation matrix, and the sliding window algorithm is used to extract the deviation change trend and generate a trend feature vector; the trend feature vector is dynamically compared with the preset performance threshold, and the performance change is predicted through piecewise linear regression to obtain a performance prediction matrix; based on the performance prediction matrix and historical execution data, a performance monitoring matrix is ​​generated through an adaptive weighting method.

[0121] S422. Obtain the task execution data and resource consumption data of the detector, apply the constraint violation detection algorithm to evaluate the constraint satisfaction during the execution process, and obtain the constraint violation vector; perform cluster analysis on the constraint violation vector, identify the default pattern through the density clustering algorithm, and generate the default pattern matrix; perform time series correlation analysis on the default pattern matrix and the historical default data, predict the default risk through the dynamic programming algorithm, and obtain the risk prediction vector; based on the risk prediction vector and the current default status, apply the multi-level evaluation model to generate the default detection matrix.

[0122] S423. Obtain the energy consumption data and task completion data of the detector, calculate the energy utilization rate of the unit task through the energy efficiency evaluation algorithm, and obtain the energy efficiency evaluation vector; apply multi-scale analysis to the energy efficiency evaluation vector, extract the energy efficiency change characteristics through wavelet transform, and generate an energy efficiency feature matrix; match and analyze the energy efficiency feature matrix with the task priority, optimize the energy use strategy through the resource allocation algorithm, and obtain the energy optimization matrix; generate the energy efficiency monitoring matrix based on the energy optimization matrix and energy constraints.

[0123] S424. Obtain a performance monitoring matrix, a default detection matrix, and an energy efficiency monitoring matrix, apply an anomaly detection algorithm to construct a multidimensional anomaly feature space, and obtain an anomaly feature tensor; perform singular value decomposition on the anomaly feature tensor, extract key anomaly indicators through a feature selection algorithm, and generate an anomaly indicator matrix; input the anomaly indicator matrix into an adaptive threshold generator, calculate the dynamic threshold through recursive least squares method, and obtain a threshold adjustment vector; perform an anomaly degree assessment based on the anomaly indicator matrix and the threshold adjustment vector, and generate an anomaly assessment result.

[0124] S425. Obtain the abnormal assessment results and the current system execution parameters, identify the key adjustment parameters through the parameter sensitivity analysis algorithm, and obtain the parameter sensitivity matrix; apply the multi-objective optimization algorithm to the parameter sensitivity matrix, calculate the optimal parameter adjustment plan, and generate the parameter adjustment vector; verify the feasibility of the parameter adjustment vector and the system constraints, and generate the adjustment feasibility matrix through the verification algorithm; finally, based on the adjustment feasibility matrix and the parameter adjustment vector, generate the adjusted execution parameters through the progressive adjustment strategy.

[0125] This embodiment improves the reliability and stability of system operation. Early warning of performance changes is achieved through trajectory deviation analysis and trend prediction; system anomalies and failures are accurately identified through constraint violation detection and pattern recognition; efficient management of energy use is achieved through energy efficiency evaluation and resource optimization; and real-time optimization of the system is achieved through multi-dimensional abnormal feature analysis and adaptive parameter adjustment. This embodiment not only improves the fault tolerance of the system, but also ensures that the system can smoothly transition to the optimal state when an anomaly occurs through a progressive parameter adjustment strategy, thereby maintaining the continuity of the detection task.

[0126] According to one aspect of the present application, step S44 is further:

[0127] S441. Based on the adjusted execution parameters, evaluation weights and performance indicators, the historical task data and execution result data of the detector are obtained, and the key features of successful execution cases are extracted through the time series pattern mining algorithm to obtain the success pattern matrix; association rule analysis is applied to the success pattern matrix, and efficient execution strategies are identified through the frequent pattern mining algorithm to generate strategy feature vectors; the strategy feature vectors are matched and analyzed with the task scenario data, and the scenario fitness matrix is ​​obtained through context similarity calculation; based on the scenario fitness matrix and the execution effect evaluation, a successful experience database is generated.

[0128] S442. Obtain the system's abnormal record data and fault processing data, apply the fault diagnosis algorithm to analyze the cause chain of the failure case, and obtain the failure cause matrix; perform hierarchical decomposition on the failure cause matrix, build a fault propagation network through the causal reasoning algorithm, and generate a fault link diagram; perform correlation analysis on the fault link diagram and the processing plan data, and obtain the solution matrix through the case reasoning algorithm; generate a failure experience database based on the solution matrix and repair effect evaluation.

[0129] S443. Obtain the environmental adaptation data and performance adjustment data of the detector, analyze the adaptive behavior of the system through the environmental feature extraction algorithm, and obtain the adaptive feature matrix; apply dynamic clustering analysis to the adaptive feature matrix, identify the adaptation pattern through the self-organizing map algorithm, and generate an adaptive pattern vector; evaluate the effect of the adaptation pattern vector and the performance improvement data, and obtain the effect evaluation matrix through a multi-dimensional analysis algorithm; generate an adaptation experience database based on the effect evaluation matrix and environmental change characteristics.

[0130] S444. Obtain a database of successful experience, a database of failed experience, and a database of adaptive experience, and construct a unified knowledge representation through a knowledge fusion algorithm to obtain a knowledge feature tensor; apply dimensionality reduction analysis to the knowledge feature tensor, extract core knowledge components through tensor-train decomposition, and generate a knowledge component matrix; conduct value assessment on the knowledge component matrix and usage frequency data, and obtain a knowledge value vector through an importance sampling algorithm; perform knowledge screening and organization based on the knowledge value vector to generate a refined knowledge base.

[0131] S445. Obtain the refined knowledge base and the current system status, build a dynamic update mechanism through the knowledge retrieval algorithm, and obtain the update trigger matrix; apply priority sorting to the update trigger matrix, generate the update sequence through the multi-criteria decision-making algorithm, and obtain the update queue vector; perform feasibility analysis on the update queue vector and the system resource constraints, and obtain the scheduling strategy matrix through the scheduling optimization algorithm; based on the scheduling strategy matrix and the update rule constraints, generate the optimized knowledge base and update rule set through the progressive update strategy.

[0132] This embodiment achieves efficient accumulation and utilization of system experience. Through time series pattern mining and association rule analysis, key experiences from successful cases are extracted; through fault diagnosis and causal reasoning, a complete fault handling knowledge system is constructed; through environmental adaptability analysis and performance evaluation, experience in system adaptability optimization is accumulated; through knowledge fusion and dynamic updating, continuous optimization of the knowledge base is achieved. This embodiment not only improves the efficiency of experience accumulation, but also enables the system to continuously learn and evolve through continuous optimization and updating of knowledge, thereby improving the intelligence level and task completion capability of the detector group.

[0133] According to one aspect of the present application, an underwater detector cluster adaptive detection system based on distributed reinforcement learning includes:

[0134] at least one processor; and,

[0135] a memory communicatively connected to at least one of the processors; wherein,

[0136] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the underwater detector cluster adaptive detection method based on distributed reinforcement learning as described in any one of the above embodiments.

[0137] The present invention constructs a complete adaptive control system for underwater detector groups, realizing intelligent control of the entire process from state perception, action optimization, strategy learning to decision execution. Through multi-level feature extraction and fusion mechanisms, accurate characterization of group states is achieved; through adaptive action space optimization, the synergy of group behavior is improved; through multi-dimensional reward mechanisms and strategy optimization, continuous optimization of group behavior is achieved; through a complete decision execution and evaluation system, the reliability and efficiency of system operation are guaranteed. The present invention not only solves the problem of collaborative control of underwater detector groups in complex environments, but also forms an adaptive, robust and scalable intelligent control framework through the organic combination of various modules, thereby improving the completion quality and efficiency of underwater detection tasks.

[0138] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. An adaptive detection method for underwater detector clusters based on distributed reinforcement learning, characterized in that: The steps include: S1. Obtain detector feature data, obtain a standardized state matrix through adaptive standardization processing; construct a spatial relationship quantization matrix based on the standardized state matrix; generate a group topology feature matrix based on the spatial relationship quantization matrix; Convert the population topology feature matrix into a probability distribution matrix and calculate the entropy feature vector; The standardized state matrix, the group topology feature matrix and the entropy feature vector are fused to obtain the final feature representation; The detector characteristic data includes detector basic status data, environmental parameter data and detector group position data; S2. Based on the final feature representation, construct an action prototype matrix and an action coupling tensor; based on the action prototype matrix and the action coupling tensor, generate a set of constraint conditions; Based on the set of constraints, the action coupling tensor is optimized to obtain the optimized action space; based on the optimized action space, the action evaluation matrix is ​​constructed, and the set of feasible actions is obtained through the screening algorithm; the set of feasible actions is multi-level encoded to generate the encoded action space; S3. Based on the final feature representation, the group topology feature matrix and the encoded action space, a basic reward tensor including efficiency reward, safety reward, collaboration reward and information acquisition reward is constructed; Based on the basic reward tensor, a dynamic weight adjustment matrix is ​​constructed to generate a comprehensive reward function; based on the comprehensive reward function, strategy optimization is performed to obtain optimized strategy parameters; based on the optimized strategy parameters, a hierarchical experience buffer is constructed and experience optimization is performed to obtain an optimized experience pool; Based on the optimized experience pool, multiple strategies are integrated to obtain the final optimization strategy; S4. Based on the final optimization strategy, a decision evaluation tensor including the immediate decision layer, the tactical decision layer and the strategic decision layer is constructed, and a fusion decision matrix is ​​obtained through multi-level fusion; Based on the fusion decision matrix, a real-time monitoring matrix is ​​constructed to detect and process anomalies and obtain the adjusted execution parameters; Based on the adjusted execution parameters, a multi-dimensional evaluation index system is constructed, and the evaluation weights and performance indicators are obtained through an adaptive weight algorithm; Based on the adjusted execution parameters, evaluation weights and performance indicators, the execution data is obtained, the experience knowledge base is constructed, and the optimized knowledge base and update rules are obtained through the knowledge extraction algorithm; Step S1 is further as follows: S11, obtaining basic state data and environmental parameter data of the detector, and performing normalization processing by calculating the dynamic mean and dynamic standard deviation and combining with a preset exponential decay smoothing factor to obtain a normalized state matrix; wherein the basic state data of the detector includes the three-dimensional spatial coordinates, velocity scalar, horizontal plane deflection angle, vertical plane deflection angle and residual energy level of the detector; and the environmental parameter data includes water pressure, temperature, conductivity and water flow velocity vector; S12, obtaining the detector group position data, constructing a position matrix and a spatial distance measurement matrix; performing a weighted combination of the spatial distance measurement matrix with the pre-configured angle correlation matrix and the energy potential difference matrix to obtain a spatial relationship quantization matrix; based on the spatial relationship quantization matrix and the position matrix, generating a group topology feature matrix through matrix operations adjusted by dynamic coefficients; S13, inputting the group topology feature matrix into the probability distribution conversion unit, and obtaining the probability distribution matrix through exponential normalization operation; performing weighted calculation on the probability distribution matrix and the pre-configured local environment weight matrix to generate a conditional information entropy matrix; extracting and combining row vectors of the conditional information entropy matrix to form an entropy feature vector; S14. Based on the standardized state matrix, the group topology feature matrix and the entropy feature vector, a multidimensional feature tensor is generated through a tensor stacking operation; a compression algorithm is applied to the multidimensional feature tensor to obtain a compressed feature tensor; and a matrix multiplication operation is performed on the compressed feature tensor and a preconfigured adaptive feature selection matrix to obtain a final feature representation.

2. The underwater detector cluster adaptive detection method based on distributed reinforcement learning according to claim 1 is characterized in that: Step S2 is further as follows: S21, based on the final feature representation, construct an action prototype matrix including a moving operation set and a detecting operation set; based on the action prototype matrix and the pre-stored distance data between detectors, calculate the action conflict detection matrix through the action coupling function; combine the action conflict detection matrix with the distance data between detectors to generate an action coupling tensor; S22. Based on the group topology feature matrix, calculate the energy constraint, distance constraint, angle constraint and speed constraint, and generate a constraint condition set; Based on the set of constraints, an optimization weight matrix is ​​constructed; Based on the optimized weight matrix and the action coupling tensor, matrix operations are performed to obtain the action optimization matrix; Apply the action space contraction algorithm to the action optimization matrix to generate the optimized action space; S23, based on the optimized action space, calculating the efficiency evaluation value, the safety evaluation value and the coordination evaluation value, and generating an action evaluation matrix; inputting the action evaluation matrix into the action screening unit, calculating the action score and performing probability adjustment, and selecting the action combination with the highest action score as the feasible action set; Based on the set of feasible actions, multi-level encoding is performed to generate the encoded action space.

3. The underwater detector cluster adaptive detection method based on distributed reinforcement learning according to claim 2 is characterized in that: Step S3 is further as follows: S31. Based on the final feature representation, the group topology feature matrix and the encoded action space, the ratio of the position change vector to the energy consumption is calculated to obtain an efficiency reward, the distance between the detector and the obstacle is calculated to obtain a safety reward, the ratio of the trace of the group topology feature matrix to the maximum eigenvalue is calculated to obtain a synergy reward, and the information entropy gain is calculated to obtain an information acquisition reward; the efficiency reward, safety reward, synergy reward and information acquisition reward are combined to form a basic reward tensor; Based on the final feature representation, a basic weight matrix is ​​constructed and a dynamic weight adjustment matrix is ​​calculated; Perform tensor contraction on the dynamic weight adjustment matrix and the basic reward tensor to obtain a comprehensive reward function. S32. Based on the comprehensive reward function, calculate the action value function and the state value function, and construct a policy gradient estimator; The policy gradient estimator is input into the adaptive update unit, and new policy parameters are obtained through normalization and scaling. The policy constraint optimization algorithm is applied to the new policy parameters to obtain the optimized policy parameters. S33, based on the optimized strategy parameters, construct a local experience buffer, including state, action, reward and next state transfer data; based on the constructed local experience buffer, generate a global experience buffer through experience importance calculation and importance sampling; Apply feature extraction, cluster analysis, representative experience selection and knowledge compression algorithms to the global experience buffer to obtain an optimized experience pool; S34. Building a predetermined number of strategy networks based on the optimized experience pool; Calculate the performance evaluation value and policy entropy of each policy network to obtain the policy weight matrix; Based on the strategy weight matrix, the output of each strategy network is weighted and combined to obtain the integrated strategy; The KL divergence between the policy networks is calculated, and based on the KL divergence, the weights of the integrated strategy are updated to obtain the updated integrated strategy; the updated integrated strategy is normalized to obtain the final optimization strategy.

4. The underwater detector cluster adaptive detection method based on distributed reinforcement learning according to claim 3 is characterized in that: Step S4 is further as follows: S41. Based on the final optimization strategy, the immediate action value is calculated to obtain the immediate decision data, the expected rewards of multiple time steps are calculated to obtain the tactical decision data, and the long-term goal completion and resource value are calculated to obtain the strategic decision data; Combine instant decision data, tactical decision data and strategic decision data to form a decision evaluation tensor; Apply the attention mechanism to the decision evaluation tensor and calculate the layer weights; Based on the hierarchical weights, decision fusion is performed, and the fusion decision matrix is ​​obtained through consistency test; S42, based on the fusion decision matrix, calculate the performance monitoring value, the default detection value and the energy efficiency monitoring value, and generate a real-time monitoring matrix in combination with the distance data between detectors; The local outlier factor algorithm is applied to the real-time monitoring matrix to detect anomalies and obtain anomaly scores. Based on the anomaly scores, the spectral clustering algorithm is used to locate faults, and the adjusted execution parameters are obtained through parameter update, task reallocation and communication strategy adjustment. S43, acquiring execution process data based on the adjusted execution parameters; Based on the execution process data, the efficiency index is obtained by calculating the task completion time ratio and energy consumption ratio, the robustness index is obtained by calculating the failure rate and the mean time between failures, and the coordination success rate and the communication graph entropy value are calculated to obtain the coordination index; the efficiency index, robustness index and coordination index are combined to form a multi-dimensional evaluation index system; Based on the multi-dimensional evaluation index system, the performance gradient and index sensitivity are calculated, and the evaluation weight is obtained through gradient updating and normalization. Based on the evaluation weights, the performance indicators are calculated; S44. Based on the adjusted execution parameters, evaluation weights and performance indicators, obtain data on successful cases, failed cases and adaptive cases in the execution process, and build an experience knowledge base; Evaluate the usage frequency and success rate of the data in the experience knowledge base to obtain the knowledge value evaluation results; Based on the knowledge value evaluation results, maximum entropy coding is applied to compress knowledge to obtain compressed knowledge data; The compressed knowledge data is integrated through a hierarchical knowledge graph, and update rules are generated based on association rule mining to finally obtain an optimized knowledge base.

5. The underwater detector cluster adaptive detection method based on distributed reinforcement learning according to claim 4 is characterized in that: Step S11 is further as follows: S111, acquiring basic state data of the detector, and arranging them in time sequence to form a time series array; based on the time series array, calculating the mean deviation of each data through a sliding window to obtain the data fluctuation range; based on the data fluctuation range, using an exponential weighting method to calculate the data reliability weight to obtain a state data reliability matrix; Perform weighted combination of detector basic state data and state data reliability matrix to generate pre-processed state data; S112, obtaining environmental parameter data, removing high-frequency noise through wavelet transform, and obtaining noise-reduced environmental data; applying local mean decomposition to the noise-reduced environmental data, extracting the change trend of the environmental parameters, and obtaining environmental trend data; Compare the noise reduction environment data with the environmental trend data, calculate the mutation degree of environmental parameters, and obtain the environmental stability matrix; S113, based on the preprocessed state data and the environmental stability matrix, a dynamic mean matrix is ​​obtained by exponential sliding calculation; Calculate volatility variance based on the dynamic mean matrix; The volatility variance and environmental stability matrix are weighted to obtain the dynamic variance matrix; The dynamic mean matrix and the dynamic variance matrix are input into the adaptive parameter generation unit, and the standardized parameter matrix is ​​calculated through recursive optimization; S114, constructing a mapping function of a dynamic interval based on the preprocessing state data and the standardized parameter matrix, performing preliminary standardization, and obtaining initial standardized data; Calculate the deviation between the initial standardized data and the pre-stored historical standardized results, and dynamically adjust the mapping function parameters based on the deviation to obtain an adjustment coefficient matrix; adaptively correct the initial standardized data and the adjustment coefficient matrix to obtain a standardized state matrix.

6. The underwater detector cluster adaptive detection method based on distributed reinforcement learning according to claim 4 is characterized in that: Step S12 is further as follows: S121, obtaining the position data of the detector group and converting it into three-dimensional coordinate data; projecting the three-dimensional coordinate data onto a unit sphere through a spherical projection algorithm to obtain spherical projection data; Based on the spherical projection data, the great circle distances between the detectors are calculated to form an initial distance matrix; the initial distance matrix is ​​combined with a preset dynamic weight function to generate a weighted distance matrix according to the distance attenuation characteristics; the local sensitive hashing algorithm is applied to the weighted distance matrix to construct a fast distance index and obtain a distance metric matrix; S122, obtaining the horizontal deflection angle data and the vertical deflection angle data of the detector, applying the quaternion interpolation algorithm, calculating the angle change rate, and obtaining the angular velocity matrix; obtaining the velocity data of the detector, combining the angular velocity matrix with the velocity data, extracting the motion features through tensor decomposition, and obtaining the motion feature tensor; based on the motion feature tensor, calculating the motion consistency between the detectors, using the cosine similarity algorithm, generating the motion correlation matrix; segmenting the motion correlation matrix through an adaptive threshold, and obtaining the angle correlation matrix; S123, obtaining the remaining energy data of the detector, predicting the energy consumption trend by piecewise linear regression, and obtaining an energy trend matrix; Based on the energy trend matrix, the energy difference between detectors is calculated, and combined with the preconfigured potential field model, an initial potential difference matrix is ​​generated; Applying energy flow analysis algorithm to the initial potential difference matrix, calculating energy transfer efficiency, and obtaining energy flow matrix; weighted fusion of energy flow matrix and initial potential difference matrix, generating energy potential difference matrix; S124, extracting main features through a spectral clustering algorithm based on the distance metric matrix, the angle correlation matrix and the energy potential difference matrix to obtain a feature component matrix; applying a dynamic weight allocation algorithm to the feature component matrix to obtain a weight allocation matrix; performing a tensor product operation on the feature component matrix and the weight allocation matrix to generate a spatial relationship quantization matrix; S125. Based on the position data of the detector group, a position matrix is ​​constructed; based on the spatial relationship quantization matrix and the position matrix, a graph embedding algorithm is applied to construct a topological structure to obtain an initial topological map; the edge weights of the initial topological map are optimized, and the key connection relationships are extracted through a recursive minimum spanning tree algorithm to obtain an optimized topological map; the optimized topological map is converted into an adjacency matrix representation, and structural features are extracted through graph convolution operations to obtain a topological feature matrix; the topological feature matrix is ​​feature-fused with the spatial relationship quantization matrix to generate a group topological feature matrix.

7. The underwater detector cluster adaptive detection method based on distributed reinforcement learning according to claim 4 is characterized in that: Step S21 is further as follows: S211, based on the final feature representation, a basic motion element set is obtained through an action decomposition algorithm; a motion feature extraction algorithm is applied to the basic motion element set, and key motion features are extracted through principal component analysis to generate a motion feature matrix; the motion feature matrix is ​​matched and analyzed with preset physical constraints, and an action feasibility vector is obtained through a feasibility verification algorithm; based on the action feasibility vector, an atomic action set is constructed to generate an action prototype matrix; S212, obtaining the relative position data and motion state data of the detectors, applying a dynamic correlation analysis algorithm, calculating the motion coupling relationship between the detectors, and obtaining a motion coupling matrix; performing spectral analysis on the motion coupling matrix, and extracting coupling modes through an eigendecomposition algorithm to generate a coupling mode vector; The coupling mode vector is mapped with the preset environmental constraints in time and space, and the coupling relationship is optimized by the dynamic programming algorithm to obtain the optimized coupling matrix; Based on the optimized coupling matrix, the action interaction model is constructed to generate the action interaction tensor; S213, based on the action prototype matrix and the action interaction tensor, extracting correlation features between actions through a tensor decomposition algorithm to obtain a correlation feature matrix; A hierarchical clustering algorithm is applied to the correlation feature matrix, and the action combination pattern is identified through the similarity calculation method to generate a combination pattern matrix; the combination pattern matrix is ​​adaptively matched with the preset task execution constraints, and an adaptive scoring vector is obtained through a multi-objective optimization algorithm; based on the adaptive scoring vector, screening and optimization are performed to generate an action combination set; S214, based on the action combination set, applying the constraint propagation algorithm, analyzing the feasibility of the action sequence, and obtaining a feasibility evaluation matrix; performing probability modeling on the feasibility evaluation matrix, and calculating the state transition probability through the Markov decision process, and generating a transition probability matrix; Based on the transition probability matrix, a comprehensive evaluation is performed to obtain the evaluation results; Based on the evaluation results, the execution cost vector is obtained through a dynamic programming algorithm; based on the execution cost vector and the preset feasibility constraints, an action constraint matrix is ​​generated; S215, based on the action combination set and the action constraint matrix, constructing the action space topology structure through the tensor network algorithm to obtain the action topology graph; Apply graph embedding algorithm to the action topology graph, extract action space features through dimensionality reduction method, and generate spatial feature matrix; Perform correlation analysis on the spatial feature matrix and the preset action execution conditions to obtain the analysis results; Based on the analysis results, a weight distribution vector is obtained through an adaptive weight algorithm; The weight assignment vector, action combination set and action constraint matrix are integrated to generate the action coupling tensor.

8. The underwater detector cluster adaptive detection method based on distributed reinforcement learning according to claim 5 is characterized in that: Step S23 is further as follows: S231, obtaining each action sequence in the optimized action space, predicting the action execution trajectory through the space-time mapping algorithm, and obtaining a set of predicted trajectories; based on the set of predicted trajectories, calculating the task completion time and energy consumption, and generating a resource consumption matrix using a multi-objective trade-off method; comparing the resource consumption matrix with the preset task time limit and energy constraints, and obtaining an efficiency evaluation matrix through piecewise linear optimization; applying adaptive normalization processing to the efficiency evaluation matrix to generate a standardized efficiency index; S232, obtaining obstacle data in the environment, and calculating the minimum safety distance through a collision prediction algorithm based on the obstacle data and the predicted trajectory set to obtain a safety distance matrix; constructing a probabilistic risk field based on the safety distance matrix, and using a Monte Carlo tree search method to evaluate the collision risk to generate a risk assessment matrix; combining the risk assessment matrix with the environmental stability matrix, and calculating a comprehensive safety factor through Bayesian reasoning to obtain a safety factor matrix; Applying a nonlinear mapping function to the safety factor matrix to generate a standardized safety index; S233, obtaining the relative motion data of the detectors in the predicted trajectory set, calculating the motion synchronization through the group consistency evaluation algorithm, and obtaining the synchronization matrix; based on the synchronization matrix and the communication topology between the detectors, using the information entropy analysis method to evaluate the information exchange efficiency, and generating a communication efficiency matrix; inputting the synchronization matrix and the communication efficiency matrix into the pre-configured synergy evaluation model, calculating the synergy effect through deep kernel regression, and obtaining a synergy evaluation matrix; applying adaptive threshold segmentation to the synergy evaluation matrix to generate a standardized synergy index; S234. Based on the standardized efficiency index, the standardized safety index and the standardized coordination index, a multi-dimensional feature fusion algorithm is applied to construct an evaluation feature space and obtain a feature space matrix; Apply principal component analysis to the feature space matrix to extract key evaluation dimensions and generate an evaluation dimension matrix; based on the evaluation dimension matrix, use recursive least squares method to calculate the weight of each dimension and obtain a weight distribution vector; perform weighted combination based on the evaluation dimension matrix and the weight distribution vector to generate an action evaluation matrix; S235. Based on the action evaluation matrix, a set of candidate actions is selected through a random priority sampling algorithm to obtain a candidate action matrix; a feasibility verification algorithm is applied to the actions in the candidate action matrix to generate a feasibility scoring vector; the feasibility scoring vector is combined with the action evaluation matrix, and screened through a multi-objective optimization algorithm to obtain a preferred action set; an action smoothing algorithm is applied to the preferred action set to generate a feasible action set; based on the feasible action set, multi-level encoding is performed to generate an encoded action space.

9. An underwater detector cluster adaptive detection system based on distributed reinforcement learning, characterized in that: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the underwater detector cluster adaptive detection method based on distributed reinforcement learning as described in any one of claims 1 to 8.

Images