Over-the-horizon single-soldier image transmission relay enhancement device and method for complex terrain

Abstract

The application belongs to the technical field of image communication, and discloses a complex-terrain-oriented beyond-visual-range individual soldier image transmission relay enhancement device and method. The method comprises the following steps: collecting target image data; collecting three-dimensional point cloud data of the environment where the individual soldier is located; dividing the three-dimensional point cloud data to obtain N point cloud data subspaces; extracting features of the N point cloud data subspaces to obtain environment feature data of the N point cloud data subspaces; collecting network operation data of the N point cloud data subspaces; extracting features of the network operation data of the N point cloud data subspaces to obtain network feature data of the N point cloud data subspaces; and adaptively optimizing the transmission path of the individual soldier relay device based on the environment feature data and the network feature data of the N point cloud data subspaces. The transmission path of the individual soldier relay device is adaptively optimized, and the problem of insufficient adaptive capacity of the traditional transmission link in complex terrain is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image communication, and more particularly, to a complex terrain-oriented beyond-visual-range individual soldier image transmission relay enhancement device and method. BACKGROUND

[0002] The existing beyond-visual-range image transmission technology has many deficiencies when facing complex terrain. In complex terrain, the traditional transmission link design lacks the ability to adaptively adjust according to the terrain and signal real-time conditions. When encountering sudden situations such as terrain undulations and signal interference, it is difficult to quickly and flexibly optimize the transmission path and parameters, which results in poor signal transmission stability, interruption or delay in actual application, affects the timeliness of information, and further leads to signal strength attenuation, making it difficult to meet the demand for long-distance and high-quality image transmission, and restricting the wide application and performance of individual soldier image transmission relay technology in complex environments.

[0003] Therefore, there is an urgent need for an optimization scheme that adaptively adjusts the transmission link in real time according to complex terrain and signal changes to meet the urgent demand for long-distance and high-quality image transmission in military operations and complex environment operations, and to ensure efficient and stable image transmission in various complex terrains. SUMMARY

[0004] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present application provides the following technical scheme: a complex terrain-oriented beyond-visual-range individual soldier image transmission relay enhancement method, comprising:

[0005] Collecting target image data;

[0006] Collecting three-dimensional point cloud data of the environment where the individual soldier is located;

[0007] Dividing the three-dimensional point cloud data to obtain N point cloud data subspaces;

[0008] Extracting features from the N point cloud data subspaces to obtain environmental feature data of the N point cloud data subspaces;

[0009] Collecting network operation data of the N point cloud data subspaces;

[0010] Extracting features from the network operation data of the N point cloud data subspaces to obtain network feature data of the N point cloud data subspaces;

[0011] Adaptively optimizing the transmission path of the individual soldier relay device based on the environmental feature data and network feature data of the N point cloud data subspaces.

[0012] Further, the method for adaptively optimizing the transmission path of the individual soldier relay device comprises:

[0013] The environment feature data and the network feature data of the N point cloud data subspaces are input into an fitness evaluation model, and fitness evaluation scores corresponding to the N point cloud data subspaces are obtained.

[0014] The point cloud data subspaces with the fitness evaluation scores greater than or equal to a preset fitness evaluation score threshold are constructed into a set of to-be-planned transmission nodes.

[0015] The to-be-planned transmission nodes in the set of to-be-planned transmission nodes are connected to each other to construct G transmission path subsegments, G being an integer greater than zero.

[0016] For each transmission path subsegment, a corresponding transmission path subsegment weight is calculated according to the environment feature data and the network feature data of the start point and the end point of the transmission path subsegment; and the G transmission path subsegment weights are constructed into a set of transmission path subsegment weights.

[0017] Based on the set of transmission path subsegment weights, the G transmission path subsegments are optimized to obtain an optimal transmission path; and the single-soldier relay device transmits the target image data according to the optimal transmission path.

[0018] Further, the method for obtaining the optimal transmission path comprises:

[0019] The number of preset transmission path subsegments is R, R being an integer greater than zero; and the optimal transmission path is initialized as empty.

[0020] The transmission path subsegment with the largest transmission path subsegment weight is selected from the set of transmission path subsegment weights, and is denoted as a to-be-planned transmission path subsegment; and the to-be-planned transmission path subsegment is removed from the set of transmission path subsegment weights.

[0021] The to-be-planned transmission path subsegment is added to the optimal transmission path; it is judged whether there is a loop in the optimal transmission path; if there is a loop in the optimal transmission path, the to-be-planned transmission path subsegment is removed from the optimal transmission path; and if there is no loop in the optimal transmission path, the to-be-planned transmission path subsegment is kept in the optimal transmission path.

[0022] The above process is repeatedly executed until the number of transmission path subsegments in the optimal transmission path reaches R.

[0023] Further, the method for judging whether there is a loop in the optimal transmission path comprises:

[0024] A transmission path subsegment is randomly selected from the optimal transmission path as a start point, and the selected transmission path subsegment is traversed; if the traversal process returns to the starting position and no transmission path subsegment is repeatedly passed through in the entire path, there is a loop in the optimal transmission path.

[0025] Further, the method for obtaining the N point cloud data subspaces comprises:

[0026] Step one: divide the three-dimensional point cloud data according to the preset sliding step, obtain N center data points of the three-dimensional point cloud data; let the initial value of n be 1, and the value range of n be 1 to N;

[0027] Step two: obtain the nth center data point, select H point cloud data points adjacent to the nth center data point from the three-dimensional point cloud data; construct the nth point cloud data subspace by the nth center data point and the H point cloud data points;

[0028] Step three: let n=n+1, if n is less than or equal to N, continue to execute step two, if n is greater than N, end the current process, and obtain N point cloud data subspaces.

[0029] Further, the method for obtaining the environmental feature data of the N point cloud data subspaces comprises:

[0030] S100: let the initial value of n be 1, and the value range of n be 1 to N;

[0031] S101: based on the three-dimensional coordinates of the center data point and the corresponding H point cloud data points in the nth point cloud data subspace, the environmental slope and the environmental surface roughness of the nth point cloud data subspace are calculated; based on the incident laser intensity and the reflected laser intensity of the center data point and the corresponding H point cloud data points in the nth point cloud data subspace, the electromagnetic wave reflectivity of the nth point cloud data subspace is calculated;

[0032] S102: according to the preset method, the environmental slope, the environmental surface roughness and the electromagnetic wave reflectivity are extracted, and the electromagnetic wave refractive index of the nth point cloud data subspace is obtained;

[0033] S103: the environmental slope, the environmental surface roughness, the electromagnetic wave reflectivity and the electromagnetic wave refractive index are constructed into the environmental feature data of the nth point cloud data point;

[0034] S104: let n=n+1, if n is less than or equal to N, continue to execute S101 to S103, if n is greater than N, obtain the corresponding environmental feature data of the N point cloud data subspaces, and end the current process.

[0035] Further, the method for obtaining the electromagnetic wave refractive index of the nth point cloud data subspace comprises:

[0036] input the environmental slope, the environmental surface roughness and the electromagnetic wave reflectivity of the nth point cloud data subspace into the medium type diagnosis model, and obtain the medium type of the nth point cloud data subspace;

[0037] Obtain the electromagnetic wave propagation speed corresponding to the medium type of the nth point cloud data subspace from a pre-built medium-propagation speed database; and calculate the electromagnetic wave refractive index of the nth point cloud data point based on the speed of light and the electromagnetic wave propagation speed.

[0038] Further, the method for obtaining network feature data of N point cloud data subspaces comprises:

[0039] S200: Set the initial value of n as 1, and the value range of n is 1 to N;

[0040] S201: Calculate the corresponding channel capacity based on the network bandwidth, signal power and noise power of the nth point cloud data subspace; calculate the corresponding link quality index based on the signal power, noise power and network packet loss rate of the nth point cloud data subspace; calculate the link throughput of the nth point cloud data subspace based on the network packet loss rate and network bandwidth; and calculate the link stability factor of the nth point cloud data subspace based on the link quality index and network delay;

[0041] S202: Construct the channel capacity, link quality index, link throughput and link stability factor into the network feature data of the nth point cloud data subspace;

[0042] S203: Set n = n + 1, if n is less than or equal to N, continue to execute S201 to S102, if n is greater than N, construct the network feature data of N point cloud data subspaces into a network feature data set, and end the current process.

[0043] Further, the training method of the fitness evaluation model comprises:

[0044] Pre-collect a fitness evaluation data set, the fitness evaluation data set comprising K groups of fitness evaluation data and K groups of fitness evaluation scores corresponding to the fitness evaluation data, K being a positive integer greater than 0, the fitness evaluation data comprising environmental feature data and network feature data; divide the fitness evaluation data set into a training set and a validation set, wherein the training set is used to train the fitness evaluation model, and the validation set is used to evaluate the generalization performance of the fitness evaluation model;

[0045] In the training process of the fitness evaluation model, the minimum cross-entropy loss function is used as the optimization target, the early stopping strategy is used to monitor the performance of the validation set, the model performance is optimized by continuously adjusting the network parameters, and when the prediction accuracy on the validation set reaches the expected accuracy, it is considered that the fitness evaluation model has converged, and the training is stopped; the fitness evaluation model is trained using a deep neural network based on a multilayer perceptron;

[0046] The fitness evaluation data is converted into a feature vector; the input layer of the fitness evaluation model receives the high-dimensional feature vector, extracts the nonlinear relationship in the data through multiple hidden layers, and finally the output layer of the fitness evaluation model calculates the probability distribution of the fitness evaluation score through the softmax activation function, and outputs the fitness evaluation score corresponding to the maximum probability as the final prediction result.

[0047] The super-long-range single-soldier image transmission relay enhancement device for complex terrain is used to realize the super-long-range single-soldier image transmission relay enhancement method for complex terrain, and comprises:

[0048] An image acquisition module is configured to acquire target image data.

[0049] A first acquisition module is configured to acquire three-dimensional point cloud data of an environment in which a single soldier is located.

[0050] A first processing module is configured to divide the three-dimensional point cloud data to obtain N point cloud data subspaces.

[0051] A second processing module is configured to extract features of the N point cloud data subspaces to obtain environmental feature data of the N point cloud data subspaces.

[0052] A second acquisition module is configured to acquire network operation data of the N point cloud data subspaces.

[0053] A third processing module is configured to extract features of the network operation data of the N point cloud data subspaces to obtain network feature data of the N point cloud data subspaces.

[0054] A path optimization module is configured to adaptively optimize a transmission path of a single-soldier relay device based on the environmental feature data and the network feature data of the N point cloud data subspaces.

[0055] Compared with the prior art, the super-long-range single-soldier image transmission relay enhancement device and method for complex terrain have the following technical effects and advantages:

[0056] First, based on the laser radar, millimeter wave radar and multi-view vision camera, three-dimensional point cloud data of an environment in which a single soldier is located is acquired in real time, key features such as environmental slope, surface roughness, electromagnetic wave reflectivity and refractivity are extracted by dividing the point cloud data subspaces, and the physical properties of terrain obstacles are accurately identified in combination with a medium type diagnosis model, thereby providing high-precision environmental data support for signal propagation path optimization.

[0057] Secondly, by dynamically monitoring network operation parameters (signal power, noise power, delay, packet loss rate, etc.), real-time calculation of channel capacity, link quality index, throughput and link stability factor, a multi-dimensional network feature index system is constructed to comprehensively evaluate the link availability. On this basis, an adaptability evaluation model is adopted to comprehensively evaluate the environmental feature data and network feature data, dynamically generate the adaptability score and select the optimal relay node, and select the transmission path through the adaptive multi-hop relay mechanism.

[0058] Finally, by dynamically adjusting the transmission path and relay node deployment, low delay and high reliability of image data transmission in complex terrain is ensured, and the problem of signal interruption caused by fixed path in traditional technology is avoided. In addition, through the loop-free transmission path planning, the data packet loop redundancy is effectively prevented, and the network resource utilization is improved. In military operations, it can help commanders to obtain real-time high-definition images of battlefield situation, and improve decision-making efficiency; in emergency rescue, geological exploration and other scenes, it can quickly establish long-distance communication link, and ensure efficient backhaul of on-site data and safe execution of tasks, which has wide military and civilian application value.

[0059] The scheme dynamically models the three-dimensional environmental features and intelligently optimizes the network link to adaptively optimize the transmission path of the individual relay device, effectively solving the problems of insufficient adaptability, serious signal attenuation and poor transmission stability of the traditional transmission link in complex terrain, and improving the data transmission distance and quality in the over-the-horizon scene. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The structure diagram of the over-the-horizon individual image transmission relay enhancement device for complex terrain of embodiment 1 of the application;

[0061] Figure 2 The flow chart of the over-the-horizon individual image transmission relay enhancement method for complex terrain of embodiment 3 of the application;

[0062] Figure 3 The structure diagram of the over-the-horizon individual image transmission relay enhancement device for complex terrain of embodiment 2 of the application;

[0063] Figure 4 The flow chart of the method for adaptively optimizing the transmission path of the individual relay device;

[0064] Figure 5 The flow chart of the method for obtaining the optimal transmission path;

[0065] Figure 6 The flow chart of the method for obtaining the environmental feature data of the N point cloud data subspaces;

[0066] Figure 7Flowchart of a method for obtaining network feature data of N point cloud data subspaces. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present invention will be described in detail, clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be noted that the specific embodiments described below are only used to better illustrate and describe the technical solutions of the present invention, and are intended to enable those skilled in the art to better understand and implement the present invention, and should not be construed as limiting the scope of protection of the present invention. Without departing from the spirit and essence of the present invention, those skilled in the art may modify, adjust or make equivalent replacements based on the contents disclosed in the present invention, and these should all be regarded as the scope of protection of the present invention.

[0068] Example 1

[0069] See also Figure 1 As shown, the beyond-visual-range individual image transmission relay enhancement device for complex terrain described in this embodiment includes an image acquisition module, a first acquisition module, a first processing module, a second processing module, a second acquisition module, a third processing module and a path optimization module. Each module realizes data transmission through wired and / or wireless connections.

[0070] The image acquisition module is used to collect target image data; the target image data is collected by remote devices such as drones, surveillance cameras, and unmanned vehicles.

[0071] It should be noted that in the beyond-line-of-sight image transmission scenario, the target image data is blocked by terrain, buildings or obstacles during transmission, and the target image data cannot be directly transmitted to the terminal. It is necessary to rely on individual relay equipment (relay equipment carried by individual soldiers) for transfer. The relay equipment carried by individual soldiers is a lightweight, low-power, intelligent wireless communication node, which is mainly used to enhance data transmission capabilities in beyond-line-of-sight (NLOS) environments and ensure the communication stability between the front-end acquisition equipment and the back-end command center or server. The relay equipment can serve as a data transfer station to forward information between different network nodes, and it can also optimize the data transmission path, thereby improving the quality and coverage of data transmission.

[0072] The first acquisition module is used to collect three-dimensional point cloud data of the environment in which the individual soldier is located; the three-dimensional point cloud data is obtained through laser radar, millimeter wave radar and multi-view camera.

[0073] The first processing module is used to divide the three-dimensional point cloud data into N point cloud data subspaces.

[0074] The method for obtaining N point cloud data subspaces includes:

[0075] Step one: divide the three-dimensional point cloud data according to a preset sliding step to obtain N center data points of the three-dimensional point cloud data; let the initial value of n be 1, and the value range of n be 1 to N;

[0076] Step two: obtain the nth center data point, select H point cloud data points adjacent to the nth center data point from the three-dimensional point cloud data; and construct the nth point cloud data subspace by the nth center data point and the H point cloud data points.

[0077] Step three: let n = n + 1, if n is less than or equal to N, continue to execute step two, if n is greater than N, end the current process, and obtain N point cloud data subspaces.

[0078] The second processing module is configured to extract features from the N point cloud data subspaces to obtain environmental feature data of the N point cloud data subspaces.

[0079] As shown in Figure 6 The method for obtaining the environmental feature data of the N point cloud data subspaces comprises:

[0080] S100: let the initial value of n be 1, and the value range of n be 1 to N;

[0081] S101: based on the three-dimensional coordinates of the center data point in the nth point cloud data subspace and the corresponding H point cloud data points, the environmental slope and the environmental surface roughness of the nth point cloud data subspace are calculated; based on the incident laser intensity and the reflected laser intensity of the center data point in the nth point cloud data subspace and the corresponding H point cloud data points, the electromagnetic wave reflectivity of the nth point cloud data subspace is calculated;

[0082] S102: according to a preset method, the environmental slope, the environmental surface roughness and the electromagnetic wave reflectivity are extracted to obtain the electromagnetic wave refractive index of the nth point cloud data subspace;

[0083] S103: the environmental slope, the environmental surface roughness, the electromagnetic wave reflectivity and the electromagnetic wave refractive index are constructed into the environmental feature data of the nth point cloud data point;

[0084] S104: let n = n + 1, if n is less than or equal to N, continue to execute S101 to S103, if n is greater than N, obtain the corresponding environmental feature data of the N point cloud data subspaces, and end the current process.

[0085] The method for obtaining the electromagnetic wave refractive index of the nth point cloud data subspace comprises:

[0086] inputting the environmental slope, the environmental surface roughness and the electromagnetic wave reflectivity of the nth point cloud data subspace into the medium type diagnosis model to obtain the medium type of the nth point cloud data subspace; the medium type comprises metal, glass, water, concrete and vegetation;

[0087] obtaining the electromagnetic wave propagation speed corresponding to the medium type of the nth point cloud data subspace from a pre-constructed medium-propagation speed database; and calculating the electromagnetic wave refractive index of the nth point cloud data point based on the speed of light and the electromagnetic wave propagation speed.

[0088] The method for calculating the electromagnetic wave refractive index comprises:

[0089]

[0090] wherein, ZSL n is the electromagnetic wave refractive index of the nth point cloud data subspace, GS is the speed of light, and the speed of light is 3×10 8 m / s, CBSD n is the electromagnetic wave propagation speed of the medium type of the nth point cloud data subspace.

[0091] The method for calculating the environmental slope comprises:

[0092]

[0093] wherein, HJPD n is the environmental slope of the nth central data point, π is a constant, and π is 3.14159, XFC n is a covariance matrix, H is the number of point cloud data points, h≤H, SWZB n is the three-dimensional coordinate of the nth central data point, is the calculation result of the transpose matrix, |XFC n | is the value of the determinant corresponding to the covariance matrix; is the eigenvector of the covariance matrix.

[0094] For example, the calculation result of is (7, 5, 4), and the corresponding transpose matrix is

[0095] The method for calculating the environmental surface roughness comprises:

[0096]

[0097] wherein, BMCD n is the environmental surface roughness of the nth central data point, XZB n is the X-axis coordinate of the nth central data point, XZBh is the X-axis coordinate of the hth point cloud data point; YZB n is the Y-axis coordinate of the nth central data point, YZB h is the Y-axis coordinate of the hth point cloud data point; ZZB n is the Z-axis coordinate of the nth central data point, ZZB h is the Z-axis coordinate of the hth point cloud data point.

[0098] The method for calculating the electromagnetic wave reflectivity comprises:

[0099]

[0100] wherein FSL n is the electromagnetic wave reflectivity of the nth central data point, FS n is the reflected laser intensity of the nth central data point, RS n is the incident laser intensity of the nth central data point, FS h is the reflected laser intensity of the hth point cloud data point, RS h is the incident laser intensity of the hth point cloud data point.

[0101] The method for training the medium type diagnosis model comprises:

[0102] A pre-constructed medium type diagnosis data set comprises Y sets of medium type diagnosis data and the medium types corresponding to the Y sets of medium type diagnosis data, Y being a positive integer greater than 0, the medium type diagnosis data comprising environmental slope, environmental surface roughness and electromagnetic wave reflectivity; the medium type diagnosis data set is divided into a medium type diagnosis data training set and a medium type diagnosis data verification set, wherein the medium type diagnosis data training set is used for parameter learning of the medium type diagnosis model, and the medium type diagnosis data verification set is used for real-time evaluation of the generalization ability of the medium type diagnosis model.

[0103] In the medium type diagnosis model training process, a deep neural network structure based on a multilayer perceptron is adopted, the medium type diagnosis data is converted into a feature vector as input, the nonlinear features in the data are extracted through a hidden layer, and finally a probability distribution of the medium type is generated by using a softmax activation function in an output layer, and the medium type corresponding to the maximum probability is output as the final prediction result; the training process aims to minimize the cross-entropy loss function, and an early stopping strategy is introduced to monitor the performance of the medium type diagnosis data verification set; when the prediction accuracy on the medium type diagnosis data verification set reaches a preset threshold, it is considered that the medium type diagnosis model has converged, and the training is stopped.

[0104] The softmax activation function is:

[0105]

[0106] wherein, Softmax(JZLX num ) is the output probability corresponding to the num-th feature vector, JZLX num is the num-th feature vector, NUM is the total number of feature vectors, and e is a constant.

[0107] The second acquisition module is configured to acquire network operation data of N point cloud data subspaces; the network operation data includes signal power, noise power, network delay, network packet loss rate, network bandwidth, etc.

[0108] The third processing module is configured to perform feature extraction on the network operation data of the N point cloud data subspaces to obtain network feature data of the N point cloud data subspaces.

[0109] As shown in Figure 7 , the method for obtaining network feature data of N point cloud data subspaces includes:

[0110] S200: Let the initial value of n be 1, and the value range of n be 1 to N;

[0111] S201: Based on the network bandwidth, signal power and noise power of the n-th point cloud data subspace, the corresponding channel capacity is calculated; based on the signal power, noise power and network packet loss rate of the n-th point cloud data subspace, the corresponding link quality index is calculated; based on the network packet loss rate and network bandwidth, the link throughput of the n-th point cloud data subspace is calculated; based on the link quality index and network delay, the link stability factor of the n-th point cloud data subspace is calculated;

[0112] S202: The channel capacity, link quality index, link throughput and link stability factor are constructed into the network feature data of the n-th point cloud data subspace;

[0113] S203: Let n=n+1, if n is less than or equal to N, continue to execute S201 to S102, if n is greater than N, the network feature data of the N point cloud data subspaces is constructed into a network feature data set, and the current process is ended.

[0114] The method for calculating the channel capacity includes:

[0115] XDRL n = WLDK × log2(1+(XHGL n -ZSGL n ));

[0116] wherein, XDRL nThe channel capacity of the nth point cloud data subspace is greater, and the data transmission capability is stronger; WLDK is the network bandwidth; XHGL n The signal power of the nth point cloud data subspace is ZSGL n The noise power of the nth point cloud data subspace is; log2(·) is a logarithmic function with base 2.

[0117] The method for calculating the link quality index comprises:

[0118]

[0119] Wherein, ZLZS n The link quality index of the nth point cloud data subspace is DBL n The network packet loss rate of the nth point cloud data subspace is higher, the link quality is better, and it is more suitable for routing decision of multi-hop network.

[0120] The method for obtaining the link throughput comprises:

[0121] TTL n =WLDK×(1-DBL n );

[0122] Wherein, TTL n The link throughput of the nth point cloud data subspace is used to reflect the real available bandwidth of the link, not just the theoretical bandwidth.

[0123] The method for obtaining the link stability factor comprises:

[0124]

[0125] Wherein, WDYZ n The link stability factor of the nth point cloud data subspace is WLYC n The network delay of the nth point cloud data subspace is, the link stability factor reflects the balance of link quality and delay, the higher the link stability factor, the better the link quality, and the more suitable for path planning.

[0126] The path optimization module adaptively optimizes the transmission path of the individual relay device based on the environmental feature data and network feature data of the N point cloud data subspaces.

[0127] As Figure 4 The method for adaptively optimizing the transmission path of the individual relay device comprises:

[0128] The environment feature data and the network feature data of the N point cloud data subspaces are input into an fitness evaluation model respectively, and N fitness evaluation scores corresponding to the N point cloud data subspaces are obtained; the fitness evaluation score is used to measure the fitness degree of the point cloud data subspace as a relay node in a transmission path, and the higher the fitness evaluation score is, the more suitable the point cloud data subspace is as a relay node in a transmission path;

[0129] The point cloud data subspace with a fitness evaluation score greater than or equal to a preset fitness evaluation score threshold is constructed into a set of to-be-planned transmission nodes;

[0130] The to-be-planned transmission nodes in the set of to-be-planned transmission nodes are connected to each other to construct G transmission path subsegments, and G is an integer greater than zero;

[0131] For each transmission path subsegment, a corresponding transmission path subsegment weight is calculated according to the environment feature data and the network feature data of the start point and the end point of the transmission path subsegment; and the G transmission path subsegment weights are constructed into a set of transmission path subsegment weights;

[0132] Based on the set of transmission path subsegment weights, the G transmission path subsegments are optimized to obtain an optimal transmission path; and the individual relay device transmits the target image data according to the optimal transmission path.

[0133] As shown in Figure 5 the method for obtaining the optimal transmission path includes:

[0134] The number of preset transmission path subsegments is R, and R is an integer greater than zero; and the optimal transmission path is initialized as empty;

[0135] The transmission path subsegment with the maximum transmission path subsegment weight is selected from the set of transmission path subsegment weights, and is recorded as a to-be-planned transmission path subsegment; and the to-be-planned transmission path subsegment is removed from the set of transmission path subsegment weights;

[0136] The to-be-planned transmission path subsegment is added to the optimal transmission path; it is judged whether there is a loop in the optimal transmission path; if there is a loop in the optimal transmission path, the to-be-planned transmission path subsegment is removed from the optimal transmission path; and if there is no loop in the optimal transmission path, the to-be-planned transmission path subsegment is kept in the optimal transmission path;

[0137] The above process is repeatedly executed until the number of transmission path subsegments in the optimal transmission path reaches R.

[0138] The method for judging whether there is a loop in the optimal transmission path includes:

[0139] Randomly select a transmission path sub-section as a starting point from the optimal transmission path, and traverse along the selected transmission path sub-section; if the starting position is returned in the traversal process, and any transmission path sub-section in the entire path is not repeated, then there is a loop in the optimal transmission path.

[0140] It should be noted that in the process of the optimal transmission path, the goal is to construct a connected but loop-free network transmission path, and loop-free is to avoid the infinite circulation of data packets in the loop and reduce redundant paths. The definition of a loop is: if a transmission path sub-section is started, a plurality of transmission path sub-sections are passed through, and finally returned to the starting node, and the entire transmission path does not have repeated transmission path sub-sections, then this transmission path forms a loop.

[0141] The transmission path sub-section weight acquisition method comprises:

[0142] ZDQZ g =HJZH g +WLZH g ;

[0143]

[0144] ZDQZ g is the transmission path sub-section weight of the gth transmission path sub-section, HJZH g is the environmental feature weight of the gth transmission path sub-section; ZSL (g，s) is the electromagnetic wave refractive index of the starting point of the gth transmission path sub-section, ZSL (g，e) is the electromagnetic wave refractive index of the ending point of the gth transmission path sub-section; FSL (g，s) is the electromagnetic wave reflectivity of the starting point of the gth transmission path sub-section, FSL (g，e) is the electromagnetic wave reflectivity of the ending point of the gth transmission path sub-section; HJPD (g，e) is the environmental slope of the ending point of the gth transmission path sub-section, HJPD (g，s) is the environmental slope of the starting point of the gth transmission path sub-section; BMCD (g，e) is the environmental surface roughness of the ending point of the gth transmission path sub-section, BMCD (g，s) is the environmental surface roughness of the starting point of the gth transmission path sub-section, sec(·) is the secant function, and csc(·) is the cosecant function.

[0145] WLZH g is the network feature weight of the gth transmission path sub-section, XDRL (g，s) is the channel capacity of the starting point of the gth transmission path sub-section, XDRL (g，e) is the channel capacity of the ending point of the gth transmission path sub-section; ZLZS (g，e)a link quality index of an end point of the gth transmission path sub-segment, ZLZS (g，s) a link quality index of a start point of the gth transmission path sub-segment; WDYZ (g，s) a link stability factor of a start point of the gth transmission path sub-segment, WDYZ (g，e) a link stability factor of an end point of the gth transmission path sub-segment; TTL (g，s) a link throughput of a start point of the gth transmission path sub-segment, TTL (g，e) a link throughput of an end point of the gth transmission path sub-segment; Ln(·) is a logarithmic function, and e is a constant.

[0146] It should be noted that the refraction and reflection characteristics of electromagnetic waves can significantly affect the propagation path and intensity of signals. The use of secant and cosecant functions can reflect the comprehensive influence of different refraction and reflectance combinations on transmission. In complex terrain, slope and surface roughness can affect signal scattering, and by calculating the environmental slope difference ratio and the environmental surface roughness difference ratio, the influence of terrain changes on signal transmission can be reflected. Channel capacity determines the amount of data that can be transmitted, and link quality index reflects transmission quality. Logarithmic operation can balance the order of magnitude difference between channel capacity and link quality index, and reasonably reflect their comprehensive influence on the transmission path. Link stability is directly related to the reliability of transmission, and throughput affects transmission efficiency. Exponential function can reflect the positive influence on transmission path weight as the stability factor increases and the throughput increases.

[0147] The training method of the fitness evaluation model comprises:

[0148] A fitness evaluation dataset is collected in advance, the fitness evaluation dataset comprises K groups of fitness evaluation data and fitness evaluation scores corresponding to the K groups of fitness evaluation data, K is a positive integer greater than 0, the fitness evaluation data comprises environmental feature data and network feature data; the fitness evaluation dataset is divided into a training set and a validation set, wherein the training set is used to train the fitness evaluation model, and the validation set is used to evaluate the generalization performance of the fitness evaluation model;

[0149] During the training process of the fitness evaluation model, the cross-entropy loss function is minimized as the optimization target, the early stopping strategy is used to monitor the performance of the validation set, the network parameters are continuously adjusted to optimize the model performance; when the prediction accuracy on the validation set reaches the expected accuracy, it is considered that the fitness evaluation model has converged, and the training is stopped; the fitness evaluation model is trained using a deep neural network based on a multilayer perceptron;

[0150] The fitness evaluation data is converted into a feature vector; the input layer of the fitness evaluation model receives a high-dimensional feature vector, extracts the nonlinear relationship in the data through multiple hidden layers, and finally the output layer of the fitness evaluation model calculates the probability distribution of the fitness evaluation score through the softmax activation function, and outputs the fitness evaluation score corresponding to the maximum probability as the final prediction result.

[0151] Embodiment 2

[0152] Please refer to Figure 3 As shown in the figure, the embodiment provides a complex terrain-oriented beyond-visual-range single-soldier image relay enhancement device, which also includes:

[0153] The forwarding optimization module performs adaptive optimization on the target image data forwarding of the single-soldier relay device based on the fitness evaluation scores of the N point cloud data subspaces.

[0154] The method for adaptively optimizing the target image data forwarding of the single-soldier relay device includes:

[0155] Sum the fitness evaluation scores of the N point cloud data subspaces to obtain a fitness evaluation score sum;

[0156] If the fitness evaluation score sum is greater than or equal to a preset score sum threshold, the target image data does not need to be further forwarded, and the target image data is transmitted to the terminal device;

[0157] If the fitness evaluation score sum is less than the preset score sum threshold, the target image data needs to be further forwarded, the data is forwarded to the next-hop single-soldier relay device, and the above determination process is repeatedly executed until the target image data is successfully transmitted to the terminal device.

[0158] It should be noted that the present scheme takes the image data transmission in the single-soldier relay mode as an example. The workflow is as follows:

[0159] During the task execution process, the single-soldier relay device (hereinafter referred to as "single-soldier relay device") carried by the single-soldier continuously monitors and collects the wireless channel state information of the surrounding environment.

[0160] When the remote data acquisition device (such as a drone, a surveillance camera, an unmanned vehicle) acquires target image data, if the remote data acquisition device cannot directly establish a stable communication link with the terminal device (such as a command center or a back-end server), the target image data is first transmitted to the nearest individual relay device through a wireless link. The individual relay device establishes a wireless connection with other individual relay devices or fixed base stations nearby through an adaptive multi-hop relay link mechanism, and dynamically adjusts the transmission path. The individual relay device determines whether the target image data needs to be further forwarded based on the fitness evaluation score of the N point cloud data subspaces; if the target image data does not need to be further forwarded, the target image data is transmitted to the terminal device; if the target image data does not need to be further forwarded, the individual relay device forwards the target image data to the next-hop individual relay device until the target image data is successfully transmitted to the terminal device.

[0161] Embodiment 3

[0162] Referring to Figure 2 The embodiment provides a complex terrain-oriented over-the-horizon individual relay method, which includes:

[0163] Acquire target image data;

[0164] Acquire three-dimensional point cloud data of the environment where the individual is located;

[0165] Divide the three-dimensional point cloud data to obtain N point cloud data subspaces;

[0166] Extract features from the N point cloud data subspaces to obtain environmental feature data of the N point cloud data subspaces;

[0167] Acquire network operation data of the N point cloud data subspaces;

[0168] Extract features from the network operation data of the N point cloud data subspaces to obtain network feature data of the N point cloud data subspaces;

[0169] Adaptively optimize the transmission path of the individual relay device based on the environmental feature data and the network feature data of the N point cloud data subspaces.

[0170] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0171] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for beyond line of sight man-portable image relay enhancement for complex terrain, characterized in that, The method comprises the following steps: collecting target image data; collecting three-dimensional point cloud data of an environment in which a single soldier is located; dividing the three-dimensional point cloud data to obtain N point cloud data subspaces; extracting features of the N point cloud data subspaces to obtain environmental feature data of the N point cloud data subspaces; collecting network operation data of the N point cloud data subspaces; extracting features of the network operation data of the N point cloud data subspaces to obtain network feature data of the N point cloud data subspaces; performing adaptive optimization on a transmission path of a single-soldier relay device based on the environmental feature data and the network feature data of the N point cloud data subspaces; The method for performing adaptive optimization on the transmission path of the single-soldier relay device comprises the following steps: inputting the environmental feature data and the network feature data of the N point cloud data subspaces into an adaptability evaluation model respectively to obtain adaptability evaluation scores corresponding to the N point cloud data subspaces; constructing point cloud data subspaces with adaptability evaluation scores greater than or equal to a preset adaptability evaluation score threshold into a set of to-be-planned transmission nodes; connecting the to-be-planned transmission nodes in the set of to-be-planned transmission nodes to each other to construct G transmission path subsegments, wherein G is an integer greater than zero; for each transmission path subsegment, calculating a corresponding transmission path subsegment weight according to the environmental feature data and the network feature data of the start point and the end point of the transmission path subsegment; and constructing the G transmission path subsegment weights into a set of transmission path subsegment weights; based on the set of transmission path subsegment weights, optimizing the G transmission path subsegments to obtain an optimal transmission path; and causing the single-soldier relay device to transmit the target image data according to the optimal transmission path.

2. The complex terrain oriented beyond line of sight man portable image relay augmentation method of claim 1 wherein, The method for obtaining the optimal transmission path comprises the following steps: presetting the number of transmission path subsegments as R, wherein R is an integer greater than zero; and initializing the optimal transmission path as empty; selecting a transmission path subsegment with the largest transmission path subsegment weight from the set of transmission path subsegment weights, denoted as a to-be-planned transmission path subsegment, and removing the to-be-planned transmission path subsegment from the set of transmission path subsegment weights; adding the to-be-planned transmission path subsegment to the optimal transmission path, and judging whether there is a loop in the optimal transmission path; if there is a loop in the optimal transmission path, removing the to-be-planned transmission path subsegment from the optimal transmission path; and if there is no loop in the optimal transmission path, leaving the to-be-planned transmission path subsegment in the optimal transmission path; repeating the above process until the number of transmission path subsegments in the optimal transmission path reaches R.

3. The complex terrain oriented beyond line of sight man portable image relay augmentation method of claim 2, wherein, The method for judging whether there is a loop in the optimal transmission path comprises the following steps: randomly selecting a transmission path subsegment from the optimal transmission path as a start point, and performing traversal along the selected transmission path subsegment; if the traversal process returns to the starting position and no transmission path subsegment is repeatedly passed through in the entire path, it is determined that there is a loop in the optimal transmission path.

4. The complex terrain oriented beyond line of sight manpack image relay augmentation method of claim 1, wherein, The method for obtaining the N point cloud data subspaces comprises the following steps: step one: dividing the three-dimensional point cloud data according to a preset sliding step to obtain N center data points of the three-dimensional point cloud data; and setting an initial value of n as 1, wherein n ranges from 1 to N; Step two: obtain the nth center data point, select H point cloud data points adjacent to the nth center data point from the three-dimensional point cloud data; construct the nth point cloud data subspace with the nth center data point and the H point cloud data points; Step three: let n = n + 1, if n is less than or equal to N, continue to execute step two, if n is greater than N, end the current process, and obtain N point cloud data subspaces.

5. The complex terrain oriented beyond line of sight man portable imagery relay augmentation method of claim 1 wherein, The method for obtaining the environmental feature data of the N point cloud data subspaces comprises: S100: let the initial value of n be 1, and the value range of n be 1 to N; S101: based on the three-dimensional coordinates of the center data point and the corresponding H point cloud data points in the nth point cloud data subspace, the environmental slope and the environmental surface roughness of the nth point cloud data subspace are calculated; based on the incident laser intensity and the reflected laser intensity of the center data point and the corresponding H point cloud data points in the nth point cloud data subspace, the electromagnetic wave reflectivity of the nth point cloud data subspace is calculated; S102: according to a predetermined method, the environmental slope, the environmental surface roughness and the electromagnetic wave reflectivity are extracted to obtain the electromagnetic wave refractive index of the nth point cloud data subspace; S103: the environmental slope, the environmental surface roughness, the electromagnetic wave reflectivity and the electromagnetic wave refractive index are constructed into the environmental feature data of the nth point cloud data point; S104: let n = n + 1, if n is less than or equal to N, continue to execute S101 to S103, if n is greater than N, obtain the corresponding environmental feature data of the N point cloud data subspaces, and end the current process.

6. The complex terrain oriented beyond line of sight man portable image relay enhancement method of claim 5, wherein, The method for obtaining the electromagnetic wave refractive index of the nth point cloud data subspace comprises: input the environmental slope, the environmental surface roughness and the electromagnetic wave reflectivity of the nth point cloud data subspace into a medium type diagnosis model to obtain the medium type of the nth point cloud data subspace; obtain the electromagnetic wave propagation speed corresponding to the medium type of the nth point cloud data subspace from a pre-constructed medium-propagation speed database; based on the speed of light and the electromagnetic wave propagation speed, the electromagnetic wave refractive index of the nth point cloud data point is calculated.

7. The complex terrain oriented beyond line of sight man portable imagery relay augmentation method of claim 1, wherein, The method for obtaining the network feature data of the N point cloud data subspaces comprises: S200: let the initial value of n be 1, and the value range of n be 1 to N; S201: based on the network bandwidth, the signal power and the noise power of the nth point cloud data subspace, the corresponding channel capacity is calculated; based on the signal power, the noise power and the network packet loss rate of the nth point cloud data subspace, the corresponding link quality index is calculated; based on the network packet loss rate and the network bandwidth, the link throughput of the nth point cloud data subspace is calculated; based on the link quality index and the network delay, the link stability factor of the nth point cloud data subspace is calculated; S202: the channel capacity, the link quality index, the link throughput and the link stability factor are constructed into the network feature data of the nth point cloud data subspace; S203: let n = n + 1, if n is less than or equal to N, continue to execute S201 to S102, if n is greater than N, construct the network feature data of the N point cloud data subspaces into a network feature data set, and end the current process.

8. The complex terrain oriented beyond line of sight manet relay enhancement method of claim 1, wherein, The training method of the fitness evaluation model comprises: Pre-collecting a fitness evaluation dataset, the fitness evaluation dataset comprising K sets of fitness evaluation data and K sets of fitness evaluation scores corresponding to the K sets of fitness evaluation data, K being a positive integer greater than 0, the fitness evaluation data comprising environmental feature data and network feature data; dividing the fitness evaluation dataset into a training set and a validation set, wherein the training set is used to train the fitness evaluation model, and the validation set is used to evaluate the generalization performance of the fitness evaluation model; During the training of the fitness evaluation model, a cross-entropy loss function is minimized as an optimization objective, an early stopping strategy is used to monitor the performance of the validation set, and the model performance is optimized by continuously adjusting network parameters; when the prediction accuracy on the validation set reaches an expected accuracy, it is considered that the fitness evaluation model has converged, and the training is stopped; the fitness evaluation model is trained using a deep neural network based on a multilayer perceptron; The fitness evaluation data is converted into a feature vector; the input layer of the fitness evaluation model receives the feature vector, extracts the nonlinear relationship in the data through multiple hidden layers, and finally the output layer of the fitness evaluation model calculates the probability distribution of the fitness evaluation score through a softmax activation function, and outputs the fitness evaluation score corresponding to the maximum probability as the final prediction result.

9. The device for complex terrain oriented beyond line of sight manpack image relay enhancement, for implementing the method for complex terrain oriented beyond line of sight manpack image relay enhancement according to any one of claims 1-8, characterized in that, Comprise: An image acquisition module for acquiring target image data; A first acquisition module for acquiring three-dimensional point cloud data of an environment in which a single soldier is located; A first processing module for dividing the three-dimensional point cloud data to obtain N point cloud data subspaces; A second processing module for extracting features of the N point cloud data subspaces to obtain environmental feature data of the N point cloud data subspaces; A second acquisition module for acquiring network operation data of the N point cloud data subspaces; A third processing module for extracting features of the network operation data of the N point cloud data subspaces to obtain network feature data of the N point cloud data subspaces; A path optimization module for adaptively optimizing the transmission path of a single-soldier relay device based on the environmental feature data and the network feature data of the N point cloud data subspaces.

Images