Power line impedance matching method and system based on space division

Through the deep reinforcement learning algorithm based on space segmentation, the values ​​of capacitors and inductor components are optimized, and the problems of insufficient complexity and adaptability of intelligent algorithms in power line impedance matching in the prior art are solved, and efficient power line impedance matching is achieved, reducing the System development cost.

CN119323232APending Publication Date: 2025-01-17NORTH CHINA ELECTRIC POWER UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411410298.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-10
Publication Date
2025-01-17

AI Technical Summary

Technical Problem

In the power line impedance matching, although intelligent algorithms such as genetic algorithms and particle swarm algorithms can achieve a good match, they have complex modeling, long iteration time and large calculations, resulting in increased system development costs and cannot dynamically adjust impedance matching. The component values ​​of the network make it difficult to achieve adaptive matching.

Method used

The deep reinforcement learning algorithm based on space segmentation is adopted to optimize the values ​​of capacitors and inductor components through database query and multi-agent distributed training, so as to achieve power line impedance matching, reduce search space, and improve optimization efficiency.

Benefits of technology

Effectively reduce program running time, improve the problem of unsatisfactory optimization due to excessive search space, improve the accuracy and efficiency of impedance matching, avoid poor optimization results caused by insufficient training of agents, and simplify system design. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119323232A_ABST
    Figure CN119323232A_ABST
Patent Text Reader

Abstract

The invention provides a power line impedance matching method and system based on space division, and belongs to the technical field of power line impedance matching based on deep learning. Acquiring a power line terminal load value; based on a database query function, querying capacitance and inductance element values and an optimal target function value during optimal matching corresponding to the obtained power line terminal load value in a database; wherein the database stores the optimal values of capacitance and inductance devices under the optimal matching condition corresponding to different power line terminal load values obtained by using a deep reinforcement learning algorithm model for optimization. According to the method, the program running time is effectively shortened, the problem of unsatisfactory agent optimization caused by too large search space is improved, and the problem of poor optimization result caused by insufficient training of the agent is avoided; the search space is reduced, and the search accuracy is improved; the space is subjected to non-uniform segmentation, and compared with multi-agent distributed optimization based on uniform segmentation, the search effect of the agents is obviously better.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of power line impedance matching based on deep learning, and in particular to a power line impedance matching method and system based on space segmentation. Background Art

[0002] Smart grid is the main trend of power grid development, so it puts forward higher requirements on communication quality. Power line communication uses existing channels for communication, does not require special communication lines, has the advantages of no need for additional wiring, strong flexibility, etc. With the advancement of science and technology, PLC has undergone great changes in many aspects, from the original analog communication to the current digital communication, from single channel to multi-channel, these changes mean that the communication quality of power line channels is required to be higher.

[0003] When the power line channel is used as a signal transmission medium, the environment is harsh when facing high-frequency signals, resulting in complex characteristics of the network's input impedance. Impedance changes may cause impedance mismatch, thereby reducing the power reception rate of the receiving end, or even causing the signal to be unable to be effectively transmitted to the receiving end, seriously affecting the communication quality. In order to maximize the power transfer from the signal source to the load end, it is particularly important to ensure impedance matching. Therefore, it is of great significance to deeply study the channel characteristics and impedance matching methods.

[0004] A simple way to achieve impedance matching on power lines is to adjust the length and type of transmission lines. For example, the details, shape, and material of the power line will affect its characteristic impedance. By accurately calculating and adjusting the length of the transmission line, its characteristic impedance can be matched with the actual impedance on the power line, thereby maximizing the transmission efficiency of the signal. An impedance converter is a device specially designed to achieve impedance matching in power line communication. It usually includes transmission lines and circuit elements, and its internal electronic component parameters, such as capacitors and inductors, can be adjusted to match the impedance of the signal source with the impedance of the power line. Its parameters can be dynamically adjusted according to real-time circuit feedback data to maximize the impedance matching effect. In more complex power line communication systems, a special matching network can be designed and implemented to achieve impedance matching. The matching network consists of circuit elements such as transmission lines, capacitors, and inductors. The best impedance matching effect can be achieved by accurately calculating and optimizing the circuit structure and parameters of the matching network.

[0005] The reflection coefficient of the matching network is measured by measuring devices such as network analyzers, and the transmission line length, component value and layout are adjusted according to the measurement results to optimize the impedance matching effect; evolutionary algorithms such as genetic algorithms and particle swarm optimization algorithms are used to automatically search and optimize the parameters of the matching network. These algorithms can find the optimal solution in the parameter space to minimize the reflection coefficient or maximize the transmission efficiency.

[0006] Reference [1] proposed a chaotic particle swarm algorithm (CSA). By combining the chaotic particle swarm algorithm with a matching network, the fitness value of the function is used to evaluate each particle, and the best fitness value is continuously sought. This algorithm requires more objective functions and constraints. Narrowband impedance matching is performed under the same frequency and different loads, and the same load and different frequencies. The single-objective optimization of the chaotic particle swarm is used to determine the parameter values ​​of the variable elements of the ideal π-type matching network. Then, the dual-objective optimization (conjugate matching, power loss) of the chaotic particle swarm is used to determine the parameter values ​​of the variable elements in the non-ideal π-type and non-ideal two-stage networks, and the performance parameters of the impedance matching are calculated at the same time. The adaptability of the proposed chaotic particle swarm impedance matching is proved. Reference [2] proposed an adjustable impedance matching coupler that can provide multiple turns ratios. Under different power line channel impedance conditions, the power transmission performance of couplers with different turns ratios is analyzed and compared. The designed adjustable impedance matching coupler can realize dynamic impedance matching between the PLC transmitter / receiver and the DC power line channel, thereby improving the receiving power of the receiver.

[0007] In summary, the current research on impedance matching algorithms is mainly focused on intelligent algorithms, such as genetic algorithms, particle swarm algorithms, and neural network algorithms. These intelligent algorithms can theoretically achieve better matching, but the modeling is relatively complex, the required iteration time is long, the amount of calculation is large, and a dedicated digital processor is often required, which increases the development cost of the system. The adjustable impedance matching coupler achieves impedance matching between the PLC transmitter / receiver and the power line channel by manually adjusting the optimal turns ratio. It is impossible to dynamically adjust the component value of the impedance matching network according to the input impedance of the power line channel (network) to achieve adaptive impedance matching.

[0008] [1] Liu Chuqun. Adaptive impedance matching of RF antenna based on chaotic particle swarm optimization[D]. Hunan University, 2014.

[0009] [2] Wang Bingting. Research on impedance matching method and impedance matching coupler for DC power line communication[D]. Nanjing University of Posts and Telecommunications, 2020.001582. Summary of the invention

[0010] The object of the present invention is to provide a power line impedance matching method and system based on space division to solve at least one technical problem existing in the above-mentioned background technology.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] In a first aspect, the present invention provides a power line impedance matching method based on space division, comprising:

[0013] Get the power line terminal load value;

[0014] Based on a database query function, the capacitor and inductor component values ​​and the optimal objective function value corresponding to the acquired power line terminal load value at the optimal matching are queried in the database; wherein, the storage table in the database stores the optimal values ​​of the capacitor and inductor devices under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot find the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

[0015] Furthermore, the training of the deep reinforcement learning algorithm model includes: on the basis of the first-level space segmentation of the space with Π-type impedance matching, the space with better spatial optimization effect is segmented at the second level, and the two spaces are divided into two subspaces respectively; the real and imaginary parts of the impedance are obtained, and the DQN algorithm is used to update the parameters of the target network and the evaluation network in each space respectively. The situation of each segmented search space is different. The fully distributed training method is used to rely on its own observation to determine the optimal action, and the optimal result and optimal path of each space are found respectively; the optimal result and optimal path obtained from the training of each space are saved to obtain the optimization result.

[0016] Furthermore, after the training of the deep reinforcement learning algorithm model is completed, the optimal result of each space is compared with the error function. If the result is less than the relative error, the result with the smallest error value in each space is read as the optimal matching result; if the training results of each space are greater than the error function, each space is searched again.

[0017] Furthermore, the deep reinforcement learning algorithm includes: initializing two evaluation networks and a target network, and initializing the weight parameters of the neural network; the intelligent agent takes the action with the largest action-value function value as the optimal action, and obtains the optimal action-value function expression; after the intelligent agent obtains the reward, it will execute the optimal action and put the obtained experience sample into the playback experience pool; the intelligent agent randomly extracts small batches of experience sample data from the experience pool; calculates the objective function of each state, calculates the target value of each state, and updates the target value through the reward after the target network executes the action; uses the gradient descent method to update the parameters of the evaluation neural network; after N iterations, passes the current target network weight parameters to the evaluation network, updates the weight parameters, and as the number of iterations increases, finds the optimal position when the target value is equal to the evaluation value.

[0018] Furthermore, the state space, action space, reward value, objective function, and learning rate, number of hidden layers, experience pool capacity, and decay rate of deep reinforcement learning are set; the intelligent agent interacts with the environment to obtain state and action information, and adopts an ε-greedy search strategy, that is, while the network makes decisions, it can explore other possible optimal behaviors with a certain probability; the acquired experience samples are stored in the experience pool, the experience samples in the experience pool are continuously updated, and small batches of experience samples are randomly extracted as inputs to the evaluation network and the target network.

[0019] Furthermore, the mean square error of the two networks is calculated as the loss function, and the weight parameters of the evaluation network are updated using the gradient descent algorithm; it is determined whether the function value obtained by the algorithm reaches the target function value at the optimal match. If so, proceed to the next step; otherwise, repeat the process.

[0020] In a second aspect, the present invention provides a power line impedance matching system based on space division, comprising:

[0021] An acquisition module, used for acquiring a power line terminal load value;

[0022] A query module is used to query the capacitor and inductor component values ​​and the optimal objective function value corresponding to the optimal matching of the acquired power line terminal load value in the database based on a database query function; wherein a storage table is stored in the database, and the storage table stores the optimal values ​​of the capacitor and inductor components under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot find the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

[0023] In a third aspect, the present invention provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the power line impedance matching method based on space division as described in the first aspect is implemented.

[0024] In a fourth aspect, the present invention provides a computer device comprising a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the power line impedance matching method based on spatial segmentation as described in the first aspect.

[0025] In a fifth aspect, the present invention provides an electronic device, comprising: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the power line impedance matching method based on spatial division as described in the first aspect.

[0026] The beneficial effects of the present invention are as follows: effectively reducing the program running time, improving the problem of unsatisfactory optimization of the intelligent agent due to an overly large search space, and avoiding poor optimization results due to insufficient training of the intelligent agent; reducing the search space, further precise search, and the search results of the remaining space are poor, and the intelligent agent performs a rough search; the space is unevenly divided, and the search effect of the intelligent agent is significantly better than the evenly divided multi-agent distributed optimization.

[0027] Additional advantages of the present invention will be more clearly given in the following description or learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0029] Figure 1 This is a flow chart of a power line impedance matching method based on space division according to an embodiment of the present invention.

[0030] Figure 2 This is a schematic diagram of the equivalent impedance of the Π-type network described in an embodiment of the present invention.

[0031] Figure 3 This is a flow chart of the DQN algorithm described in an embodiment of the present invention.

[0032] Figure 4 This is a flow chart of the multi-agent algorithm described in an embodiment of the present invention.

[0033] Figure 5 This is a schematic diagram of the multi-process parallelism described in an embodiment of the present invention.

[0034] Figure 6 This is a parallel program kernel distribution diagram according to an embodiment of the present invention. Figure 6 (a) is uniform distribution; Figure 6 (b) is the case of non-uniform distribution.

[0035] Figure 7 This is a flow chart of the improved multi-agent algorithm described in an embodiment of the present invention.

[0036] Figure 8 It is a schematic diagram of the optimization effect of 200+30j spaces according to an embodiment of the present invention.

[0037] Fig. 9 The figure is a schematic diagram for comparing the accuracy of different algorithms described in the embodiments of the present invention.

[0038] Fig.10 This is a database workflow diagram described in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below by the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0040] It should be understood by those skilled in the art that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0041] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with that in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0042] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements and / or groups thereof.

[0043] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. Different embodiments or examples described in this specification and features of different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

[0044] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings, and the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0045] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0046] The present invention provides a method for solving impedance matching parameters by deep reinforcement learning based on multi-agent joint optimization of space segmentation. On the basis of equating the impedance matching space solution problem to the geometric space position search problem, the matching space is segmented at the first level, and multi-agent distributed optimization is introduced. The maximum parallel processing of the algorithm is achieved through software programming and combined with the processor architecture. On the basis of the first-level space segmentation, the probability of the optimal solution appearing in each subspace is counted, and the two subspaces with the highest probability of the optimal solution appearing are selected for the second level segmentation. Each subspace is divided into two small spaces, and the number of agents is increased. The remaining subspaces remain unchanged. Combined with the local database, the optimal solution is stored. For repeated matching situations, the table is directly looked up without learning; for new matching situations, training is performed, and the optimal solution is uploaded, which is convenient for direct table lookup and retrieval when it appears again next time, further improving the solution efficiency.

[0047] Example 1

[0048] In this embodiment 1, a power line impedance matching system based on space segmentation is first provided, including: an acquisition module, used to obtain a power line terminal load value; a query module, used to query the capacitor, inductor component values ​​and the optimal objective function value corresponding to the acquired power line terminal load value in the database based on a database query function; wherein a storage table is stored in the database, and the storage table stores the optimal values ​​of capacitors and inductors under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot query the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

[0049] Specifically, in this embodiment, the establishment of the above system includes the following steps:

[0050] First, establish a new connection in Navicat software, and fill in the basic information of the database in the dialog box. Set the connection name, host name, port number, user name and password, etc. Then, in pycharm software, introduce the datetime library, use the pymysql.connect statement to connect Navicat and MySQL. After the database is established, set the key information of the table, and establish storage tables in narrowband environment and broadband environment respectively. Different storage tables use different keywords. By changing the power line terminal load value, run the DQN program multiple times, get the optimal values ​​of capacitors and inductors under the optimal matching condition, and store the result information of the program running in different tables, and complete the database design. Establish a database query function selectdata, input the real and imaginary values ​​of the power line terminal load, and query in the database. If the data information is stored in the database, directly output the corresponding capacitor and inductor component values ​​and the optimal objective function value when the optimal match occurs. If the data does not exist in the database, output "no such data" and mobilize the deep reinforcement learning algorithm program for optimization. The database initially does not have matching information for the load. After running the algorithm program, the narrowband impedance matching information library and broadband impedance matching information library are newly added, and the running results are uploaded in time to ensure that the database information can be directly retrieved the next time the same load value is entered.

[0051] In this embodiment, the above-mentioned system is used to implement a power line impedance matching method based on space segmentation, including: using an acquisition module to obtain a power line terminal load value; using a query module to query the database based on a database query function for the capacitor, inductor component values ​​and the optimal objective function value corresponding to the optimal matching of the acquired power line terminal load value; wherein a storage table is stored in the database, and the storage table stores the optimal values ​​of capacitors and inductors under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot find the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

[0052] In this embodiment, when the matching network is optimized for parameters, various intelligent algorithms are often used for parameter optimization, such as genetic algorithm, particle swarm algorithm and neural network algorithm, etc. However, the modeling of these intelligent algorithms is often complex, and the required iteration time is long, the amount of calculation is large, and sometimes they fall into the problem of local optimality. To solve the above problems, this embodiment proposes a multi-agent joint optimization deep reinforcement learning algorithm based on space segmentation to solve impedance matching.

[0053] like Figure 1 As shown, the multi-agent joint optimization deep reinforcement learning impedance matching solution method based on space segmentation includes the following steps: (1) Selecting a Π-type matching network for impedance matching, firstly, the spatial solution of impedance matching is equivalent to a position search problem in a geometric space, and the space is divided into 8 uniform subspaces to prepare for the subsequent multi-agent simultaneous optimization. From the spatial segmentation method of step (1), it can be seen that the search spaces are different, and there is no need for the agents to communicate with each other. Therefore, the present invention adopts a fully distributed training method for multi-agent optimization, and adopts a multi-process parallel method to achieve multi-agent simultaneous optimization, and uses a deep reinforcement learning algorithm to obtain the optimal solution for each space. By step (2), the optimal solution of impedance matching is found in the 8 spaces, the probability of the optimal solution is sorted, and the two spaces with the highest probability of the optimal solution are selected for the second-level segmentation, and the number of agents is increased, and the other spaces remain unchanged. After the second-level segmentation, the 8 uniformly divided spaces become 10 unevenly divided spaces. After the second-level segmentation, the number of agents is increased, and parameter optimization is performed in the unevenly divided space. Use the MySQL database platform to store the optimal solutions, set keyword information for different scenarios, and facilitate query and retrieval of data. There is no need to learn repeated matching situations and you can directly look up the table. For new situations, train and upload the optimal solutions to the database to facilitate the next table lookup.

[0054] With the rapid development of PLC and the rise of smart grids, the problem of mismatching between signal source and load impedance seriously affects the quality of power line carrier communication, so the impedance matching problem of power line carrier communication has become an important research direction. Since the mismatch between the signal source load and the terminal load in the power line communication system leads to low load receiving power, the design and method of adaptive impedance matching network plays a vital role in power line carrier communication. Selecting a suitable matching network structure is a key step in impedance matching. To solve the above problems, in this embodiment, three commonly used structures are compared: L-type network, T-type network, and Π-type network. The L-type matching network has a simple network structure and uses two passive components. Its advantage is that the cost is low and it is only suitable for narrowband matching scenarios. The disadvantage is that the matching range is limited and there is a matching forbidden area. The T-type network uses three passive components to adjust the matching terminal load value, and adds a node on the basis of the L-type network to form a T-type network. The advantage is that compared with the L-type matching network, there is no problem of matching forbidden area, and the terminal load impedance value can be matched arbitrarily. However, there are problems of limited frequency selectivity and limited low-frequency performance. The Π-type matching network is similar to the T-type network in principle and composition. However, for some applications that need to process low-frequency signals, the Π-type network is better than the T-type structure. In comparison with the three commonly used structures, this embodiment adopts a Π-type matching network. The Π-type matching network has the advantages of strong harmonic suppression ability, relatively simple circuit structure, wide adaptability, and no matching forbidden area. It can better match with changing load impedance and changing frequency. Figure 2 shown.

[0055] The calculation expressions between the output voltage and transfer function of the matching network and the impedance are shown in formula (1) and formula (2).

[0056]

[0057] Where: V o (f) represents the output voltage; V i (f) represents input voltage; Z CH Indicates the impedance value of the channel; Z C1 , Z C2 are the impedance values ​​of capacitors C1 and C2 respectively; Z L is the impedance value of the inductor L; Z d It is the communication terminal load.

[0058] According to different application scenarios, power line impedance matching problems can be divided into narrowband matching and broadband matching. Narrowband matching is mainly for fixed frequency and variable load; broadband matching is mainly for fixed load and different frequency. The following matching problems are all based on the above-mentioned Π-type network. In order to ensure the maximum output power of the signal source at the transmitter, the internal resistance of the signal source and the equivalent impedance formed by the load and the matching network must be equal to the impedance conjugate to achieve the best matching condition.

[0059] Impedance matching in narrowband situations, such as Figure 2 As shown, the communication terminal load Z d =R+jX; inductor L, capacitor C1, and capacitor C2 form a π-type matching network connecting the power supply and the load. For narrowband conditions, the condition for achieving optimal impedance matching is input impedance Z in With load Z d Achieve conjugate matching. The resistance values ​​of each segment are shown in formula (3), formula (4), and formula (5).

[0060]

[0061] Z2=jwZ L +Z1 (4)

[0062]

[0063] Where R is the real part of the impedance of the power line load; X is the imaginary part of the impedance of the power line load; w is the operating frequency of the circuit; Z1 is the load Z d The equivalent resistance after being connected in parallel with capacitor C2; Z2 is the equivalent total resistance after the equivalent resistance Z1 and inductor L are connected in series; Z in It is the input impedance formed by the load and the π-type matching network.

[0064] For broadband impedance matching, the concept of transducer power gain (TPG) needs to be introduced. The ratio of the actual power absorbed by the load to the maximum power is defined as the transducer power gain TPG, as shown in formula (6). For broadband conditions, the condition for achieving optimal impedance matching is R in =R CH , X in =-X CH , that is, f itness =1, as shown in formula (7).

[0065]

[0066]

[0067] Among them, T PG (Wk ) is the frequency W k The conversion power gain at A is the received power of the load when the transfer power gain is maximum; P L is the actual power absorbed by the load; R CH (W k ),X CH (W k ) are the real and imaginary parts of the channel impedance respectively; R in (W k ),X in (W k ) are the real and imaginary parts of the equivalent input impedance of the load and matching network respectively; f itness is the objective function for measuring the broadband matching standard; m is the number of frequency points in the entire frequency band.

[0068] When performing impedance matching, in order to find the optimal matching parameters that meet the narrowband and broadband conditions, this embodiment uses a deep reinforcement learning algorithm to perform parameter optimization. The relevant principles are as follows:

[0069] Deep reinforcement learning (Deep Q-Networks, DQN), first initialize two evaluation networks Q and target networks And initialize the weight parameters of the neural network In each interaction process, a state s is obtained. t , the agent selects the action with the greatest value for output, so the action value function Q under the strategy π is defined as π (s,a), as shown in formula (8).

[0070]

[0071] Where E is the mean square error function; For state s t+1 The mean square error function when r t+1 is the reward value at time t+1; γ is the discount rate, which indicates the impact of subsequent states on the return of the current state; s t is the state of the agent at time t; a t is the action performed by the agent at time t.

[0072] The agent takes the action with the largest Q value as the optimal action and obtains the optimal action value function expression, as shown in formula (9).

[0073]

[0074] in, Execute the optimal policy π for the agent * When in state s tExecute the best action a t The expected value of the maximum cumulative return obtained after

[0075] After the agent receives the reward, it will perform the optimal action and send the obtained experience sample (s t ,a t ,r t ,s t+1 ) into the replay experience pool. The agent randomly extracts small batches of experience sample data from the experience pool.

[0076] The next step is to calculate the objective function for each state, the objective function Y i The calculation is shown in formula (10).

[0077]

[0078] Among them, θ i is the neural network weight parameter; Q(s t+1 ,a t+1 |θ i ) is the value function for evaluating the network.

[0079] Calculate the target value of each state through the target network Reward update after performing an action Value. Loss function L(θ i ) is shown in formula (11).

[0080]

[0081] Among them, Q(s t ,a t |θ i ) is the value function of the target network; is the experience sample (s t ,a,r,s t+1 ). By adjusting the parameter θ in the loss function i By calculating the partial derivative, we get the gradient formula as shown in formula (12).

[0082]

[0083] Use gradient descent to update the parameters θ of the evaluation neural network i After N iterations, the current target network weight parameters are passed to the evaluation network to update the weight parameters. The ultimate goal of the DQN algorithm is to find the current target network weight parameters as the number of iterations increases. The optimal position when .

[0084] The following (1)-(3) supplementary explanations are given for the above DQN process in a specific impedance matching application scenario of this embodiment:

[0085] (1) The state refers to the position of the agent after each action and the value of the objective function at that position. In narrowband scenarios, the state is the position of the coordinate point and the input equivalent impedance value corresponding to the coordinate. In broadband scenarios, the state is the position of the coordinate point and the f corresponding to the coordinate point. itness .

[0086] (2) The action described above refers to the change in the value of the adjustable capacitor and inductor. In the algorithm, the action of each adjustable device is defined as two, with a total of three devices and six actions, namely (1,0,0), (-1,0,0), (0,1,0), (0,-1,0), (0,0,1), and (0,0,-1).

[0087] The reward mentioned in (3) is shown in formula (13) and formula (14). In the narrowband scenario, the reward function F reward The expression is:

[0088]

[0089] Among them, t arget The equivalent input impedance value calculated for the deep reinforcement learning algorithm. The difference between the equivalent input impedance and the signal source input impedance is used as the variable of the reward function.

[0090] In the broadband scenario, the reward function F reward The expression is:

[0091]

[0092] The algorithm calculates the objective function f for each coordinate point itness , the objective function f itness With the ideal state f itness The difference when 1 is used as the variable of the reward function.

[0093] The algorithm flow of deep reinforcement learning is as follows Figure 3 The specific steps are as follows:

[0094] Step 1: Build the experimental environment and initialize the environment. Set the state space, action space, reward value, objective function, and the learning rate, number of hidden layers, experience pool capacity, decay rate, etc. of deep reinforcement learning.

[0095] Step 2: In the exploration phase, the agent interacts with the environment to obtain state and action information, and uses the ε-greedy search strategy, which means that while the network makes decisions, it can also explore other possible optimal behaviors with a certain probability.

[0096] Step 3: Storing and extracting experience samples. The acquired experience samples are stored in the experience pool, the experience samples in the experience pool are continuously updated, and small batches of experience samples are randomly extracted as inputs to the evaluation network and the target network.

[0097] Step 4: Calculate the loss function. Calculate the mean square error of the two networks as the loss function, and use the gradient descent algorithm to update the weight parameters of the evaluation network.

[0098] Step 5: Determine whether the function value obtained by the algorithm reaches the target function value at the optimal match. If so, proceed to the next step; otherwise, repeat the process.

[0099] Step 6: Determine the end of the iteration. Determine the absolute value of the difference between the cumulative evaluation Q values ​​of adjacent iterations |Q n+1 -Q n |Is it less than a certain threshold? If so, the iteration ends; otherwise, the iterative training continues.

[0100] Step 7: Determine and save the optimal strategy and end the program.

[0101] When performing impedance matching and using intelligent agents to search for optimal parameters, it is particularly important to increase the diversity of optimization and avoid falling into local optimal solutions. In addition, a search space that is too large will also lead to problems such as unsatisfactory optimization of the intelligent agent and low optimization efficiency. Improving the efficiency of the intelligent agent in exploring spatial solutions is also an urgent problem to be solved. To solve the above problems, this embodiment adopts a multi-agent joint optimization deep reinforcement learning method based on space segmentation. The optimization space of the intelligent agent is divided to solve the problem of unsatisfactory optimization caused by too large a space, and multiple subspace agents are optimized simultaneously in parallel, which improves the problem of low efficiency in finding the optimal solution. The multi-agent algorithm flow chart is shown below. Figure 4 The specific steps are as follows:

[0102] Step 1: The adjustable ranges of device capacitance C1, capacitance C2, and inductance L correspond to the ranges of the search space x, y, and z axes, respectively. The space of Π-type impedance matching is divided into a first-level space, that is, the search space size in the DQN algorithm is divided into 8 uniform subspaces, in preparation for the subsequent simultaneous optimization of multiple agents. Step 2: Input the real and imaginary parts of the impedance, and transmit the impedance values ​​to each space through the central controller. A single agent is stored in each space to interact with the environment. Step 3: The single agent in each space uses the DQN algorithm to update the parameters of the target network and the evaluation network. The situation of each divided search space is different, so a fully distributed training method is adopted to rely on its own observation to determine the optimal action. And a multi-process parallel method is used to realize the function of multi-agents searching together in the space, and find the optimal result and optimal path of each space respectively. Step 4: Save the optimal result and optimal path obtained from the training of each space to obtain the optimization result. Step 5: After the training is completed, the agent reads the optimization result and transmits the optimal result and the optimal path model to the central controller. The central controller compares the transmitted result with the error function. If the result is less than the relative error, the result with the smallest error value in each space is read as the optimal matching result. If the training results of each space are greater than the error function, the agents in each space will search again. Step 6: Output the optimal result of multi-agent parallel optimization.

[0103] In this embodiment, additional explanation is required for step three:

[0104] (1) Multi-agent Deep Q-Networks (MADQN), multiple agents learn and interact in the environment at the same time. Multi-agent deep reinforcement learning (MADRL) is divided into fully distributed, fully centralized, centralized training, and distributed execution according to the information interaction relationship between agents. Fully distributed refers to distributed training and distributed execution. Each agent has its own reward function and relies entirely on its own observation of the environment to learn the decision-making strategy in its own environment. Each agent is independent of each other and does not communicate with each other. It updates the parameters of the target network and the evaluation network based on its own interactive experience samples with the environment. For the impedance matching problem in this embodiment, the situation of each divided search space is different, and there is no need for the agents to communicate with each other. Therefore, a fully distributed training method is adopted to rely on their own observations to determine the optimal action. The multi-agent fully distributed training method effectively solves the problem of large search space and unsatisfactory optimization of agents. (2) In order to realize the function of multiple agents searching together in the space, it is necessary to use Python language to realize the simultaneous running of multiple programs based on the DQN algorithm. In this embodiment, a multi-process parallel method is used for impedance matching. The process is the basic unit of dynamic execution of the operating system and the basic unit of resource allocation and scheduling in the operating system. A program can generate multiple processes, and the memory space between multiple processes is independent and does not affect each other. This method is applicable to various operating systems, simple to implement, easy to understand and maintain; each process has independent resources and memory, which can be used to save the running intelligent agent strategy. A problem in one process will not affect the operation of other processes, and will not affect the optimization of other intelligent agents. The parallel principle diagram of multiple processes is as follows: Figure 5 In this embodiment, the multi-process operation mode is parallel. Parallel means that when the computer has a multi-core CPU, multiple processes are processed by different cores at the same time, and multiple processes run tasks at the same time.

[0105] This embodiment uses a 4-core CPU, which can evenly distribute the processes to two CPU cores, and process the tasks of each process separately. The core usage of each process is as follows: Figure 6 (a) is shown. The processes can also be distributed unevenly, and the kernel usage of each process is shown as Figure 6 (b) as shown.

[0106] In this embodiment, the feasibility of using a multi-agent joint optimization deep reinforcement learning method based on space segmentation to improve the problem of unsatisfactory optimization and low optimization efficiency caused by too large a space is verified through simulation. The simulation results are compared as follows:

[0107] Analysis of simulation results of narrowband power line system:

[0108] The optimization results of the DQN algorithm and the MADQN algorithm were compared. Different real and imaginary impedance values ​​were selected in the range of 10Ω-700Ω, and the two programs were run respectively to compare the optimization results. As shown in Table 1. The optimized MADQN algorithm has a smaller search range, and the agent can better learn the optimal strategy. Compared with the DQN algorithm, the optimization effect is improved, and the error with the optimal value is small.

[0109] Table 1 Comparison of optimal values

[0110]

[0111] Table 2 records the training time and number of trainings of each algorithm. It can be seen from Table 2 that after the matching space is divided, the training time of the MADQN algorithm is effectively shortened. When the number of trainings is reduced to 3000 or more, the training results tend to be stable, which is significantly reduced compared to the single-agent DQN algorithm.

[0112] Table 2 Algorithm time comparison

[0113]

[0114] Analysis of simulation results of broadband power line system:

[0115] At a fixed frequency of 340kHz, by changing the impedance value of the load, the optimization effects of the DQN and MADQN algorithms are compared, and the optimal matching values ​​obtained by the algorithms are recorded and compared as shown in Table 3. Compared with the DQN algorithm, the MADQN algorithm has a higher optimization value and better optimization effect. This shows that the optimization effect of the improved algorithm is improved, the relative error is smaller, and the matching accuracy is higher.

[0116] Table 3 Comparison of optimization values ​​between DQN and MADQN

[0117]

[0118] The number of training iterations and the time required for training when the running results of the MADQN algorithm and the DQN algorithm tend to be stable are recorded, as shown in Table 4. The training time of the MADQN algorithm is effectively shortened, and when the number of training times is reduced to 2000 or more, the training results tend to be stable.

[0119] Table 4 Algorithm time comparison

[0120]

[0121] From the above, it can be seen that the multi-agent optimization in this embodiment is to evenly divide the search space into 8 subspaces, and the impedance matching effect can be further improved based on the obtained results. After the multi-agent optimization, if it is found that the optimization effects of different spaces are quite different, then the space with better optimization effect can be selected in the impedance matching to perform a secondary division, that is, on the basis of the uniform division of the first-level space, the probability of the optimal solution appearing in each subspace is counted, and several subspaces with the highest probability of the optimal solution appearing are selected for the second-level division, which are divided into two spaces respectively to achieve uneven division of the space, reduce the search range, and further refine the search. Because the search effect of the remaining spaces is poor, the x, y, and z axis ranges are larger, and the agent performs a rough search to improve the impedance matching effect. The flow chart of the multi-agent improved algorithm proposed in this embodiment is as follows. Figure 7 As shown, including:

[0122] Step 1: On the basis of the first-level space segmentation of the space with Π-type impedance matching, the space with better spatial optimization effect is segmented at the second level, and the two spaces are equally divided into two subspaces, that is, the search space size in the DQN algorithm is divided into 10 uneven subspaces, in preparation for the subsequent simultaneous optimization of multiple agents. Step 2: Input the real and imaginary parts of the impedance, and transmit the impedance value to each space through the central controller. A single agent is stored in each space to interact with the environment. Step 3: The single agent in each space uses the DQN algorithm to update the parameters of the target network and the evaluation network. The situation of each segmented search space is different. The fully distributed training method is adopted to rely on its own observation to determine the optimal action. And the multi-process parallel method is used to realize the function of multi-agents searching together in the space. Find the optimal result and optimal path of each space respectively. Step 4: Save the optimal result and optimal path obtained by training each space to obtain the optimization result. Step 5: After the training is completed, the agent reads the optimization result and transmits the optimal result and the optimal path model to the central controller. The central controller compares the transmitted result with the error function. If the result is less than the relative error, the result with the smallest error value in each space is read as the optimal matching result. If the training results of each space are greater than the error function, the agents in each space will search again. Step 6: Output the optimal result of multi-agent parallel optimization.

[0123] When simulating and verifying the evenly divided multi-agent parallel optimization method, it was found that when impedance matching is performed in a broadband environment, the optimal values ​​appear mainly in two spaces. For example, when the load is 200+30j, Figure 8As shown in the figure, the optimal solutions are mainly concentrated in space 1 and space 5, so the secondary division of these two spaces can further improve the matching effect. Compared with MADQN, the search range of some spaces is shortened, two search spaces are added, and the search effect of the intelligent agent is significantly higher than other algorithms. It is not only feasible on a theoretical basis, but also further proved the feasibility of the scheme in simulation verification. Fig. 9 shown.

[0124] After the space is unevenly divided, the number of trainings for space 1.1, space 1.2, space 5.1, and space 5.2 can be shortened to 500 times, the training time is shortened to 25 seconds (0.41 minutes), and the FPS of the total space can reach 105, and the algorithm time is effectively shortened. The time comparison of different algorithms is shown in Table 5.

[0125] Table 5 Time comparison of different algorithms

[0126]

[0127] Repeated calculations will cause a waste of time and resources. When the amount of data is large, repeated model training and calculations will increase the burden on the system. To solve the above problems, further improvements are made in this embodiment. In combination with the local database, the optimal solution that has been obtained is stored. For repeated matching situations, the stored results can be directly retrieved from the database without repeated learning. The table lookup mechanism simplifies the matching process, making the system more efficient and easy to maintain. In addition, new data can be uploaded using the database. When the database initially has no matching information for the load, the newly added information will be added to the narrowband impedance matching information library and the broadband impedance matching information library after running the algorithm program. Avoid unnecessary model training and calculations, save time and resources, and further improve the solution efficiency.

[0128] This embodiment uses the MySQL database, which is a large database that can process tens of thousands of records and has the advantages of fast processing speed and low cost. Fig.10As shown. The specific steps are as follows: Step 1: First, establish a new connection in the Navicat software, and fill in the basic information of the database in the dialog box. Set the connection name, host name, port number, user name and password, etc. Then, in the pycharm software, introduce the datetime library, and use the pymysql.connect statement to connect Navicat and MySQL. Step 2: After the database is established, set the key information of the table, and establish storage tables under narrowband environments and broadband environments respectively. Different storage tables use different keywords. Step 3: By changing the power line terminal load value, run the DQN program multiple times to obtain the optimal values ​​of capacitors and inductors under the optimal matching condition, and store the result information of the program running in different tables, and complete the database design. Step 4: Establish a database query function selectdata, input the real and imaginary values ​​of the power line terminal load, and query in the database. If the data information is stored in the database, directly output the corresponding capacitor and inductor component values ​​and the optimal objective function value when the optimal match occurs. If the data does not exist in the database, output "no such data" and mobilize the deep reinforcement learning algorithm program for optimization. Step 5: Initially, the database does not have matching information for the load. After running the algorithm program, the narrowband impedance matching information library and broadband impedance matching information library are newly added, and the running results are uploaded in time to ensure that the database information can be directly retrieved the next time the same load value is entered.

[0129] Example 2

[0130] This embodiment 2 provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the power line impedance matching method based on space division as described above is implemented. The method includes:

[0131] Get the power line terminal load value;

[0132] Based on a database query function, the capacitor and inductor component values ​​and the optimal objective function value corresponding to the acquired power line terminal load value at the optimal matching are queried in the database; wherein a storage table is stored in the database, and the storage table stores the optimal values ​​of the capacitor and inductor components under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot find the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

[0133] Example 3

[0134] This embodiment 3 provides a computer device, including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, the processor calls the program instructions to execute the power line impedance matching method based on space division as described above, the method comprising:

[0135] Get the power line terminal load value;

[0136] Based on a database query function, the capacitor and inductor component values ​​and the optimal objective function value corresponding to the acquired power line terminal load value at the optimal matching are queried in the database; wherein a storage table is stored in the database, and the storage table stores the optimal values ​​of the capacitor and inductor components under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot find the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

[0137] Example 4

[0138] This embodiment 4 provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes instructions for implementing the power line impedance matching method based on space division as described above, the method including:

[0139] Get the power line terminal load value;

[0140] Based on a database query function, the capacitor and inductor component values ​​and the optimal objective function value corresponding to the acquired power line terminal load value at the optimal matching are queried in the database; wherein a storage table is stored in the database, and the storage table stores the optimal values ​​of the capacitor and inductor components under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot find the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

[0141] In summary, the embodiment of the present invention aims at the problems that the DQN algorithm has long agent optimization time and slow training convergence speed when training the optimal strategy of intelligent learning. A multi-agent joint optimization deep reinforcement learning impedance matching solution method based on space segmentation is proposed. The search space is evenly divided, and there is an agent in each space. The multi-process parallel operation mode is adopted to make it only responsible for interacting and learning with the local space environment. The agents are not connected to each other. The optimal result of the operation is transmitted to the central controller, and the optimal result is selected for output. Compared with the DQN algorithm, it can effectively reduce the program running time, improve the problem of unsatisfactory agent optimization due to the large search space, and avoid poor optimization results due to insufficient training of the agent. The results of multi-agent simultaneous optimization after uniform space division show that in a broadband environment, the optimal solution of impedance matching is mainly concentrated in two spaces, so the two spaces are further divided twice, which reduces the search space and further searches accurately. The search results of the remaining spaces are poor, and the agents perform rough searches. The space is unevenly divided, and the search effect of the agent is significantly better than the evenly divided multi-agent distributed optimization. The MySQL database is used. For repeated matching situations, the stored results can be directly retrieved from the database without repeated learning. The table lookup mechanism simplifies the matching process, making the system more efficient and easy to maintain. The database can also be used to upload new data and continuously update the database storage content to avoid repeated learning, further saving time and resources and improving the efficiency of solving problems.

[0142] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative work on the basis of the technical solution disclosed in the present invention should be included in the scope of protection of the present invention.

Claims

1. A power line impedance matching method based on space division, characterized in that: include: Get the power line terminal load value; Based on a database query function, the capacitor and inductor component values ​​and the optimal objective function value corresponding to the acquired power line terminal load value at the optimal matching are queried in the database; wherein a storage table is stored in the database, and the storage table stores the optimal values ​​of the capacitor and inductor components under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot find the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

2. The power line impedance matching method based on space division according to claim 1, characterized in that: The training of the deep reinforcement learning algorithm model includes: on the basis of the first-level space segmentation of the space with Π-type impedance matching, the space with better spatial optimization effect is segmented at the second level, and the two spaces are divided into two subspaces respectively; the real and imaginary parts of the impedance are obtained, and the DQN algorithm is used in each space to update the parameters of the target network and the evaluation network respectively. The situation of each segmented search space is different. The fully distributed training method is used to rely on its own observation to determine the optimal action, and the optimal result and optimal path of each space are found respectively; the optimal result and optimal path obtained from the training of each space are saved to obtain the optimization result.

3. The power line impedance matching method based on space division according to claim 2, characterized in that: After the deep reinforcement learning algorithm model training is completed, the optimal result of each space is compared with the error function. If the result is less than the relative error, the result with the smallest error value in each space is read as the optimal matching result; If the training results of each space are greater than the error function, each space will be searched again.

4. The power line impedance matching method based on space division according to claim 3, characterized in that: The deep reinforcement learning algorithm includes: initializing two evaluation networks and the target network, and initializing the weight parameters of the neural network; the intelligent agent takes the action with the largest action-value function value as the optimal action, and obtains the optimal action-value function expression; after the intelligent agent obtains the reward, it will execute the optimal action and put the obtained experience samples into the playback experience pool; the intelligent agent randomly extracts small batches of experience sample data from the experience pool; calculates the objective function of each state, calculates the target value of each state, and updates the target value through the reward after the target network executes the action; uses the gradient descent method to update the parameters of the evaluation neural network; after N iterations, passes the current target network weight parameters to the evaluation network, updates the weight parameters, and as the number of iterations increases, finds the optimal position when the target value is equal to the evaluation value.

5. The power line impedance matching method based on space division according to claim 4, characterized in that: Set the state space, action space, reward value, objective function, and learning rate, number of hidden layers, experience pool capacity, and decay rate of deep reinforcement learning; the agent interacts with the environment to obtain state and action information, and uses the ε-greedy greedy strategy to search, that is, while the network makes decisions, it can explore other possible optimal behaviors with a certain probability; store the acquired experience samples in the experience pool, continuously update the experience samples in the experience pool, and randomly extract small batches of experience samples as inputs for the evaluation network and the target network.

6. The power line impedance matching method based on space division according to claim 5, characterized in that: Calculate the mean square error of the two networks as the loss function, and use the gradient descent algorithm to update the weight parameters of the evaluation network; determine whether the function value obtained by the algorithm reaches the target function value at the optimal match, if so, proceed to the next step; Otherwise, repeat the process.

7. A power line impedance matching system based on space division, characterized in that: include: An acquisition module, used for acquiring a power line terminal load value; A query module is used to query the capacitor and inductor component values ​​and the optimal objective function value corresponding to the optimal matching of the acquired power line terminal load value in the database based on a database query function; wherein a storage table is stored in the database, and the storage table stores the optimal values ​​of the capacitor and inductor components under the optimal matching conditions corresponding to different power line terminal load values ​​obtained by optimizing using a deep reinforcement learning algorithm model; if the database cannot find the matching information of the acquired power line terminal load value, the deep reinforcement learning algorithm model is used to obtain the optimal matching information of the acquired power line terminal load value, and the information is stored in the storage table to ensure that the optimal matching information is directly retrieved when the same power line terminal load value is obtained next time.

8. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the power line impedance matching method based on space division as described in any one of claims 1-6 is implemented.

9. A computer device, characterized in that: It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the power line impedance matching method based on space division as described in any one of claims 1-6.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the power line impedance matching method based on space division as described in any one of claims 1-6.

Citation Information

Cited By

  • Matching algorithm construction method and device of radio frequency power supply matcher

    CN121637099A