A method and device for generating flow channel planning of a hydraulic housing

Abstract

The present application provides a method and device for generating flow path planning for a hydraulic housing, which belongs to the field of hydraulic fluid control. It includes: initializing a reinforcement learning network according to the design information of the hydraulic housing, and constructing a hydraulic housing environment; calculating the first path point of the hydraulic housing according to the reinforcement learning network, interacting the first path point with the hydraulic housing environment, and calculating the attribute information of the first path point; judging whether the end point has been reached according to the attribute information corresponding to the path point, if the end point has not been reached, inputting the attribute information of the calculated path point into the reinforcement learning network, calculating the next path point, and interacting with the hydraulic housing environment according to the predicted next path point; determining the flow path according to all calculated path points, and establishing a hydraulic housing model according to the flow path. The method and device for generating flow path planning for a hydraulic housing provided in the present application can make full use of historical experience, greatly improve design efficiency, and shorten the design cycle.

Description

Technical Field

[0001] The present application relates to the field of hydraulic fluid control technology, and in particular to a method and device for generating hydraulic housing flow channel planning. Background Art

[0002] The hydraulic housing is one of the core components of the hydraulic system. It distributes hydraulic oil by connecting the oil cylinder and other control valves to control the actuator to perform different actions. It plays an important role in engineering machinery, transportation and even modern industry.

[0003] At present, the traditional design method still mainly relies on manual design and modeling, which has a large amount of repetitive work, a high degree of dependence on experience, low design efficiency, and unstable design quality. Modern industrial systems have higher and higher performance requirements for hydraulic control systems, resulting in more complex shapes of hydraulic housing parts and an increasing application of new materials. With the development of additive manufacturing technology, additive manufacturing of hydraulic housings is also being explored step by step. Limited by the principles and characteristics of additive manufacturing, additively manufactured products often need to be finely processed to meet higher surface roughness requirements in order to meet design requirements. Therefore, additive manufacturing products should also take into account the machinability requirements of subtractive manufacturing in the design engineering.

[0004] Under this situation, the existing technology proposes to use the A* algorithm for path planning and obstacle avoidance between two interfaces for additive manufacturing. However, the A* algorithm is a static algorithm, which essentially traverses all nodes to find a feasible path. This algorithm does not have the ability to learn, which makes it unable to use historical experience and takes a long time to find the path. Summary of the invention

[0005] In view of this, the present application provides a method and device for generating flow channel planning for a hydraulic housing, which can make full use of historical experience through reinforcement learning, greatly improve design efficiency, and shorten the design cycle.

[0006] Specifically, the present application is implemented through the following technical solutions:

[0007] In a first aspect, the present application provides a method for generating a hydraulic housing flow channel plan, the method comprising:

[0008] Initialize the reinforcement learning network according to the hydraulic housing design information and build the hydraulic housing environment;

[0009] Calculating a first path point of the hydraulic housing according to the reinforcement learning network, interacting the first path point with the hydraulic housing environment, and calculating attribute information of the first path point, the attribute information including at least scoring information;

[0010] Determining whether the end point has been reached according to the attribute information corresponding to the path point, if the end point has not been reached, inputting the attribute information of the calculated path point into the reinforcement learning network, calculating the next path point, and interacting with the hydraulic housing environment according to the predicted next path point;

[0011] A flow path is determined according to all calculated path points, and a hydraulic housing model is established according to the flow path.

[0012] The second aspect of the present application provides a hydraulic housing flow channel planning generation device, the device comprising a processing module, an interaction module, a judgment module and a building module; wherein:

[0013] The processing module is used to initialize the reinforcement learning network according to the hydraulic housing design information and construct the hydraulic housing environment;

[0014] The interaction module is used to calculate a first path point of the hydraulic housing according to the reinforcement learning network, interact the first path point with the hydraulic housing environment, and calculate attribute information of the first path point, wherein the attribute information at least includes scoring information;

[0015] The judgment module is used to judge whether the end point has been reached according to the attribute information corresponding to the path point. If the end point has not been reached, the attribute information of the calculated path point is input into the reinforcement learning network, the next path point is calculated, and the next path point is predicted and interacts with the hydraulic housing environment;

[0016] The establishment module is used to determine the flow path according to all calculated path points, and establish the hydraulic housing model according to the flow path.

[0017] The hydraulic housing flow path planning generation method and device provided in the present application automatically generate path points through a reinforcement learning network, which reduces the time and labor intensity of manual design and greatly improves the design efficiency. In this process, through the feedback of each step, the reinforcement learning network can dynamically adjust the path points, adaptively adjust the path when the environment changes, and generate the optimal path. In addition, the interaction between each path point and the hydraulic housing environment can provide real-time feedback information, making full use of historical experience and shortening the time of design iteration. In terms of quality, the algorithm is used to perform global design iteration, which improves the design quality and avoids the limitations of manual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a first embodiment of a method for planning and generating a flow channel for a hydraulic housing provided in the present application;

[0019] Figure 2 A schematic diagram of an action ball shown as an exemplary embodiment of the present application;

[0020] Figure 3A A schematic diagram of a rectangular cube shown as an exemplary embodiment of the present application;

[0021] Figure 3B for Figure 3A Schematic diagram of information storage on the rectangular cube face shown in;

[0022] Figure 4A A schematic diagram of merging two rectangular cubes shown as an exemplary embodiment of the present application;

[0023] Figure 4B for Figure 4A Schematic diagram of the combined results;

[0024] Figure 5 A schematic diagram of the calculation principle of the threshold value of the machinable angle shown in an exemplary embodiment of the present application;

[0025] Fig. 6A A schematic diagram of environmental scanning information shown as an exemplary embodiment of the present application;

[0026] Figure 6B For the purpose of illustration Fig. 6A Part of the environmental scanning information result map;

[0027] Figure 7 A schematic diagram showing a new flow channel point approximated to the projection point of the point on the installation vector according to an exemplary embodiment of the present application;

[0028] Figure 8 A schematic diagram of a feasible cone region is shown for an exemplary embodiment of the present application;

[0029] Fig. 9A A schematic diagram of all paths after iteration shown in an exemplary embodiment of the present application;

[0030] Fig. 9B for Fig. 9A A schematic diagram showing the path with the highest score among all the paths;

[0031] Fig.10 This is a structural schematic diagram of embodiment 1 of the hydraulic housing flow channel planning and generation device provided in this application. DETAILED DESCRIPTION

[0032] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0033] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] Specific embodiments are given below to introduce the technical solution of the present application in detail.

[0036] Figure 1 This is a flow chart of the first embodiment of the method for planning and generating flow channels of a hydraulic housing provided by this application. Figure 1 , the method provided in this embodiment may include:

[0037] S101. Initialize a reinforcement learning network according to the hydraulic housing design information and construct a hydraulic housing environment.

[0038] It should be noted that hydraulic housings are usually used in hydraulic systems to protect and accommodate hydraulic components, such as valves, pumps, and hydraulic oil circuits. In addition to protecting hydraulic components, they also connect hydraulic channels, seal, and maintain pressure.

[0039] It should be noted that the hydraulic housing design information is manually input information, which can be specifically divided into two aspects: design space and flow channel information. Among them, the design space can be abstracted as a combination of several rectangular spaces, so by picking up the diagonal points of the rectangle, the coordinate range of the rectangular space and the equations of each face can be obtained, where the coordinate range is used to determine whether the path point is in the design space, and the face equation is used to calculate the feasible step length of the path point in each direction. The flow channel information mainly includes the initial direction and depth of the flow channel, which is mainly for the installation requirements of external components such as cartridge valves. The inlet and outlet of the flow channel are used to set the starting and end points of the path finding. The pressure flow of the flow channel is mainly used to calculate the inner diameter and outer diameter of the flow channel.

[0040] It should also be noted that, for example, in one embodiment, the reinforcement learning network can be an A3C reinforcement learning network, wherein A3C (Asynchronous Advantage Actor-Critic) is an advanced algorithm for reinforcement learning, which plays a significant role in solving decision-making problems in large-scale and complex environments. Its core idea is to accelerate the learning process and improve the stability and robustness of the strategy by training multiple agents in parallel.

[0041] Specifically, the main components of the A3C reinforcement learning network include: Actor-Critic architecture, advantage function, asynchronous update, and multi-task learning. Among them, the Actor (actor) in the Actor-Critic architecture is responsible for generating actions (i.e., strategies) and selecting the next action to be taken according to the current state. The Critic (evaluator) evaluates the performance of the current strategy and calculates the value of the action (i.e., state-action value function); the advantage function is used to reduce estimation errors, thereby improving the efficiency of strategy updates; multi-task learning is used to share experience between multiple tasks, accelerate the learning process, and improve performance on different tasks; asynchronous update is used to run multiple agents in parallel, each agent explores and learns in different copies of the environment, and these agents send their respective experiences to the global network for asynchronous updates, which can improve sample efficiency and avoid sample correlation problems common in traditional reinforcement learning algorithms.

[0042] The specific implementation process of the common A3C reinforcement learning network is as follows:

[0043] (1) Initialize the global network and multiple local networks.

[0044] (2) Each agent runs independently in its local environment and generates state-action sequences.

[0045] (3) Use the local network to calculate the gradient and accumulate it.

[0046] (4) Asynchronously update global network parameters.

[0047] (5) Reset local network parameters to global network parameters.

[0048] (6) Repeat steps (2)-(5) until a stopping condition is reached (such as a fixed number of iterations or a certain performance indicator).

[0049] It should also be noted that initializing the reinforcement learning network may include defining the network architecture, initializing parameters, defining the reward function and training the network, wherein defining the network architecture is to use a deep neural network (such as a convolutional neural network CNN or a recursive neural network RNN) to build a reinforcement learning model, which may include defining the input layer, hidden layer and output layer structure of the network; initializing parameters may include setting hyperparameters such as weights, biases and learning rates; defining the reward function may include designing a reward function for guiding the learning of the intelligent agent, and the reward function should consider processability scores, step ratio scores, direction rewards, etc.; training the network may include training using the A3C algorithm or other suitable reinforcement learning algorithms.

[0050] Specifically, in order to be suitable for the path-finding environment of reinforcement learning, this scheme builds a hydraulic shell environment suitable for flow path planning, which is based on the global rectangular coordinate system of the design geometry space 、Local rectangular coordinates of path points (The origin of the coordinate system is located at the latest path point of the path, and the x-axis is along the direction of the previous path , the XOZ plane is perpendicular to the XOY plane of the global coordinate system, and the Z axis is perpendicular to the previous path) and the motion ball coordinate system , the three planes of the motion ball coordinate system coincide with C2, such as Figure 2 As shown, P0 is the origin of the path point local rectangular coordinate system C2, and P1 is the origin of the action ball coordinate system C3.

[0051] Specifically, constructing a hydraulic housing environment at least includes designing a geometric space, an action space, and a state space; wherein the geometric space is used to define the physical layout in the hydraulic housing environment, including the shape, size, and coordinate positions of target path points of the hydraulic housing; the action space is used to define the conditions of all actions in each time step in the hydraulic housing; and the state space is used to define and represent the characteristics and attributes of each state in the hydraulic housing.

[0052] Specifically, the design geometric space includes:

[0053] (1) The geometric space is abstracted as a combination of rectangular cubes, wherein information of each rectangular cube includes a coordinate range and endpoints of a face; the coordinate range includes upper and lower bounds of the coordinates of the rectangular cube in the x-axis, y-axis, and z-axis directions in a global rectangular coordinate system.

[0054] Specifically, the geometric space is abstracted as a combination of rectangular cubes, where the information of each cube includes the coordinate range and the endpoints of the face. Figure 3A This is a schematic diagram of a rectangular cube shown in an exemplary embodiment of the present application, please refer to Figure 3A, assuming that the diagonal points of a rectangle are (0,0,0) and (10,10,10), the coordinate range storage information format is as follows: 0,10,0; 10,0,10. Among them, each data is the upper limit (0,10,0) and lower limit (10,0,10) of the coordinates of the rectangle in the x, y, and z directions.

[0055] (2) Simplifying the coordinates of each face of the rectangular cube according to the coordinate range of the rectangular cube and the parallel relationship and distance between the endpoints of the face and the coordinate plane.

[0056] Figure 3B for Figure 3A Schematic diagram of information storage on the rectangular cube face shown in . Please also refer to Figure 3A and Figure 3B ,in Figure 3B The penultimate digit indicates which coordinate plane the face is parallel to (0, 1, 2 correspond to parallel to the YOZ, XOZ, and XOY planes, respectively), the penultimate digit indicates the distance of the face from the parallel coordinate plane, and the rest are the simplified coordinates of the endpoints of the plane. Figure 3B In the figure, the endpoints of face2 plane are (10,0,0), (10,0,10), (10,10,10), (10,10,0), which is parallel to the YOZ plane and has the same x-coordinate, so the coordinates can be simplified to (0,0), (0,10), (10,10), (10,0).

[0057] (3) Combine the coordinate ranges of the two rectangular cubes into an array and display the two rectangular cubes together to determine the relationship between the new geometric space and the old geometric space.

[0058] Figure 4A This is a schematic diagram of merging two rectangular cubes according to an exemplary embodiment of the present application. Figure 4B for Figure 4A Please also refer to the schematic diagram of the combined results Figure 4A and Figure 4B When merging rectangles, for the coordinate range, the coordinate ranges of the two rectangles are stored in an array; when merging faces, by judging the relationship between the upper and lower bounds of the plane coordinates with the same parallel mark, it is possible to determine in which direction the new space is located in the old space and the overlapped surface of the two spaces. By subtracting the two planes, the merged plane can be obtained, and the remaining planes are stored in a list in sequence.

[0059] It should be noted that the action space contains three actions: pitch angle , deflection angle , step ratio R_ratio, and the spherical coordinate system , , R three dimensions correspond. In particular, is a discrete angle between [0°, 180°] with an interval of 5°, is a discrete angle between [0°, 180°] with an interval of 5°. The step percentage R_ratio is a decimal between 0 and 1, indicating the percentage of the maximum step length that can be advanced.

[0060] It should also be noted that a threshold of a processable angle can be calculated based on the action space. Figure 5 This is a schematic diagram of the calculation principle of the threshold value of the processable angle shown in an exemplary embodiment of the present application. Please refer to Figure 5 , D i Indicates the diameter of the interface, D p represents the flow channel diameter, h is the interface depth, and the processable angle is:

[0061] ;

[0062] Since the action space is a discrete angle, the angle obtained by dividing gamma by 5 is used as the threshold of the processable angle.

[0063] It should be noted that the state space includes a target vector and environmental scanning information; the target vector is a vector from the current path point to the end point of the path finding; the environmental scanning information is a feasible step length along the target vector; wherein, the process of obtaining the environmental scanning information includes: making a ray from the path point toward the hemisphere of the spherical coordinates along a specified angle; based on all the rays, calculating the points at which each ray and other paths meet the minimum flow channel interval, obtaining the maximum step length in the direction corresponding to the ray, forming a matrix, and using the matrix as the environmental scanning information result.

[0064] Fig. 6A This is a schematic diagram of environmental scanning information shown in an exemplary embodiment of the present application. Figure 6B For the purpose of illustration Fig. 6A Please also refer to the partial environmental scan information results diagram. Fig. 6A and Figure 6B , from the path point to the spherical coordinate system ∈[0°, 180°] hemisphere, along Draw a ray in the direction, find the point where the ray and other paths meet the minimum flow channel interval (if the interval is greater than the minimum interval, find the distance to the boundary), get the maximum step length in the direction, and finally get a 37*37 matrix as the final scanning information result. Figure 6B Only the 13*13 portion is shown.

[0065] S102, calculating a first path point of the hydraulic housing according to the reinforcement learning network, interacting the first path point with the hydraulic housing environment, and calculating attribute information of the first path point, wherein the attribute information includes at least scoring information.

[0066] Specifically, calculating the first path point of the hydraulic housing according to the reinforcement learning network includes: extracting the design information of the hydraulic housing, obtaining the inlet and outlet coordinates, initial direction and pressure and flow rate information of each flow channel; initializing the reinforcement learning network based on the extracted information; wherein the reinforcement learning network is used for reinforcement learning of hydraulic housing flow channel planning, and the input information of the reinforcement learning network includes the state space, the current path vector, the vector from the current path point to the end point and the scanning space result; the output information of the reinforcement learning network includes the action space. Specifically, please refer to the description above for the contents of the state space and the action space, which will not be repeated here. The scanning space result is the result of scanning the current hydraulic housing environment or design space, which is used to identify possible obstacles and feasible areas. For example, three-dimensional space information generated by laser scanning or other sensor data can be used. Specifically, the principle of scanning space (environmental scanning information) has been introduced above, which will not be repeated here.

[0067] It should also be noted that the path point needs to interact with the hydraulic housing environment. In specific implementation, the process of the path point interacting with the hydraulic housing environment includes:

[0068] (1) Before each iteration, the parameters and the hydraulic housing environment are initialized.

[0069] It should be noted that at the beginning of each iteration, the parameters and the hydraulic housing environment need to be initialized, which may include key information such as action base point, base vector, target vector, initial feasible environment, processable angle and action mask. In combination with the above example, when the feasible space is a 37*37 matrix, the pre-calculation method can be used to reduce the time of each iteration.

[0070] (2) The state space and the action mask are input into the reinforcement learning network, and the generated action is obtained and transmitted into the hydraulic shell environment.

[0071] Specifically, the state space and action mask are input into the neural network to generate three actions (pitch angle , deflection angle , step ratio R_ratio), and pass it into the hydraulic housing environment.

[0072] (3) Calculate a new path and path vector according to the generated action, and determine whether the flow channel has reached the end point based on the new path.

[0073] Specifically, after receiving the three actions, first find the feasible length corresponding to the angle in the feasible space according to the action index and multiply it by the step ratio to get the step length. In particular, in order to avoid the step length being too small or 0, when the current forward angle is not (90°, 90°) (that is, not along the direction of the previous path), the flow channel step length must not be less than 1.5 times the flow channel diameter; when the current forward angle is (90°, 90°), the step length should be greater than 0. Otherwise, the end mark is activated, the error code is 1, the status and scoring information are returned, and the current iteration round ends. After obtaining the step length, the action in the spherical coordinate C3 is converted into a new path coordinate point and a new path vector in the local rectangular coordinate C2 and the global rectangular coordinate C1.

[0074] Optionally, in the actual flow channel layout, the flow channel should be as right angles as possible if conditions permit. Since the step ratio is a continuous value, it is difficult to obtain a specific step ratio, so we approximate the flow channel points. Figure 7 As shown, when the angle between the line from the new flow channel point to the path finding end point and the flow channel installation vector is less than 5 degrees, the new flow channel point is approximated to the projection point of the point on the installation vector.

[0075] It should be noted that, in order to encourage the agent to generate a right-angle path, the ideal step length ratio step_des is calculated based on the flow channel base point and flow channel direction of the step, which is used for scoring. In theory, the flow channel angle should not be sharp, that is, "backtracking", so the new path vector should satisfy Any of the items is greater than 0, and Any item in is greater than 0 to ensure that the agent is gradually approaching the end point. is the end point coordinate in C2, is the action base point of this step. If it is not satisfied, the end mark is activated, the error code is 2, the status and score information are returned, and the current iteration round ends. Finally, the new path point is stored in the runner path point list.

[0076] Specifically, judging whether the flow channel reaches the end point based on the new path includes:

[0077] After the agent enters the feasible cone, it determines whether the line from the current path point to the end point meets the minimum spacing requirements with other channels and boundaries; if so, the current channel planning is determined to be over and the end mark is activated; if not, the reward mark is triggered and subsequent operations are performed.

[0078] It should be noted that in reinforcement learning, an agent refers to an entity that can perceive its environment, take actions, and learn optimization strategies through interaction with the environment. The goal of the agent is to find the optimal decision path by maximizing the cumulative reward.

[0079] Figure 8This is a schematic diagram of a feasible cone of an exemplary embodiment of the present application. Figure 8 , taking the path finding end point as the vertex and the axis of the installation hole as the axis of symmetry, a feasible domain cone can be obtained.

[0080] It should be noted that in order to speed up the iteration, when the agent enters the feasible cone, it indicates that the current path can be directly connected to the end point and meets the processability conditions. Therefore, it is necessary to determine whether the connection line from the current path point to the end point meets the minimum spacing requirements with other flow channels and boundaries. If so, the current flow channel planning is determined to be over, the end mark is activated, the error code is 3, and the status and score are returned; if not, the reward mark mid_mark is triggered (reward for entering the feasible cone), and subsequent operations are performed.

[0081] As an optional embodiment, the flow channel planning specifically includes determining whether the interface type is a preset shape, the preset shape is an annular groove, if it is a preset shape, determining the line between the latest planned flow channel point and the path finding end point, and taking whether the angle between the line and the flow channel installation vector meets the preset condition as the latest end point judgment condition to complete the flow channel planning. In particular, for a flow channel with an end interface as an annular groove, its flow channel path L2 must be perpendicular to the valve sleeve hole axis L1 where the annular groove is located, and there is no feasible domain cone for the interface. In this case, the flow channel planning needs to meet specific vertical requirements and cannot rely on the feasible domain cone to determine the feasibility of the path. Such a design can ensure the connection and function of the flow channel and the annular groove. In specific implementation, if the end point interface is an annular groove and there is no feasible domain cone, the intelligent agent must enter the approximate angle mentioned above (referring to when the angle between the line from the new flow channel point to the path finding end point and the flow channel installation vector is less than 5 degrees) before it is judged that it can directly reach the end point.

[0082] (4) Before reaching the end point, calculate the remaining length of the flow channel, scan the environmental information, and calculate the action score.

[0083] Due to process restrictions, the flow channel hole should meet the aspect ratio requirements. The remaining forward length in the current direction is calculated based on the preset aspect ratio and flow channel diameter. After calculating the remaining length of the flow channel, scan the feasible domain information for the next step and use the remaining length to cut the feasible step length in the (90°, 90°) direction. Calculate the action score based on the error code.

[0084] (5) Obtain the attribute information of the path points and update the reinforcement learning network.

[0085] It should be noted that the scoring information in the attribute information includes scoring rules. After calculation according to the scoring rules, the reinforcement learning network calculates the next path point based on the calculation results, and gradually optimizes the design of the flow path to update the reinforcement learning network.

[0086] Specifically, calculating the attribute information of the first path point includes:

[0087] When the status information is not an error, the main line reward is calculated based on the distance change between the current path point and the target point;

[0088] Calculate branch line bonus based on the deviation of the flow path's starting angle from the ideal angle;

[0089] Calculate the step length reward based on the current path step length and the ideal step length;

[0090] The runner spacing score is calculated based on the spacing between the path points in the runner and the comprehensive score of the runner path without error is calculated based on the machinability score of the runner;

[0091] When the status information is reaching the end point or not reaching the end point, the path completion is evaluated according to the angle and position of the flow path end point, and the end reward is calculated;

[0092] The total score of the flow path is obtained based on the main line, branch line, step length, end reward calculation results and the comprehensive score of the flow path without error reporting.

[0093] Specifically, the scoring rules are composed of mainline rewards and branch rewards, where the mainline rewards are positive rewards and branch rewards are negative rewards. When the status information is a non-error reward, the end mark is 0. At this time (ABCD...HIJ... etc. involved below are all constants):

[0094] ;

[0095] Where i=1, 2, 3, ..., As the main line reward,

[0096] For branch rewards,

[0097] The specific calculation method is as follows:

[0098] ;

[0099] in, Indicates the ratio of the distance from the target after the current action to the target distance of the previous step. Indicates the distance from the current path point to the end point. Indicates the distance from the previous path point to the end point;

[0100] ;

[0101] in, Indicates the starting angle score of the flow channel, and Indicates the angle in the action;

[0102] ;

[0103] in, Indicates the ratio of the current step length to the ideal step length for this angle, represents the step length of the action, Indicates the ideal step size for the current angle;

[0104] ;

[0105] in, It represents the comprehensive score of the flow channel;

[0106] ;

[0107] in, Indicates the score of the distance between the flow channel and other flow channels, Indicates the minimum channel spacing, which is a set constant; Indicates the flow channel spacing;

[0108] in, It represents the machinability score of the flow channel. The calculation method is to determine whether the intersection of the straight line where the flow channel path is located and the boundary is within the interface range of the flow channel. If so, it is P, otherwise it is a score less than P.

[0109] When the status information is reaching the end point, the end mark is 3:

[0110] ;

[0111] ;

[0112] in, It represents the angle between the last section of the flow channel and the axis of the terminal interface. A score that represents the angle between the last section of the flow channel and the axis of the terminal interface;

[0113] When the status information is not reached, the end mark is 1 or 2:

[0114] ;

[0115] in, is a normal number, is a negative constant.

[0116] In summary, the score of a flow channel is .

[0117] It should also be noted that after iterating a certain number of times at each path point and obtaining the best result as the current path point, it is necessary to repeatedly iterate for the next path point, and the next path point needs to be repeatedly calculated until the end of the flow channel is reached.

[0118] S103, judging whether the end point has been reached according to the attribute information corresponding to the path point; if the end point has not been reached, inputting the attribute information of the path point obtained by calculation into the reinforcement learning network, calculating the next path point, and interacting with the hydraulic housing environment according to the predicted next path point.

[0119] For details, please refer to the previous description of the process, which will not be repeated here.

[0120] S104, determining a flow path according to all calculated path points, and establishing a hydraulic housing model according to the flow path.

[0121] It should be noted that after the iteration is completed, the path with the highest score is selected as the final flow path, and the remaining results are kept for backup. Then, according to the flow path characteristics, the feature library template is called. For example, the NX Open script can be used to achieve the modeling of the hydraulic housing through parameter drive. Fig. 9A This is a schematic diagram of all paths after iteration according to an exemplary embodiment of the present application. Fig. 9B for Fig. 9A A schematic diagram of the path with the highest score among all paths is shown, where Fig. 9A The redder the line is, the higher the score is.

[0122] The hydraulic housing flow path planning generation method provided in this embodiment initializes a reinforcement learning network according to the hydraulic housing design information, and constructs a hydraulic housing environment, thereby calculating a first path point of the hydraulic housing according to the reinforcement learning network, interacting the first path point with the hydraulic housing environment, and calculating attribute information of the first path point, wherein the attribute information at least includes state information, a score, and an end mark, and then judging whether the end point has been reached according to the attribute information corresponding to the path point; if the end point has not been reached, inputting the attribute information of the calculated path point into the reinforcement learning network, calculating the next path point, and interacting with the hydraulic housing environment according to the predicted next path point; finally, determining the flow path according to all the calculated path points, and establishing a hydraulic housing model according to the flow path. In this way, the path points are automatically generated through the reinforcement learning network, which reduces the time and labor intensity of manual design and greatly improves the design efficiency. In this process, through the feedback of each step, the reinforcement learning network can dynamically adjust the path points, adaptively adjust the path when the environment changes, and generate the optimal path. In addition, the interaction between each path point and the hydraulic housing environment can provide real-time feedback information, making full use of historical experience and shortening the time of design iteration. In terms of quality, the algorithm is used to perform global design iteration, which improves the design quality and avoids the limitations of manual experience.

[0123] Corresponding to the aforementioned embodiment of a method for generating a hydraulic housing flow channel plan, the present application also provides an embodiment of a device for generating a hydraulic housing flow channel plan.

[0124] Fig.10 This is a structural diagram of the first embodiment of the hydraulic housing flow channel planning and generation device provided by this application. Fig.10 The device provided in this embodiment includes a processing module 1010, an interaction module 1020, a judgment module 1030 and an establishment module 1040; wherein,

[0125] The processing module 1010 is used to initialize the reinforcement learning network according to the hydraulic housing design information and construct the hydraulic housing environment;

[0126] The interaction module 1020 is used to calculate a first path point of the hydraulic housing according to the reinforcement learning network, interact the first path point with the hydraulic housing environment, and calculate attribute information of the first path point, wherein the attribute information at least includes scoring information;

[0127] The judgment module 1030 is used to judge whether the end point has been reached according to the attribute information corresponding to the path point. If the end point has not been reached, the attribute information of the calculated path point is input into the reinforcement learning network, the next path point is calculated, and the next path point is predicted and interacted with the hydraulic housing environment;

[0128] The establishment module 1040 is used to determine the flow path according to all calculated path points, and establish the hydraulic housing model according to the flow path.

[0129] The device of this embodiment can be used to perform Figure 1 The steps, specific implementation principles and implementation processes of the method embodiment shown are similar and will not be repeated here.

[0130] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0131] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0132] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for generating flow channel planning for a hydraulic housing, characterized in that: The method comprises: Initialize the reinforcement learning network according to the hydraulic housing design information and build the hydraulic housing environment; Calculating a first path point of the hydraulic housing according to the reinforcement learning network, interacting the first path point with the hydraulic housing environment, and calculating attribute information of the first path point, the attribute information including at least scoring information; Determining whether the end point has been reached according to the attribute information corresponding to the path point, if the end point has not been reached, inputting the attribute information of the calculated path point into the reinforcement learning network, calculating the next path point, and interacting with the hydraulic housing environment according to the predicted next path point; determining a flow path according to all calculated path points, and establishing a hydraulic housing model according to the flow path; The process of waypoints interacting with the hydraulic housing environment includes: Before each iteration begins, initializing parameters and the hydraulic housing environment; Inputting the state space and the action mask into the reinforcement learning network, obtaining the generated action and transmitting it into the hydraulic housing environment; Calculating a new path and a path vector according to the generated motion, and determining whether the flow path has reached an end point based on the new path; Before reaching the end point, calculate the remaining length of the flow channel, scan the environmental information and calculate the action score; Get the attribute information of the waypoints and update the reinforcement learning network; The determining whether the flow path reaches the end point based on the new path includes: After the agent enters the feasible cone, it determines whether the line from the current path point to the end point meets the minimum spacing requirements with other flow channels and boundaries; If satisfied, the current flow channel planning is determined to be finished and the end mark is activated; If not satisfied, the reward mark is triggered and subsequent calculations are performed.

2. The method according to claim 1, characterized in that Calculating the first path point of the hydraulic housing according to the reinforcement learning network comprises: Extracting the hydraulic housing design information, obtaining the inlet and outlet coordinates, initial direction and pressure and flow rate information of each flow channel; Initialize the reinforcement learning network based on the extracted information; Among them, the reinforcement learning network is used for reinforcement learning of hydraulic housing flow channel planning, and the input information of the reinforcement learning network includes state space, current path vector, vector from current path point to end point and scanning space result; the output information of the reinforcement learning network includes action space.

3. The method according to claim 1, characterized in that The construction of the hydraulic housing environment at least includes designing a geometric space, an action space and a state space; wherein the geometric space is used to define the physical layout in the hydraulic housing environment, including the shape, size and coordinate positions of the target path points of the hydraulic housing; the action space is used to define the conditions of all actions in each time step in the hydraulic housing; and the state space is used to define and represent the characteristics and attributes of each state in the hydraulic housing.

4. The method according to claim 3, characterized in that The design geometry space includes: The geometric space is abstracted as a combination of rectangular cubes, wherein the information of each rectangular cube includes a coordinate range and endpoints of a face; the coordinate range includes an upper bound and a lower bound of the coordinates of the rectangular cube in the x-axis, y-axis and z-axis directions in a global rectangular coordinate system; Simplifying the coordinates of each face of the rectangular cube according to the coordinate range of the rectangular cube and the parallel relationship and distance between the end points of the face and the coordinate plane; Combine the coordinate ranges of the two rectangular cubes into an array, merge and display the two rectangular cubes to determine the relationship between the new geometric space and the old geometric space.

5. The method according to claim 3, characterized in that: The state space includes a target vector and environment scanning information; the target vector is a vector from the current path point to the path finding end point; the environment scanning information is a feasible step length along the target vector; wherein the process of obtaining the environment scanning information includes: Draw a ray from the path point toward the hemisphere of spherical coordinates at the specified angle; Based on all the rays, the points where each ray and other paths satisfy the minimum flow channel interval are calculated, the maximum step length in the direction corresponding to the ray is obtained, a matrix is ​​formed, and the matrix is ​​used as the environment scanning information result.

6. The method according to claim 3, characterized in that The action space includes a pitch angle, a yaw angle and a step ratio, and corresponds to three dimensions of a spherical coordinate system respectively; and a threshold of a processable angle is calculated based on the action space.

7. The method according to claim 1, characterized in that The calculating of the attribute information of the first path point includes: When the status information is not an error, the main line reward is calculated based on the distance change between the current path point and the target point; Calculate branch line bonus based on the deviation of the flow path's starting angle from the ideal angle; Calculate the step length reward based on the current path step length and the ideal step length; The runner spacing score is calculated based on the spacing between the path points in the runner and the comprehensive score of the runner path without error is calculated based on the machinability score of the runner; When the status information is reaching the end point or not reaching the end point, the path completion is evaluated according to the angle and position of the flow path end point, and the end reward is calculated; The total score of the flow path is obtained based on the main line, branch line, step length, end reward calculation results and the comprehensive score of the flow path without error reporting.

8. A hydraulic housing flow channel planning and generating device, characterized in that: The device includes a processing module, an interaction module, a judgment module and an establishment module; wherein, The processing module is used to initialize the reinforcement learning network according to the hydraulic housing design information and construct the hydraulic housing environment; The interaction module is used to calculate a first path point of the hydraulic housing according to the reinforcement learning network, interact the first path point with the hydraulic housing environment, and calculate attribute information of the first path point, wherein the attribute information at least includes scoring information; The judgment module is used to judge whether the end point has been reached according to the attribute information corresponding to the path point. If the end point has not been reached, the attribute information of the calculated path point is input into the reinforcement learning network, the next path point is calculated, and the next path point is predicted and interacts with the hydraulic housing environment; The establishment module is used to determine the flow path according to all calculated path points, and establish the hydraulic housing model according to the flow path; The process of waypoints interacting with the hydraulic housing environment includes: Before each iteration begins, initializing parameters and the hydraulic housing environment; Inputting the state space and the action mask into the reinforcement learning network, obtaining the generated action and transmitting it into the hydraulic housing environment; Calculating a new path and a path vector according to the generated motion, and determining whether the flow path has reached an end point based on the new path; Before reaching the end point, calculate the remaining length of the flow channel, scan the environmental information and calculate the action score; Get the attribute information of the waypoints and update the reinforcement learning network; The determining whether the flow path reaches the end point based on the new path includes: After the agent enters the feasible cone, it determines whether the line from the current path point to the end point meets the minimum spacing requirements with other flow channels and boundaries; If satisfied, the current flow channel planning is determined to be finished and the end mark is activated; If not satisfied, the reward mark is triggered and subsequent calculations are performed.

Images