Ammeter shooting path optimization method based on reinforcement learning

Abstract

The invention discloses an electricity meter shooting path optimization method based on reinforcement learning, and the method comprises the steps: collecting the building structure information of a high-rise building, generating building space structure data, and constructing building-floor-electricity meter position hierarchical description graph data; building-floor-electricity meter position layering description graph data and inspection equipment state information data are called to initialize a three-layer layering reinforcement learning agent; under the driving of a building hierarchical strategy network, obtaining candidate floor sequence data; calling a nested Monte Carlo tree search algorithm for the candidate floor sequence data to perform floor level search expansion to obtain sorted floor sequence data; floor inner meter position access path data is generated; taking the access path data of the epitopes in the floors as evaluation input, and outputting batch candidate path data; and generating an optimal batch shooting path. According to the method, the state space dimension is remarkably compressed, and the path planning efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric meter photography, and in particular to an electric meter photography path optimization method based on reinforcement learning. Background Art

[0002] With the continuous development of smart buildings, the Internet of Things, and artificial intelligence, the demand for intelligent inspection and automatic reading of electricity meters in high-rise buildings is increasing. How to efficiently complete image acquisition and data feedback for dozens of floors and thousands of electricity meters in large-scale building environments has become a key challenge in the digital transformation of urban energy management and property management.

[0003] Existing technologies face a series of significant challenges when dealing with the complex spatial structures and extremely large number of nodes in high-rise buildings. First, single-layer shortest path algorithms or traditional Q-learning methods face state space explosion when faced with the state combinations of dozens of floors and thousands of electricity meters. This leads to an exponential increase in the computational resources required for path planning, making it difficult to implement real-time planning and adaptive scheduling in real-world scenarios. Second, existing batch inspection solutions often only optimize for the shortest path or a single objective, failing to consider the power consumption of inspection terminals, actual operating hours, the unique constraints of high-rise elevators, and the image capture clarity and ambient lighting quality indicators. Summary of the Invention

[0004] One purpose of the present invention is to propose a method for optimizing the path of electric meter photography based on reinforcement learning. The present invention significantly compresses the state space dimension and improves the efficiency of path planning.

[0005] A method for optimizing an electric meter photographing path based on reinforcement learning according to an embodiment of the present invention includes: Collect high-rise building structural information, generate building space structure data, and perform spatial calibration to obtain building space structure calibration data. At the same time, construct building-floor-meter position hierarchical description diagram data; And call the building-floor-meter position hierarchical description map data and inspection equipment status information data to initialize the three-layer hierarchical reinforcement learning agent; Driven by the building-level strategy network, candidate floor sequences are sampled from the accessible floor set according to the building-floor-meter hierarchical description graph data to obtain candidate floor sequence data; Calling the nested Monte Carlo tree search algorithm to perform floor-level search expansion on the candidate floor sequence data, calculating the floor access cost value data, and prioritizing the candidate floor sequence data to obtain sorted floor sequence data; Driven by the floor-level strategy network, a nested Monte Carlo tree search algorithm is called on the first floor node in the sorted floor sequence data to perform epitope-level path planning to generate epitope access path data within the floor. The epitope access path data within the floor is used as evaluation input to update the three-layer hierarchical reinforcement learning, perform strategy updates and experience feedback, and output batch candidate path data; Generate the optimal batch shooting path, output it and send it to the inspection equipment to complete batch shooting of electricity meters in high-rise buildings.

[0006] Optionally, the building-floor-electricity meter location hierarchical description diagram data is constructed, including: Collect 3D point cloud information of the building information model and uniformly map it to the building's global coordinates to form building spatial structure data; Extract the elevator shaft centerline data, corridor centerline data and electricity meter position initial coordinate data from the building space structure data, and store them as elevator information data, corridor information data and electricity meter position information data; Set the floor reference plane, map the same point in the building's global coordinate system to the position in the floor's local coordinate system through the rigid body transformation matrix, and perform coordinate transformation operations to convert elevator information data, corridor information data, and electricity meter location information data into the floor's local coordinate system; The laser scanning calibration value is used to compensate the error of the local coordinates of the floor to obtain the building space structure calibration data including the compensated coordinates; Constructing hierarchical node sets and hierarchical edge sets based on building spatial structure calibration data; The hierarchical node set and the hierarchical edge set are serialized and stored to generate building-floor-meter position hierarchical description graph data.

[0007] Optionally, the construction of the three-layer hierarchical reinforcement learning includes: Get the device position vector, remaining battery scalar, remaining time scalar, and shooting success rate scalar; Call the rigid body transformation matrix in the building space structure calibration data to convert the equipment position vector from the building global coordinate system to the equipment floor local position vector in the floor local coordinate system; According to the set order, the local position vector of the equipment floor, the remaining power scalar, the remaining time scalar and the shooting success rate scalar are spliced ​​into a one-dimensional state vector; Establish an association between the one-dimensional state vector and the building-floor-electricity meter position hierarchical description diagram data, and store it as the initialization environment state data; Create a building-level policy network based on the initialized environmental status data , Floor-level strategy network Meter-level strategy network ; Perform parameter initialization operation to set the building level strategy network parameter vector , floor-level strategy network parameter vector and the meter level strategy network parameter vector The building-level policy network, floor-level policy network, and meter-level policy network are written, and a one-dimensional state vector is used as the initial state input to complete the synchronous initialization of the three-layer hierarchical reinforcement learning agent.

[0008] Optionally, the construction of the building-level policy network includes: Based on the building-level strategy network, the building-level strategy network parameter vector is used to take the building-floor-meter hierarchical description graph data as input to calculate the probability value of each floor node being selected as the next floor to be visited; Sort the probability values ​​of all floor nodes in descending order, and filter out floor nodes with probability values ​​greater than or equal to the candidate floor screening threshold according to the preset candidate floor screening threshold to obtain the initial candidate floor node set; For each floor node in the initial candidate floor node set, the difference in the number of floors between the floor where the inspection device is currently located and the floor node is calculated. Based on the set maximum number of floors allowed to be crossed, the reachability constraint value of each floor node is determined. Only when the difference in the number of floors is less than or equal to the maximum number of floors allowed to be crossed, the reachability constraint value of the floor node is 1, indicating that the floor is accessible. Otherwise, the reachability constraint value is 0, indicating that the floor is inaccessible. All floor nodes with a reachability constraint value of 1 are sorted from large to small according to the corresponding probability values ​​to form the final candidate floor sequence data.

[0009] Optionally, the nested Monte Carlo tree search algorithm performs a floor-level search expansion, including: The final candidate floor sequence data is used as the initial node set of the Monte Carlo tree search, and each floor node in the initial node set is used as an initial search node in the Monte Carlo tree search; For each floor node in the initial search node set, call the floor-level nested Monte Carlo tree search algorithm to perform multiple extended searches to obtain floor access cost value data; The floor access cost value data of all floor nodes are sorted in descending order, and the sorted floor node sequence is the sorted floor sequence data.

[0010] Optionally, each expansion process in the expanded search includes: Based on the building-floor-meter location hierarchical description graph data, an internal meter location subtree of the current floor node is constructed. Each meter location node in the internal meter location subtree is used as a leaf node in the Monte Carlo tree search, and a local pruning operation is performed on each leaf node with a capture success rate constraint. For each leaf node retained after the local pruning operation, the inner Monte Carlo tree search is called to perform local evaluation. The result of the local evaluation is the local node benefit value of the leaf node; Based on the local node benefit values ​​of all leaf nodes retained within each floor node, a soft termination strategy nested modeling is used to aggregate the outer node benefit values ​​of the current floor node; The floor access cost value data of each floor node is calculated using the outer node benefit value.

[0011] Optionally, the nested modeling of the soft termination strategy refers to taking the maximum value of the expected benefit of withdrawing the current batch inspection task to the starting point in advance and the expected benefit of continuing to complete the current batch inspection task, and then adding the discount factor and the mean to obtain the outer node benefit value of each floor node.

[0012] Optionally, calling a nested Monte Carlo tree search algorithm to perform epitope-level path planning includes: Driven by the floor-level strategy network, the first floor node is extracted from the sorted floor sequence data, and the building-floor-meter hierarchical description graph data is combined with the corridor information data to construct the floor-corridor connectivity graph corresponding to the floor node. Identify a set of all electricity meter location nodes on each floor in the floor corridor connectivity graph; Calling a nested Monte Carlo tree search algorithm with the inspection device entrance position as the root node to calculate the meter access cost value data for each meter meter node in the meter meter node set; According to the meter access cost data, the upper confidence interval strategy is used to select the extended nodes for all nodes in the meter meter node set, and the optimal access path covering the meter meter node set is generated through multiple rounds of iteration.

[0013] Optionally, the strategy update and experience feedback include: The floor-level table access path data is input into the floor-level strategy network as the evaluation input. Based on the input table access path data and inspection equipment status information data, the policy gradient algorithm is used to update the current floor-level strategy network value estimation parameters. Feeding back the updated value estimation parameters of the floor-level strategy network to the building-level strategy network through parameter feedback, and dynamically correcting the inter-floor path planning using the updated value estimation parameters; The building-level policy network outputs batch candidate path data based on the updated value estimation parameters and the feedback information between the hierarchical policies.

[0014] Optionally, the generation of the optimal batch shooting path includes: Based on the remaining power data of the inspection equipment, the remaining operation time window data of the inspection equipment, the batch candidate path data and the building-floor-electricity meter position layered description map data, the continuing action data and the early termination action data are generated. The batch candidate path data corresponding to the continuing action data and the early termination action data are compared in value, the action data with the greatest value is selected, and the final batch shooting path data is output; The final batch shooting path data is sent to the inspection equipment for execution. The inspection equipment moves according to the final batch shooting path data and shoots images at each target meter position to complete the batch shooting of high-rise building meters.

[0015] The beneficial effects of the present invention are: This application not only significantly compresses the state space dimension and improves the path planning efficiency, but also enables the inspection terminal to achieve a dynamic balance between the three goals of resource consumption, inspection coverage and shooting quality, meeting the actual engineering needs of batch recording of electricity meters in high-rise buildings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 This is a flowchart of a method for optimizing the path of electric meter photography based on reinforcement learning proposed by the present invention. DETAILED DESCRIPTION

[0017] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0018] refer to Figure 1 , a method for optimizing the path of electric meter shooting based on reinforcement learning, comprising: Collect high-rise building structural information to generate building space structure data, perform spatial calibration on the elevator information, corridor information, and electricity meter position information in the building space structure data to obtain building space structure calibration data, and construct building-floor-electricity meter position hierarchical description diagram data based on the building space structure calibration data; Obtain inspection equipment status information data, and call the building-floor-meter position hierarchical description map data and inspection equipment status information data to initialize the three-layer hierarchical reinforcement learning agent, and write the building-level policy network, floor-level policy network, and meter position-level policy network into the internal parameter table of the three-layer hierarchical reinforcement learning agent; Driven by the building-level strategy network, candidate floor sequences are sampled from the accessible floor set according to the building-floor-meter hierarchical description graph data to obtain candidate floor sequence data; The nested Monte Carlo tree search algorithm is used to perform floor-level search expansion on the candidate floor sequence data. The floor access cost data is calculated by integrating the remaining power data of the inspection equipment, the remaining operation time window data of the inspection equipment, and the elevator information data. The candidate floor sequence data is prioritized according to the floor access cost data to obtain the sorted floor sequence data. Driven by the floor-level strategy network, a nested Monte Carlo tree search algorithm is called on the first floor node in the sorted floor sequence data to perform table-level path planning. Table-level access cost data is calculated based on corridor information data and meter table information data to generate table-level access path data within the floor. The epitope access path data within the floor is used as evaluation input to update the three-layer hierarchical reinforcement learning, perform strategy updates and experience feedback, and output batch candidate path data; Based on the remaining power data of the inspection equipment, the remaining operating time window data of the inspection equipment, the batch candidate path data and the building-floor-meter position layered description map data, the optimal batch shooting path is generated, output and sent to the inspection equipment to complete the batch shooting of high-rise building meters.

[0019] In this embodiment, the building-floor-meter location hierarchical description diagram data is constructed, including: Collect 3D point cloud information of floor outlines, load-bearing walls, structural columns, and fire escape passages in the building information model, and uniformly map the 3D point cloud information to the building's global coordinates to form building spatial structure data; Extract the elevator shaft centerline data, corridor centerline data, and electricity meter initial coordinate data from the building space structure data based on the semantic tags of the building information model, and store them as elevator information data, corridor information data, and electricity meter information data; Set the floor reference plane, map the same point in the building's global coordinate system to the floor's local coordinate system through the rigid body transformation matrix, perform coordinate transformation operations on all key point coordinates in the 3D point cloud information, and convert elevator information data, corridor information data, and electricity meter location information data into the floor's local coordinate system; The laser scanning calibration value is used to compensate for the errors of the local floor coordinates. The laser scanning calibration value is a three-dimensional vector. Each item in the three-dimensional vector represents the calibration offset in the x-axis direction, the y-axis direction, and the z-axis direction. After error compensation, the building space structure calibration data containing the compensated coordinates is obtained. Based on the building spatial structure calibration data, a hierarchical node set and a hierarchical edge set are constructed. The hierarchical node set includes the building root node, all floor nodes, and all electricity meter nodes on each floor. The building root node is used to represent the entire building, the floor node is used to represent each floor, and the electricity meter node is used to represent all electricity meter nodes on each floor. The total number of floors is L, and the number of electricity meter nodes on the i-th floor is ,The hierarchical edge set includes the edges between the building root node and ,each floor node, and the edges between each floor node and all ,electricity meter locations under it.,The hierarchical edge set describes the hierarchical ,connection relationship between buildings, floors and electricity meter locations; The hierarchical node set and the hierarchical edge set are serialized and stored to generate building-floor-meter position hierarchical description graph data.

[0020] In this embodiment, the construction of three-layer hierarchical reinforcement learning includes: Obtain the current location information of the inspection device, obtain the device location vector, obtain the remaining power information of the inspection device, obtain the remaining power scalar, obtain the remaining operation time window information of the inspection device, obtain the remaining time scalar, obtain the historical shooting success rate information of the inspection device, and obtain the shooting success rate scalar; Call the rigid body transformation matrix in the building space structure calibration data to convert the equipment position vector from the building global coordinate system to the equipment floor local position vector in the floor local coordinate system. The equipment floor local position vector represents the position of the inspection equipment in the floor local coordinate system. The local position vector of the device floor, the remaining power scalar, the remaining time scalar, and the shooting success rate scalar are concatenated into a one-dimensional state vector in a set order. Each element of the one-dimensional state vector corresponds to the local position vector of the device floor, the remaining power scalar, the remaining time scalar, and the shooting success rate scalar information in sequence; Establish an association between the one-dimensional state vector and the building-floor-electricity meter position hierarchical description diagram data, and store it as the initialization environment state data; Create a building-level policy network based on the initialized environmental status data , Floor-level strategy network Meter-level strategy network , and set the building-level strategy network parameter dimensions, floor-level strategy network parameter dimensions and meter-level strategy network parameter dimensions respectively; Perform parameter initialization operation to set the building level strategy network parameter vector , floor-level strategy network parameter vector and the meter level strategy network parameter vector The building-level policy network, floor-level policy network, and meter-level policy network are written, and a one-dimensional state vector is used as the initial state input to complete the synchronous initialization of the three-layer hierarchical reinforcement learning agent.

[0021] Building-level strategy network parameter vector , floor-level strategy network parameter vector and the meter level strategy network parameter vector It can be obtained by loading a pre-trained parameter vector, or initialized by a normalized Gaussian distribution or a set initial constant.

[0022] The building level corresponds to the global decision-making of the entire building. It is responsible for selecting the set of floors to be visited in this batch from all floors under global resource constraints. The building level uses the global state of the environment as input and outputs a sequence of candidate floors, taking into account path length, elevator scheduling and inspection resources.

[0023] The floor level is responsible for planning the optimal inspection sequence covering all target meter locations within a determined single floor. The input is the spatial connectivity, meter location distribution and historical collection feedback of the current floor, and the output is the meter location access path.

[0024] The meter level describes the micro-operations of specific meter nodes, and decides in real time whether to shoot, reshoot, or skip, thereby improving the shooting success rate and energy utilization rate of the overall path.

[0025] In this embodiment, the construction of the building-level policy network includes: Based on the building-level strategy network, the building-level strategy network parameter vector is used to take the building-floor-meter hierarchical description graph data as input to calculate the probability value of each floor node being selected as the next floor to be visited; The probability value is calculated by the building-level strategy network based on the one-dimensional state vector. The probability value reflects the priority of each floor node as the next visit target; Sort the probability values ​​of all floor nodes in descending order, and filter out floor nodes with probability values ​​greater than or equal to the candidate floor screening threshold according to the preset candidate floor screening threshold to obtain the initial candidate floor node set; Each floor node in the initial candidate floor node set is a floor node that meets the probability screening condition, and the candidate floor screening threshold is a scalar between 0 and 1; For each floor node in the initial candidate floor node set, the floor number difference between the inspection device's current floor and the floor node is calculated. Based on the set maximum number of floors allowed to be crossed, the reachability constraint value of each floor node is determined. Only when the floor number difference is less than or equal to the maximum number of floors allowed to be crossed, the reachability constraint value of the floor node is 1, indicating that the floor is accessible. Otherwise, the reachability constraint value is 0, indicating that the floor is inaccessible. All floor nodes with a reachability constraint value of 1 are sorted from large to small according to the corresponding probability values ​​to form the final candidate floor sequence data.

[0026] In this embodiment, the nested Monte Carlo tree search algorithm performs a floor-level search expansion, including: The final candidate floor sequence data is used as the initial node set of the Monte Carlo tree search, and each floor node in the initial node set is used as an initial search node in the Monte Carlo tree search; For each floor node in the initial search node set, call the floor-level nested Monte Carlo tree search algorithm to perform multiple expansion searches; The floor access cost value data of all floor nodes are sorted in descending order, and the sorted floor node sequence is the sorted floor sequence data.

[0027] In this embodiment, each expansion process in the expanded search includes: Based on the building-floor-meter-position hierarchical description graph data, an internal meter-position subtree of the current floor node is constructed. Each meter-position node in the internal meter-position subtree is used as a leaf node in the Monte Carlo tree search. A local pruning operation is performed on each leaf node, constrained by the capture success rate. The local pruning operation is achieved by removing meter-position nodes with a capture success rate below a preset threshold. For each leaf node retained after the local pruning operation, the inner Monte Carlo tree search is called to perform local evaluation. The result of the local evaluation is the local node benefit value of the leaf node. The local node benefit value is used to measure the comprehensive benefit of accessing the meter node from the current location of the inspection equipment. The evaluation indicators of the comprehensive benefit include remaining power, expected power consumption, remaining operation time, expected consumption time, historical shooting success rate and path entropy penalty value. The path entropy penalty value is used to suppress the path strategy from being overly concentrated on some nodes to improve path diversity. The remaining power, expected power consumption, remaining operation time, expected consumption time, historical shooting success rate and path entropy penalty value are respectively assigned weighted coefficients, and the sum of all weighted coefficients is guaranteed to be 1. The local node benefit value is obtained by linearly adding each indicator according to the weighted coefficient. The weight distribution is determined according to actual business needs. The remaining power refers to the real-time remaining available power of the inspection equipment under the current batch task. The remaining power value is obtained in real time through the battery management unit of the equipment and is expressed in percentage or absolute energy units. Each time the path planning and movement action of a floor or meter is executed, the current remaining power is dynamically updated according to the equipment movement trajectory, power consumption model and actual operation data.

[0028] The estimated power consumption refers to the path from the device's current location to the target floor or meter node. Based on the path length and climbing height, combined with the device's unit distance power consumption and actual access obstacles, a dynamic estimate is made to obtain the energy consumption forecast value from the current location to each candidate node.

[0029] The remaining operation time refers to the remaining time from the current moment to the latest completion time specified for the batch task. The remaining operation time is calculated in real time based on the task schedule, the start and end times of the shift, and the actual time consumed for each path update, and serves as the time constraint in path planning.

[0030] The estimated time is the time it is expected to take to reach the target floor or meter node from the device's current location. This time is dynamically simulated and calculated based on the physical length of the building's internal paths, elevator waiting and running times, and channel congestion prediction factors. The estimated time for each feasible path is output based on the device's current speed, the historical average time taken for the same path, and the real-time building congestion status.

[0031] The historical capture success rate refers to the probability of successful image acquisition and automatic recognition in the device's historical tasks at the target meter node. The historical capture success rate is calculated by statistically analyzing and attributing the historical capture data of each node, analyzing the environmental factors that affect the capture success rate, and providing real-time feedback on the capture results of the latest task. This serves as an important reward signal for adjusting the policy network. Nodes with high historical capture success rates are given higher weights in path selection to improve the success rate and efficiency of the overall batch task.

[0032] The path entropy penalty value is used to measure and adjust the diversity of path planning to prevent the decision-making strategy from being overly concentrated on some frequently visited nodes. The specific construction method is as follows: The frequency distribution of each node visited during the historical path planning process is statistically analyzed, and the diversity index of the current path selection is calculated based on the information entropy theory. When certain epitope nodes are repeatedly preferred in multiple tasks, the path entropy penalty value increases, prompting the active exploration of other possible paths, thereby improving the diversity and robustness of the overall path.

[0033] Based on the local node benefit values ​​of all leaf nodes retained within each floor node, a soft termination strategy nested modeling is used to aggregate the outer node benefit values ​​of the current floor node; The outer node benefit value is used to measure the overall expected benefit of the current floor node. The outer node benefit value is the weighted superposition of the mean benefit value of all local nodes and the expected benefit of the soft termination strategy. The soft termination strategy nested modeling refers to taking the maximum value of the expected benefit of withdrawing the current batch of inspection tasks to the starting point in advance and the expected benefit of continuing to complete the current batch of inspection tasks. The outer node benefit value of each floor node is obtained by adding the maximum value of the expected benefit of withdrawing the current batch of inspection tasks to the starting point in advance and the expected benefit of continuing to complete the current batch of inspection tasks. The discount factor is used to adjust the impact of future rewards on current decisions. The floor access cost value data of each floor node is calculated using the outer node benefit value. The floor access cost value data is used to measure the comprehensive priority of the current floor node as the next access target; The floor visit cost data is obtained by weightedly superimposing the outer node benefit value and the exploration weight coefficient based on historical visit frequency. The exploration weight coefficient is used to balance the priority between explored paths and unexplored paths. ; in, is the outer node benefit value of the currently expanded i-th floor node, Indicates the floor access cost value data of the currently expanded i-th floor node, is the exploration weight coefficient of the Monte Carlo tree search algorithm, which is a dimensionless scalar greater than 0. is the historical visit count of the parent node of the current floor node, The historical visit count of the current floor node.

[0034] In this embodiment, a nested Monte Carlo tree search algorithm is called to perform epitope-level path planning, including: Driven by the floor-level strategy network, the first floor node is extracted from the sorted floor sequence data, and the building-floor-meter hierarchical description graph data is combined with the corridor information data to construct the floor-corridor connectivity graph corresponding to the floor node. The floor corridor connectivity diagram is used to describe the traffic relationship between each spatial area on the floor and the actual spatial distribution of the electricity meter nodes. The floor corridor connectivity diagram uses the inspection equipment entrance location as the root node and all the electricity meter nodes on the floor as the coverage nodes. Identify the set of all electricity meter nodes on each floor in the floor corridor connectivity graph; The electricity meter node set is used to represent all electricity meter nodes on a floor that need to be accessed and photographed. The number of nodes is the total number of electricity meter nodes on the floor. Calling a nested Monte Carlo tree search algorithm with the inspection device entrance position as the root node to calculate the meter access cost value data for each meter meter node in the meter meter node set; Based on the meter access cost data, the upper confidence interval strategy is used to select the expansion nodes for all nodes in the meter meter node set, and the optimal access path covering the meter meter node set is generated through multiple rounds of iterations; The upper confidence interval strategy is used to comprehensively evaluate the historical benefits and exploration potential of each node in each round of search, giving priority to expanding nodes with high historical benefits and low exploration times. The optimal access path is the access path data of the table within the floor.

[0035] In this implementation, strategy updates and experience feedback include: The floor-level epitope access path data is input into the floor-level strategy network as evaluation input. The floor-level strategy network uses the policy gradient algorithm to update the value estimation parameters of the current floor-level strategy network based on the input epitope access path data and inspection equipment status information data. The value estimation parameters reflect the cumulative return performance of different path strategies in historical tasks. The updated value estimation parameters of the floor-level strategy network are fed back to the building-level strategy network through parameter feedback. The building-level strategy network uses the updated value estimation parameters to dynamically correct the inter-floor path planning, thus achieving value information sharing among the hierarchical strategies and global strategy convergence. Based on the updated value estimation parameters and the feedback information between the hierarchical strategies, the building-level strategy network integrates the meter access path data on each floor, the inspection equipment status information data, and the building-floor-meter meter hierarchical description map data to output batch candidate path data. The batch candidate path data is used to represent the optimal batch shooting task plan under the current resource constraints and benefit trade-offs.

[0036] In this embodiment, the generation of the optimal batch shooting path includes: Based on the remaining power data of the inspection equipment, the remaining operation time window data of the inspection equipment, the batch candidate path data and the building-floor-electricity meter position layered description map data, the continuing action data and the early termination action data are generated. The batch candidate path data corresponding to the continuing action data and the early termination action data are compared in value, the action data with the greatest value is selected, and the final batch shooting path data is output; The final batch shooting path data is sent to the inspection equipment for execution. The inspection equipment moves according to the final batch shooting path data and shoots images at each target meter position to complete the batch shooting of high-rise building meters.

[0037] Example 1: In a high-rise building's electric meter batch automatic reading task, inspection robot A was assigned to inspect all electric meters on the 15th and 16th floors starting from the elevator hall on the 15th floor. During the task initialization phase, inspection robot A automatically obtained a layered description diagram of the building-floor-electricity meter position through the environmental database. The system identified that robot A's current remaining power was 68%, the remaining operation time was 52 minutes, and the historical shooting success rate was 97.5%. Environmental monitoring data showed that due to temporary construction, the northwest corridor of the 15th floor had narrowed the passage space for some meters, and the system automatically updated the passage access status.

[0038] Inspection robot A determined that the 15th and 16th floors were the optimal target floors for this batch through the building-level strategy network and nested Monte Carlo tree search algorithm. Robot A first entered the 15th floor according to the sorted floor sequence issued by the dispatcher. The system combined the corridor connectivity and the distribution of electricity meters to generate a spatial distribution map of all 78 electricity meters on the 15th floor, and identified that 7 of them were located in the construction area. The success rate of the past three rounds of shooting was less than 90%. The system used local pruning to automatically skip the low-success rate meters and marked them as targets for subsequent re-shooting.

[0039] After entering the elevator hall on the 15th floor, inspection robot A visited the remaining 71 electricity meters in sequence along the optimized path. Every time it arrived at an electricity meter, the robot automatically took a photo. After the photo was taken, it immediately performed image quality analysis and QR code recognition on the photo. When it arrived at meter No. 31 on the 15th floor, the robot detected that the ambient illumination was only 21lx, which was lower than the set threshold of 35lx. It automatically adjusted the fill light intensity to 80%. Then the photo recognition passed and the results were uploaded to the background synchronously. During the process, the robot was blocked by debris in front when taking a photo at meter No. 44. The system located and identified the obstacle, and the path replanning took only 1.4 seconds. The meter was adjusted to be accessed from the rear channel, and the shooting success rate was not affected.

[0040] After completing the 15th floor, inspection robot A checked its remaining battery power and found it had dropped to 49%. The system re-evaluated the remaining tasks. Considering that the elevator was about to reach the peak waiting period, the system evaluated the time and energy consumption of the robot moving to the 16th floor, and finally decided to continue completing the 16th-floor task. After entering the 16th floor, the robot photographed all 84 electricity meters point by point. Among them, the QR code recognition failed for 5 meters due to special lighting angles. The system automatically triggered the soft termination mechanism, planned the retreat path in advance, and pushed the reshoot task to the next batch of scheduling.

[0041] During the entire batch inspection, Robot A completed the shooting tasks of 155 electricity meters, with a total shooting success rate of 99.2%. It took an average of 32 seconds per meter, and the batch energy consumption was controlled at 19%. After the task was completed, the system automatically generated an inspection report, which included the spatial coordinates of each meter, the captured pictures, the image recognition results, the re-shooting list and the entire path trajectory.

[0042] Compared with traditional manual inspections, manual paths are usually based on the order of floors, and manual judgment is often required to detour when encountering obstacles, resulting in an average time of 54 seconds per inspection and easy omission of inspections in special areas. The re-shooting rate is as high as 7%. The single-target shortest path method does not consider energy consumption, elevator scheduling and lighting environment, so the probability of batch execution interruption is high, and the re-shooting rate is about 4.3%.

[0043] In the embodiment of the present invention, automatic pruning, real-time replanning, soft termination mechanism and multi-objective comprehensive optimization are used to not only improve the meter coverage efficiency, but also significantly reduce shooting failures and repeated operations caused by obstacles and illumination.

[0044] This application systematically optimizes the process of optimizing the path of batch shooting of electricity meters in high-rise buildings by introducing a nested Monte Carlo tree search algorithm and a three-layer hierarchical reinforcement learning agent. Specifically, the building information model data is first collected to complete the high-precision spatial calibration of elevators, corridors and electricity meter locations to construct a hierarchical description map of buildings, floors and electricity meter locations. The three-layer hierarchical reinforcement learning agent is initialized based on the remaining power of the inspection equipment, the operation period and the historical shooting success rate to achieve hierarchical strategy modeling at the building level, floor level and electricity meter location level. During the path planning process, the building-level strategy network dynamically screens the reachable candidate floors according to the global state and resource constraints, and uses the nested Monte Carlo tree search algorithm to perform multi-objective evaluation, taking into account the remaining power of the equipment, the remaining time of the operation, the elevator scheduling and the real-time environmental constraints to prioritize the candidate floors. On each floor, a spatial subgraph is established through the corridor connectivity and meter location information. A meter-level nested Monte Carlo tree search algorithm is used, and multiple indicators are integrated in the node evaluation process to automatically prune nodes with low success rates, effectively improving the unit workload and recognition success rate within the batch and reducing secondary re-shooting.

[0045] This application designs a nested modeling of soft termination strategies, which supports intelligent decision-making for early retraction and batch termination when shooting quality degrades or resources are tight. It also implements multi-level decision closed-loop optimization through layered value feedback, and achieves real-time adaptive updates and global convergence of the strategy network through batch candidate paths and experience replay mechanisms. This reduces planning time caused by excessive nodes and complex constraints in high-rise building inspection environments, and effectively supports online replanning and dynamic reshooting tasks within seconds. The overall solution not only significantly compresses the state space dimension and improves path planning efficiency, but also enables inspection terminals to achieve a dynamic balance between the three goals of resource consumption, inspection coverage, and shooting quality, meeting the actual engineering needs of batch meter reading in high-rise buildings.

[0046] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for optimizing the path of electric meter shooting based on reinforcement learning, characterized in that: include: Collect high-rise building structural information, generate building space structure data, and perform spatial calibration to obtain building space structure calibration data. At the same time, construct building-floor-meter position hierarchical description diagram data; And call the building-floor-meter position hierarchical description map data and inspection equipment status information data to initialize the three-layer hierarchical reinforcement learning agent; Driven by the building-level strategy network, candidate floor sequences are sampled from the accessible floor set according to the building-floor-meter hierarchical description graph data to obtain candidate floor sequence data; Calling the nested Monte Carlo tree search algorithm to perform floor-level search expansion on the candidate floor sequence data, calculating the floor access cost value data, and prioritizing the candidate floor sequence data to obtain sorted floor sequence data; Driven by the floor-level strategy network, a nested Monte Carlo tree search algorithm is called on the first floor node in the sorted floor sequence data to perform epitope-level path planning to generate epitope access path data within the floor. The epitope access path data within the floor is used as evaluation input to update the three-layer hierarchical reinforcement learning, perform strategy updates and experience feedback, and output batch candidate path data; Generate the optimal batch shooting path, output it and send it to the inspection equipment to complete batch shooting of electricity meters in high-rise buildings.

2. The method for optimizing the electric meter shooting path based on reinforcement learning according to claim 1, characterized in that: The building-floor-electricity meter position hierarchical description diagram data is constructed, including: Collect 3D point cloud information of the building information model and uniformly map it to the building's global coordinates to form building spatial structure data; Extract the elevator shaft centerline data, corridor centerline data and electricity meter position initial coordinate data from the building space structure data, and store them as elevator information data, corridor information data and electricity meter position information data; Set the floor reference plane, map the same point in the building's global coordinate system to the position in the floor's local coordinate system through the rigid body transformation matrix, and perform coordinate transformation operations to convert elevator information data, corridor information data, and electricity meter location information data into the floor's local coordinate system; The laser scanning calibration value is used to compensate the error of the local coordinates of the floor to obtain the building space structure calibration data including the compensated coordinates; Constructing hierarchical node sets and hierarchical edge sets based on building spatial structure calibration data; The hierarchical node set and the hierarchical edge set are serialized and stored to generate building-floor-meter position hierarchical description graph data.

3. The method for optimizing the electric meter shooting path based on reinforcement learning according to claim 2, characterized in that: The construction of the three-layer hierarchical reinforcement learning includes: Get the device position vector, remaining battery scalar, remaining time scalar, and shooting success rate scalar; Call the rigid body transformation matrix in the building space structure calibration data to convert the equipment position vector from the building global coordinate system to the equipment floor local position vector in the floor local coordinate system; According to the set order, the local position vector of the equipment floor, the remaining power scalar, the remaining time scalar and the shooting success rate scalar are spliced ​​into a one-dimensional state vector; Establish an association between the one-dimensional state vector and the building-floor-electricity meter position hierarchical description diagram data, and store it as the initialization environment state data; Create a building-level policy network based on the initialized environmental status data , Floor-level strategy network Meter-level strategy network ; Perform parameter initialization operation to set the building level strategy network parameter vector , floor-level strategy network parameter vector and the meter level strategy network parameter vector The building-level policy network, floor-level policy network, and meter-level policy network are written, and a one-dimensional state vector is used as the initial state input to complete the synchronous initialization of the three-layer hierarchical reinforcement learning agent.

4. The method for optimizing the electric meter shooting path based on reinforcement learning according to claim 3, characterized in that: The construction of the building-level strategy network includes: Based on the building-level strategy network, the building-level strategy network parameter vector is used to take the building-floor-meter hierarchical description graph data as input to calculate the probability value of each floor node being selected as the next floor to be visited; Sort the probability values ​​of all floor nodes in descending order, and filter out floor nodes with probability values ​​greater than or equal to the candidate floor screening threshold according to the preset candidate floor screening threshold to obtain the initial candidate floor node set; For each floor node in the initial candidate floor node set, the difference in the number of floors between the floor where the inspection device is currently located and the floor node is calculated. Based on the set maximum number of floors allowed to be crossed, the reachability constraint value of each floor node is determined. Only when the difference in the number of floors is less than or equal to the maximum number of floors allowed to be crossed, the reachability constraint value of the floor node is 1, indicating that the floor is accessible. Otherwise, the reachability constraint value is 0, indicating that the floor is inaccessible. All floor nodes with a reachability constraint value of 1 are sorted from large to small according to the corresponding probability values ​​to form the final candidate floor sequence data.

5. The method for optimizing the electric meter shooting path based on reinforcement learning according to claim 4, characterized in that: The nested Monte Carlo tree search algorithm performs floor-level search expansion, including: The final candidate floor sequence data is used as the initial node set of the Monte Carlo tree search, and each floor node in the initial node set is used as an initial search node in the Monte Carlo tree search; For each floor node in the initial search node set, call the floor-level nested Monte Carlo tree search algorithm to perform multiple extended searches to obtain floor access cost value data; The floor access cost value data of all floor nodes are sorted in descending order, and the sorted floor node sequence is the sorted floor sequence data.

6. The method for optimizing the electric meter shooting path based on reinforcement learning according to claim 5, characterized in that: Each expansion process in the expanded search includes: Based on the building-floor-meter location hierarchical description graph data, an internal meter location subtree of the current floor node is constructed. Each meter location node in the internal meter location subtree is used as a leaf node in the Monte Carlo tree search, and a local pruning operation is performed on each leaf node with a capture success rate constraint. For each leaf node retained after the local pruning operation, the inner Monte Carlo tree search is called to perform local evaluation. The result of the local evaluation is the local node benefit value of the leaf node; Based on the local node benefit values ​​of all leaf nodes retained within each floor node, a soft termination strategy nested modeling is used to aggregate the outer node benefit values ​​of the current floor node; The floor access cost value data of each floor node is calculated using the outer node benefit value.

7. The method for optimizing the meter shooting path based on reinforcement learning according to claim 5, characterized in that: The nested modeling of the soft termination strategy refers to taking the maximum value of the expected benefit of withdrawing the current batch inspection task to the starting point in advance and the expected benefit of continuing to complete the current batch inspection task, and then adding the discount factor and the mean to obtain the outer node benefit value of each floor node.

8. The method for optimizing the electric meter shooting path based on reinforcement learning according to claim 5, characterized in that: The calling of the nested Monte Carlo tree search algorithm to perform epitope-level path planning includes: Driven by the floor-level strategy network, the first floor node is extracted from the sorted floor sequence data, and the building-floor-meter hierarchical description graph data is combined with the corridor information data to construct the floor-corridor connectivity graph corresponding to the floor node. Identify a set of all electricity meter location nodes on each floor in the floor corridor connectivity graph; Calling a nested Monte Carlo tree search algorithm with the inspection device entrance position as the root node to calculate the meter access cost value data for each meter meter node in the meter meter node set; According to the meter access cost data, the upper confidence interval strategy is used to select the extended nodes for all nodes in the meter meter node set, and the optimal access path covering the meter meter node set is generated through multiple rounds of iteration.

9. The method for optimizing the electric meter shooting path based on reinforcement learning according to claim 8, characterized in that: The strategy update and experience feedback include: The floor-level table access path data is input into the floor-level strategy network as the evaluation input. Based on the input table access path data and inspection equipment status information data, the policy gradient algorithm is used to update the current floor-level strategy network value estimation parameters. Feeding back the updated value estimation parameters of the floor-level strategy network to the building-level strategy network through parameter feedback, and dynamically correcting the inter-floor path planning using the updated value estimation parameters; The building-level policy network outputs batch candidate path data based on the updated value estimation parameters and the feedback information between the hierarchical policies.

10. The method for optimizing the path of electric meter shooting based on reinforcement learning according to claim 9, characterized in that: The generation of the optimal batch shooting path includes: Based on the remaining power data of the inspection equipment, the remaining operation time window data of the inspection equipment, the batch candidate path data and the building-floor-electricity meter position layered description map data, the continuing action data and the early termination action data are generated. The batch candidate path data corresponding to the continuing action data and the early termination action data are compared in value, the action data with the greatest value is selected, and the final batch shooting path data is output; The final batch shooting path data is sent to the inspection equipment for execution. The inspection equipment moves according to the final batch shooting path data and shoots images at each target meter position to complete the batch shooting of high-rise building meters.

Images