Wheel-legged robot plant inspection path simulation method based on digital twinning

Abstract

The application provides a wheeled quadruped robot factory area inspection path simulation method based on digital twinning, and belongs to the technical field of inspection; solves the problem of low robot inspection route planning efficiency; the specific method is as follows: a digital twinning model is constructed based on point clouds and aerial images of the factory area, video monitoring is connected to identify personnel and vehicles, road interference is counted in real time, and the motion constraints of the robot are defined; according to historical data of the factory area, the area failure probability is estimated, the inspection priority is divided, and the path is planned: the robot pose is collected in real time to check the running state, and the walking mode is switched according to the obstacle height if normal; the application realizes the whole process supervision of environment modeling, risk assessment, path planning and motion control through the robot, reduces the route searching difficulty of the robot, and improves the inspection efficiency.

Description

Technical Field

[0001] This invention relates to a simulation method for factory inspection paths of wheeled quadruped robots based on digital twins, belonging to the field of inspection technology. Background Technology

[0002] The existing methods for determining factory inspection routes have the following shortcomings: The uniform inspection model is prone to misallocation of inspection resources: Traditional factory inspections generally adopt a fixed route and fixed frequency of uniform coverage across the entire area, without distinguishing the differences in equipment failure risk in different production areas, and performing full-coverage inspections according to a uniform standard. This extensive inspection strategy will aggravate the problem of resource misallocation. For example, in high-risk areas where key production equipment is concentrated, operating load is high, and failure rate is high, the fixed inspection frequency limits the timeliness of hidden danger investigation, making it difficult to detect and deal with equipment failures in a timely manner, which can easily lead to unplanned shutdowns and other production safety accidents, and make it difficult to ensure the reliable operation of equipment.

[0003] Fixed-area task allocation leads to low efficiency in inspection resource utilization: Traditional inspection task allocation often adopts a fixed-area responsibility division model, dividing the factory area into fixed zones and matching corresponding robots to perform inspections, without dynamic scheduling optimization based on the real-time battery status of the robots. This rigid allocation method easily leads to an imbalance and waste of inspection resources: some robots with sufficient remaining battery power return to recharge early after completing their zone tasks, resulting in significant battery redundancy and idle energy consumption; some robots run out of battery power due to long inspection mileage in their zones, forcing them to turn back to recharge midway, lengthening the total inspection time per round and reducing overall inspection efficiency.

[0004] The experience-driven approach to iteration is costly: Traditional factory inspection solutions rely heavily on the on-site experience of maintenance personnel. Core parameters such as path planning, mode switching thresholds, and task scheduling rules all require on-site debugging and optimization, lacking a full-process digital simulation verification process. This offline debugging model not only has a long deployment cycle and high labor costs, but may also disrupt normal production order due to robot debugging and operation, resulting in high trial and error costs and time costs for solution adjustments. At the same time, when the factory expands, new equipment is added, or inspection requirements are upgraded, the iteration of traditional solutions requires repeating the lengthy process of on-site testing, parameter adjustment, and on-site verification, resulting in slow response speed, high adaptation costs, and difficulty in quickly adapting to new factory layouts and maintenance needs. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a simulation method for factory inspection paths of wheeled quadruped robots based on digital twins, thereby solving the problem of low efficiency in robot inspection route planning.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: a simulation method for factory inspection paths of a wheeled quadruped robot based on digital twins includes: Acquire point cloud data and aerial images of the factory area, establish a digital virtual model of the factory area, associate all cameras in the factory area, load the personnel and vehicles captured by each camera into the virtual model of the factory area, monitor the flow of people and vehicles on each path in the factory area in real time, obtain the path interference of each path, and then define the motion constraints of the quadruped wheeled robot. Acquire and analyze historical operating data for each area within the plant, and then combine this with actual operating data for each area within the plant to estimate the probability of failure in each area. Based on the probability of failure in each area, the inspection level of each area will be divided, and then different inspection routes will be selected according to the actual situation of the inspection level of each area. Scenario 1: If all areas have the same inspection level and are all low inspection levels, then select a robot with sufficient power to perform an overall inspection of all areas. Scenario 2: If the inspection levels of different areas are different, then the inspections will be carried out in stages according to the order of inspection level from high to low. The robot acquires the pose data of the quadruped wheeled robot in real time and determines whether the robot is functioning normally based on its motion constraints. If it is functioning normally, the robot selectively enables its quadruped walking mode and defines motion constraints based on the actual obstacle height on the inspection path. If it is not functioning normally, a new quadruped wheeled robot is assigned to complete the inspection task.

[0007] Furthermore, the steps for establishing a digital virtual model are as follows: Spatial data of various buildings and structures within the factory area are acquired to establish the global spatial coordinates of the factory area; aerial imagery and ground-based LiDAR point cloud data of the factory area are acquired simultaneously, and preprocessing such as image distortion correction, panoramic stitching and point cloud denoising, and single-station stitching are performed respectively; using ground control points as a common reference, aerial imagery and point cloud data are fused together. Aerial data is used to fill in the scanning blind spots of point cloud data. Then, the height data of each building and structure in the point cloud data is corrected based on the known height of ground control points to obtain the full-area fused point cloud data of the factory area. The full-area fused point cloud data is then calibrated a second time based on the CAD drawing data of the factory area to construct a three-dimensional geometric model of the factory area and import it into the three-dimensional engine to obtain the virtual model of the factory area. The system acquires and decodes all video streams from the factory area, then obtains the pose data of the quadruped robot, uses a target detection algorithm, and extracts the pixel coordinates of personnel and vehicles from the video stream. Using the intrinsic and extrinsic parameters of each camera and the homogeneous matrix, the personnel and vehicles extracted from the video stream are converted from two-dimensional pixel coordinates to three-dimensional world coordinates. The virtual model of the factory area and the personnel and vehicles in the video stream are loaded into the 3D engine. The position and orientation of the virtual robot, personnel and vehicles are updated frame by frame according to the received real-time coordinates. At the same time, the flow of people and vehicles on each path in the factory area are counted as the path interference of each path.

[0008] Furthermore, the steps for defining motion constraints are as follows: Let the world coordinates of building A within the factory area be (x （w） y （w） , z （w） Let the coordinates of the building inside the robot be (x, y). （b） y （b） , z （b） Let the robot's heading angle be φ. （航） , ; Let h be the vertical distance from the robot's main camera to the ground. （b） The homogeneous matrix T of the robot is obtained. （b） Then define the positioning constraints of the robot in world coordinates and robot interior coordinates: Let the radius of the robot wheel be r. （b） The wheelbase is b （b） The angular velocities of the robot's left and right tires are ω and ω, respectively. （左） and ω （右） The robot's actual forward speed is obtained as v. （机） With steering angular velocity; Let the robot's position in the world coordinate system be (x... o（w） y o（w） The heading angle is φo （航） Define the constraints on the position changes of the robot during its movement; .

[0009] Furthermore, the steps for estimating the probability of failure in each region are as follows: In the historical operational data, the operational data of each region when no fault occurred and the operational data when fault occurred are transformed into feature vectors of the same dimension and then standardized to eliminate dimensions. Obtain the equipment models corresponding to each area in the factory, and classify the historical operating data of all areas according to the same equipment model; Suppose there are areas in the factory area where equipment A is used. For all areas using equipment A, summarize the feature vectors of each area when no fault occurs, as vector set 'a'; summarize the feature vectors of each area when a fault occurs, as vector set 'b'. Calculate the mean vectors of the corresponding sets in vector sets 'a' and 'b' respectively, to obtain the center vector et of equipment A used when no fault occurs. （甲） With the center vector ef where the fault occurred （甲）; Obtain the number of regions using device A, nj; obtain the real-time operating data for all regions corresponding to device A; and convert this real-time operating data into a vector et with respect to the center. （甲） or center vector ef （甲） Feature vectors of the same dimension are used to obtain feature vectors ei for the regions corresponding to devices A from the 1st to the njth. （1） ~ei （nj） ; Calculate ei sequentially （1） ~ei （nj） Regarding the center vector et respectively （甲） With the center vector ef （甲） The center distance is used to obtain dt. （1） ~dt （nj） With df （1） ~df （nj） .

[0010] Furthermore, the subsequent steps for estimating the probability of failure in each region are as follows: Calculate the prior probability P of the regions corresponding to the first to njth users of device A. 0（1） ~P 0（nj） ; Let event A be: A malfunction occurs in the area where equipment A is used; Let event B be: the current running data appears in the corresponding area using device A; The prior probability P 0（1） ~P 0（nj） This is converted into the probability of a fault occurring in the area corresponding to the 1st to njth devices using device A, resulting in P(A). （1） ~P(A) （nj） ; Let P(B|A) be the likelihood probability of the first to nj regions corresponding to device A when a fault is detected and the current running data is available. （1） ~P(B|A) （nj） ; Based on the law of total probability, the probability of the current running data appearing in the area corresponding to the njth to njth devices using device A is calculated as P(B). （1） ~P(B) （nj） ; Based on Bayes' theorem, the probability of a fault occurring in the regions corresponding to the first to njth devices using device A under the current operating data is calculated, yielding P(A|B). （1） ~P(A|B) （nj） ; Calculate the probability of other regions experiencing failures under the current operating data of each region; Set a reference range for the probability of failure in each area, and classify different inspection levels for different areas based on the probability of failure. Then, different inspection routes are selected based on the actual situation of the inspection level in each area.

[0011] Furthermore, the steps for selecting different inspection routes are as follows: Get the total number of regions ns within the factory area, and get the total number of paths rn within the factory area. Each path connects one region to another, and there can be multiple paths from one region to another within the factory area. Obtain the path interference amount of each path within the factory area, introduce the interference amount coefficient, and set a piecewise function to assign different interference amount coefficient values ​​to the path interference amount of different intervals. Let the robot's initial moving speed be v. （0） The actual speed of the robot on the road is calculated to be v. （路） ; Scenario 1: If all areas have the same inspection level and are all low inspection levels, then select a robot with sufficient power to perform an overall inspection of all areas. Scenario 2: If the inspection levels of different areas are different, then the inspections will be carried out in stages according to the order of inspection level from high to low. Classify all areas within the factory according to inspection level, repeat the pathfinding steps in Scenario 1, and select the path with the least cumulative time cost for high inspection level, medium inspection level, and low inspection level in sequence.

[0012] Furthermore, the specific steps for scenario one are as follows: Branch 1: Minimum energy consumption; Calculate the energy consumption of the robot as it travels from path 1 to path rn; Define an energy cost matrix C of size ns × ns. （能） ; Matrix C （能） In matrix C, both rows and columns represent regions. （能） Each element in the array represents the minimum cumulative energy cost from the region corresponding to a row element or column element to the region corresponding to a column element or row element. The specific correspondence process is as follows: Let matrix C （能） The element in row a, column b is c. （a，b） There are an paths from region a to region b in the entire network. The energy consumption of these paths is: ei （1） ~ei （an） ; In ei （1） ~ei （an） Choose the smallest number as c. （a，b） The value; If there is no path from region a to region b in the entire path, then c （a，b） The value is ∞; If c （a，b） If the two subscripts in c are equal, then c （a，b） The value is zero; Use Floyd dynamic programming to iteratively find the path with the minimum energy consumption: In the entire region, determine a start point and an end point. Then, select one region from the remaining regions as a transit node. Traverse all paths from the start point to the end point. For each path, calculate the cumulative energy consumption for both paths that pass through transit nodes and those that do not. Iteratively select the path with the minimum cumulative energy consumption and update the matrix C. （能） The corresponding position in the middle.

[0013] Furthermore, the specific iterative process involves the following steps: Let the starting point be s （a） The endpoint is f （e） The transit node is t （c） Traverse all paths from the starting point to the ending point in the entire factory area and select the path with the minimum cumulative power consumption as the target path. Determine whether the target path passes through a transit node; If it has not been processed, then no action will be taken; If the path has been traversed, then extract the path from the starting point s in the target path. （a） To transit node t （c） The cumulative energy consumption is calculated, and matrix C is replaced with this cumulative energy loss. （能） Middle starting point s （a） With transit node t （c） The corresponding element; extract the transit node t from the target path. （c） to the finish line f （e） The cumulative energy consumption is calculated, and matrix C is replaced with this cumulative energy loss. （能） transit node t （c） and the endpoint f （e） The corresponding element; After completing one iteration, the endpoint f is... （e） As a new starting point, a new transit node is selected to continue the iteration until a path is obtained from the starting point s. （a） Paths leading to all areas of the factory; The path obtained through Floyd dynamic programming iteration is taken as the minimum energy consumption path of class A. Calculate the cumulative energy consumption of each path in the minimum energy consumption path of class A, and obtain E. （to） ; Determine if the robot's battery capacity is greater than E. （to） If it is not greater than, then no action is taken; If the length is greater than the specified length, obtain the length of all paths, calculate the time cost of each path based on the robot's actual speed on each path, reselect the path with the shortest inspection time, and proceed to branch two.

[0014] Furthermore, the specific steps for branch two are as follows: Branch 2: Shortest time; matrix C （能） The element replacement in the matrix is: the minimum cumulative time cost from the region corresponding to the row element or column element to the region corresponding to the column element or row element, resulting in matrix T. （时） ; Calculate the degree of each region to determine if there are any odd-degree points; If it does not exist, it means that there is an Eulerian circuit in all paths of the factory area. Use the Hierholzer algorithm to traverse all Eulerian circuits in the factory area and calculate the cumulative time cost corresponding to all Eulerian circuits. Select the Eulerian circuit with the cumulative time cost as the shortest inspection path in the factory area. If it exists, extract all regions corresponding to odd degrees as quasi-regions; In all paths, for each region, retain the shortest path from that region to any other region, so that the path from any region to another region in the factory area is unique. Pairing up two vehicles in all quasi-regions yields several pairing schemes; Calculate the cumulative time cost after pairing all quasi-regions in each scheme; select the scheme with the minimum cumulative time cost as the target scheme; increment the degree of all regions corresponding to odd degrees by 1, and adjust matrix T according to the pairing results of the target scheme. （时） Elements in; Based on the adjusted matrix T （时） We use Floyd dynamic programming to iteratively select the path with the minimum cumulative time cost.

[0015] Furthermore, the steps to define the quadrupedal walking motion constraints are as follows: Set the height threshold H for the tire to pass over ground obstacles. （th） Then determine whether the height of the obstacle exceeds H. （th） If the limit is not exceeded, no action is taken; if the limit is exceeded, the robot's quadrupedal walking model is enabled, and the motion constraints for quadrupedal walking are defined: Let the three-dimensional world coordinates of the obstacle be (xa （w） ya （w） , za （w） Using the robot's homogeneous matrix, the obstacle coordinates are projected from world coordinates to the robot's internal coordinates, resulting in (xb). （b） yb （b） zb（b） ); Obtain the robot's hip offset l （a） Thigh length l （b） calf length l （c） ; Let θa be the hip lateral swing angle of the i-th leg of the machine. （i） The hip pitch angle is θb （i） The knee joint angle is θc （i） ; The movement constraints for a single leg are: .

[0016] Compared with the prior art, the beneficial effects of the present invention are: A fault probability-driven hierarchical inspection mechanism: This invention mines historical operating data from various areas of the plant to estimate the probability of equipment failure in different areas, and uses this as a basis to classify inspection levels and match inspection path strategies, solving the problem of resource misallocation in the traditional equal inspection mode across the entire area. This method establishes a three-level mapping logic of "fault probability - inspection level - path strategy": areas with high fault probability correspond to high inspection levels and are included in the priority inspection sequence, ensuring the timeliness of hidden danger investigation in key production areas. This differentiated inspection mode can tilt limited inspection resources toward high-risk areas, improving the inspection effect of the plant area while ensuring inspection reliability.

[0017] Enhancing the robustness and fault tolerance of inspection tasks: This invention establishes a full-link anomaly handling logic encompassing real-time robot pose monitoring, operational status determination, and backup robot takeover. This enables path simulation to not only verify inspection effectiveness under ideal working conditions but also assess task assurance capabilities under abnormal scenarios. Traditional inspection path simulations are mostly based on the ideal assumption of normal robot operation throughout the entire process, without considering sudden situations such as mid-journey failures or posture anomalies, and lack corresponding task succession schemes, resulting in the inability to complete inspection tasks. In contrast, this invention compares robot pose data with motion constraints in real time as the basis for determining whether the state is normal or not. Once an anomaly is determined, a backup robot is immediately dispatched to take over and complete the remaining inspection tasks, improving the fault tolerance of the entire inspection system.

[0018] Supporting the Iterative Implementation of Inspection Solutions: This invention covers the process links of environmental modeling, risk assessment, path planning, and motion control, forming a complete digital simulation closed loop. It can support the rapid verification and iteration of inspection solutions, reducing the trial and error costs of on-site implementation. This invention also relies on a digital twin model to complete the simulation verification of the entire inspection process in virtual space. It can flexibly adjust key parameters such as fault probability threshold, inspection level classification standards, obstacle mode switching height, and backup scheduling rules. It can quickly compare the comprehensive indicators of different solutions, such as inspection coverage, task duration, energy consumption level, and fault tolerance, and select the optimal inspection solution that best suits the actual situation of the plant. Attached Figure Description

[0019] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the method of the present invention; Figure 2 This is a schematic diagram of the four robot groups of the present invention; Figure 3 This is a functional guidance diagram of the present invention. Detailed Implementation

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] Please see Figure 1 and Figure 2 The simulation method for factory inspection paths of wheeled quadruped robots based on digital twins includes: Acquire point cloud data and aerial images of the factory area, establish a digital virtual model of the factory area, associate all cameras in the factory area, load the personnel and vehicles captured by each camera into the virtual model of the factory area, monitor the flow of people and vehicles on each path in the factory area in real time, obtain the path interference of each path, and then define the motion constraints of the quadruped wheeled robot. Using satellite positioning technology, spatial data of various buildings and structures within the factory area (such as factory walls, area outlines, passageway boundaries, steps, low-lying areas, guardrails, pipeline supports, etc.) is acquired to establish the global spatial coordinates of the factory area. Simultaneously, aerial imagery and ground-based LiDAR point cloud data of the factory area are acquired, and preprocessing steps such as image distortion correction, panoramic stitching and point cloud denoising, and single-station stitching are performed. Using ground control points as a common reference, the aerial imagery and point cloud data are fused. Aerial data is used to fill in the scanning blind spots of the point cloud data. Then, the height data of each building and structure in the point cloud data is corrected based on the known heights of ground control points to obtain the full-area fused point cloud data of the factory area. The CAD drawing data of the factory area is imported, and the full-area fused point cloud data is calibrated a second time based on the engineering dimensions in the CAD drawing data. The calibrated full-area fused point cloud data is imported into modeling software to obtain the three-dimensional geometric model of the factory area. The geometric model is imported into the 3D engine and rendered to obtain the virtual model of the factory area. The system utilizes the RTSP / RTMP protocol to acquire and decode all video streams monitored within the factory area. Then, it employs the ROS and MQTT protocols to obtain the quadruped robot's pose data (including position coordinates, heading angle, and body posture). Using the YOLO target detection algorithm, it identifies and continuously tracks personnel and vehicles on various paths within the factory area from all video streams, extracting the pixel coordinates of personnel and vehicles from the video streams. Finally, using the intrinsic and extrinsic parameters of each camera, as well as the homogeneous matrix, it converts the extracted personnel and vehicle coordinates from the video streams from two-dimensional pixel coordinates into three-dimensional world coordinates. The 3D world coordinates of people and vehicles in the video stream are transmitted to the 3D engine using WebSocket or MQTT protocols. Kalman filtering is applied to the coordinate data to suppress detection jitter and noise. The virtual model of the factory area and the people and vehicles in the video stream are loaded into the 3D engine. The position and orientation of the virtual robot, people and vehicles are updated frame by frame according to the received real-time coordinates. The flow of people and vehicles on each path in the factory area is counted in real time as the path interference of each path. Define the motion constraints for a quadrupedal wheeled robot: A world coordinate system is established with the factory entrance as the origin, north as the positive vertical axis, east as the positive horizontal axis, and the vertical axis pointing upwards from the ground as the positive vertical axis. A robot coordinate system is established with the position of the robot's main camera as the origin, north as the positive vertical axis, east as the positive horizontal axis, and the vertical axis pointing upwards from the ground as the positive vertical axis. Let the world coordinates of building A within the factory area be (x （w） y （w） , z （w） Let the coordinates of the building inside the robot be (x, y). （b） y （b） , z （b） Let the robot's heading angle be φ. （航） The heading angle represents the angle between the robot's own horizontal axis (i.e., the x-axis) and the corresponding horizontal axis in the real world when the robot is moving forward. Let h be the vertical distance from the robot's main camera to the ground. （b） The homogeneous matrix T of the robot （b） for: ; where (x （w，原点） y （w，原点） This indicates the position of the origin in the robot's coordinate system within the world coordinate system; Define the robot's positioning constraints in world coordinates and robot interior coordinates (Constraint 1): ; Let the radius of the robot wheel be r. （b） The wheelbase is b （b）The angular velocities of the robot's left and right tires are ω and ω, respectively. （左） and ω （右） The robot's actual forward speed is v. （机） : ; The angular velocity of the steering is ω （机） : ; Let the robot's position in the world coordinate system be (x... o（w） y o（w） The heading angle is φo （航） Define the position change constraints of the robot during its movement (Constraint 2). ; Let the robot's position in the world coordinate system at time t be (x, y). o（w，t） y o（w，t） The heading angle is φo （航，t） The forward speed is v （机，t） angular velocity is ω （机，t） ; Let the robot's position in the world coordinate system at time (t+1) be (x... o（w，t+1） y o（w，t+1） The heading angle is φo （航，t+1） ; Let the time difference between time t and (t+1) be Δt. （机） Define the constraints on the robot's posture changes during movement (Constraint 3): .

[0022] Acquire and analyze historical operating data for each area within the plant (historical operating data includes: the existing operating time of each area within the plant, the number of times each area has experienced a failure, the duration of each area's failure, the operating data of each area when no failure occurred, and the operating data of each area when a failure occurred), and then combine this with the actual operating data of each area within the plant to estimate the probability of a failure in each area. Based on the probability of failure in each area, the inspection level of each area is divided, and then different inspection routes are selected according to the actual situation of the inspection level of each area. Scenario 1: If all areas have the same inspection level and are all low inspection levels, then select a robot with sufficient power to perform an overall inspection of all areas. Scenario 2: If the inspection levels of different areas are different, then the inspections will be carried out in stages according to the order of inspection level from high to low. Please see Figure 3 The estimated probability of failure in each region is as follows: Calculate the frequency f of failures in each region from historical operational data.（区） :f （区） =Total duration of failure in a single area / Total uptime of a single area; It should be noted that, unless otherwise specified, the term "area" in this invention refers to the area to be inspected within the factory premises. Based on the Poisson distribution, the prior probability of a region experiencing a fault at any time is P0(f （区） ): Where e represents the natural base; In the historical operational data, the operational data of each region when no fault occurred and the operational data when fault occurred are transformed into feature vectors of the same dimension and then standardized to eliminate dimensions. Obtain the equipment model (or equipment type) corresponding to each area in the factory, and classify the historical operation data corresponding to all areas according to the same equipment model (or equipment type); Suppose there are areas in the factory area that use equipment A. For all areas using equipment A, summarize the feature vectors of each area when no fault occurs (corresponding to operational data), as vector set 'a'; summarize the feature vectors of each area when a fault occurs (corresponding to operational data), as vector set 'b'. Calculate the mean vectors of the corresponding sets in vector sets 'a' and 'b' respectively, to obtain the center vector 'et' of equipment A when no fault occurs. （甲） With the center vector ef where the fault occurred （甲） ; Obtain the number of regions using device A, nj; obtain the real-time operating data for all regions corresponding to device A; and convert this real-time operating data into a vector et with respect to the center. （甲） or center vector ef （甲） Feature vectors of the same dimension are used to obtain the 1st, 2nd, and so on, up to the njth feature vector ei corresponding to the region using device A. （1） ei （2） ~ei （nj） ; Calculate ei sequentially （1） ei （2） ~ei （nj） Regarding the center vector et respectively （甲） With the center vector ef （甲） The center distance is used to obtain dt. （1） dt （2） ~dt （nj） With df （1） df （2） ~df （nj） ; Define ei （1） The corresponding region's fault association probability wi （1） : ; Define ei （2） The corresponding region's fault association probability wi （2） : ; And so on, we define ei （nj） The corresponding region's fault association probability wi （nj） : ; Based on the above definition, the prior probability is P0(f) （区） The formula is used to calculate the prior probability P of the region corresponding to the 1st, 2nd, up to the njth region using device A. 0（1） P 0（2） ~P 0（nj） ; Let event A be: A malfunction occurs in the area where equipment A is used; Let event B be: the current running data appears in the corresponding area using device A; The prior probability P 0（1） P 0（2） ~P 0（nj） This is transformed into the probability of a fault occurring in the area corresponding to the 1st, 2nd, and so on, up to the njth device using equipment A, resulting in P(A). （1） P(A) （2） ~P(A) （nj） (Equivalent to the inherent probability of failure in each region) Where, P(A) （1） P(A) （2） ~P(A) （nj） The value of P 0（1） P 0（2） ~P 0（nj） The values ​​correspond one-to-one; Let the likelihood probability of the first, second, and so on, up to the njth region corresponding to device A being detected as P(B|A) be P(B|A). （1） P(B|A) （2） ~P(B|A) （nj） Wherein, P(B|A) （1） P(B|A) （2） ~P(B|A) （nj） The value of wi （1） wi （2） ~wi （nj） The values ​​correspond one-to-one; Based on the law of total probability, the probability of the current running data appearing in the area corresponding to the 1st, 2nd, up to the njth device using device A is derived as P(B). （1） P(B) （2） ~P(B) （nj）; Where, P(B) （1） The formula for calculation is: ; P(B) （2） The formula for calculation is: ; And so on, P(B) （nj） The formula for calculation is: ; Based on Bayes' theorem, the probability of a fault occurring in the area corresponding to the 1st, 2nd, up to the njth device using equipment A under the current operating data is calculated, yielding P(A|B). （1） P(A|B) （2） ~P(A|B) （nj） ; Where, P(A|B) （1） The calculation formula is: ; P(A|B) （2） The calculation formula is: ; And so on, P(A|B) （nj） The calculation formula is: ; Repeatedly calculate P(A|B) （1） ~P(A|B) （nj） Using the same steps, calculate the probability of other regions failing under the current operating data of each region; if the historical operating data of a region using a certain model of equipment is relatively small (i.e., the proportion of historical operating data of the region using the same model of equipment in the "all historical operating data" is less than 2%), then when calculating the prior probability of these regions, the Laplace smoothing coefficient is introduced for probability correction. Set the probability reference range for each region [ma] （1） ma （2） The probability of failure will be less than ma. （1） Setting the area to a low inspection level will result in a failure probability within the range [ma]. （1） ma （2） The area within [ ] is set to medium inspection level, which will result in a failure probability greater than ma. （2） The area is set to a high inspection level; Then, different inspection routes are selected based on the actual situation of the inspection level in each area; Get the total number of regions ns within the factory area, and get the total number of paths rn within the factory area. Each path connects one region to another, and there can be multiple paths from one region to another within the factory area. Obtain the path interference amount for each path within the factory area, and introduce the interference amount coefficient λ. （扰） And set a piecewise function to assign different interference coefficient values ​​(interference coefficient λ) to the path interference in different intervals. （扰） (This represents the percentage decrease in the robot's movement speed during the inspection process). Let the robot's initial moving speed be v. （0） The robot's actual speed on the road is v. （路） : v （路） =v （0） ×(1-λ （扰） ); Scenario 1: If all areas have the same inspection level and are all low inspection levels, then select a robot with sufficient power to perform an overall inspection of all areas. Branch 1: Minimum energy consumption; Obtain the slope and rolling friction coefficient of each path along its mileage. Also obtain the robot's overall mass, gravitational acceleration, motor power-speed characteristic curves, and motor energy conversion efficiency. Combined with the robot's actual travel speed on each path, construct expressions for the instantaneous travel resistance and instantaneous traction power of each path's infinitesimal segment. Then, perform integration along the path mileage to obtain the energy consumption of the robot after traversing paths 1, 2, and finally rn: Eh （1） Eh （2） ~Eh （rn） ; Define an energy cost matrix C of size ns × ns. （能） ; Matrix C （能） The rows and columns in matrix C represent areas to be inspected within the factory area. （能） Each element in the array represents the minimum cumulative energy cost from the region corresponding to a row element or column element to the region corresponding to a column element or row element. The specific correspondence process is as follows: Let matrix C （能） The element in row a, column b is c. （a，b） There are an paths from region a to region b in the entire network. The energy consumption of these paths is: ei （1） ~ei （an） ; where, {ei （1） ~ei （an）}⊂{Eh （1） ~Eh （rn）}, the values ​​of a and b are both in the range of 1 to ns; In ei （1） ~ei （an） Choose the smallest number as c. （a，b） The value; If there is no path from region a to region b in the entire path, then c （a，b） The value is ∞; If c （a，b） If the two subscripts in c are equal (i.e., a = b), then c （a，b） The value is zero; Obtain the energy cost matrix C （能） Then, Floyd-Warshall dynamic programming is used to iteratively find the path with the minimum energy consumption: In the entire region, determine a start point and an end point. Then, in the remaining regions (i.e., regions other than the start and end points), select one region as a transit node. Traverse all paths from the start point to the end point. For each path, calculate the cumulative energy consumption for the path that passes through transit nodes and for the path that does not pass through transit nodes. Select the path with the minimum cumulative energy consumption and update the matrix C. （能） The specific iteration process at the corresponding position is as follows: Let the starting point be s （a） The endpoint is f （e） The transit node is t （c） Traverse all paths from the starting point to the ending point in the entire factory area and select the path with the minimum cumulative power consumption as the target path. Determine whether the target path passes through a transit node; If it has not been processed, then no action will be taken; If the path has been traversed, then extract the path from the starting point s in the target path. （a） To transit node t （c） The cumulative energy consumption is calculated, and matrix C is replaced with this cumulative energy loss. （能） Middle starting point s （a） With transit node t （c） The corresponding element; extract the transit node t from the target path. （c） to the finish line f （e） The cumulative energy consumption is calculated, and matrix C is replaced with this cumulative energy loss. （能） transit node t （c） and the endpoint f （e） The corresponding element; After completing one round (transfer point) of iteration, the destination f will be... （e） As a new starting point, a new transit node is selected to continue the iteration (i.e., the transit point iteration continues) until a path is obtained from the starting point s. （a） Paths leading to all areas of the factory (or for matrix C) （能） Update all elements in the file. The path obtained through Floyd dynamic programming iteration is taken as the minimum energy consumption path of class A. Calculate the cumulative energy consumption of each path in the minimum energy consumption path of class A, and obtain E. （to） ; Determine if the robot's battery capacity is greater than E. （to） If it is not greater than, then no action is taken; If it is greater than the length of all paths, calculate the time cost of each path based on the robot's actual speed on each path, reselect the path with the shortest inspection time, and proceed to branch two. Branch 2: Shortest time; matrix C （能） The element replacement in the matrix is: the minimum cumulative time cost from the region corresponding to the row element or column element to the region corresponding to the column element or row element, resulting in matrix T. （时） ; Calculate the degree of each region (degree represents the number of paths connected from a point) and determine if there are any odd-degree points; If it does not exist, it means that there is an Eulerian circuit in all paths of the factory area. Use the Hierholzer algorithm to traverse all Eulerian circuits in the factory area and calculate the cumulative time cost corresponding to all Eulerian circuits. Select the Eulerian circuit with the cumulative time cost as the shortest inspection path in the factory area. If it exists, extract all regions corresponding to odd degrees as quasi-regions; In all paths, for each region, retain the shortest path from that region to any other region, so that the path from any region to another region in the factory area is unique. Pairing up two vehicles in all quasi-regions yields several pairing schemes; Calculate the cumulative time cost after pairing all quasi-regions in each scheme; select the scheme with the minimum cumulative time cost as the target scheme; increment the degree of all regions corresponding to odd degrees by 1, and adjust matrix T according to the pairing results of the target scheme. （时） Elements in; Based on the adjusted matrix T （时） Floyd dynamic programming is used to iteratively select the path with the minimum cumulative time cost; Scenario 2: If the inspection levels of different areas are different, then the inspections will be carried out in stages according to the order of inspection level from high to low. Classify all areas within the factory area according to the inspection level, repeat the branch two pathfinding steps in Case 1, and select the path with the least cumulative time cost for high inspection level, medium inspection level and low inspection level in turn. The robot acquires its pose data (including position coordinates, heading angle, and body posture) in real time and determines whether the robot is functioning correctly based on its motion constraints. If it is functioning correctly, the robot's quadrupedal walking model is selectively activated based on the actual height of obstacles encountered by the robot on the inspection path. If it is not functioning correctly, a new quadrupedal wheeled robot is assigned to complete the inspection task. Motion constraints for robots when using quadrupedal walking: Set the height threshold H for the tire to pass over ground obstacles. （th） That is, the robot's tires will drive over heights exceeding H. （th） When encountering an object, stop the tire rotation and switch to quadrupedal walking inspection: The robot is equipped with a machine vision processor, which can identify obstacles encountered during inspection and guide the robot to avoid them. If the robot cannot avoid an obstacle (e.g., there are vehicles or a large number of pedestrians on the path), it determines whether the height of the obstacle exceeds H. （th） If the limit is not exceeded, no action is taken; if the limit is exceeded, the robot's quadrupedal walking model is enabled, and the motion constraints for quadrupedal walking are defined: Let the three-dimensional world coordinates of the obstacle be (xa （w） ya （w） za （w） Using the robot's homogeneous matrix, the obstacle coordinates are projected from world coordinates to the robot's internal coordinates, resulting in (xb). （b） yb （b） zb （b） ); Obtain the robot's hip offset l （a） Thigh length l （b） calf length l （c） ; Let θa be the hip lateral swing angle of the i-th leg of the machine. （i） The hip pitch angle is θb （i） The knee joint angle is θc （i） ; Wherein, hip lateral swing angle θa （1） The hip joint lateral swing motor at the base of the robot's leg drives the entire leg to swing laterally to the left and right sides of the robot body, achieving leg abduction and adduction. Hip pitch angle θb （i） The hip joint pitch motor controls the robot's hip joint pitch, which drives the entire thigh to swing back and forth, and is the core of quadrupedal walking forward and backward. knee joint angle θc （i） The robot controls the knee joint motor at the connection between the thigh and lower leg, driving the lower leg to bend and extend relative to the thigh; when lifting the leg to take a step, the knee bends to raise the foot wheel, and when landing to support the body, the knee extends to cushion the impact; The movement constraints for a single leg are: .

[0023] The above formulas are all dimensionless calculations. The formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation. For example, there are weighting coefficients and proportional coefficients. The values ​​set are to quantify each parameter to obtain a specific value, which is convenient for subsequent comparison. The values ​​of the weighting coefficients and proportional coefficients are only required to not affect the proportional relationship between the parameters and the quantified values.

[0024] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A simulation method for factory inspection paths of a wheeled quadruped robot based on digital twins, characterized in that, Simulation methods include: Acquire point cloud data and aerial images of the factory area, establish a digital virtual model of the factory area, load personnel and vehicles into the virtual model of the factory area, monitor the flow of people and vehicles on each path in the factory area in real time, obtain the path interference of each path, and define the motion constraints of the quadruped wheeled robot. Acquire and analyze historical operating data for each area within the plant, and combine this with actual operating data for each area to estimate the probability of failure in each area. Based on the probability of failure in each area, the inspection level of each area will be divided. Then, based on the actual situation of the inspection level of each area, different inspection routes will be selected to carry out overall inspection or phased inspection. The robot acquires the pose data of the quadruped wheeled robot in real time and determines whether the robot is functioning normally based on its motion constraints. If it is functioning normally, the robot selectively enables its quadruped walking mode and defines motion constraints based on the actual obstacle height on the inspection path. If it is not functioning normally, a new quadruped wheeled robot is assigned to complete the inspection task.

2. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 1, characterized in that, The steps for creating a digital virtual model are as follows: Spatial data of various buildings and structures within the factory area were acquired to establish the global spatial coordinates of the factory area. Simultaneously, aerial imagery and ground-based LiDAR point cloud data of the factory area were acquired, and image distortion correction, panoramic stitching and point cloud denoising, and single-station stitching preprocessing were performed respectively. Using ground control points as a common reference, aerial imagery and point cloud data were fused. Aerial data is used to fill in the scanning blind spots of point cloud data. The height data of each building and structure in the point cloud data is corrected based on the known height of the ground control points to obtain the full-area fused point cloud data of the factory area. The full-area fused point cloud data is then calibrated a second time based on the CAD drawing data of the factory area to construct a three-dimensional geometric model of the factory area and import it into the three-dimensional engine to obtain the virtual model of the factory area. The system acquires and decodes all video streams from the factory area, then obtains the pose data of the quadruped robot. Using a target detection algorithm, it extracts the pixel coordinates of personnel and vehicles from the video stream. By utilizing the intrinsic and extrinsic parameters of each camera and the homogeneous matrix, the personnel and vehicles extracted from the video stream are converted from two-dimensional pixel coordinates to three-dimensional world coordinates. The virtual model of the factory area and the personnel and vehicles in the video stream are loaded into the 3D engine. The position and orientation of the virtual robot, personnel and vehicles are updated frame by frame according to the received real-time coordinates. At the same time, the flow of people and vehicles on each path in the factory area are counted as the path interference of each path.

3. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 1, characterized in that, The steps for defining motion constraints are as follows: Let the world coordinates of building A within the factory area be (x （w） y （w） , z （w） Let the coordinates of the building inside the robot be (x, y). （b） y （b） , z （b） Let the robot's heading angle be φ. （航） , ; Let h be the vertical distance from the robot's main camera to the ground. （b） The homogeneous matrix T of the robot is obtained. （b） Define the positioning constraints of the robot in world coordinates and robot interior coordinates: Let the radius of the robot wheel be r. （b） The wheelbase is b （b） The angular velocities of the robot's left and right tires are ω and ω, respectively. （左） and ω （右） The robot's actual forward speed is obtained as v. （机） and Steering angular velocity ω （机） ; Let the robot's position in the world coordinate system be (x... o（w） y o（w） The heading angle is φo （航） Define the constraints on the position changes of the robot during its movement; 。 4. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 2, characterized in that, The steps for estimating the probability of failure in each area are as follows: In the historical operational data, the operational data of each region when no fault occurred and the operational data when fault occurred are transformed into feature vectors of the same dimension and then standardized to eliminate dimensions. Obtain the equipment models corresponding to each area in the factory, and classify the historical operating data of all areas according to the same equipment model; Suppose there are areas in the factory area where equipment A is used. For all areas using equipment A, summarize the feature vectors of each area when no fault occurs, as vector set 'a'; summarize the feature vectors of each area when a fault occurs, as vector set 'b'. Calculate the mean vectors of the corresponding sets in vector sets 'a' and 'b' respectively, to obtain the center vector et of equipment A used when no fault occurs. （甲） With the center vector ef where the fault occurred （甲） ; Obtain the number of regions using device A, nj; obtain the real-time operating data for all regions corresponding to device A; and convert this real-time operating data into a vector et with respect to the center. （甲） or center vector ef （甲） Feature vectors of the same dimension are used to obtain feature vectors ei for the regions corresponding to devices A from the 1st to the njth. （1） ~ei （nj） ; Calculate ei sequentially （1） ~ei （nj） Regarding the center vector et respectively （甲） With the center vector ef （甲） The center distance is used to obtain dt. （1） ~dt （nj） With df （1） ~df （nj） .

5. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 4, characterized in that, The following steps follow the estimation of the probability of failure in each area: Calculate the prior probability P of the regions corresponding to the first to njth users of device A. 0（1） ~P 0（nj） ; Let event A be: A malfunction occurs in the area where equipment A is used; Let event B be: the current running data appears in the corresponding area using device A; The prior probability P 0（1） ~P 0（nj） This is converted into the probability of a fault occurring in the area corresponding to the 1st to njth devices using device A, resulting in P(A). （1） ~P(A) （nj） ; Let P(B|A) be the likelihood probability of the first to nj regions corresponding to device A when a fault is detected and the current running data is available. （1） ~P(B|A) （nj） ; Calculate the probability P(B) of the current running data appearing in the regions corresponding to the nth to nth devices using device A. （1） ~P(B) （nj） ; Based on Bayes' theorem, the probability of a fault occurring in the regions corresponding to the 1st to njth devices using device A under the current operating data is calculated, yielding P(A|B). （1） ~P(A|B) （nj） ; Calculate the probability of other regions experiencing failures under the current operating data of each region; Set a reference range for the probability of failure in each area, and classify different inspection levels for different areas based on the probability of failure. Different inspection routes are selected based on the actual situation of each area's inspection level.

6. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 4, characterized in that, The steps for selecting different inspection routes are as follows: Get the total number of regions ns within the factory area, and get the total number of paths rn within the factory area. Each path connects one region to another. Obtain the path interference amount of each path within the factory area, introduce the interference amount coefficient, and set a piecewise function to assign different interference amount coefficient values ​​to the path interference amount of different intervals. Let the robot's initial moving speed be v. （0） The actual speed of the robot on the road is calculated to be v. （路） ; Scenario 1: If all areas have the same inspection level and are all low inspection levels, then select a robot with sufficient power to perform an overall inspection of all areas. Scenario 2: If the inspection levels of different areas are different, then the inspections will be carried out in stages according to the order of inspection level from high to low. Classify all areas within the factory according to inspection level, repeat the pathfinding steps in Scenario 1, and select the path with the least cumulative time cost for high inspection level, medium inspection level, and low inspection level in sequence.

7. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 6, characterized in that, The specific steps for scenario one are as follows: Branch 1: Minimum energy consumption; Calculate the energy consumption of the robot as it travels from path 1 to path rn; Define an energy cost matrix C of size ns × ns. （能） ; Matrix C （能） In matrix C, both rows and columns represent regions. （能） Each element in the array represents the minimum cumulative energy cost from the region corresponding to a row element or column element to the region corresponding to a column element or row element. The specific correspondence process is as follows: Let matrix C （能） The element in row a, column b is c. （a，b） There are an paths from region a to region b in the entire network. The energy consumption of these paths is: ei （1） ~ei （an） ; In ei （1） ~ei （an） Choose the smallest number as c. （a，b） The value; If there is no path from region a to region b in the entire path, then c （a，b） The value is ∞; If c （a，b） If the two subscripts in c are equal, then c （a，b） The value is zero; Use Floyd dynamic programming to iteratively find the path with the minimum energy consumption: In the entire region, determine a start point and an end point. In the remaining regions, select one region as a transit node. Traverse all paths from the start point to the end point. For each path, calculate the cumulative energy consumption for both paths that pass through transit nodes and those that do not. Iteratively select the path with the minimum cumulative energy consumption and update the matrix C. （能） The corresponding position in the middle.

8. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 7, characterized in that, The specific iterative process involves the following steps: Let the starting point be s （a） The endpoint is f （e） The transit node is t （c） Traverse all paths from the starting point to the ending point in the entire factory area and select the path with the minimum cumulative power consumption as the target path. Determine whether the target path passes through a transit node; If it has not been processed, then no action will be taken; If the path has been traversed, then extract the path from the starting point s in the target path. （a） To transit node t （c） The cumulative energy consumption is calculated, and matrix C is replaced with this cumulative energy loss. （能） Middle starting point s （a） With transit node t （c） The corresponding element; extract the transit node t from the target path. （c） to the finish line f （e） The cumulative energy consumption is calculated, and matrix C is replaced with this cumulative energy loss. （能） transit node t （c） and the endpoint f （e） The corresponding element; After completing one iteration, the endpoint f is... （e） As a new starting point, a new transit node is selected to continue the iteration until a path is obtained from the starting point s. （a） Paths leading to all areas of the factory; The path obtained through Floyd dynamic programming iteration is taken as the minimum energy consumption path of class A. Calculate the cumulative energy consumption of each path in the minimum energy consumption path of class A, and obtain E. （to） ; Determine if the robot's battery capacity is greater than E. （to） If it is not greater than, then no action is taken; If the length is greater than the specified length, obtain the length of all paths, calculate the time cost of each path based on the robot's actual speed on each path, reselect the path with the shortest inspection time, and proceed to branch two.

9. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 8, characterized in that, The specific steps for branch two are as follows: Branch 2: Shortest time; matrix C （能） The element replacement in the matrix is: the minimum cumulative time cost from the region corresponding to the row element or column element to the region corresponding to the column element or row element, resulting in matrix T. （时） ; Calculate the degree of each region to determine if there are any odd-degree points; If it does not exist, it means that there is an Eulerian circuit in all paths of the factory area. Traverse all Eulerian circuits in the factory area and calculate the cumulative time cost corresponding to all Eulerian circuits. Select the Eulerian circuit with the cumulative time cost as the shortest inspection path in the factory area. If it exists, extract all regions corresponding to odd degrees as quasi-regions; In all paths, for each region, retain the shortest path from that region to any other region, so that the path from any region to another region in the factory area is unique. Pairing up two vehicles in all quasi-regions yields several pairing schemes; Calculate the cumulative time cost after pairing all quasi-regions in each scheme; The solution with the minimum cumulative time cost is selected as the target solution. Increment the degree of all regions corresponding to odd degrees by 1, and adjust matrix T according to the pairing results of the target scheme. （时） Elements in; Based on the adjusted matrix T （时） We use Floyd dynamic programming to iteratively select the path with the minimum cumulative time cost.

10. The simulation method for factory inspection path of a wheeled quadruped robot based on digital twin as described in claim 1, characterized in that, The steps to define quadrupedal walking motion constraints are as follows: Set the height threshold H for the tire to pass over ground obstacles. （th） Determine whether the height of the obstacle exceeds H. （th） If the limit is not exceeded, no action is taken; if the limit is exceeded, the robot's quadrupedal walking model is enabled, and the motion constraints for quadrupedal walking are defined: Let the three-dimensional world coordinates of the obstacle be (xa （w） ya （w） za （w） Using the robot's homogeneous matrix, the obstacle coordinates are projected from world coordinates to the robot's internal coordinates, resulting in (xb). （b） yb （b） zb （b） ); Obtain the robot's hip offset l （a） Thigh length l （b） calf length l （c） ; set up The hip lateral swing angle of the i-th leg of the machine is θa （i） The hip pitch angle is θb （i） The knee joint angle is θc （i） ; The movement constraints for a single leg are: 。

Images