A Visual Intelligent Inspection Method and System for Rail Transit Based on Digital Twin

Abstract

This invention provides a digital twin-based visual intelligent inspection method and system for rail transit, belonging to the field of visual intelligent inspection technology. The method includes: constructing a three-dimensional digital twin of the rail transit scene; adaptively performing visual inspection based on the three-dimensional digital twin; generating a response plan when an abnormal event is detected; simulating and verifying the response plan; and deploying the verified response plan to the rail transit scene for execution. Based on the constructed three-dimensional digital twin of the rail transit scene, adaptive visual inspection is performed. When an abnormal event is detected, a response plan is generated, and after simulation verification, it is deployed to the rail transit scene for execution. This reduces the manpower cost of inspection personnel, improves the comprehensiveness and accuracy of inspections, enhances the timeliness of the occurrence and response to emergencies in the rail transit scene, and ensures the safe operation of rail transit.

Description

Technical Field

[0001] This invention relates to the field of visual intelligent inspection technology, and in particular to a visual intelligent inspection method and system for rail transit based on digital twins. Background Technology

[0002] Currently, with the rapid development of modern rail transit, its safety and operational efficiency have become important goals for the industry.

[0003] In the operation and maintenance of rail transit, equipment inspection is one of the key links to ensure safe operation. Traditional inspection methods usually rely on manual inspection, where inspectors need to check each piece of equipment one by one and then deal with any problems found.

[0004] However, manual inspection has certain limitations, such as a limited number of inspectors, high workload, difficulty in ensuring the comprehensiveness and accuracy of inspections, and in the event of an emergency, manual inspection often cannot detect and respond in a timely manner, affecting the safe operation of rail transit. Summary of the Invention

[0005] One of the objectives of this invention is to provide a digital twin-based visual intelligent inspection method for rail transit. Based on a constructed three-dimensional digital twin of the rail transit scene, the method adaptively performs visual inspections. When an abnormal event is detected, a response plan for the abnormal event is generated and verified through simulation before being deployed to the rail transit scene for execution. This reduces the manpower cost of inspection personnel, improves the comprehensiveness and accuracy of inspections, and enhances the timeliness of the occurrence and response to emergencies in the rail transit scene, thus ensuring the safe operation of rail transit.

[0006] This invention provides a visual intelligent inspection method for rail transit based on digital twins, comprising:

[0007] Constructing a three-dimensional digital twin of a rail transit scenario;

[0008] Based on a 3D digital twin, adaptive visual inspection is performed;

[0009] When visual inspection detects an abnormal event, a response plan for the abnormal event is generated.

[0010] The response plan was simulated and verified.

[0011] The solutions that have passed the simulation verification will be deployed to the rail transit scenario for implementation.

[0012] Optionally, the construction of the three-dimensional digital twin of the rail transit scene includes:

[0013] Invoke the digital twin engine;

[0014] Configure the digital twin engine with autonomous custom modeling capabilities;

[0015] Based on the digital twin engine configured with autonomous evolutionary modeling capabilities, a three-dimensional digital twin is constructed according to the multimodal on-site information of the rail transit scenario.

[0016] Optionally, the adaptive visual inspection based on a 3D digital twin includes:

[0017] Deploy an array of inspection cameras within a three-dimensional digital twin;

[0018] Based on an inspection camera array, adaptive visual inspection is performed in a 3D digital twin.

[0019] Optionally, when the visual inspection detects an abnormal event, the decision to generate a response plan for the abnormal event includes:

[0020] Identify the corresponding response plan decision knowledge from the response plan decision knowledge base;

[0021] By comparing the decision-making knowledge of response plans, a response plan for abnormal events is generated.

[0022] Optionally, the countermeasures are simulated and verified, including:

[0023] The response plan will be simulated and executed in a three-dimensional digital twin;

[0024] Continuously monitor the process information of the simulated execution of the response plan in the 3D digital twin;

[0025] Based on the simulation verification and evaluation system, simulation verification and evaluation are carried out according to process information to obtain evaluation values;

[0026] When the evaluation value exceeds the evaluation threshold, the simulation verification of the solution is considered successful.

[0027] Optionally, after constructing the three-dimensional digital twin of the rail transit scene, the method further includes:

[0028] When a target person in a rail transit scenario inputs a non-standard event mapping request, the system assists the target person in mapping the non-standard event into a 3D digital twin.

[0029] Optionally, the mapping of non-standard events into the three-dimensional digital twin body by the assisting target personnel includes:

[0030] Send a quick selection table of mapping scenarios to the target personnel;

[0031] Based on the preset mapping indication knowledge corresponding to the mapping scenario selected by the target personnel from the mapping scenario quick selection table, a fuzzy acquisition route is planned.

[0032] The system assists target personnel in collecting coarse information about events by moving along a fuzzy acquisition route.

[0033] Preprocess the coarse information of the event to obtain non-standard events;

[0034] Non-standard events are mapped into a three-dimensional digital twin.

[0035] Optionally, the steps for determining the mapping scenario quick selection table are as follows:

[0036] Identify multiple key points within a preset range surrounding the current location of the target person from a 3D digital twin;

[0037] Draw the intention-time curve for each key point;

[0038] Extract multiple target peak sets from each intention-time curve; wherein each target peak set contains more than N peaks that originate from different intention-time curves, and the absolute value of the time difference between each pair of peaks in each target peak set does not exceed the absolute value threshold, and each peak in each target peak set exceeds the peak threshold; N is an integer threshold greater than or equal to 2.

[0039] Multiple key point sets are identified; each key point set contains key points corresponding to the intention-time curves from which the peak values ​​contained in the target peak set originate.

[0040] Determine the target candidate mapping scenarios corresponding to each key point set from the candidate mapping scenario library;

[0041] When the total number of target candidate mapping scenarios corresponding to each key point set does not exceed the number threshold, the target candidate mapping scenarios corresponding to each key point set are arranged in a unified planar list to obtain a quick selection table of mapping scenarios; otherwise, the target candidate mapping scenarios corresponding to each key point set are arranged in a multi-planar list according to the priority of the scenarios from large to small to obtain a quick selection table of mapping scenarios.

[0042] The steps for plotting the intention-time curve for any key point are as follows:

[0043] The intention degree between the transient trajectory of the target person at different times within the most recent preset time and any of the key points is calculated, and the corresponding time is associated to form a one-to-one correlation degree and time.

[0044] The one-to-one correlation degree and time are mapped into the template curve to obtain multiple curve coordinates;

[0045] Connect the coordinates of each curve in sequence to obtain the intention-time curve;

[0046] The steps for calculating the intentionality between the transient trajectory and key points are as follows:

[0047] Obtain the first and second standard ranges of the key points respectively;

[0048] Extract the first partial trajectory from the transient trajectory after entering the first standard range and before entering the second standard range;

[0049] Obtain the standard trajectory corresponding to the first local trajectory;

[0050] Extract the second local trajectory from the transient trajectory after entering the second standard range;

[0051] The similarity between the second local trajectory and the standard trajectory is used as the intentionality.

[0052] The steps for determining the priority of any candidate target mapping scenario are as follows:

[0053] Based on the average value of the peak values ​​contained in the target peak set corresponding to the key point set of the target candidate mapping scenario and the minimum time difference between the corresponding time and the current time, the scenario priority is calculated; where the average value is positively correlated with the scenario priority, and the minimum time difference is negatively correlated with the scenario priority.

[0054] Optionally, the step of planning a fuzzy acquisition route based on the preset mapping indication knowledge corresponding to the mapping scenario selected by the target personnel from the mapping scenario quick selection table includes:

[0055] Based on mapping indication knowledge, multiple acquisition targets are identified from a 3D digital twin;

[0056] Generate multiple sets of acquisition targets; wherein each set of acquisition targets contains at least two acquisition targets and at least one non-acquisition target, and the non-acquisition targets contained therein conform to the constraints corresponding to the relative relationships between the acquisition targets contained therein;

[0057] Plan the target route for each set of data collection targets; where the target route is the shortest route that sequentially passes through each data collection target and non-data collection target in the same set of data collection targets.

[0058] The shortest route connecting the target routes of each set of acquisition targets is used to obtain the fuzzy acquisition route.

[0059] This invention provides a digital twin-based visual intelligent inspection system for rail transit, comprising:

[0060] The building module is used to construct a 3D digital twin of a rail transit scenario;

[0061] The visual inspection module is used to perform adaptive visual inspection based on a 3D digital twin.

[0062] The decision module is used to generate a response plan for abnormal events when visual inspection detects them.

[0063] The simulation verification module is used to simulate and verify the response plan;

[0064] The execution module is used to distribute the simulated and verified response solutions to the rail transit scenario for execution.

[0065] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0066] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0067] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0068] Figure 1 This is a schematic diagram of a visual intelligent inspection method for rail transit based on digital twins, as described in an embodiment of the present invention.

[0069] Figure 2 This is a schematic diagram of a digital twin-based visual intelligent inspection system for rail transit, as described in an embodiment of the present invention. Detailed Implementation

[0070] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0071] This invention provides a visual intelligent inspection method for rail transit based on digital twins, such as... Figure 1 As shown, it includes:

[0072] S1. Construct a three-dimensional digital twin of the rail transit scenario;

[0073] S2. Based on a 3D digital twin, adaptive visual inspection is performed.

[0074] S3. When visual inspection detects an abnormal event, a response plan for the abnormal event is generated.

[0075] S4. Conduct simulation verification of the response plan;

[0076] S5. Deploy the simulated and verified response plan to the rail transit scenario for implementation.

[0077] The rail transit scenario refers to the operation scenario of rail transit. After constructing a 3D digital twin, the operation status of the rail transit scenario will be displayed in real time in the 3D digital twin. Therefore, visual inspection can be adaptively performed based on the 3D digital twin. Abnormal events refer to operational abnormal events that occur in the rail transit operation scenario. When an abnormal event is detected by visual inspection, a response plan for the abnormal event is generated. After the response plan is generated, it is simulated and verified, and then the response plan that passes the simulation verification is sent to the rail transit scenario for execution.

[0078] This application uses a constructed 3D digital twin of a rail transit scenario to adaptively perform visual inspections. When an abnormal event is detected by the visual inspection, a response plan for the abnormal event is generated. After simulation verification, the plan is then sent to the rail transit scenario for execution. This reduces the manpower cost of inspection personnel, improves the comprehensiveness and accuracy of inspections, and enhances the timeliness of the occurrence and response to emergencies in the rail transit scenario, thus ensuring the safe operation of rail transit.

[0079] In one embodiment, the construction of the three-dimensional digital twin of the rail transit scene includes:

[0080] Invoke the digital twin engine;

[0081] Configure the digital twin engine with autonomous custom modeling capabilities;

[0082] Based on the digital twin engine configured with autonomous evolutionary modeling capabilities, a three-dimensional digital twin is constructed according to the multimodal on-site information of the rail transit scenario.

[0083] A digital twin engine is a tool for implementing digital twin technology, such as PTC ThingWorx. Configuring autonomous adaptive modeling capabilities refers to enabling the digital twin engine to dynamically evolve and configure a 3D digital twin based on dynamically changing on-site information (such as environmental changes and complex scene changes). This allows the digital twin engine to continuously optimize and adjust the model based on real-time data and feedback, thereby better adapting to the complex changes and uncertainties of the physical system. The multimodal on-site information of the rail transit scene includes at least: on-site images of the rail transit scene collected by multiple sensors and monitoring equipment, equipment operation information, staff distribution, and staff dynamics. After configuring autonomous adaptive modeling capabilities, the digital twin engine can construct a 3D digital twin based on the multimodal on-site information of the rail transit scene.

[0084] In one embodiment, the adaptive visual inspection based on a 3D digital twin includes:

[0085] Deploy an array of inspection cameras within a three-dimensional digital twin;

[0086] Based on an inspection camera array, adaptive visual inspection is performed in a 3D digital twin.

[0087] The inspection camera array contains multiple virtual cameras, which move dynamically and in a distributed manner within a 3D digital twin to continuously detect abnormal events and achieve visual inspection.

[0088] In one embodiment, when an abnormal event is detected by visual inspection, the step of generating a response plan for the abnormal event includes:

[0089] Identify the corresponding response plan decision knowledge from the response plan decision knowledge base;

[0090] By comparing the decision-making knowledge of response plans, a response plan for abnormal events is generated.

[0091] A pre-set response plan decision knowledge base is established, which contains response plan decision knowledge corresponding to different abnormal events. The response plan decision knowledge indicates how to handle abnormal events, so response plans for abnormal events can be generated by referring to the response plan decision knowledge.

[0092] In one embodiment, the simulation verification of the countermeasure includes:

[0093] The response plan will be simulated and executed in a three-dimensional digital twin;

[0094] Continuously monitor the process information of the simulated execution of the response plan in the 3D digital twin;

[0095] Based on the simulation verification and evaluation system, simulation verification and evaluation are carried out according to process information to obtain evaluation values;

[0096] When the evaluation value exceeds the evaluation threshold, the simulation verification of the solution is considered successful.

[0097] During simulation verification, the solution is simulated and executed in a 3D digital twin, and process information is continuously monitored. The process information includes at least the latest dynamics of abnormal events after simulation execution. A simulation verification evaluation system is preset, and the simulation verification is evaluated based on the process information to obtain evaluation values. The evaluation values ​​represent the degree of effectiveness of the simulation verification. An evaluation threshold is preset. When the evaluation value exceeds the evaluation threshold, it indicates that the effectiveness is good and the simulation verification of the solution is passed.

[0098] The comprehensiveness of the 3D digital twin construction directly determines the effectiveness of subsequent visual inspection based on it. However, in the actual application scenarios of this application, there are often some events that cannot be reflected by the 3D digital twin constructed solely based on multimodal on-site information, such as events occurring in blind spots of monitoring in rail transit scenarios, monitoring information lost due to abnormal events causing related monitoring equipment to malfunction, and information that requires comprehensive analysis in special on-site situations.

[0099] Therefore, to address the aforementioned issues, in one embodiment, after constructing the three-dimensional digital twin of the rail transit scene, the method further includes:

[0100] When a target person in a rail transit scenario inputs a non-standard event mapping request, the system assists the target person in mapping the non-standard event into a 3D digital twin.

[0101] When personnel patrolling a rail transit scenario and encounter events—non-standard events—that cannot be reflected in the 3D digital twin, they can input a non-standard event mapping request. The system will then assist the personnel in mapping these non-standard events into the 3D digital twin. This significantly improves the comprehensiveness of the 3D digital twin construction and enhances the effectiveness of visual inspections based on it.

[0102] However, the scenes where non-standard events are discovered are often quite complex. It is quite troublesome for the target personnel to map non-standard events into a 3D digital twin. They need to collect on-site information in a targeted manner first, and then organize the information to prepare for mapping. The process is very cumbersome. Moreover, if non-standard events are not mapped in time, visual inspection may miss a lot of new abnormal events.

[0103] Furthermore, to address these issues, in one embodiment, the assisting target person mapping non-standard events into the three-dimensional digital twin includes:

[0104] Send a quick selection table of mapping scenarios to the target personnel;

[0105] Based on the preset mapping indication knowledge corresponding to the mapping scenario selected by the target personnel from the mapping scenario quick selection table, a fuzzy acquisition route is planned.

[0106] The system assists target personnel in collecting coarse information about events by moving along a fuzzy acquisition route.

[0107] Preprocess the coarse information of the event to obtain non-standard events;

[0108] Non-standard events are mapped into a three-dimensional digital twin.

[0109] The system comprehensively infers which mapping scenario the target personnel currently want to map non-standard events to, generating a quick selection table for mapping scenarios. Each mapping scenario corresponds to pre-set mapping indications, specifying which targets need information collection. Therefore, a fuzzy acquisition route can be planned based on this, helping the target personnel roughly understand how to move to collect information. This assists in collecting coarse event information based on the fuzzy acquisition route. The coarse event information needs to be pre-processed to obtain non-standard events, which are then mapped into a 3D digital twin. During pre-processing, the coarse event information can be converted into events of a corresponding format based on a pre-set event format, and operations such as information denoising and outlier removal can also be performed.

[0110] When the target personnel see the mapping scenario quick selection table, they quickly select the mapping scenario and then move along the fuzzy acquisition route with the assistance of the system to collect coarse event information. At this point, all the work that the target personnel need to do is completed, which greatly improves the convenience of mapping non-standard events for the target personnel. The system preprocesses the coarse event information to obtain non-standard events, and then maps the non-standard events into the 3D digital twin, which improves the timeliness of non-standard event mapping and avoids visual inspection from missing a lot of new abnormal events, thus greatly improving the system's work efficiency.

[0111] The input of a scenario involves multiple scenario elements. It is time-consuming for the target personnel to input the information themselves. Therefore, it is necessary to generate a mapping scenario quick selection table. However, the on-site environment of the target personnel and their inspection history are relatively difficult to determine. How to accurately and comprehensively infer which non-standard event the target personnel want to map to has become an urgent problem to be solved.

[0112] Therefore, in one embodiment, the steps for determining the mapping scenario quick selection table are as follows:

[0113] Identify multiple key points within a preset range surrounding the current location of the target person from a 3D digital twin;

[0114] Draw the intention-time curve for each key point;

[0115] Extract multiple target peak sets from each intention-time curve; wherein each target peak set contains more than N peaks that originate from different intention-time curves, and the absolute value of the time difference between each pair of peaks in each target peak set does not exceed the absolute value threshold, and each peak in each target peak set exceeds the peak threshold; N is an integer threshold greater than or equal to 2.

[0116] Multiple key point sets are identified; each key point set contains key points corresponding to the intention-time curves from which the peak values ​​contained in the target peak set originate.

[0117] Determine the target candidate mapping scenarios corresponding to each key point set from the candidate mapping scenario library;

[0118] When the total number of target candidate mapping scenarios corresponding to each key point set does not exceed the number threshold, the target candidate mapping scenarios corresponding to each key point set are arranged in a unified planar list to obtain a quick selection table of mapping scenarios; otherwise, the target candidate mapping scenarios corresponding to each key point set are arranged in a multi-planar list according to the priority of the scenarios from large to small to obtain a quick selection table of mapping scenarios.

[0119] The surrounding preset range refers to a circular area with a radius of 50 meters around the current location of the target personnel. Key points are equipment locations that the target personnel may want to use as the basis for mapping non-standard scenarios, such as operating equipment and monitoring equipment. An intention-time curve is plotted for each key point, reflecting the recent change in the target personnel's intention to use the key point as a basis for mapping non-standard scenarios. The absolute threshold can be 120 seconds; the peak threshold is a threshold representing a large peak value, for example, 20. There is a one-to-one correspondence between the key point set and the target peak set. The key points in the key point set represent equipment locations where the target personnel have consistently shown a high intention to use them as the basis for mapping non-standard scenarios in the recent period. Therefore, it can reflect what non-standard scenarios the target personnel may want to map. Thus, the target candidate mapping scenarios corresponding to each key point set are determined from the candidate mapping scenario library. The target candidate mapping scenarios are the mappings that the target personnel may want to perform, as reflected by the key point set. In non-standard scenarios, such as when the key points in a key point set are either faulty devices or multiple connected devices of those faulty devices, the target candidate mapping scenario is the scenario where the faulty device affects its connected devices. The number threshold represents the number of target candidate mapping scenarios that the target personnel can view globally at once, for example, a value of 5. When the total number of target candidate mapping scenarios corresponding to each key point set does not exceed the number threshold, the target candidate mapping scenarios corresponding to each key point set are arranged in a unified planar list layout. The unified planar list layout means that the target candidate mapping scenarios corresponding to each key point set are arranged on a single planar list, making it easy for the target personnel to view them globally at once. Otherwise, the target candidate mapping scenarios corresponding to each key point set are arranged in multiple planar lists according to their priority from highest to lowest to obtain a quick selection table for mapping scenarios. The multiple planar list layout means that scenarios with higher priority are placed on the planar lists that are displayed first, and the target personnel need to flip through the pages to view the planar lists.

[0120] This invention first determines key locations, then draws the intention-time curve for each key location, extracts multiple target peak sets from each intention-time curve, then determines multiple key location sets, and identifies the target candidate mapping scenarios corresponding to each key location set from the candidate mapping scenario library. Based on the number of these scenarios, a unified planar list layout is reasonably constructed, or a multi-planar list layout is constructed according to the priority of the scenarios from largest to smallest, ultimately forming a mapping scenario quick selection table. This greatly improves the accuracy of determining the mapping scenario quick selection table, enabling accurate and comprehensive inference of which non-standard event the target personnel currently want to map, and significantly improving the efficiency of the mapping scenario quick selection table for the target personnel to quickly select mapping scenarios.

[0121] The steps for plotting the intention-time curve for any key point are as follows:

[0122] The intention degree between the transient trajectory of the target person at different times within the most recent preset time and any of the key points is calculated, and the corresponding time is associated to form a one-to-one correlation degree and time.

[0123] The one-to-one correlation degree and time are mapped into the template curve to obtain multiple curve coordinates;

[0124] Connect the coordinates of each curve in sequence to obtain the intention-time curve;

[0125] The most recent preset time can be within the most recent 10 minutes. The transient trajectory is the trajectory of the target person moving within the most recent preset time until a certain moment. The horizontal axis of the template curve is time, and the vertical axis is the degree of correlation. When the corresponding degree of correlation and time are mapped into the template curve, multiple curve coordinates will be obtained. Connecting the curve coordinates in sequence will yield the intention-time curve.

[0126] The steps for calculating the intentionality between the transient trajectory and key points are as follows:

[0127] Obtain the first and second standard ranges of the key points respectively;

[0128] Extract the first partial trajectory from the transient trajectory after entering the first standard range and before entering the second standard range;

[0129] Obtain the standard trajectory corresponding to the first local trajectory;

[0130] Extract the second local trajectory from the transient trajectory after entering the second standard range;

[0131] The similarity between the second local trajectory and the standard trajectory is used as the intentionality.

[0132] The steps for determining the priority of any candidate target mapping scenario are as follows:

[0133] Based on the average value of the peak values ​​contained in the target peak set corresponding to the key point set of the target candidate mapping scenario and the minimum time difference between the corresponding time and the current time, the scenario priority is calculated; where the average value is positively correlated with the scenario priority, and the minimum time difference is negatively correlated with the scenario priority.

[0134] The second standard range of the key point is a circular area with a radius of 5 meters around the key point; the first standard range is a circular area with a radius of 8 meters around the key point minus the annular area of ​​the second standard range; the first local trajectory reflects the initial state of the target personnel approaching the key point, and its corresponding standard trajectory is the movement trajectory that should be generated if the target personnel want the key point as the basis for mapping non-standard situations. It can be set in advance by technicians. Therefore, the similarity between the second local trajectory and the standard trajectory can be used as the degree of intent.

[0135] When calculating scenario priority, the average value and the minimum time difference can be assigned separate weights. A weighted calculation is performed where the average value and scenario priority are positively correlated, and the minimum time difference and scenario priority are negatively correlated. The weighted sum is then used as the scenario priority. The average value represents the average level of intent, therefore it is positively correlated with scenario priority. The minimum time difference represents the freshness of the peak values ​​contained in the target peak set; the smaller the minimum time difference, the more recent the generation time, therefore the minimum time difference is negatively correlated with scenario priority.

[0136] In one embodiment, planning the fuzzy data acquisition route based on the preset mapping indication knowledge corresponding to the mapping scenario selected by the target personnel from the mapping scenario quick selection table includes:

[0137] Based on mapping indication knowledge, multiple acquisition targets are identified from a 3D digital twin;

[0138] Generate multiple sets of acquisition targets; wherein each set of acquisition targets contains at least two acquisition targets and at least one non-acquisition target, and the non-acquisition targets contained therein conform to the constraints corresponding to the relative relationships between the acquisition targets contained therein;

[0139] Plan the target route for each set of data collection targets; where the target route is the shortest route that sequentially passes through each data collection target and non-data collection target in the same set of data collection targets.

[0140] The shortest route connecting the target routes of each set of acquisition targets is used to obtain the fuzzy acquisition route.

[0141] The mapping indicator indicates which target needs to be acquired; therefore, multiple acquisition targets can be determined from the 3D digital twin based on this. Based on the constraints corresponding to the relative relationships between at least two acquisition targets, non-acquisition targets are identified. These are then combined with the at least two acquisition targets to obtain an acquisition target set. The relative relationships between acquisition targets can be relative positional relationships, target type associations, etc. Non-acquisition targets refer to targets whose information may need to be acquired, as reflected in the relative relationships, such as devices of the same target type that are very close to the at least two acquisition targets. A target route is planned for each acquisition target set. During acquisition, information from both acquisition targets and non-acquisition targets in the acquisition target set can be continuously acquired, facilitating subsequent comprehensive analysis to determine whether information from non-acquisition targets needs to be acquired. The shortest route connecting the target routes of each acquisition target set is obtained as a fuzzy acquisition route.

[0142] This invention provides a digital twin-based visual intelligent inspection system for rail transit, such as... Figure 2 As shown, it includes:

[0143] Module 1 is used to construct a three-dimensional digital twin of the rail transit scene;

[0144] Visual inspection module 2 is used for adaptive visual inspection based on a 3D digital twin;

[0145] Decision module 3 is used to generate a response plan for abnormal events when visual inspection detects abnormal events.

[0146] Simulation verification module 4 is used to simulate and verify the response plan;

[0147] Execution module 5 is used to distribute the simulated and verified response solutions to the rail transit scenario for execution.

[0148] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A visual intelligent inspection method for rail transit based on digital twins, characterized in that, include: Constructing a three-dimensional digital twin of a rail transit scenario; Based on a 3D digital twin, adaptive visual inspection is performed; When visual inspection detects an abnormal event, a response plan for the abnormal event is generated. The response plan was simulated and verified. The solutions that have passed the simulation verification will be deployed to the rail transit scenario for implementation; Following the construction of the three-dimensional digital twin of the rail transit scene, the following is also included: When a target person in a rail transit scenario inputs a non-standard event mapping request, the system assists the target person in mapping the non-standard event into a 3D digital twin. The mapping of non-standard events from the target personnel to the three-dimensional digital twin includes: Send a quick selection table of mapping scenarios to the target personnel; Based on the preset mapping indication knowledge corresponding to the mapping scenario selected by the target personnel from the mapping scenario quick selection table, a fuzzy acquisition route is planned. The system assists target personnel in collecting coarse information about events by moving along a fuzzy acquisition route. Preprocess the coarse information of the event to obtain non-standard events; Non-standard events are mapped into a three-dimensional digital twin.

2. The rail transit visual intelligent inspection method based on digital twin as described in claim 1, characterized in that, The three-dimensional digital twin of the rail transit scene includes: Invoke the digital twin engine; Configure the digital twin engine with autonomous custom modeling capabilities; Based on the digital twin engine configured with autonomous evolutionary modeling capabilities, a three-dimensional digital twin is constructed according to the multimodal on-site information of the rail transit scenario.

3. The rail transit visual intelligent inspection method based on digital twin as described in claim 1, characterized in that, The adaptive visual inspection based on a 3D digital twin includes: Deploy an array of inspection cameras within a three-dimensional digital twin; Based on an inspection camera array, adaptive visual inspection is performed in a 3D digital twin.

4. The rail transit visual intelligent inspection method based on digital twin as described in claim 1, characterized in that, When an abnormal event is detected by visual inspection, a response plan for the abnormal event is generated, including: Identify the corresponding response plan decision knowledge from the response plan decision knowledge base; By comparing the decision-making knowledge of response plans, a response plan for abnormal events is generated.

5. The rail transit visual intelligent inspection method based on digital twin as described in claim 1, characterized in that, The proposed countermeasures are simulated and verified, including: The response plan will be simulated and executed in a three-dimensional digital twin; Continuously monitor the process information of the simulated execution of the response plan in the 3D digital twin; Based on the simulation verification and evaluation system, simulation verification and evaluation are carried out according to process information to obtain evaluation values; When the evaluation value exceeds the evaluation threshold, the simulation verification of the solution is considered successful.

6. The rail transit visual intelligent inspection method based on digital twin as described in claim 1, characterized in that, The steps for determining the mapping scenario quick selection table are as follows: Identify multiple key points within a preset range surrounding the current location of the target person from a 3D digital twin; Draw the intention-time curve for each key point; Extract multiple target peak sets from each intention-time curve; wherein each target peak set contains more than N peaks that originate from different intention-time curves, and the absolute value of the time difference between each pair of peaks in each target peak set does not exceed the absolute value threshold, and each peak in each target peak set exceeds the peak threshold; N is an integer threshold greater than or equal to 2. Multiple key point sets are identified; each key point set contains key points corresponding to the intention-time curves from which the peak values ​​contained in the target peak set originate. Determine the target candidate mapping scenarios corresponding to each key point set from the candidate mapping scenario library; When the total number of target candidate mapping scenarios corresponding to each key point set does not exceed the number threshold, the target candidate mapping scenarios corresponding to each key point set are arranged in a unified planar list to obtain a quick selection table of mapping scenarios; otherwise, the target candidate mapping scenarios corresponding to each key point set are arranged in a multi-planar list according to the priority of the scenarios from large to small to obtain a quick selection table of mapping scenarios. The steps for plotting the intention-time curve for any key point are as follows: The intention degree between the transient trajectory of the target person at different times within the most recent preset time and any of the key points is calculated, and the corresponding time is associated to form a one-to-one correlation degree and time. The one-to-one correlation degree and time are mapped into the template curve to obtain multiple curve coordinates; Connect the coordinates of each curve in sequence to obtain the intention-time curve; The steps for calculating the intentionality between the transient trajectory and key points are as follows: Obtain the first and second standard ranges of the key points respectively; Extract the first partial trajectory from the transient trajectory after entering the first standard range and before entering the second standard range; Obtain the standard trajectory corresponding to the first local trajectory; Extract the second local trajectory from the transient trajectory after entering the second standard range; The similarity between the second local trajectory and the standard trajectory is used as the intentionality. The steps for determining the priority of any candidate target mapping scenario are as follows: Based on the average value of the peak values ​​contained in the target peak set corresponding to the key point set of the target candidate mapping scenario and the minimum time difference between the corresponding time and the current time, the scenario priority is calculated; where the average value is positively correlated with the scenario priority, and the minimum time difference is negatively correlated with the scenario priority.

7. The rail transit visual intelligent inspection method based on digital twin as described in claim 1, characterized in that, The step of planning a fuzzy data acquisition route based on the preset mapping indication knowledge corresponding to the mapping scenario selected by the target personnel from the mapping scenario quick selection table includes: Based on mapping indication knowledge, multiple acquisition targets are identified from a 3D digital twin; Generate multiple sets of acquisition targets; wherein each set of acquisition targets contains at least two acquisition targets and at least one non-acquisition target, and the non-acquisition targets contained in the acquisition target set conform to the constraints corresponding to the relative relationships between the acquisition targets contained in the acquisition target set; Plan the target route for each set of data collection targets; where the target route is the shortest route that sequentially passes through each data collection target and non-data collection target in the same set of data collection targets. The shortest route connecting the target routes of each set of acquisition targets is used to obtain the fuzzy acquisition route.

8. A digital twin-based visual intelligent inspection system for rail transit, characterized in that, include: The building module is used to construct a 3D digital twin of a rail transit scenario; The visual inspection module is used to perform adaptive visual inspection based on a 3D digital twin. The decision module is used to generate a response plan for abnormal events when visual inspection detects them. The simulation verification module is used to simulate and verify the response plan; The execution module is used to distribute the simulated and verified response solutions to the rail transit scenario for execution. After the construction module constructs a three-dimensional digital twin of the rail transit scene, it also includes: When a target person in a rail transit scenario inputs a non-standard event mapping request, the system assists the target person in mapping the non-standard event into a 3D digital twin. The mapping of non-standard events from the target personnel to the three-dimensional digital twin includes: Send a quick selection table of mapping scenarios to the target personnel; Based on the preset mapping indication knowledge corresponding to the mapping scenario selected by the target personnel from the mapping scenario quick selection table, a fuzzy acquisition route is planned. The system assists target personnel in collecting coarse information about events by moving along a fuzzy acquisition route. Preprocess the coarse information of the event to obtain non-standard events; Non-standard events are mapped into a three-dimensional digital twin.

Images