Elevator safety management method and system based on visual detection

Abstract

The present invention provides an elevator safety management method and system based on visual detection, wherein the method includes: based on the first visual detection information of the elevator in the first time period, triggering and determining the safety management opportunity and safety management situation; when entering the safety management opportunity, based on the second visual detection information of the elevator in the second time period, generating a digital model of the safety management situation; wherein the second time period includes the first time period; based on the digital model, the elevator is correspondingly managed safely. The elevator safety management method and system based on visual detection of the present invention does not require management personnel to manually patrol and inspect the elevator regularly, which reduces labor costs, improves management efficiency, realizes real-time monitoring and intelligent digital analysis of elevators, and avoids the omission of safety hazards due to human factors.

Description

Technical Field

[0001] The present invention relates to the technical field of visual detection, and in particular to an elevator safety management method and system based on visual detection. Background Art

[0002] In modern high-rise buildings, elevators are important and common means of vertical transportation, and their safety management is particularly critical.

[0003] However, existing elevator safety management technologies often rely on regular manual inspections by managers, which has high labor costs, insufficient management efficiency, lack of real-time monitoring and intelligent digital analysis capabilities, and are prone to omissions of safety hazards due to human factors.

[0004] Therefore, a solution is urgently needed. Summary of the invention

[0005] One of the purposes of the present invention is to provide an elevator safety management method based on visual detection. Based on visual detection technology, visual detection information in different time periods is obtained to trigger the determination of safety management opportunities and safety management situations, and a digital model of the safety management situation is generated when the safety management opportunity is entered. Finally, the elevator is safely managed accordingly based on the digital model. There is no need for management personnel to manually patrol and inspect the elevators on a regular basis, which reduces labor costs, improves management efficiency, realizes real-time monitoring and intelligent digital analysis of elevators, and avoids the omission of safety hazards due to human factors.

[0006] An elevator safety management method based on visual detection provided by an embodiment of the present invention includes:

[0007] Based on the first visual detection information of the elevator in the first time period, triggering and determining the safety management opportunity and safety management situation;

[0008] When entering the safety management opportunity, generating a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period; wherein the second time period includes the first time period;

[0009] Based on the digital model, the elevators can be managed accordingly for safety;

[0010] The triggering and determining of the safety management opportunity and the safety management situation based on the first visual detection information of the elevator in the first time period includes:

[0011] Based on the trigger determination history, determining a plurality of template conditions and respective corresponding characterization values;

[0012] Based on each template condition, multiple target trigger extraction templates are screened out from the trigger extraction template library; wherein each target trigger extraction template meets at least i template conditions, and the sum of the characterization values ​​corresponding to the template conditions that meet the conditions exceeds the threshold of the sum of the characterization values; wherein the rounded-up value of the quotient of N divided by i is the rounding threshold; and N is the total number of template conditions;

[0013] Based on each target trigger extraction template, try to extract the trigger determination basis from the first visual detection information respectively;

[0014] When the attempt is successful, a security management opportunity and a security management situation are generated based on the extracted trigger determination basis.

[0015] Optionally, the determining of a plurality of template conditions and their corresponding characterization values ​​based on the trigger determination history includes:

[0016] Perform a timeline representation of the trigger determination history to obtain a historical timeline;

[0017] Determine multiple time node clusters from the historical timeline; wherein the maximum node time difference between two time nodes in each time node cluster does not exceed a threshold value of the maximum node time difference;

[0018] Perform commonality analysis on each time node cluster to obtain multiple commonality results and corresponding commonality values;

[0019] Based on the commonality results whose commonality values ​​exceed the threshold value of the commonality value, a template condition is generated, and the commonality value of the commonality result based on which the template condition is generated is used as the corresponding characterization value.

[0020] Optionally, the generating a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period includes:

[0021] Performing a graph representation on the second visual detection information to obtain an information graph;

[0022] Based on the mapping acquisition template of each of the multiple sub-situations in the security management situation, the mapping value of each sub-situation is obtained according to the information graph;

[0023] Generate digital mechanisms;

[0024] Based on the digital mechanism, the original model is digitally set;

[0025] The original model after digital setting is used as the digital model;

[0026] Wherein, the digital mechanism includes:

[0027] Traverse each sub-case in order from large to small according to the mapping value;

[0028] During each traversal, based on the digital processing template corresponding to the traversed sub-scenario and according to the second visual detection information, the original model is digitally processed, and after the digital processing, the traversal stay request is continuously detected for a preset first time period;

[0029] When detected, stay at the preset second time length, and then continue to traverse the next sub-situation; otherwise, stay at the preset third time length corresponding to the difference between the mapping value of the next traversed sub-situation and the mapping value of the traversed sub-situation, and then continue to traverse the next sub-situation; wherein the third time length and the difference are negatively correlated.

[0030] Optionally, the elevator is managed accordingly based on the digital model, including:

[0031] Input the digital model into a preset neural network to make a first management decision;

[0032] When the situation value of the safety management situation does not exceed the threshold of the situation value, the first management strategy of the neural network making the first management decision is executed on the elevator; otherwise, the digital model is output to the expert personnel, and the target local sequence is determined from the historical progress sequence of the neural network making the first management decision; wherein the target local sequence is constrained by the first condition;

[0033] Based on the target local sequence, determining a target history from the output history of the digitized model; wherein the target history is subject to a second condition;

[0034] Control the digital model to reproduce the output target history to the expert personnel and prompt the expert personnel to make a second management decision;

[0035] Based on the second management strategy of the expert personnel making the second management decision, the sub-strategies corresponding to each historical progress in the target local sequence in the first management strategy are replaced to obtain the final management strategy;

[0036] Implement final management strategy for elevators;

[0037] The first condition includes:

[0038] The first maximum matching degree between the first multimodal detail feature of the first and last historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold value of the first maximum matching degree;

[0039] or,

[0040] The second maximum matching degree between the second multimodal detail feature of the first historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold of the second maximum matching degree, and there is a progress association between any two adjacent historical progresses in the target local sequence;

[0041] The second condition includes:

[0042] The target history is related to or represents at least one historical progress in the target partial sequence.

[0043] An elevator safety management system based on visual detection provided by an embodiment of the present invention includes:

[0044] A triggering and determining module, used for triggering and determining a safety management opportunity and a safety management situation based on first visual detection information of the elevator in a first time period;

[0045] A model generation module, used for generating a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period when entering the safety management opportunity; wherein the second time period includes the first time period;

[0046] Safety management module, used to manage elevator safety accordingly based on digital models;

[0047] The trigger determination module triggers and determines the safety management opportunity and safety management situation based on the first visual detection information of the elevator in the first time period, including:

[0048] Based on the trigger determination history, determining a plurality of template conditions and respective corresponding characterization values;

[0049] Based on each template condition, multiple target trigger extraction templates are screened out from the trigger extraction template library; wherein each target trigger extraction template meets at least i template conditions, and the sum of the characterization values ​​corresponding to the template conditions that meet the conditions exceeds the threshold of the sum of the characterization values; wherein the rounded-up value of the quotient of N divided by i is the rounding threshold; and N is the total number of template conditions;

[0050] Based on each target trigger extraction template, try to extract the trigger determination basis from the first visual detection information respectively;

[0051] When the attempt is successful, a security management opportunity and a security management situation are generated based on the extracted trigger determination basis.

[0052] Optionally, the trigger determination module determines a plurality of template conditions and their corresponding characterization values ​​based on the trigger determination history, including:

[0053] Perform a timeline representation of the trigger determination history to obtain a historical timeline;

[0054] Determine multiple time node clusters from the historical timeline; wherein the maximum node time difference between two time nodes in each time node cluster does not exceed a threshold value of the maximum node time difference;

[0055] Perform commonality analysis on each time node cluster to obtain multiple commonality results and corresponding commonality values;

[0056] Based on the commonality results whose commonality values ​​exceed the threshold value of the commonality value, a template condition is generated, and the commonality value of the commonality result based on which the template condition is generated is used as the corresponding characterization value.

[0057] Optionally, the model generation module generates a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period, including:

[0058] Performing a graph representation on the second visual detection information to obtain an information graph;

[0059] Based on the mapping acquisition template of each of the multiple sub-situations in the security management situation, the mapping value of each sub-situation is obtained according to the information graph;

[0060] Generate digital mechanisms;

[0061] Based on the digital mechanism, the original model is digitally set;

[0062] The original model after digital setting is used as the digital model;

[0063] Wherein, the digital mechanism includes:

[0064] Traverse each sub-case in order from large to small according to the mapping value;

[0065] During each traversal, based on the digital processing template corresponding to the traversed sub-scenario and according to the second visual detection information, the original model is digitally processed, and after the digital processing, the traversal stay request is continuously detected for a preset first time period;

[0066] When detected, stay at the preset second time length, and then continue to traverse the next sub-situation; otherwise, stay at the preset third time length corresponding to the difference between the mapping value of the next traversed sub-situation and the mapping value of the traversed sub-situation, and then continue to traverse the next sub-situation; wherein the third time length and the difference are negatively correlated.

[0067] Optionally, the safety management module performs corresponding safety management of the elevator based on the digital model, including:

[0068] Input the digital model into a preset neural network to make a first management decision;

[0069] When the situation value of the safety management situation does not exceed the threshold of the situation value, the first management strategy of the neural network making the first management decision is executed on the elevator; otherwise, the digital model is output to the expert personnel, and the target local sequence is determined from the historical progress sequence of the neural network making the first management decision; wherein the target local sequence is constrained by the first condition;

[0070] Based on the target local sequence, determining a target history from the output history of the digitized model; wherein the target history is subject to a second condition;

[0071] Control the digital model to reproduce the output target history to the expert personnel and prompt the expert personnel to make a second management decision;

[0072] Based on the second management strategy of the expert personnel making the second management decision, the sub-strategies corresponding to each historical progress in the target local sequence in the first management strategy are replaced to obtain the final management strategy;

[0073] Implement final management strategy for elevators;

[0074] The first condition includes:

[0075] The first maximum matching degree between the first multimodal detail feature of the first and last historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold value of the first maximum matching degree;

[0076] or,

[0077] The second maximum matching degree between the second multimodal detail feature of the first historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold of the second maximum matching degree, and there is a progress association between any two adjacent historical progresses in the target local sequence;

[0078] The second condition includes:

[0079] The target history is related to or represents at least one historical progress in the target partial sequence.

[0080] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0081] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0083] Figure 1 Schematic diagram of an elevator safety management method based on visual detection in an embodiment of the present invention;

[0084] Figure 2Schematic diagram of an elevator safety management system based on visual detection in an embodiment of the present invention. DETAILED DESCRIPTION

[0085] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0086] The embodiment of the present invention provides an elevator safety management method based on visual detection, such as Figure 1 As shown, including:

[0087] S1. triggering and determining a safety management opportunity and a safety management situation based on first visual detection information of the elevator in a first time period;

[0088] In step S1, the first time period may be within the most recent 100 seconds; a high-resolution industrial camera is installed in the elevator machine room, shaft and car in advance to continuously photograph the elevator's traction machine, brake, traction wheel, wire rope and other key components and the interior of the car in real time; the image captured by the high-resolution industrial camera is subjected to pre-processing steps such as denoising and enhancement to form visual detection information; the first visual detection information is generated within the first time period; the safety management opportunity is the opportunity when the elevator needs to be safely managed, and the safety management situation is the situation when the elevator needs to be safely managed; the first visual detection information will reflect the latest operating status of the elevator, so the safety management opportunity and safety management situation can be determined based on its triggering, for example: the first visual detection information reflects that the elevator's traction machine is suspected to be abnormal, then the trigger determines that the safety management opportunity is the next time the visual detection finds that the traction machine is suspected to be abnormal, and the safety management situation is the suspected abnormality of the traction machine;

[0089] S2. When entering the safety management opportunity, generating a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period; wherein the second time period includes the first time period;

[0090] In step S2, the second time period may be within 600 seconds before and after the moment of entering the safety management opportunity; the second visual detection information is generated in the second time period; in order to ensure the comprehensiveness of the information displayed by the digital model, the second time period is set to include the first time period; the digital model is a three-dimensional model corresponding to the safety management situation and based on the second visual detection information, and the digital model displays the second visual detection information so that the system can quickly and in detail understand the safety management situation;

[0091] S3. Based on the digital model, carry out corresponding safety management of elevators.

[0092] In step S3, the digital model reflects the safety management situation in detail by displaying the second visual detection information, so the system can perform corresponding safety management of the elevator based on it.

[0093] The embodiment of the present invention is based on visual detection technology to obtain visual detection information in different time periods to trigger the determination of safety management opportunities and safety management situations, and generate a digital model of the safety management situation when entering the safety management opportunity. Finally, the elevator is safely managed accordingly based on the digital model. There is no need for management personnel to manually patrol and inspect the elevators on a regular basis, which reduces labor costs, improves management efficiency, realizes real-time monitoring and intelligent digital analysis of elevators, and avoids the omission of safety hazards due to human factors.

[0094] In S1, based on the first visual detection information of the elevator in the first time period, triggering and determining the safety management opportunity and the safety management situation includes:

[0095] S11, determining a plurality of template conditions and their corresponding characterization values ​​based on the trigger determination history;

[0096] In step S11, the trigger determination history includes: historical records of triggering and determining different safety management opportunities and different safety management situations based on different first visual detection information in the past; the trigger determination history can reflect how the elevator may need to perform safety management in the future; the template condition includes: screening out conditions that can extract a target trigger extraction template corresponding to the trigger determination basis, the trigger determination basis corresponding to how the elevator may need to perform safety management in the future reflected by the trigger determination history, and the corresponding characterization value includes: representing the degree to which the template condition can ensure that the target trigger extraction template extracts the trigger determination basis;

[0097] S12, based on each template condition, select multiple target trigger extraction templates from the trigger extraction template library; wherein each target trigger extraction template meets at least i template conditions, and the sum of the characterization values ​​corresponding to the template conditions that meet the conditions exceeds the threshold of the sum of the characterization values; wherein the rounded-up value of the quotient of N divided by i is the rounding threshold; and N is the total number of template conditions;

[0098] In step S12, there are different trigger extraction templates in the trigger extraction template library; a target trigger extraction template is screened out based on each template condition; the threshold of the sum of the characterization values ​​may be, for example, 80; the rounding threshold may be, for example, 2; it is ensured that each target trigger extraction template meets at least i template conditions, and the sum of the characterization values ​​corresponding to the template conditions that meet the conditions exceeds the threshold of the sum of the characterization values, so that the screened target trigger extraction template is sufficient to extract the trigger determination basis corresponding to how the elevator may need to be safely managed in the future as reflected by the trigger determination history;

[0099] S13, based on each target trigger extraction template, try to extract the trigger determination basis from the first visual detection information respectively;

[0100] In step S13, when the target trigger extraction template is used, for example: an image reflecting suspected abnormal performance of the traction machine is extracted from the first visual detection information;

[0101] S14. When the attempt is successful, a security management opportunity and a security management situation are generated based on the extracted trigger determination basis.

[0102] In step S14, accordingly, when generating the safety management opportunity and the safety management situation, for example: analyzing the features of the extracted image reflecting the suspected abnormal performance of the hoisting machine, the safety management opportunity is generated as the moment when the feature appears in the future visual detection information, and the safety management situation is generated as the suspected abnormality of the hoisting machine.

[0103] The embodiment of the present invention triggers the determination of the safety management opportunity and the safety management situation based on the first visual detection information of the elevator in the first time period. Based on the trigger determination history, multiple template conditions and their corresponding characterization values ​​are determined, and multiple target trigger extraction templates are accurately screened out from the trigger extraction template library. Based on each target trigger extraction template, attempts are made to extract the trigger determination basis from the first visual detection information. When the attempt is successful, the trigger is achieved, and then the safety management opportunity and the safety management situation are generated based on the extracted trigger determination basis, which greatly improves the accuracy, comprehensiveness and efficiency of triggering the determination of the safety management opportunity and the safety management situation.

[0104] In one embodiment, the step S11, determining a plurality of template conditions and their corresponding characterization values ​​based on the trigger determination history, includes:

[0105] S111, performing a timeline representation of the trigger determination history to obtain a historical timeline;

[0106] In step S111, when performing timeline representation, the trigger determines that the history is split into multiple historical content items, and the historical content items are set at corresponding moments on the timeline according to their generation time. The historical content items form time nodes together at the moments at which they are set; finally, the timeline after all time nodes are formed is used as the historical timeline;

[0107] S112, determining a plurality of time node clusters from the historical timeline; wherein the maximum node time difference between any two time nodes in each time node cluster does not exceed a threshold value of the maximum node time difference;

[0108] Among them, in step S112, a time node cluster is formed by a plurality of consecutive time nodes on the historical timeline; the node time difference between any two time nodes is the difference between the moments on the corresponding timeline; the threshold of the maximum node time difference can be 300 seconds; it is ensured that the maximum node time difference between any two time nodes in each time node cluster does not exceed the threshold of the maximum node time difference, so that each time node in a time node cluster can jointly reflect the stage situation of continuous trigger determination within a certain period of history;

[0109] S113, performing commonality analysis on each time node cluster to obtain multiple commonality results and corresponding commonality values;

[0110] Among them, in step S113, during the commonality analysis, the same points between the time node clusters are analyzed, the same points are taken as the commonality results, and the total number of time node clusters with the same points is quantized into a commonality value (a quantization table can be pre-set, and there are commonality values ​​corresponding to different total numbers in the quantization table); since a time node cluster reflects the situation of the corresponding stage, the commonality result reflects the commonality relationship between the multi-stage situations;

[0111] S114 . Generate a template condition based on a commonality result whose commonality value exceeds a threshold value of the commonality value, and use the commonality value of the commonality result based on which the template condition is generated as a corresponding characterization value.

[0112] Among them, in step S114, the threshold of the commonality value can be 8, for example; the larger the commonality value, the greater the degree of existence of the commonality result in each time node cluster; based on the commonality result whose commonality value exceeds the threshold of the commonality value, a template condition is generated. Specifically, when generating, for example: the commonality result is a safety management situation that triggers a suspected abnormality of the traction machine, then the generated template condition is the target trigger extraction template that needs to have the ability to extract the suspected abnormal performance of the traction machine, and the commonality value of the commonality result based on which the template condition is generated is used as the corresponding characterization value.

[0113] The embodiment of the present invention introduces a historical timeline when determining multiple template conditions and their corresponding characterization values ​​based on a trigger determination history, determines multiple time node clusters from the historical timeline, performs commonality analysis on each time node cluster, obtains multiple commonality results and corresponding commonality values, generates template conditions based on the commonality results whose commonality values ​​exceed the threshold of the commonality values, and uses the commonality values ​​of the commonality results based on which the template conditions are generated as the corresponding characterization values, thereby greatly improving the accuracy and efficiency of determining multiple template conditions and their corresponding characterization values.

[0114] In one embodiment, in S2, generating a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period includes:

[0115] S21, performing a graph representation on the second visual detection information to obtain an information graph;

[0116] In step S21, when performing the atlas representation, the second visual detection information is distributed and set on the template atlas, and the template atlas after the distribution setting is completed is used as the information atlas; the template atlas has atlas nodes for setting different types of information (such as traction machine related information, brake related information, traction wheel related information, wire rope related information, etc.), and the second visual detection information is set on the corresponding atlas node according to its type;

[0117] S22, based on the mapping acquisition template of each of the multiple sub-situations in the security management situation, and according to the information map, obtaining the mapping value of each sub-situation;

[0118] In step S22, the safety management situation is divided into several sub-situations, for example, the situation of suspected abnormality of the traction machine is divided into sub-situations such as traction machine traction wheel management and traction machine control system management; different sub-situations are pre-set with mapping acquisition templates, and the mapping value of the sub-situation can be obtained according to the information map using the mapping acquisition template. The mapping value represents the degree to which the relevant details of the sub-situation need to be displayed first; when using the mapping acquisition template, the map node related to the sub-situation is first found from the information map, and the total number of related map nodes is quantified (when quantizing, similarly, the corresponding value of the total number is determined from the pre-set quantization table) to obtain the first quantization value; The total number of node connections between different related graph nodes is quantized (the quantization process is the same as the quantization process of the first quantization value), and the second quantization value is obtained. The sum of the first quantization value and the second quantization value is used as the mapping value of the sub-scenario; when there is a node connection between two graph nodes, it means that the setting information on the two graph nodes is related, for example: the respective information may reflect the cause of the same fault, the respective information belongs to the interconnected components, etc.; secondly, the more the total number of related graph nodes is, the greater the relevant information of the sub-scenario is, and the more suitable it is for current safety management; therefore, the mapping value reasonably represents the degree to which the relevant details of the sub-scenario need to be displayed first;

[0119] S23, generate digital mechanism;

[0120] S24, digitally setting the original model based on the digital mechanism;

[0121] In step S24, the original model is a visualization model to be digitized and displayed without content;

[0122] S25, using the original model after digital setting as the digital model;

[0123] Wherein, the digital mechanism includes:

[0124] Traverse each sub-case in order from large to small according to the mapping value;

[0125] During each traversal, based on the digital processing template corresponding to the traversed sub-scenario and according to the second visual detection information, the original model is digitally processed, and after the digital processing, the traversal stay request is continuously detected for a preset first time period;

[0126] When detected, stay at the preset second time length, and then continue to traverse the next sub-situation; otherwise, stay at the preset third time length corresponding to the difference between the mapping value of the next traversed sub-situation and the mapping value of the traversed sub-situation, and then continue to traverse the next sub-situation; wherein the third time length and the difference are negatively correlated.

[0127] In the digitization mechanism, each sub-situation is traversed in order from large to small according to the mapping value, so that the sub-situation that needs to display relevant details first can be displayed first; the sub-situation is pre-set with a digital processing template, and the digital processing template can digitize the original model according to the second visual detection information. The original model after digitization can display the relevant details of the sub-situation. For example: if the sub-situation is the management of the winch control system, when the digital processing is performed based on the corresponding digital processing template, the image of the components related to the winch control system is searched in the second visual detection information, and the original model is controlled to display the component image in sequence according to the image generation time sequence, and the abnormal or suspected abnormal position of the component image is marked; the preset first duration can be 30 seconds ; The traversal stay request can be input by the viewing object of the digital model; the preset second duration is 200 seconds; when the difference is a positive number, it means that the mapping value of the next traversal sub-case is relatively large, and the more necessary it is to display the relevant details as soon as possible, the smaller the third duration; when the difference is a negative number, it means that the mapping value of the next traversal sub-case is relatively small, and the more necessary it is to display the relevant details of the currently traversed sub-case in more detail, the larger the third duration; therefore, there is a negative correlation between the three durations and the difference; the event viewing object that continuously detects can analyze the display content. When detected, continue to set the time for its analysis. When not detected, set a reasonable time to stay according to the difference in the mapping value to ensure that the viewing object analyzes different sub-cases at a reasonable rhythm.

[0128] When the embodiment of the present invention generates a digital model of a security management situation, an information graph is introduced, mapping values ​​are obtained from the information graph based on a mapping acquisition template, a digital mechanism is reasonably generated based on the mapping values, and the original model is digitally set based on the digital mechanism. The original model after the digital setting is used as the digital model, which greatly improves the rationality, accuracy and comprehensiveness of the digital model generation and improves the applicability of the system.

[0129] In one embodiment, the step S3, based on the digital model, performs corresponding safety management on the elevator, including:

[0130] S31, inputting the digital model into a preset neural network to make a first management decision;

[0131] In step S31, the neural network is an artificial intelligence model that is trained in advance using a large amount of expert experience in elevator safety management based on visual inspection information of elevators as training samples until convergence, and it can make elevator safety management decisions according to the information displayed by the digital model according to the digital mechanism;

[0132] S32, when the situation value of the safety management situation does not exceed the threshold value of the situation value, executing the first management strategy of the neural network making the first management decision on the elevator; otherwise, outputting the digital model to the expert personnel, and determining the target local sequence from the historical progress sequence of the neural network making the first management decision; wherein the target local sequence is constrained by the first condition;

[0133] In step S32, the situation value represents the severity of the safety management situation; the threshold of the situation value can be 10; when the situation value does not exceed the threshold, it means that the severity is small, and the first management strategy of the first management decision of the neural network is directly executed on the elevator; otherwise, it means that the severity is large, and experts need to intervene in a targeted manner; when the neural network makes the first management decision, it will continuously record the progress of the decision, that is, the historical progress, and the historical progress is sorted in order of progress to form a historical progress sequence; under the constraint of the first condition, the target local sequence determined from the historical progress sequence can be used as a basis for determining the target history, and controlling the digital model to re-output the target history can assist experts in making targeted interventions;

[0134] S33, determining a target history from the output history of the digital model based on the target local sequence; wherein the target history is subject to a second condition;

[0135] In step S33, when the digital model is output, the output history is recorded, and the output history includes different contents outputted in history and corresponding output time, etc.;

[0136] S34, controlling the digital model to reproduce the output target history to the expert personnel, and prompting the expert personnel to make a second management decision;

[0137] In step S34, re-outputting refers to controlling the digital model to replay the target history; when prompting the expert to make a second management decision, corresponding information prompts may be given to the expert;

[0138] S35, based on the second management strategy of the expert personnel making the second management decision, the sub-strategies corresponding to the historical progress in the target local sequence in the first management strategy are replaced to obtain the final management strategy;

[0139] In step S35, after the expert makes the second management decision, a second management strategy is obtained, which replaces the sub-strategies corresponding to each historical progress in the target local sequence in the first management strategy to obtain the final management strategy; the correspondence between the sub-strategy and the historical progress means that the decision to generate the sub-strategy is made when the historical progress is carried out;

[0140] S36, executing the final management strategy for the elevator;

[0141] In step S36, finally, the final management strategy is executed on the elevator;

[0142] The first condition includes:

[0143] The first maximum matching degree between the first multimodal detail feature of the first and last historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold value of the first maximum matching degree;

[0144] or,

[0145] The second maximum matching degree between the second multimodal detail feature of the first historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold of the second maximum matching degree, and there is a progress association between any two adjacent historical progresses in the target local sequence;

[0146] In the first condition, the first multimodal detail feature and the second multimodal detail feature include: the amount of relevant information in the training sample based on which the neural network executes the corresponding historical progress, the completion time of the historical progress, etc.; the standard feature is a multimodal detail feature that represents the need for targeted intervention by experts, for example: the amount of relevant information is 10, and the completion time is 0.2 seconds, that is, the basis is small, the completion time is short, and the historical progress execution may be unreliable; the threshold of the first maximum matching degree can be 60%; when the first maximum matching degree between the first multimodal detail feature of the first and last historical progress in the target local sequence and each of the multiple standard features exceeds the threshold of the first maximum matching degree, Adjacent progresses affect each other, indicating that each continuous historical progress in the target local sequence needs to be redone, and targeted intervention by experts is required; the threshold of the second maximum matching degree can be 75%; the existence of progress association between adjacent historical progresses means that the execution of the two historical progresses is to obtain the same decision result, etc.; when the second maximum matching degree between the second multimodal detail feature of the first historical progress in the target local sequence and multiple standard features exceeds the threshold of the second maximum matching degree, and there is a progress association between two adjacent historical progresses in the target local sequence, it also indicates that each continuous historical progress in the target local sequence needs to be redone, and targeted intervention by experts is required;

[0147] The second condition includes:

[0148] The target history is related to or represents at least one historical progress in the target partial sequence.

[0149] In the second condition, the target history represents at least one historical progress in the target local sequence, which means that when the viewing object of the digital model views the target history, it can know the progress of making elevator safety management decisions. For example, if the target history is to view the fault history of the traction machine, the historical progress that represents it is the historical fault comparison. It is ensured that the target history is related to at least one historical progress in the target local sequence or represents at least one historical progress in the target local sequence, so that the control digital model to re-output the target history can assist experts in targeted intervention.

[0150] The embodiment of the present invention performs corresponding safety management of the elevator based on the digital model. It determines whether the neural network is fully responsible according to the situation value. If not, auxiliary experts perform targeted intervention quickly and accurately based on the digital model, which greatly improves the efficiency of elevator safety management and is more intelligent.

[0151] For the elevator brake, taking it as an example, the specific visual detection implementation method is as follows:

[0152] The elevator brake is one of the core components to ensure the safe operation of the elevator. Its main function is to ensure the stability of the elevator during operation and to stop the elevator quickly in an emergency. Therefore, the safety performance of the elevator brake is directly related to the life safety of passengers. Traditional elevator brake inspection methods rely on manual inspection and regular maintenance, but this method has problems such as long inspection cycle, low efficiency, and high risk of missed inspection. In order to improve the safety management efficiency, accuracy, and real-time performance of elevator brakes, an automated inspection solution based on visual inspection has emerged.

[0153] This solution aims to use visual inspection technology to automatically monitor the status and safety performance of elevator brakes in real time, detect potential faults or damage in a timely manner, and ensure the safe operation of the elevator; by integrating high-precision image acquisition and processing technology, combined with deep learning and image recognition algorithms, it can achieve comprehensive and accurate inspection of elevator brakes.

[0154] 1. System architecture

[0155] The system architecture includes hardware and software. The hardware part consists of cameras, light sources, computing platforms, and supplementary sensors; the software part is mainly responsible for image acquisition, image processing, image analysis and recognition, anomaly detection, alarm processing, data recording, and report generation.

[0156] 1.1 Hardware

[0157] Camera: A high-resolution industrial-grade camera with autofocus and high-speed shooting functions is used to clearly capture the details and dynamic status of the brake.

[0158] Light source: Uses adjustable LED light source to ensure uniform lighting and eliminates interference of shadows or reflections on image quality.

[0159] Computing platform: Image processing and analysis are performed through local computers or cloud servers, supporting edge computing or cloud computing.

[0160] Supplementary sensors: including temperature, vibration and other sensors, which are used to combine with visual inspection data to provide more comprehensive status monitoring.

[0161] 1.2 Software

[0162] Image acquisition and processing: Acquire image data of elevator brakes in real time, and improve image quality through preprocessing (such as denoising and image enhancement) to provide high-quality input for subsequent analysis.

[0163] Image analysis and recognition: Deep learning and convolutional neural network (CNN) technologies are used to automatically identify key components of elevator brakes (such as brake wheels, brake bands, brake shoes, etc.) to determine whether there are abnormalities such as wear, cracks, looseness, corrosion, etc.

[0164] Abnormal detection and alarm: Based on the set safety standards (such as brake thickness, loose clearance, etc.), the system monitors various indicators of the elevator brake in real time. Once an abnormal situation is found, the system will automatically alarm and mark the problem area.

[0165] Data recording and report generation: The system will automatically record the test results, generate detailed test reports, and support report export for later analysis and maintenance reference.

[0166] 2. Image Processing and Analysis

[0167] 2.1 Image Preprocessing

[0168] In order to improve the image quality and the accuracy of analysis, Gaussian filtering, edge enhancement and other technologies are used for image preprocessing. By removing the noise in the image, the recognizability of the brake components is enhanced, ensuring the accuracy of subsequent analysis. At the same time, the image alignment algorithm is used to standardize the image input under different shooting angles or lighting conditions to eliminate the impact of external environmental factors on image quality.

[0169] 2.2 Image Recognition and Deep Learning

[0170] In the image recognition stage, we first collect a large amount of normal and abnormal brake image data to train the deep learning model. Through data enhancement and transfer learning technology, we improve the generalization ability of the model so that it can adapt to the detection needs of different elevator brakes. We use convolutional neural networks (CNN) to extract key features in the image, accurately identify various parts of the brake (such as brake discs, brake bands, brake shoes, etc.), and determine whether there are abnormal problems such as wear, cracks, corrosion, etc. according to the set safety standards.

[0171] The trained model can process the image data uploaded by the camera in real time, ensuring that each frame of the image can be quickly fed back to the monitoring system. When an abnormality is found, the system will trigger an alarm in time and notify the maintenance personnel through SMS, email, etc.

[0172] 3. Implementation and Installation

[0173] In actual deployment, it is necessary to first select suitable cameras, light sources and sensor equipment according to the structural characteristics and use environment of the elevator brake to ensure that the detection requirements are met. Then, through on-site investigation of the installation location and spatial conditions of the elevator brake, the installation location of the camera and light source is reasonably designed to ensure that all key components can be fully and clearly captured.

[0174] The installation process includes the installation of cameras, light sources and sensors, power access and equipment debugging. During the debugging phase, it is necessary to ensure that the hardware equipment is properly connected to the computing platform, the image acquisition and processing processes can run efficiently, and the detection system can achieve real-time monitoring and abnormality identification.

[0175] 4. System advantages and effects

[0176] By combining advanced visual inspection and artificial intelligence technology, this solution provides an efficient and intelligent safety management method for elevator brakes. The advantages of this solution include:

[0177] Automation and real-time monitoring: realize full-process automated monitoring of elevator brakes to avoid omissions and inefficiencies caused by manual inspection.

[0178] High-precision detection: Using high-resolution image acquisition and deep learning technology, it can accurately detect tiny abnormalities of the brake and ensure the safety of the elevator.

[0179] Abnormal warning and report generation: The system can immediately alarm when an abnormality occurs and generate a detailed detection report for subsequent analysis and maintenance reference.

[0180] Data support and optimization: Automated detection data can provide real-time feedback for elevator maintenance management, helping to optimize maintenance plans and improve equipment reliability. The embodiment of the present invention provides an elevator safety management system based on visual detection, such as Figure 2 As shown, including:

[0181] A triggering and determining module 1, configured to trigger and determine a safety management opportunity and a safety management situation based on first visual detection information of the elevator in a first time period;

[0182] Model generation module 2, used for generating a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period when entering the safety management opportunity; wherein the second time period includes the first time period;

[0183] The safety management module 3 is used to perform corresponding safety management of the elevator based on the digital model.

[0184] The trigger determination module triggers and determines the safety management opportunity and safety management situation based on the first visual detection information of the elevator in the first time period, including:

[0185] Based on the trigger determination history, determining a plurality of template conditions and respective corresponding characterization values;

[0186] Based on each template condition, multiple target trigger extraction templates are screened out from the trigger extraction template library; wherein each target trigger extraction template meets at least i template conditions, and the sum of the characterization values ​​corresponding to the template conditions that meet the conditions exceeds the threshold of the sum of the characterization values; wherein the rounded-up value of the quotient of N divided by i is the rounding threshold; and N is the total number of template conditions;

[0187] Based on each target trigger extraction template, try to extract the trigger determination basis from the first visual detection information respectively;

[0188] When the attempt is successful, a security management opportunity and a security management situation are generated based on the extracted trigger determination basis.

[0189] The trigger determination module determines a plurality of template conditions and their corresponding characterization values ​​based on the trigger determination history, including:

[0190] Perform a timeline representation of the trigger determination history to obtain a historical timeline;

[0191] Determine multiple time node clusters from the historical timeline; wherein the maximum node time difference between two time nodes in each time node cluster does not exceed a threshold value of the maximum node time difference;

[0192] Perform commonality analysis on each time node cluster to obtain multiple commonality results and corresponding commonality values;

[0193] Based on the commonality results whose commonality values ​​exceed the threshold value of the commonality value, a template condition is generated, and the commonality value of the commonality result based on which the template condition is generated is used as the corresponding characterization value.

[0194] The model generation module generates a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period, including:

[0195] Performing a graph representation on the second visual detection information to obtain an information graph;

[0196] Based on the mapping acquisition template of each of the multiple sub-situations in the security management situation, the mapping value of each sub-situation is obtained according to the information graph;

[0197] Generate digital mechanisms;

[0198] Based on the digital mechanism, the original model is digitally set;

[0199] The original model after digital setting is used as the digital model;

[0200] Wherein, the digital mechanism includes:

[0201] Traverse each sub-case in order from large to small according to the mapping value;

[0202] During each traversal, based on the digital processing template corresponding to the traversed sub-scenario and according to the second visual detection information, the original model is digitally processed, and after the digital processing, the traversal stay request is continuously detected for a preset first time period;

[0203] When detected, stay at the preset second time length, and then continue to traverse the next sub-situation; otherwise, stay at the preset third time length corresponding to the difference between the mapping value of the next traversed sub-situation and the mapping value of the traversed sub-situation, and then continue to traverse the next sub-situation; wherein the third time length and the difference are negatively correlated.

[0204] The safety management module performs corresponding safety management of the elevator based on the digital model, including:

[0205] Input the digital model into a preset neural network to make a first management decision;

[0206] When the situation value of the safety management situation does not exceed the threshold of the situation value, the first management strategy of the neural network making the first management decision is executed on the elevator; otherwise, the digital model is output to the expert personnel, and the target local sequence is determined from the historical progress sequence of the neural network making the first management decision; wherein the target local sequence is constrained by the first condition;

[0207] Based on the target local sequence, determining a target history from the output history of the digitized model; wherein the target history is subject to a second condition;

[0208] Control the digital model to reproduce the output target history to the expert personnel and prompt the expert personnel to make a second management decision;

[0209] Based on the second management strategy of the expert personnel making the second management decision, the sub-strategies corresponding to each historical progress in the target local sequence in the first management strategy are replaced to obtain the final management strategy;

[0210] Implement final management strategy for elevators;

[0211] The first condition includes:

[0212] The first maximum matching degree between the first multimodal detail feature of the first and last historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold value of the first maximum matching degree;

[0213] or,

[0214] The second maximum matching degree between the second multimodal detail feature of the first historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold of the second maximum matching degree, and there is a progress association between any two adjacent historical progresses in the target local sequence;

[0215] The second condition includes:

[0216] The target history is related to or represents at least one historical progress in the target partial sequence.

[0217] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An elevator safety management method based on visual detection, characterized in that: include: Based on the first visual detection information of the elevator in the first time period, triggering and determining the safety management opportunity and safety management situation; When entering the safety management opportunity, generating a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period; wherein the second time period includes the first time period; Based on the digital model, the elevators can be managed accordingly for safety; The triggering and determining of the safety management opportunity and the safety management situation based on the first visual detection information of the elevator in the first time period includes: Based on the trigger determination history, determining a plurality of template conditions and respective corresponding characterization values; Based on each template condition, multiple target trigger extraction templates are screened out from the trigger extraction template library; wherein each target trigger extraction template meets at least i template conditions, and the sum of the characterization values ​​corresponding to the template conditions that meet the conditions exceeds the threshold of the sum of the characterization values; wherein the rounded-up value of the quotient of N divided by i is the rounding threshold; and N is the total number of template conditions; Based on each target trigger extraction template, try to extract the trigger determination basis from the first visual detection information respectively; When the attempt is successful, a safety management opportunity and a safety management situation are generated based on the extracted trigger determination basis; The step of generating a digital model of a safety management situation based on the second visual detection information of the elevator in the second time period includes: Performing a graph representation on the second visual detection information to obtain an information graph; Based on the mapping acquisition template of each of the multiple sub-situations in the security management situation, the mapping value of each sub-situation is obtained according to the information graph; Generate digital mechanisms; Based on the digital mechanism, the original model is digitally set; The original model after digital setting is used as the digital model; Wherein, the digital mechanism includes: Traverse each sub-case in order from large to small according to the mapping value; During each traversal, based on the digital processing template corresponding to the traversed sub-scenario and according to the second visual detection information, the original model is digitally processed, and after the digital processing, the traversal stay request is continuously detected for a preset first time period; When detected, stay at the preset second time length, and then continue to traverse the next sub-situation; otherwise, stay at the preset third time length corresponding to the difference between the mapping value of the next traversed sub-situation and the mapping value of the traversed sub-situation, and then continue to traverse the next sub-situation; wherein the third time length and the difference are negatively correlated.

2. The elevator safety management method based on visual detection according to claim 1, characterized in that: The determining of a plurality of template conditions and their corresponding characterization values ​​based on the trigger determination history includes: Perform a timeline representation of the trigger determination history to obtain a historical timeline; Determine multiple time node clusters from the historical timeline; wherein the maximum node time difference between two time nodes in each time node cluster does not exceed a threshold value of the maximum node time difference; Perform commonality analysis on each time node cluster to obtain multiple commonality results and corresponding commonality values; Based on the commonality results whose commonality values ​​exceed the threshold value of the commonality value, a template condition is generated, and the commonality value of the commonality result based on which the template condition is generated is used as the corresponding characterization value.

3. The elevator safety management method based on visual detection according to claim 1, characterized in that: Based on the digital model, the elevator is managed accordingly for safety, including: Input the digital model into a preset neural network to make a first management decision; When the situation value of the safety management situation does not exceed the threshold of the situation value, the first management strategy of the neural network making the first management decision is executed on the elevator; otherwise, the digital model is output to the expert personnel, and the target local sequence is determined from the historical progress sequence of the neural network making the first management decision; wherein the target local sequence is constrained by the first condition; Based on the target local sequence, determining a target history from the output history of the digitized model; wherein the target history is subject to a second condition; Control the digital model to reproduce the output target history to the expert personnel and prompt the expert personnel to make a second management decision; Based on the second management strategy of the expert personnel making the second management decision, the sub-strategies corresponding to each historical progress in the target local sequence in the first management strategy are replaced to obtain the final management strategy; Implement final management strategy for elevators; The first condition includes: The first maximum matching degree between the first multimodal detail feature of the first and last historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold value of the first maximum matching degree; or, The second maximum matching degree between the second multimodal detail feature of the first historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold of the second maximum matching degree, and there is a progress association between any two adjacent historical progresses in the target local sequence; The second condition includes: The target history is related to or represents at least one historical progress in the target partial sequence.

4. An elevator safety management system based on visual detection, characterized in that: include: A triggering and determining module, used for triggering and determining a safety management opportunity and a safety management situation based on first visual detection information of the elevator in a first time period; A model generation module, used for generating a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period when entering the safety management opportunity; wherein the second time period includes the first time period; Safety management module, used to manage elevator safety accordingly based on digital models; The trigger determination module triggers and determines the safety management opportunity and safety management situation based on the first visual detection information of the elevator in the first time period, including: Based on the trigger determination history, determining a plurality of template conditions and respective corresponding characterization values; Based on each template condition, multiple target trigger extraction templates are screened out from the trigger extraction template library; wherein each target trigger extraction template meets at least i template conditions, and the sum of the characterization values ​​corresponding to the template conditions that meet the conditions exceeds the threshold of the sum of the characterization values; wherein the rounded-up value of the quotient of N divided by i is the rounding threshold; and N is the total number of template conditions; Based on each target trigger extraction template, try to extract the trigger determination basis from the first visual detection information respectively; When the attempt is successful, a safety management opportunity and a safety management situation are generated based on the extracted trigger determination basis; The model generation module generates a digital model of the safety management situation based on the second visual detection information of the elevator in the second time period, including: Performing a graph representation on the second visual detection information to obtain an information graph; Based on the mapping acquisition template of each of the multiple sub-situations in the security management situation, the mapping value of each sub-situation is obtained according to the information graph; Generate digital mechanisms; Based on the digital mechanism, the original model is digitally set; The original model after digital setting is used as the digital model; Wherein, the digital mechanism includes: Traverse each sub-case in order from large to small according to the mapping value; During each traversal, based on the digital processing template corresponding to the traversed sub-scenario and according to the second visual detection information, the original model is digitally processed, and after the digital processing, the traversal stay request is continuously detected for a preset first time period; When detected, stay at the preset second time length, and then continue to traverse the next sub-situation; otherwise, stay at the preset third time length corresponding to the difference between the mapping value of the next traversed sub-situation and the mapping value of the traversed sub-situation, and then continue to traverse the next sub-situation; wherein the third time length and the difference are negatively correlated.

5. The elevator safety management system based on visual detection according to claim 4, characterized in that: The trigger determination module determines a plurality of template conditions and their corresponding characterization values ​​based on the trigger determination history, including: Perform a timeline representation of the trigger determination history to obtain a historical timeline; Determine multiple time node clusters from the historical timeline; wherein the maximum node time difference between two time nodes in each time node cluster does not exceed a threshold value of the maximum node time difference; Perform commonality analysis on each time node cluster to obtain multiple commonality results and corresponding commonality values; Based on the commonality results whose commonality values ​​exceed the threshold value of the commonality value, a template condition is generated, and the commonality value of the commonality result based on which the template condition is generated is used as the corresponding characterization value.

6. The elevator safety management system based on visual detection according to claim 4, characterized in that: The safety management module performs corresponding safety management of the elevator based on the digital model, including: Input the digital model into a preset neural network to make a first management decision; When the situation value of the safety management situation does not exceed the threshold of the situation value, the first management strategy of the neural network making the first management decision is executed on the elevator; otherwise, the digital model is output to the expert personnel, and the target local sequence is determined from the historical progress sequence of the neural network making the first management decision; wherein the target local sequence is constrained by the first condition; Based on the target local sequence, determining a target history from the output history of the digitized model; wherein the target history is subject to a second condition; Control the digital model to reproduce the output target history to the expert personnel and prompt the expert personnel to make a second management decision; Based on the second management strategy of the expert personnel making the second management decision, the sub-strategies corresponding to each historical progress in the target local sequence in the first management strategy are replaced to obtain the final management strategy; Implement final management strategy for elevators; The first condition includes: The first maximum matching degree between the first multimodal detail feature of the first and last historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold value of the first maximum matching degree; or, The second maximum matching degree between the second multimodal detail feature of the first historical progress in the target local sequence and each of the plurality of standard features exceeds a threshold of the second maximum matching degree, and there is a progress association between any two adjacent historical progresses in the target local sequence; The second condition includes: The target history is related to or represents at least one historical progress in the target partial sequence.

Images