Intelligent Management System and Method for Electronic Safety Tools

Abstract

The present application discloses an intelligent management system and method for electronic safety tools. By automatically capturing images of the tools when they are stored in and taken out of the warehouse, and using computer vision technology based on deep learning to perform multi-level feature analysis on the tool images, the boundary features and global spatial structure features of the tools are respectively captured. Furthermore, through the joint perception and dependence relationship modeling of the two, multi-level tool status information is obtained, so as to realize the intelligent detection of the integrity of the tools and the intelligent management of the storage in and out of the tools. In this way, the efficiency and accuracy of tool management can be significantly improved, human errors can be reduced, and operation safety can be ensured.

Description

Technical Field

[0001] This application relates to the field of intelligent management, and more specifically, to an intelligent management system and method for electronic safety tools. Background Art

[0002] In today's industrial society, industries such as power, chemical, and manufacturing form the cornerstone of the global economy. In these industries, whether it is a towering power plant or a busy production line, safety is always the top priority of enterprise operation, and the management of safety tools is a key link to ensure the safe and efficient progress of production operations, and its importance cannot be ignored.

[0003] Safety tools, including protective clothing, safety helmets, insulating tools, safety glasses, etc., are important barriers for workers to prevent accidental injuries during work, and the integrity of their state directly affects the life safety of operators and the normal progress of production activities.

[0004] Traditionally, the management of tools mainly relies on means such as manual records and regular inspections. However, this method has many deficiencies, such as low efficiency, easy to make mistakes, and difficult to trace. First of all, manual records are prone to errors and it is difficult to track the current state and location of tools in real time, resulting in low management efficiency. Secondly, regular inspections often rely on the experience and subjective judgment of operators, and it is difficult to ensure the comprehensiveness and accuracy of inspections, especially when faced with a large number and various types of tools, this problem is particularly prominent. Especially for occasions that require high safety guarantees, such as power system maintenance, chemical production environments, etc., traditional manual management methods often cannot meet the tool management requirements of high safety standards and real-time monitoring.

[0005] Therefore, an intelligent management method for electronic safety tools is expected. Summary of the Invention

[0006] In order to solve the above technical problems, this application is proposed. The embodiments of this application provide an intelligent management system and method for electronic safety tools, which automatically capture images of tools when they are stored in and out of the warehouse, and use computer vision technology based on deep learning to perform multi-level feature analysis on the tool images, respectively capturing the boundary features and global spatial structure features of the tools, and then through the joint perception and dependence relationship modeling of the two to obtain multi-level tool state information, so as to realize the intelligent detection of tool integrity and the intelligent management of tool storage in and out of the warehouse. In this way, the efficiency and accuracy of tool management can be significantly improved, human errors can be reduced, and operation safety can be ensured.

[0007] According to one aspect of this application, an intelligent management system for electronic safety tools is provided, which includes:

[0008] The tool ledger management module is used to input the information of tools and predict the inspection cycle based on the types and usage information of the tools;

[0009] The warehousing and outbound management module is used to automatically capture images of the tools when they are warehoused or out of warehouse, and conduct integrity inspections and archive records based on the images of the tools;

[0010] The expected reminder module is used to automatically remind of the due inspection based on the inspection cycle;

[0011] The statistical analysis module is used to count the usage frequency and usage time of the tools and generate statistical reports;

[0012] Among them, the warehousing and outbound management module includes:

[0013] The tool image acquisition unit is used to acquire images of the tools;

[0014] The image multi-level feature extraction unit is used to extract the boundary features and global context features of the images of the tools respectively to obtain the tool surface state boundary feature map and the tool surface state global feature map;

[0015] The feature autocorrelation enhancement unit is used to enhance the feature autocorrelation of the tool surface state global feature map to obtain the tool surface state enhanced global feature map;

[0016] The multi-level feature joint perception unit is used to jointly perceive the tool surface state boundary feature map and the tool surface state enhanced global feature map to obtain the tool surface state multi-scale joint perception feature map;

[0017] The tool integrity analysis unit is used to generate the integrity detection result of the tool based on the tool surface state multi-scale joint perception feature map.

[0018] According to another aspect of the present application, there is provided an intelligent management method for electronic safety tools, which includes:

[0019] Acquire images of the tools;

[0020] Extract the boundary features and global context features of the images of the tools respectively to obtain the tool surface state boundary feature map and the tool surface state global feature map;

[0021] Enhance the feature autocorrelation of the tool surface state global feature map to obtain the tool surface state enhanced global feature map;

[0022] Jointly sense the boundary feature map of the surface state of the tool and the global enhanced feature map of the surface state of the tool to obtain a multi-scale joint perception feature map of the surface state of the tool;

[0023] Generate an integrity detection result of the tool based on the multi-scale joint perception feature map of the surface state of the tool.

[0024] Compared with the prior art, an intelligent management system and method for electronic safety tools provided by the present application automatically captures tool images when the tools are stored in and out of the warehouse, and uses computer vision technology based on deep learning to perform multi-level feature analysis on the tool images, respectively capturing the boundary features and global spatial structure features of the tools. Furthermore, through the joint perception and dependence relationship modeling of the two, multi-level tool state information is obtained, so as to realize the intelligent detection of the integrity of the tools and the intelligent management of the storage and out of the warehouse of the tools. In this way, the efficiency and accuracy of tool management can be significantly improved, human errors can be reduced, and operation safety can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present application will become more obvious. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 It is a block diagram of an intelligent management system for electronic safety tools according to an embodiment of the present application;

[0027] Figure 2 It is a schematic diagram of data flow of an intelligent management system for electronic safety tools according to an embodiment of the present application;

[0028] Figure 3 It is a block diagram of an inbound and outbound management module in an intelligent management system for electronic safety tools according to an embodiment of the present application;

[0029] Figure 4 It is a block diagram of an image multi-level feature extraction unit in an intelligent management system for electronic safety tools according to an embodiment of the present application;

[0030] Figure 5 It is a flowchart of an intelligent management method for electronic safety tools according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0032] As shown in the present application and the claims, unless the context clearly indicates an exception, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0033] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0034] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the previous or following operations are not necessarily executed precisely in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.

[0035] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0036] Traditionally, the management of tools mainly relies on means such as manual records and regular inspections. However, this method has many deficiencies, such as low efficiency, error-proneness, and difficulty in traceability. First of all, manual records are prone to errors and it is difficult to track the current status and location of tools in real time, resulting in low management efficiency. Secondly, regular inspections often rely on the experience and subjective judgment of operators, and it is difficult to ensure the comprehensiveness and accuracy of inspections, especially when facing a large number and variety of tools, this problem is particularly prominent. Especially for occasions that require a high level of safety assurance, such as power system maintenance, chemical production environments, etc., the traditional manual management method often cannot meet the tool management requirements of high safety standards and real-time monitoring. Therefore, an intelligent electronic safety tool management method is expected.

[0037] In the technical solution of the present application, an intelligent management system for electronic safety tools is proposed. Figure 1It is a block diagram of an intelligent management system for electronic safety tools according to an embodiment of the present application. Figure 2 It is a schematic diagram of data flow of an intelligent management system for electronic safety tools according to an embodiment of the present application. As Figure 1 and Figure 2 shown, the intelligent management system 300 for electronic safety tools according to an embodiment of the present application includes: a tool ledger management module 310, configured to enter information of the tools and predict the inspection period based on the type and usage information of the tools; an inbound / outbound management module 320, configured to automatically capture an image of the tool when the tool is inbound / outbound, and perform integrity inspection and archive records based on the image of the tool; an expected reminder module 330, configured to automatically remind of the due inspection based on the inspection period; and a statistical analysis module 340, configured to count the usage frequency and usage time of the tools and generate a statistical report.

[0038] Specifically, the tool ledger management module 310 is configured to enter information of the tools and predict the inspection period based on the type and usage information of the tools. It should be understood that different types of tools may have different inspection periods. For example, high-risk tools may require more frequent inspections. In the technical solution of the present application, by optimizing the inspection plan of the tools, it is ensured that the tools are regularly inspected and maintained. Specifically, a computer vision technology based on deep learning is used to perform multi-level feature analysis on the tool images, respectively capturing the boundary features and global spatial structure features of the tools, and then through joint perception and dependency relationship modeling of the two, to obtain multi-level tool status information, so as to realize intelligent detection of tool integrity and intelligent management of tool inbound / outbound, thereby improving the efficiency and usage safety of tool management.

[0039] Specifically, the inbound / outbound management module 320 is configured to automatically capture an image of the tool when the tool is inbound / outbound, and perform integrity inspection and archive records based on the image of the tool. Specifically, in a specific example of the present application, as Figure 3As shown, the incoming and outgoing warehouse management module 320 includes: a tool image acquisition unit 321 for acquiring an image of the tool; an image multi-level feature extraction unit 322 for respectively extracting the boundary feature and the global context feature of the tool image to obtain a tool surface state boundary feature map and a tool surface state global feature map; a feature self-correlation enhancement unit 323 for performing feature self-correlation enhancement on the tool surface state global feature map to obtain a tool surface state enhanced global feature map; a multi-level feature joint perception unit 324 for jointly perceiving the tool surface state boundary feature map and the tool surface state enhanced global feature map to obtain a tool surface state multi-scale joint perception feature map; and a tool integrity analysis unit 325 for generating an integrity detection result of the tool based on the tool surface state multi-scale joint perception feature map.

[0040] Specifically, the tool image acquisition unit 321 is used to acquire an image of the tool. It should be understood that collecting the image of the tool through image capture technology can not only realize automatic integrity detection by combining computer vision algorithms, but also save the image as a historical record for subsequent data analysis and information traceability, such as for statistics of information such as the usage frequency and damage condition of the tool, providing data support for the maintenance, replacement and management of the tool.

[0041] Specifically, the image multi-level feature extraction unit 322 is used to respectively extract the boundary feature and the global context feature of the tool image to obtain a tool surface state boundary feature map and a tool surface state global feature map. In particular, in a specific example of the present application, as Figure 4 shown, the image multi-level feature extraction unit 322 includes: an image grayscale conversion subunit 3221 for performing grayscale processing on the tool image to obtain a tool surface state grayscale image; and a multi-level feature extraction subunit 3222 for inputting the tool surface state grayscale image into an MBCNet model including a boundary feature extraction branch and a backbone network to obtain the tool surface state boundary feature map and the tool surface state global feature map.

[0042] More specifically, the image grayscale conversion subunit 3221 is configured to perform grayscale processing on the image of the tool to obtain a grayscale image of the surface state of the tool. Considering that during the image acquisition process, the image of the tool may be affected by changes in illumination, resulting in color deviation in the image, thereby affecting the accuracy of integrity detection. Therefore, in this application, the image of the tool is further subjected to grayscale processing to obtain a grayscale image of the surface state of the tool, so as to eliminate color differences, simplify the complexity of subsequent feature extraction, and make the edge texture features of the tool in the image more prominent, so as to improve the accuracy of subsequent feature analysis.

[0043] More specifically, the multi-level feature extraction subunit 3222 is configured to input the grayscale image of the surface state of the tool into the MBCNet model including a boundary feature extraction branch and a backbone network to obtain the boundary feature map of the surface state of the tool and the global feature map of the surface state of the tool. Considering that during the process of using a neural network model to extract features from the grayscale image of the surface state of the tool, as the number of network layers deepens, the granularity of feature expression will gradually refine from local to global, resulting in the loss of image boundary information. Therefore, in order to simultaneously retain the boundary features and global structural features of the tool, this application uses the MBCNet model including a boundary feature extraction branch and a backbone network to process the grayscale image of the surface state of the tool. Among them, the boundary feature extraction branch adopts a lightweight network structure with a small receptive field, focusing on capturing the edges and local details of the tool to generate the boundary feature map of the surface state of the tool; while the backbone network adopts a deep network structure with a large receptive field to obtain the global context information of the tool and generate the global feature map of the surface state of the tool. This network design with a forked structure can ensure that boundary information is not lost while extracting global features, thereby improving the accuracy of tool integrity detection.

[0044] It is worth mentioning that in other specific examples of this application, the boundary features and global context features of the image of the tool can also be extracted separately by other means to obtain the boundary feature map of the surface state of the tool and the global feature map of the surface state of the tool. For example: input the image of the tool; use an edge detection algorithm (such as Canny edge detection) to extract the boundary from the tool image to obtain the boundary feature map of the surface state of the tool; among them, the boundary feature map is a binary image, where the boundary pixels are 1 and the non-boundary pixels are 0. Use a global pooling layer to extract the global context features from the tool image to obtain the global feature map of the surface state of the tool.

[0045] Specifically, the feature self - correlation enhancement unit 323 is used to perform feature self - correlation enhancement on the global feature map of the surface state of the tool to obtain the enhanced global feature map of the surface state of the tool. In a specific example of the present application, the global feature map of the surface state of the tool is input into the self - attention enhanced cross - channel feature space structure preservation module to obtain the enhanced global feature map of the surface state of the tool. It should be understood that in order to further enhance the global structural feature representation of the surface state of the tool, the present application introduces the self - attention enhanced cross - channel feature space structure preservation module to perform feature enhancement processing on the global feature map of the surface state of the tool. Specifically, this module first performs layer normalization on the global feature map of the surface state of the tool to eliminate the scale difference between layers and ensure the stability of subsequent feature processing. Then, through multi - layer convolution operations, it captures the channel context correlation and spatial structure information of the global feature map of the surface state of the tool, and uses its channel context correlation information as the query, and the spatial structure information as the key and value, and performs feature cross - channel global correlation interaction based on the self - attention mechanism. Thus, while retaining the spatial structure of the feature map, it fuses the channel correlation information between feature structures in units of the feature space structure, enhances the expression ability of the feature map, and generates the enhanced global feature map of the surface state of the tool, providing a more accurate feature representation for the integrity assessment of the tool.

[0046] In the embodiment of the present application, performing feature self - correlation enhancement on the global feature map of the surface state of the tool to obtain the enhanced global feature map of the surface state of the tool includes: performing layer normalization on the global feature map of the surface state of the tool to obtain the normalized global feature map of the surface state of the tool; performing point convolution processing on the normalized global feature map of the surface state of the tool to obtain the feature map representing the channel context correlation of the surface state of the tool; performing convolutional encoding on the feature map representing the channel context correlation of the surface state of the tool to obtain the feature map representing the spatial context correlation of the surface state of the tool; performing channel - space global interaction attention fusion on the feature map representing the channel context correlation of the surface state of the tool and the feature map representing the spatial context correlation of the surface state of the tool to obtain the enhanced global feature map of the surface state of the tool.

[0047] Among them, the process of performing channel - space global interaction attention fusion on the tool surface state channel context - associated representation feature map and the tool surface state space context - associated representation feature map to obtain the tool surface state enhanced global feature map includes: copying the tool surface state space context - associated representation feature map to obtain a backup tool surface state space context - associated representation feature map; reshaping the feature shapes of the tool surface state channel context - associated representation feature map, the tool surface state space context - associated representation feature map, and the backup tool surface state space context - associated representation feature map to obtain a tool surface state channel context - associated representation feature matrix, a tool surface state space context - associated representation feature matrix, and a backup tool surface state space context - associated representation feature matrix; calculating the cross - channel cross - covariance matrix between the tool surface state channel context - associated representation feature matrix and the tool surface state space context - associated representation feature matrix; using the Softmax function to activate the cross - channel cross - covariance matrix to obtain a tool surface state feature global interaction attention matrix; calculating the product between the backup tool surface state space context - associated representation feature matrix and the tool surface state feature global interaction attention matrix to obtain an attention - enhanced tool surface state feature representation matrix; reshaping the feature shape of the attention - enhanced tool surface state feature representation matrix to obtain the tool surface state enhanced global feature map.

[0048] In summary, in the above - mentioned embodiments, performing feature autocorrelation enhancement on the tool surface state global feature map to obtain the tool surface state enhanced global feature map includes: processing the tool surface state global feature map with the following autocorrelation attention enhancement formula to obtain the tool surface state enhanced global feature map, where the autocorrelation attention enhancement formula is:

[0049] F ln = Layer Normalization(F i )

[0050] F Q = Conv 1×1 (F ln )

[0051] F K = Conv 3×3 (Conv 1×1 (F ln ))

[0052] F V = Copy(F K )

[0053] MQ = Reshape(F Q )

[0054] M K = Reshape(F K )

[0055] M V = Reshape(F V )

[0056]

[0057]

[0058] where F i represents the global feature map of the surface state of the tool, Layer Normalization(·) represents the layer normalization operation, F ln represents the normalized global feature map of the surface state of the tool, Conv 1×1 represents point convolution, F Q represents the feature map of the channel context correlation representation of the surface state of the tool, Conv 3×3 represents the convolution process based on a 3×3 convolution kernel, F K represents the feature map of the spatial context correlation representation of the surface state of the tool, Copy(·) represents the copy operation, F V represents the feature map of the backup spatial context correlation representation of the surface state of the tool, reshape(·) represents the reshaping of the feature shape, M Q 、M K and M V represent the feature matrix of the channel context correlation representation of the surface state of the tool, the feature matrix of the spatial context correlation representation of the surface state of the tool, and the feature matrix of the backup spatial context correlation representation of the surface state of the tool respectively, M a represents the cross-channel cross-covariance matrix, θ is the scaling factor, softmax represents the normalized exponential function, represents the matrix multiplication operation, F a represents the enhanced global feature map of the surface state of the tool.

[0059] Specifically, the multi-level feature joint perception unit 324 is used to jointly perceive the tool surface state boundary feature map and the tool surface state enhanced global feature map to obtain a tool surface state multi-scale joint perception feature map. In a specific example of the present application, the tool surface state boundary feature map and the tool surface state enhanced global feature map are input into a saliency joint perception module based on the attention mechanism to obtain the tool surface state multi-scale joint perception feature map. It should be understood that the tool surface state boundary feature map mainly focuses on details such as the edges and contours of the tool surface, while the tool surface state enhanced global feature map emphasizes the global shape and structural features of the tool. To more comprehensively describe the integrity state of the tool, the present application introduces a saliency joint perception module based on the attention mechanism to fuse the tool surface state boundary feature map and the tool surface state enhanced global feature map. Based on the dependency relationship between the boundary features and the global structural features on the tool surface, the deep fusion of multi-level features is realized, and the integrity of the tool is jointly perceived. Specifically, the module first reshapes the shapes of the tool surface state boundary feature map and the tool surface state enhanced global feature map into feature matrices to adapt their dimensions and shapes to the input requirements of the attention mechanism. Then, the two-way dependency relationship between the two groups of feature matrices is calculated through the feature-by-channel interaction perception module, the interaction between features is quantified, and boundary-global and global-boundary dependency relationship matrices are generated as the guiding basis for feature attention optimization and fusion. At the same time, in order to reduce the sensitivity of the model to noise and prevent overfitting, the two groups of dependency relationship matrices are further subjected to dropout processing, and the generalization ability of the model is enhanced by selectively ignoring or suppressing a part of feature interactions. Furthermore, based on the two groups of dependency relationship matrices after dropout processing, the two groups of reshaped feature matrices are weighted and optimized, and then restored to the original feature map form through reshaping and weighted fusion to generate a tool surface state multi-scale joint perception feature map, thus effectively improving the diversity and richness of feature representation and making the integrity assessment of the tool more accurate and comprehensive.

[0060] In the embodiments of the present application, joint perception of the tool surface state boundary feature map and the tool surface state enhanced global feature map is performed to obtain a tool surface state multi-scale joint perception feature map, including: reshaping the feature shapes of the tool surface state boundary feature map and the tool surface state enhanced global feature map to obtain a tool surface state boundary feature shape reshaping matrix and a tool surface state global feature shape reshaping matrix; inputting the tool surface state boundary feature shape reshaping matrix and the tool surface state global feature shape reshaping matrix into a feature per-channel interactive perception module to obtain a boundary-global dependence relationship matrix and a global-boundary dependence relationship matrix; inputting the boundary-global dependence relationship matrix and the global-boundary dependence relationship matrix into a dropout module to obtain a pruned boundary-global dependence relationship matrix and a pruned global-boundary dependence relationship matrix; multiplying the pruned boundary-global dependence relationship matrix by the tool surface state global feature shape reshaping matrix to obtain a dependence relationship optimized tool surface state global feature matrix; multiplying the pruned global-boundary dependence relationship matrix by the tool surface state boundary feature shape reshaping matrix to obtain a dependence relationship optimized tool surface state boundary feature matrix; reshaping the feature shapes of the dependence relationship optimized tool surface state boundary feature matrix and the dependence relationship optimized tool surface state global feature matrix to obtain an optimized tool surface state boundary feature map and an optimized tool surface state enhanced global feature map; calculating the weighted sum of the optimized tool surface state boundary feature map and the optimized tool surface state enhanced global feature map to obtain the tool surface state multi-scale joint perception feature map.

[0061] Among them, the process of inputting the tool surface state boundary feature shape reshaping matrix and the tool surface state global feature shape reshaping matrix into a feature per-channel interactive perception module to obtain a boundary-global dependence relationship matrix and a global-boundary dependence relationship matrix includes: calculating the tool surface state boundary feature shape reshaping matrix multiplied by the transpose matrix of the tool surface state global feature shape reshaping matrix to obtain a tool surface state boundary-global correlation representation matrix; dividing the tool surface state boundary-global correlation representation matrix by the scale of the tool surface state global feature shape reshaping matrix and then inputting it into the softmax function to obtain the boundary-global dependence relationship matrix; calculating the transpose matrix of the tool surface state boundary feature shape reshaping matrix multiplied by the tool surface state global feature shape reshaping matrix to obtain a tool surface state global-boundary correlation representation matrix; dividing the tool surface state boundary-global correlation representation matrix by the scale of the tool surface state boundary feature shape reshaping matrix and then inputting it into the softmax function to obtain the global-boundary dependence relationship matrix.

[0062] In summary, in the above embodiments, the joint perception of the tool surface state boundary feature map and the tool surface state enhanced global feature map is performed to obtain the tool surface state multi-scale joint perception feature map, including: processing the tool surface state boundary feature map and the tool surface state enhanced global feature map with the following joint perception formula to obtain the tool surface state multi-scale joint perception feature map, where the joint perception formula is:

[0063] M1 = Reshape(F1)

[0064] M2 = Reshape(F2)

[0065]

[0066]

[0067] W 21 = Dropout(M 21 )

[0068] W 12 = Dropout(M 12 )

[0069]

[0070]

[0071] F1’ = Reshape(M 1′ )

[0072] F2’ = Reshape(M 2′ )

[0073] F c = αF1’ + βF2’

[0074] Wherein, F1 represents the tool surface state boundary feature map, F2 represents the tool surface state enhanced global feature map, reshape(·) represents feature shape reshaping, M1 represents the tool surface state boundary feature shape reshaping matrix, M2 represents the tool surface state global feature shape reshaping matrix, represents matrix multiplication, (·) T represents the transpose of the matrix, S represents the scale of the tool surface state boundary feature shape reshaping matrix, that is, the width multiplied by the height of the tool surface state boundary feature shape reshaping matrix, and the scales of the tool surface state boundary feature shape reshaping matrix and the tool surface state global feature shape reshaping matrix are the same, Softmax is the normalized exponential function, M 21Denote the boundary-global dependency matrix, M 12 Denote the global-boundary dependency matrix, Dropout(·) represents the dropout process, W 21 Denote the pruned boundary-global dependency matrix, W 12 Denote the pruned global-boundary dependency matrix, M 1′ Denote the boundary feature matrix of the surface state of the dependency relationship optimization tool, M 2′ Denote the global feature matrix of the surface state of the dependency relationship optimization tool, F1’ represents the boundary feature map of the optimized tool surface state, F2’ represents the enhanced global feature map of the optimized tool surface state, α and β respectively represent different weight coefficients, F c Denote the multi-scale joint perception feature map of the surface state of the tool.

[0075] Specifically, the tool integrity analysis unit 325 is configured to generate an integrity detection result of the tool based on the multi-scale joint perception feature map of the surface state of the tool. In a specific example of the present application, the multi-scale joint perception feature map of the surface state of the tool is input into an integrity detection result generation module based on a classifier to obtain the integrity detection result, and the integrity detection result is used to indicate whether the tool is complete.

[0076] In a preferred example, inputting the multi-scale joint perception feature map of the surface state of the tool into an integrity detection result generation module based on a classifier to obtain the integrity detection result includes the steps of:

[0077] Perform clustering on all eigenvalues of the multi-scale joint perception feature map of the surface state of the tool based on the L2 distance between eigenvalues, and arrange the clustering features into a multi-scale joint perception clustering vector of the surface state of the tool;

[0078] Determine the clustering ratio value of the number of eigenvalues of the multi-scale joint perception clustering vector of the surface state of the tool to the number of eigenvalues of the multi-scale joint perception feature map of the surface state of the tool;

[0079] Divide the two-norm of the multi-scale joint perception clustering vector of the surface state of the tool by the two-norm of the multi-scale joint perception feature vector of the surface state of the tool obtained after unfolding the multi-scale joint perception feature map of the surface state of the tool to obtain a multi-scale joint perception conflict representation value of the surface state of the tool;

[0080] Divide the first power value of the one-norm of the multi-scale joint perception clustering vector of the surface state of the tool with the clustering ratio value as the exponent by the second power value of the one-norm of the multi-scale joint perception feature vector of the surface state of the tool with the clustering ratio value as the exponent to obtain a multi-scale joint perception adversarial representation value of the surface state of the tool;

[0081] For each eigenvalue of the multi-scale joint perception clustering vector of the surface state of the tool, multiply it by the reciprocal of the difference between the multi-scale joint perception conflict representation value and the multi-scale joint perception confrontation representation value of the surface state of the tool to obtain the optimized eigenvalue of the multi-scale joint perception clustering vector of the surface state of the tool;

[0082] For each eigenvalue outside the clustering in the multi-scale joint perception feature map of the surface state of the tool, multiply it by the reciprocal of the sum of the multi-scale joint perception conflict representation value and the multi-scale joint perception confrontation representation value of the surface state of the tool to obtain the optimized out-of-class eigenvalue of the multi-scale joint perception feature map of the surface state of the tool;

[0083] Combine the optimized eigenvalues of the multi-scale joint perception clustering vector of the surface state of the tool and the optimized out-of-class eigenvalues of the multi-scale joint perception feature map of the surface state of the tool to form an optimized multi-scale joint perception feature map of the surface state of the tool; and

[0084] Input the optimized multi-scale joint perception feature map of the surface state of the tool into the integrity detection result generation module based on the classifier to obtain the integrity detection result.

[0085] Expressed as:

[0086]

[0087] where V n is the multi-scale joint perception feature vector of the surface state of the tool, n is the number of eigenvalues of the multi-scale joint perception feature vector of the surface state of the tool, is the multi-scale joint perception clustering vector of the surface state of the tool, k is the number of eigenvalues of the multi-scale joint perception clustering vector of the surface state of the tool, represents the clustering feature set corresponding to the multi-scale joint perception clustering vector of the surface state of the tool, ‖·‖2 and respectively represent the second norm of the vector and the λ-th power of the first norm.

[0088] That is, considering that the surface state boundary feature map and the surface state enhanced global feature map of the tool represent the image shallow edge enhancement feature and the image global feature of the surface state gray image of the tool respectively, during the further significant joint perception based on the attention mechanism, due to the difference in the attention weights of the image shallow-deep fusion, there will be a significant joint alignment conflict in the image semantics, resulting in the loss of aggregated key information and affecting the expression effect of the multi-scale joint perception feature map of the surface state of the tool.

[0089] Thus, in order to avoid the loss of key suffix semantic information of the multi-scale joint perception feature map of the tool surface state relative to the overall original feature set due to aggregation conflict based on the aggregated features, by using the clustering ratio of the number of eigenvalues of the multi-scale joint perception clustering vector of the tool surface state to the number of eigenvalues of the multi-scale joint perception feature map of the tool surface state as a decision function, an adversarial decision of the absolute representation of the set of the first norms of the multi-scale joint perception clustering vector of the tool surface state and the multi-scale joint perception feature vector of the tool surface state is performed, and positive and negative interactions are respectively carried out with the clustering internal conflict representation of the second norms of the multi-scale joint perception clustering vector of the tool surface state and the multi-scale joint perception feature vector of the tool surface state, so as to construct a firm alignment guardrail of the optimized multi-scale joint perception feature map of the tool surface state based on the aggregated features with the overall original feature set, thereby realizing the mitigation of the harmful information loss intention based on the aggregatable risk transferability of the optimized multi-scale joint perception feature map of the tool surface state, improving the expression effect of the optimized multi-scale joint perception feature map of the tool surface state, and thus improving the accuracy of the integrity detection result obtained by the integrity detection result generation module based on the input of the multi-scale joint perception feature map of the tool surface state through a classifier.

[0090] Specifically, the expected reminder module 330 and the statistical analysis module 340 are used to automatically remind of the expiration of inspection based on the detection cycle; and count the usage frequency and usage time of the tool and generate a statistical report. It should be understood that tools without inspection and maintenance may malfunction, thus posing a safety hazard, and automatically reminding of the expiration of inspection can help ensure that the tools are inspected and maintained within the recommended time, so as to simplify the tool management process and improve efficiency. Further, counting the usage frequency and usage time of the tool can help track the usage of the tool. By analyzing the usage of the tool, the tool management can be further optimized.

[0091] As described above, the electronic safety tool intelligent management system 300 according to the embodiments of the present application can be implemented in various wireless terminals, such as a server with an electronic safety tool intelligent management algorithm. In a possible implementation manner, the electronic safety tool intelligent management system 300 according to the embodiments of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the electronic safety tool intelligent management system 300 can be a software module in the operating system of the wireless terminal, or can be an application program developed for the wireless terminal; of course, the electronic safety tool intelligent management system 300 can also be one of the numerous hardware modules of the wireless terminal.

[0092] Alternatively, in another example, the intelligent management system 300 for electronic safety tools and the wireless terminal may also be separate devices, and the intelligent management system 300 for electronic safety tools can be connected to the wireless terminal through a wired and / or wireless network and transmit interaction information according to a predefined data format.

[0093] Furthermore, an intelligent management method for electronic safety tools is also provided.

[0094] Figure 5 FIG. is a flowchart of an intelligent management method for electronic safety tools according to an embodiment of the present application. As Figure 5 shown, the intelligent management method for electronic safety tools according to an embodiment of the present application includes the steps of: S1, acquiring an image of the tool; S2, respectively extracting the boundary feature and the global context feature of the image of the tool to obtain a tool surface state boundary feature map and a tool surface state global feature map; S3, performing feature autocorrelation enhancement on the tool surface state global feature map to obtain a tool surface state enhanced global feature map; S4, performing joint perception on the tool surface state boundary feature map and the tool surface state enhanced global feature map to obtain a tool surface state multi-scale joint perception feature map; S5, generating an integrity detection result of the tool based on the tool surface state multi-scale joint perception feature map.

[0095] In summary, the intelligent management method for electronic safety tools according to an embodiment of the present application is elucidated. By automatically capturing images of tools when they are checked in and out, and using computer vision technology based on deep learning to perform multi-level feature analysis on the tool images, the boundary features and global spatial structure features of the tools are respectively captured. Then, through joint perception and dependency relationship modeling of the two, multi-level tool state information is obtained, thereby realizing intelligent detection of the integrity of the tools and intelligent management of tool check-in and check-out. In this way, the efficiency and accuracy of tool management can be significantly improved, human errors can be reduced, and operation safety can be ensured.

[0096] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technologies in the market, or to enable other ordinary skilled persons in the technical field to understand the disclosed embodiments.

Claims

1. An intelligent management system for electronic safety tools, characterized in that, Including: The tool ledger management module is used to input the information of tools and predict the detection period based on the type and usage information of the tools; The inventory management module is used to automatically capture the images of the tools when the tools are in and out of storage, and perform integrity checks and archive records based on the images of the tools; The expected reminder module is used to automatically remind of the due inspection based on the detection period; The statistical analysis module is used to count the usage frequency and usage time of the tools and generate statistical reports; Among them, the inventory management module includes: The tool image acquisition unit is used to acquire the images of the tools; The image multi-level feature extraction unit is used to extract the boundary features and global context features of the tool images respectively to obtain the tool surface state boundary feature map and the tool surface state global feature map; The feature self-correlation enhancement unit is used to perform feature self-correlation enhancement on the tool surface state global feature map to obtain the tool surface state enhanced global feature map; The multi-level feature joint perception unit is used to jointly perceive the tool surface state boundary feature map and the tool surface state enhanced global feature map to obtain the tool surface state multi-scale joint perception feature map; The tool integrity analysis unit is used to generate the integrity detection result of the tool based on the tool surface state multi-scale joint perception feature map; The feature self-correlation enhancement unit includes: The layer normalization sub-unit is used to perform layer normalization on the tool surface state global feature map to obtain the normalized tool surface state global feature map; The channel context correlation feature extraction sub-unit is used to perform point convolution processing on the normalized tool surface state global feature map to obtain the tool surface state channel context correlation representation feature map; The spatial context correlation feature extraction sub-unit is used to perform convolutional coding on the tool surface state channel context correlation representation feature map to obtain the tool surface state spatial context correlation representation feature map; The global interaction attention fusion sub-unit is used to: Copy the tool surface state spatial context correlation representation feature map to obtain a backup tool surface state spatial context correlation representation feature map; Perform feature shape reshaping on the tool surface state channel context correlation representation feature map, the tool surface state spatial context correlation representation feature map, and the backup tool surface state spatial context correlation representation feature map to obtain the tool surface state channel context correlation representation feature matrix, the tool surface state spatial context correlation representation feature matrix, and the backup tool surface state spatial context correlation representation feature matrix; Calculate the cross-channel cross-covariance matrix between the tool surface state channel context correlation representation feature matrix and the tool surface state spatial context correlation representation feature matrix; Use the Softmax function to activate the cross-channel cross-covariance matrix to obtain the tool surface state feature global interaction attention matrix; Calculate the product between the context - associated representation feature matrix of the surface state of the backup tool and the global interaction attention matrix of the surface state features of the tool to obtain the attention - enhanced surface state feature representation matrix of the tool; Reshape the feature shape of the attention - enhanced surface state feature representation matrix of the tool to obtain the enhanced global feature map of the surface state of the tool.

2. The intelligent management system for electronic safety tools according to claim 1, characterized in that The image multi - level feature extraction unit includes: An image grayscale conversion sub - unit, which is used to perform grayscale processing on the image of the tool to obtain the grayscale image of the surface state of the tool; A multi - level feature extraction sub - unit, which is used to input the grayscale image of the surface state of the tool into the MBCNet model including a boundary feature extraction branch and a backbone network to obtain the boundary feature map of the surface state of the tool and the global feature map of the surface state of the tool.

3. The intelligent management system for electronic safety tools according to claim 2, characterized in that The feature self - correlation enhancement unit is used for: Input the global feature map of the surface state of the tool into a self - attention enhanced cross - channel feature space structure preservation module to obtain the enhanced global feature map of the surface state of the tool.

4. The intelligent management system for electronic safety tools according to claim 3, wherein The multi - level feature joint perception unit is used for: Input the boundary feature map of the surface state of the tool and the enhanced global feature map of the surface state of the tool into a significant joint perception module based on the attention mechanism to obtain the multi - scale joint perception feature map of the surface state of the tool.

5. The intelligent management system for electronic safety tools according to claim 4, wherein The multi - level feature joint perception unit includes: A first feature shape reshaping sub - unit, which is used to reshape the feature shapes of the boundary feature map of the surface state of the tool and the enhanced global feature map of the surface state of the tool to obtain the reshaped matrix of the boundary feature shape of the surface state of the tool and the reshaped matrix of the global feature shape of the surface state of the tool; A two - way dependence relationship calculation sub - unit, which is used to input the reshaped matrix of the boundary feature shape of the surface state of the tool and the reshaped matrix of the global feature shape of the surface state of the tool into a feature per - channel interaction perception module to obtain the boundary - global dependence relationship matrix and the global - boundary dependence relationship matrix; A dropout sub - unit, which is used to input the boundary - global dependence relationship matrix and the global - boundary dependence relationship matrix into a dropout module to obtain the pruned boundary - global dependence relationship matrix and the pruned global - boundary dependence relationship matrix; A semantic feature dependence relationship optimization sub - unit, which is used to multiply the pruned boundary - global dependence relationship matrix with the reshaped matrix of the global feature shape of the surface state of the tool to obtain the optimized global feature matrix of the surface state of the tool with dependence relationship; A shallow - layer feature dependence relationship optimization sub - unit, which is used to multiply the pruned global - boundary dependence relationship matrix with the reshaped matrix of the boundary feature shape of the surface state of the tool to obtain the optimized boundary feature matrix of the surface state of the tool with dependence relationship; A second feature shape reshaping sub - unit, which is used to reshape the optimized boundary feature matrix of the surface state of the tool and the optimized global feature matrix of the surface state of the tool with dependence relationship to obtain the optimized boundary feature map of the surface state of the tool and the optimized enhanced global feature map of the surface state of the tool; A multi-scale joint subunit for calculating the weighted sum of the optimized tool surface state boundary feature map and the optimized tool surface state enhanced global feature map to obtain the tool surface state multi-scale joint perception feature map.

6. The intelligent management system for electronic safety tools according to claim 5, wherein The bidirectional dependence relationship calculation subunit is used for: Calculating the product of the tool surface state boundary feature shape reshaping matrix and the transpose matrix of the tool surface state global feature shape reshaping matrix to obtain the tool surface state boundary-global correlation representation matrix; Dividing the tool surface state boundary-global correlation representation matrix by the scale of the tool surface state global feature shape reshaping matrix and then inputting it into the softmax function to obtain the boundary-global dependence relationship matrix; Calculating the product of the transpose matrix of the tool surface state boundary feature shape reshaping matrix and the tool surface state global feature shape reshaping matrix to obtain the tool surface state global-boundary correlation representation matrix; Dividing the tool surface state boundary-global correlation representation matrix by the scale of the tool surface state boundary feature shape reshaping matrix and then inputting it into the softmax function to obtain the global-boundary dependence relationship matrix.

7. The intelligent management system for electronic safety tools according to claim 6, characterized in that, The tool integrity analysis unit is used for: Inputting the tool surface state multi-scale joint perception feature map into the integrity detection result generation module based on the classifier to obtain the integrity detection result, and the integrity detection result is used to indicate whether the tool is complete.

8. An intelligent management method for electronic safety tools, which is executed by the intelligent management system for electronic safety tools according to any one of claims 1 to 7, characterized in that It includes: Obtaining an image of the tool; Respectively extracting the boundary feature and the global context feature of the image of the tool to obtain the tool surface state boundary feature map and the tool surface state global feature map; Performing feature self-correlation enhancement on the tool surface state global feature map to obtain the tool surface state enhanced global feature map; Performing joint perception on the tool surface state boundary feature map and the tool surface state enhanced global feature map to obtain the tool surface state multi-scale joint perception feature map; Generating an integrity detection result of the tool based on the tool surface state multi-scale joint perception feature map.

Images