Intelligent electricity utilization safety inspection system
Through the intelligent power safety inspection system, multimodal knowledge base and intelligent terminal data collection, combined with intelligent cognition and knowledge reasoning technology, the problem of insufficient accuracy and decision-making support capabilities of power safety inspection in the existing technology is solved. It realizes efficient identification of safety hazards of power equipment and grid safety management.
Patent Information
- Application Number
- CN202411258511.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-22
AI Technical Summary
Existing power safety inspections rely on manual inspections, resulting in inaccurate and timely detection, difficult to adapt to complex multi-factor faults, and lack decision-making support capabilities, which cannot meet the needs of intelligent power grids and digital marketing management.
Design an intelligent power safety inspection system, including multimodal knowledge base, intelligent terminal data acquisition and processing, multimodal intelligent diagnosis and decision support module and power customer power safety inspection service module, using intelligent cognition and knowledge reasoning Technology uses safety hazard identification and evaluation, and provides panoramic display and real-time monitoring functions.
It realizes accurate and efficient identification of safety hazards of power equipment, improves the efficiency of power safety inspection and decision-making support capabilities, and meets the needs of intelligent power grids and digital marketing management.
Smart Images

Figure CN119003739A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to power operation management, and in particular to an intelligent power safety inspection system. Background Art
[0002] In the continuous development of the power system, electricity safety inspection plays a vital role. Its goal is to assist power customers in improving the safety management standards of power equipment, quickly discover and promptly deal with potential safety risks in the operation of grid-related power equipment, so as to enhance the overall security of the power grid.
[0003] At present, electricity safety inspections mainly rely on regular inspections by professionals. This method not only consumes a lot of resources, but also relies heavily on the experience and skills of personnel. Due to the differences in subjective judgment and skill levels among different personnel, the detection and identification of safety hazards are often not accurate and timely, which undoubtedly increases the risk of power grid operation. At the same time, with the increasing complexity of power equipment, safety hazards have gradually changed from single faults to more complex multi-factor faults. Traditional manual electricity safety inspection methods are difficult to adapt to this complexity, and thus cannot achieve accurate and efficient identification of safety hazards.
[0004] In addition, power safety inspections also lack the ability to support decision-making. The current business process has encountered many difficulties in risk assessment and equipment intelligence, including unclear status information of key power equipment, lack of ability to troubleshoot non-electrical safety hazards, insufficient decision support capabilities, low intelligence level of on-site operations, and limited integration capabilities. These challenges not only restrict the development of smart grids, but also make it difficult to meet the needs of digital marketing management. Summary of the invention
[0005] 1. Technical issues to be solved
[0006] In view of the above-mentioned shortcomings of the prior art, the present invention provides an intelligent electricity safety inspection system, which can effectively overcome the defect of the prior art that it is difficult to accurately and efficiently identify safety hazards of electrical equipment.
[0007] (II) Technical solution
[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:
[0009] An intelligent electricity safety inspection system, including a multimodal knowledge base establishment and retrieval module, an intelligent terminal multimodal data acquisition and processing module, a multimodal intelligent diagnosis and decision support module, and an electricity customer electricity safety inspection service module;
[0010] Multimodal knowledge base establishment and retrieval module, which builds and maintains a comprehensive multimodal knowledge base for power customer electricity safety inspection services, including various knowledge related to electricity safety inspection services, and provides intelligent retrieval and dialogue functions to support on-site service personnel to quickly obtain relevant knowledge;
[0011] Intelligent terminal multimodal data collection and processing module, which collects multimodal data through intelligent terminals and is responsible for developing information fusion and recognition algorithms to achieve real-time monitoring and analysis of equipment operating status and environment, and provide data support for risk prediction and safety management;
[0012] The multimodal intelligent diagnosis and decision support module uses intelligent cognition and knowledge reasoning technology to identify and evaluate potential safety hazards in electricity use based on the knowledge provided by the multimodal knowledge base establishment and retrieval module and the data provided by the multimodal data acquisition and processing module of the intelligent terminal.
[0013] The power customer electricity safety inspection service module integrates the output results of the above modules to build a system for efficient data processing and analysis. It also provides panoramic display and real-time monitoring functions, so that the power supply service or power emergency command center can fully understand the on-site situation and achieve rapid response and accurate decision-making.
[0014] Preferably, the multimodal knowledge base establishment and retrieval module constructs and maintains a comprehensive multimodal knowledge base for power customer electricity safety inspection services, including various knowledge related to electricity safety inspection services, and provides intelligent retrieval and dialogue functions to support on-site service personnel to quickly obtain relevant knowledge, including:
[0015] Based on the typical electricity safety inspection requirements for key electrical equipment, relevant knowledge is sorted out, a comprehensive multimodal knowledge base of electricity safety inspection services for power customers is built and maintained, and a natural language-based knowledge retrieval function is provided.
[0016] Preferably, the typical power safety inspection requirements for key power equipment are sorted out, and a comprehensive multimodal knowledge base of power customer power safety inspection services is constructed and maintained, and a knowledge retrieval function based on natural language is provided, including:
[0017] S1. Convert documents of various formats into processable text data through document loader and OCR text recognition;
[0018] S2. Use text preprocessing technology to clean and normalize text data;
[0019] S3, dividing the preprocessed text data into multiple text blocks, and converting the text blocks into vector models using an embedding model;
[0020] S4. construct a vector index and store the vector model to obtain a multimodal knowledge base of power safety inspection service for power customers;
[0021] S5. When a user queries, the voice input is converted into a vector model, and the vector index is used to search and match with the vector model in the multimodal knowledge base of the power customer electricity safety inspection service to find the target document;
[0022] S6. Combine the target document and the user's voice input and use a large language model to generate accurate and rich response content;
[0023] Among them, the embedding models include BERT and Word2Vec, and FAISS and Annoy technologies are used to store and retrieve vector models in the multimodal knowledge base of the power customer electricity safety inspection service. The large language model includes ChatGLM-6B.
[0024] Preferably, the intelligent terminal multimodal data collection and processing module collects multimodal data through the intelligent terminal, and is responsible for developing information fusion and recognition algorithms to achieve real-time monitoring and analysis of the equipment operation status and environment, and provide data support for risk prediction and safety management, including:
[0025] Smart terminals are used to collect multimodal data including on-site operation behavior, power safety-related operating status data, distance perception, and helmet wearing. Smart terminals transmit data through integrated sensors, modules, and communication protocols.
[0026] Build a defect detection model for power scenarios to perform defect detection on multimodal data collected by smart terminals;
[0027] Among them, smart terminals include smart helmets and smart phones. The communication protocol specifies the composition of the data packet in detail, including length, information ID, valid data, packet checksum and command tail, to ensure accurate transmission and parsing of data.
[0028] Preferably, the defect detection model is constructed based on a YOLOX network, and the YOLOX network includes an input layer Input, a backbone network Backbone, a neck network Neck, a head network Head and a prediction layer Prediction;
[0029] Input layer, receiving image data;
[0030] Backbone network uses the Focus structure and CSPDarknet network to extract feature information from image data. The Focus structure reduces the amount of calculation by sampling alternate columns and splicing channels, while improving the detection accuracy of small targets. The CSPDarknet network further enhances the semantic understanding ability of features through CBS, CSP and SPP modules;
[0031] Neck network Neck, which fuses multi-scale features through FPN structure and PAN structure, enhances the network's detection ability for targets of different sizes;
[0032] The head network adopts a decoupled design to optimize category and boundary prediction, shortening training time while improving average accuracy.
[0033] The prediction layer outputs the detection results, including bounding boxes, categories, and confidence levels.
[0034] Among them, in view of the diversity and complexity of targets in power scenarios, the YOLOX network introduced the CSEA module to optimize feature extraction and suppress the propagation of invalid information. In addition, the GMP module combines the advantages of maximum pooling and convolution operations to reduce the number of parameters and improve the reasoning speed. In the design of the loss function, the YOLOX network uses IOU-Based Focal Loss to solve the problem of mismatch between classification and positioning scores, and uses Focal Loss to handle sample imbalance.
[0035] Preferably, the multimodal intelligent diagnosis and decision support module uses intelligent cognition and knowledge reasoning technology to identify and evaluate potential safety hazards of electricity use based on the knowledge provided by the multimodal knowledge base establishment and retrieval module and the data provided by the intelligent terminal multimodal data acquisition and processing module, including:
[0036] By building a database of electrical equipment characteristics that includes historical information, safety regulations, and fault characteristics, combined with intelligent recognition technology, and using instrument reading models to identify and evaluate instrument readings, the safe operation of the power system can be ensured.
[0037] Preferably, the instrument reading model includes a space transformer network STN and a text recognition module;
[0038] The spatial transformer network STN uses the MobileNetV3 network as the backbone network, generates an affine transformation matrix through a fully connected layer, and introduces a coordinate attention mechanism to replace the SE module in the MobileNetV3 network to enhance the learning ability of spatial feature information. The network outputs the affine transformation parameters and calculates the transformed output feature map through the Sampler.
[0039] The text recognition module uses the MobileNetV3 network to extract features and adds the SE module to it to enhance the feature representation capability. Then, the feature map is fed into the self-attention mechanism after pooling and fully connected network, and the weight parameters are dynamically generated to process the input sequence of indefinite length. Finally, the module uses the CTC algorithm as the loss function and optimizes the network parameters through back propagation to realize the recognition of the instrument panel text line image.
[0040] Among them, the coordinate attention mechanism encodes the coordinate position along the horizontal and vertical directions through the average pooling kernel, and then calculates the importance between channels through linear transformation and sigmoid function to accurately obtain the dependency of position information and inter-channel relationships.
[0041] Preferably, the power customer power safety inspection service module integrates the output results of the above modules to build a system to efficiently process and analyze data, and at the same time provides panoramic display and real-time monitoring functions, so that the power supply service or power emergency command center can fully understand the on-site situation and achieve rapid response and accurate decision-making, including:
[0042] Establish an electricity safety inspection system to identify the multimodal data collected by smart terminals and reflect the appearance and internal status of on-site equipment.
[0043] Preferably, the electricity safety inspection system includes a device management module, a work order management module, a video call module and a voice assistant module;
[0044] In the equipment management module, administrators can deeply review the detailed information of each equipment, including basic information, historical work orders and fault records. Administrators can access and browse all historical work orders that the equipment has participated in, and directly jump to the work order details interface. The work order details interface provides a comprehensive overview of equipment use, including dynamic playback of equipment tracks and detailed alarm logs. The dynamic playback of equipment tracks uses the longitude and latitude and timestamp data in the database and the intelligent map technology to display the movement path of the equipment in an intuitive simulation form, providing a powerful visualization tool for tracking and analyzing equipment usage.
[0045] The work order management module allows users to easily reply to work orders. When users reply as executors, their feedback will directly affect the change of the work order status. Specifically, when users reply, the module will automatically update the work order status according to the content of the reply, including five work order statuses: pending confirmation, pending, in progress, completed, and revoked, which clearly indicate the progress and status of the work order. When users choose to process a work order, the module will guide users to a special reply interface, which integrates the device selection function so that users can correctly select and use the corresponding device to complete the task, ensuring the accuracy and efficiency of work order processing, while also improving the transparency and traceability of the entire workflow.
[0046] The video call module is designed through the videoRoom plug-in of the Janus framework. This module allows users to join or exit a dynamic virtual conference room at any time. This virtual conference room operates on a flexible publish / subscribe model. In the publish / subscribe model, each participant can publish his or her own real-time audio and video streams, which are subscribed and viewed by other participants in the room at any time, thereby achieving efficient information sharing. The design of this module supports a variety of application scenarios and greatly enhances the flexibility and real-time nature of communication;
[0047] The voice assistant module is based on the Chinese dialogue system of ChatGLM3-6B and BGE-Large-ZH-V1.5, and is used to assist in electricity safety inspections. It uses a model to answer questions by storing various knowledge and questions about electricity safety inspections in a database format. When answering user questions, it quotes reference content in the database to help users solve various professional problems in electricity safety inspections more intelligently.
[0048] (III) Beneficial effects
[0049] Compared with the prior art, the intelligent electricity safety inspection system provided by the present invention has the following beneficial effects:
[0050] 1) Make full use of multimodal data collected from a variety of smart terminals including smart helmets and smart phones to detect defects in electrical equipment, and better reveal the correlation between data;
[0051] 2) Due to factors such as weather and camera position, the dashboard images of instruments may have different brightness, tilt angles, and clarity. For these situations, an instrument reading model is proposed to accurately read the instruments.
[0052] 3) The innovative video conferencing function is introduced in the power safety inspection system, allowing multiple parties involved in the power safety inspection to hold consultations, effectively improving the efficiency of power safety inspections;
[0053] 4) Use large language models for dialogue generation, so as to provide better services when responding to consulting needs in power scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0055] Figure 1 It is a system schematic diagram of the present invention;
[0056] Figure 2 It is a schematic diagram of the construction and retrieval process of the multimodal knowledge base of the power customer electricity safety inspection service in the present invention. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0058] An intelligent power safety inspection system, such as Figure 1 As shown, it includes a multimodal knowledge base establishment and retrieval module, a smart terminal multimodal data acquisition and processing module, a multimodal intelligent diagnosis and decision support module, and a power customer electricity safety inspection service module;
[0059] Multimodal knowledge base establishment and retrieval module, which builds and maintains a comprehensive multimodal knowledge base for power customer electricity safety inspection services, including various knowledge related to electricity safety inspection services, and provides intelligent retrieval and dialogue functions to support on-site service personnel to quickly obtain relevant knowledge;
[0060] Intelligent terminal multimodal data collection and processing module, which collects multimodal data through intelligent terminals and is responsible for developing information fusion and recognition algorithms to achieve real-time monitoring and analysis of equipment operating status and environment, and provide data support for risk prediction and safety management;
[0061] The multimodal intelligent diagnosis and decision support module uses intelligent cognition and knowledge reasoning technology to identify and evaluate potential safety hazards in electricity use based on the knowledge provided by the multimodal knowledge base establishment and retrieval module and the data provided by the multimodal data acquisition and processing module of the intelligent terminal.
[0062] The power customer electricity safety inspection service module integrates the output results of the above modules to build a system for efficient data processing and analysis. It also provides panoramic display and real-time monitoring functions, so that the power supply service or power emergency command center can fully understand the on-site situation and achieve rapid response and accurate decision-making.
[0063] ① Multimodal knowledge base establishment and retrieval module builds and maintains a comprehensive multimodal knowledge base for power customer electricity safety inspection services, including various knowledge related to electricity safety inspection services, and provides intelligent retrieval and dialogue functions to support on-site service personnel to quickly obtain relevant knowledge, including:
[0064] Based on the typical electricity safety inspection requirements for key electrical equipment, relevant knowledge is sorted out, a comprehensive multimodal knowledge base of electricity safety inspection services for power customers is built and maintained, and a natural language-based knowledge retrieval function is provided.
[0065] Specifically, we organize relevant knowledge for typical power safety inspection requirements of key power equipment, build and maintain a comprehensive multimodal knowledge base of power customer power safety inspection services, and provide natural language-based knowledge retrieval functions, such as Figure 2 As shown, including:
[0066] S1. Convert documents of various formats into processable text data through document loader and OCR text recognition;
[0067] S2. Use text preprocessing technology to clean and normalize text data;
[0068] S3, dividing the preprocessed text data into multiple text blocks, and converting the text blocks into vector models using an embedding model;
[0069] S4. construct a vector index and store the vector model to obtain a multimodal knowledge base of power safety inspection service for power customers;
[0070] S5. When a user queries, the voice input is converted into a vector model, and the vector index is used to search and match with the vector model in the multimodal knowledge base of the power customer electricity safety inspection service to find the target document;
[0071] S6. Combine the target document and the user's voice input and use a large language model to generate accurate and rich response content;
[0072] Among them, the embedding models include BERT and Word2Vec, and FAISS and Annoy technologies are used to store and retrieve vector models in the multimodal knowledge base of the power customer electricity safety inspection service. The large language model includes ChatGLM-6B.
[0073] ② The intelligent terminal multimodal data acquisition and processing module collects multimodal data through intelligent terminals and is responsible for developing information fusion and recognition algorithms to achieve real-time monitoring and analysis of equipment operation status and environment, and provide data support for risk prediction and safety management, including:
[0074] Smart terminals are used to collect multimodal data including on-site operation behavior, power safety-related operating status data, distance perception, and helmet wearing. Smart terminals transmit data through integrated sensors, modules, and communication protocols.
[0075] Build a defect detection model for power scenarios to perform defect detection on multimodal data collected by smart terminals;
[0076] Among them, smart terminals include smart helmets and smart phones. The communication protocol specifies the composition of the data packet in detail, including length, information ID, valid data, packet checksum and command tail, to ensure accurate transmission and parsing of data.
[0077] In the technical solution of this application, the defect detection model is built based on the YOLOX network, which includes an input layer Input, a backbone network Backbone, a neck network Neck, a head network Head and a prediction layer Prediction;
[0078] Input layer, receiving image data;
[0079] Backbone network uses the Focus structure and CSPDarknet network to extract feature information from image data. The Focus structure reduces the amount of calculation by sampling alternate columns and splicing channels, while improving the detection accuracy of small targets. The CSPDarknet network further enhances the semantic understanding ability of features through CBS, CSP and SPP modules;
[0080] Neck network Neck, which fuses multi-scale features through FPN structure and PAN structure, enhances the network's detection ability for targets of different sizes;
[0081] The head network adopts a decoupled design to optimize category and boundary prediction, shortening training time while improving average accuracy.
[0082] The prediction layer outputs the detection results, including bounding boxes, categories, and confidence levels.
[0083] Among them, in view of the diversity and complexity of targets in power scenarios, the YOLOX network introduced the CSEA module to optimize feature extraction and suppress the propagation of invalid information. In addition, the GMP module combines the advantages of maximum pooling and convolution operations to reduce the number of parameters and improve the reasoning speed. In the design of the loss function, the YOLOX network uses IOU-Based Focal Loss to solve the problem of mismatch between classification and positioning scores, and uses Focal Loss to handle sample imbalance.
[0084] ③ The multimodal intelligent diagnosis and decision support module uses the knowledge provided by the multimodal knowledge base establishment and retrieval module and the data provided by the multimodal data acquisition and processing module of the intelligent terminal to identify and evaluate the hidden dangers of power safety by using intelligent cognition and knowledge reasoning technology, including:
[0085] By building a database of electrical equipment characteristics that includes historical information, safety regulations, and fault characteristics, combined with intelligent recognition technology, and using instrument reading models to identify and evaluate instrument readings, the safe operation of the power system can be ensured.
[0086] In the technical solution of the present application, the instrument reading model includes a space transformation network STN and a text recognition module;
[0087] The spatial transformer network STN uses the MobileNetV3 network as the backbone network, generates an affine transformation matrix through a fully connected layer, and introduces a coordinate attention mechanism to replace the SE module in the MobileNetV3 network to enhance the learning ability of spatial feature information. The network outputs the affine transformation parameters and calculates the transformed output feature map through the Sampler.
[0088] The text recognition module uses the MobileNetV3 network to extract features and adds the SE module to it to enhance the feature representation capability. Then, the feature map is fed into the self-attention mechanism after pooling and fully connected network, and the weight parameters are dynamically generated to process the input sequence of indefinite length. Finally, the module uses the CTC algorithm as the loss function and optimizes the network parameters through back propagation to realize the recognition of the instrument panel text line image.
[0089] Among them, the coordinate attention mechanism encodes the coordinate position along the horizontal and vertical directions through the average pooling kernel, and then calculates the importance between channels through linear transformation and sigmoid function to accurately obtain the dependency of position information and inter-channel relationships.
[0090] ④ The power customer power safety inspection service module integrates the output results of the above modules to build a system to efficiently process and analyze the data. At the same time, it provides panoramic display and real-time monitoring functions, so that the power supply service or power emergency command center can fully understand the on-site situation and achieve rapid response and accurate decision-making, including:
[0091] Establish an electricity safety inspection system to identify the multimodal data collected by smart terminals and reflect the appearance and internal status of on-site equipment.
[0092] In the technical solution of this application, the electricity safety inspection system includes an equipment management module, a work order management module, a video call module and a voice assistant module;
[0093] In the equipment management module, administrators can deeply review the detailed information of each equipment, including basic information, historical work orders and fault records. Administrators can access and browse all historical work orders that the equipment has participated in, and directly jump to the work order details interface. The work order details interface provides a comprehensive overview of equipment use, including dynamic playback of equipment tracks and detailed alarm logs. The dynamic playback of equipment tracks uses the longitude and latitude and timestamp data in the database and the intelligent map technology to display the movement path of the equipment in an intuitive simulation form, providing a powerful visualization tool for tracking and analyzing equipment usage.
[0094] The work order management module allows users to easily reply to work orders. When users reply as executors, their feedback will directly affect the change of the work order status. Specifically, when users reply, the module will automatically update the work order status according to the content of the reply, including five work order statuses: pending confirmation, pending, in progress, completed, and revoked, which clearly indicate the progress and status of the work order. When users choose to process a work order, the module will guide users to a special reply interface, which integrates the device selection function so that users can correctly select and use the corresponding device to complete the task, ensuring the accuracy and efficiency of work order processing, while also improving the transparency and traceability of the entire workflow.
[0095] The video call module is designed through the videoRoom plug-in of the Janus framework. This module allows users to join or exit a dynamic virtual conference room at any time. This virtual conference room operates on a flexible publish / subscribe model. In the publish / subscribe model, each participant can publish their own real-time audio and video streams, which are subscribed and viewed by other participants in the room at any time, thereby achieving efficient information sharing. The design of this module supports a variety of application scenarios and greatly enhances the flexibility and real-time nature of communication. For example, inspectors can use this tool to communicate seamlessly with team members or other experts no matter where they are. This instant remote collaboration capability not only improves work efficiency, but also provides strong support for on-site decision-making.
[0096] The voice assistant module is based on the Chinese dialogue system of ChatGLM3-6B and BGE-Large-ZH-V1.5, and is used to assist in electricity safety inspections. It uses a model to answer questions by storing various knowledge and questions about electricity safety inspections in a database format. When answering user questions, it quotes reference content in the database to help users solve various professional problems in electricity safety inspections more intelligently.
[0097] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An intelligent electricity safety inspection system, characterized in that: It includes a multimodal knowledge base establishment and retrieval module, a smart terminal multimodal data acquisition and processing module, a multimodal intelligent diagnosis and decision support module, and a power customer electricity safety inspection service module; Multimodal knowledge base establishment and retrieval module, which builds and maintains a comprehensive multimodal knowledge base for power customer electricity safety inspection services, including various knowledge related to electricity safety inspection services, and provides intelligent retrieval and dialogue functions to support on-site service personnel to quickly obtain relevant knowledge; Intelligent terminal multimodal data collection and processing module, which collects multimodal data through intelligent terminals and is responsible for developing information fusion and recognition algorithms to achieve real-time monitoring and analysis of equipment operating status and environment, and provide data support for risk prediction and safety management; The multimodal intelligent diagnosis and decision support module uses intelligent cognition and knowledge reasoning technology to identify and evaluate potential safety hazards in electricity use based on the knowledge provided by the multimodal knowledge base establishment and retrieval module and the data provided by the multimodal data acquisition and processing module of the intelligent terminal. The power customer electricity safety inspection service module integrates the output results of the above modules to build a system for efficient data processing and analysis. It also provides panoramic display and real-time monitoring functions, so that the power supply service or power emergency command center can fully understand the on-site situation and achieve rapid response and accurate decision-making.
2. The intelligent electricity safety inspection system according to claim 1 is characterized in that: The multimodal knowledge base establishment and retrieval module builds and maintains a comprehensive multimodal knowledge base for power customer electricity safety inspection services, including various knowledge related to electricity safety inspection services, and provides intelligent retrieval and dialogue functions to support on-site service personnel to quickly obtain relevant knowledge, including: Based on the typical electricity safety inspection requirements for key electrical equipment, relevant knowledge is sorted out, a comprehensive multimodal knowledge base of electricity safety inspection services for power customers is built and maintained, and a natural language-based knowledge retrieval function is provided.
3. The intelligent electricity safety inspection system according to claim 2 is characterized in that: The typical power safety inspection requirements for key power equipment are sorted out, and a comprehensive multimodal knowledge base of power customer power safety inspection services is constructed and maintained, and a knowledge retrieval function based on natural language is provided, including: S1. Convert documents of various formats into processable text data through document loader and OCR text recognition; S2. Use text preprocessing technology to clean and normalize text data; S3, dividing the preprocessed text data into multiple text blocks, and converting the text blocks into vector models using an embedding model; S4. construct a vector index and store the vector model to obtain a multimodal knowledge base of power safety inspection service for power customers; S5. When a user queries, the voice input is converted into a vector model, and the vector index is used to search and match with the vector model in the multimodal knowledge base of the power customer electricity safety inspection service to find the target document; S6. Combine the target document and the user's voice input and use a large language model to generate accurate and rich response content; Among them, the embedding models include BERT and Word2Vec, and FAISS and Annoy technologies are used to store and retrieve vector models in the multimodal knowledge base of the power customer electricity safety inspection service. The large language model includes ChatGLM-6B.
4. The intelligent electricity safety inspection system according to claim 1 is characterized in that: The intelligent terminal multimodal data acquisition and processing module collects multimodal data through intelligent terminals and is responsible for developing information fusion and recognition algorithms to achieve real-time monitoring and analysis of equipment operation status and environment, and provide data support for risk prediction and safety management, including: Smart terminals are used to collect multimodal data including on-site operation behavior, power safety-related operating status data, distance perception, and helmet wearing. Smart terminals transmit data through integrated sensors, modules, and communication protocols. Build a defect detection model for power scenarios to perform defect detection on multimodal data collected by smart terminals; Among them, smart terminals include smart helmets and smart phones. The communication protocol specifies the composition of the data packet in detail, including length, information ID, valid data, packet checksum and command tail, to ensure accurate transmission and parsing of data.
5. The intelligent electricity safety inspection system according to claim 4 is characterized in that: The defect detection model is built based on the YOLOX network, which includes an input layer Input, a backbone network Backbone, a neck network Neck, a head network Head and a prediction layer Prediction; Input layer, receiving image data; Backbone network uses the Focus structure and CSPDarknet network to extract feature information from image data. The Focus structure reduces the amount of calculation by sampling alternate columns and splicing channels, while improving the detection accuracy of small targets. The CSPDarknet network further enhances the semantic understanding ability of features through CBS, CSP and SPP modules; Neck network Neck, which fuses multi-scale features through FPN structure and PAN structure, enhances the network's detection ability for targets of different sizes; The head network adopts a decoupled design to optimize category and boundary prediction, shortening training time while improving average accuracy. The prediction layer outputs the detection results, including bounding boxes, categories, and confidence levels. Among them, in view of the diversity and complexity of targets in power scenarios, the YOLOX network introduced the CSEA module to optimize feature extraction and suppress the propagation of invalid information. In addition, the GMP module combines the advantages of maximum pooling and convolution operations to reduce the number of parameters and improve the reasoning speed. In the design of the loss function, the YOLOX network uses IOU-BasedFocalLoss to solve the problem of mismatch between classification and positioning scores, and uses FocalLoss to handle sample imbalance problems.
6. The intelligent electricity safety inspection system according to claim 1 is characterized in that: The multimodal intelligent diagnosis and decision support module uses intelligent cognition and knowledge reasoning technology to identify and evaluate potential safety hazards in electricity use based on the knowledge provided by the multimodal knowledge base establishment and retrieval module and the data provided by the multimodal data acquisition and processing module of the intelligent terminal, including: By building a database of electrical equipment characteristics that includes historical information, safety regulations, and fault characteristics, combined with intelligent recognition technology, and using instrument reading models to identify and evaluate instrument readings, the safe operation of the power system can be ensured.
7. The intelligent electricity safety inspection system according to claim 6 is characterized in that: The instrument reading model includes a space transformation network STN and a text recognition module; The spatial transformer network STN uses the MobileNetV3 network as the backbone network, generates an affine transformation matrix through a fully connected layer, and introduces a coordinate attention mechanism to replace the SE module in the MobileNetV3 network to enhance the learning ability of spatial feature information. The network outputs the affine transformation parameters and calculates the transformed output feature map through the Sampler. The text recognition module uses the MobileNetV3 network to extract features and adds the SE module to it to enhance the feature representation capability; Then, the feature map is fed into the self-attention mechanism after pooling and fully connected network, and the weight parameters are dynamically generated to process the input sequence of indefinite length. Finally, the module uses the CTC algorithm as the loss function and optimizes the network parameters through back propagation to realize the recognition of the instrument panel text line image. Among them, the coordinate attention mechanism encodes the coordinate position along the horizontal and vertical directions through the average pooling kernel, and then calculates the importance between channels through linear transformation and sigmoid function to accurately obtain the dependency of position information and inter-channel relationships.
8. The intelligent electricity safety inspection system according to claim 1 is characterized in that: The power customer power safety inspection service module integrates the output results of the above modules to build a system to efficiently process and analyze data, and at the same time provides panoramic display and real-time monitoring functions, so that the power supply service or power emergency command center can fully understand the on-site situation and achieve rapid response and accurate decision-making, including: Establish an electricity safety inspection system to identify the multimodal data collected by smart terminals and reflect the appearance and internal status of on-site equipment.
9. The intelligent electricity safety inspection system according to claim 8, characterized in that: The electricity safety inspection system includes an equipment management module, a work order management module, a video call module and a voice assistant module; In the equipment management module, administrators can deeply review the detailed information of each equipment, including basic information, historical work orders and fault records. Administrators can access and browse all historical work orders that the equipment has participated in, and directly jump to the work order details interface. The work order details interface provides a comprehensive overview of equipment use, including dynamic playback of equipment tracks and detailed alarm logs. The dynamic playback of equipment tracks uses the longitude and latitude and timestamp data in the database and the intelligent map technology to display the movement path of the equipment in an intuitive simulation form, providing a powerful visualization tool for tracking and analyzing equipment usage. The work order management module allows users to easily reply to work orders. When users reply as executors, their feedback will directly affect the change of the work order status. Specifically, when users reply, the module will automatically update the work order status according to the content of the reply, including five work order statuses: pending confirmation, pending, in progress, completed, and revoked, which clearly indicate the progress and status of the work order. When users choose to process a work order, the module will guide users to a special reply interface, which integrates the device selection function so that users can correctly select and use the corresponding device to complete the task, ensuring the accuracy and efficiency of work order processing, while also improving the transparency and traceability of the entire workflow. The video call module is designed through the videoRoom plug-in of the Janus framework. This module allows users to join or exit a dynamic virtual conference room at any time. This virtual conference room operates on a flexible publish / subscribe model. In the publish / subscribe model, each participant can publish his or her own real-time audio and video streams, which are subscribed and viewed by other participants in the room at any time, thereby achieving efficient information sharing. The design of this module supports a variety of application scenarios and greatly enhances the flexibility and real-time nature of communication; The voice assistant module is based on the Chinese dialogue system of ChatGLM3-6B and BGE-Large-ZH-V1.5, and is used to assist in electricity safety inspections. It uses a model to answer questions by storing various knowledge and questions about electricity safety inspections in a database format. When answering user questions, it quotes reference content in the database to help users solve various professional problems in electricity safety inspections more intelligently.
Citation Information
Patent Citations
Intelligent multi-mode fusion electricity utilization inspection method, device, system, equipment and medium
CN115114448A
Power utilization safety inspection system based on mobile terminal
CN117688503A
Cited By
Power generation equipment maintenance knowledge question-answering system and method based on multi-modal large model
CN119671532A
