A method and system for identifying electrical safety hazards

Abstract

The present application provides a method and system for identifying public safety hazards related to electricity. After obtaining a target monitoring image to be used for safety hazard identification, a hidden danger image library query operation is performed using the target monitoring image as a query image, and a hidden danger related image set associated with the target monitoring image, including at least one hidden danger related image, is determined based on the known hidden danger images included in the hidden danger image library query result; based on the extracted image representation vector of the target monitoring image and the image representation vectors of each hidden danger related image, the image safety hazard category of the target monitoring image is inferred to obtain the inferred image safety hazard category of the target monitoring image. When identifying safety hazards of the target monitoring image, the features of the hidden danger related images associated with the target monitoring image are integrated into the features of the target monitoring image, and feature information not in the target monitoring image is added based on the hidden danger related image, so that the accuracy of safety hazard identification is higher.

Description

Technical Field

[0001] The present application relates to, but is not limited to, the field of data processing technology, and in particular to a method and system for identifying electricity-related public safety hazards. Background Art

[0002] With the rapid development of society and the continuous advancement of science and technology, public safety issues are increasingly attracting people's attention. Especially in the field of electricity, since power equipment is widely used in various fields, the identification and prevention of its safety hazards are particularly important. Traditional methods for identifying public safety hazards in electricity mainly rely on manual inspections and regular testing. However, these methods are inefficient and easily affected by human factors, and it is difficult to meet the efficient and accurate requirements of modern society for safety management. In recent years, with the rapid development of image processing technology and machine learning technology, image-based safety hazard identification methods have gradually become a research hotspot. This type of method usually uses a camera to capture real-time images of power equipment and identifies potential safety hazards through image processing and analysis technology. However, when dealing with complex backgrounds and changing lighting conditions, the existing image-based safety hazard identification methods often have low recognition accuracy and are easily affected by noise and interference. Summary of the invention

[0003] In view of this, the embodiment of the present application at least provides a method and system for identifying electrical public safety hazards. The technical solution of the embodiment of the present application is implemented as follows:

[0004] On the one hand, an embodiment of the present application provides a method for identifying electricity-related public safety hazards, which is applied to an identification system. The method includes:

[0005] Acquire a target monitoring image for potential safety hazards identification; the target monitoring image is an image taken of a target electrical public area;

[0006] Performing a hidden danger image library query operation with the target monitoring image as a query image to obtain a hidden danger image library query result, and determining a hidden danger related image set associated with the target monitoring image based on known safety hidden danger images included in the hidden danger image library query result; the hidden danger related image set includes at least one hidden danger related image obtained based on each known safety hidden danger image;

[0007] Respectively extracting the image representation vector of the target monitoring image and the image representation vector of each hidden danger related image;

[0008] Based on the image representation vector of the target monitoring image and the image representation vectors of each hazard-related image, the image safety hazard category of the target monitoring image is inferred to obtain the inferred image safety hazard category of the target monitoring image.

[0009] In some embodiments, the image representation vector of the target monitoring image includes the embedded representation vectors of each monitoring image block in the target monitoring image, and the image representation vector of the target hidden danger related image in the hidden danger related image set includes the embedded representation vector of each related image block in the target hidden danger related image; the image safety hidden danger category of the target monitoring image is inferred based on the image representation vector of the target monitoring image and the image representation vectors of each hidden danger related image to obtain the inferred image safety hidden danger category of the target monitoring image, including:

[0010] Performing a representation vector adjustment on the image representation vector of the target monitoring image based on the image representation vectors of each hidden danger related image to obtain an inference representation vector of the target monitoring image;

[0011] The inference representation vector of the target monitoring image is obtained based on the image block inference representation vectors of each monitoring image block in the target monitoring image;

[0012] Based on the inference representation vector of the target monitoring image, inferring the inference image safety hazard category of the target monitoring image from a plurality of pre-determined image safety hazard categories;

[0013] Among them, the image block inference representation vector of the target monitoring image block in the target monitoring image is obtained based on the representation information commonality measurement result between the target monitoring image block and the relevant image blocks in each hidden danger-related image, and the representation information commonality measurement result between the target monitoring image block and any relevant image block is obtained based on the embedded representation vector of the target monitoring image block and the embedded representation vector of any relevant image block.

[0014] In some embodiments, the step of performing a representation vector adjustment on the image representation vector of the target monitoring image based on the image representation vectors of each hidden danger related image to obtain an inference representation vector of the target monitoring image includes:

[0015] Determine a commonality measurement result of representation information between the target monitoring image block and each relevant image block of the target hidden danger related image in the hidden danger related image set based on the embedded representation vector of the target monitoring image block in the target monitoring image and based on the embedded representation vector of each relevant image block of the target hidden danger related image in the hidden danger related image set;

[0016] Determining an enhanced representation vector of the target monitoring image block compared to the image block of the target hidden danger related image based on a commonality measurement result of the representation information between the target monitoring image block and each related image block of the target hidden danger related image;

[0017] Obtaining an image block enhancement representation vector of the target monitoring image block compared to each other hidden danger related image in the hidden danger related image set;

[0018] Performing a representation vector integration operation on the embedded representation vector of the target monitoring image block and the image block enhancement representation vector of each hidden danger related image in the hidden danger related image set compared with the target monitoring image block, to obtain an image block reasoning representation vector of the target monitoring image block;

[0019] Based on the image block inference representation vector of the target monitoring image block, an inference representation vector of the target monitoring image is obtained.

[0020] In some embodiments, the determining, based on the commonality measurement result of the representation information between the target monitoring image block and each relevant image block of the target hidden danger related image, an enhanced representation vector of the target monitoring image block compared with the image block of the target hidden danger related image includes:

[0021] Standardizing the commonality measurement results of the representation information between the target monitoring image block and each relevant image block of the target hidden danger related image, respectively, to obtain the commonality influence coefficient of each relevant image block of the target hidden danger related image relative to the target monitoring image block;

[0022] Based on the common influence coefficients of each relevant image block of the target hidden danger related image relative to the target monitoring image block, the embedded representation vectors of each relevant image block of the target hidden danger related image are adjusted and integrated to obtain an enhanced representation vector of the target monitoring image block compared to the image block of the target hidden danger related image.

[0023] In some embodiments, the embedding representation vector of the target monitoring image block is integrated with the image block enhancement representation vector of each hidden danger related image in the hidden danger related image set compared with the target monitoring image block to obtain the image block reasoning representation vector of the target monitoring image block, including:

[0024] Performing a representation vector combination operation on the target monitoring image block compared with the image block enhancement representation vectors of each hidden danger related image in the hidden danger related image set to obtain a combined enhancement representation vector corresponding to the target monitoring image block;

[0025] Based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block, an image block inference representation vector of the target monitoring image block is determined.

[0026] In some embodiments, determining the image block inference representation vector of the target monitoring image block based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block includes:

[0027] A representation vector combination operation is performed on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block to obtain an image block inference representation vector of the target monitoring image block.

[0028] In some embodiments, determining the image block inference representation vector of the target monitoring image block based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block includes:

[0029] Performing a vector mapping operation on the combined enhanced representation vector corresponding to the target monitoring image block based on a preset mapping parameter variable to obtain a mapping enhanced representation vector corresponding to the target monitoring image block;

[0030] The number of vector elements of the mapping enhancement representation vector corresponding to the target monitoring image block is the same as the number of vector elements of the embedding representation vector of the target monitoring image block;

[0031] The embedded representation vector of the target monitoring image block, the mapping enhancement representation vector corresponding to the target monitoring image block, and the error representation vector between the embedded representation vector of the target monitoring image block and the mapping enhancement representation vector corresponding to the target monitoring image block are subjected to a representation vector combination operation to obtain an image block inference representation vector of the target monitoring image block.

[0032] In some embodiments, the inference characterization vector of the target surveillance image is used to infer the inference image safety hazard category of the target surveillance image from a plurality of pre-determined image safety hazard categories, including:

[0033] Performing confidence reasoning on the inference representation vector of the target monitoring image to obtain the inference confidence that the target monitoring image is inferred to be each of the plurality of predetermined image safety hazard categories;

[0034] Among the multiple predetermined image safety hazard categories, a predetermined image safety hazard category that meets the confidence value requirement is determined as the inferred image safety hazard category of the target monitoring image.

[0035] In some embodiments, the inferred image safety hazard category of the target monitoring image is obtained by inference based on a safety hazard identification neural network; the safety hazard identification neural network is calibrated by the following steps:

[0036] Acquire a learning sample of an electrical surveillance image, a hidden danger-related image learning sample set associated with the learning sample of the electrical surveillance image, and a priori labels of image safety hidden danger categories;

[0037] The hidden danger related image learning sample set includes at least one hidden danger related image learning sample, and the image safety hidden danger category priori mark is used to indicate the image safety hidden danger category of the electrical monitoring image learning sample;

[0038] The image representation vectors of the electrical monitoring image learning samples and the image representation vectors of the image learning samples related to each hazard are respectively extracted through the representation information extraction layer in the safety hazard identification neural network to be adjusted;

[0039] Through the representation information adjustment layer in the safety hazard identification neural network to be adjusted, the image representation vector of the electrical monitoring image learning sample is adjusted based on the image representation vector of each hazard-related image learning sample to obtain an inference representation vector of the electrical monitoring image learning sample;

[0040] By using the image safety hazard category inference layer in the safety hazard identification neural network to be calibrated, confidence inference is performed on the inference representation vector of the electrical monitoring image learning sample to obtain the inference confidence that the electrical monitoring image learning sample is inferred as each of the multiple predetermined image safety hazard categories;

[0041] Based on the error between the reasoning confidence corresponding to the learning sample of the electrical monitoring image and the prior label of the image safety hazard category, the parameters of the safety hazard identification neural network to be calibrated are optimized to obtain the safety hazard identification neural network.

[0042] In some embodiments, the performing confidence reasoning on the inference representation vector of the target monitoring image to obtain the inference confidence that the target monitoring image is inferred as each of the plurality of predetermined image safety hazard categories includes:

[0043] Performing a linear transformation on the inference representation vector of the target monitoring image based on the representation information mapping parameter to obtain a linear transformation result;

[0044] The obtained linear transformation result is used with a preset classifier to obtain the inference confidence that the target monitoring image is inferred to be each of a plurality of predetermined image safety hazard categories; wherein the representation information mapping parameter is a network parameter of the safety hazard identification neural network, the network parameter is obtained when the safety hazard identification neural network to be calibrated, and the classifier is a softmax classifier.

[0045] In a second aspect, the present application provides an identification system, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the steps in the above-described method when executing the program.

[0046] The beneficial effects of this application include at least:

[0047] After acquiring the target monitoring image for which safety hazard identification is to be performed, the present application uses the target monitoring image as the query image to perform a hidden danger image library query operation, and based on the known safety hazard images included in the hidden danger image library query result, determines a hidden danger related image set associated with the target monitoring image, including at least one hidden danger related image; based on the extracted image representation vector of the target monitoring image and the image representation vectors of each hidden danger related image, infers the image safety hazard category of the target monitoring image to obtain the inferred image safety hazard category of the target monitoring image. When identifying safety hazards for the target monitoring image, the features of the hidden danger related images associated with the target monitoring image are integrated into the features of the target monitoring image, and feature information not present in the target monitoring image is added based on the hidden danger related image, so that the accuracy of safety hazard identification is higher.

[0048] After extracting the image representation vector of the target monitoring image and the image representation vectors of each hidden danger related image, the image representation vector of the target monitoring image is adjusted based on the image representation vectors of each hidden danger related image to obtain the inference representation vector of the target monitoring image, and then the inference image safety hidden danger category of the target monitoring image can be inferred from multiple predetermined image safety hidden danger categories based on the inference representation vector of the target monitoring image; wherein, when the image representation vector of the target monitoring image is adjusted based on the image representation vectors of each hidden danger related image to obtain the inference representation vector of the target monitoring image, the image block enhancement representation vector of the target monitoring image block compared with the target hidden danger related image is determined by measuring the commonality of the representation information between the target monitoring image block in the target monitoring image and each related image block of the target hidden danger related image in the hidden danger related image set; the embedded representation vector of the target monitoring image block is integrated with the image block enhancement representation vector of the target monitoring image block compared with each hidden danger related image in the hidden danger related image set to obtain the image block inference representation vector of the target monitoring image block; the inference representation vector of the target monitoring image block is obtained based on the image block inference representation vector of each monitoring image block in the target monitoring image.

[0049] When determining the image block enhancement representation vector of the target monitoring image block compared to the target hidden danger related image, the embedded representation vector of each relevant image block in the target hidden danger related image is integrated according to the attention weight of each relevant image block and the target hidden danger related image to represent the influence of the target hidden danger related image on the target monitoring image block in the target monitoring image. The embedded representation vector of the target monitoring image block is integrated with the image block enhancement representation vector of the target monitoring image block compared to each hidden danger related image in the hidden danger related image set. When the inference image safety hidden danger category of the target monitoring image is obtained, the feature information of the hidden danger related image is added, that is, the influence of each hidden danger related image associated with the target monitoring image on the target monitoring image is added, and the features of the target monitoring image are improved, which can increase the accuracy of safety hidden danger identification.

[0050] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0052] Figure 1A schematic diagram of the implementation flow of a method for identifying electricity-related public safety hazards provided in an embodiment of the present application.

[0053] Figure 2 A hardware entity diagram of an identification system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0055] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0057] The embodiment of the present application provides a method for identifying electrical public safety hazards, which can be executed by a processor of an identification system, wherein the identification system can refer to a device with data processing capabilities such as a server, a laptop, a tablet computer, and a desktop computer.

[0058] Figure 1 A schematic diagram of the implementation process of a method for identifying electrical safety hazards provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0059] Step S10: Acquire a target monitoring image for potential safety hazard identification; the target monitoring image is an image taken of a target electrical-related public area.

[0060] In step S10, the target monitoring images that need to be identified for potential safety hazards specifically refer to images taken of public areas involving electricity, including but not limited to key power infrastructure such as power grid facilities, substations, transmission and distribution lines, and meter rooms. In order to obtain these images, the recognition system is usually connected to surveillance cameras in the target public areas involving electricity, which are strategically installed around power facilities to monitor potential safety risks in real time. For example, a camera may be installed on the wall of a substation to capture the situation inside and outside the wall and detect any unusual activities or potential safety threats in a timely manner.

[0061] In actual applications, when the camera captures video streams, the recognition system will periodically or in real time capture static images from these video streams, which are used as target monitoring images. The recognition system will then perform subsequent analysis and processing on these images to identify safety hazards.

[0062] For example, suppose a camera is installed next to a high-voltage power pole. The camera continuously monitors the status of the pole. One day, due to strong winds, the pole sways slightly. The camera captures this process and transmits the video signal to the recognition system. The recognition system then captures an image from the video stream, which clearly shows the shaking of the pole. This captured image is a target monitoring image, which will be used in the subsequent safety hazard identification process.

[0063] Step S20: Perform a hidden danger image library query operation with the target monitoring image as a query image to obtain a hidden danger image library query result, and determine a hidden danger related image set associated with the target monitoring image based on the known safety hidden danger images included in the hidden danger image library query result; the hidden danger related image set includes at least one hidden danger related image obtained based on each known safety hidden danger image.

[0064] In step S20, the recognition system uses the target monitoring image as a query image and performs a query operation in the hidden danger image library. The hidden danger image library is a database that stores a large number of known safety hidden danger images. These images cover various possible safety hidden danger scenarios, such as equipment failure, abnormal conditions, risks caused by environmental factors, etc. The purpose of the query operation is to find a hidden danger related image set associated with the target monitoring image. This image set contains hidden danger related images obtained based on various known safety hidden danger images, which may be similar or related to the scenes in the target monitoring image.

[0065] Specifically, the recognition system will compare the features of the target monitoring image with the image features in the hidden danger image library to find similar images. For example, the comparison is based on some basic image features, such as color histogram, SIFT feature points, etc., which can include visual information such as color, texture, shape, etc., or deeper features extracted by machine learning models.

[0066] For example, suppose the target surveillance image shows a smoking transformer. The recognition system compares the features of this image with images in the hidden danger image library to find similar images, which may show smoking transformers at different angles and under different lighting conditions, or other types of power equipment failure scenes. Once a similar set of hidden danger-related images is found, the recognition system can perform subsequent safety hazard identification and analysis based on these images. This set of hidden danger-related images not only provides scenes similar to the target surveillance image, but may also contain more contextual information about the safety hazards, thereby helping the recognition system to more accurately determine the category of safety hazards in the target surveillance image.

[0067] In practical applications, the hidden danger image library can be enriched and improved by continuously adding new safety hidden danger images to improve the accuracy and coverage of queries. At the same time, with the continuous development of technology, the recognition system can also adopt more advanced image retrieval and machine learning algorithms to improve query efficiency and accuracy.

[0068] Step S30: extracting the image representation vector of the target monitoring image and the image representation vector of each hidden danger related image respectively.

[0069] In step S30, the task of the recognition system is to extract the image representation vectors of the target monitoring image and each hidden danger related image. The image representation vector is a digital representation of the image, which can capture and summarize the key information and features of the image, and provide a data basis for the subsequent safety hazard identification.

[0070] In actual operation, the recognition system usually uses a pre-trained safety hazard recognition neural network to complete this task. This type of neural network usually includes a convolutional network layer, which is specifically used to extract image features. When the recognition system inputs an image, the convolutional network layer extracts key features such as edges, textures, and shapes in the image layer by layer, and converts these features into a high-dimensional vector, namely the image representation vector.

[0071] Taking the identification of public safety hazards related to electricity as an example, suppose the target monitoring image is a high-voltage power pole. The recognition system will first extract the shape, color, texture, and relative position of the pole to the surrounding environment through a neural network, and encode these features into an image representation vector. Similarly, for each hazard-related image in the hazard image library, the recognition system will also perform the same feature extraction and vector encoding operations.

[0072] Specifically, an image representation vector may be an array containing hundreds or thousands of elements, each of which represents the intensity or presence of a particular feature in the image. For example, one element in the vector may represent the proportion of red pixels in the image, another element may represent the sharpness of the edges in the image, and so on.

[0073] Through this step, the recognition system can transform the original image data into a more abstract and quantitative form, which is convenient for subsequent reasoning and identification of safety hazard categories. This neural network-based feature extraction method not only improves the accuracy of recognition, but also enhances the system's adaptability to different scenes and lighting conditions.

[0074] It should be noted that in step S20, it is mentioned that the features of the target monitoring image are compared with the image features in the hidden danger image library. The feature comparison here is a relatively simple similarity matching process, which is used to retrieve hidden danger related images similar to the target monitoring image from the hidden danger image library. This comparison may be based on some basic image features, such as color histogram, SIFT feature points, etc., and does not necessarily involve deep semantic features. In step S30, the image representation vector mentioned is a more abstract and advanced feature representation extracted by a specific machine learning model (such as the convolutional network layer in the safety hazard identification neural network). These feature representations can capture the deep structure and semantic information of the image, which is crucial for the subsequent safety hazard identification. Therefore, although both steps S20 and S30 involve the extraction and comparison of image features, their purposes and the technologies used are different. The feature comparison in step S20 is mainly to retrieve similar images from the hidden danger image library, while the image representation vector extraction in step S30 is to obtain more accurate and deep image features for the subsequent safety hazard identification.

[0075] Step S40: Based on the image representation vector of the target monitoring image and the image representation vectors of each hazard-related image, the image safety hazard category of the target monitoring image is inferred to obtain the inferred image safety hazard category of the target monitoring image.

[0076] In step S40, the recognition system uses the image representation vectors extracted from the target monitoring image and each hidden danger related image to infer the safety hidden danger category. The core of this step is to compare and analyze these vectors to determine the type of safety hidden danger that may exist in the target monitoring image.

[0077] Specifically, the recognition system uses machine learning algorithms, such as support vector machines (SVM), random forests, or neural networks, to process these image representation vectors. These algorithms are able to learn and recognize the characteristic patterns of different safety hazard categories. Taking neural networks as an example, the recognition system can use a well-trained multi-layer perceptron (MLP) or convolutional neural network (CNN) for classification. These networks have been trained on a large amount of labeled safety hazard image data, so they can predict the category of safety hazards based on the input image representation vector.

[0078] In actual operation, the recognition system will input the image representation vector of the target monitoring image into the neural network, which will process this vector and output the probability distribution of one or more safety hazard categories. For example, if the target monitoring image shows a broken wire, the neural network may output a high-probability "broken wire" category. In addition, the recognition system may also combine the image representation vector of the hazard-related images to enhance the accuracy of reasoning. For example, if the hazard-related images contain multiple scenes of broken wires, the system can compare the representation vector of the target monitoring image with the representation vectors of these related images, thereby more accurately identifying safety hazards.

[0079] Finally, the recognition system determines one or more most likely safety hazard categories and outputs these categories as inference results. This result can be a specific safety hazard name, such as "broken wire", "equipment overheating", etc., or a probability distribution showing the likelihood of different safety hazards.

[0080] In this way, step S40 enables the identification system to automatically and accurately identify potential safety hazards in public areas involving electricity, providing important support for timely prevention and response to potential risks.

[0081] As an implementation method, the image representation vector of the target monitoring image includes the embedded representation vectors (i.e., coded features) of each monitoring image block in the target monitoring image, and the image representation vector of the target hidden danger related image in the hidden danger related image set includes the embedded representation vectors of each related image block in the target hidden danger related image. Based on this, the step S40, based on the image representation vector of the target monitoring image and the image representation vectors of each hidden danger related image, infers the image safety hidden danger category of the target monitoring image to obtain the inferred image safety hidden danger category of the target monitoring image, may specifically include:

[0082] Step S41: Based on the image representation vectors of each hidden danger related image, the image representation vector of the target monitoring image is adjusted to obtain an inference representation vector of the target monitoring image; the inference representation vector of the target monitoring image is obtained based on the image block inference representation vectors of each monitoring image block in the target monitoring image.

[0083] In step S41, the recognition system will make detailed adjustments to the image representation vector of the target monitoring image. This process is called representation vector adjustment, that is, feature adjustment. The core of this step is to use the image representation vector of the hidden danger-related image to enhance the feature representation of the target monitoring image, so as to more accurately identify potential safety hazards. Specifically, the recognition system will first focus on the various monitoring image blocks in the target monitoring image. These blocks are obtained by dividing the target monitoring image into several small blocks, each of which contains a part of the image content. For example, if the target monitoring image is a scene of a substation, one image block may focus on the transformer, and another may show wires or switchgear.

[0084] Next, the recognition system extracts an embedded representation vector for each image block, which is achieved through a deep learning model (such as a convolutional neural network) that has been trained on a large amount of image data and can effectively extract key features from the image blocks. After extracting the embedded representation vector of the target surveillance image, the recognition system refers to the image representation vector of the hidden danger related images. These hidden danger related images are images that are similar or related to the target surveillance image in content, and they provide additional information about possible safety hazards. For example, if the target surveillance image shows a utility pole, the hidden danger related images may contain photos of various damages or abnormalities of the utility pole.

[0085] The recognition system calculates the similarity between the embedded representation vector of each image block of the target monitoring image and the embedded representation vector of the corresponding image block of the hidden danger related image. This similarity calculation can adopt methods such as cosine similarity and Euclidean distance. By comparing these vectors, the system can identify the features in the target monitoring image that are most relevant to safety hazards. Then, based on these similarity calculation results, the recognition system adjusts the embedded representation vector of the target monitoring image. This adjustment may be weighting, fusion or transformation of the original vector to enhance those features that are closely related to safety hazards and suppress irrelevant or noise features. For example, if an image block is very similar to a broken utility pole in a hidden danger related image, the embedded representation vector of the block may be adjusted to emphasize this feature more.

[0086] Finally, the adjusted image block embedding representation vectors are combined or fused to form an inference representation vector of the target monitoring image. This inference representation vector is a more powerful and more focused feature representation of safety hazards, which will be used for subsequent safety hazard category reasoning. In this way, step S41 can significantly improve the sensitivity and accuracy of the recognition system for electricity-related public safety hazards.

[0087] Step S42: Based on the inference representation vector of the target monitoring image, inferring the inference image safety hazard category of the target monitoring image from a plurality of pre-determined image safety hazard categories.

[0088] Among them, the image block inference representation vector of the target monitoring image block in the target monitoring image is obtained based on the representation information commonality measurement result between the target monitoring image block and the relevant image blocks in each hidden danger-related image, and the representation information commonality measurement result between the target monitoring image block and any relevant image block is obtained based on the embedded representation vector of the target monitoring image block and the embedded representation vector of any relevant image block.

[0089] In step S42, the recognition system uses the inference characterization vector of the target monitoring image to determine the possible safety hazard category. This process involves in-depth analysis of the inference characterization vector and comparison with predefined safety hazard categories.

[0090] First, the recognition system focuses on the reasoning representation vector previously obtained in step S41. This reasoning representation vector is derived based on the reasoning representation vectors of each image block in the target monitoring image. It integrates the feature similarity information between the target monitoring image and the hidden danger-related image, so it has stronger expressiveness and pertinence. Next, the recognition system compares this reasoning representation vector with multiple pre-determined safety hazard categories. These safety hazard categories may include but are not limited to exposed wires, damaged equipment, fire risks, flooding risks, etc. Each category has corresponding feature descriptions and judgment criteria.

[0091] The comparison process can be achieved through machine learning models, such as classifiers such as support vector machines (SVM), decision trees or neural networks. These classifiers have been trained on a large amount of labeled image data and are therefore able to predict the most likely category of safety hazards based on the input reasoning representation vector.

[0092] Taking neural networks as an example, the recognition system may use a multi-layer perceptron (MLP) or convolutional neural network (CNN) for classification. When the inference representation vector is input into the neural network, the network processes this vector through its complex connection and activation functions, and gives the probability distribution of each safety hazard category at the output layer. For example, if the inference representation vector of the target monitoring image is very close to the characteristics of the "exposed wire" category, then the output probability of the neural network on the "exposed wire" category will be very high. Based on this, the recognition system will determine that there is a safety hazard of "exposed wire" in the target monitoring image.

[0093] In addition, step S42 also involves a detailed analysis of the image blocks. In the previous step, the recognition system has calculated the similarity between the embedded representation vectors of each image block in the target monitoring image and the corresponding image block in the hidden danger related image. This similarity information will also be used in step S42 to help more accurately determine the type and location of the safety hazard.

[0094] Ultimately, the recognition system outputs one or more of the most likely safety hazard categories, as well as the specific location information of these categories in the image (if possible). This information is crucial for subsequent prevention and response measures, and can help managers detect and resolve potential safety risks in a timely manner.

[0095] As an implementation manner, the above step S41, adjusting the image representation vector of the target monitoring image based on the image representation vectors of each hidden danger related image to obtain the inference representation vector of the target monitoring image, may specifically include:

[0096] Step S411: Based on the embedded representation vector of the target monitoring image block in the target monitoring image and based on the embedded representation vector of each relevant image block of the target hidden danger related image in the hidden danger related image set, determine the commonality measurement result of the representation information between the target monitoring image block and each relevant image block of the target hidden danger related image.

[0097] In step S411, the recognition system first processes the image data in the target monitoring image and the hidden danger related image set to determine the commonality measurement result of the representation information between them. It can be understood that the representation information is the information that represents the corresponding image features, such as the image representation vector, and the commonality measurement result of the representation information is the feature similarity. This step helps the recognition system understand the similarity between the target monitoring image and the known hidden danger image, and then determine whether there is a safety hazard.

[0098] Specifically, the recognition system first focuses on a specific area in the target surveillance image, namely the target surveillance image patch, and extracts an embedded representation vector for it. An embedded representation vector is a mathematical representation that captures the key features of an image, and it is usually extracted from the image through a deep learning model (such as a convolutional neural network). For example, if the target surveillance image patch contains an image of a transformer, then its embedded representation vector may contain information about the transformer's shape, color, texture and other visual features.

[0099] At the same time, the recognition system will select a target hidden danger related image from the hidden danger related image set and extract embedding representation vectors for each related image block. The hidden danger related image set contains various known safety hidden danger images, which have been processed and annotated in the previous steps.

[0100] Next, the recognition system calculates the commonality measurement results between the embedded representation vector of the target monitoring image block and the embedded representation vector of each relevant image block of the target hidden danger related image. This commonality measurement, namely feature similarity, is an indicator to measure the similarity between two vectors. It can be obtained by calculating the cosine similarity, Euclidean distance or other similarity calculation methods of two vectors.

[0101] Taking cosine similarity as an example, if the angle between two vectors is smaller, the similarity between them is higher. In the scenario of identifying public safety hazards involving electricity, this means that the target monitoring image block and the target hazard-related image block have high similarity in visual features. For example, if the target monitoring image block shows a local overheating area of ​​a transformer, and there is an image block in the hazard-related image that shows a similar overheating phenomenon, then the cosine similarity between the embedded representation vectors of the two image blocks will be very high.

[0102] Through this step, the recognition system can accurately quantify the similarity between the target monitoring image and the hidden danger related image, providing important data support for the subsequent safety hazard identification and classification.

[0103] Step S412: Determine an enhanced representation vector of the target monitoring image block compared to the image block of the target hidden danger related image based on the commonality measurement result of the representation information between the target monitoring image block and each related image block of the target hidden danger related image.

[0104] In step S412, the recognition system determines an enhanced representation vector of the target monitoring image block compared to the image block of the target hidden danger related image based on the representation information commonality measurement result calculated in the previous step S411. This vector can also be called an extended representation vector.

[0105] First, the recognition system reviews the feature similarity data obtained in step S411. These data reflect the degree of similarity between the target monitoring image block and the various related image blocks of the target hidden danger-related image at the feature level. For example, if the feature similarity is very high, it means that the target monitoring image block is visually very close to a hidden danger-related image block and may contain similar safety hazard information. Next, the recognition system uses this information to construct an enhanced representation vector. The purpose of this vector is to highlight and strengthen those features that are related to the target hidden danger, while suppressing or ignoring features that are not related to the target hidden danger. Specifically, the system may weight features that are highly similar to the target hidden danger-related image blocks to increase their importance in the enhanced representation vector.

[0106] For example, suppose that the target monitoring image block contains an area with exposed wires, and there is an image block in the target hidden danger-related image that also shows a similar exposed wire situation. In step S411, the feature similarity of the two image blocks is calculated and found to be very high. In step S412, the recognition system will use this high similarity information to assign higher weights to those features related to exposed wires (such as the color, shape, texture, etc. of the wires), thereby constructing an enhanced representation vector that can highlight this safety hazard. The construction of this enhanced representation vector may involve complex mathematical operations and machine learning techniques. For example, linear or nonlinear transformations can be used to adjust the weights of the original features, or the attention mechanism in deep learning can be used to automatically learn and assign weights.

[0107] Ultimately, this enhanced representation vector will serve as part of the inference representation vector in subsequent steps, helping the recognition system to more accurately determine whether there are electrical public safety hazards in the target monitoring image, and classify and locate them.

[0108] As an implementation manner, the step S412, based on the commonality measurement result of the representation information between the target monitoring image block and each relevant image block of the target hidden danger related image, determines the enhanced representation vector of the target monitoring image block compared with the image block of the target hidden danger related image, which may specifically include:

[0109] Step S4121: performing standardization operations on the commonality measurement results of the representation information between the target monitoring image block and each relevant image block of the target hidden danger related image, respectively, to obtain the commonality influence coefficient of each relevant image block of the target hidden danger related image relative to the target monitoring image block.

[0110] In step S4121, the recognition system will standardize the commonality measurement results of the representation information between the target monitoring image block and each related image block of the target hidden danger related image. The purpose of this step is to obtain the commonality influence coefficient, that is, the similarity weight, which reflects the influence of each hidden danger related image block on the target monitoring image block.

[0111] The purpose of standardization is to eliminate the influence of dimensional differences and numerical values ​​between different features, so that each feature is comparable. In the scenario of identifying public safety hazards involving electricity, the calculation of common influence coefficients is crucial for subsequent hazard identification. In specific operations, the recognition system will first review the commonality measurement results of the characterization information calculated in the previous step, which are usually similarity scores in numerical form. However, these scores may vary due to different calculation methods or data ranges, so it is necessary to unify the scale through standardization operations. Standardization operations can be performed in a variety of ways, such as Min-Max Normalization or Z-score normalization. Taking Min-Max Normalization as an example, this method maps the original data to a specified range (usually between 0 and 1), thereby eliminating the influence of the original data unit. In this process, the recognition system finds the maximum and minimum values ​​in the commonality measurement results of the characterization information, and then uses a formula to convert the data into a specified range.

[0112] After standardization, the recognition system will obtain a series of common influence coefficients, which are on a unified scale, which is convenient for comparison and calculation in subsequent steps. For example, if the common influence coefficient of the target monitoring image block and a hidden danger related image block is 0.8 after standardization, it means that there is a high similarity between the two, and the hidden danger related image block has a relatively large impact on the target monitoring image block.

[0113] Step S4121 provides a unified similarity weight for subsequent hidden danger identification through standardized operations, so that the recognition system can more accurately determine the potential safety hazards in the target monitoring image.

[0114] Step S4122: Based on the common influence coefficients of each relevant image block of the target hidden danger related image relative to the target monitoring image block, the embedded representation vectors of each relevant image block of the target hidden danger related image are adjusted and integrated to obtain an enhanced representation vector of the target monitoring image block compared to the image block of the target hidden danger related image.

[0115] In step S4122, the recognition system uses the common influence coefficient calculated in step S4121 to adjust and integrate the embedded representation vectors of each relevant image block of the target hidden danger related image. The core purpose of this step is to strengthen the features related to safety hazards in the target monitoring image block by integrating the information of multiple relevant image blocks, thereby obtaining an enhanced representation vector.

[0116] Specifically, the recognition system first reviews the common influence coefficients obtained in step S4121, which represent the similarity weights between each relevant image block and the target monitoring image block. Then, the recognition system adjusts the embedded representation vectors of the relevant image blocks according to these weights. The adjustment method may be weighted summation, weighted average, or other forms of vector combination, depending on the system design and actual needs.

[0117] Taking weighted averaging as an example, if the common influence coefficient of a related image block is high, which means that it is more similar to the target monitoring image block, then its embedded representation vector will occupy a greater weight in the integration process. In this way, the enhanced representation vector obtained by weighted averaging will more reflect the characteristics of this highly similar image block.

[0118] For example, assume that the target monitoring image block is an image of a utility pole, and the target hidden danger-related image contains multiple image blocks related to utility poles. In step S4121, the system has calculated the similarity weights of these image blocks and the target monitoring image blocks. Now, in step S4122, the system will integrate the embedded representation vectors of each image block according to these weights. If an image block shows a broken utility pole and its common influence coefficient is very high, then the representation vector of this image block will account for a larger proportion in the final enhanced representation vector.

[0119] In this way, the recognition system can construct an enhanced representation vector that integrates information from multiple highly similar image blocks. It not only contains the feature information of the target monitoring image block itself, but also incorporates the feature information of the highly similar hidden danger-related image blocks, which helps to more accurately identify the subsequent electricity-related public safety hazards in the target monitoring image. For example, if there are safety hazards such as broken or tilted electric poles, this enhanced representation vector can better reflect these abnormal features and improve the accuracy and reliability of the recognition system.

[0120] Step S413: Obtain an image block enhancement representation vector of the target monitoring image block compared with other hidden danger related images in the hidden danger related image set.

[0121] It can be understood that the process of obtaining the image block enhancement representation vector of the target monitoring image block compared to other hidden danger-related images in the hidden danger-related image set and the process of obtaining the image block enhancement representation vector of the target monitoring image block compared to the target hidden danger-related image are similar, and will not be elaborated here.

[0122] Step S414: performing a representation vector integration operation on the embedded representation vector of the target monitoring image block and the image block enhanced representation vector of each hidden danger related image in the hidden danger related image set compared with the target monitoring image block to obtain the image block inference representation vector of the target monitoring image block.

[0123] In step S414, the recognition system integrates the embedded representation vector of the target monitoring image block with the enhanced representation vector obtained in the previous step. The purpose of this step is to obtain a more comprehensive image block reasoning representation vector rich in multi-level information, which will help improve the accuracy of subsequent identification of public safety hazards related to electricity.

[0124] Specifically, the recognition system first obtains the embedded representation vector of the target monitoring image block, which reflects the feature information of the image block itself. Next, the enhanced representation vector of the target monitoring image block obtained in step S413 compared with the image block of each hidden danger related image in the hidden danger related image set is reviewed. These enhanced representation vectors not only contain the features of the target monitoring image block, but also incorporate the feature information of various safety hazards related to it. Next, the recognition system will integrate these two types of representation vectors. The integration method may include but is not limited to splicing, weighted averaging, or through a specific fusion strategy, such as attention mechanism, etc., depending on the design requirements and implementation details of the recognition system. The integrated image block reasoning representation vector will have both the features of the target monitoring image block itself and the feature information related to various safety hazards. For example, assuming that the target monitoring image block is a transformer, its embedded representation vector may contain basic information such as the shape, color, and texture of the transformer. The enhanced representation vector may incorporate the safety hazard features that may exist in the transformer, such as oil stains, damage, and excessive temperature. By integrating these two types of vectors, the resulting image block reasoning representation vector will be able to more comprehensively describe the current state of the transformer, including whether it has safety hazards. In this way, the recognition system can generate an image block reasoning representation vector rich in multi-level information, providing more accurate and comprehensive feature support for subsequent safety hazard identification. This will also greatly improve the performance of the recognition system in the task of identifying public safety hazards involving electricity.

[0125] As an implementation manner, the step S414 performs a representation vector integration operation on the embedded representation vector of the target monitoring image block and the image block enhanced representation vector of each hidden danger related image in the hidden danger related image set compared with the target monitoring image block to obtain the image block inference representation vector of the target monitoring image block, which may specifically include:

[0126] Step S4141: performing a representation vector combination operation on the target monitoring image block compared with the image block enhancement representation vectors of each hidden danger related image in the hidden danger related image set to obtain a combined enhancement representation vector corresponding to the target monitoring image block;

[0127] Step S4142: Determine an image block inference representation vector of the target monitoring image block based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block.

[0128] In step S4141, the recognition system first performs a combination operation on the target monitoring image block and the image block enhancement representation vectors of each hidden danger related image in the hidden danger related image set. This combination operation actually concatenates multiple enhancement representation vectors to form a longer combined enhancement representation vector. This vector not only contains the feature information of the target monitoring image block itself, but also integrates the feature information of multiple hidden danger related image blocks, thereby providing a more comprehensive and multi-angle feature description.

[0129] For example, if the target monitoring image block is a utility pole, and the image blocks in the hidden danger related image set include broken utility poles, tilted utility poles, etc., then by combining these enhanced representation vectors, the recognition system can obtain a combined enhanced representation vector covering multiple utility pole safety hazard features.

[0130] Next, in step S4142, the recognition system uses the embedded representation vector of the target monitoring image block and the combined enhanced representation vector just obtained to determine the final image block reasoning representation vector. This process may involve further integration or fusion of the two vectors, such as through weighted averaging, attention mechanism, etc., to ensure that the final reasoning representation vector can simultaneously reflect the characteristics of the target monitoring image block itself and the characteristic information related to various safety hazards.

[0131] Continuing with the example of utility poles, the recognition system may fuse the embedded representation vector of the utility pole (including the basic shape, color and other features of the utility pole) with the combined enhanced representation vector (including multiple safety hazard features) to obtain an image block inference representation vector that contains both the basic information of the utility pole and the safety hazard features.

[0132] In this way, the recognition system can generate a comprehensive and multi-level information-rich image block reasoning representation vector, providing accurate and comprehensive feature support for subsequent safety hazard identification. This is of great significance in applications such as power-related public safety hazard identification, as it can greatly improve the recognition system's ability to perceive safety hazards and its recognition accuracy.

[0133] As an implementation mode, the step S4142 determines the image block inference representation vector of the target monitoring image block based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block. Specifically, it may include: performing a representation vector combination operation on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block to obtain the image block inference representation vector of the target monitoring image block.

[0134] In this implementation of step S4142, the recognition system combines the embedded representation vector of the target monitoring image block with the combined enhanced representation vector corresponding to the target monitoring image block to obtain the image block inference representation vector of the target monitoring image block.

[0135] Specifically, the recognition system will first obtain the embedded representation vector of the target monitoring image block, which mainly reflects the inherent characteristics of the image block itself, such as shape, color, texture, etc. At the same time, the system will also obtain the combined enhanced representation vector corresponding to the target monitoring image block. This vector is obtained by fusing the features of multiple hidden danger images related to the target monitoring image block, and it contains the feature information of various safety hazards related to the target monitoring image block.

[0136] Next, the recognition system combines the two vectors. The combination operation here can be to directly concatenate the two vectors together to form a longer vector, or to fuse the features of the two vectors through a specific fusion strategy, such as weighted averaging, attention mechanism, etc., to generate a new vector. This new vector is the image block reasoning representation vector of the target monitoring image block.

[0137] For example, assuming that the target monitoring image block is an image of a transformer, its embedded representation vector may contain basic information such as the shape, color, and size of the transformer. At the same time, by fusing the features of other hidden danger-related images, a combined enhanced representation vector is obtained, which may contain the safety hazard features that may exist in the transformer, such as oil stains, cracks, etc. By combining these two vectors, an image block reasoning representation vector containing both the basic information of the transformer and the safety hazard features is obtained.

[0138] This image segmentation inference representation vector will serve as an important basis for subsequent safety hazard identification. In this way, the recognition system can more comprehensively capture various feature information of the target monitoring image segment, thereby improving the accuracy and reliability of safety hazard identification.

[0139] Or; as another implementation manner, the step S4142, based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block, determines the image block inference representation vector of the target monitoring image block, which may specifically include:

[0140] Step S41421: Perform a vector mapping operation on the combined enhanced representation vector corresponding to the target monitoring image block based on preset mapping parameters to obtain a mapping enhanced representation vector corresponding to the target monitoring image block; the number of vector elements of the mapping enhanced representation vector corresponding to the target monitoring image block is the same as the number of vector elements of the embedded representation vector of the target monitoring image block.

[0141] In step S41421, the recognition system uses a preset mapping parameter to perform feature transformation on the combined enhanced representation vector of the target monitoring image block to obtain a mapped enhanced representation vector. The mapping parameter can be a transformation matrix or other forms of transformation parameters, which can map the feature space of the combined enhanced representation vector to a new space while ensuring that the mapped vector has the same number of elements, that is, dimension, as the embedded representation vector of the target monitoring image block.

[0142] Specifically, assume that the embedded representation vector of the target surveillance image block is a 50-dimensional vector, and the combined enhanced representation vector is originally a 100-dimensional vector. In order to achieve effective fusion of these two vectors, the recognition system needs to first map the 100-dimensional combined enhanced representation vector to a 50-dimensional space. At this time, the system will use a preset mapping parameter (such as a 100×50 transformation matrix) to perform a linear transformation, mapping the original combined enhanced representation vector from the 100-dimensional space to the 50-dimensional space, thereby obtaining a new 50-dimensional mapping enhanced representation vector.

[0143] Taking the identification of public safety hazards related to electricity as an example, assuming that the target monitoring image block is an image area of ​​a transformer, its embedded representation vector reflects the basic shape, color, texture and other information of the transformer. At the same time, the combined enhanced representation vector may incorporate multiple features related to transformer safety hazards, such as temperature anomalies, oil stains, rust, etc. Through the mapping operation, the recognition system can compress these safety hazard features into a space of the same dimension as the embedded representation vector, which is convenient for subsequent feature fusion and reasoning analysis.

[0144] In this way, the recognition system can effectively align feature vectors of different dimensions, providing more comprehensive and accurate feature support for subsequent safety hazard identification.

[0145] Step S41422: Perform a representation vector combination operation on the embedded representation vector of the target monitoring image block, the mapping enhancement representation vector corresponding to the target monitoring image block, and the error representation vector between the embedded representation vector of the target monitoring image block and the mapping enhancement representation vector corresponding to the target monitoring image block to obtain the image block inference representation vector of the target monitoring image block.

[0146] Specifically, the recognition system first obtains the embedded representation vector of the target monitoring image block, which captures the basic features of the image block, such as shape, texture, and color. Taking electricity-related public safety as an example, if the target monitoring image block is a utility pole, the embedded representation vector may contain the shape, color, and texture information of the utility pole. Next, the mapping enhancement representation vector corresponding to the target monitoring image block is obtained. This vector is obtained by mapping the combined enhancement representation vector in the previous step S41421, and it incorporates the safety hazard features related to the target monitoring image block. Continuing with the example of the utility pole, the mapping enhancement representation vector may contain features of safety hazards such as rust, cracks, or tilt.

[0147] Then, the recognition system calculates the error representation vector between the embedded representation vector and the mapped enhanced representation vector. The error representation vector is calculated by bitwise subtracting the two vectors and taking the absolute value. This error representation vector reflects the difference between the original feature and the mapped safety hazard feature and is an interactive feature.

[0148] Finally, the recognition system concatenates these three vectors (embedded representation vector, mapping enhancement representation vector, and error representation vector) to form a longer image block reasoning representation vector. This new vector not only contains the basic features of the image block, but also incorporates the safety hazard features and the difference information between the two, thus providing richer and more comprehensive data support for subsequent safety hazard identification.

[0149] In this way, the recognition system can more accurately identify safety hazards in public electricity-related scenes, such as rust and cracks on utility poles, and take necessary measures in a timely manner to prevent potential safety accidents.

[0150] Step S415: obtaining an inference representation vector of the target monitoring image based on the image block inference representation vector of the target monitoring image block.

[0151] For example, suppose the recognition system is processing a surveillance image of a public power scene containing multiple devices (such as transformers, utility poles, distribution boxes, etc.). In the previous step, the recognition system has segmented the image into multiple image blocks and extracted image block reasoning representation vectors for each block. These vectors capture the feature information of each block, including shape, texture, color, and possible safety hazard features.

[0152] In step S415, the recognition system aggregates the inference representation vectors of these image blocks to form an inference representation vector of the entire target monitoring image. This process can be implemented in a variety of ways. For example, the system can average or weighted average the inference representation vectors of all image blocks, or adopt more complex fusion strategies, such as attention mechanisms, to emphasize the features of certain important blocks.

[0153] Taking weighted averaging as an example, assume that the system assigns a weight to the inference representation vector of each image block based on its importance in identifying safety hazards. Then, the system calculates the average of all weighted inference representation vectors to obtain the inference representation vector of the entire target monitoring image. This inference representation vector not only contains the feature information of each part of the image, but also reflects the importance of different parts in identifying safety hazards through weights. This enables the recognition system to understand the image content more comprehensively and accurately identify the safety hazards therein.

[0154] Finally, the recognition system can use this inference representation vector as input to classify and identify safety hazards through machine learning models (such as support vector machines, neural networks, etc.). In this way, the system can automatically discover potential safety hazards in public power-related scenes, such as equipment failures, line abnormalities, etc., and issue alarms or perform other corresponding processing in a timely manner.

[0155] In one implementation, the step S42, based on the inference representation vector of the target monitoring image, inferring the inference image safety hazard category of the target monitoring image from a plurality of pre-determined image safety hazard categories, may specifically include:

[0156] Step S421: performing confidence reasoning on the reasoning representation vector of the target monitoring image to obtain the reasoning confidence that the target monitoring image is inferred to be each of the plurality of predetermined image safety hazard categories;

[0157] Step S422: Determine, among the plurality of predetermined image safety hazard categories, a predetermined image safety hazard category that meets the confidence value requirement as the inferred image safety hazard category of the target monitoring image.

[0158] In step S421, the recognition system performs confidence reasoning on the reasoning representation vector of the target monitoring image. This process can be implemented by using a machine learning model, such as a support vector machine (SVM), a random forest, or a neural network classifier, such as the image safety hazard category reasoning layer in the safety hazard recognition neural network provided in the subsequent embodiments. These classifiers have been trained to identify different safety hazard categories, and will output the reasoning confidence of each category based on the input reasoning representation vector.

[0159] For example, suppose there are four pre-determined safety hazard categories: exposed wires, aging equipment, overload operation, and ground fault. For a given inference representation vector of a target monitoring image, the classifier will output the confidence that the image belongs to the above four categories, such as the confidence of exposed wires is 0.8, the confidence of aging equipment is 0.1, the confidence of overload operation is 0.05, and the confidence of ground fault is 0.05.

[0160] In step S422, the recognition system determines the safety hazard category of the target monitoring image based on the confidence value requirement. Usually, this requirement is a preset threshold, and only when the confidence of a certain category exceeds this threshold, the recognition system considers it to belong to this category. Continuing with the above example, if the set threshold is 0.7, then only when the confidence of exposed wires exceeds 0.7, the recognition system infers the safety hazard category of the target monitoring image as exposed wires. In this way, the recognition system can automatically match the monitoring image with the preset safety hazard category, thereby providing timely and accurate hazard identification services for public safety related to electricity. This technology has broad application prospects in preventing power accidents and improving the safety of power equipment operation.

[0161] As an implementation scheme, the inference image safety hazard category of the target monitoring image is obtained based on the safety hazard identification neural network inference. The following introduces the calibration process of the safety hazard identification neural network, including the following steps S110~S150:

[0162] Step S110: Obtain an electrical-related monitoring image learning sample, a hidden danger-related image learning sample set associated with the electrical-related monitoring image learning sample, and an image safety hidden danger category prior label; the hidden danger-related image learning sample set includes at least one hidden danger-related image learning sample, and the image safety hidden danger category prior label is used to indicate the image safety hidden danger category of the electrical-related monitoring image learning sample.

[0163] The learning samples of power-related monitoring images are image samples used to train neural networks. They are image data collected from actual monitoring systems. These images may contain various power-related equipment, scenes, and potential safety hazards. For example, one learning sample may be an image containing high-voltage wires, and another may be an image of a working transformer. The set of learning samples of hidden danger-related images associated with the learning samples of power-related monitoring images refers to a set of images of possible safety hazards associated with each power-related monitoring image. These images show various problems that may occur in power-related equipment and scenes, such as aging equipment, exposed wires, overloaded operation, etc. For example, for image learning samples containing high-voltage wires, the hidden danger-related images associated with them may include images of safety hazards such as broken wires and damaged insulation.

[0164] The prior labeling of image safety hazard categories is used to indicate the labeling information of the safety hazard categories in the learning samples of power-related monitoring images. These labels are annotated in advance by experts or based on historical data to guide the training process of the neural network. For example, for an image containing exposed wires, its prior labeling may indicate the safety hazard category of "exposed wires".

[0165] In step S110, the recognition system needs to collect these learning examples and hidden danger related images from various sources (such as monitoring system videos, historical image databases, etc.), and sort out the corresponding safety hazard category prior labels. These data will serve as the basis for subsequent neural network training to help the neural network learn the ability to identify safety hazards from images.

[0166] Step S120: extracting the image representation vectors of the electrical monitoring image learning samples and the image representation vectors of each hazard-related image learning sample through the representation information extraction layer in the safety hazard identification neural network to be calibrated.

[0167] The recognition system uses the representation information extraction layer, usually the convolutional layer, in the safety hazard recognition neural network to be adjusted to extract the image representation vectors of the electrical monitoring image learning samples and the image representation vectors of the hazard-related image learning samples. Specifically, the role of the representation information extraction layer is to extract key feature information from the input image and convert this information into a mathematical expression, namely the image representation vector. This vector can capture key visual features such as shape, texture, and color in the image, and is the basis for subsequent image recognition and analysis.

[0168] For example, suppose an example of an electrical monitoring image is an image of a transformer. When this image is input into the neural network, the representation information extraction layer starts to work, extracting features from the image through operations such as convolution operations, such as the shape, color, and texture of the transformer, and encoding these features into an image representation vector. This vector may be a high-dimensional array, such as [0.23, -0.15, 0.87, ...], where each value represents the intensity or presence of a specific feature in the image.

[0169] Similarly, for hidden danger-related image learning samples associated with power-related monitoring images, such as images of transformer overheating or oil leakage, the representation information extraction layer will also extract their image representation vectors in the same way. These vectors will be used for subsequent safety hazard identification and analysis. In step S120, the recognition system automatically completes the conversion from the original image to the feature vector through a neural network, which is an important step in realizing the automatic identification of public safety hazards related to electricity. In this way, the system can more effectively process and analyze a large amount of monitoring image data and improve the accuracy and efficiency of safety hazard identification.

[0170] Step S130: Through the representation information adjustment layer in the safety hazard identification neural network to be calibrated, the image representation vector of the electrical monitoring image learning sample is adjusted based on the image representation vector of each hazard-related image learning sample to obtain an inference representation vector of the electrical monitoring image learning sample.

[0171] In step S130, the recognition system uses the representation information adjustment layer in the safety hazard recognition neural network to be calibrated to adjust the image representation vector of the electrical monitoring image learning sample.

[0172] Specifically, the function of the representation information adjustment layer is to receive the image representation vector of the learning sample of the electricity-related monitoring image and the image representation vector of each hidden danger-related image learning sample associated with it, and then perform a series of operations and adjustments based on these inputs. This adjustment process can be understood as fusing the feature information of the hidden danger-related image into the electricity-related monitoring image, thereby enhancing the feature expression of the electricity-related monitoring image in terms of safety hazards. For example, suppose that the learning sample of the electricity-related monitoring image is an image of a normally operating transformer, and its image representation vector captures the normal state characteristics of the transformer. The hidden danger-related image learning samples associated with it may include images of different safety hazards such as transformer oil leakage and overheating, and their image representation vectors capture the characteristics of these safety hazards respectively. In step S130, the representation information adjustment layer will fuse the feature vectors of these safety hazards with the feature vectors of the electricity-related monitoring image to obtain an inference representation vector that fuses various safety hazard features. This inference representation vector not only contains the features of the electricity-related monitoring image itself, but also incorporates the features of various safety hazards associated with it, so it is more representative and can more comprehensively reflect the information of the electricity-related monitoring image in terms of safety hazards. This will help improve the accuracy and reliability of subsequent safety hazard identification.

[0173] Step S140: Confidence inference is performed on the inference representation vector of the electrical surveillance image learning sample through the image safety hazard category inference layer in the safety hazard identification neural network to be calibrated, and the inference confidence of the electrical surveillance image learning sample being inferred as each of multiple predetermined image safety hazard categories.

[0174] In step S140, the recognition system performs confidence reasoning on the reasoning representation vector of the electrical monitoring image learning sample through the image safety hazard category reasoning layer in the safety hazard recognition neural network to be calibrated.

[0175] The image safety hazard category inference layer is an important component of the neural network. It is responsible for receiving the inference representation vectors and inferring the safety hazard category that the image may belong to based on these vectors. This inference layer usually uses some classification algorithms, such as the Softmax classifier, to assign an inference confidence to each safety hazard category. The inference confidence is a numerical value that indicates the degree of confidence of the neural network that the electrical monitoring image belongs to a specific safety hazard category. This value is usually between 0 and 1. The higher the value, the more confident the neural network is that the image belongs to that category.

[0176] For example, suppose there are three pre-determined safety hazard categories: exposed wires, aging equipment, and overload operation. For a learning example of an electrical monitoring image containing a transformer, its inference representation vector is obtained after the previous steps. In step S140, the image safety hazard category inference layer receives this inference representation vector and outputs the inference confidence that the image belongs to the above three categories, such as [0.8, 0.1, 0.1]. This means that the neural network has 80% confidence that this image belongs to the "exposed wire" category, and only 10% confidence that it belongs to the "aging equipment" or "overload operation" category.

[0177] In this way, step S140 helps the recognition system to make a preliminary judgment on the category of potential safety hazards in the electrical monitoring image, and provides an important decision-making basis for the subsequent potential safety hazard identification. At the same time, this reasoning confidence can also be used as a feedback signal in the neural network calibration process to help optimize the parameters and performance of the neural network.

[0178] Step S150: Based on the error between the inference confidence corresponding to the electrical monitoring image learning sample and the prior label of the image safety hazard category, the safety hazard identification neural network to be calibrated is optimized to obtain the safety hazard identification neural network.

[0179] Step S150 involves optimizing the parameters of the safety hazard recognition neural network to be calibrated based on the error between the inference confidence corresponding to the learning sample of the electrical monitoring image and the prior label of the image safety hazard category.

[0180] Specifically, the recognition system first calculates the error between the reasoning confidence corresponding to the learning sample of the electrical monitoring image and the prior label of the image safety hazard category. This error reflects the difference between the neural network reasoning result and the real safety hazard category. For example, if the neural network gives a low reasoning confidence for an image containing exposed wire hazards, but the prior label of the image clearly indicates the existence of exposed wire hazards, then there is an error between the two. Next, the recognition system uses this error information to optimize the parameters of the safety hazard recognition neural network to be adjusted. The goal of the optimization is to reduce the error between the reasoning confidence and the prior label, so that the reasoning result of the neural network is closer to the real safety hazard category. This process usually uses some optimization algorithms, such as the gradient descent algorithm, to adjust the parameters such as the weight and bias of the neural network.

[0181] For example, assuming that there is a large error between the neural network's inference result and the prior labeling for a certain electrical monitoring image, then through parameter optimization, the neural network may increase its attention to specific safety hazard features in the image, thereby improving its ability to identify the safety hazard.

[0182] Finally, after multiple iterations of optimization, the recognition system will obtain a calibrated safety hazard recognition neural network, which can more accurately identify the types of safety hazards in electrical surveillance images and provide more reliable protection for public safety.

[0183] Step S150 optimizes the parameters of the neural network by reducing the error between the inference confidence and the priori mark, thereby improving the neural network's ability to identify electricity-related public safety hazards.

[0184] As an implementation manner, the step S421, performing confidence reasoning on the reasoning representation vector of the target monitoring image to obtain the reasoning confidence that the target monitoring image is inferred as each of the multiple predetermined image safety hazard categories, may specifically include:

[0185] Step S4211: performing a linear transformation on the inference representation vector of the target monitoring image based on the representation information mapping parameter to obtain a linear transformation result;

[0186] Step S4212: Use a preset classifier to obtain the linear transformation result obtained to obtain the inference confidence that the target monitoring image is inferred to be each of a plurality of predetermined image safety hazard categories; wherein the characterization information mapping parameter is a network parameter of the safety hazard identification neural network, the network parameter is obtained when the safety hazard identification neural network to be calibrated, and the classifier is a softmax classifier.

[0187] In step S4211, the recognition system uses the representation information mapping parameters to perform a linear transformation on the inference representation vector of the target monitoring image. These parameters are network parameters obtained during the calibration of the safety hazard recognition neural network. The purpose of the linear transformation is to map the inference representation vector from the original feature space to a feature space that is easier to classify.

[0188] For example, assuming that the inference representation vector is a high-dimensional array, such as [0.23, -0.15, 0.87, ...], the representation information mapping parameter may be a set of weights and bias values, and the original inference representation vector is converted into a new linear transformation result through a linear transformation formula (such as y=Wx+b, where W is the weight matrix and b is the bias vector).

[0189] In step S4212, the recognition system uses a preset classifier (softmax classifier) ​​to process the linear transformation results. The Softmax classifier can convert the linear transformation results into a probability distribution, that is, the reasoning confidence of each safety hazard category. These reasoning confidences represent the possibility that the target monitoring image belongs to each safety hazard category.

[0190] For example, suppose there are three categories of safety hazards: exposed wires, aging equipment, and overloaded operation. After being processed by the softmax classifier, an inference confidence array similar to [0.8, 0.15, 0.05] may be obtained, indicating that the target monitoring image has an 80% probability of belonging to the "exposed wire" category, a 15% probability of belonging to the "aging equipment" category, and a 5% probability of belonging to the "overloaded operation" category.

[0191] In summary, the derivative scheme of step S421 can accurately infer the possible safety hazard categories and their corresponding reasoning confidences of the target monitoring image through linear transformation and classifier processing, providing an important basis for subsequent safety hazard identification and processing.

[0192] In some embodiments, the representation information extraction layer in the safety hazard identification neural network is a convolutional network layer, the representation information adjustment layer in the safety hazard identification neural network includes a saliency processing layer (i.e., an attention layer) and a feature integration layer, and the classifier selected for the image safety hazard category inference layer in the safety hazard identification neural network is a softmax classifier; after the recognition system obtains the target monitoring image and a hazard-related image set associated with the target monitoring image and including at least one hazard-related image, the image representation vector of the target monitoring image and the image representation vectors of each hazard-related image can be extracted respectively through the representation information extraction layer convolutional network layer in the safety hazard identification neural network, wherein the image representation vector of the target monitoring image includes: an embedded representation vector of each monitoring image block in the target monitoring image, and the image representation vector of the target hazard-related image in the hazard-related image set includes an embedded representation vector of each related image block in the target hazard-related image.

[0193] In addition, based on the saliency processing layer in the representation information adjustment layer in the safety hazard identification neural network, based on the embedded representation vector of the target monitoring image block in the target monitoring image, and based on the embedded representation vectors of each relevant image block of the target hidden danger related image in the hidden danger related image set, the representation information commonality measurement result between the target monitoring image block and each relevant image block of the target hidden danger related image can be determined; based on the representation information commonality measurement result between the target monitoring image block and each relevant image block of the target hidden danger related image, the image block enhancement representation vector of the target monitoring image block compared with the target hidden danger related image is determined; the image block enhancement representation vector of the target monitoring image block compared with each other hidden danger related image in the hidden danger related image set is obtained; based on the feature integration layer in the representation information adjustment layer in the safety hazard identification neural network, the embedded representation vector of the target monitoring image block and the image block enhancement representation vector of the target monitoring image block compared with each hidden danger related image in the hidden danger related image set are integrated to obtain the image block reasoning representation vector of the target monitoring image block; based on the image block reasoning representation vector of the target monitoring image block, the reasoning representation vector of the target monitoring image is obtained. The image safety hazard category inference layer in the safety hazard identification neural network can be used to perform confidence inference on the inference representation vector of the target monitoring image to obtain the inference confidence of the target monitoring image being inferred as each of the multiple predetermined image safety hazard categories; among the multiple predetermined image safety hazard categories, the predetermined image safety hazard category that meets the confidence value requirements is determined as the inferred image safety hazard category of the target monitoring image.

[0194] In summary, the method for identifying electrical public safety hazards provided in the embodiments of the present application significantly improves the accuracy of safety hazard identification by introducing feature information of hazard-related images.

[0195] The method first obtains the target surveillance image and finds the associated hidden danger related image set through the hidden danger image library query. These related images provide additional feature information for the target surveillance image and help identify those safety hidden danger features that may not be obvious or missing in the original surveillance image.

[0196] After extracting the representation vectors of the target monitoring image and the hidden danger related image, this method uses the representation vectors of these related images to adjust the representation vector of the target monitoring image. This step actually incorporates the features of the associated hidden danger image on the basis of the features of the original monitoring image, thereby enriching the feature expression and making the identification of safety hazards more accurate.

[0197] By dividing the target monitoring image and the hidden danger related image into blocks and calculating the commonality measure of the representation information between the blocks, this method can determine the enhanced representation vector of the target monitoring image block compared with the hidden danger related image block. This block processing method helps to capture the local feature differences and further improve the precision of recognition.

[0198] When determining the enhanced representation vector of the target monitoring image block compared to the hidden danger related image block, this method integrates the embedded representation vectors of each related image block according to the attention weight. This attention mechanism can highlight important features and suppress irrelevant information, making the integrated features more representative.

[0199] Through the above steps, this method has fully integrated the feature information of the hazard-related image when inferring the safety hazard category of the target monitoring image. This fusion not only improves the feature expression of the target monitoring image, but also increases the accuracy of safety hazard identification, allowing the recognition system to more accurately discover and locate potential safety hazards.

[0200] In summary, this method effectively enhances and improves the features of the target monitoring image by integrating the feature information of hidden danger-related images, and significantly improves the recognition accuracy of electricity-related public safety hazards.

[0201] It should be noted that in the embodiments of the present application, if the above-mentioned method for identifying electrical public safety hazards is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0202] An embodiment of the present application provides an identification system, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0203] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0204] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0205] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0206] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0207] Figure 2 A schematic diagram of a hardware entity of an identification system provided in an embodiment of the present application, such as Figure 2 As shown, the hardware entity of the identification system 1000 includes: a processor 1001 and a memory 1002, wherein the memory 1002 stores a computer program that can be run on the processor 1001, and the processor 1001 implements the steps in the method of any of the above embodiments when executing the program.

[0208] The memory 1002 stores computer programs that can be run on the processor. The memory 1002 is configured to store instructions and applications executable by the processor 1001. It can also cache data to be processed or processed by the processor 1001 and each module in the recognition system 1000 (for example, image data, audio data, voice communication data, and video communication data). This can be achieved through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0209] When the processor 1001 executes the program, the steps of any of the above-mentioned methods for identifying electrical safety hazards are implemented. The processor 1001 generally controls the overall operation of the identification system 1000 .

[0210] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the method for identifying electricity-related public safety hazards in any of the above embodiments.

[0211] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For the technical details not disclosed in the storage medium and device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding. The above processor can be at least one of a target application integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD), a programmable logic device (Programmable Logic Device, PLD), a field programmable gate array (Field Programmable Gate Array, FPGA), a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, and a microprocessor. It can be understood that the electronic device that realizes the above processor function can also be other, and the embodiments of the present application are not specifically limited.

[0212] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0213] It should be understood that the "one embodiment" or "one embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" appearing in various places throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence number of the embodiment of the present application is only for description and does not represent the advantages and disadvantages of the embodiment. It should be noted that, in this article, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0214] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0215] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0216] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0217] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0218] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0219] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for identifying hidden dangers of public safety related to electricity, characterized in that: Applied to an identification system, the method comprises: Acquire a target monitoring image for potential safety hazards identification; the target monitoring image is an image taken of a target electrical public area; Performing a hidden danger image library query operation with the target monitoring image as a query image to obtain a hidden danger image library query result, and determining a hidden danger related image set associated with the target monitoring image based on known safety hidden danger images included in the hidden danger image library query result; the hidden danger related image set includes at least one hidden danger related image obtained based on each known safety hidden danger image; Respectively extracting the image representation vector of the target monitoring image and the image representation vector of each hidden danger related image; Inferring the image safety hazard category of the target monitoring image based on the image representation vector of the target monitoring image and the image representation vectors of each hazard-related image to obtain an inferred image safety hazard category of the target monitoring image; The image representation vector of the target monitoring image includes the embedded representation vectors of each monitoring image block in the target monitoring image, and the image representation vector of the target hidden danger related image in the hidden danger related image set includes the embedded representation vector of each related image block in the target hidden danger related image; the image safety hidden danger category of the target monitoring image is inferred based on the image representation vector of the target monitoring image and the image representation vectors of each hidden danger related image to obtain the inferred image safety hidden danger category of the target monitoring image, including: Performing a representation vector adjustment on the image representation vector of the target monitoring image based on the image representation vectors of each hidden danger related image to obtain an inference representation vector of the target monitoring image; The inference representation vector of the target monitoring image is obtained based on the image block inference representation vectors of each monitoring image block in the target monitoring image; Based on the inference representation vector of the target monitoring image, inferring the inference image safety hazard category of the target monitoring image from a plurality of pre-determined image safety hazard categories; The image block reasoning representation vector of the target monitoring image block in the target monitoring image is obtained based on the representation information commonality measurement result between the target monitoring image block and the relevant image blocks in each hidden danger related image, and the representation information commonality measurement result between the target monitoring image block and any relevant image block is obtained based on the embedded representation vector of the target monitoring image block and the embedded representation vector of any relevant image block; The step of adjusting the image representation vector of the target monitoring image based on the image representation vectors of each hidden danger related image to obtain the inference representation vector of the target monitoring image includes: Determine a commonality measurement result of representation information between the target monitoring image block and each relevant image block of the target hidden danger related image in the hidden danger related image set based on the embedded representation vector of the target monitoring image block in the target monitoring image and based on the embedded representation vector of each relevant image block of the target hidden danger related image in the hidden danger related image set; Determining an enhanced representation vector of the target monitoring image block compared to the image block of the target hidden danger related image based on a commonality measurement result of the representation information between the target monitoring image block and each related image block of the target hidden danger related image; Obtaining an image block enhancement representation vector of the target monitoring image block compared to each other hidden danger related image in the hidden danger related image set; Performing a representation vector integration operation on the embedded representation vector of the target monitoring image block and the image block enhancement representation vector of each hidden danger related image in the hidden danger related image set compared with the target monitoring image block, to obtain an image block reasoning representation vector of the target monitoring image block; Based on the image block inference representation vector of the target monitoring image block, an inference representation vector of the target monitoring image is obtained.

2. The method according to claim 1, characterized in that The determining, based on the commonality measurement result of the representation information between the target monitoring image block and each relevant image block of the target hidden danger related image, an enhanced representation vector of the target monitoring image block compared with the image block of the target hidden danger related image comprises: Standardizing the commonality measurement results of the representation information between the target monitoring image block and each relevant image block of the target hidden danger related image, respectively, to obtain the commonality influence coefficient of each relevant image block of the target hidden danger related image relative to the target monitoring image block; Based on the common influence coefficients of each relevant image block of the target hidden danger related image relative to the target monitoring image block, the embedded representation vectors of each relevant image block of the target hidden danger related image are adjusted and integrated to obtain an enhanced representation vector of the target monitoring image block compared to the image block of the target hidden danger related image.

3. The method according to claim 1, characterized in that The embedding representation vector of the target monitoring image block is integrated with the image block enhancement representation vector of each hidden danger related image in the hidden danger related image set compared with the target monitoring image block to obtain the image block reasoning representation vector of the target monitoring image block, including: Performing a representation vector combination operation on the target monitoring image block compared with the image block enhancement representation vectors of each hidden danger related image in the hidden danger related image set to obtain a combined enhancement representation vector corresponding to the target monitoring image block; Based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block, an image block inference representation vector of the target monitoring image block is determined.

4. The method according to claim 3, characterized in that The step of determining the image block inference representation vector of the target monitoring image block based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block includes: A representation vector combination operation is performed on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block to obtain an image block inference representation vector of the target monitoring image block.

5. The method according to claim 3, characterized in that The step of determining the image block inference representation vector of the target monitoring image block based on the embedded representation vector of the target monitoring image block and the combined enhanced representation vector corresponding to the target monitoring image block includes: Performing a vector mapping operation on the combined enhanced representation vector corresponding to the target monitoring image block based on a preset mapping parameter variable to obtain a mapping enhanced representation vector corresponding to the target monitoring image block; The number of vector elements of the mapping enhancement representation vector corresponding to the target monitoring image block is the same as the number of vector elements of the embedding representation vector of the target monitoring image block; The embedded representation vector of the target monitoring image block, the mapping enhancement representation vector corresponding to the target monitoring image block, and the error representation vector between the embedded representation vector of the target monitoring image block and the mapping enhancement representation vector corresponding to the target monitoring image block are subjected to a representation vector combination operation to obtain an image block inference representation vector of the target monitoring image block.

6. The method according to claim 1, characterized in that The inference characterization vector based on the target monitoring image is used to infer the inference image safety hazard category of the target monitoring image from a plurality of pre-determined image safety hazard categories, including: Performing confidence reasoning on the inference representation vector of the target monitoring image to obtain the inference confidence that the target monitoring image is inferred to be each of the plurality of predetermined image safety hazard categories; Among the multiple predetermined image safety hazard categories, a predetermined image safety hazard category that meets the confidence value requirement is determined as the inferred image safety hazard category of the target monitoring image.

7. The method according to claim 6, characterized in that The inference image safety hazard category of the target monitoring image is obtained by inference based on the safety hazard identification neural network; the safety hazard identification neural network is adjusted by the following steps: Acquire a learning sample of an electrical surveillance image, a hidden danger-related image learning sample set associated with the learning sample of the electrical surveillance image, and a priori labels of image safety hidden danger categories; The hidden danger related image learning sample set includes at least one hidden danger related image learning sample, and the image safety hidden danger category priori mark is used to indicate the image safety hidden danger category of the electrical monitoring image learning sample; The image representation vectors of the electrical monitoring image learning samples and the image representation vectors of the image learning samples related to each hazard are respectively extracted through the representation information extraction layer in the safety hazard identification neural network to be adjusted; Through the representation information adjustment layer in the safety hazard identification neural network to be adjusted, the image representation vector of the electrical monitoring image learning sample is adjusted based on the image representation vector of each hazard-related image learning sample to obtain an inference representation vector of the electrical monitoring image learning sample; By using the image safety hazard category inference layer in the safety hazard identification neural network to be calibrated, confidence inference is performed on the inference representation vector of the electrical monitoring image learning sample to obtain the inference confidence that the electrical monitoring image learning sample is inferred as each of the multiple predetermined image safety hazard categories; Based on the error between the reasoning confidence corresponding to the learning sample of the electrical monitoring image and the prior label of the image safety hazard category, the parameters of the safety hazard identification neural network to be calibrated are optimized to obtain the safety hazard identification neural network.

8. The method according to claim 7, characterized in that The confidence reasoning is performed on the reasoning representation vector of the target monitoring image to obtain the reasoning confidence that the target monitoring image is inferred as each of the plurality of predetermined image safety hazard categories, including: Performing a linear transformation on the inference representation vector of the target monitoring image based on the representation information mapping parameter to obtain a linear transformation result; The obtained linear transformation result is used with a preset classifier to obtain the inference confidence that the target monitoring image is inferred to be each of a plurality of predetermined image safety hazard categories; wherein the representation information mapping parameter is a network parameter of the safety hazard identification neural network, the network parameter is obtained when the safety hazard identification neural network to be calibrated, and the classifier is a softmax classifier.

9. An identification system, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps in the method according to any one of claims 1 to 8 are implemented.

Images