Distribution network electrical equipment defect identification method based on image voiceprint multi-modal data

Abstract

The invention provides a distribution network electrical equipment defect identification method based on image voiceprint multi-modal data. Comprising the following steps: constructing an image complex mapping model according to a mapping relation between a light distribution signal and a light source position and a light source light intensity, and establishing first relation knowledge among background light intensity, local light intensity change polarity, absolute brightness change and equipment internal defect characteristics; constructing a voiceprint complex mapping model according to the mapping relationship between the voiceprint signal and the internal defect feature of the equipment, and establishing second relationship knowledge between the local sound pressure change polarity, the frequency change, the absolute sound pressure change and the internal defect feature of the equipment according to the voiceprint complex mapping model; and constructing an electrical equipment defect identification model according to the image complex mapping model, the first relation knowledge, the voiceprint complex mapping model and the second relation knowledge, and identifying the defect of the distribution network electrical equipment by adopting the electrical equipment defect identification model. The problem that an existing equipment defect identification method is low in defect identification accuracy is solved.

Description

Technical Field

[0001] The present application relates to the technical field of distribution network electrical equipment defect identification based on image voiceprint multimodal data, and specifically, to a distribution network electrical equipment defect identification method based on image voiceprint multimodal data, a distribution network electrical equipment defect identification device based on image voiceprint multimodal data, a computer-readable storage medium and an electronic device. Background Art

[0002] The stable operation of the power system is of great significance. Once a power equipment fails, it will affect the safe and stable operation of the power system, and even cause power supply interruption, which will have a major impact on the country and society. If the defects of the power equipment are not discovered and handled in time, it may cause power equipment failure. Therefore, it is important and necessary to manage the defects of power equipment. Defect identification is an important process in the defect management of power equipment. Maintenance personnel usually collect the working status of power equipment through daily or special inspections to find defects. If the defects cannot be identified in a timely, accurate and comprehensive manner, it will directly affect the subsequent defect management work. Therefore, it is urgent to design an electrical equipment defect identification method based on image voiceprint multimodal data fusion to meet the challenges in power equipment management and promote the intelligent development of power systems.

[0003] However, the existing electrical equipment defect recognition methods still have the following two shortcomings: First, the existing technology does not fully consider the equipment characteristics. The established relationship is not equipment-characterized, resulting in the failure to effectively utilize the unique information of the equipment. In addition, the method cannot construct the relationship knowledge between the equipment-characterized characteristic signal and the equipment defect based on the occlusion inside the equipment and the unique structural information inside the equipment. This lack of targeted information integration limits the accuracy and effectiveness of defect recognition. Second, since the networks of various channels such as images and voiceprints work independently and cannot influence each other, the complementarity of information is not fully utilized, resulting in reduced recognition accuracy. When facing different environments and fault types, it is difficult to flexibly adjust its decision-making basis, thereby affecting the overall adaptability. Channels refer to different ways of using different modal data to identify defects separately. Channels that have not been evaluated for performance may perform poorly under specific fault types. The lack of targeted evaluation may cause some channels to be mistakenly considered reliable, thereby reducing classification accuracy and increasing the risk of misjudgment and missed judgment. Summary of the invention

[0004] The main purpose of the present application is to provide a distribution network electrical equipment defect identification method based on image voiceprint multimodal data, a distribution network electrical equipment defect identification device based on image voiceprint multimodal data, a computer-readable storage medium and an electronic device, so as to at least solve the problem of low defect identification accuracy in existing electrical equipment defect identification methods.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for identifying defects of distribution network electrical equipment based on image voiceprint multimodal data is provided, including: constructing an image complex mapping model according to the mapping relationship between the light distribution signal and the light source position and the light source intensity, and establishing a first relationship knowledge between the background light intensity, the local light intensity change polarity, the absolute brightness change and the internal defect characteristics of the equipment according to the image complex mapping model, wherein the light distribution signal is the distribution signal of the light intensity in the optical fiber bundle; constructing a voiceprint complex mapping model according to the mapping relationship between the voiceprint signal and the internal defect characteristics of the equipment, and establishing a second relationship knowledge between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the equipment according to the voiceprint complex mapping model, wherein the voiceprint signal is the spectrum distribution signal of the sound inside the equipment; constructing an electrical equipment defect recognition model according to the image complex mapping model, the first relationship knowledge, the voiceprint complex mapping model and the second relationship knowledge, and using the electrical equipment defect recognition model to identify defects of distribution network electrical equipment.

[0006] Optionally, an electrical equipment defect recognition model is constructed based on the image complex mapping model, the first relationship knowledge, the voiceprint complex mapping model and the second relationship knowledge, including: determining a first probability distribution of each fault type of the electrical equipment based on the image complex mapping model and the first relationship knowledge, and determining a second probability distribution of each fault type of the electrical equipment based on the voiceprint complex mapping model and the second relationship knowledge; constructing the electrical equipment defect recognition model based on the image complex mapping model, the first probability distribution, the voiceprint complex mapping model and the second probability distribution.

[0007] Optionally, determining a first probability distribution of each fault type of the electrical equipment according to the image complex mapping model and the first relationship knowledge, and determining a second probability distribution of each fault type of the electrical equipment according to the voiceprint complex mapping model and the second relationship knowledge, includes: according to a first formula: Determine the first probability distribution in, is the background light intensity, is the polarity of the local light intensity change, is the absolute brightness change, x ji is the guidance value for the i-th type of fault of the j-th type of electrical equipment, is the original output value of the image complex mapping model; according to the second formula: Determine the second probability distribution in, is the polarity of local sound pressure change, is the frequency change, is the absolute sound pressure change, is the guiding value of the complex voiceprint mapping model, is the fault probability distribution calculated using the Softmax function.

[0008] Optionally, the electrical equipment defect recognition model includes an image recognition channel, a voiceprint recognition channel, and image and voiceprint interleaved neurons. The electrical equipment defect recognition model is used to identify defects in distribution network electrical equipment, including: obtaining information to be detected of the distribution network electrical equipment, inputting the information to be detected into the corresponding image recognition channel and the voiceprint recognition channel respectively, and outputting a first fault probability result, wherein the information to be detected includes background light intensity changes, local light intensity changes, absolute brightness changes, local sound pressure changes, local frequency changes, and absolute sound pressure changes; determining the image fault confidence and voiceprint fault confidence of the distribution network electrical equipment based on the fault probability result, and performing comprehensive channel adaptability weighted processing on the image fault confidence and the voiceprint fault confidence to obtain a probability distribution analysis result, and the probability distribution result is obtained. The rate distribution analysis result is input into the image-voiceprint interleaving neuron to obtain the image-voiceprint adaptive weight, wherein the image-voiceprint adaptive weight includes the image adaptive weight and the voiceprint adaptive weight; the image recognition channel and the voiceprint recognition channel are adjusted according to the image-voiceprint adaptive weight, so that the adjusted image recognition channel and the voiceprint recognition channel output a second fault probability result; the channel validity index is determined according to the second fault probability result, and the voiceprint channel weight and the image channel weight are determined according to the channel validity index, and the defect of the distribution network electrical equipment is determined according to the voiceprint channel weight, the image channel weight and the second fault probability result, wherein the channel validity index includes the image channel validity index and the voiceprint channel validity index.

[0009] Optionally, inputting the probability distribution analysis result into the image voiceprint interleaving neuron to obtain the image voiceprint adaptive weight includes: according to the third formula: Determine the image voiceprint adaptive weight, wherein the image voiceprint adaptive weight includes the image adaptive weight Voiceprint adaptive weight Θ(·) is the range normalization function, is the image fault confidence, is the voiceprint fault confidence.

[0010] Optionally, determining the defect of the distribution network electrical equipment according to the voiceprint channel weight, the image channel weight and the second fault probability result includes: according to a fourth formula: Determine the defect probability distribution of the distribution network electrical equipment And determine the defect of the distribution network electrical equipment according to the defect probability distribution, where w j,image is the image channel weight, w j,s o is the voiceprint channel weight, is the image failure probability distribution in the second failure probability result, is the voiceprint failure probability distribution in the second failure probability result.

[0011] Optionally, before determining the channel effectiveness index according to the second fault probability result, the method further includes: using a range normalization method to standardize the second fault probability result to obtain a standardized second fault probability result.

[0012] According to another aspect of the present application, a distribution network electrical equipment defect identification device based on image voiceprint multimodal data is provided, including: a first construction unit, used to construct an image complex mapping model according to the mapping relationship between the light distribution signal and the light source position and the light source intensity, and establish the first relationship knowledge between the background light intensity, the local light intensity change polarity, the absolute brightness change and the internal defect characteristics of the equipment according to the image complex mapping model, wherein the light distribution signal is the distribution signal of the light intensity in the optical fiber bundle; a second construction unit, used to construct a voiceprint complex mapping model according to the mapping relationship between the voiceprint signal and the internal defect characteristics of the equipment, and establish the second relationship knowledge between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the equipment according to the voiceprint complex mapping model, wherein the voiceprint signal is the spectrum distribution signal of the sound inside the equipment; a third construction unit, used to construct an electrical equipment defect identification model according to the image complex mapping model, the first relationship knowledge, the voiceprint complex mapping model and the second relationship knowledge, and use the electrical equipment defect identification model to identify defects of distribution network electrical equipment.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the distribution network electrical equipment defect identification methods based on image voiceprint multimodal data.

[0014] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the distribution network electrical equipment defect identification methods based on image voiceprint multimodal data.

[0015] By applying the technical solution of the present application, an image complex mapping model is constructed according to the mapping relationship between the light distribution signal and the position and intensity of the light source, and the first relationship knowledge between the background light intensity, the polarity of the local light intensity change, the absolute brightness change and the internal defect characteristics of the equipment is established according to the image complex mapping model, wherein the light distribution signal is the distribution signal of the light intensity in the optical fiber bundle; a voiceprint complex mapping model is constructed according to the mapping relationship between the voiceprint signal and the internal defect characteristics of the equipment, and the second relationship knowledge between the polarity of the local sound pressure change, the frequency change, the absolute sound pressure change and the internal defect characteristics of the equipment is established according to the voiceprint complex mapping model, wherein the voiceprint signal is the spectrum distribution signal of the sound inside the equipment; an electrical equipment defect recognition model is constructed according to the image complex mapping model, the first relationship knowledge, the voiceprint complex mapping model and the second relationship knowledge, and the electrical equipment defect recognition model is used to identify the defects of the electrical equipment in the distribution network. In this way, information can be integrated more effectively, and the accuracy and effectiveness of defect recognition can be improved. This targeted equipment-based relationship knowledge construction ensures more efficient and accurate fault recognition capabilities, and solves the problem of low defect recognition accuracy in the existing electrical equipment defect recognition methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for identifying defects of electrical equipment in a distribution network based on image and voiceprint multimodal data provided in an embodiment of the present application is shown;

[0018] Figure 2 A schematic flow chart of a method for identifying defects of electrical equipment in a distribution network based on image and voiceprint multimodal data according to an embodiment of the present application is shown;

[0019] Figure 3 A schematic diagram of a flow chart of a method for identifying defects of electrical equipment in a distribution network based on image and voiceprint multimodal data according to an embodiment of the present application is shown;

[0020] Figure 4 A schematic diagram of the structure of an electrical equipment defect recognition model with two channels of image and voiceprint neural network interwoven according to an embodiment of the present application is shown;

[0021] Figure 5 A structural block diagram of a distribution network electrical equipment defect identification device based on image voiceprint multimodal data provided according to an embodiment of the present application is shown.

[0022] The above drawings include the following reference numerals:

[0023] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As introduced in the background technology, existing electrical equipment defect identification methods have a low defect identification accuracy. In order to solve the problem of low defect identification accuracy in existing electrical equipment defect identification methods, the embodiments of the present application provide a distribution network electrical equipment defect identification method based on image voiceprint multimodal data, a distribution network electrical equipment defect identification device based on image voiceprint multimodal data, a computer-readable storage medium and an electronic device.

[0028] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a method for identifying defects of distribution network electrical equipment based on image and voiceprint multimodal data according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the distribution network electrical equipment defect identification method based on image voiceprint multimodal data in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above networks include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. The above network specific examples may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a distribution network electrical equipment defect identification method based on image voiceprint multimodal data running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0032] Figure 21 is a flow chart of a method for identifying defects of distribution network electrical equipment based on image and voiceprint multimodal data according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0033] Step S201, constructing an image complex mapping model according to the mapping relationship between the light distribution signal and the light source position and the light source intensity, and establishing first relationship knowledge between the background light intensity, the polarity of the local light intensity change, the absolute brightness change and the internal defect characteristics of the equipment according to the above-mentioned image complex mapping model, wherein the above-mentioned light distribution signal is a distribution signal of the light intensity in the optical fiber bundle;

[0034] Specifically, in order to improve the accuracy of electrical equipment defect identification, the influence of factors such as light reflection and occlusion on the internal defect detection of electrical equipment is considered, and the internal defects of electrical equipment are detected through optical fiber bundles. Based on the neural network, a complex image mapping model of the internal light distribution signal of the electrical equipment and the position and intensity of the light source is constructed. Among them, the light distribution signal is the distribution of light intensity in the optical fiber bundle. The intensity of the reflected light inside the electrical equipment, the area blocked by the internal fault detection of the electrical equipment, and the light distribution signal are input into the neural network as parameters, and feature extraction and parameter analysis are performed in the neural network to output the position of the light source and the intensity of the light source. The background light intensity is the overall light intensity level inside the electrical equipment, and the polarity of the local light intensity change refers to the increase or decrease of the local light intensity. The absolute brightness change refers to the absolute value of the light intensity change. There is a certain correlation between the internal defect characteristics of the equipment and the background light intensity, the polarity of the local light intensity change, and the absolute brightness change. The first relationship knowledge is determined based on this correlation.

[0035] Step S202, constructing a complex voiceprint mapping model according to the mapping relationship between the voiceprint signal and the internal defect characteristics of the above-mentioned device, and establishing second relationship knowledge between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the above-mentioned device according to the complex voiceprint mapping model, wherein the above-mentioned voiceprint signal is a spectrum distribution signal of the sound inside the device;

[0036] Specifically, in order to improve the accuracy of defect detection of distribution network electrical equipment, the influence of factors such as occlusion of voiceprint signals when propagating inside the equipment on defect detection is considered, and the defects inside the distribution network electrical equipment are detected through voiceprint signals. A complex voiceprint mapping model of the internal voiceprint signals of distribution network electrical equipment and the internal defect characteristics of the equipment is constructed based on deep learning. Among them, the voiceprint signal is the spectrum distribution of the sound inside the equipment. The local sound pressure change polarity, frequency change, and absolute sound pressure change inside the equipment are input as parameters into the deep learning network, and feature extraction and parameter analysis are performed in the network to output the internal defect characteristics of the equipment. Based on the constructed complex voiceprint mapping model, the relationship knowledge between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the equipment is established. The internal defect characteristics of the equipment include arc discharge, mechanical wear, poor contact, overheating, etc. There is a certain correlation between the internal defect characteristics of the equipment and the local sound pressure change polarity, frequency change, and absolute sound pressure change.

[0037] Step S203, constructing an electrical equipment defect recognition model based on the above-mentioned image complex mapping model, the above-mentioned first relationship knowledge, the above-mentioned voiceprint complex mapping model and the above-mentioned second relationship knowledge, and using the above-mentioned electrical equipment defect recognition model to identify defects of distribution network electrical equipment.

[0038] Specifically, voiceprint detection is suitable for use in dynamic monitoring environments, such as running equipment such as motors or transformers. By monitoring the changes in acoustic wave signals in real time, voiceprint detection can promptly detect early signs of faults such as wear and looseness, thereby effectively avoiding serious damage to the equipment. On the other hand, image detection is more suitable for detecting defects with obvious visual features, such as surface cracks, corrosion or dirt on equipment such as distribution boxes, transformers and cable connectors. With the help of image processing technology, image detection can perform multi-dimensional analysis to improve the accuracy of defect identification, especially in detection tasks that require high efficiency and high repeatability. This interleaving mechanism and weight adaptive adjustment strategy enable the system to flexibly respond to multimodal data under different working conditions, thereby achieving accurate identification of electrical equipment faults.

[0039] Through this embodiment, an image complex mapping model is constructed according to the mapping relationship between the light distribution signal and the position and light intensity of the light source, and the first relationship knowledge between the background light intensity, the polarity of the local light intensity change, the absolute brightness change and the internal defect characteristics of the equipment is established according to the image complex mapping model, wherein the light distribution signal is the distribution signal of the light intensity in the optical fiber bundle; a voiceprint complex mapping model is constructed according to the mapping relationship between the voiceprint signal and the internal defect characteristics of the equipment, and the second relationship knowledge between the polarity of the local sound pressure change, the frequency change, the absolute sound pressure change and the internal defect characteristics of the equipment is established according to the voiceprint complex mapping model, wherein the voiceprint signal is the spectrum distribution signal of the sound inside the equipment; an electrical equipment defect recognition model is constructed according to the image complex mapping model, the first relationship knowledge, the voiceprint complex mapping model and the second relationship knowledge, and the electrical equipment defect recognition model is used to identify the defects of the electrical equipment of the distribution network. In this way, information can be integrated more effectively, and the accuracy and effectiveness of defect recognition can be improved. This targeted equipment-based relationship knowledge construction ensures more efficient and accurate fault recognition capabilities, and solves the problem of low defect recognition accuracy in the existing electrical equipment defect recognition method.

[0040] In the specific implementation process, an electrical equipment defect recognition model is constructed according to the above-mentioned image complex mapping model, the above-mentioned first relationship knowledge, the above-mentioned voiceprint complex mapping model and the above-mentioned second relationship knowledge, including: determining the first probability distribution of each fault type of the electrical equipment according to the above-mentioned image complex mapping model and the above-mentioned first relationship knowledge, and determining the second probability distribution of each fault type of the electrical equipment according to the above-mentioned voiceprint complex mapping model and the above-mentioned second relationship knowledge; constructing the above-mentioned electrical equipment defect recognition model according to the above-mentioned image complex mapping model, the above-mentioned first probability distribution, the above-mentioned voiceprint complex mapping model and the above-mentioned second probability distribution.

[0041] Specifically, determining a first probability distribution of each fault type of the electrical equipment according to the above-mentioned image complex mapping model and the above-mentioned first relationship knowledge, and determining a second probability distribution of each fault type of the electrical equipment according to the above-mentioned voiceprint complex mapping model and the above-mentioned second relationship knowledge, including: according to the first formula: Determine the first probability distribution above in, is the above background intensity, is the polarity of the local light intensity change, is the absolute brightness change mentioned above, is the guidance value for the i-th type of fault of the j-th type of electrical equipment, is the original output value of the above image complex mapping model; according to the second formula: Determine the second probability distribution above in, is the polarity of local sound pressure change, is the frequency change, is the absolute sound pressure change, is the guiding value of the above complex voiceprint mapping model, is the fault probability distribution calculated using the Softmax function.

[0042] This method is based on the constructed image complex mapping model. The background light intensity at different collection points, the polarity of local light intensity change during discharge and the absolute brightness change are obtained through the light distribution signal. The relationship knowledge between the background light intensity, the polarity of local light intensity change during discharge, the absolute brightness change and the internal defect characteristics of the equipment is further established, and converted into the probability distribution of each fault type; based on the constructed voiceprint complex mapping model, the local sound pressure change polarity, frequency change and absolute sound pressure change at different collection points are obtained through the voiceprint signal, and the relationship knowledge between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the equipment is further calculated and converted into probability distribution; in this way, the constructed electrical equipment defect recognition model can effectively integrate information from multiple sources and improve the recognition ability of different fault categories.

[0043] More specifically, the electrical equipment defect recognition model includes an image recognition channel, a voiceprint recognition channel, and image and voiceprint interleaved neurons. The electrical equipment defect recognition model is used to identify defects in distribution network electrical equipment, including: obtaining information to be detected of the distribution network electrical equipment, inputting the information to be detected into the corresponding image recognition channel and the voiceprint recognition channel respectively, and outputting a first fault probability result, wherein the information to be detected includes background light intensity changes, local light intensity changes, absolute brightness changes, local sound pressure changes, local frequency changes, and absolute sound pressure changes; determining the image fault confidence and voiceprint fault confidence of the distribution network electrical equipment based on the fault probability results, and performing comprehensive channel adaptability weighted processing on the image fault confidence and the voiceprint fault confidence to obtain a probability distribution analysis result, and the above The probability distribution analysis result is input into the above-mentioned image and voiceprint interleaving neurons to obtain the image and voiceprint adaptive weights, wherein the above-mentioned image and voiceprint adaptive weights include image adaptive weights and voiceprint adaptive weights; the above-mentioned image recognition channel and the above-mentioned voiceprint recognition channel are adjusted and processed respectively according to the above-mentioned image and voiceprint adaptive weights, so that the above-mentioned image recognition channel and the above-mentioned voiceprint recognition channel after the adjustment process output a second fault probability result; the channel validity index is determined according to the above-mentioned second fault probability result, and the voiceprint channel weight and the image channel weight are determined according to the above-mentioned channel validity index, and the defects of the above-mentioned distribution network electrical equipment are determined according to the above-mentioned voiceprint channel weight, the above-mentioned image channel weight and the above-mentioned second fault probability result, wherein the above-mentioned channel validity index includes the image channel validity index and the voiceprint channel validity index.

[0044] The method is based on the failure probability results of the outputs of two identification channel networks, and

[0045] Calculate the confidence level of the jth device failure and

[0046] Among them, the higher the confidence value, the stronger the confidence of the network in its output results, and the more accurate the results of equipment defect identification are.

[0047] The confidence values ​​calculated by the two networks are weighted by the channel adaptability to obtain a comprehensive probability distribution analysis. The results of the probability analysis are finally input into the interleaved neurons to obtain adaptive weights. In each neural network, the interleaved neurons are initially in an inactive state. In this way, each network will not interfere with each other when making independent judgments, and the accuracy and independence of the initial results will be maintained. After one network outputs the result, the output probability distribution and confidence information are transmitted to the interleaved neurons of the other network, and the interleaved neurons are activated when receiving the input. The adaptive weight value can affect the final defect recognition results of the two neural networks. When the confidence of a neural network is low and the channel adaptability is poor, the calculated adaptive weight value will decrease, and the corresponding interleaved neural activation level will be low, which will comprehensively consider the results of the two channels; conversely, the obtained probability distribution will be more dependent on a single network. This mechanism allows the two networks to interweave with each other, and the output results obtained will affect each other, thereby improving the accuracy of the final equipment defect recognition.

[0048] The calculated adaptive weights are then input into the interleaved neurons, and the algorithms of the image recognition channel and the voiceprint recognition channel are updated, so that the image recognition channel and the voiceprint recognition channel output more accurate second fault probability results, and then the second fault probability results are standardized using the range normalization method.

[0049] Then calculate and in is the proportion of the probability of the jth device having the i-th type of failure output by the standardized voiceprint channel to all failure types, It is the proportion of the probability of the i-th type of failure occurring in the j-th device output by the standardized image channel among all fault types.

[0050] Subsequently, based on these ratios, D j,sound With D j,image , where D j,sound is the voiceprint channel effectiveness index of the jth device, D j,imageis the image channel effectiveness index of the jth device. The greater the effectiveness, the greater the discreteness of the output probability, indicating that the channel has a higher prediction value for the probability of occurrence of a specific fault and can occupy a greater weight in comprehensive interleaving identification. j,sound With D j,image The calculation process is expressed as:

[0051]

[0052] Where k is a constant, usually set to k = 1 / ln(I). According to the effectiveness index, the voiceprint and image channel weights w corresponding to the jth device can be obtained. j,sound and w j,image , expressed as:

[0053]

[0054] Finally, according to the weight w j,sound and w j,image Get the probability distribution of the interleaved neural network output

[0055]

[0056] The fault with the highest probability is selected as the final defect identification result. By introducing interleaved neurons and adaptive weight settings, this method can dynamically adjust the sensitivity to faults according to the output results and confidence levels of each network, thereby improving the ability to accurately identify defects in electrical equipment under a variety of environments.

[0057] Furthermore, the above probability distribution analysis result is input into the above image voiceprint interleaving neuron to obtain the image voiceprint adaptive weight, including: according to the third formula: Determine the image voiceprint adaptive weight, wherein the image voiceprint adaptive weight includes the image adaptive weight Voiceprint adaptive weight Θ(·) is the range normalization function, is the fault confidence of the above image, is the confidence level of the above voiceprint failure.

[0058] Furthermore, determining the defect of the distribution network electrical equipment according to the voiceprint channel weight, the image channel weight and the second fault probability result includes: according to the fourth formula: Determine the defect probability distribution of the above distribution network electrical equipment And determine the defects of the above distribution network electrical equipment according to the above defect probability distribution, where w j,image is the above image channel weight, w j,s o is the above voiceprint channel weight, is the image failure probability distribution in the above second failure probability result, is the voiceprint failure probability distribution in the above second failure probability result.

[0059] Specifically, before determining the channel effectiveness index according to the second fault probability result, the method further includes: using a range normalization method to perform normalization processing on the second fault probability result to obtain a second fault probability result after normalization processing.

[0060] This method uses the range standardization method to standardize the failure probability results, which can eliminate the dimensional differences between different features and make the data more stable and reliable.

[0061] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the distribution network electrical equipment defect identification method based on image voiceprint multimodal data of the present application will be described in detail below in combination with specific embodiments.

[0062] This embodiment relates to a specific method for identifying defects of distribution network electrical equipment based on image voiceprint multimodal data, such as Figure 3 As shown, it specifically includes the following contents:

[0063] S1. A method for constructing knowledge about the association between multi-dimensional information of signal strength and internal event characteristics of a device: extract features and parse parameters in a neural network, output image and voiceprint feature signals, construct a training sample set based on experimental data, train the above neural network, consider issues such as occlusion of image and voiceprint signals, and deduce through the trained neural network model to construct knowledge about the relationship between image, voiceprint feature signals and internal defect characteristics of the device.

[0064] S1.1. Construct a mapping relationship model between light distribution signal and light source position and light source intensity:

[0065] In order to improve the accuracy of electrical equipment defect recognition, the internal defects of electrical equipment are detected through optical fiber bundles, considering the influence of light reflection, occlusion and other factors on the internal defect detection of electrical equipment. A complex image mapping model of the internal light distribution signal of electrical equipment and the position and intensity of the light source is constructed based on a neural network. Among them, the light distribution signal is the distribution of light intensity in the optical fiber bundle. The intensity of the reflected light inside the electrical equipment, the occluded area of ​​the internal fault detection of the electrical equipment, and the light distribution signal are input as parameters into the neural network, and feature extraction and parameter analysis are performed in the neural network, and then the position and intensity of the light source are output.

[0066] To train the above neural network, multiple experiments were conducted to simulate the internal discharge of electrical equipment using a light-emitting device, and a training sample set was constructed based on the experimental data. The mapping relationship between the light distribution signal inside the electrical equipment and the light source position and light source intensity was deduced through the trained neural network model, thereby ensuring the high accuracy of the mapping relationship between the light distribution signal and the light source position and light source intensity. Based on the neural network fitting, the light distribution signal G and the light source position L and light source intensity Q were obtained. source The mapping relationship is expressed as: (L,Q source )←(S,Q reflection ,G)(1);

[0067] Where S represents the shielded area of ​​the internal fault detection of the electrical equipment, Q reflection Indicates the intensity of reflected light inside electrical equipment.

[0068] S1.2. Establish the knowledge of the relationship between background light intensity, local light intensity change polarity, absolute brightness change and internal defect characteristics of the equipment:

[0069] Based on the constructed image complex mapping model, the relationship between background light intensity, local light intensity change polarity, absolute brightness change and internal defect characteristics of the equipment is established. Background light intensity is the overall light intensity level inside the electrical equipment. Local light intensity change polarity refers to the increase or decrease of local light intensity. Absolute brightness change refers to the absolute value of light intensity change. A total of J types of internal defect characteristics of type I in equipment are considered, and the equipment set is represented by J = {1,...,j,...J}, where j represents the jth type of equipment, and the fault set is represented by in represents the i-th type of fault that occurs in the j-th device. The set of image acquisition points is defined as N image ={1,…,n image ,…,N image}, where N image is the total number of image acquisition points, n image is the image point index variable. There is a certain correlation between the internal defect characteristics of the equipment and the background light intensity, the polarity of local light intensity change, and the absolute brightness change.

[0070] Based on the constructed image complex mapping model, the background light intensity at different collection points is obtained through the light distribution signal. Polarity of local light intensity change during discharge Absolute brightness change Further build The relationship between the internal defect characteristics of the equipment is converted into the probability distribution of each fault type, which is expressed as:

[0071] in, Background light intensity Polarity of local light intensity change Absolute brightness change Internal defect characteristics of the equipment The probability distribution derived from the relationship knowledge between It is the original output value of the image complex mapping model, representing the guidance value for the i-th type of fault of the j-th equipment.

[0072] S1.3. Construct a mapping relationship model between voiceprint signals and internal defect characteristics of the equipment:

[0073] In order to improve the accuracy of defect detection of distribution network electrical equipment, the impact of factors such as occlusion of voiceprint signals when propagating inside the equipment on defect detection is considered, and the defects inside the distribution network electrical equipment are detected through voiceprint signals. Based on deep learning, a complex voiceprint mapping model of the internal voiceprint signals of distribution network electrical equipment and the internal defect characteristics of the equipment is constructed. Among them, the voiceprint signal is the frequency spectrum distribution of the sound inside the equipment. The local sound pressure change polarity, frequency change, and absolute sound pressure change inside the equipment are input as parameters into the deep learning network, and feature extraction and parameter analysis are performed in the network to output the internal defect characteristics of the equipment. The mapping relationship between the voiceprint signal distribution V, sound source position Y, sound source sound intensity A, and Mel-spectrogram feature MEL obtained based on neural network fitting is expressed as:

[0074] (Y,A,MEL)←(V,ΔP,f) (3);

[0075] Among them, ΔP represents the change in sound pressure, and f represents the frequency of the voiceprint signal.

[0076] In order to train the above deep learning network, multiple experiments were conducted by simulating the sound signals generated by internal defects of the equipment, and a training sample set was constructed based on the experimental data. The mapping relationship between the voiceprint signal and the internal defect characteristics of the equipment was deduced through the trained deep learning model, thereby ensuring the high accuracy of the mapping relationship between the voiceprint signal and the internal defect characteristics of the equipment.

[0077] S1.4. Establish the knowledge of the relationship between the polarity of local sound pressure change, frequency change, absolute sound pressure change and the internal defect characteristics of the equipment:

[0078] Based on the constructed complex mapping model of voiceprint, the relationship between the polarity of local sound pressure change, frequency change, absolute sound pressure change and the internal defect characteristics of the equipment is established. The internal defect characteristics of the equipment include arc discharge, mechanical wear, poor contact, overheating, etc. There is a certain correlation between the internal defect characteristics of the equipment and the polarity of local sound pressure change, frequency change, and absolute sound pressure change. The set of voiceprint collection points is defined as N sound ={1,...,n sound ,...,N sound}, where N soundis the total number of collection points, n sound is an index variable. There is a certain relationship between the internal defect characteristics of electrical equipment and the voiceprint signal, the polarity of local sound pressure change, the frequency change, and the absolute sound pressure change.

[0079] Based on the constructed voiceprint complex mapping model, the polarity of local sound pressure changes at different collection points is obtained through voiceprint signals. Frequency Change Absolute sound pressure change The relationship between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the equipment is further calculated and converted into a probability distribution, which is expressed as:

[0080] in, is the polarity of local sound pressure change Frequency Change Absolute sound pressure change and the internal defect characteristics of the equipment x ji The failure probability obtained by the relationship knowledge between is the guiding value of the complex voiceprint mapping model, The fault probability distribution is calculated using the Softmax function. The Softmax function can convert the guidance value of each type of fault into a probability distribution, which is suitable for multi-device defect identification scenarios.

[0081] This embodiment performs feature extraction in a neural network and outputs image and voiceprint feature signals adapted to the equipment environment. Next, considering issues such as occlusion inside the device and the unique structural information of the device, a complex image mapping model and a complex voiceprint mapping model are constructed to ensure that the established relationship can reflect the knowledge system of equipment characterization. Finally, the trained image and voiceprint complex mapping models are used for deduction to construct the relationship knowledge between image and voiceprint feature signals and equipment defects, which is converted into a probability distribution model of equipment failure. This method can more effectively integrate information and improve the accuracy and effectiveness of defect identification. This targeted equipment-based relationship knowledge construction ensures more efficient and accurate fault identification capabilities.

[0082] S2. Distribution network electrical equipment defect recognition method based on image and voiceprint channel interweaving:

[0083] By introducing interleaved neurons, the image recognition channel and the voiceprint recognition channel are interwoven and combined. By combining the channel adaptability and confidence value, an electrical equipment defect interleaved identification method based on the probability distribution information is obtained, which enhances the accuracy of electrical equipment defect identification.

[0084] S2.1. Construct a two-channel interleaving model of image and voiceprint neural network:

[0085] In the fault identification system, the interweaving method of the two neural network channels of image and voiceprint is adopted to achieve more accurate fault identification. The interweaving model of the two channels of image and voiceprint neural network is as follows: Figure 4 As shown, the specific structure is introduced as follows:

[0086] First, the image recognition channel contains an image complex mapping model to process the input image data. The channel consists of an input layer, a hidden layer, and an output layer. In the input layer, the image data is converted into a format suitable for model processing. The hidden layer consists of multiple convolutional layers and fully connected layers, which are responsible for extracting image features and capturing important information. Finally, in the output layer, the recognition probability of each fault is generated, indicating the possibility of the fault corresponding to the image data.

[0087] Meanwhile, the voiceprint recognition channel contains a complex mapping model of voiceprints to analyze the input sound data. Its structure also includes input layer, hidden layer and output layer. The input layer of the voiceprint converts the audio signal into a feature vector, and the hidden layer processes these features to identify patterns in the sound. Finally, in the output layer, the fault probability associated with the voiceprint data is calculated, which reflects the possibility of fault occurrence obtained through sound analysis.

[0088] On this basis, the interleaved neuron mechanism is introduced so that the output of one network can affect the decision-making process of another network. The input of an interleaved neuron of a neural network is the device feature information, channel adaptability and the output result of another channel.

[0089] Among them, channel adaptability analysis can improve the adaptability of electrical equipment defect identification in different environments, so that in a specific environment, the network that shows higher accuracy has a greater weight in the final identification. For example, voiceprint detection is suitable for use in dynamic monitoring environments, such as running equipment such as motors or transformers. By monitoring the changes in acoustic wave signals in real time, voiceprint detection can promptly detect early signs of faults such as wear and looseness, thereby effectively avoiding serious damage to the equipment. On the other hand, image detection is more suitable for detecting defects with obvious visual features, such as surface cracks, corrosion or dirt on equipment such as distribution boxes, transformers and cable connectors. With the help of image processing technology, image detection can perform multi-dimensional analysis to improve the accuracy of defect identification, especially in detection tasks that require high efficiency and high repeatability. This interleaving mechanism and weight adaptive adjustment strategy enable the system to flexibly respond to multimodal data under different working conditions, thereby achieving accurate identification of electrical equipment faults.

[0090] When a network shows a significantly higher probability of a certain fault than other faults in fault detection, this indicates that the network has a higher accuracy in identifying that specific fault. At this point, the output of the network can fully activate the interleaved neurons, enabling them to obtain more reliable knowledge and conclusions. If the network's output probability distribution does not have a significant bias, this indicates that the uncertainty of the recognition result is high. In this case, the activation level of the interleaved neurons is low, and it is not possible to fully rely on the judgment of a single network. Therefore, the interleaved neurons will be more inclined to comprehensively consider the opinions of the two networks. At this point, the interleaved neurons can extract effective information from the output of the two networks to form a more comprehensive and reliable judgment, thereby improving the system's adaptability in complex environments.

[0091] S2.2. Electrical equipment defect identification method based on output probability distribution:

[0092] The neural network process of this defect recognition method is divided into two main stages. The first stage is called interleaved neuron activation. In this stage, the interleaved neurons are independent of the neural network, analyze the preliminary output of the network, integrate information from multiple sources by introducing channel adaptability, and calculate the confidence value of each fault category. Subsequently, these confidence values ​​are combined with channel adaptability to form a dynamically adjusted weight, which is input into the interleaved neurons to complete their activation. The second stage is called defect recognition judgment. In this stage, the input information is expanded to include weighted equipment characteristics, channel adaptability, and output results from another channel. Interleaved neurons use the confidence weights obtained in the first stage to exert a weighted influence on the output results of another channel. Finally, all weighted inputs are combined and processed by a weighting method based on the output probability information amount, and finally a series of fault probability distributions are output. The effective combination of these two stages enables the model to more accurately identify equipment defects in complex environments and make reliable decisions.

[0093] Interwoven neuron activation: Channel adaptability refers to the adaptability of two channels to identify various types of faults occurring in various devices. The higher the channel adaptability, the more suitable the channel is for defect identification of this type of fault in this scenario. It can be set based on historical data or expert experience. Through channel adaptability analysis, the adaptability weight E can be obtained. ji,image and E ji,sound , which enables the neural network to adapt to different environments and fault types. In specific environments, the network channels that show higher accuracy have a greater weight in the final identification.

[0094] The failure probability based on the output of the two networks, and

[0095] Calculate the confidence level of the jth device failure and

[0096] Among them, the higher the confidence value, the stronger the confidence of the network in its output results, and the more accurate the results of equipment defect identification are.

[0097] The confidence values ​​calculated by the two networks are weighted by the channel adaptability to obtain a comprehensive probability distribution analysis. The results of the probability analysis are finally input into the interleaved neurons to obtain the adaptive weights. and

[0098]

[0099] Where Θ(·) is the range normalization function. In each neural network, the interleaved neurons are initially in an inactive state. In this way, each network will not interfere with each other when making independent judgments, maintaining the accuracy and independence of the initial results. When one network outputs a result, the output probability distribution and confidence information are transmitted to the interleaved neurons of the other network, and the interleaved neurons are activated when receiving the input. The adaptive weight value can affect the final defect recognition results of the two neural networks. When the confidence of a neural network is low and the channel adaptability is poor, the calculated adaptive weight value will decrease, and the corresponding interleaved neural activation level will be low, which will comprehensively consider the results of the two channels; conversely, the obtained probability distribution will be more dependent on a single network. This mechanism allows the two networks to interweave with each other, and the output results obtained influence each other, thereby improving the accuracy of the final equipment defect recognition.

[0100] Defect fault interleaving identification: After the three inputs are input into the interleaved neurons through the above-mentioned weighted method, the channel output results are weighted using a weighting method based on the output probability information amount. The steps are as follows.

[0101] First, the output results of the two neural networks are standardized using the range normalization method.

[0102] Next, calculate and in is the proportion of the probability of the jth device having the i-th type of failure output by the standardized voiceprint channel to all types of failures, p image (x ji ) is the proportion of the probability of the j-th device having the i-th type of failure output by the standardized image channel to all failure types.

[0103] Subsequently, based on these ratios, D j,sound With D j,image , where D j,soundis the voiceprint channel effectiveness index of the jth device, D j,image is the image channel effectiveness index of the jth device. The greater the effectiveness, the greater the discreteness of the output probability, indicating that the channel has a higher prediction value for the probability of occurrence of a specific fault and can occupy a greater weight in comprehensive interleaving identification. j,sound With D j,image The calculation process is expressed as:

[0104]

[0105] Where k is a constant, usually set to k = 1 / ln(I). According to the effectiveness index, the voiceprint and image channel weights w corresponding to the jth device can be obtained. j,sound and w j,image , expressed as:

[0106]

[0107] Finally, according to the weight w j,sound and w j,image Get the probability distribution of the interleaved neural network output

[0108]

[0109] The fault with the highest probability is selected as the final defect identification result. By introducing interleaved neurons and adaptive weight settings, this method can dynamically adjust the sensitivity to faults according to the output results and confidence levels of each network, thereby improving the ability to accurately identify defects in electrical equipment under a variety of environments.

[0110] This embodiment first uses the device characteristic signal as input in the activation stage of the interleaved neuron to generate a preliminary fault probability distribution, calculate the confidence value of each fault category, and form a dynamically adjusted weight in combination with the channel adaptability, thereby activating the interleaved neuron, enabling it to effectively integrate information from multiple sources and improve the ability to identify different fault categories. Secondly, the input information of each channel is expanded to include weighted device characteristics, channel adaptability, and the output results of another channel. The confidence weight obtained in the first stage is used to apply an interleaved influence to the output of another channel, so that the results with high adaptability and high accuracy are increased in weight, while the results with low adaptability and uncertainty are reduced in weight. Finally, the probability distribution results obtained based on the interleaved input state are processed by a weighting method based on the output probability information to generate defect recognition results. This process ensures that the model can more accurately identify equipment defects in complex environments and make more reliable decisions.

[0111] The embodiment of the present application also provides a distribution network electrical equipment defect identification device based on image voiceprint multimodal data. It should be noted that the distribution network electrical equipment defect identification device based on image voiceprint multimodal data of the embodiment of the present application can be used to execute the distribution network electrical equipment defect identification method based on image voiceprint multimodal data provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0112] The following is an introduction to the distribution network electrical equipment defect identification device based on image voiceprint multimodal data provided in an embodiment of the present application.

[0113] Figure 5 Schematic diagram of a distribution network electrical equipment defect recognition device based on image voiceprint multimodal data according to an embodiment of the present application. Figure 5 As shown, the device comprises:

[0114] The first construction unit 51 is used to construct an image complex mapping model according to the mapping relationship between the light distribution signal and the light source position and the light source intensity, and to establish first relationship knowledge between the background light intensity, the polarity of the local light intensity change, the absolute brightness change and the internal defect characteristics of the device according to the above-mentioned image complex mapping model, wherein the above-mentioned light distribution signal is a distribution signal of the light intensity in the optical fiber bundle;

[0115] The second construction unit 52 is used to construct a complex voiceprint mapping model according to the mapping relationship between the voiceprint signal and the internal defect characteristics of the above-mentioned device, and establish second relationship knowledge between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the above-mentioned device according to the complex voiceprint mapping model, wherein the above-mentioned voiceprint signal is a spectrum distribution signal of the sound inside the device;

[0116] The third construction unit 53 is used to construct an electrical equipment defect recognition model based on the above-mentioned image complex mapping model, the above-mentioned first relationship knowledge, the above-mentioned voiceprint complex mapping model and the above-mentioned second relationship knowledge, and use the above-mentioned electrical equipment defect recognition model to identify defects of distribution network electrical equipment.

[0117] In this embodiment, the first construction unit is used to construct an image complex mapping model according to the mapping relationship between the light distribution signal and the light source position and the light source intensity, and to establish the first relationship knowledge between the background light intensity, the local light intensity change polarity, the absolute brightness change and the internal defect characteristics of the equipment according to the above-mentioned image complex mapping model, wherein the above-mentioned light distribution signal is the distribution signal of the light intensity in the optical fiber bundle; the second construction unit is used to construct a voiceprint complex mapping model according to the mapping relationship between the voiceprint signal and the internal defect characteristics of the equipment, and to establish the second relationship knowledge between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the equipment according to the above-mentioned voiceprint complex mapping model, wherein the above-mentioned voiceprint signal is the spectrum distribution signal of the sound inside the equipment; the third construction unit is used to construct an electrical equipment defect recognition model according to the above-mentioned image complex mapping model, the above-mentioned first relationship knowledge, the above-mentioned voiceprint complex mapping model and the above-mentioned second relationship knowledge, and use the above-mentioned electrical equipment defect recognition model to identify the defects of the electrical equipment in the distribution network. This can more effectively integrate information and improve the accuracy and effectiveness of defect identification. This targeted equipment-based relationship knowledge construction ensures more efficient and accurate fault identification capabilities and solves the problem of low defect identification accuracy in existing electrical equipment defect identification methods.

[0118] As an optional scheme, the third construction unit includes a first determination module and a first construction module; the first determination module is used to determine the first probability distribution of each fault type of the electrical equipment based on the above-mentioned image complex mapping model and the above-mentioned first relationship knowledge, and determine the second probability distribution of each fault type of the electrical equipment based on the above-mentioned voiceprint complex mapping model and the above-mentioned second relationship knowledge; the first construction module is used to construct the above-mentioned electrical equipment defect recognition model based on the above-mentioned image complex mapping model, the above-mentioned first probability distribution, the above-mentioned voiceprint complex mapping model and the above-mentioned second probability distribution.

[0119] In an optional solution, the first determination module includes a first determination submodule and a second determination submodule; the first determination submodule is used to determine the first formula: Determine the first probability distribution above in, is the above background intensity, is the polarity of the local light intensity change, is the absolute brightness change mentioned above, is the guidance value for the i-th type of fault of the j-th type of electrical equipment, is the original output value of the above-mentioned image complex mapping model; the second determination submodule is used to determine the value according to the second formula: Determine the second probability distribution above in, is the polarity of local sound pressure change, is the frequency change, is the absolute sound pressure change, is the guiding value of the above complex voiceprint mapping model, is the fault probability distribution calculated using the Softmax function.

[0120] An optional solution, the third construction unit also includes an acquisition module, a second determination module, an adjustment processing module and a third determination module; the acquisition module is used to acquire the information to be detected of the above-mentioned distribution network electrical equipment, input the above-mentioned information to be detected into the corresponding above-mentioned image recognition channel and the above-mentioned voiceprint recognition channel respectively, and output a first fault probability result, and the information to be detected includes background light intensity changes, local light intensity changes, absolute brightness changes, local sound pressure changes, local frequency changes and absolute sound pressure changes; the second determination module is used to determine the image fault confidence and voiceprint fault confidence of the above-mentioned distribution network electrical equipment based on the above-mentioned fault probability results, and perform comprehensive channel adaptability weighted processing on the above-mentioned image fault confidence and the above-mentioned voiceprint fault confidence to obtain a probability distribution analysis result, and input the above-mentioned probability distribution analysis result into the above-mentioned image and voiceprint interlaced In the neuron, the image voiceprint adaptive weight is obtained, wherein the image voiceprint adaptive weight includes the image adaptive weight and the voiceprint adaptive weight; the adjustment processing module is used to adjust the image recognition channel and the voiceprint recognition channel respectively according to the image voiceprint adaptive weight, so that the image recognition channel and the voiceprint recognition channel after the adjustment process output the second fault probability result; the third determination module is used to determine the channel validity index according to the second fault probability result, and determine the voiceprint channel weight and the image channel weight according to the channel validity index, and determine the defect of the distribution network electrical equipment according to the voiceprint channel weight, the image channel weight and the second fault probability result, wherein the channel validity index includes the image channel validity index and the voiceprint channel validity index.

[0121] In an optional solution, the second determination module includes a third determination submodule, which is used to determine the value according to a third formula: Determine the image voiceprint adaptive weight, wherein the image voiceprint adaptive weight includes the image adaptive weight Voiceprint adaptive weight Θ(·) is the range normalization function, is the fault confidence of the above image, is the confidence level of the above voiceprint failure.

[0122] In an optional solution, the third determination module includes a fourth determination submodule, which is used to determine the value according to a fourth formula: Determine the defect probability distribution of the above distribution network electrical equipment And determine the defects of the above distribution network electrical equipment according to the above defect probability distribution, where w j,imageis the above image channel weight, w j,s o is the above voiceprint channel weight, is the image failure probability distribution in the above second failure probability result, is the voiceprint failure probability distribution in the above second failure probability result.

[0123] In an optional solution, the third construction unit also includes a standardization processing module, which is used to standardize the above-mentioned second fault probability result using a range standardization method before determining the channel effectiveness index based on the above-mentioned second fault probability result to obtain the standardized second fault probability result.

[0124] The above-mentioned distribution network electrical equipment defect identification device based on image voiceprint multimodal data includes a processor and a memory. The above-mentioned first construction unit, second construction unit, third construction unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in the form of any combination.

[0125] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem of low defect recognition accuracy in the existing electrical equipment defect recognition method can be solved by adjusting kernel parameters.

[0126] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0127] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the distribution network electrical equipment defect identification method based on image voiceprint multimodal data.

[0128] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the distribution network electrical equipment defect identification method based on image voiceprint multimodal data when running.

[0129] An embodiment of the present invention provides an electronic device, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the steps of the above-mentioned method for identifying defects of distribution network electrical equipment based on image voiceprint multimodal data are implemented. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0130] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes at least the steps of the above-mentioned distribution network electrical equipment defect identification method based on image voiceprint multimodal data.

[0131] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0132] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0137] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0138] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0139] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0140] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying defects of distribution network electrical equipment based on image voiceprint multimodal data, characterized in that: include: An image complex mapping model is constructed according to the mapping relationship between the light distribution signal and the light source position and the light source intensity, and first relationship knowledge between the background light intensity, the polarity of the local light intensity change, the absolute brightness change and the internal defect characteristics of the device is established according to the image complex mapping model, wherein the light distribution signal is a distribution signal of the light intensity in the optical fiber bundle; A complex voiceprint mapping model is constructed according to the mapping relationship between the voiceprint signal and the internal defect characteristics of the device, and second relationship knowledge between the local sound pressure change polarity, frequency change, absolute sound pressure change and the internal defect characteristics of the device is established according to the complex voiceprint mapping model, wherein the voiceprint signal is a frequency spectrum distribution signal of the sound inside the device; An electrical equipment defect recognition model is constructed based on the image complex mapping model, the first relationship knowledge, the voiceprint complex mapping model and the second relationship knowledge, and the electrical equipment defect recognition model is used to identify defects of distribution network electrical equipment.

2. The method according to claim 1, characterized in that Constructing an electrical equipment defect recognition model according to the image complex mapping model, the first relationship knowledge, the voiceprint complex mapping model, and the second relationship knowledge, including: Determine a first probability distribution of each fault type of the electrical equipment according to the complex image mapping model and the first relationship knowledge, and determine a second probability distribution of each fault type of the electrical equipment according to the complex voiceprint mapping model and the second relationship knowledge; The electrical equipment defect recognition model is constructed according to the image complex mapping model, the first probability distribution, the voiceprint complex mapping model and the second probability distribution.

3. The method according to claim 2, characterized in that Determining a first probability distribution of each fault type of the electrical equipment according to the complex image mapping model and the first relationship knowledge, and determining a second probability distribution of each fault type of the electrical equipment according to the complex voiceprint mapping model and the second relationship knowledge, including: According to the first formula: Determine the first probability distribution in, is the background light intensity, is the polarity of the local light intensity change, is the absolute brightness change, is the guidance value for the i-th type of fault of the j-th type of electrical equipment, is the original output value of the image complex mapping model; According to the second formula: Determine the second probability distribution in, is the polarity of local sound pressure change, is the frequency change, is the absolute sound pressure change, is the guiding value of the complex voiceprint mapping model, is the fault probability distribution calculated using the Softmax function.

4. The method according to claim 1, characterized in that: in, The electrical equipment defect recognition model includes an image recognition channel, a voiceprint recognition channel, and an image and voiceprint interlaced neuron. The electrical equipment defect recognition model is used to identify defects of distribution network electrical equipment, including: Acquire the information to be detected of the distribution network electrical equipment, input the information to be detected into the corresponding image recognition channel and the voiceprint recognition channel respectively, and output a first fault probability result, wherein the information to be detected includes background light intensity change, local light intensity change, absolute brightness change, local sound pressure change, local frequency change and absolute sound pressure change; Based on the fault probability result, the image fault confidence and the voiceprint fault confidence of the distribution network electrical equipment are determined, and the image fault confidence and the voiceprint fault confidence are subjected to comprehensive channel adaptability weighting processing to obtain a probability distribution analysis result, and the probability distribution analysis result is input into the image voiceprint interleaving neuron to obtain an image voiceprint adaptive weight, wherein the image voiceprint adaptive weight includes an image adaptive weight and a voiceprint adaptive weight; The image recognition channel and the voiceprint recognition channel are adjusted according to the image and voiceprint adaptive weights, so that the image recognition channel and the voiceprint recognition channel after the adjustment process output a second fault probability result; A channel validity index is determined according to the second fault probability result, and a voiceprint channel weight and an image channel weight are determined according to the channel validity index, and the defect of the distribution network electrical equipment is determined according to the voiceprint channel weight, the image channel weight and the second fault probability result, wherein the channel validity index includes an image channel validity index and a voiceprint channel validity index.

5. The method according to claim 4, characterized in that Inputting the probability distribution analysis result into the image voiceprint interleaving neuron to obtain the image voiceprint adaptive weight includes: According to the third formula: Determine the image voiceprint adaptive weight, wherein the image voiceprint adaptive weight includes the image adaptive weight Voiceprint adaptive weight Θ(·) is the range normalization function, is the image fault confidence, is the voiceprint fault confidence.

6. The method according to claim 4, characterized in that Determining the defect of the distribution network electrical equipment according to the voiceprint channel weight, the image channel weight and the second fault probability result includes: According to the fourth formula: Determine the defect probability distribution ξ of the distribution network electrical equipment final (x ji ), and determine the defects of the distribution network electrical equipment according to the defect probability distribution, where w j,image is the image channel weight, w j,sound is the voiceprint channel weight is the image failure probability distribution in the second failure probability result, is the voiceprint failure probability distribution in the second failure probability result.

7. The method according to claim 4, characterized in that Before determining the channel effectiveness index according to the second failure probability result, the method further includes: The second fault probability result is standardized by using a range standardization method to obtain a standardized second fault probability result.

8. A distribution network electrical equipment defect recognition device based on image voiceprint multimodal data, characterized in that: include: A first construction unit is used to construct an image complex mapping model according to the mapping relationship between the light distribution signal and the light source position and the light source intensity, and to establish first relationship knowledge between the background light intensity, the polarity of the local light intensity change, the absolute brightness change and the internal defect characteristics of the device according to the image complex mapping model, wherein the light distribution signal is a distribution signal of the light intensity in the optical fiber bundle; a second construction unit, configured to construct a complex voiceprint mapping model according to a mapping relationship between a voiceprint signal and the internal defect characteristics of the device, and to establish second relationship knowledge between a local sound pressure change polarity, a frequency change, an absolute sound pressure change and the internal defect characteristics of the device according to the complex voiceprint mapping model, wherein the voiceprint signal is a frequency spectrum distribution signal of the sound inside the device; The third construction unit is used to construct an electrical equipment defect recognition model based on the image complex mapping model, the first relationship knowledge, the voiceprint complex mapping model and the second relationship knowledge, and use the electrical equipment defect recognition model to identify defects of distribution network electrical equipment.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the distribution network electrical equipment defect identification method based on image voiceprint multimodal data as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the distribution network electrical equipment defect identification method based on image voiceprint multimodal data as described in any one of claims 1 to 7.

Images