Gearbox fault detection method, device, equipment, storage medium and program product

Abstract

This application discloses a gearbox fault detection method, apparatus, device, storage medium, and program product, relating to the field of wind power generation technology. The method includes: acquiring an image to be detected, which is obtained by photographing the interior of the gearbox; performing semantic segmentation on the image to be detected using a semantic segmentation model to obtain fault regions in the image; determining whether the image to be detected contains a fault type corresponding to the semantic segmentation model based on the fault regions, and generating a fault type identification result, thereby obtaining a fault detection result for the image to be detected based on the identification results corresponding to each fault type. This approach can improve the efficiency and accuracy of fault detection.

Description

Technical Field

[0001] This application relates to the field of wind power generation technology, and in particular to a gearbox fault detection method, device, equipment, storage medium, and program product. Background Technology

[0002] As a crucial component of wind turbine generators, the gearbox's operational status directly impacts the generator's power generation efficiency and safety. Gearbox malfunctions can not only lead to generator shutdowns for maintenance but also potentially trigger more serious safety accidents, threatening the safety of personnel and equipment. Therefore, gearbox fault detection is of paramount importance for improving the safety and reliability of wind turbine generators.

[0003] In related technologies, gearbox fault detection typically employs manual diagnosis. Specifically, this involves using an endoscope to capture images of the gearbox's interior, followed by manual visual inspection to determine the presence and type of fault. However, this method is time-consuming, relies heavily on human experience, and is prone to misjudgments or omissions due to human error or lack of experience, resulting in low efficiency and accuracy in fault detection. Summary of the Invention

[0004] This application provides a gearbox fault detection method, apparatus, device, storage medium, and program product, which can improve the efficiency and accuracy of fault detection.

[0005] In a first aspect, embodiments of this application provide a gearbox fault detection method, comprising: acquiring an image to be detected, the image to be detected being obtained by photographing the interior of a gearbox; performing semantic segmentation on the image to be detected using a semantic segmentation model to obtain fault regions in the image to be detected; determining whether the image to be detected has a fault type corresponding to the semantic segmentation model based on the fault regions, and generating a fault type identification result, so as to obtain a fault detection result of the image to be detected based on the identification results corresponding to each fault type.

[0006] Optionally, a semantic segmentation model is used to perform semantic segmentation on the image to be detected to obtain fault regions in the image to be detected, including: inputting the image to be detected into the semantic segmentation model for semantic segmentation to obtain a grayscale image of the image to be detected, the grayscale image being used to indicate the fault probability of each pixel in the image to be detected; based on the fault probability of each pixel, the grayscale image is binarized according to a first threshold to obtain a binarized image, the binarized image being used to indicate fault regions in the image to be detected.

[0007] Optionally, based on the fault region, determine whether the image to be detected has a fault type corresponding to the semantic segmentation model, and generate a fault type identification result, including: eliminating noise regions in the binarized image through an erosion algorithm to obtain fault regions, the fault regions including one or more connected regions; determining whether the number of pixels in the largest connected region in each connected region is greater than a second threshold, if it is greater than the second threshold, then determining that the image to be detected has a fault type corresponding to the semantic segmentation model, and the identification result is that a fault type exists.

[0008] Optionally, based on the fault region, determine whether the image to be detected has a fault type corresponding to the semantic segmentation model, and generate a fault type identification result, including: calculating the total number of pixels in the fault region and determining whether the total is greater than a third threshold; if it is greater than the third threshold, determine that the image to be detected has a fault type corresponding to the semantic segmentation model, and the identification result is that a fault type exists.

[0009] Optionally, the fault detection result of the image to be detected is obtained based on the recognition result corresponding to each fault type, including: when the recognition result indicates that a fault type exists, the recognition result is used as the fault detection result of the image to be detected; when the recognition result indicates that no fault type exists, the semantic segmentation model of the next priority fault type is used to perform semantic segmentation on the image to be detected, wherein different fault types have different preset priorities.

[0010] Optionally, the fault detection results of the image to be detected can be obtained based on the recognition results corresponding to each fault type, including: integrating the recognition results corresponding to each fault type to obtain the fault detection results of the image to be detected.

[0011] Secondly, embodiments of this application provide a gearbox fault detection device, comprising: an image acquisition module for acquiring an image to be detected, wherein the image to be detected is obtained by photographing the interior of the gearbox; a fault identification module for performing semantic segmentation on the image to be detected using a semantic segmentation model to obtain fault regions in the image to be detected; and a result acquisition module for determining whether the image to be detected has a fault type corresponding to the semantic segmentation model based on the fault regions, and generating a fault type identification result, thereby obtaining a fault detection result of the image to be detected based on the identification results corresponding to each fault type.

[0012] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory storing program instructions, wherein the processor is configured to execute the gearbox fault detection method described above when running the program instructions.

[0013] Fourthly, embodiments of this application provide a storage medium storing program instructions, wherein the program instructions, when executed, perform the gearbox fault detection method as described above.

[0014] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements the gearbox fault detection method described above.

[0015] The gearbox fault detection method provided in this application acquires an image of the inside of the gearbox as the image to be detected, and uses a semantic segmentation model to obtain the fault region in the image to be detected. Further, based on the fault region, it determines whether the image to be detected contains a corresponding fault type to generate a recognition result. Finally, based on the recognition results corresponding to each fault type, it obtains the fault detection result of the image to be detected. This approach, using a semantic segmentation model to identify the fault region in the image to be detected to determine whether the image contains a corresponding fault type, enables rapid and high-precision detection of image faults. Compared to manual review, it eliminates the need for manual inspection of each image, avoiding misjudgments and omissions due to human negligence or lack of experience, thus improving the efficiency and accuracy of fault detection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a gearbox fault detection method provided in an embodiment of this application; Figure 2 This is a flowchart of another gearbox fault detection method provided in the embodiments of this application; Figure 3 This is a schematic diagram of an embodiment of the gearbox fault detection process provided in this application; Figure 4 This is a schematic diagram of another embodiment of the gearbox fault detection process provided in this application; Figure 5 This is a schematic diagram of yet another embodiment of the gearbox fault detection process provided in this application; Figure 6 This is a schematic diagram of a gearbox fault detection device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0019] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0020] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0021] In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0022] As a crucial component of wind turbine generators, the gearbox's operational status directly impacts the generator's power generation efficiency and safety. Gearbox malfunctions can not only lead to generator shutdowns for maintenance but also potentially trigger more serious safety accidents, threatening the safety of personnel and equipment. Therefore, gearbox fault detection is of paramount importance for improving the safety and reliability of wind turbine generators.

[0023] In related technologies, gearbox fault detection typically employs manual diagnostics. Specifically, this involves using an endoscope to capture images of the gearbox's interior. These images are then visually inspected to determine the presence and type of fault. This relies on the inspector's professional experience to observe the images, identify fault characteristics, and determine the fault type. However, this method is time-consuming and dependent on human experience, making it prone to misjudgments and omissions due to human error or insufficient experience, resulting in low efficiency and accuracy in fault detection.

[0024] Based on this, embodiments of this application provide a gearbox fault detection method, apparatus, device, storage medium, and program product. By utilizing a semantic segmentation model to identify fault regions in the image to be detected, and thus identifying whether the image to be detected contains a corresponding fault type, rapid and high-precision detection of image faults can be achieved. Compared to manual review, it eliminates the need for manual inspection of each image for faults, avoiding misjudgments and omissions caused by human negligence or lack of experience, thereby improving the efficiency and accuracy of fault detection.

[0025] The subject of this application is an electronic device, such as a server, laptop, desktop computer, mobile phone, tablet computer, etc., and the embodiments of this application are not limited thereto.

[0026] The permission control method of this application embodiment will now be described in detail. For example, please refer to... Figure 1 , Figure 1 This is a flowchart illustrating a gearbox fault detection method provided in an embodiment of this application. This embodiment includes: S101: Acquire the image to be detected, which is obtained by photographing the inside of the gearbox.

[0027] In this embodiment, the image to be inspected can be obtained by taking pictures of the inside of the gearbox using an endoscope. For example, a miniature industrial endoscope can be inserted into the gearbox to photograph the gears, bearings, and other components, obtaining photos or videos of each component. The image to be inspected can be a photo taken directly by the endoscope or a video frame extracted from a video captured by the endoscope. Furthermore, images can be taken focusing on areas prone to failure, such as gear meshing surfaces and bearing raceways. Further, the electronic device establishes a communication connection with the endoscope to acquire the image to be inspected transmitted by the endoscope.

[0028] S102: Use a semantic segmentation model to perform semantic segmentation on the image to be detected in order to obtain the fault region in the image to be detected; In this embodiment, the electronic device pre-stores a trained semantic segmentation model. Since there may be multiple fault types within the gearbox, a corresponding trained semantic segmentation model is stored for each fault type. Fault types include, for example, indentation faults, pitting faults, fracture faults, spalling faults, corrosion faults, and crack faults. Specifically, indentation faults refer to localized depressions or pits on the surface of the component; pitting faults refer to dense, fine pits on the surface of the component; fracture faults refer to through-cracks that separate the component; spalling faults refer to flaky or blocky metal detachment from the surface of the component; corrosion faults refer to a yellowish-brown or reddish-brown oxide layer on the surface of the component; and crack faults refer to linear gaps on or inside the surface of the component.

[0029] In this embodiment, the electronic device inputs the image to be detected into a pre-trained semantic segmentation model for semantic segmentation and outputs the fault probability corresponding to each pixel in the image to be detected. The fault probability is used to characterize the likelihood that the corresponding pixel belongs to the fault region, so as to obtain the fault region composed of multiple pixels based on the fault probability of each pixel.

[0030] S103: Determine whether the image to be detected has a fault type corresponding to the semantic segmentation model based on the fault region, and generate the identification result of the fault type, so as to obtain the fault detection result of the image to be detected based on the identification result corresponding to each fault type.

[0031] In this embodiment, after acquiring the fault region in the image to be detected, the electronic device matches and judges the features of the fault region with the features of the corresponding fault type. If the fault region matches the features of the fault type, it is determined that the fault type exists in the image to be detected, and a corresponding recognition result is generated. For example, the image to be detected is input into the semantic segmentation model corresponding to the indentation fault. After acquiring the fault region, it is judged whether the fault region matches the features of the indentation fault. If they match, it is determined that the fault type exists in the image to be detected, and the recognition result is "indentation fault exists".

[0032] Furthermore, the image to be detected may contain multiple fault types. For different fault types, corresponding semantic segmentation models are required to detect the image. Specifically, the electronic device can input the image to be detected into the semantic segmentation models corresponding to each fault type in a preset order, obtain the fault regions output by each semantic segmentation model, and then determine whether the corresponding fault type exists based on the fault regions, generating the corresponding recognition result. Finally, the electronic device combines the recognition results corresponding to multiple fault types to obtain the final fault detection result for the image to be detected. This fault detection result indicates which fault types exist or whether there are no faults in the image. Simultaneously, the electronic device can also visualize the fault detection result, for example, by marking the corresponding position on the image, making it easy for users to quickly understand the gearbox's fault condition and take appropriate measures.

[0033] The gearbox fault detection method provided in this application acquires an image of the inside of the gearbox as the image to be detected, and uses a semantic segmentation model to obtain the fault region in the image to be detected. Further, based on the fault region, it determines whether the image to be detected contains a corresponding fault type to generate a recognition result. Finally, based on the recognition results corresponding to each fault type, it obtains the fault detection result of the image to be detected. This approach, using a semantic segmentation model to identify the fault region in the image to be detected to determine whether the image contains a corresponding fault type, enables rapid and high-precision detection of image faults. Compared to manual review, it eliminates the need for manual inspection of each image, avoiding misjudgments and omissions due to human negligence or lack of experience, thus improving the efficiency and accuracy of fault detection.

[0034] Optionally, in the above embodiments, during the process of the electronic device performing semantic segmentation on the image to be detected using a semantic segmentation model to obtain fault regions in the image to be detected, the electronic device inputs the image to be detected into the semantic segmentation model for semantic segmentation to obtain a grayscale image of the image to be detected. This grayscale image is used to indicate the fault probability of each pixel in the image to be detected. Then, the electronic device can binarize the grayscale image based on the fault probability of each pixel according to a first threshold to obtain a binarized image, which is used to indicate fault regions in the image to be detected.

[0035] Reference Figure 2 As shown, Figure 2 This is a flowchart of another gearbox fault detection method provided in this application embodiment. In this embodiment, the electronic device uses a semantic segmentation model to obtain the fault probability of each pixel in the image to be detected. The fault probability is used to represent the probability that the corresponding pixel belongs to the fault region, and the value range is between 0 and 1. The fault probability of each pixel can be presented in a visual way through a grayscale image. Specifically, the fault probability can be represented by the size of the pixel value. For example, the larger the pixel value, the higher the fault probability of the pixel, and the smaller the pixel value, the lower the fault probability of the pixel.

[0036] Furthermore, the electronic device classifies each pixel using a first threshold to divide the grayscale image into faulty and non-faulty regions. The first threshold is, for example, a value such as 0.8 or 0.9, and can be adjusted according to actual needs. Specifically, the electronic device sets the pixel values ​​of pixels with a fault probability greater than the first threshold to the first pixel value, and sets the pixel values ​​of pixels with a fault probability less than or equal to the first threshold to the second pixel value, thereby generating a binary image containing only the first and second pixel values. The region consisting of pixels with the first pixel value is the faulty region. For example, the first pixel value might be 255, and the second pixel value might be 0. For each pixel in the grayscale image, the electronic device compares the fault probability of that pixel with the first threshold. If the fault probability is greater than the first threshold, the pixel value is set to 255 (white); if the fault probability is less than the first threshold, the pixel value is set to 0 (black). In this case, the faulty region is the white region in the binary image.

[0037] This approach, by setting a first threshold to binarize the grayscale image, can clearly distinguish between faulty and non-faulty areas, making the location of faulty areas more accurate.

[0038] Optionally, in the above embodiments, during the process of the electronic device determining whether the image to be detected contains a fault type corresponding to the semantic segmentation model based on the fault region and generating a fault type identification result, the electronic device eliminates noise regions in the binarized image using an erosion algorithm to obtain the fault region, which includes one or more connected regions. Then, it is determined whether the number of pixels in the largest connected region is greater than a second threshold. If it is greater than the second threshold, it is determined that the image to be detected contains a fault type corresponding to the semantic segmentation model, and the identification result is that the fault type exists.

[0039] In this embodiment, the region composed of pixels with a first pixel value contains multiple connected regions. A connected region refers to a set of pixels with the same pixel value that are interconnected through a specific adjacency relationship. These multiple connected regions include not only faulty regions but also potentially scattered, small connected regions or weakly connected regions, which are considered noise regions. Since these noise regions are not true faulty regions, they can be eliminated using an erosion algorithm, such as the OpenCV erosion algorithm.

[0040] Further, continue to refer to Figure 2After eliminating noise regions, the faulty region consists of one or more remaining connected regions. The electronic device determines whether the number of pixels in the largest connected region among these connected regions is greater than a preset second threshold. This largest connected region refers to the connected region with the most pixels among all connected regions. If the number of pixels in the largest connected region is greater than the second threshold, it indicates the presence of a corresponding fault type, that is, it is determined that the image to be detected contains the fault type corresponding to this semantic segmentation model. For example, if the second threshold is set to 10000, when the number of pixels in the largest connected region is greater than 10000, the electronic device determines that the image to be detected contains the corresponding fault type. Furthermore, the electronic device can set different second thresholds for different fault types to adapt to the feature differences of different faults.

[0041] This approach, which uses an erosion algorithm to eliminate noisy regions and combines the size of connected regions to determine the fault type, can further improve the accuracy and reliability of fault detection.

[0042] Optionally, in the above embodiments, during the process of the electronic device determining whether the image to be detected has a fault type corresponding to the semantic segmentation model based on the fault area and generating the identification result of the fault type, the electronic device calculates the total number of pixels in the fault area and determines whether the total number is greater than the third threshold. If it is greater than the third threshold, it is determined that the image to be detected has a fault type corresponding to the semantic segmentation model, and the identification result is that the fault type exists.

[0043] It should be noted that, when certain faults occur, the fault area may not be concentrated in a single area, but rather scattered across multiple locations in the image. For example, when the fault type is a crack or fracture, the fault area is concentrated in a specific area, while when the fault type is a pitting or indentation fault, the fault characteristics are manifested as scattered small areas with a small number of pixels in each connected region. In such cases, judging the number of pixels in the largest connected region of the fault area cannot accurately identify the fault type. Therefore, in this embodiment, the electronic device calculates the sum of the number of pixels in the fault area, which represents the overall area size of the fault region. If the sum is greater than a preset third threshold, it indicates the existence of the corresponding fault type, that is, it is determined that the image to be detected contains the fault type corresponding to the semantic segmentation model. Furthermore, the electronic device can set different third thresholds for different fault types to adapt to the feature differences of different faults.

[0044] By using this approach, the total number of pixels in the fault area can be calculated and compared with a third threshold, thus identifying fault types with discrete fault area distributions and making the identification of fault types more comprehensive.

[0045] Optionally, in the above embodiments, during the process of obtaining the fault detection result of the image to be detected based on the identification result corresponding to each fault type, when the identification result indicates that the fault type exists, the identification result is used as the fault detection result of the image to be detected; when the identification result indicates that the fault type does not exist, the semantic segmentation model of the next priority fault type is used to perform semantic segmentation on the image to be detected. Different fault types have different preset priorities.

[0046] In this embodiment, after acquiring the image to be detected, the electronic device will sequentially use semantic segmentation models corresponding to different fault types for detection according to a preset priority order. Different fault types have different preset priorities, which can be set from high to low according to the severity of the fault. For example, among the six fault types of indentation, pitting, fracture, peeling, corrosion, and crack, their severity is: fracture > peeling > crack > corrosion > pitting > indentation. Therefore, their priority order is set from high to low as fracture, peeling, crack, corrosion, pitting, and indentation.

[0047] Specifically, the electronic device first selects the semantic segmentation model corresponding to the highest priority fault type, performs semantic segmentation on the image to be detected, and determines whether the fault type exists based on the steps in the above embodiments. After obtaining the recognition result, if the recognition result indicates that the fault type exists, the electronic device directly uses the recognition result as the fault detection result for the image to be detected, without needing to continue detecting subsequent fault types. If the recognition result indicates that the fault type does not exist, the electronic device selects the semantic segmentation model corresponding to the next priority fault type according to a preset priority order, performs semantic segmentation on the image to be detected again, and repeats the steps in the above embodiments until the existing fault type is identified, or until all fault types have been detected.

[0048] For example, such as Figure 3 As shown, Figure 3 This is a schematic diagram of an embodiment of the gearbox fault detection process provided in this application. The electronic device first determines whether the image to be detected contains a fracture fault with the highest priority. If it does, the identification result of "fracture fault exists" is taken as the fault detection result. If it does not exist, the identification result is "fracture fault does not exist," and then it continues to determine whether there is a peeling fault, and so on. If all fault types have been detected and no fault is found, the final fault detection result is no fault.

[0049] By adopting this scheme, electronic devices can detect various faults in images in an orderly manner by pre-setting the priority of fault types. This allows them to output detection results in a timely manner when a fault is detected, thereby improving detection efficiency and saving computing resources.

[0050] Optionally, in the above embodiments, during the process of obtaining the fault detection result of the image to be detected based on the identification results corresponding to each fault type, the electronic device can integrate the identification results corresponding to each fault type to obtain the fault detection result of the image to be detected.

[0051] In this embodiment, after acquiring the image to be detected, the electronic device uses semantic segmentation models corresponding to different fault types to detect the image, obtaining recognition results for each fault type. After obtaining the recognition results for each fault type, the electronic device integrates these recognition results to generate a fault detection result. If a fault exists, the fault detection result contains all existing fault types; if all recognition results indicate that no corresponding fault type exists, the final fault detection result is "no fault exists."

[0052] Reference Figure 4 As shown, Figure 4 This is a schematic diagram of another embodiment of the gearbox fault detection process provided in this application. The electronic device inputs the image to be detected into multiple semantic segmentation models corresponding to different fault types, and detects fault types such as fracture faults, spalling faults, and crack faults in parallel, and outputs the corresponding recognition results respectively. Then, the electronic device will... Figure 4 The identification results of six fault types are summarized: if a fault is detected, the output fault detection result is a fault list, which lists all fault types present in the image to be detected. Otherwise, the final fault detection result is that no fault exists.

[0053] This approach allows for the acquisition of multiple fault types that may exist in the image to be detected at once, without the need to detect them one by one according to priority. By integrating the identification results of each fault type, the fault status of the image to be detected can be more comprehensively reflected.

[0054] Optionally, in another embodiment, fault types can be pre-classified, that is, multiple fault types are divided into different level groups. Fault types within the same level group have the same priority, while fault types in different level groups have different priorities. After the electronic device acquires the image to be detected, it uses the semantic segmentation model corresponding to different fault types within the same level group to perform fault detection, obtaining the recognition result corresponding to each fault type within that level group. If a fault is detected, all existing fault types within that level group are integrated, and a fault list is output as the fault detection result. If all recognition results within that level group indicate that no corresponding fault type exists, the image to be detected is then input into the semantic segmentation model corresponding to different fault types in the next priority level group for various fault type detections, and so on, until an existing fault type is identified, or until all fault types in all level groups have been detected.

[0055] For example, fracture, spalling, and cracking failures are classified into the severe failure level group; corrosion and pitting failures are classified into the moderate failure level group; and indentation failures are classified into the mild failure level group. Their priority, from highest to lowest, is severe failure, moderate failure, and mild failure. (Refer to...) Figure 5 As shown, Figure 5 This is a schematic diagram of another embodiment of the gearbox fault detection process provided in this application. The electronic device first determines whether there is a fault in the severe fault level group of the image to be detected. If a fault is detected, the fault list containing the existing fault types is used as the fault detection result. If no fault is detected in the severe fault level group, the detection continues for the medium fault level group to obtain the identification results of all fault types in this level group. If a fault is detected, the fault list containing the existing fault types is used as the fault detection result. If no fault is detected in the medium fault level group, the detection continues for the mild fault level group to obtain the identification results of the indentation fault type in this level group. If a fault is detected, the presence of indentation fault is used as the fault detection result. If no fault is detected, the final fault detection result is no fault.

[0056] This approach, which categorizes fault types into different levels for hierarchical detection, ensures comprehensive detection and enables timely output of results when serious faults are detected, thereby effectively saving computing resources.

[0057] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0058] Figure 6This is a schematic diagram of a gearbox fault detection device provided in an embodiment of this application. The device 600 includes: an image acquisition module 61, a fault identification module 62, and a result acquisition module 63.

[0059] Image acquisition module 61 is used to acquire the image to be detected, which is obtained by taking a picture of the inside of the gearbox; The fault identification module 62 is used to perform semantic segmentation on the image to be detected using a semantic segmentation model in order to obtain the fault region in the image to be detected. The result acquisition module 63 is used to determine whether the image to be detected has a fault type corresponding to the semantic segmentation model based on the fault area, and to generate the identification result of the fault type, so as to obtain the fault detection result of the image to be detected based on the identification result corresponding to each fault type.

[0060] In one feasible implementation, when the fault identification module 62 uses a semantic segmentation model to perform semantic segmentation on the image to be detected to obtain fault regions in the image to be detected, it inputs the image to be detected into the semantic segmentation model for semantic segmentation to obtain a grayscale image of the image to be detected. The grayscale image is used to indicate the fault probability of each pixel in the image to be detected. Based on the fault probability of each pixel, the grayscale image is binarized according to a first threshold to obtain a binarized image. The binarized image is used to indicate the fault regions in the image to be detected.

[0061] In one feasible implementation, when the result acquisition module 63 determines whether the image to be detected has a fault type corresponding to the semantic segmentation model based on the fault region and generates the fault type identification result, it is used to eliminate the noise region in the binarized image through the erosion algorithm to obtain the fault region, which includes one or more connected regions; it is determined whether the number of pixels of the largest connected region in each connected region is greater than a second threshold. If it is greater than the second threshold, it is determined that the image to be detected has a fault type corresponding to the semantic segmentation model, and the identification result is that a fault type exists.

[0062] In one feasible implementation, when the result acquisition module 63 determines whether the image to be detected has a fault type corresponding to the semantic segmentation model based on the fault region and generates the identification result of the fault type, it is used to calculate the total number of pixels in the fault region and determine whether the total number is greater than the third threshold; if it is greater than the third threshold, it is determined that the image to be detected has a fault type corresponding to the semantic segmentation model, and the identification result is that a fault type exists.

[0063] In one feasible implementation, when the result acquisition module 63 acquires the fault detection result of the image to be detected based on the recognition result corresponding to each fault type, it is used to take the recognition result as the fault detection result of the image to be detected when the recognition result indicates that a fault type exists; when the recognition result indicates that no fault type exists, it continues to use the semantic segmentation model of the next priority fault type to perform semantic segmentation on the image to be detected, wherein different fault types have different preset priorities.

[0064] In one feasible implementation, when the result acquisition module 63 acquires the fault detection result of the image to be detected based on the recognition result corresponding to each fault type, it is used to integrate the recognition results corresponding to each fault type to obtain the fault detection result of the image to be detected.

[0065] The gearbox fault detection device provided in this application embodiment is used to execute the gearbox fault detection method in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0066] Combination Figure 7 As shown, this application provides an electronic device 700, including a processor 701 and a memory 702. Optionally, the device may further include a communication interface 703 and a bus 704. The processor 701, memory 702, and communication interface 703 can communicate with each other via the bus 704. The communication interface 703 can be used for information transmission. The processor 701 can call logical instructions in the memory 702 to execute the gearbox fault detection method described in the above embodiment.

[0067] Furthermore, the logic instructions in the aforementioned memory 702 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0068] The memory 702, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 701 executes functional applications and data processing by running the program instructions / modules stored in the memory 702, thereby implementing the gearbox fault detection method in the above embodiments.

[0069] The memory 702 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 702 may include high-speed random access memory and may also include non-volatile memory.

[0070] This application provides a storage medium storing computer-executable instructions, which are configured to execute the gearbox fault detection method described in the above embodiments.

[0071] The aforementioned storage medium can be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0072] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0073] This application provides a computer program product, including a computer program, which, when executed by a processor, implements the access control method described above.

[0074] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0075] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0076] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0077] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description; sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for detecting gearbox faults, characterized in that, include: The image to be detected is obtained by photographing the inside of the gearbox; The image to be detected is semantically segmented using a semantic segmentation model to obtain the fault region in the image to be detected. Based on the fault region, determine whether the image to be detected has a fault type corresponding to the semantic segmentation model, and generate the identification result of the fault type, so as to obtain the fault detection result of the image to be detected based on the identification result corresponding to each fault type.

2. The method according to claim 1, characterized in that, The step of using a semantic segmentation model to perform semantic segmentation on the image to be detected to obtain fault regions in the image to be detected includes: The image to be detected is input into the semantic segmentation model for semantic segmentation to obtain a grayscale image of the image to be detected. The grayscale image is used to indicate the fault probability of each pixel in the image to be detected. Based on the fault probability of each pixel, the grayscale image is binarized according to a first threshold to obtain a binarized image, which is used to indicate the fault region in the image to be detected.

3. The method according to claim 2, characterized in that, The step of determining whether the image to be detected contains the fault type corresponding to the semantic segmentation model based on the fault region, and generating the identification result of the fault type, includes: The noise region in the binarized image is eliminated by an erosion algorithm to obtain the fault region, which includes one or more connected regions. Determine whether the number of pixels in the largest connected region in each of the connected regions is greater than a second threshold. If it is greater than the second threshold, then determine that the image to be detected has a fault type corresponding to the semantic segmentation model, and the recognition result is that the fault type exists.

4. The method according to claim 2, characterized in that, The step of determining whether the image to be detected contains the fault type corresponding to the semantic segmentation model based on the fault region, and generating the identification result of the fault type, includes: Calculate the total number of pixels in the fault area and determine whether the total number is greater than a third threshold; If the value is greater than the third threshold, it is determined that the image to be detected contains the fault type corresponding to the semantic segmentation model, and the recognition result is that the fault type exists.

5. The method according to claim 3 or 4, characterized in that, The step of obtaining the fault detection result of the image to be detected based on the recognition result corresponding to each fault type includes: When the identification result indicates the presence of the fault type, the identification result is taken as the fault detection result of the image to be detected; When the identification result indicates that the fault type does not exist, the semantic segmentation model of the next priority fault type is used to perform semantic segmentation on the image to be detected. Different fault types have different preset priorities.

6. The method according to claim 3 or 4, characterized in that, The step of obtaining the fault detection result of the image to be detected based on the recognition result corresponding to each fault type includes: By integrating the identification results corresponding to each fault type, the fault detection result of the image to be detected is obtained.

7. A gearbox fault detection device, characterized in that, include: The image acquisition module is used to acquire the image to be detected, which is obtained by photographing the inside of the gearbox; The fault identification module is used to perform semantic segmentation on the image to be detected using a semantic segmentation model in order to obtain the fault region in the image to be detected. The result acquisition module is used to determine whether the image to be detected has a fault type corresponding to the semantic segmentation model based on the fault region, and to generate the identification result of the fault type, so as to obtain the fault detection result of the image to be detected based on the identification result corresponding to each fault type.

8. An electronic device comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to perform the method as described in any one of claims 1 to 6 when executing the program instructions.

9. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they perform the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images