Engine surface anomaly real-time detection method and device, and storage medium

Abstract

The present disclosure relates to an engine surface anomaly real-time detection method and device, and a storage medium. The engine surface anomaly real-time detection method comprises: collecting an engine surface picture of an aero-engine test site; constructing an aero-engine surface anomaly detection model; inputting the engine surface picture into the aero-engine surface anomaly detection model to obtain an engine anomaly detection result. The present disclosure can automatically and real-timely detect surface anomalies in the aero-engine test process.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of aero-engine manufacturing, and particularly relates to an engine surface anomaly real-time detection method and device and a storage medium. BACKGROUND

[0002] In the development process of an aero-engine, a test run is often needed. During the test run, the staff needs to leave the test run workshop to avoid some unexpected accidents that may occur halfway. However, during the test run, the staff needs to observe the surface state of the engine in real time and record the abnormal problems occurring in the test run process for subsequent development improvement. SUMMARY

[0003] The inventors have found that, in the related art, the main method for the staff to observe outside the test run workshop is to use multiple cameras to shoot various parts of the engine, to observe the multiple camera pictures in real time with naked eyes, to record the results if any abnormality is observed, and to intervene in real time and analyze afterwards. If the abnormal information recorded in the test run process needs to be linked with an information system, a manual input method is usually used to input the abnormal information into the information system. The whole process usually accompanies the whole process of a test run, and the time can be as long as 3 to 5 days. The manual observation and recording method needs multiple special staff to take turns, which is very monotonous and easy to make mistakes due to fatigue.

[0004] In view of at least one of the above technical problems, the present disclosure provides an engine surface anomaly real-time detection method and device and a storage medium, which can automatically and in real time detect surface anomalies in the test run process of an aero-engine.

[0005] According to one aspect of the present disclosure, an engine surface anomaly real-time detection method is provided, comprising:

[0006] collecting engine surface pictures of an aero-engine test run site;

[0007] constructing an aero-engine surface anomaly detection model;

[0008] inputting the engine surface pictures into the aero-engine surface anomaly detection model to obtain an engine anomaly detection result.

[0009] In some embodiments of the present disclosure, the constructing of the aero-engine surface anomaly detection model comprises:

[0010] constructing an aero-engine surface anomaly data set;

[0011] training the aero-engine surface anomaly detection model;

[0012] verifying the effect of the aero-engine surface anomaly detection model on a verification set.

[0013] In some embodiments of the present disclosure, the constructing the aero-engine surface anomaly dataset comprises:

[0014] screening the collected engine surface pictures;

[0015] annotating the engine surface anomalies of the specified anomaly type in the engine surface pictures;

[0016] constructing a training set and a validation set of the aero-engine surface anomaly data.

[0017] In some embodiments of the present disclosure, the training the aero-engine surface anomaly detection model comprises:

[0018] adopting a multi-layer convolutional neural network and a pyramid feature detection network as the aero-engine surface anomaly detection model;

[0019] calculating the model prediction loss value of a single iteration by using a model loss function;

[0020] correcting the parameter weight of the aero-engine surface anomaly detection model according to the back propagation of the model prediction loss value.

[0021] In some embodiments of the present disclosure, the inputting the engine surface picture into the aero-engine surface anomaly detection model to obtain the engine anomaly detection result comprises:

[0022] initializing and assigning values to the aero-engine surface anomaly detection model;

[0023] converting the engine surface picture into a predetermined size;

[0024] inputting the engine surface picture of the predetermined size into the aero-engine surface anomaly detection model to obtain the anomaly prediction result by forward calculation.

[0025] In some embodiments of the present disclosure, the inputting the engine surface picture into the aero-engine surface anomaly detection model to obtain the engine anomaly detection result further comprises:

[0026] performing non-maximum suppression processing on the anomaly prediction result;

[0027] filtering the anomaly prediction result according to a pre-set threshold.

[0028] In some embodiments of the present disclosure, the specified anomaly type comprises at least one of connector loosening, oil leakage and firelight.

[0029] According to another aspect of the present disclosure, an engine surface anomaly real-time detection device is provided, comprising:

[0030] a picture collection module configured to collect engine surface pictures of an aero-engine test site;

[0031] a model construction module, configured to construct an aero-engine surface anomaly detection model;

[0032] an anomaly result acquisition module, configured to input the engine surface picture into the aero-engine surface anomaly detection model to obtain an engine anomaly detection result;

[0033] In some embodiments of the present disclosure, the engine surface anomaly real-time detection device is configured to perform operations to implement the engine surface anomaly real-time detection method according to any of the above embodiments.

[0034] According to another aspect of the present disclosure, an engine surface anomaly real-time detection device is provided, comprising:

[0035] a memory, configured to store instructions;

[0036] a processor, configured to execute the instructions, so that the engine surface anomaly real-time detection device performs operations to implement the engine surface anomaly real-time detection method according to any of the above embodiments.

[0037] According to another aspect of the present disclosure, a non-transitory computer readable storage medium is provided, wherein the non-transitory computer readable storage medium stores computer instructions, and the instructions, when executed by a processor, implement the engine surface anomaly real-time detection method according to any of the above embodiments.

[0038] The present disclosure can automatically and real-time detect surface anomalies in the aero-engine test run process. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0040] Figure 1 a schematic diagram of some embodiments of the engine surface anomaly real-time detection method of the present disclosure.

[0041] Figure 2 a schematic diagram of some other embodiments of the engine surface anomaly real-time detection method of the present disclosure.

[0042] Figure 3 a schematic diagram of the aero-engine surface anomaly detection model in some embodiments of the present disclosure.

[0043] Figure 4A schematic diagram of applying the aero-engine surface anomaly detection model to an actual test run in some embodiments of the present disclosure.

[0044] Figure 5 A schematic diagram of the engine surface anomaly real-time detection device in some embodiments of the present disclosure.

[0045] Figure 6 A structural schematic diagram of the engine surface anomaly real-time detection device in some other embodiments of the present disclosure. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, but not all the embodiments. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present disclosure.

[0047] Unless otherwise specified, the relative arrangement, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0048] It should be understood that the sizes of the various parts shown in the drawings are not drawn in accordance with the actual proportion relationship for the convenience of description.

[0049] The technology, methods and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered as part of the authorized description when appropriate.

[0050] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as a limitation. Thus, other examples of the exemplary embodiments can have different values.

[0051] It should be noted that similar reference numbers and letters represent similar items in the following drawings, and thus, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0052] The inventors have found through research that during the aero-engine test run, it is a time-consuming and laborious task for a person to continuously observe the camera screen in the test run workshop, and it is difficult to avoid omissions during the execution process.

[0053] The technical problem to be solved by the present disclosure is to provide a method and device capable of automatically and real-time detecting surface abnormalities in the aero-engine test run process, so as to overcome the problems of high cost and error-prone in the related art relying on manpower to observe the camera and record abnormal information. The above embodiments of the present disclosure will be described below through specific embodiments.

[0054] Figure 1 A schematic diagram of some embodiments of the present disclosure of the engine surface abnormality real-time detection method. Preferably, the present embodiment can be executed by the engine surface abnormality real-time detection device of the present disclosure. The method can include at least one of steps 11-13, wherein:

[0055] Step 11, collecting engine surface pictures of the aero-engine test site.

[0056] Step 12, constructing an aero-engine surface abnormality detection model.

[0057] In some embodiments of the present disclosure, step 12 can include Figure 2 Steps S2-S4 in the embodiment.

[0058] Step 13, inputting the engine surface pictures into the aero-engine surface abnormality detection model to obtain the engine abnormality detection result.

[0059] Figure 2 A schematic diagram of another embodiment of the present disclosure of the engine surface abnormality real-time detection method. Preferably, the present embodiment can be executed by the engine surface abnormality real-time detection device of the present disclosure.

[0060] The method can include at least one of steps S1-S5, wherein:

[0061] Step S1, collecting engine surface pictures of the aero-engine test site.

[0062] In some embodiments of the present disclosure, step S1 can include using an industrial camera to collect pictures of the on-site experiment, collecting more than 500 pictures for each of the three abnormalities of oil leakage, connector loosening, and firelight, and making a data set.

[0063] In some embodiments of the present disclosure, the pictures are collected to ensure that they are clear and identifiable.

[0064] In some embodiments of the present disclosure, step S1 can include at least one of steps S11 and S12.

[0065] Step S11, before the test run, manual collection is used; during the test run, automatic collection is used. Manual collection requires at least two people to cooperate, one person is responsible for moving the camera and fixing the camera at each position using a tripod, and the other person is responsible for watching the camera screen and taking a picture after the screen is stable and clear. After the light source facilities are fixed, images are collected in the six fixed positions in turn, and then the camera is moved to simulate the movement trajectory of the mechanical arm to collect defects from various angles and distances (500mm-1000mm). Automatic collection uses the picture periodic saving function of the system to obtain pictures during operation.

[0066] Step S12, before the actual test run, an artificial simulation method can be used to make abnormal scenes. The artificial simulation of abnormalities must be as close to the real scene as possible and include as many possible scenarios as possible. For oil leakage, oil droplets can be artificially added at possible leakage locations, for loose connectors, the connectors can be artificially loosened, and for fire, artificial controllable fire can be created around the engine to simulate it. Generally, for each simulated abnormality, 5 pictures are collected for each angle and each distance. Automatic collection is to set multiple cameras monitoring the surface of the engine to automatic continuous shooting mode during the test run.

[0067] Step S2, constructing an aero-engine surface abnormality data set.

[0068] In some embodiments of the present disclosure, step S2 can include at least one of steps S21-S23, wherein:

[0069] Step S21, screening the collected engine surface pictures.

[0070] In some embodiments of the present disclosure, step S21 can include further screening for very large original picture data collected in step S1.

[0071] In some embodiments of the present disclosure, step S21 can include removing most of the useless pictures, including pictures that are not clear and pictures with highly similar contents. The remaining pictures mainly include four types: loose connectors, oil leakage, fire, and no abnormal background pictures.

[0072] Step S22, labeling the engine surface abnormalities of the specified abnormality type in the engine surface pictures.

[0073] In some embodiments of the present disclosure, the specified abnormality type can include at least one of a loose connector, oil leakage, and fire.

[0074] The above embodiments of the present disclosure must limit the labeling standard of abnormal defects because the system adopts a visual recognition method, that is, actually, visual analysis is performed on the image collected by the camera, otherwise, the pixel ratio of the abnormal defect in the image is too small to be identified.

[0075] In some embodiments of the present disclosure, the labeling standard for leaking oil is that the area size of a single leaking oil region must be greater than 8mm*5mm; and the position of the leaking oil region that can be clearly identified by the human eye in the picture is labeled. For abnormal defects that are not clear enough due to factors such as shooting angle problems, defect positions at the edge of the picture, camera focus blur, camera motion picture blur, etc., no labeling is performed.

[0076] In some embodiments of the present disclosure, the labeling standard for loose connectors is that the misalignment distance of the loose connector must be greater than 5mm; and the position of the loose connector that can be clearly identified by the human eye in the picture is labeled. For abnormal defects that are not clear enough due to factors such as shooting angle problems, defect positions at the edge of the picture, camera focus blur, camera motion picture blur, etc., no labeling is performed.

[0077] In some embodiments of the present disclosure, the labeling standard for firelight is that the area size of a single firelight must be greater than 8mm*5mm; and the position of the firelight that can be clearly identified by the human eye in the picture is labeled. For abnormal defects that are not clear enough due to factors such as shooting angle problems, defect positions at the edge of the picture, camera focus blur, camera motion picture blur, etc., no labeling is performed.

[0078] Step S23, constructing a training set and a verification set of the aero-engine surface abnormal data.

[0079] In some embodiments of the present disclosure, step S23 can include: selecting 500-1000 labeled abnormal pictures of each of the three types, and 1000 pure background pictures, to form a data set; randomly extracting 80% of the data set as a training set, and the remaining 20% as a verification set.

[0080] Step S3, training an aero-engine surface abnormality detection model.

[0081] In some embodiments of the present disclosure, step S3 can include at least one of steps S31-S33, wherein:

[0082] Step S31, using a multi-layer CNN (Convolutional Neural Network) and FPN (Feature Pyramid Networks) as the aero-engine surface abnormality detection model.

[0083] In some embodiments of this disclosure, the multilayer convolutional neural network can be a 53-layer CNN network.

[0084] Figure 3 This is a schematic diagram of an aero-engine surface anomaly detection model in some embodiments of this disclosure. For example... Figure 3 As shown, the aero-engine surface anomaly detection model includes a 53-layer CNN network and a pyramid feature detection network based on FPN.

[0085] In some embodiments of this disclosure, such as Figure 3 As shown, the 53-layer CNN network 3 is a fully convolutional network, without fully connected layers, and extensively uses residual layer skip connections. First, a 3x3 convolutional kernel with a stride of 1 (Conv) is used to perform preliminary feature extraction on the original 416*416*3 input. Then, a 3x3 convolutional kernel with a stride of 2 is used to downsample the feature map. Next, a residual layer is used for downsampling, resulting in an image resolution of 104*104. After eight more residual layers, downsampling occurs again, yielding a feature map with a resolution of 52*52. This is followed by eight more residual layers, resulting in another downsampling to a feature map with a resolution of 26*26. Finally, four residual layers are used to obtain a feature map with a resolution of 13*13.

[0086] In some embodiments of this disclosure, such as Figure 3 As shown, Avgpool is average pooling. Connected is for concatenation, Unsample is for upsampling, Softmax is for Softmax output, and Concat is a function that concatenates two or more arrays.

[0087] In some embodiments of this disclosure, such as Figure 3 As shown, the FPN-based pyramid feature detection network uses an object detection head to obtain structured object detection outputs on feature maps with resolutions of 13*13, 26*26, and 52*52. Taking the 13*13 feature map as an example, a fully convolutional network is used to obtain a feature representation of 13*13*(3*(5+c)). For the original image grid division of 13*13, each grid can predict 3 object boxes. The information of each object box includes the probability of c (number of categories), the coordinate information of the object box, and the object discrimination information. The 13*13 and 26*26 resolution feature maps are also upsampled and fused with the 26*26 and 52*52 resolution features, respectively, to realize the flow of information in the spatial pyramid and ensure the compatibility and balance between high-order semantic information and low-order detail information in the high-resolution feature map.

[0088] Step S32: Calculate the model prediction loss value for a single iteration using the model loss function.

[0089] In some embodiments of the present disclosure, formula (1) is a model loss function of the present disclosure.

[0090] L(O, o, C, c, l, g) = λ1L conf (o, c) + λ2L cla (O, C) + λ3L loc (l, g) (1)

[0091] In formula (1), the loss function can include three parts: target positioning loss L loc (l, g), target confidence loss L conf (o, c), and target classification loss L cla (O, C). In formula (1), λ1 is a target confidence loss weight, o is a real background, and c is a predicted background; λ2 is a target classification loss weight, O is a real classification, and C is a predicted classification; λ3 is a target positioning loss weight, l is a predicted position, and g is a real position.

[0092] Step S33, correcting the parameter weight of the aero-engine surface anomaly detection model according to the model prediction loss value back propagation.

[0093] In some embodiments of the present disclosure, step S33 can include using a method of back propagation operation to train the parameters of the entire network.

[0094] In some embodiments of the present disclosure, steps S32 and S33 can include: first inputting a batch of images, calculating the loss of the batch, and using a stochastic gradient descent method to update the weight of the network according to the corresponding loss. After a fixed number of iterations, the learning rate is reduced by a certain proportion, so that the weight of the network is updated relatively gently, and finally converges to a global optimal solution.

[0095] Step S4, verifying the effect of the aero-engine surface anomaly detection model on the verification set.

[0096] In some embodiments of the present disclosure, the evaluation index of the detection algorithm is accuracy. The definition of correct detection result is that the detection result and the actual result category are consistent, and the bounding box roughly coincides (the coincidence rate is more than 50%). The definition of accuracy is the number of correct detection results divided by the number of all detection results. The definition of IoU (Intersection over Union) is the area of the intersection of two bounding boxes divided by the area of the union of the bounding boxes.

[0097] In some embodiments of the present disclosure, the verification method of step S4 can comprise: using the trained neural network model, sequentially inputting all pictures in the verification set into the neural network model, performing forward calculation, and giving a prediction result; comparing the prediction result with the actual result, and counting the accuracy rate of each category of the prediction result.

[0098] In some embodiments of the present disclosure, the verification effect of the above-mentioned embodiments of the present disclosure is shown in Table 1:

[0099] Table 1

[0100]

[0101] The verification result of the above-mentioned embodiments of the present disclosure shows that the trained CNN neural network model has a high accuracy rate and can be used in actual test running.

[0102] Step S5, applying the aero-engine surface anomaly detection model to the actual test running process.

[0103] In some embodiments of the present disclosure, step S5 can comprise: inputting the engine surface picture into the aero-engine surface anomaly detection model to obtain an engine anomaly detection result.

[0104] Figure 4 A schematic diagram for applying the aero-engine surface anomaly detection model to the actual test running process in some embodiments of the present disclosure. As shown in Figure 4 , the step S5 of the embodiments can comprise at least one of steps S501-S506, wherein: Figure 2 Step 501, initializing and assigning values to the aero-engine surface anomaly detection model.

[0105] In some embodiments of the present disclosure, step 501 can comprise: constructing a neural network with a pre-designed structure, and then assigning the trained parameter weight to the network.

[0106] Step 502, converting the engine surface picture into a predetermined size.

[0107] In some embodiments of the present disclosure, step 502 can comprise: scaling the picture to 416*416 size.

[0108] Step 503, inputting the engine surface picture of the predetermined size into the aero-engine surface anomaly detection model.

[0109] In some embodiments of the present disclosure, step 503 can comprise: copying the 416*416 picture data to the entrance of the neural network.

[0110] In some embodiments of the present disclosure, step 503 can comprise: copying the 416*416 picture data to the entrance of the neural network.

[0111] Step 504, the forward calculation neural network obtains an abnormal prediction result.

[0112] In some embodiments of the present disclosure, step 504 can include: using a 3x3 step length of 2 convolution kernel to down-sample the feature map. After passing through a residual layer, down-sampling processing is performed, and at this time, the image resolution is 104*104. After passing through 8 residual layers, down-sampling is performed again, and a feature map with a resolution of 52*52 is obtained. After sequentially passing through 8 residual layers, down-sampling is performed again to obtain a feature map with a resolution of 26*26. Finally, after passing through 4 residual layers, a feature map with a resolution of 13*13 is obtained. For the 13*13 feature map, a full convolution network is used to obtain a 13*13*(3*(5+c)) feature representation. For the 13*13 original image grid division, each grid can predict 3 target boxes, and the information of each target box includes the probability of c (the number of categories) categories, the coordinate information of the target box, and the object discrimination degree information.

[0113] Step 505, performing NMS (Non-Maximum Suppression) processing on the abnormal prediction result.

[0114] In some embodiments of the present disclosure, step 505 can include: first sorting the confidence of each abnormal result, first selecting the first one, that is, the abnormal result with the highest confidence, which is the abnormal result that we want to keep. Then compare it with the remaining abnormal results, if the IOU (Intersection over Union) is greater than a certain threshold, it means that the two have high overlap, and should be removed. The filtered abnormal result is the abnormal result with low overlap with the first abnormal result, and the first iteration ends. The second time, the abnormal result with the highest confidence is selected from the remaining abnormal results, and the above process is repeated until no box is kept.

[0115] Step 506, filtering the abnormal prediction result according to a pre-set threshold.

[0116] In some embodiments of the present disclosure, step 506 can include: if the confidence of the predicted abnormal result exceeds the preset threshold, an immediate warning is given to remind the on-site personnel whether to intervene in the manual intervention of the test process, and the warning information is recorded into a storage database for subsequent analysis.

[0117] The engine surface anomaly real-time detection method provided by the above-mentioned embodiments of the present disclosure can automatically and real-timely detect the surface anomaly of the aero-engine during the test run, can assist the artificial to better complete the observation and recording, plays a role of checking and supplementing the artificial observation, can reduce the labor cost and improve the execution quality of the observation task to a certain extent, can better record the anomaly during the test run, can be used for real-time intervention and post-analysis, and reduces the probability of major accidents.

[0118] Figure 5 FIG. 1 is a schematic diagram of some embodiments of the engine surface anomaly real-time detection device of the present disclosure. As shown in FIG. 1, the engine surface anomaly real-time detection device of the present disclosure can include a picture acquisition module 51, a model construction module 52, and an anomaly result acquisition module 53, wherein: Figure 5

[0119] The picture acquisition module 51 is configured to acquire the engine surface pictures of the aero-engine test site.

[0120] The model construction module 52 is configured to construct the aero-engine surface anomaly detection model.

[0121] In some embodiments of the present disclosure, the model construction module 52 can be configured to construct the aero-engine surface anomaly data set, train the aero-engine surface anomaly detection model, and verify the effect of the aero-engine surface anomaly detection model on the verification set.

[0122] In some embodiments of the present disclosure, in the case of constructing the aero-engine surface anomaly data set, the model construction module 52 can be configured to filter the acquired engine surface pictures, label the engine surface anomaly of a specified anomaly type in the engine surface pictures, and construct the training set and the verification set of the aero-engine surface anomaly data.

[0123] In some embodiments of the present disclosure, the specified anomaly type can include at least one of connector loosening, oil leakage, and firelight.

[0124] In some embodiments of the present disclosure, in the case of training the aero-engine surface anomaly detection model, the model construction module 52 can be configured to adopt a multi-layer convolutional neural network and a pyramid feature detection network as the aero-engine surface anomaly detection model, calculate the model prediction loss value of a single iteration by using a model loss function, and correct the parameter weight of the aero-engine surface anomaly detection model according to the back propagation of the model prediction loss value.

[0125] The anomaly result acquisition module 53 is configured to input the engine surface pictures into the aero-engine surface anomaly detection model to obtain the engine anomaly detection result.

[0126] ​In some embodiments of the present disclosure, the abnormal result obtaining module 53 can be configured to initialize and assign values to the aero-engine surface anomaly detection model; convert the engine surface picture into a predetermined size; input the engine surface picture of the predetermined size into the aero-engine surface anomaly detection model, and obtain an abnormal prediction result through forward calculation.

[0127] In some embodiments of the present disclosure, the abnormal result obtaining module 53 can also be configured to perform non-maximum suppression processing on the abnormal prediction result; and filter the abnormal prediction result according to a pre-set threshold.

[0128] In some embodiments of the present disclosure, the engine surface anomaly real-time detection device is configured to perform operations of the engine surface anomaly real-time detection method as described in any of the above embodiments (for example Figures 1-4 any embodiment).

[0129] Figure 6 FIG. 6 is a structural schematic diagram of another embodiment of the engine surface anomaly real-time detection device according to the present disclosure. As shown in FIG. 6, the engine surface anomaly real-time detection device includes a memory 61 and a processor 62. Figure 6

[0130] The memory 61 is configured to store instructions, and the processor 62 is coupled to the memory 61. The processor 62 is configured to perform operations of the engine surface anomaly real-time detection method as described in any of the above embodiments (for example Figures 1-4 any embodiment) based on the instructions stored in the memory.

[0131] As shown in FIG. 6, the engine surface anomaly real-time detection device further includes a communication interface 63 configured to interact with other devices. Meanwhile, the engine surface anomaly real-time detection device further includes a bus 64, and the processor 62, the communication interface 63, and the memory 61 complete communication with each other through the bus 64. Figure 6

[0132] The memory 61 can include a high-speed RAM memory, and can also include a non-volatile memory such as at least one disk memory. The memory 61 can also be a memory array. The memory 61 can also be divided into blocks, and the blocks can be combined into a virtual volume according to a certain rule.

[0133] In addition, the processor 62 can be a central processing unit CPU, or can be an application-specific integrated circuit ASIC, or one or more integrated circuits configured to implement embodiments of the present disclosure.

[0134] ​​The engine surface abnormality real-time detection device provided by the above-mentioned embodiments of the present disclosure uses visual technology to automatically detect and analyze real-time camera pictures in a test workshop, uses deep learning CNN technology to detect the camera pictures in real time, gives an immediate warning if an abnormality is found, reminds on-site personnel to pay attention to whether manual intervention is needed in the test process, records the warning information into a storage database for subsequent analysis. The biggest difference between the present disclosure and related technologies in key technologies is that deep learning CNN technology is used, which can automatically identify and find abnormal situations that may occur in the camera pictures, and the working mode is continuous and uninterrupted, which can achieve real-time analysis and identification of multiple camera pictures, and the discovery of abnormal situations is very timely, avoiding further expansion of more serious abnormal situations that need to be handled.

[0135] The above-mentioned embodiments of the present disclosure provide an engine surface abnormality real-time detection method and device in an aero-engine test process.

[0136] According to another aspect of the present disclosure, a non-transitory computer readable storage medium is provided, wherein the non-transitory computer readable storage medium stores computer instructions, and the instructions are executed by a processor to implement the engine surface abnormality real-time detection method according to any one of the above-mentioned embodiments (for example Figures 1-4 any embodiment).

[0137] The non-transitory computer readable storage medium provided by the above-mentioned embodiments of the present disclosure can automatically and real-time detect surface abnormalities in an aero-engine test process during a test process, can assist manual observation and recording, can find missing observations, can reduce labor costs and improve observation task execution quality to a certain extent, can better record abnormalities in the test process, can be used for real-time intervention and post-analysis, and can reduce the probability of major accidents.

[0138] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, an apparatus, or a computer program product. Therefore, the present disclosure can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can be in the form of a computer program product implemented on one or more computer usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0139] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0140] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0141] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0142] The engine surface anomaly real-time detection device described above can be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or any appropriate combination thereof for performing the functions described in the present application.

[0143] So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein according to the above description.

[0144] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete the related hardware, and the program can be stored in a non-transitory computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0145] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are chosen and described in order to best explain the principles of the present disclosure and its practical application, and to enable others skilled in the art to understand the present disclosure in order to design various embodiments with various modifications for specific use cases.

Claims

1. A method for real-time detection of anomalies on the engine surface, characterized in that, include: Before the test run, an abnormal scenario of the engine surface of a specified abnormal type is created by artificial simulation, and the engine surface images of the test run site are collected by manual acquisition. The specified abnormal type includes at least one of the following: loose connector, oil leakage, and sparks. During the test run, images of the engine surface at the test site were collected using an automatic acquisition method. Construct a surface anomaly detection model for aero-engines; The engine surface images collected during the test run are input into the aero-engine surface anomaly detection model to obtain the engine anomaly detection results; The construction of the aero-engine surface anomaly detection model includes: Construct a dataset of surface anomalies for aero-engines; Training a model for detecting surface anomalies on aero-engines; Verify the effectiveness of the aero-engine surface anomaly detection model on the validation set; The construction of the aero-engine surface anomaly dataset includes: The collected images of the engine surface are then filtered. Mark the engine surface anomalies of the specified anomaly type in the engine surface image. The marking criteria for oil leakage are: the area of ​​a single oil leakage area is greater than 8mm*5mm; the marking criteria for loose connectors are: the misalignment distance of the loose connector is greater than 5mm; and the marking criteria for sparks are: the area of ​​a single spark is greater than 8mm*5mm. Construct training and validation sets for surface anomaly data of aero-engines.

2. The real-time detection method for engine surface anomalies according to claim 1, characterized in that, The trained aero-engine surface anomaly detection model includes: A multi-layer convolutional neural network and a pyramid feature detection network are used as a model for detecting anomalies on the surface of aero-engines. The model prediction loss value for a single iteration is calculated using the model loss function; The parameter weights of the aero-engine surface anomaly detection model are corrected by backpropagation based on the model's predicted loss value.

3. The real-time detection method for engine surface anomalies according to claim 1 or 2, characterized in that, The step of inputting engine surface images into the aero-engine surface anomaly detection model to obtain engine anomaly detection results includes: Initialize and assign values ​​to the surface anomaly detection model of the aero-engine; Convert the engine surface image to a predetermined size; An image of the engine surface of a predetermined size is input into the aero-engine surface anomaly detection model, and forward calculation is performed to obtain the anomaly prediction result.

4. The real-time detection method for engine surface anomalies according to claim 3, characterized in that, The step of inputting engine surface images into the aero-engine surface anomaly detection model to obtain engine anomaly detection results also includes: Non-maximum suppression is applied to the abnormal prediction results; Abnormal prediction results are filtered according to a pre-set threshold.

5. A real-time detection device for engine surface anomalies, characterized in that, include: The image acquisition module is used to acquire images of the engine surface at the aero-engine test site; The model building module is used to build an aero-engine surface anomaly detection model. The anomaly result acquisition module is used to input engine surface images into the aero-engine surface anomaly detection model to obtain engine anomaly detection results. The engine surface anomaly real-time detection device is used to perform the operation of implementing the engine surface anomaly real-time detection method as described in any one of claims 1-4.

6. A real-time detection device for engine surface anomalies, characterized in that, include: Memory, used to store instructions; A processor is configured to execute the instructions, causing the engine surface anomaly real-time detection device to implement the engine surface anomaly real-time detection method as described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium, characterized in that, The non-transient computer-readable storage medium stores computer instructions that, when executed by a processor, implement the real-time detection method for engine surface anomalies as described in any one of claims 1-4.

Images