Engine fault detection method, device and equipment based on image processing

Abstract

The present invention relates to an image processing-based engine fault detection method, apparatus, and device. The method can obtain image data at least including an engine plume within a preset time period, wherein the image data includes at least first image data and second image data. Plume region extraction is performed on the first image data and the second image data, respectively, to obtain first plume region image data and second plume region image data. A differential operation is performed on the first plume region image data and the second plume region image data, respectively, to obtain first differential image data and second differential image data. The first differential image data and the second differential image data are processed based on preset statistics, respectively, to obtain first and second statistical sets. An engine fault is determined based on the first and second statistical sets and a preset statistical threshold set. The present invention can quickly and accurately detect engine anomalies.

Description

Technical Field

[0001] The present invention relates to the technical field of engine testing, and in particular to an engine fault detection method, device and equipment based on image processing. Background Art

[0002] Space engines are highly complex and sophisticated thermal machines. As the heart of various spacecraft, they not only power these vehicles but also serve as a vital driving force for the development of the space industry. Launching a carrier rocket, a primary application area for space engines, is a rigorous undertaking characterized by high risk, high cost, instantaneous nature, and the rapid onset of failures. Numerous factors can impact a successful rocket launch, with engine failure being one of the most significant. Promptly detecting engine failures facilitates the implementation of emergency mitigation measures to minimize losses. Therefore, accurately and reliably detecting engine failures has become a pressing technical challenge. Summary of the Invention

[0003] In order to solve the problems of untimely fault detection and low detection accuracy in the prior art, the present invention provides an engine fault detection method, device and equipment based on image processing, which has the characteristics of accurate fault detection and higher detection efficiency.

[0004] According to a specific embodiment of the present invention, an engine fault detection method based on image processing is provided, comprising:

[0005] Acquiring image data of at least the engine plume within a preset time period, the image data comprising at least first image data and second image data, the first image data being obtained based on visible light imaging, and the second image data being obtained based on infrared light imaging;

[0006] performing plume region extraction on the first image data and the second image data respectively, to obtain first plume region image data and second plume region image data respectively;

[0007] performing a differential operation on the first plume region image data and the second plume region image data respectively, to obtain first differential image data and second differential image data respectively;

[0008] processing the first differential image data and the second differential image data based on preset statistics to obtain a first statistical set and a second statistical set respectively;

[0009] An engine fault is determined based on the first statistic set, the second statistic set, and a preset statistic threshold set.

[0010] Furthermore, the determining of the engine fault based on the first statistic set, the second statistic set, and a preset statistic threshold set includes:

[0011] If the value of any statistic in the first statistic set and / or the second statistic set is greater than the corresponding threshold in the preset statistic threshold set, it is determined that the engine is faulty.

[0012] Furthermore, the acquiring of image data including at least the engine plume within a preset time period includes:

[0013] Image data including the engine plume within the preset time period is acquired from at least one direction perpendicular to the engine plume using an infrared camera and a visible light camera.

[0014] Furthermore, the extracting the plume region from the first image data and the second image data respectively to obtain the first plume region image data and the second plume region image data respectively includes:

[0015] Image stabilization processing and region of interest cropping are performed on the first image data and the second image data in sequence to obtain the first plume region image data and the second plume region image data respectively.

[0016] Furthermore, the performing image stabilization processing and cropping of a region of interest on the first image data and the second image data in sequence respectively includes:

[0017] Image stabilization processing is performed on the first image data and the second image data in sequence based on feature point matching.

[0018] Furthermore, the preset statistics include at least: a mean, a standard deviation, and a variance of an image. The first difference image data and the second difference image data are processed based on the preset statistics to obtain a first set of statistics and a second set of statistics, respectively, including:

[0019] The mean, standard deviation, and variance of the first differential image data and the second differential image data at each time point are calculated respectively to obtain the first statistical quantity set and the second statistical quantity set.

[0020] Furthermore, performing a differential operation on the first plume region image data and the second plume region image data respectively to obtain first differential image data and second differential image data respectively includes:

[0021] performing a differential operation on two frames of image data at adjacent time points in the first plume region image data to obtain the first differential image data;

[0022] A difference operation is performed on two frames of image data at adjacent time points in the second plume region image data to obtain the second difference image data.

[0023] According to a specific embodiment of the present invention, an engine fault detection device based on image processing is provided, comprising:

[0024] an image acquisition module, configured to acquire image data of at least the engine plume within a preset time period, the image data comprising at least first image data and second image data, the first image data being obtained based on visible light imaging, and the second image data being obtained based on infrared light imaging;

[0025] a feature extraction module, configured to perform plume region extraction on the first image data and the second image data, respectively, to obtain first plume region image data and second plume region image data;

[0026] a differential operation module, configured to perform differential operations on the first plume region image data and the second plume region image data, respectively, to obtain first differential image data and second differential image data;

[0027] a statistics module, configured to process the first differential image data and the second differential image data based on preset statistics, respectively, to obtain a first statistics set and a second statistics set; and

[0028] A fault determination module is configured to determine an engine fault based on the first statistic set, the second statistic set, and a preset statistic threshold set.

[0029] A device provided according to a specific embodiment of the present invention includes: a data collector, a memory and a processor;

[0030] The data collector is used to collect image data of the engine plume;

[0031] The memory is used to store the program, image data and all results of the program execution;

[0032] The processor is used to execute the program to implement the various steps of the engine fault detection method based on image processing as described above.

[0033] The image processing-based engine fault detection method provided by the present invention can obtain image data at least including the engine plume within a preset time period, wherein the image data includes at least first image data and second image data, the first image data being obtained based on visible light imaging, and the second image data being obtained based on infrared light imaging. Plume region extraction is performed on the first and second image data, respectively, to obtain first plume region image data and second plume region image data. A differential operation is performed on the first and second plume region image data, respectively, to obtain first differential image data and second differential image data. The first and second differential image data are processed based on preset statistics, respectively, to obtain first and second statistical sets. An engine fault is determined based on the first and second statistical sets and a preset statistical threshold set. The present invention achieves that fault identification based on engine plume images can effectively improve the sensitivity of fault detection, thereby quickly and accurately detecting engine anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 is a flow chart of an engine fault detection method based on image processing according to an exemplary embodiment;

[0036] Figure 2 is a diagram of a structure for acquiring a plume image according to an exemplary embodiment;

[0037] Figure 3 is a structural diagram of an engine fault detection device based on image processing according to an exemplary embodiment;

[0038] Figure 4 is a structural diagram of a device provided according to an exemplary embodiment. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other implementations obtained by those of ordinary skill in the art without inventive effort are within the scope of protection of the present invention.

[0040] Reference Figure 1 As shown, an embodiment of the present invention provides an engine fault detection method based on image processing, which may include the following steps:

[0041] 101. Acquire image data of at least an engine plume within a preset time period, the image data including at least first image data and second image data, the first image data being obtained based on visible light imaging, and the second image data being obtained based on infrared light imaging.

[0042] Specifically, an infrared camera and a visible light camera can be used to obtain image data including the engine plume within a preset time period from at least one direction perpendicular to the engine plume. Figure 2 As shown, four sets of infrared cameras and visible light cameras can be symmetrically placed around the sides of the aerospace engine. Depending on the engine model and the required imaging quality, the distance between each set of cameras and the engine can be controlled to approximately 20 meters. The visible light camera's imaging is color, with three RGB channels, while the infrared camera's thermal imaging is grayscale, with the original thermal image being a single channel. The visible light camera and infrared camera can image the engine plume from different perspectives, thereby comprehensively presenting the plume's different image features. In specific implementations, image data can be acquired by shooting a video, so that subsequent image data processing can be performed based on the acquisition of each frame of image data. The video image data captured by the visible light camera can be used as the first image data, and the video image data captured by the infrared camera can be used as the second image data.

[0043] It is understandable that those skilled in the art can adjust the configuration and quantity of the above-mentioned camera combination according to actual detection needs, and the present invention does not impose any limitation thereto.

[0044] 102. Perform plume region extraction on the first image data and the second image data respectively, to obtain first plume region image data and second plume region image data accordingly.

[0045] The plume region is extracted from the first and second image data captured by each camera group. Image stabilization and cropping of the region of interest are performed on the first and second image data, respectively, to obtain corresponding first and second plume region image data. Image stabilization is achieved using feature point matching technology. By tracking feature points between two consecutive frames, an affine change matrix representing the motion is derived based on the changes in feature points between the two images. The motion trajectory is calculated based on the affine change matrix and then smoothed. Based on the smoothed motion trajectory, the affine change matrix after smoothing the motion is obtained, and the stabilized image is then obtained based on the smoothed affine change matrix.

[0046] After obtaining a stable image, the plume region can be used as a region of interest, and the image of the plume region can be extracted from each frame of image. The specific implementation of the extraction will not be described in detail in this invention.

[0047] 103. Perform a differential operation on the first plume region image data and the second plume region image data respectively to obtain first differential image data and second differential image data accordingly.

[0048] A differential operation may be performed on two frames of image data at adjacent time points in the first plume region image data to obtain first differential image data, and a differential operation may be performed on two frames of image data at adjacent time points in the second plume region image data to obtain second differential image data.

[0049] For example, a differential image may be obtained by performing a differential operation on two adjacent frames of the first plume region image data and the second plume region image data. The differential operation may be performed by subtracting pixel values at corresponding positions of the two images, thereby obtaining a pixel transformation of the plume region over time.

[0050] 104. Process the first differential image data and the second differential image data based on preset statistics, and obtain a first statistical set and a second statistical set respectively.

[0051] For the first differential image data from the visible light camera, the mean, standard deviation, and variance of the differential image in the RGB channels can be calculated separately. For the second differential image data from the infrared camera, the mean, standard deviation, and variance of each grayscale image can be directly calculated. This yields the first set of statistics for the visible light camera and the second set of statistics for the infrared camera.

[0052] 105. Determine engine failure based on the first statistic set, the second statistic set, and the preset statistic threshold set.

[0053] When performing real-time anomaly detection, the mutation values of the image's mean, standard deviation, and variance, under normal circumstances, fluctuate within a certain range. Exceeding the upper limit of this range indicates an engine anomaly. Therefore, a threshold is set for each statistic. Based on this threshold, the resulting image statistics are compared, allowing rapid identification of engine issues based on plume images.

[0054] The engine fault detection method calculates the statistics of the components of the plume area image on different channels and can quickly and accurately detect anomalies based on the changes in the statistics, thereby improving the sensitivity of engine fault detection.

[0055] In another specific embodiment of the present invention, when determining an engine fault based on the first statistic set, the second statistic set and the preset statistic threshold set, the engine fault can be determined if the value of any statistic in the first statistic set and / or the second statistic set is greater than the corresponding threshold in the preset statistic threshold set.

[0056] Specifically, to ensure the sensitivity and accuracy of the judgment, a one-to-one comparison method can be adopted. That is, as long as any value of the three statistical quantities of mean, standard deviation and variance exceeds the corresponding item in the preset threshold, it can be determined that the engine has a fault, thereby improving the sensitivity and accuracy of engine fault judgment and reducing the time for fault discovery.

[0057] Based on the same design idea, refer to Figure 3 The embodiment of the present invention further provides an engine fault detection device based on image processing, which can implement the various steps of the above-mentioned engine fault detection method based on image processing when running. The device may include:

[0058] The image acquisition module 301 is used to acquire image data including at least the engine plume within a preset time period. The image data includes at least first image data and second image data. The first image data is obtained based on visible light imaging, and the second image data is obtained based on infrared light imaging.

[0059] The feature extraction module 302 is configured to perform plume region extraction on the first image data and the second image data, respectively, to obtain first plume region image data and second plume region image data respectively.

[0060] The difference operation module 303 is used to perform a difference operation on the first plume region image data and the second plume region image data respectively, and obtain first difference image data and second difference image data accordingly.

[0061] The statistics module 304 is used to process the first differential image data and the second differential image data based on the preset statistics, and obtain the first statistics set and the second statistics set respectively.

[0062] The fault determination module 305 is configured to determine engine faults based on the first statistic set, the second statistic set, and a preset statistic threshold set.

[0063] Furthermore, the fault determination module 305 is specifically configured to determine that an engine fault occurs if the value of any statistic in the first statistic set and / or the second statistic set is greater than a corresponding threshold in the preset statistic threshold set.

[0064] Furthermore, the image acquisition module 301 is specifically configured to acquire image data including the engine plume within a preset time period from at least one direction perpendicular to the engine plume based on the infrared camera and the visible light camera.

[0065] Furthermore, the feature extraction module 302 is specifically configured to perform image stabilization processing and cropping of the region of interest on the first image data and the second image data in sequence, so as to obtain first plume region image data and second plume region image data respectively.

[0066] The first image data and the second image data are subjected to image stabilization processing in sequence based on feature point matching.

[0067] Furthermore, the statistics module 304 is specifically configured to calculate the mean, standard deviation, and variance of the first differential image data and the second differential image data at each time point, respectively, to obtain a first statistics set and a second statistics set.

[0068] Furthermore, the differential operation module 303 is specifically configured to perform a differential operation on two frames of image data at adjacent time points in the first plume region image data to obtain first differential image data.

[0069] A difference operation is performed on two frames of image data at adjacent time points in the second plume region image data to obtain second difference image data.

[0070] The image processing-based engine fault detection device has the same beneficial effects as the above-mentioned image processing-based engine fault detection method. Its specific implementation method can refer to the embodiment of the above-mentioned image processing-based engine fault detection method, and the present invention will not repeat it here.

[0071] Reference Figure 4 As shown, an embodiment of the present invention further provides a device, which may include: a data collector 401, a memory 402 and a processor 403.

[0072] The data collector 401 is used to collect image data of the engine plume.

[0073] The memory 402 is used to store the program, image data and all the results of the program execution.

[0074] The processor 403 is configured to execute the program to implement the various steps of the engine fault detection method based on image processing as described in the above embodiment.

[0075] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0076] The above description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it will be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, to the extent that the term "comprising" is used in the specification or claims, the term is intended to be encompassed in a manner similar to the term "including", as explained by the use of "including" as a transitional word in the claims. In addition, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or".

[0077] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An engine fault detection method based on image processing, characterized in that: include: Acquiring image data of at least the engine plume within a preset time period, the image data comprising at least first image data and second image data, the first image data being obtained based on visible light imaging, and the second image data being obtained based on infrared light imaging; performing plume region extraction on the first image data and the second image data respectively, to obtain first plume region image data and second plume region image data respectively; performing a differential operation on the first plume region image data and the second plume region image data respectively, to obtain first differential image data and second differential image data respectively; processing the first differential image data and the second differential image data based on preset statistics to obtain a first statistical set and a second statistical set respectively; An engine fault is determined based on the first statistic set, the second statistic set, and a preset statistic threshold set.

2. The method according to claim 1, characterized in that The determining of the engine fault based on the first statistic set, the second statistic set, and a preset statistic threshold set includes: If the value of any statistic in the first statistic set and / or the second statistic set is greater than the corresponding threshold in the preset statistic threshold set, it is determined that the engine is faulty.

3. The method according to claim 1, characterized in that The acquiring of image data including at least the engine plume within a preset time period includes: Image data including the engine plume within the preset time period is acquired from at least one direction perpendicular to the engine plume using an infrared camera and a visible light camera.

4. The method according to claim 1, wherein The extracting the plume region from the first image data and the second image data respectively to obtain the first plume region image data and the second plume region image data accordingly includes: Image stabilization processing and region of interest cropping are performed on the first image data and the second image data in sequence to obtain the first plume region image data and the second plume region image data respectively.

5. The method according to claim 4, characterized in that The performing image stabilization processing and cropping of a region of interest on the first image data and the second image data in sequence respectively includes: Image stabilization processing is performed on the first image data and the second image data in sequence based on feature point matching.

6. The method according to claim 1, characterized in that The preset statistics include at least: a mean, a standard deviation, and a variance of an image. The first differential image data and the second differential image data are processed based on the preset statistics to obtain a first set of statistics and a second set of statistics, respectively, including: The mean, standard deviation, and variance of the first differential image data and the second differential image data at each time point are calculated respectively to obtain the first statistical quantity set and the second statistical quantity set.

7. The method according to claim 1, characterized in that The performing of a differential operation on the first plume region image data and the second plume region image data respectively to obtain first differential image data and second differential image data respectively includes: performing a differential operation on two frames of image data at adjacent time points in the first plume region image data to obtain the first differential image data; A difference operation is performed on two frames of image data at adjacent time points in the second plume region image data to obtain the second difference image data.

8. An engine fault detection device based on image processing, characterized in that: include: an image acquisition module, configured to acquire image data of at least the engine plume within a preset time period, the image data comprising at least first image data and second image data, the first image data being obtained based on visible light imaging, and the second image data being obtained based on infrared light imaging; a feature extraction module, configured to perform plume region extraction on the first image data and the second image data, respectively, to obtain first plume region image data and second plume region image data; a differential operation module, configured to perform differential operations on the first plume region image data and the second plume region image data, respectively, to obtain first differential image data and second differential image data; a statistics module, configured to process the first differential image data and the second differential image data based on preset statistics, respectively, to obtain a first statistics set and a second statistics set; as well as A fault determination module is configured to determine an engine fault based on the first statistic set, the second statistic set, and a preset statistic threshold set.

9. A device, characterized in that include: Data collector, memory and processor; The data collector is used to collect image data of the engine plume; The memory is used to store programs, image data and all results of the program execution; The processor is configured to execute the program to implement each step of the engine fault detection method based on image processing as claimed in any one of claims 1 to 7.

Images