A fuse detection method and system based on projection technology

Abstract

The present invention relates to the technical field of fuse detection, and in particular to a fuse detection method and system based on projection technology, wherein the detection method comprises: obtaining an image of an object to be detected including a fastener; determining the fastener information of the object to be detected based on the image of the object to be detected; obtaining a fastener image based on the fastener information, and determining the geometric features of the fastener; generating a fuse image based on the geometric features of the fastener, and fusing the fastener image and the fuse image to generate a projection image; projecting the projection image onto the surface of the object to be detected, and determining that the fuse on the object to be detected is qualified when the difference between the fuse on the surface of the object to be detected and the fuse image in the projection image meets a preset condition. The detection method and system can effectively solve the problems of low efficiency and difficulty in ensuring accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of fuse detection, and in particular to a fuse detection method and system based on projection technology. Background Art

[0002] A fuse is a metal wire used to prevent fasteners from loosening. In the aerospace industry, fuses are often used to prevent bolts from loosening. Compared with other types of anti-loosening parts, fuses are often used in places where it is not suitable to assemble nuts and other anti-loosening parts to prevent bolts from loosening due to high-speed vibration. The appearance defect detection of aircraft engines is an important part of aircraft engine quality control. In order to ensure the safe operation of aircraft engines, after the aircraft engine is assembled, the installation status of thousands of specific parts needs to be inspected, especially the installation direction of fuses.

[0003] In the prior art, a Chinese invention patent document with publication number CN110853091A and publication date February 28, 2020 is proposed. The technical solution disclosed in the patent document is as follows: an image recognition method and system for engine fuse winding direction defects, including: collecting fuse images from different parts of the engine, constructing a hexagonal bolt detection model, and realizing coarse positioning; using the polar coordinate image of the bolt coarse positioning result to realize accurate detection of the bolt edge, and calculating the center point and radius of the bolt approximate circle; obtaining the single-pixel skeleton of each area in the segmentation result, and constructing a skeleton tree to remove burrs on the skeleton; combining the spatial position constraints of the fuse and the hexagonal bolt and the texture features of the fuse to realize the extraction of the fuse skeleton; quickly calculating the vector of the extension direction of the fuse skeleton; calculating the cross product of the bolt position vector and the fuse extension direction vector to identify the fuse winding direction, and the wrong fuse winding direction is regarded as a fuse winding direction defect.

[0004] In the prior art, a Chinese invention patent document with publication number CN114419026A and publication date April 29, 2022 has also been proposed. The technical solution disclosed in the patent document is as follows: A system and method for identifying the winding direction of an aircraft engine fuse based on a visual attention mechanism, comprising: using an industrial camera to construct a multi-eye stereo vision sensor to collect multi-exposure images of the same part of the aircraft engine from different perspectives; fusing multi-exposure images from different perspectives to obtain a multi-channel image; constructing a multi-class fuse detection model based on deep learning to obtain the position information of the fuse in the multi-channel image; a multi-branch fuse winding direction recognition model based on a visual attention mechanism to obtain the installation winding direction of the fuse in the multi-channel image; fusing the fuse winding direction recognition results from different perspectives to realize the installation defect recognition of aircraft engine fuses.

[0005] The above two technical solutions can complete fuse image acquisition and fuse winding direction identification through different methods, but the detection method has the problems of low efficiency and difficulty in ensuring accuracy. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a fuse detection method and system based on projection technology, which can effectively solve the problems of low efficiency and difficulty in ensuring accuracy.

[0007] The present invention is achieved by adopting the following technical solutions:

[0008] A fuse detection method based on projection technology comprises the following steps:

[0009] Step S1. Acquire an image of an object to be detected including a fastener;

[0010] Step S2. Based on the image of the object to be detected, determining the fastener information of the object to be detected;

[0011] Step S3. Obtaining a fastener image based on the fastener information and determining geometric features of the fastener;

[0012] Step S4. Generate a fuse image based on the geometric features of the fastener, and fuse the fastener image and the fuse image to generate a projection image;

[0013] Step S5. Projecting the projection image onto the surface of the object to be inspected, when the difference between the fuse on the surface of the object to be inspected and the fuse image in the projection image meets a preset condition, determining that the fuse on the object to be inspected is qualified.

[0014] The step S1 specifically includes: acquiring images of the object to be inspected including the fastener from different angles.

[0015] Determining the fastener information of the object to be detected in step S2 specifically refers to: determining the fastener information of the object to be detected based on historical data or a recognition model.

[0016] The step S3 specifically comprises the following steps:

[0017] Step S 31 Based on the fastener information, determine the fastening area in the image of the object to be detected and obtain the fastener image;

[0018] Step S 32 . Preprocessing the fastener image to obtain a processed fastener image;

[0019] Step S 33 . Determine the geometric features of the fastener based on the fastener information and the processed fastener image.

[0020] The fastening area is an area where all fasteners of the same type are located, or the fastening area is a minimum rectangular area that can contain all fasteners of the same type.

[0021] Generating a fuse image based on geometric features of a fastener specifically refers to: generating a fuse image using a trained generation model.

[0022] The generative model includes a generator and a discriminator obtained through separate training, wherein the generator is used to process the geometric features of the fastener and the processed fastener image to determine the initial fuse image; and the discriminator is used to process the initial fuse image to determine the fuse image when the fuse is correctly installed.

[0023] The training method of the discriminator includes: first fixing the initial generator, using the initial generator to randomly simulate and generate an initial fuse image as a negative sample, and sampling from historical data to obtain a positive sample; inputting the above positive sample and negative sample into the initial discriminator, and calculating the error according to the output of the initial discriminator and the sample label; finally using the error back propagation algorithm to update the parameters of the initial discriminator; and obtaining a trained discriminator through parameter updating.

[0024] The training method of the generator includes: first fixing the initial discriminator, using the initial generator to randomly simulate and generate negative samples, and inputting them into the initial discriminator; calculating the error according to the output of the initial discriminator and the sample label, and finally using the error back propagation algorithm to update the parameters of the initial generator; and obtaining a trained generator through parameter updating.

[0025] The fusing of the fastener image and the fuse image in step S4 specifically refers to: overlapping the fastener image and the fuse image to generate a projection image; or, based on the image recognition model, extracting fastener features from the fastener image and fusing them into corresponding positions in the fuse image to generate a projection image; or, based on the image recognition model, extracting fuse features from the fuse image and fusing them into corresponding positions in the fastener image to generate a projection image.

[0026] The step S5 specifically comprises the following steps:

[0027] Step S 51 Determine the scaling ratio of the projection image, determine the projection area based on the fastening area, and project the projection image onto the projection area on the surface of the object to be detected;

[0028] Step S 52 . Compare the similarity between the fuse on the surface of the object to be inspected and the fuse image in the projection image. If the similarity is greater than a preset threshold, the preset condition is met and it is determined that the fuse on the object to be inspected is qualified.

[0029] A fuse detection system based on projection technology, comprising:

[0030] An image acquisition module, used to acquire an image of an object to be inspected including a fastener;

[0031] A first determination module, configured to determine fastener information of the object to be detected based on the image of the object to be detected;

[0032] A second determination module is used to obtain a fastener image based on the fastener information and determine the geometric features of the fastener;

[0033] An image generation module, used for generating a fuse image based on geometric features of the fastener, and fusing the fastener image with the fuse image to generate a projection image;

[0034] The projection judgment module is used to project the projection image onto the surface of the object to be detected, and when the difference between the fuse on the surface of the object to be detected and the fuse image in the projection image meets the preset condition, it is determined that the fuse on the object to be detected is qualified.

[0035] The image acquisition module acquires an image of the object to be detected including the fastener through an image acquisition device, and the number of the image acquisition device is at least one.

[0036] A storage module is also included for storing data and / or instructions.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] 1. In the present invention, the fuse on the surface of the object to be detected is detected by using projection technology, which is simple to operate. While ensuring the detection accuracy, the fuse can be judged more simply and intuitively, thereby improving the efficiency of fuse detection and saving labor costs. In addition, the use of projection technology for fuse detection can also store the relevant data obtained during the detection in real time, which can not only ensure the reliability of the recorded information, but also facilitate daily random inspections.

[0039] 2. In the present invention, the image of the object to be detected including the fastener is acquired from different angles, thereby improving the accuracy of identifying the fastening area in the image of the object to be detected.

[0040] 3. In the present invention, a fastener image is obtained based on the fastener information, and the fastener image is preprocessed to obtain a processed fastener image. Then, based on the fastener information and the processed fastener image, the geometric features of the fastener can be determined more quickly and accurately, thereby laying a foundation for improving the efficiency and accuracy of fuse detection.

[0041] 4. In the present invention, the fuse image is generated by using a trained generation model, which can not only ensure the accuracy of the fuse image, but also improve the generation efficiency of the fuse image.

[0042] 5. The present invention generates a fuse image based on geometric features, which can ensure the reliability of the generated fuse image; the fastener image is merged with the above-mentioned fuse image to generate a projection image, which can intuitively and accurately display the type and connection relationship of the fuse that should be correctly installed by various types of fasteners, so as to better perform fuse detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, wherein:

[0044] Figure 1 It is a schematic diagram of the process of the detection method in the present invention;

[0045] Figure 2 It is a structural schematic diagram of the detection system in the present invention;

[0046] Figure 3 It is a schematic diagram of an application scenario of the present invention;

[0047] Markings in the figure:

[0048] 1. Image acquisition device, 2. Network, 3. Processor, 4. Projection device, 5. Storage device. DETAILED DESCRIPTION

[0049] Example 1

[0050] As a basic implementation mode of the present invention, the present invention includes a fuse detection method based on projection technology, comprising the following steps:

[0051] Step S1. Acquire an image of an object to be inspected including a fastener.

[0052] Step S2: Based on the image of the object to be detected, determine the fastener information of the object to be detected, so that the corresponding fuse type can be determined.

[0053] Step S3: Obtain a fastener image based on the fastener information and determine geometric features of the fastener.

[0054] Step S4. Generate a fuse image based on the geometric features of the fastener, and fuse the fastener image and the fuse image to generate a projection image. Through this step, an image of a correctly installed fuse can be determined based on the combined features of the fuse type and the fastener.

[0055] Step S5. Projecting the projection image onto the surface of the object to be inspected, when the difference between the fuse on the surface of the object to be inspected and the fuse image in the projection image meets a preset condition, determining that the fuse on the object to be inspected is qualified.

[0056] Example 2

[0057] As a preferred embodiment of the present invention, the present invention includes a fuse detection method based on projection technology, comprising the following steps:

[0058] Step S1. Obtain an image of the object to be detected including the fastener. More specifically, the image of the object to be detected can be obtained based on uploading by the user through a terminal (such as a mobile phone, tablet computer, etc.). For example, the user can take a photo of the object to be detected using a mobile phone, and then upload the photo, i.e., the image of the object to be detected.

[0059] Step S2. Based on the image of the object to be detected, determine the fastener information of the object to be detected. More specifically, the fastener information of the object to be detected can be determined based on historical data. For example, the current image of the object to be detected can be matched with the historical data, and then the historical image of the object to be detected with the highest similarity to the current image of the object to be detected in the historical data is used as the reference image of the object to be detected, and the historical fastener information corresponding to the reference image of the object to be detected is directly used as the fastener information of the current image of the object to be detected. Among them, historical data can refer to a collection of all past data, including but not limited to historical images of the object to be detected and historical fastener information. The historical image of the object to be detected can refer to the image of the object to be detected taken before this. The reference image of the object to be detected can refer to the historical image of the object to be detected with the highest similarity to the current image of the object to be detected in the historical data. The historical fastener information can refer to the fastener information corresponding to the historical object to be detected.

[0060] Step S3. Obtain a fastener image based on the fastener information, and then determine the geometric features of the fastener by matching with the historical data based on the historical data. The geometric features of the fastener may refer to the features of the geometric figure formed by the cross section of the fastener, such as the perimeter, center of gravity, axis of symmetry, etc. of the geometric figure.

[0061] Step S4. Generate a fuse image based on the geometric features of the fastener, and fuse the fastener image and the fuse image to generate a projection image. The projection image may be generated by directly acquiring the projection image from the historical data based on the historical data.

[0062] Step S5: Project the projection image onto the surface of the object to be inspected, and when the difference between the fuse on the surface of the object to be inspected and the fuse image in the projection image meets a preset condition, determine that the fuse on the object to be inspected is qualified. The preset condition may refer to a pre-set judgment condition.

[0063] Example 3

[0064] As another preferred embodiment of the present invention, the present invention includes a fuse detection system based on projection technology, Figure 2 , including: an image acquisition module, a first determination module, a second determination module, an image generation module and a projection judgment module.

[0065] Wherein, the image acquisition module is used to acquire an image of the object to be detected including the fastener.

[0066] The first determination module can be used to determine the fastener information of the object to be detected based on the image of the object to be detected. More specifically, the first determination module can include a recognition model, and the fastener information of the object to be detected is determined by the recognition model based on the image of the object to be detected. The recognition model is a machine learning model, the input includes the image of the object to be detected, and the output includes the fastener information, wherein the fastener information includes at least the fastener type and the fastener position.

[0067] The second determination module can be used to obtain a fastener image based on the fastener information and determine the geometric features of the fastener. More specifically, the second determination module can be used to: determine the fastening area in the image of the object to be detected based on the fastener information and obtain the fastener image; pre-process the fastener image to obtain a processed fastener image; determine the geometric features of the fastener based on the fastener information and the processed fastener image, the geometric features at least including the outline and centroid of the fastener.

[0068] The image generation module is used to generate a fuse image based on the geometric features of the fastener, and fuse the fastener image and the fuse image to generate a projection image.

[0069] The projection judgment module is used to project the projection image onto the surface of the object to be detected. When the difference between the fuse on the surface of the object to be detected and the fuse image in the projection image meets a preset condition, it is determined that the fuse on the object to be detected is qualified.

[0070] The image acquisition module, the first determination module, the second determination module, the image generation module and the projection judgment module may be different modules in one system, or one module may realize the functions of two or more of the above modules. For example, each module may share a storage module, or each module may have its own storage module.

[0071] Example 4

[0072] As the best embodiment of the present invention, the present invention includes a fuse detection system based on projection technology, including an image acquisition module, a first determination module, a second determination module, an image generation module, a projection judgment module and a storage module. Figure 3 The detection system can be specifically implemented based on the following hardware: an image acquisition device 1, a processor 3, a projection device 4 and a storage device 5.

[0073] More specifically, the image acquisition module can acquire the image of the object to be detected including the fastener from the image acquisition device 1 through the network 2. The image acquisition device 1 can be used to acquire the image of the object to be detected. The image acquisition device 1 includes but is not limited to a camera, a camera, a still camera, etc. The number of image acquisition devices 1 is not limited, and can include only one to acquire and shoot the image of the object to be detected; or can include multiple to acquire the image of the object to be detected from different angles, thereby improving the accuracy of identifying the fastening area in the image of the object to be detected.

[0074] Wherein, the network 2 can be a wired network or a wireless network, etc. or any combination thereof. The detection system may include one or more network access points, such as a base station and / or a wireless access point, etc. One or more hardware in the detection system may be connected to a network access point in the network 2 to exchange data and / or information. The image acquisition device 1, the processor 3, the projection device 4, the storage device 5, etc. may exchange information and / or data via the network 2. For example, the processor 3 may obtain an image of an object to be detected from the image acquisition device 1 via the network 2.

[0075] The first determination module is used to determine the fastener information of the object to be detected based on the image of the object to be detected. The second determination module is used to obtain the fastener image based on the fastener information and determine the geometric features of the fastener. The image generation module is used to generate a fuse image based on the geometric features of the fastener, and fuse the fastener image and the fuse image to generate a projection image. The projection judgment module is used to control the projection device 4 to project the projection image onto the surface of the object to be detected. When the difference between the fuse on the surface of the object to be detected and the fuse image in the projection image meets the preset condition, it is determined that the fuse on the object to be detected is qualified.

[0076] The functions implemented by the above-mentioned first determination module, second determination module, image generation module and projection judgment module can be implemented based on the processor 3. The processor 3 can be used to process information and / or data related to the fuse detection system based on projection technology to perform one or more functions described in this specification. Among them, the processor 3 may include one or more processing engines (for example, a single-chip processing engine or a multi-chip processing engine). The processor 3 can process the data and / or information obtained from the image acquisition device 1 and the storage device 5. For example, the processor 3 can process the image of the object to be detected obtained from the image acquisition device 1, and determine the fastener information of the object to be detected by identifying the model. For another example, the processor 3 can process the current image of the object to be detected and the historical data (i.e., the historical image of the object to be detected) obtained from the image acquisition device 1 and the storage device 5, respectively, and determine the fastener information of the current image to be detected by matching the current image of the object to be detected with the historical data.

[0077] Processor 3 may be a single server or a server group. The server group may be centralized or distributed. For example, processor 3 may be a distributed system. Processor 3 may be local or remote. For example, processor 3 may be connected to storage device 5 via network 2 to access stored information and / or data. For another example, processor 3 may be directly connected to storage device 5 to access stored information and / or data. Processor 3 may also be implemented on a cloud platform or provided in a virtual manner.

[0078] The projection device 4 can communicate directly with the processor 3. For example, the projection device 4 can directly obtain the projection image output by the processor 3, and project the projection image onto the surface of the object to be detected, so as to detect the fuse on the object to be detected, thereby determining whether the fuse meets the requirements. Alternatively, the projection device 4 can communicate indirectly with the processor 3. For example, the projection device 4 can obtain the projection image output by the processor 3 via the network 2.

[0079] The functions implemented by the storage module can be implemented based on the storage device 5. The storage device 5 can store data and / or instructions. The storage device 5 can store data obtained from the image acquisition device 1. The storage device 5 can also store data and / or instructions used by the processor 3 to execute or use to complete the exemplary method described in this application. For example, fastener information of the object to be detected, geometric features of the fastener, projection images, etc. The storage device 5 can also be implemented on a cloud platform.

[0080] Further, the storage device 5 may be connected to the network 2 to communicate with one or more of the image acquisition device 1, the processor 3 and the projection device 4. One or more of the image acquisition device 1, the processor 3 and the projection device 4 may access the data or instructions stored in the storage device 5 via the network 2, or directly connect to communicate. Furthermore, the storage device 5 may be a part of the processor 3.

[0081] Based on the above detection system, the present invention also proposes a fuse detection method based on projection technology, which can be executed by the processor 3. Figure 1 , specifically including the following steps:

[0082] Step S1. Acquire an image of the object to be detected including the fastener. This step may be performed by an image acquisition module. More specifically, the image of the object to be detected may be acquired based on an image acquisition device 1 (such as a camera, a still camera, etc.). For example, a camera may be used to directly photograph the object to be detected to acquire an image of the object to be detected; for another example, multiple cameras may be used to photograph the object to be detected from multiple different angles to acquire images of the object to be detected at different angles.

[0083] The fastener may refer to a type of mechanical part used when two or more parts (or components) are fastened and connected to form a whole. For example, bolts, studs, plugs, pipe joints, cable joints, etc. A fuse may refer to a metal wire used to prevent a fastener from loosening. Two or more fasteners can be connected together using a fuse to prevent them from loosening. Fuses correspond to fasteners. For example, a bolt fuse is used to prevent a bolt from loosening, a plug fuse is used to prevent a plug from loosening, etc.

[0084] The object to be detected may refer to a component that needs to be tested for fuses, such as an aircraft engine, etc. The image of the object to be detected may refer to an image of the area where fasteners and fuses are located on the object to be detected, for example, the structural connection of an aircraft engine, etc.

[0085] Step S2: Based on the image of the object to be detected, determine the fastener information of the object to be detected. This step can be performed by the first determination module.

[0086] Among them, fastener information may refer to information related to the fastener. For example, fastener information may include fastener type and fastener position. Fastener type may refer to the type of fastener. For example, bolts, screws, nuts, etc. Fastener position may refer to the specific position of the fastener on the object to be detected. The fastener position may be represented by coordinates, such as (5, 3), (3, 4), etc. The coordinate origin may be determined based on a preset coordinate rule. Among them, the preset coordinate rule may refer to a pre-set rule for determining the coordinate origin.

[0087] More specifically, the fastener information of the object to be detected can be determined by a trained recognition model. The recognition model can be a machine learning model. For example, the recognition model can include any one or combination of a convolutional neural network model, a deep neural network model, a recurrent neural network model, or other custom model structures. The input of the recognition model can be an image of the object to be detected, and its output can be the fastener information.

[0088] Furthermore, the initial recognition model can be trained based on a large number of labeled training samples. The labeled training samples are input into the initial recognition model, and the parameters of the initial recognition model are updated through training. The training samples may include images of the sample objects to be detected. The labels may be the fastener information corresponding to the samples. The labels may be obtained based on manual annotation and other methods. The initial recognition model may be trained in various ways based on the above samples to update the parameters of the initial recognition model. For example, the training may be based on the gradient descent method and the like. When the trained initial recognition model meets the preset conditions, the training ends and the trained recognition model is obtained. Among them, the preset conditions may be that the loss function result converges or is less than a preset threshold and the like.

[0089] Based on the image of the object to be detected, the fastener information of the object to be detected is determined through a trained recognition model, which can not only ensure the recognition efficiency, but also ensure the accuracy of the fastener information, which is conducive to improving the reliability of fuse detection.

[0090] Step S3. Obtain a fastener image based on the fastener information and determine the geometric features of the fastener. This step can be performed by the second determination module. More specifically, it includes the following steps:

[0091] Step S 31 Based on the fastener information, determine the fastening area in the image of the object to be detected and obtain the fastener image.

[0092] The fastening area may refer to the area where all fasteners of the same type are located. For example, a plug fastening area, a bolt fastening area, etc. Alternatively, the fastening area may be the smallest rectangular area that can contain all fasteners of the same type. More specifically, the fasteners may be classified according to the fastener information, and then according to the classification result and the location of the fasteners of the same type, the smallest rectangular area containing all fasteners of the same type is determined as the corresponding fastening area.

[0093] After the fastening region in the image of the object to be detected is determined, the image of the object to be detected can be segmented based on the fastening region, and the image of the area where the fastening region in the image of the object to be detected is located can be directly determined as the fastener image.

[0094] Step S 32. Preprocess the fastener image to obtain a processed fastener image.

[0095] The preprocessing method may be: converting the fastener image into a grayscale image, and performing inverse binarization processing on the grayscale image according to a preset threshold value, so as to obtain a binary image of the fastener, and the binary image is the processed fastener image. A grayscale image refers to an image in which each pixel has only one sampled color; a preset threshold value may refer to a pre-set grayscale value; and inverse binarization processing may refer to a method for converting a grayscale image into a binary image. A binary image may refer to an image in which each pixel is black or white.

[0096] Step S 33 . Determine the geometric features of the fastener based on the fastener information and the processed fastener image.

[0097] The geometric features of the fastener include at least the outline and centroid of the fastener. The outline of the fastener may refer to the boundary or outline of the fastener pattern; the centroid of the fastener may refer to the geometric center of the fastener.

[0098] Among them, the trained first image recognition model can be used to determine the geometric features of the fastener. Specifically, the outline of the fastener can be determined by the first image recognition model, and then the centroid of the fastener can be determined by a preset algorithm in combination with the position of the fastener. Among them, the preset algorithm can be a preset centroid coordinate calculation method. For example, a centroid coordinate calculation formula, etc. Therefore, the processed fastener image can be processed by the first image recognition model to determine the outline of the fastener, and then the centroid coordinates of the fastener can be determined by the centroid coordinate calculation formula based on the coordinate information of the fastener position.

[0099] The first image recognition model may be a machine learning model. Training samples may be obtained based on historical data, sample labels may be determined by manual annotation, etc., and the first image recognition model may be trained based on a large number of labeled training samples by various methods (such as gradient descent method, etc.) to obtain a trained first image recognition model.

[0100] Step S4. Generate a fuse image based on the geometric features of the fastener, and fuse the fastener image and the fuse image to generate a projection image. This step can be performed by an image generation module. There will be different fuse images based on the type of fuse and the connection relationship of the fuse. For example, a bolt fuse image, a plug fuse image, a cable plug fuse image, etc.

[0101] Specifically, based on the geometric features of the fastener, a fuse image may be generated by a preset method, wherein the preset method may refer to a preset method for generating a fuse image.

[0102] The preset method may be: based on the geometric features of the fastener, by matching with historical data, the fuse image generated by the geometric features of the historical fastener with the highest similarity to the geometric features of the current fastener in the historical data is directly used as the fuse image generated by the geometric features of the current fastener, etc. The geometric features of the historical fastener may refer to the geometric features of the fastener determined based on the historical fastener information.

[0103] The preset method may also be: generating a fuse image based on preset rules. The preset rules may include determining the connection position of the fuse on each fastener according to the geometric features of the fastener. For example, the centroid positions and centroid lines of adjacent plug fasteners may be determined as their connection positions and the connection mode of the fuse, and the plug fasteners may be connected in sequence based on the above connection positions to generate a fuse image; for another example, if the connection positions of adjacent bolt fasteners are located on both sides of the centroid line, the bolt fasteners may be connected in sequence based on the connection positions on both sides of the centroid line to generate a fuse image.

[0104] The preset method may also be: generating a fuse image based on a trained generative model. The generative model may refer to a model for generating a fuse image. The generative model may be a machine learning model. For example, the generative model may include any one or a combination of a deep belief network model, a variational autoencoder model, or other custom model structures. The input of the generative model may include the geometric features of the fastener, the processed fastener image, and the output thereof may include a fuse image. It should be noted that the fuse image here may be considered to be an image of a fuse when the fastener is correctly installed with a fuse.

[0105] The generative model may include a generator and a discriminator. The generator may process the geometric features of the fastener and the processed fastener image to determine the fuse image. The input of the generator may be the geometric features of the fastener and the processed fastener image, and the output may be the initial fuse image. The generator may be a machine learning model. The discriminator may process the initial fuse image to determine the fuse image when the fuse is correctly installed. The input of the discriminator may be the initial fuse image, and the output may be the fuse image when the fastener is correctly installed with the fuse. The discriminator may also be a machine learning model.

[0106] The generative model can be obtained by training the generator and the discriminator separately. The initial generator and the initial discriminator can be trained based on a large number of labeled training samples. The training samples can be various types of fastener images, and the labels can be various types of fasteners and fuse images when the fuses are correctly installed on them. The training samples can be determined based on historical data. The labels can be determined based on manual annotation and other methods.

[0107] The training process includes: when training the initial discriminator, first fix the initial generator, use the initial generator to randomly simulate and generate the initial fuse image as a negative sample, and sample the positive sample from the historical data (i.e., the fuse image when the fastener correctly installs the fuse); input the above positive and negative samples into the initial discriminator, and calculate the error according to the output of the initial discriminator and the sample label; finally, use the error back propagation algorithm (i.e., based on the output error back propagation, to determine the error signal of each unit of the initial discriminator, and then correct the weight of each unit of the initial discriminator) to update the parameters of the initial discriminator. When training the initial generator, first fix the initial discriminator, use the initial generator to randomly simulate and generate the negative sample, and input it into the initial discriminator; calculate the error according to the output of the initial discriminator and the sample label, and finally use the error back propagation algorithm (i.e., based on the output error back propagation, to determine the error signal of each unit of the initial generator, and then correct the weight of each unit of the initial discriminator) to update the parameters of the initial generator. Through parameter update, the trained generator and discriminator are obtained.

[0108] Generating fuse images by using a trained generative model can not only ensure the accuracy of the fuse images, but also improve the generation efficiency of the fuse images.

[0109] Furthermore, the fastener image and the fuse image are fused to generate a projection image, which can be achieved in a variety of ways:

[0110] Based on the fastener information and the geometric features of the fastener, the fastener image and the fuse image are overlapped to achieve fusion and generate a projection image. For example, based on the fastener type, the fastener position and the geometric features of the fastener, multiple overlap points in the fastener image and the fuse image are determined; then based on the multiple overlap points, the fastener image and the fuse image are overlapped to achieve fusion of the two to generate a projection image. The overlap point can be a reference point when the fastener image and the fuse image overlap, such as a connection position on the fastener (i.e., the end point of the fuse connection relationship), etc.

[0111] Alternatively, based on the trained second image recognition model, the fastener image and the fuse image are fused to generate a projection image. For example, the fastener features in the fastener image are extracted by the second image recognition model, and are fused into the corresponding position in the fuse image, thereby achieving the fusion of the two and generating a projection image. For another example, the fuse features of the fuse image are extracted by the second image recognition model, and are fused into the corresponding position of the fastener image to generate a projection image. The second image recognition model can be a machine learning model, which can obtain training samples based on historical data, determine sample labels by manual annotation, etc., and train the second image recognition model based on a large number of labeled training samples through various methods (such as gradient descent method, etc.) to obtain a trained second image recognition model.

[0112] Step S5. Use the projection device 4 (such as a projector, etc.) to project the projection image onto the surface of the object to be detected. When the difference between the fuse on the surface of the object to be detected and the fuse image in the projection image meets the preset condition, it is determined that the fuse on the object to be detected is qualified. This step can be performed by the projection judgment module. Specifically, it includes the following steps:

[0113] Step S 51 The processor 3 can determine the distance between the projection device 4 and the surface of the object to be detected through the sensing element, and determine the scaling ratio of the projection image based on the distance. The projection area is determined based on the fastening area, and the projection device 4 is controlled to project the projection image onto the projection area on the surface of the object to be detected. When the projection image is projected onto the surface of the object to be detected, its projection size can be close to the size of the actual fastener and the corresponding fuse, so that the fuse on the object to be detected can be better detected.

[0114] Among them, the method of adjusting the projection size can be determined based on a preset adjustment rule. The preset adjustment rule can be a pre-set rule for adjusting the size of the projection image. For example, the preset adjustment rule can be that the distance between the projection device 4 and the surface of the object to be detected is different, and the projection image corresponds to different scaling ratios. For example, the distance between the projection device 4 and the surface of the object to be detected is between (0mm, 5mm], and the size of the projection image remains unchanged; the distance between the projection device 4 and the surface of the object to be detected is between (5mm, 10mm], and the size of the projection image = the size of the original projection image × 0.9, etc. The specific scaling ratio of the projection image can be determined based on historical experience, experiments, etc., and the present invention is not limited to this.

[0115] Step S 52The processor 3 can compare the similarity between the fuse on the surface of the object to be detected and the fuse image in the projection image through the projection judgment module. The more the type, winding direction, and tightness of the fuse on the surface of the object to be detected and the fuse image in the projection image are consistent, the higher the similarity between the two. If the similarity is greater than a preset threshold, the preset condition is met, and it is determined that the fuse on the object to be detected is qualified. When the preset condition is not met, it can be considered that the fuse on the object to be detected is unqualified.

[0116] In summary, after reading the present invention document, ordinary technicians in this field can make various other corresponding transformation schemes based on the technical scheme and technical concept of the present invention without creative mental labor, which all fall within the scope of protection of the present invention.

Claims

1. A fuse detection method based on projection technology, characterized in that: The following steps are involved: Step S1. Acquire an image of an object to be detected including a fastener; Step S2. Based on the image of the object to be detected, determining the fastener information of the object to be detected; Step S3. Obtaining a fastener image based on the fastener information and determining geometric features of the fastener; Step S4. Generate a fuse image based on the geometric features of the fastener, and fuse the fastener image and the fuse image to generate a projection image; Step S5. Projecting the projection image onto the surface of the object to be detected, when the difference between the fuse on the surface of the object to be detected and the fuse image in the projection image meets a preset condition, determining that the fuse on the object to be detected is qualified; The fusion of the fastener image and the fuse image in step S4 specifically refers to: overlapping the fastener image and the fuse image to generate a projection image; or, based on the image recognition model, extracting fastener features from the fastener image and fusing them into corresponding positions in the fuse image to generate a projection image; or, based on the image recognition model, extracting fuse features from the fuse image and fusing them into corresponding positions in the fastener image to generate a projection image.

2. The method for detecting fuses based on projection technology according to claim 1, characterized in that: The step S1 specifically includes: acquiring images of the object to be inspected including the fastener from different angles.

3. The method for detecting fuses based on projection technology according to claim 1, characterized in that: Determining the fastener information of the object to be detected in step S2 specifically refers to: determining the fastener information of the object to be detected based on historical data or a recognition model.

4. The method for detecting fuses based on projection technology according to claim 1, characterized in that: The step S3 specifically comprises the following steps: Step S 31 Based on the fastener information, determine the fastening area in the image of the object to be detected and obtain the fastener image; Step S 32 . Preprocessing the fastener image to obtain a processed fastener image; Step S 33 . Determine the geometric features of the fastener based on the fastener information and the processed fastener image.

5. The method for detecting fuses based on projection technology according to claim 4, characterized in that: The fastening area is an area where all fasteners of the same type are located, or the fastening area is a minimum rectangular area that can contain all fasteners of the same type.

6. The method for detecting fuses based on projection technology according to claim 1, characterized in that: Generating a fuse image based on geometric features of a fastener specifically refers to: generating a fuse image using a trained generation model.

7. The method for detecting fuses based on projection technology according to claim 6, characterized in that: The generative model includes a generator and a discriminator obtained through separate training, wherein the generator is used to process the geometric features of the fastener and the processed fastener image to determine the initial fuse image; and the discriminator is used to process the initial fuse image to determine the fuse image when the fuse is correctly installed.

8. The method for fuse detection based on projection technology according to claim 7, characterized in that: The training method of the discriminator includes: first fixing the initial generator, using the initial generator to randomly simulate and generate an initial fuse image as a negative sample, and sampling from historical data to obtain a positive sample; inputting the above positive sample and negative sample into the initial discriminator, and calculating the error according to the output of the initial discriminator and the sample label; finally using the error back propagation algorithm to update the parameters of the initial discriminator; and obtaining a trained discriminator through parameter updating.

9. The method for fuse detection based on projection technology according to claim 7, characterized in that: The training method of the generator includes: first fixing the initial discriminator, using the initial generator to randomly simulate and generate negative samples, and inputting them into the initial discriminator; calculating the error according to the output of the initial discriminator and the sample label, and finally using the error back propagation algorithm to update the parameters of the initial generator; and obtaining a trained generator through parameter updating.

10. The method for fuse detection based on projection technology according to claim 4, characterized in that: The step S5 specifically comprises the following steps: Step S 51 Determine the scaling ratio of the projection image, determine the projection area based on the fastening area, and project the projection image onto the projection area on the surface of the object to be detected; Step S 52 . Compare the similarity between the fuse on the surface of the object to be inspected and the fuse image in the projection image. If the similarity is greater than a preset threshold, the preset condition is met and it is determined that the fuse on the object to be inspected is qualified.

11. A fuse detection system based on projection technology, characterized in that: include: An image acquisition module, used to acquire an image of an object to be inspected including a fastener; A first determination module, configured to determine fastener information of the object to be detected based on the image of the object to be detected; A second determination module is used to obtain a fastener image based on the fastener information and determine the geometric features of the fastener; An image generation module is used to generate a fuse image based on the geometric features of the fastener, and fuse the fastener image and the fuse image to generate a projection image; wherein fusing the fastener image and the fuse image specifically refers to: overlapping the fastener image and the fuse image, fusing them to generate a projection image; or, based on an image recognition model, extracting fastener features from the fastener image, fusing them into corresponding positions in the fuse image, and generating a projection image; or, based on an image recognition model, extracting fuse features from the fuse image, fusing them into corresponding positions in the fastener image, and generating a projection image; The projection judgment module is used to project the projection image onto the surface of the object to be detected, and when the difference between the fuse on the surface of the object to be detected and the fuse image in the projection image meets the preset condition, it is determined that the fuse on the object to be detected is qualified.

12. The fuse detection system based on projection technology according to claim 11, characterized in that: The image acquisition module acquires an image of the object to be detected including the fastener through an image acquisition device, and the number of the image acquisition device is at least one.

13. The fuse detection system based on projection technology according to claim 11, characterized in that: A storage module is also included for storing data and / or instructions.

Images