Method and system for detecting internal structure defects of a product based on aoi multispectral imaging

Abstract

The application discloses a product internal structure defect detection method and system based on AOI multispectral imaging. The method comprises the following steps: placing a detection device at a predicted position of a product to be detected; acquiring an initial multispectral image of the product to be detected through a multispectral imaging module, and acquiring a starting position of a corresponding movable probe; controlling the movable probe to move along a preset path, and collecting multispectral image data of the position of the movable probe in real time; performing image preprocessing on the multispectral image and detecting defect features in the product, obtaining a product defect detection result, acquiring corresponding spatial position information, acquiring real-time depth position information of the corresponding movable probe based on the starting position and the preset path; and performing spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information. The application provides a product defect detection method which can meet the requirements of modern industrial production on efficiency and cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, in particular to a product internal structure defect detection method and system based on AOI multispectral imaging. BACKGROUND

[0002] At present, AOI (Automated Optical Inspection, automatic optical inspection) technology has been widely used in various industrial production, such as agriculture, remote sensing, medical treatment, safety inspection, etc. However, the traditional AOI technology mainly focuses on the detection of product surface defects, and has limited detection capability for product internal structure and deep defects. At present, for the detection of product internal structure and deep defects, X-ray, ultrasonic wave and other technologies are often used, but these methods have problems such as high cost, complex operation and high skill requirement for operators, etc. It is difficult to meet the requirements of modern industrial production on efficiency and cost. SUMMARY

[0003] In order to provide a product defect detection method which can meet the requirements of modern industrial production on efficiency and cost, the present application provides a product internal structure defect detection method and system based on AOI multispectral imaging.

[0004] In the first aspect, the application purpose is realized by adopting the following technical scheme:

[0005] The product internal structure defect detection method based on AOI multispectral imaging comprises a detection device, the detection device comprises a multispectral imaging module and at least one movable probe, the movable probe can move along the surface and inside of the product to be detected according to the control instruction, and the method comprises:

[0006] Placing the detection device at the predicted position of the product to be detected, acquiring the initial multispectral image of the product to be detected by the multispectral imaging module, and acquiring the starting position of the movable probe corresponding to the initial multispectral image;

[0007] Controlling the movable probe to move along a preset path on the surface and inside of the product to be detected, the multispectral imaging module real-time collects multispectral image data of the position where the movable probe is located, and the preset path is generated according to the structure information of the product to be detected;

[0008] Image pre-processing is performed on the images of the initial multispectral image and the multispectral image data, and defect feature detection is performed on the pre-processed initial multispectral image and multispectral image data to obtain the product defect detection result;

[0009] acquire corresponding spatial position information based on the product defect detection result, acquire real-time depth position information of the movable probe based on the starting position and the preset path; and perform spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information.

[0010] By adopting the technical solution, the multispectral imaging module is used to emit light of different wavelengths to the surface and interior of the product to be detected, and can capture multispectral image data reflected or projected thereby. Meanwhile, the multispectral imaging module acquires multispectral image data of the position where the probe is located in real time. The present application identifies subtle differences in the internal structure of the product and potential defect areas by analyzing the spectral characteristics of different wavelengths in the multispectral image, so as to obtain a product defect detection result. The movable probe moves in a preset path planning mode according to the structural characteristics of the product to be detected, which can greatly improve the efficiency of internal defect detection of the product, is conducive to saving defect detection costs, and can identify the size difference, shape abnormality, etc. in the interior of the product based on defect feature recognition of the internal defects of the product. The multispectral imaging technology can analyze the product to be detected from multiple angles and multiple levels, which is conducive to improving the accuracy of product defect detection. During the detection of the product to be detected, the detection result (i.e., the product defect detection result) can be fed back in real time. In modern industrial production, the real-time feedback of the product defect detection result can help the staff on the production line to timely adjust and optimize the production process, reduce the scrap rate, and improve the production efficiency. Therefore, the present application provides a product defect detection method that can meet the requirements of efficiency and cost in modern industrial production.

[0011] In a preferred example of the present application, the initial multispectral image and the multispectral image data after preprocessing are subjected to defect feature detection in the interior of the product to obtain a product defect detection result, which includes:

[0012] Acquire continuous multispectral image data of the same product to be tested;

[0013] Extract defect features in the interior of the product from the initial multispectral image and the continuous multispectral image data after preprocessing of the same product to be tested, and construct a multispectral feature set of the product. The extracted features include color features, texture features, shape features, and spectral features.

[0014] In a preset defect detection tracking model, a machine learning algorithm is used to analyze the multispectral feature set, identify potential defect areas in the internal structure and surface of the product to be detected, obtain corresponding potential defect detection areas, and perform product internal structure defect detection on the potential defect detection areas based on a preset defect detection algorithm to obtain a product defect detection result.

[0015] classify the initial product defect detection result based on a preset defect classification standard, determine the classification type and defect cause of the defect, and obtain defect detection classification information; and optimize the product defect detection result based on the defect detection classification information.

[0016] By using the above technical solution, the machine learning algorithm is used to deeply analyze the multispectral feature set, which can more accurately identify the potential defect area of the internal structure and surface of the product, improve the detection accuracy, and the application can realize omnibearing and multi-angle defect detection of the surface and internal structure of the product by constructing the multispectral feature set and comprehensively applying multiple features, which helps to find the defects with strong concealment and difficult to detect, ensures the overall control of product quality, enhances the comprehensiveness of product defect detection, and through continuous acquisition of multiple multispectral image data and real-time processing, the method can complete the defect detection of the internal structure of the product in a short time, and improve the real-time detection.

[0017] In a preferred example of the application: after the initial product defect detection result is classified based on the preset defect classification standard, the classification type and defect cause of the defect are determined, and the defect detection classification information is obtained, the application further comprises:

[0018] Obtain three-dimensional size information of the product to be detected, and construct a product defect detection model based on the three-dimensional size information and the product defect target detection information;

[0019] Based on the defect detection classification information of different classification types, different display states are correspondingly used for associated display in the product defect detection model.

[0020] By using the above technical solution, the three-dimensional visualization of the defect detection result is realized, the product defect detection model is constructed, which not only intuitively displays the defect position, size and shape of the internal structure of the product, but also provides more rich spatial information, which helps the detection personnel to more comprehensively understand the defect situation, and based on the defect detection classification information of different classification types, the different classification types are distinguished and associated displayed, in actual application, based on the changes of different colors, transparency, highlight and other visual elements, it is beneficial to help the detection personnel to quickly identify different types of defects and preliminarily judge the causes and severity, which provides strong support for subsequent processing and decision-making.

[0021] In a preferred example of the application: the product internal structure defect detection method based on AOI multispectral imaging further comprises:

[0022] Obtain material characteristic information of the current product to be detected, and adjust the spectral range and light source intensity of the multispectral imaging module based on the material characteristic information;

[0023] If a defect feature is detected in the initial multi-spectral image, the obtained current light source parameter information is compared with the light source parameter reference interval corresponding to the material characteristic information;

[0024] When the light source parameter information exceeds the corresponding light source parameter reference interval, light source parameter adjustment information is obtained; based on the light source parameter adjustment information, a light source parameter adjustment instruction is triggered and sent to the multi-spectral imaging module.

[0025] By adopting the above technical solution, an adaptive spectrum optimization strategy is provided, the material characteristic information of the current product to be detected is obtained, and the spectrum range and light source intensity of the multi-spectral imaging module are adjusted accordingly, so that adaptive optimization of the detection condition is realized, thereby significantly improving the sensitivity and accuracy of defect detection; further, the light source parameter comparison and calibration mode of comparing the obtained current light source parameter information with the light source parameter reference interval corresponding to the material characteristic information and obtaining light source parameter adjustment information when the light source parameter information exceeds the corresponding light source parameter reference interval is beneficial to ensuring that the light source parameter of the multi-spectral imaging module remains in the best state, thereby obtaining higher-quality multi-spectral imaging data, which not only can adapt to the product detection requirements of different materials, but also can effectively reduce the misjudgment or missed detection caused by improper light source setting, thereby improving the overall detection performance and improving the stability and reliability of the internal structure defect detection of the product.

[0026] In a preferred example of the present application: before the space matching of the spatial position information and the corresponding real-time depth position information to obtain the product defect target detection information, the following steps are further included:

[0027] The initial spatial position of the multi-spectral imaging module and the movable probe is determined, the initial spatial position includes geographic coordinates or relative position relative to a preset reference point of the product to be detected; and the initial spatial position is converted into coordinate values in the same three-dimensional coordinate system;

[0028] During the movement of the movable probe along the preset path, the depth position information of the movable probe is recorded in real time, and each multi-spectral image data is mapped to the corresponding three-dimensional space coordinate through a space transformation algorithm to obtain the spatial coordinates of the image pixels in each multi-spectral image data;

[0029] The image sequence collected by the multi-spectral imaging module is time-stamped, and a three-dimensional space information database based on the image sequence is constructed in combination with the spatial coordinates of each multi-spectral image, the three-dimensional space information database records the spectral characteristics of the internal structure of the product at different times and positions, and the product is associated with a product identifier.

[0030] By adopting the above technical solutions, the initial spatial positions of the multispectral imaging module and the movable probe are converted into a unified three-dimensional coordinate system, realizing high-precision spatial positioning of internal structural defects of a product. During movement of the movable probe along the preset path, the depth position information of the movable probe is recorded in real time, improving the real-time depth perception detection capability of internal structural defects of a product, significantly improving the detection comprehensiveness and depth of the internal structure of the product. Meanwhile, by constructing a three-dimensional spatial information database, the spectral characteristics of the internal structure of the product at different times and positions can be systematically recorded to provide rich data support. By associating the product identifier with the three-dimensional spatial information database, the internal structural defects of the product can be more accurately found, and the detection cost and time cost are reduced.

[0031] In a second aspect, the application aims to achieve the following technical solutions:

[0032] The product internal structural defect detection system based on AOI multispectral imaging comprises a detection device and a data processing mechanism. The detection device comprises a multispectral imaging module and at least one movable probe. The movable probe can move along the surface and inside of the product to be detected according to the control instruction. The multispectral imaging module acquires the initial multispectral image of the product to be detected.

[0033] The data processing mechanism acquires the starting position of the movable probe. The data processing mechanism is used to generate a preset path according to the structural information of the product to be detected, and output a movement control instruction to control the movable probe to move along the preset path on the surface and inside of the product to be detected.

[0034] The data processing mechanism is used to pre-process the images of the initial multispectral image and multispectral image data. The pre-processed initial multispectral image and multispectral image data are used to detect the internal defect features of the product, obtaining the product defect detection result. The corresponding spatial position information is obtained based on the product defect detection result. The real-time depth position information of the movable probe is obtained based on the starting position and the preset path. The spatial position information and the corresponding real-time depth position information are matched in space, obtaining the product defect target detection information.

[0035] By adopting the technical scheme, the multispectral imaging module is used for emitting light rays of different wavelengths to the surface and the interior of the product to be detected, and can capture multispectral image data reflected or projected thereby, and the multispectral imaging module can collect multispectral image data of a position where the probe is located in real time; the application can analyze spectral characteristics of different wavelengths in the multispectral image, identify subtle differences and potential defect areas of the internal structure of the product, and obtain a product defect detection result. The movable probe can be moved according to a preset path based on the structural characteristics of the product to be detected, which can greatly improve the efficiency of internal defect detection of the product, is beneficial to saving the cost of defect detection, and can identify the size difference, shape abnormality and the like of the internal structure of the product based on the internal defect of the product. The multispectral imaging technology can analyze the product to be detected from multiple angles and multiple levels, which is beneficial to improving the accuracy of product defect detection. During the detection of the product to be detected, the detection result (i.e., the product defect detection result) can be fed back in real time. In modern industrial production, the real-time feedback of the product defect detection result can help the staff on the production line to timely adjust and optimize the production process, reduce the waste rate, and improve the production efficiency. Therefore, the application provides a product defect detection method which can meet the requirements of efficiency and cost in modern industrial production.

[0036] In a third aspect, the application achieves the above technical effects by adopting the following technical scheme:

[0037] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned product internal structure defect detection method based on AOI multispectral imaging when executing the computer program.

[0038] In a fourth aspect, the application achieves the above technical effects by adopting the following technical scheme:

[0039] A computer-readable storage medium stores a computer program, and the computer program implements the steps of the above-mentioned product internal structure defect detection method based on AOI multispectral imaging when executed by a processor.

[0040] In summary, the application has at least one of the following beneficial technical effects:

[0041] 1. The multispectral imaging module is used to emit light of different wavelengths to the surface and interior of the product to be detected, and can capture the multispectral image data of its reflection or projection. At the same time, the multispectral imaging module can collect multispectral image data in real time at the position of the probe. The application can identify the subtle differences and potential defect areas of the internal structure of the product by analyzing the spectral characteristics of different wavelengths in the multispectral image, so as to obtain the product defect detection result. Based on the movable probe, the product defect detection result can be greatly improved by planning the movement mode of the preset path according to the structural characteristics of the product to be detected, which is beneficial to save the defect detection cost. Based on the internal defect of the product, the defect feature recognition can identify the size difference, shape anomaly and other internal defects of the product. Through the multispectral imaging technology, the product to be detected can be analyzed from multiple angles and multiple levels, which is beneficial to improve the accuracy of product defect detection. During the detection of the product to be detected, the detection result (referring to the product defect detection result) can be fed back in real time. In modern industrial production, the real-time feedback of the product defect detection result can help the staff on the production line to adjust and optimize the production process in time, reduce the waste rate, and improve the production efficiency, so that the application provides a product defect detection method which can meet the requirements of modern industrial production in efficiency and cost;

[0042] 2. The initial spatial position of the multispectral imaging module and the movable probe is converted into a unified three-dimensional coordinate system, realizing high-precision spatial positioning of the internal structure defect of the product. During the movement of the movable probe along the preset path, the depth position information of the movable probe is recorded in real time, so as to improve the real-time depth perception detection ability of the internal structure defect of the product, and significantly improve the detection comprehensiveness and depth of the internal structure of the product. At the same time, by constructing a three-dimensional spatial information database, the spectral characteristics of the internal structure of the product at different times and positions can be systematically recorded to provide rich data support. By associating the product identification with the three-dimensional spatial information database, the defects of the internal structure of the product can be more accurately found, and the detection cost and time cost can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a flowchart of the product internal structure defect detection method based on AOI multispectral imaging in an embodiment of the application;

[0044] Figure 2 is a flowchart of step S3 in the product internal structure defect detection method based on AOI multispectral imaging in an embodiment of the application;

[0045] Figure 3 is a device schematic diagram in an embodiment of the application. DETAILED DESCRIPTION

[0046] The application will be described in further detail below with reference to the drawings.

[0047] In an embodiment, as shown in Figure 1 The application discloses a product internal structure defect detection method based on AOI multi-spectral imaging, which specifically comprises the following steps:

[0048] S1: Place the detection device at the predicted position of the product to be detected. Place the detection device at the predicted position of the product to be detected. Obtain the initial multi-spectral image of the product to be detected by the multi-spectral imaging module, and obtain the starting position of the corresponding movable probe.

[0049] In this embodiment, according to the design drawing or actual size of the product to be detected, the detection device is placed at a suitable position to ensure that the multi-spectral imaging module can cover the key detection area of the product.

[0050] S2: Control the movable probe to move along the preset path on the surface and inside of the product to be detected, and the multi-spectral imaging module collects multi-spectral image data in real time at the position of the movable probe; the preset path is generated according to the structure information of the product to be detected.

[0051] In this embodiment, the path planning algorithm (such as RRT algorithm) is used to generate the preset path of the movable probe according to the three-dimensional scanning data of the product to be detected; the preset path should cover all the key detection areas of the product and consider avoiding the non-detection part or the vulnerable area of the product; the preset path is converted into control instructions and sent to the control system of the movable probe; the control system drives the probe to move along the preset path while maintaining a proper distance from the surface of the product.

[0052] Specifically, during the movement of the probe, the multi-spectral imaging module collects multi-spectral image data in real time at the position of the probe. A high-precision servo motor or a stepping motor is used to drive the movable probe.

[0053] S3: Image pre-processing is performed on the initial multi-spectral image and the multi-spectral image data; defect feature detection is performed on the pre-processed initial multi-spectral image and multi-spectral image data to obtain the product defect detection result.

[0054] In this embodiment, image pre-processing includes denoising, contrast enhancement, distortion correction, etc. on the initial multi-spectral image and the real-time collected multi-spectral image data, which is beneficial to improve the image quality; image processing techniques (such as edge detection, texture analysis and morphological operations) and machine learning algorithms (such as convolutional neural networks, support vector machines, etc.) are used to extract defect features from the pre-processed images, wherein the extracted features include color abnormalities, texture changes, irregular shapes and spectral features, etc.

[0055] Specifically, the extracted defect features are compared with preset defect thresholds to identify potential defect areas inside the product and generate product defect detection results.

[0056] S4: Obtain the corresponding spatial location information based on the product defect detection results, and obtain the real-time depth location information of the corresponding movable probe based on the starting position and preset path; perform spatial matching between the spatial location information and the corresponding real-time depth location information to obtain the product defect target detection information.

[0057] In this embodiment, based on the pixel position information in the product defect detection results and combined with the relative position of the camera and the product, the specific position information of the product defect in three-dimensional space is calculated; the movable probe is equipped with a position sensor, and the depth position information of the probe is recorded in real time through the position sensor of the movable probe (such as a laser rangefinder); the product defect target detection information includes the defect position, size, shape, and depth information.

[0058] Specifically, the spatial location information of the defect is matched with the real-time depth location information of the probe to ensure spatial consistency between the two, so as to map the two-dimensional defect detection results into the three-dimensional product model.

[0059] In one embodiment, such as Figure 2 As shown, in step S3, the preprocessed initial multispectral image and multispectral image data are used to detect internal defect features of the product, and the product defect detection results are obtained, including:

[0060] S31: Acquire multispectral image data of multiple consecutive frames of the same product under test.

[0061] In this embodiment, the installation location of the multispectral imaging module can cover the entire detection area of ​​the product under test as much as possible.

[0062] Specifically, the multispectral imaging equipment is activated, and appropriate parameters such as exposure time and gain are set to capture multiple frames of multispectral images of the product under test in continuous mode.

[0063] S32: Extract defect features from the preprocessed initial multispectral image and multiple consecutive frames of multispectral image data of the same product under test, and construct a multispectral feature set of the product. The extracted features include color features, texture features, shape features and spectral features.

[0064] In this embodiment, the pixel values of different wavebands in the image are analyzed, the color distribution and color histogram features are extracted, the texture information of the image is extracted by using the gray level co-occurrence matrix and local binary pattern method, and the shape features in the image are identified and quantified by using edge detection and contour extraction technology, and the image differences under different spectral wavebands are analyzed to extract the spectral curve and spectral peak features.

[0065] Specifically, the extracted color, texture, shape and spectral features are integrated into a multi-spectral feature set.

[0066] S33: In the preset defect detection tracking model, the multi-spectral feature set is analyzed by using a machine learning algorithm to identify the internal structure and potential defect area of the surface of the product to be detected, obtain the corresponding potential defect detection area, and perform product internal structure defect detection on the potential defect detection area based on a preset defect detection algorithm to obtain a product defect detection result.

[0067] In this embodiment, the defect detection tracking model is obtained by training a large number of labeled multi-spectral image data in advance; for the potential defect area, a preset defect detection algorithm (such as three-dimensional reconstruction, deep learning segmentation, etc.) is applied for further detection to confirm the existence and nature of the defect.

[0068] S34: Classify the initial product defect detection result based on a preset defect classification standard to determine the classification type and defect cause of the defect to obtain defect detection classification information; and optimize the product defect detection result based on the defect detection classification information.

[0069] Specifically, according to the preset defect classification standard (such as defect type, severity, cause, etc.), the initial product defect detection result is classified to obtain the corresponding defect detection classification information; and based on the defect classification and cause analysis result, the initial defect detection result is optimized, such as removing false positives, supplementing missed detection, correcting classification, etc., to realize the optimization operation of the product defect detection result.

[0070] In an embodiment, after step S34, the product internal structure defect detection method based on AOI multi-spectral imaging further comprises:

[0071] S341: Obtain three-dimensional size information of the product to be detected, and construct a product defect detection model based on the three-dimensional size information and the product defect target detection information.

[0072] Specifically, the three-dimensional dimensions of the product to be inspected are obtained through a 3D scanner; the defect detection classification information (including defect location, size, shape, etc.) obtained in the previous steps is spatially aligned with the three-dimensional model of the product to ensure that the defect information can be accurately mapped onto the three-dimensional model; based on the defect detection classification information, the defect area is marked on the three-dimensional model and assigned corresponding attributes (such as defect type, severity, etc.).

[0073] S342: Based on the defect detection classification information of different classification types, different display states are used to display the corresponding information in the product defect detection model.

[0074] In this embodiment, different display states include: different colors, transparency, icons, etc.

[0075] Specifically, when a user views the product defect detection model, the corresponding display state will be automatically applied based on the defect classification information. The product defect detection model of this application allows users to view defects of specific categories or switch display states through interactive operations (such as clicking, selecting, etc.).

[0076] In one embodiment, the product internal structural defect detection method based on AOI multispectral imaging further includes:

[0077] S51: Obtain the material properties information of the product to be tested, and adjust the spectral range and light source intensity of the multispectral imaging module based on the material properties information.

[0078] In this embodiment, the material property information includes, but is not limited to, the absorption spectrum, reflection spectrum, transmission spectrum, and possible fluorescence properties of the material. Based on the acquired material property information, its response characteristics to different spectral bands are analyzed to determine which bands are most sensitive to defect detection. Based on the material property analysis results, the spectral range of the multispectral imaging module is adjusted to ensure coverage of the spectral bands that are crucial for defect detection. At the same time, based on the reflectivity, transmittance, and other characteristics of the material, the intensity of the light source is adjusted to optimize the signal-to-noise ratio and contrast of the image, which facilitates the subsequent extraction of defect features.

[0079] Specifically, this application takes into account the differences between different materials when adjusting the spectral range and light source intensity, that is, it adopts a flexible spectral parameter adjustment strategy.

[0080] S52: If defect features are detected in the initial multispectral image, the current light source parameter information is compared with the light source parameter reference range corresponding to the material property information.

[0081] Specifically, the current light source parameter information includes the light source type, wavelength, and intensity.

[0082] S53: When the light source parameter information is out of the corresponding light source parameter reference interval, obtain light source parameter adjustment information; based on the light source parameter adjustment information, trigger a light source parameter adjustment instruction and send it to the multispectral imaging module.

[0083] Specifically, if the light source parameter information is out of the corresponding light source parameter reference interval, appropriate light source parameter adjustment information is calculated or queried according to the material characteristic information and the defect detection requirement, so as to effectively guarantee long-time high-precision defect detection of the internal structure of the product.

[0084] Further, the adjustment of the light source parameter should be smoothly transitioned to avoid sudden impact on the imaging quality.

[0085] In an embodiment, before step S4, the product internal structure defect detection method based on AOI multispectral imaging further comprises:

[0086] S41: Determine the initial spatial position of the multispectral imaging module and the movable probe, the initial spatial position including geographic coordinates or relative position with respect to a preset reference point of the product to be detected; convert the initial spatial position into coordinate values in the same three-dimensional coordinate system.

[0087] In this embodiment, the initial position information includes X, Y, Z axis coordinates and rotation angles (such as pitch angle, yaw angle, roll angle); the initial geographic coordinates of the multispectral imaging module and the movable probe or the relative position with respect to a preset reference point of the product to be detected are determined using a high-precision positioning device (such as GPS, laser range finder or mechanical arm encoder).

[0088] Specifically, when converting the initial spatial position into coordinate values in the same three-dimensional coordinate system, the designed coordinate conversion includes translation, rotation and scaling of the coordinate system and the like; the unified coordinate system provides a basis for subsequent image processing and spatial matching, ensuring the accuracy and consistency of the data

[0089] S42: During the movement of the movable probe along the preset path, the depth position information of the movable probe is recorded in real time, and each multispectral image data is mapped to the corresponding three-dimensional spatial coordinate through a spatial transformation algorithm to obtain the spatial coordinates of the image pixel points in each multispectral image data.

[0090] Specifically, during the movement of the movable probe along the preset path, the depth position information of the probe is recorded in real time using a position depth sensing device; for each multispectral image data, a spatial transformation algorithm is used to map the pixel points in the image to the corresponding three-dimensional spatial coordinates; spatial transformation and mapping enable multispectral image data to be analyzed and processed in three-dimensional space, improving the accuracy and intuitiveness of detection.

[0091] S43: Time stamping the image sequence collected by the multispectral imaging module, combining the spatial coordinates of each frame of multispectral image, constructing a three-dimensional spatial information database based on the image sequence, the three-dimensional spatial information database recording the spectral characteristics of the internal structure of the product at different times and positions, and the product being associated with a product identifier.

[0092] Specifically, a three-dimensional spatial information database based on the image sequence is constructed in combination with the spatial coordinates and time stamp information of each frame of multispectral image; the three-dimensional spatial information database should be able to record the spectral characteristics of the internal structure of the product at different times and positions; by associating the product identifier with the three-dimensional spatial information database, subsequent data retrieval and analysis are facilitated, and the association of the product identifier ensures the traceability and manageability of the data.

[0093] It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of the processes should be determined according to their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0094] In an embodiment, a product internal structure defect detection system based on AOI multispectral imaging is provided, which corresponds to the product internal structure defect detection method based on AOI multispectral imaging described above.

[0095] The product internal structure defect detection system based on AOI multispectral imaging comprises a detection device and a data processing mechanism. The detection device comprises a multispectral imaging module and at least one movable probe. The movable probe can move along the surface and inside of the product to be detected according to the control instruction. The multispectral imaging module acquires the initial multispectral image of the product to be detected.

[0096] The data processing mechanism acquires the starting position of the movable probe. The data processing mechanism is used to generate a preset path according to the structure information of the product to be detected, and outputs a movement control instruction to control the movable probe to move along the preset path on the surface and inside of the product to be detected.

[0097] The data processing mechanism is used to perform image preprocessing on the images of the initial multispectral image and the multispectral image data. The preprocessed initial multispectral image and multispectral image data are subjected to internal defect feature detection of the product to obtain a product defect detection result. Based on the product defect detection result, corresponding spatial position information is acquired. Based on the starting position and the preset path, real-time depth position information of the movable probe is acquired. The spatial position information and the corresponding real-time depth position information are spatially matched to obtain product defect target detection information.

[0098] The specific limitations of the product internal structure defect detection system based on AOI multispectral imaging can refer to the limitations of the product internal structure defect detection method based on AOI multispectral imaging described above, and will not be repeated here; each module in the product internal structure defect detection system based on AOI multispectral imaging described above can be realized by software, hardware and their combination; the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each module.

[0099] In one embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in Figure 3 The computer device includes a processor, a memory, a network interface and a database connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store product defect detection results and product defect target detection information. The network interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a product internal structure defect detection method based on AOI multispectral imaging.

[0100] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the following steps:

[0101] S1: placing the detection device at the predicted position of the product to be detected, obtaining the initial multispectral image of the product to be detected by the multispectral imaging module, and obtaining the starting position of the corresponding movable probe;

[0102] S2: controlling the movable probe to move along a preset path on the surface and inside of the product to be detected, and the multispectral imaging module to collect multispectral image data in real time at the position of the movable probe; the preset path is generated according to the structure information of the product to be detected;

[0103] S3: image pre-processing of the initial multispectral image and the multispectral image data; defect feature detection of the pre-processed initial multispectral image and the multispectral image data to obtain product defect detection results;

[0104] S4: obtain corresponding spatial position information based on the product defect detection result, obtain real-time depth position information of the corresponding movable probe based on the starting position and the preset path; and perform spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information.

[0105] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0106] S1: place the detection device at a predicted position of the product to be detected, obtain an initial multispectral image of the product to be detected by a multispectral imaging module, and obtain a starting position of the corresponding movable probe;

[0107] S2: control the movable probe to move along a preset path on the surface and inside of the product to be detected, and the multispectral imaging module collects multispectral image data of the position of the movable probe in real time; the preset path is generated according to the structure information of the product to be detected;

[0108] S3: perform image preprocessing on the images of the initial multispectral image and the multispectral image data; and perform defect feature detection inside the product on the preprocessed initial multispectral image and the multispectral image data to obtain a product defect detection result;

[0109] S4: obtain corresponding spatial position information based on the product defect detection result, obtain real-time depth position information of the corresponding movable probe based on the starting position and the preset path; and perform spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information.

[0110] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified. In actual application, the above-mentioned functions can be completed by different functional units or modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions.

[0112] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the features. Such modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for detecting internal structural defects of a product based on AOI multispectral imaging, characterized in that, The method comprises a detection device comprising a multispectral imaging module and at least one movable probe which can move along the surface and inside of a product to be detected according to control instructions, and the method comprises: placing the detection device at a predicted position of the product to be detected, acquiring an initial multispectral image of the product to be detected by the multispectral imaging module, and acquiring a starting position of the movable probe; controlling the movable probe to move along a preset path on the surface and inside of the product to be detected, and the multispectral imaging module acquires multispectral image data of the position of the movable probe in real time; the preset path is generated according to the structure information of the product to be detected; image preprocessing is performed on the initial multispectral image and the image data of the multispectral image, and defect feature detection is performed on the preprocessed initial multispectral image and the multispectral image data to obtain a product defect detection result; based on the product defect detection result, corresponding spatial position information is acquired, and real-time depth position information of the movable probe is acquired based on the starting position and the preset path; the spatial position information and the corresponding real-time depth position information are spatially matched to obtain product defect target detection information; the defect feature detection on the preprocessed initial multispectral image and the multispectral image data to obtain the product defect detection result comprises: acquiring continuous multispectral image data of the same product to be tested; extracting defect features inside the product from the preprocessed initial multispectral image and the continuous multispectral image data of the same product to be tested to construct a multispectral feature set of the product, and the extracted features include color features, texture features, shape features and spectral features; in a preset defect detection tracking model, a machine learning algorithm is used to analyze the multispectral feature set, identify potential defect areas of the internal structure and surface of the product to be detected, obtain corresponding potential defect detection areas, and perform product internal structure defect detection on the potential defect detection areas based on a preset defect detection algorithm to obtain a product defect detection result; based on a preset defect classification standard, the product defect detection result is classified to determine the classification type and defect cause of the defect, and defect detection classification information is obtained; and the product defect detection result is optimized based on the defect detection classification information.

2. The method for product internal structure defect detection based on AOI multispectral imaging according to claim 1, characterized in that, after the product defect detection result is classified based on the preset defect classification standard, the classification type and defect cause of the defect are determined, and the defect detection classification information is obtained, the method further comprises: acquiring three-dimensional size information of the product to be detected, and constructing a product defect detection model based on the three-dimensional size information and the product defect target detection information; based on the defect detection classification information of different classification types, different display states are correspondingly adopted for associated display in the product defect detection model.

3. The method for product internal structure defect detection based on AOI multispectral imaging according to claim 1, characterized in that, Further comprising: acquiring material characteristic information of the current product to be detected, and adjusting the spectral range and light source intensity of the multispectral imaging module based on the material characteristic information; If a defect feature is detected in the initial multi-spectral image, the obtained current light source parameter information is compared with a light source parameter reference interval corresponding to the material characteristic information; When the light source parameter information exceeds the corresponding light source parameter reference interval, light source parameter adjustment information is obtained; based on the light source parameter adjustment information, a light source parameter adjustment instruction is triggered and sent to the multi-spectral imaging module.

4. The method for product internal structure defect detection based on AOI multispectral imaging according to claim 1, characterized in that, The spatial position information and the corresponding real-time depth position information are spatially matched to obtain product defect target detection information. Before the spatial position information and the corresponding real-time depth position information are spatially matched to obtain product defect target detection information, the method further includes: determining an initial spatial position of the multi-spectral imaging module and the movable probe, the initial spatial position including a geographic coordinate or a relative position with respect to a preset reference point of the product to be detected; and converting the initial spatial position into coordinate values in a same three-dimensional coordinate system; During movement of the movable probe along the preset path, depth position information of the movable probe is recorded in real time, and each multi-spectral image data is mapped to a corresponding three-dimensional spatial coordinate through a spatial transformation algorithm to obtain spatial coordinates of image pixels in each multi-spectral image data; The image sequence collected by the multi-spectral imaging module is time-stamped, and a three-dimensional spatial information database based on the image sequence is constructed in combination with spatial coordinates of each multi-spectral image, the three-dimensional spatial information database recording spectral characteristics of an internal structure of the product at different times and positions, and the product being associated with a product identifier.

5. A system for detecting internal structural defects in a product based on AOI multispectral imaging, characterized in that, The system includes a detection device and a data processing mechanism, the detection device including a multi-spectral imaging module and at least one movable probe, the movable probe being movable along a surface and inside of a product to be detected according to a control instruction; and the multi-spectral imaging module acquiring an initial multi-spectral image of the product to be detected. The data processing mechanism acquires a starting position of the movable probe, generates a preset path according to structure information of the product to be detected, and outputs a movement control instruction to control the movable probe to move along the preset path on the surface and inside of the product to be detected. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The data processing mechanism is configured to perform image preprocessing on the initial multi-spectral image and images of multi-spectral image data. The initial multi-spectral image and the continuous multi-frame multi-spectral image data of the pre-processing of the same product to be tested are used for defect feature extraction in the product, a multi-spectral feature set of the product is constructed, and the extracted features include color features, texture features, shape features, and spectral features; In a preset defect detection tracking model, a machine learning algorithm is used to analyze the multi-spectral feature set, identify potential defect areas of the internal structure and surface of the product to be detected, obtain corresponding potential defect detection areas, and perform product internal structure defect detection on the potential defect detection areas based on a preset defect detection algorithm to obtain a product defect detection result; The product defect detection result is classified based on a preset defect classification standard, the classification type and defect cause of the defect are determined, and defect detection classification information is obtained; and the product defect detection result is optimized based on the defect detection classification information.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the product internal structure defect detection method based on AOI multi-spectral imaging according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor to implement the steps of the product internal structure defect detection method based on AOI multi-spectral imaging according to any one of claims 1 to 4.

Images