Product internal structure defect detection method and system based on AOI multispectral imaging

Abstract

The invention discloses a product internal structure defect detection method and system based on AOI multispectral imaging, and the method comprises the steps: placing a detection device at a prediction position of a to-be-detected product, obtaining an initial multispectral image of the to-be-detected product through a multispectral imaging module, and obtaining the initial position of a corresponding movable probe; the movable probe is controlled to move along a preset path, and multispectral image data of the position where the movable probe is located are collected in real time; image preprocessing is conducted on the multispectral image, defect feature detection is conducted on the interior of the product, a product defect detection result is obtained, corresponding spatial position information is obtained, and real-time depth position information of a corresponding movable probe is obtained based on the initial position and a preset path; performing spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information; the invention provides a product defect detection mode which is convenient for meeting the requirements of modern industrial production on efficiency and cost.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular to a method and system for detecting internal structural defects of products based on AOI multispectral imaging. Background Art

[0002] At present, AOI (Automated Optical Inspection) technology has been widely used in various industrial production, such as agriculture, remote sensing, medical treatment, safety inspection, etc.; however, traditional AOI technology mainly focuses on the detection of surface defects of products, and has limited detection capabilities for internal structures and deep defects of products; currently, X-ray, ultrasonic and other technologies are often used to detect internal structures and deep defects of products, but these methods have problems such as high cost, complex operation and high skill requirements for operators; it is difficult to meet the efficiency and cost requirements of modern industrial production. Summary of the invention

[0003] In order to provide a product defect detection method that can easily meet the efficiency and cost requirements of modern industrial production, the present application provides a product internal structure defect detection method and system based on AOI multi-spectral imaging.

[0004] In the first aspect, the invention objective of the present application is achieved by adopting the following technical solutions: A method for detecting internal structural defects of a product based on AOI multispectral imaging includes a detection device, the detection device includes a multispectral imaging module and at least one movable probe, the movable probe can move along the surface and inside of the product to be detected according to a control instruction, and the method includes: Placing the detection device at the predicted position of the product to be detected, acquiring an initial multispectral image of the product to be detected through a multispectral imaging module, and acquiring a corresponding starting position of the movable probe; The movable probe is controlled to move along a preset path on the surface and inside of the product to be inspected, and the multispectral imaging module collects multispectral image data of the position of the movable probe in real time; the preset path is generated according to the structural information of the product to be inspected; Performing image preprocessing on the initial multispectral image and the multispectral image data; performing defect feature detection inside the product on the preprocessed initial multispectral image and the multispectral image data to obtain a product defect detection result; Based on the product defect detection result, the corresponding spatial position information is obtained, and based on the starting position and the preset path, the corresponding real-time depth position information of the movable probe is obtained; the spatial position information and the corresponding real-time depth position information are spatially matched to obtain product defect target detection information.

[0005] By adopting the above technical solution, the multispectral imaging module is used to emit light of different wavelengths to the surface and interior of the product to be inspected, and can capture the multispectral image data reflected or projected by it. At the same time, the multispectral imaging module collects multispectral image data at the location of the probe in real time; the present application analyzes the spectral characteristics at different wavelengths in the multispectral image, identifies the subtle differences in the internal structure of the product and the potential defect areas, so as to obtain the product defect detection results. The planning movement mode of the movable probe according to the structural characteristics of the product to be inspected can greatly improve the efficiency of the internal defect detection of the product, which is conducive to saving the defect detection cost, and the defect feature identification is performed based on the internal defects of the product. The size difference, shape abnormality, etc. inside the product can be identified through the defect detection algorithm inside the product. The multispectral imaging technology can be used to perform multi-angle and multi-level analysis on the product to be inspected, which is conducive to improving the accuracy of product defect detection. In the process of inspecting the product to be inspected, the inspection results (referring to the product defect detection results) can also be fed back in real time. In modern industrial production, the real-time feedback of product defect detection results can help the staff on the production line to adjust and optimize the production process in time, reduce the scrap rate, and improve production efficiency. Therefore, the present application provides a product defect detection method that is easy to meet the efficiency and cost requirements of modern industrial production.

[0006] In a preferred example of the present application, the defect feature detection inside the product is performed on the pre-processed initial multispectral image and the multispectral image data to obtain the product defect detection result, including: Acquire multi-spectral image data of multiple consecutive frames of the same product to be tested; Extract the defect features inside the product from the preprocessed initial multispectral image and the continuous multi-frame multispectral image data of the same product to be tested, and build a multispectral feature set of the product. The extracted features include color features, texture features, shape features and spectral features. In a preset defect detection and tracking model, a machine learning algorithm is used to analyze a multispectral feature set, identify the internal structure and surface potential defect areas of the product to be detected, obtain corresponding potential defect detection areas, and perform product internal structure defect detection on the potential defect detection areas based on a preset defect detection algorithm to obtain product defect detection results; The initial product defect detection result is classified based on a preset defect classification standard, the defect classification type and defect cause are determined, and defect detection classification information is obtained; and the product defect detection result is optimized based on the defect detection classification information.

[0007] By adopting the above-mentioned technical scheme and using machine learning algorithms to conduct in-depth analysis of multispectral feature sets, it is possible to more accurately identify potential defect areas in the internal structure and surface of products and improve detection accuracy. This application constructs a multispectral feature set and comprehensively applies multiple features to achieve all-round and multi-angle defect detection on the surface and interior of products, which helps to discover those hidden and difficult-to-detect defects in products, ensure comprehensive control of product quality, enhance the comprehensiveness of product defect detection, and by continuously acquiring multi-frame multispectral image data and performing real-time processing, this method can complete defect detection of the internal structure of the product in a short time and improve the real-time performance of detection.

[0008] In a preferred example of the present application, the initial product defect detection result is classified based on a preset defect classification standard, the defect classification type and defect cause are determined, and after the defect detection classification information is obtained, the following is further included: Acquire the three-dimensional size information of the product to be detected, and build a product defect detection model based on the three-dimensional size information and the product defect target detection information; Based on defect detection classification information of different classification types, different display states are used to perform associated display in the product defect detection model.

[0009] By adopting the above technical solution, three-dimensional visualization of defect detection results is achieved. The construction of a product defect detection model not only intuitively displays the defect location, size and shape of the internal structure of the product, but also provides richer spatial information, which helps inspectors to understand the defect situation more comprehensively. Defect detection classification information based on different classification types is distinguished and associated and displayed. In actual application, changes in visual elements such as color, transparency, and highlight can be used to assist inspectors to quickly identify different types of defects, and preliminarily determine their causes and severity, providing strong support for subsequent processing and decision-making.

[0010] In a preferred example of the present application: a method for detecting internal structural defects of a product based on AOI multispectral imaging, further comprising: Acquire material characteristic information of the current product to be inspected, and adjust the spectral range and light source intensity of the multi-spectral imaging module based on the material characteristic information; If defect features are detected in the initial multispectral image, comparing the acquired current light source parameter information with a light source parameter reference interval corresponding to the material characteristic information; When the light source parameter information exceeds the corresponding light source parameter reference interval, light source parameter adjustment information is obtained; based on the light source parameter adjustment information, a light source parameter adjustment instruction is triggered and sent to the multispectral imaging module.

[0011] By adopting the above technical scheme, an adaptive spectral optimization strategy is provided, which realizes adaptive optimization of detection conditions by obtaining material characteristic information of the current product to be detected and adjusting the spectral range and light source intensity of the multispectral imaging module accordingly, thereby significantly improving the sensitivity and accuracy of defect detection; further, by comparing the current light source parameter information obtained with the light source parameter reference interval corresponding to the material characteristic information, and when the light source parameter information exceeds the corresponding light source parameter reference interval, the light source parameter comparison and calibration method of obtaining the light source parameter adjustment information is beneficial to ensure that the light source parameters of the multispectral imaging module are maintained in the optimal state, thereby obtaining higher quality multispectral imaging data, which can not only meet the product detection needs of materials, but also effectively reduce false detection or missed detection caused by improper light source settings, thereby improving the overall detection performance and improving the stability and reliability of internal structural defect detection of products.

[0012] In a preferred example of the present application, the method further includes: obtaining corresponding spatial position information based on the product defect detection result, obtaining corresponding real-time depth position information of the movable probe based on the starting position and the preset path; and spatially matching the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information. Determine the initial spatial position of the multispectral imaging module and the movable probe, the initial spatial position including geographic coordinates or relative positions relative to a preset reference point of the product to be detected; convert the initial spatial position into coordinate values ​​of the same three-dimensional coordinate system; When the movable probe moves along the preset path, the depth position information of the movable probe is recorded in real time, and each multispectral image data is mapped to the corresponding three-dimensional spatial coordinates through a spatial transformation algorithm to obtain the spatial coordinates of the image pixel points in each multispectral image data; The image sequence collected by the multispectral imaging module is timestamped, and a three-dimensional spatial information database based on the image sequence is constructed in combination with the spatial coordinates of each frame of the multispectral image. The three-dimensional spatial information database records the spectral characteristics of the internal structure of the product at different times and locations, and the product is associated with a product identification.

[0013] By adopting the above technical solution, the initial spatial positions of the multispectral imaging module and the movable probe are converted into a unified three-dimensional coordinate system, thereby achieving high-precision spatial positioning of internal structural defects of the product; and in the process of the movable probe moving along the preset path, the depth position information of the movable probe is recorded in real time to improve the real-time depth perception detection capability of the internal structural defects of the product, significantly improving the comprehensiveness and depth of the detection of the internal structure of the product; at the same time, by constructing a three-dimensional spatial information database, the spectral characteristics of the internal structure of the product at different times and positions can be systematically recorded to provide rich data support, and the product identification can be associated with the three-dimensional spatial information database, which can also more accurately discover the defects of the internal structure of the product and reduce the detection cost and time cost.

[0014] In the second aspect, the invention objective of the present application is achieved by adopting the following technical solutions: A product internal structural defect detection system based on AOI multispectral imaging, the system comprising a detection device and a data processing mechanism, the detection device comprising a multispectral imaging module and at least one movable probe, the movable probe being movable along the surface and interior of the product to be detected according to a control instruction; the multispectral imaging module acquiring an initial multispectral image of the product to be detected; The data processing mechanism obtains the starting position of the movable probe, and the data processing mechanism is used to generate a preset path according to the structural information of the product to be detected, and output a movement control instruction to control the movable probe to move along the preset path on the surface and inside of the product to be detected; The data processing mechanism is used to perform image preprocessing on the images of the initial multispectral image and the multispectral image data; perform defect feature detection inside the product on the preprocessed initial multispectral image and the multispectral image data to obtain product defect detection results; obtain corresponding spatial position information based on the product defect detection results, and obtain corresponding real-time depth position information of the movable probe based on the starting position and the preset path; and spatially match the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information.

[0015] By adopting the above technical solution, the multispectral imaging module is used to emit light of different wavelengths to the surface and interior of the product to be inspected, and can capture the multispectral image data reflected or projected by it. At the same time, the multispectral imaging module collects multispectral image data at the location of the probe in real time; the present application analyzes the spectral characteristics at different wavelengths in the multispectral image, identifies the subtle differences in the internal structure of the product and the potential defect areas, so as to obtain the product defect detection results. The planning movement mode of the movable probe according to the structural characteristics of the product to be inspected can greatly improve the efficiency of the internal defect detection of the product, which is conducive to saving the defect detection cost, and the defect feature identification is performed based on the internal defects of the product. The size difference, shape abnormality, etc. inside the product can be identified through the defect detection algorithm inside the product. The multispectral imaging technology can be used to perform multi-angle and multi-level analysis on the product to be inspected, which is conducive to improving the accuracy of product defect detection. In the process of inspecting the product to be inspected, the inspection results (referring to the product defect detection results) can also be fed back in real time. In modern industrial production, the real-time feedback of product defect detection results can help the staff on the production line to adjust and optimize the production process in time, reduce the scrap rate, and improve production efficiency. Therefore, the present application provides a product defect detection method that is easy to meet the efficiency and cost requirements of modern industrial production.

[0016] In the third aspect, the invention objective of the present application is achieved by adopting the following technical solutions: A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the product internal structure defect detection method based on AOI multispectral imaging are implemented.

[0017] In a fourth aspect, the invention objective of the present application is achieved by adopting the following technical solutions: A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned product internal structure defect detection method based on AOI multispectral imaging.

[0018] In summary, the present application includes at least one of the following beneficial technical effects: 1. The multispectral imaging module is used to emit light of different wavelengths to the surface and interior of the product to be inspected, and can capture the multispectral image data reflected or projected by it. At the same time, the multispectral imaging module collects multispectral image data at the location of the probe in real time; the present application analyzes the spectral characteristics of different wavelengths in the multispectral image, identifies the subtle differences in the internal structure of the product and the potential defect areas, so as to obtain the product defect detection results. The movable probe is based on the structural characteristics of the product to be inspected. The planned movement mode of the preset path can greatly improve the efficiency of the internal defect detection of the product, which is conducive to saving the defect detection cost. The defect feature is identified based on the internal defects of the product. The size difference, shape abnormality, etc. inside the product can be identified through the defect detection algorithm inside the product. The multispectral imaging technology can be used to analyze the product to be inspected from multiple angles and multiple levels, which is conducive to improving the accuracy of product defect detection. In the process of inspecting the product to be inspected, the inspection results (referring to the product defect detection results) can also be fed back in real time. In modern industrial production, the real-time feedback of product defect detection results can help the staff on the production line to adjust and optimize the production process in time, reduce the scrap rate, and improve production efficiency. Therefore, the present application provides a product defect detection method that is easy to meet the efficiency and cost requirements of modern industrial production; 2. The initial spatial position of the multispectral imaging module and the movable probe is converted into a unified three-dimensional coordinate system, realizing high-precision spatial positioning of the internal structural defects of the product; and in the process of the movable probe moving along the preset path, the depth position information of the movable probe is recorded in real time to improve the real-time depth perception detection capability of the internal structural defects of the product, significantly improving the comprehensiveness and depth of the detection of the internal structure of the product; at the same time, by constructing a three-dimensional spatial information database, the spectral characteristics of the internal structure of the product at different times and positions can be systematically recorded to provide rich data support, and the product identification can be associated with the three-dimensional spatial information database, which can also more accurately discover the defects of the internal structure of the product and reduce the detection cost and time cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a method for detecting internal structural defects of a product based on AOI multispectral imaging in one embodiment of the present application; Figure 2 This is a flowchart of step S3 in the product internal structure defect detection method based on AOI multispectral imaging in one embodiment of the present application; Figure 3 It is a schematic diagram of a device in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The present application is further described in detail below in conjunction with the accompanying drawings.

[0021] In one embodiment, if Figure 1 As shown, the present application discloses a method for detecting internal structural defects of products based on AOI multispectral imaging, which specifically includes the following steps: S1: Place the detection device at the predicted position of the product to be detected, obtain an initial multispectral image of the product to be detected through the multispectral imaging module, and obtain the corresponding starting position of the movable probe.

[0022] In this embodiment, the detection device is placed at a suitable position according to the design drawings or actual size of the product to be detected, ensuring that the multi-spectral imaging module can cover the key detection area of ​​the product.

[0023] S2: Control the movable probe to move along a preset path on the surface and inside of the product to be inspected, and the multispectral imaging module collects multispectral image data of the position of the movable probe in real time; the preset path is generated according to the structural information of the product to be inspected.

[0024] In this embodiment, a path planning algorithm (such as an RRT algorithm) is used to generate a preset path for the movable probe based on the three-dimensional scanning data of the product to be inspected; the preset path should cover all key inspection areas of the product and consider avoiding non-inspection parts or vulnerable areas of the product; the preset path is converted into a control instruction and sent to the control system of the movable probe; the control system drives the probe to move along the preset path while maintaining an appropriate distance from the product surface.

[0025] Specifically, during the movement of the probe, the multispectral imaging module collects multispectral image data of the position of the probe in real time. A high-precision servo motor or a stepper motor is used to drive the movable probe.

[0026] S3: performing image preprocessing on the initial multispectral image and the multispectral image data; performing defect feature detection inside the product on the preprocessed initial multispectral image and the multispectral image data to obtain a product defect detection result.

[0027] In this embodiment, image preprocessing includes denoising, contrast enhancement, distortion correction, etc. of the initial multispectral image and the multispectral image data collected in real time, which is beneficial to improving the image quality; image processing technology (such as edge detection, texture analysis and morphological operation) and machine learning algorithms (such as convolutional neural networks, support vector machines, etc.) are used to extract defect features from the preprocessed image, wherein the extracted features include color anomalies, texture changes, irregular shapes, and spectral features.

[0028] Specifically, the extracted defect features are compared with the preset defect threshold, the potential defect area inside the product is identified, and the product defect detection result is generated.

[0029] S4: Obtain corresponding spatial position information based on the product defect detection result, and obtain corresponding real-time depth position information of the movable probe based on the starting position and the preset path; perform spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information.

[0030] In this embodiment, based on the pixel position information in the product defect detection results and combined with the relative position of the camera and the product, the specific position information of the product defect in the three-dimensional space is calculated; the movable probe is provided with a position sensor, and the depth position information of the probe is recorded in real time by the position sensor of the movable probe (such as a laser rangefinder, etc.); the product defect target detection information includes the defect position, size, shape, and depth information.

[0031] Specifically, the spatial position information of the defect is matched with the real-time depth position information of the probe to ensure the spatial consistency of the two, so as to map the two-dimensional defect detection results into the three-dimensional product model.

[0032] In one embodiment, if Figure 2 As shown, in step S3, the pre-processed initial multispectral image and multispectral image data are subjected to defect feature detection inside the product to obtain a product defect detection result, including: S31: Acquire multiple frames of multispectral image data of the same product to be tested.

[0033] In this embodiment, the installation position of the multi-spectral imaging module can cover the entire detection area of ​​the product to be tested as much as possible.

[0034] Specifically, the multispectral imaging device is started, and appropriate parameters such as exposure time and gain are set to capture multi-frame multispectral images of the product to be tested in a continuous mode.

[0035] S32: extracting defect features inside the product from the preprocessed initial multispectral image and the continuous multi-frame multispectral image data of the same product to be tested, and constructing a multispectral feature set of the product, wherein the extracted features include color features, texture features, shape features, and spectral features.

[0036] In this embodiment, the pixel values ​​of different bands in the image are analyzed to extract the color distribution and color features of the color histogram; the texture information of the image is extracted using methods such as grayscale co-occurrence matrix and local binary pattern to obtain texture features; the shape features in the image are identified and quantified through edge detection, contour extraction and other technologies; the image differences in different spectral bands are analyzed to extract features such as spectral curves and spectral peaks.

[0037] Specifically, the extracted color, texture, shape, and spectral features are integrated into a multispectral feature set.

[0038] S33: In a preset defect detection and tracking model, a machine learning algorithm is used to analyze the multispectral feature set to identify the internal structure and surface potential defect areas of the product to be inspected, and the corresponding potential defect detection areas are obtained. Based on the preset defect detection algorithm, the potential defect detection areas are subjected to product internal structure defect detection to obtain product defect detection results.

[0039] In this embodiment, the defect detection and tracking model is pre-trained using a large amount of labeled multispectral image data; for potential defect areas, a preset defect detection algorithm (such as three-dimensional reconstruction, deep learning segmentation, etc.) is applied to perform more in-depth detection to confirm the existence and nature of the defects.

[0040] S34: classify the initial product defect detection results based on a preset defect classification standard, determine the defect classification type and defect cause, and obtain defect detection classification information; optimize the product defect detection results based on the defect detection classification information.

[0041] Specifically, the initial product defect detection results are classified according to preset defect classification standards (such as defect type, severity, cause, etc.) to obtain corresponding defect detection classification information; based on the defect classification and cause analysis results, the initial defect detection results are optimized, such as removing false detections, supplementing missed detections, correcting classifications, etc., to achieve optimization operations on product defect detection results.

[0042] In one embodiment, after step S34, the method for detecting internal structural defects of products based on AOI multispectral imaging further includes: S341: Acquire three-dimensional size information of the product to be inspected, and build a product defect detection model based on the three-dimensional size information and product defect target detection information.

[0043] Specifically, the three-dimensional size information of the product to be inspected is obtained through a three-dimensional scanner; the defect detection classification information (including defect location, size, shape, etc.) obtained in the previous step is spatially aligned with the three-dimensional model of the product to ensure that the defect information can be accurately mapped to the three-dimensional model; based on the defect detection classification information, the defect area is marked on the three-dimensional model and assigned corresponding attributes (such as defect type, severity, etc.).

[0044] S342: Based on the defect detection classification information of different classification types, different display states are used to perform associated display in the product defect detection model.

[0045] In this embodiment, different display states include: different colors, transparencies, icons, etc.

[0046] Specifically, when a user views the product defect detection model, the corresponding display state will be automatically applied according to the classification information of the defect; the product defect detection model of the present application allows the user to view defects of a specific classification type or switch display states through interactive operations (such as clicking, selecting, etc.).

[0047] In one embodiment, the method for detecting internal structural defects of a product based on AOI multispectral imaging further includes: S51: Obtain material characteristic information of the current product to be inspected, and adjust the spectral range and light source intensity of the multi-spectral imaging module based on the material characteristic information.

[0048] In this embodiment, the material characteristic information includes but is not limited to the absorption spectrum, reflection spectrum, transmission spectrum and possible fluorescence characteristics of the material; based on the acquired material characteristic information, its response characteristics to different spectral bands are analyzed to determine which bands are most sensitive to defect detection; based on the material characteristic analysis results, the spectral range of the multi-spectral imaging module is adjusted to ensure coverage of spectral bands that are critical to defect detection, and at the same time, according to the reflectivity, transmittance and other characteristics of the material, the intensity of the light source is adjusted to optimize the signal-to-noise ratio and contrast of the image to facilitate the subsequent extraction of defect features.

[0049] Specifically, the present application takes into account the differences between different materials when adjusting the spectral range and the light source intensity, that is, adopts a flexible spectral parameter adjustment strategy.

[0050] S52: If defect features are detected in the initial multispectral image, the acquired current light source parameter information is compared with a light source parameter reference interval corresponding to the material characteristic information.

[0051] Specifically, the current light source parameter information includes light source type, wavelength, intensity, and the like.

[0052] S53: When the light source parameter information exceeds the corresponding light source parameter reference interval, obtain light source parameter adjustment information; based on the light source parameter adjustment information, trigger a light source parameter adjustment instruction and send it to the multispectral imaging module.

[0053] Specifically, if the light source parameter information exceeds the corresponding light source parameter reference range, the appropriate light source parameter adjustment information is calculated or queried based on the material characteristic information and defect detection requirements to effectively ensure long-term and high-precision defect detection of the product's internal structure.

[0054] Furthermore, the adjustment of light source parameters should be smooth to avoid sudden impact on imaging quality.

[0055] In one embodiment, before step S4, the product internal structure defect detection method based on AOI multi-spectral imaging further includes: S41: Determine the initial spatial position of the multispectral imaging module and the movable probe, the initial spatial position including the geographic coordinates or the relative position relative to a preset reference point of the product to be inspected; convert the initial spatial position into a coordinate value of the same three-dimensional coordinate system.

[0056] In this embodiment, the initial position information includes X-, Y-, and Z-axis coordinates and rotation angles (such as pitch angle, yaw angle, and roll angle). A high-precision positioning device (such as GPS, a laser rangefinder, or a robotic arm encoder) is used to determine the initial geographic coordinates of the multispectral imaging module and the movable probe or their relative positions relative to a preset reference point of the product to be inspected.

[0057] Specifically, when converting the initial spatial position into the coordinate value of the same three-dimensional coordinate system, the designed coordinate transformation includes transformations such as translation, rotation and scaling of the coordinate system; the unified coordinate system provides the basis for subsequent image processing and spatial matching, ensuring the accuracy and consistency of the data. S42: When the movable probe moves along the preset path, the depth position information of the movable probe is recorded in real time, and each multispectral image data is mapped to the corresponding three-dimensional spatial coordinates through a spatial transformation algorithm to obtain the spatial coordinates of the image pixel points in each multispectral image data.

[0058] Specifically, when the movable probe moves along a preset path, a position depth sensing device is used to record the depth position information of the probe in real time; a spatial transformation algorithm is used to map the pixel points in the image to the corresponding three-dimensional space coordinates for each multispectral image data; spatial transformation and mapping enable the multispectral image data to be analyzed and processed in three-dimensional space, thereby improving the accuracy and intuitiveness of detection.

[0059] S43: Timestamp the image sequence collected by the multispectral imaging module, and build a three-dimensional spatial information database based on the image sequence in combination with the spatial coordinates of each frame of the multispectral image. The three-dimensional spatial information database records the spectral characteristics of the internal structure of the product at different times and locations, and the product is associated with a product identification.

[0060] Specifically, a three-dimensional spatial information database based on the image sequence is constructed by combining the spatial coordinates and timestamp information of each frame of the multispectral image; the three-dimensional spatial information database should be able to record the spectral characteristics of the internal structure of the product at different times and locations; by associating the product identification with the three-dimensional spatial information database, subsequent data retrieval and analysis are facilitated, and the association of the product identification ensures the traceability and manageability of the data.

[0061] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0062] In one embodiment, a product internal structure defect detection system based on AOI multi-spectral imaging is provided, and the product internal structure defect detection system based on AOI multi-spectral imaging corresponds to the product internal structure defect detection method based on AOI multi-spectral imaging in the above embodiment.

[0063] The product internal structure defect detection system based on AOI multispectral imaging includes a detection device and a data processing mechanism. The detection device includes a multispectral imaging module and at least one movable probe. The movable probe can move along the surface and inside of the product to be detected according to the control instruction; the multispectral imaging module obtains the initial multispectral image of the product to be detected; The data processing mechanism obtains the starting position of the movable probe, and the data processing mechanism is used to generate a preset path according to the structural information of the product to be detected, and output a movement control instruction to control the movable probe to move along the preset path on the surface and inside of the product to be detected; The data processing mechanism is used to perform image preprocessing on the initial multispectral image and the multispectral image data; perform defect feature detection inside the product on the preprocessed initial multispectral image and the multispectral image data to obtain product defect detection results; obtain corresponding spatial position information based on the product defect detection results, and obtain corresponding real-time depth position information of the movable probe based on the starting position and the preset path; perform spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information.

[0064] For the specific limitations of the product internal structure defect detection system based on AOI multi-spectral imaging, please refer to the limitations of the product internal structure defect detection method based on AOI multi-spectral imaging above, which will not be repeated here; each module in the above-mentioned product internal structure defect detection system based on AOI multi-spectral imaging can be fully or partially implemented by software, hardware and their combination; each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0065] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store product defect detection results and product defect target detection information, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a product internal structure defect detection method based on AOI multi-spectral imaging is implemented.

[0066] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: S1: placing the detection device at the predicted position of the product to be detected, obtaining an initial multispectral image of the product to be detected through a multispectral imaging module, and obtaining a corresponding starting position of the movable probe; S2: Control the movable probe to move along a preset path on the surface and inside of the product to be inspected, and the multispectral imaging module collects multispectral image data of the position of the movable probe in real time; the preset path is generated according to the structural information of the product to be inspected; S3: performing image preprocessing on the initial multispectral image and the image of the multispectral image data; performing defect feature detection inside the product on the preprocessed initial multispectral image and the multispectral image data to obtain a product defect detection result; S4: Obtain corresponding spatial position information based on the product defect detection result, and obtain corresponding real-time depth position information of the movable probe based on the starting position and the preset path; perform spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information.

[0067] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: S1: placing the detection device at the predicted position of the product to be detected, obtaining an initial multispectral image of the product to be detected through a multispectral imaging module, and obtaining a corresponding starting position of the movable probe; S2: Control the movable probe to move along a preset path on the surface and inside of the product to be inspected, and the multispectral imaging module collects multispectral image data of the position of the movable probe in real time; the preset path is generated according to the structural information of the product to be inspected; S3: performing image preprocessing on the initial multispectral image and the image of the multispectral image data; performing defect feature detection inside the product on the preprocessed initial multispectral image and the multispectral image data to obtain a product defect detection result; S4: Obtain corresponding spatial position information based on the product defect detection result, and obtain corresponding real-time depth position information of the movable probe based on the starting position and the preset path; perform spatial matching on the spatial position information and the corresponding real-time depth position information to obtain product defect target detection information.

[0068] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0069] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0070] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the features thereof may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting internal structural defects of products based on AOI multispectral imaging, characterized in that: The method comprises a detection device, the detection device comprises a multi-spectral imaging module and at least one movable probe, the movable probe can move along the surface and inside of the product to be detected according to a control instruction, and the method comprises: Placing the detection device at the predicted position of the product to be detected, placing the detection device at the predicted position of the product to be detected, obtaining an initial multispectral image of the product to be detected through a multispectral imaging module, and obtaining a corresponding starting position of the movable probe; The movable probe is controlled to move along a preset path on the surface and inside of the product to be inspected, and the multispectral imaging module collects multispectral image data of the position of the movable probe in real time; the preset path is generated according to the structural information of the product to be inspected; Performing image preprocessing on the initial multispectral image and the multispectral image data; performing defect feature detection inside the product on the preprocessed initial multispectral image and the multispectral image data to obtain a product defect detection result; Based on the product defect detection result, the corresponding spatial position information is obtained, and based on the starting position and the preset path, the corresponding real-time depth position information of the movable probe is obtained; the spatial position information and the corresponding real-time depth position information are spatially matched to obtain product defect target detection information.

2. The method for detecting internal structural defects of products based on AOI multispectral imaging according to claim 1, characterized in that: The method of performing defect feature detection on the preprocessed initial multispectral image and multispectral image data to obtain a product defect detection result includes: Acquire multi-spectral image data of multiple consecutive frames of the same product to be tested; Extract the defect features inside the product from the preprocessed initial multispectral image and the continuous multi-frame multispectral image data of the same product to be tested, and build a multispectral feature set of the product. The extracted features include color features, texture features, shape features and spectral features. In a preset defect detection and tracking model, a machine learning algorithm is used to analyze a multispectral feature set to identify the internal structure and surface potential defect areas of the product to be detected, obtain corresponding potential defect detection areas, and perform product internal structure defect detection on the potential defect detection areas based on a preset defect detection algorithm to obtain product defect detection results; The initial product defect detection result is classified based on a preset defect classification standard, the defect classification type and defect cause are determined, and defect detection classification information is obtained; and the product defect detection result is optimized based on the defect detection classification information.

3. The method for detecting internal structural defects of products based on AOI multispectral imaging according to claim 2, characterized in that: After classifying the initial product defect detection result based on a preset defect classification standard, determining the defect classification type and defect cause, and obtaining defect detection classification information, the method further includes: Acquire the three-dimensional size information of the product to be detected, and build a product defect detection model based on the three-dimensional size information and the product defect target detection information; Based on defect detection classification information of different classification types, different display states are used to perform associated display in the product defect detection model.

4. The method for detecting internal structural defects of products based on AOI multispectral imaging according to claim 1, characterized in that: Also includes: Acquire material characteristic information of the current product to be inspected, and adjust the spectral range and light source intensity of the multi-spectral imaging module based on the material characteristic information; If defect features are detected in the initial multispectral image, comparing the acquired current light source parameter information with a light source parameter reference interval corresponding to the material characteristic information; When the light source parameter information exceeds the corresponding light source parameter reference interval, light source parameter adjustment information is obtained; based on the light source parameter adjustment information, a light source parameter adjustment instruction is triggered and sent to the multispectral imaging module.

5. The method for detecting internal structural defects of products based on AOI multispectral imaging according to claim 1, characterized in that: The corresponding spatial position information is obtained based on the product defect detection result, and the corresponding real-time depth position information of the movable probe is obtained based on the starting position and the preset path; Before spatially matching the spatial position information with the corresponding real-time depth position information to obtain product defect target detection information, the method further includes: Determine the initial spatial position of the multispectral imaging module and the movable probe, the initial spatial position including geographic coordinates or relative positions relative to a preset reference point of the product to be detected; convert the initial spatial position into coordinate values ​​of the same three-dimensional coordinate system; When the movable probe moves along the preset path, the depth position information of the movable probe is recorded in real time, and each multispectral image data is mapped to the corresponding three-dimensional spatial coordinates through a spatial transformation algorithm to obtain the spatial coordinates of the image pixel points in each multispectral image data; The image sequence collected by the multispectral imaging module is timestamped, and a three-dimensional spatial information database based on the image sequence is constructed in combination with the spatial coordinates of each frame of the multispectral image. The three-dimensional spatial information database records the spectral characteristics of the internal structure of the product at different times and locations, and the product is associated with a product identification.

6. The product internal structure defect detection system based on AOI multispectral imaging is characterized by: The system comprises a detection device and a data processing mechanism, wherein the detection device comprises a multispectral imaging module and at least one movable probe, wherein the movable probe can move along the surface and inside of the product to be detected according to a control instruction; the multispectral imaging module obtains an initial multispectral image of the product to be detected; The data processing mechanism obtains the starting position of the movable probe, and the data processing mechanism is used to generate a preset path according to the structural information of the product to be detected, and output a movement control instruction to control the movable probe to move along the preset path on the surface and inside of the product to be detected; The data processing mechanism is used to perform image preprocessing on the initial multispectral image and the image of the multispectral image data; Performing defect feature detection inside the product on the preprocessed initial multispectral image and multispectral image data to obtain a product defect detection result; Acquire corresponding spatial position information based on the product defect detection result, and acquire corresponding real-time depth position information of the movable probe based on the starting position and the preset path; The spatial position information and the corresponding real-time depth position information are spatially matched to obtain product defect target detection information.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the product internal structure defect detection method based on AOI multi-spectral imaging as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the product internal structure defect detection method based on AOI multi-spectral imaging as claimed in any one of claims 1 to 5 are implemented.

Images