Hardware image retrieval method based on deep learning

Abstract

The invention relates to the technical field of data retrieval, in particular to a hardware image retrieval method based on deep learning. The method comprises the following steps: acquiring a multi-angle image of hardware to be retrieved, and generating hardware feature data after geometric correction and feature enhancement; and performing shape fuzzy retrieval on a preset hardware image database by using the feature data to generate a shape-like hardware data set. And carrying out surface grating structure construction and optical response simulation on the shape-like data set to generate hardware grating data. Based on the grating data, simulating a light field and collecting four-dimensional reflected light to obtain a four-dimensional data tensor; and associating the tensor with a preset hardware material attribute, and establishing a spectrum-material attribute relationship. Feature matching is carried out according to the relation, and target hardware data are accurately retrieved; according to the hardware image retrieval method, the hardware image retrieval method can be faster and more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of data retrieval, and in particular to a hardware image retrieval method based on deep learning. Background Art

[0002] In modern manufacturing, hardware products are of various types and shapes. Traditional image retrieval methods often rely on manual feature extraction and simple similarity matching. This method is prone to low retrieval accuracy and poor generalization when dealing with complex hardware shapes and surface features. With the growth of industrial demand, especially in the fields of quality control and product traceability, the demand for efficient and accurate hardware image retrieval technology has become more urgent. In recent years, the application of deep learning technology in the field of computer vision has made significant progress, especially in image feature extraction and classification tasks. However, existing hardware image retrieval methods based on deep learning still face challenges in dealing with multi-angle, complex shapes and surface grating features. These methods usually lack in-depth analysis of three-dimensional light field information and material properties, resulting in inaccuracy and lack of robustness in retrieval results when facing hardware with complex surface textures. Summary of the invention

[0003] Based on this, it is necessary to provide a hardware image retrieval method based on deep learning to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a hardware image retrieval method based on deep learning includes the following steps:

[0005] Step S1: Acquire multi-angle images of hardware to be retrieved; perform geometric correction on the multi-angle images of hardware to be retrieved, and perform feature enhancement processing to obtain hardware feature data;

[0006] Step S2: performing shape fuzzy retrieval on a preset hardware image database based on hardware feature data to generate a shape-like hardware data set;

[0007] Step S3: constructing a surface grating structure for the shape-like hardware data set, and performing optical response simulation to obtain hardware grating data;

[0008] Step S4: simulating the light field of the multi-angle image of the hardware to be retrieved based on the hardware grating data, and collecting four-dimensional reflected light to generate a four-dimensional data tensor;

[0009] Step S5: mapping the four-dimensional data tensor according to the preset hardware material properties to obtain the spectrum-material property relationship;

[0010] Step S6: Based on the spectrum-material property relationship, perform feature matching and precise retrieval on the shape-like hardware data set to generate retrieved hardware data.

[0011] The present invention can effectively improve the accuracy and robustness of hardware feature data and ensure the consistency of features extracted from different viewing angles by performing geometric correction and feature enhancement processing on multi-angle images of hardware to be retrieved. Based on the hardware feature data, the preset hardware image database is subjected to shape fuzzy retrieval, which can flexibly handle hardware objects with complex shapes and generate a similar shape hardware data set, thereby improving the matching accuracy in the retrieval process. By constructing a surface grating structure and simulating the optical response of the similar shape hardware data set, the optical properties of the hardware surface can be accurately simulated to generate hardware grating data, providing high-precision input for subsequent light field simulation and reflected light acquisition. By simulating the light field and collecting four-dimensional reflected light for the multi-angle images of the hardware to be retrieved based on the hardware grating data, the generated four-dimensional data tensor provides a multi-dimensional and high-information basis for the subsequent material property association and spectral feature analysis. By mapping the association relationship between the four-dimensional data tensor and the preset hardware material properties, the relationship between the spectral characteristics and material properties of the hardware can be accurately described, providing higher accuracy for subsequent feature matching. Based on the spectrum-material property relationship, the similar shape hardware data set is subjected to feature matching and accurate retrieval, which can significantly improve the accuracy and speed of hardware image retrieval and meet the hardware product recognition needs in complex industrial environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of the steps of a hardware image retrieval method based on deep learning;

[0013] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0014] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0015] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0016] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0017] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0018] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0019] To achieve this, please refer to Figures 1 to 3 , a hardware image retrieval method based on deep learning, comprising the following steps:

[0020] Step S1: Acquire multi-angle images of hardware to be retrieved; perform geometric correction on the multi-angle images of hardware to be retrieved, and perform feature enhancement processing to obtain hardware feature data;

[0021] Step S2: performing shape fuzzy retrieval on a preset hardware image database based on hardware feature data to generate a shape-like hardware data set;

[0022] Step S3: constructing a surface grating structure for the shape-like hardware data set, and performing optical response simulation to obtain hardware grating data;

[0023] Step S4: simulating the light field of the multi-angle image of the hardware to be retrieved based on the hardware grating data, and collecting four-dimensional reflected light to generate a four-dimensional data tensor;

[0024] Step S5: mapping the four-dimensional data tensor according to the preset hardware material properties to obtain the spectrum-material property relationship;

[0025] Step S6: Based on the spectrum-material property relationship, perform feature matching and precise retrieval on the shape-like hardware data set to generate retrieved hardware data.

[0026] The present invention can effectively improve the accuracy and robustness of hardware feature data and ensure the consistency of features extracted from different viewing angles by performing geometric correction and feature enhancement processing on multi-angle images of hardware to be retrieved. Based on the hardware feature data, the preset hardware image database is subjected to shape fuzzy retrieval, which can flexibly handle hardware objects with complex shapes and generate a similar shape hardware data set, thereby improving the matching accuracy in the retrieval process. By constructing a surface grating structure and simulating the optical response of the similar shape hardware data set, the optical properties of the hardware surface can be accurately simulated to generate hardware grating data, providing high-precision input for subsequent light field simulation and reflected light acquisition. By simulating the light field and collecting four-dimensional reflected light for the multi-angle images of the hardware to be retrieved based on the hardware grating data, the generated four-dimensional data tensor provides a multi-dimensional and high-information basis for the subsequent material property association and spectral feature analysis. By mapping the association relationship between the four-dimensional data tensor and the preset hardware material properties, the relationship between the spectral characteristics and material properties of the hardware can be accurately described, providing higher accuracy for subsequent feature matching. Based on the spectrum-material property relationship, the similar shape hardware data set is subjected to feature matching and accurate retrieval, which can significantly improve the accuracy and speed of hardware image retrieval and meet the hardware product recognition needs in complex industrial environments.

[0027] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of the steps of a hardware image retrieval method based on deep learning of the present invention. In this example, the hardware image retrieval method based on deep learning includes the following steps:

[0028] Step S1: Acquire multi-angle images of hardware to be retrieved; perform geometric correction on the multi-angle images of hardware to be retrieved, and perform feature enhancement processing to obtain hardware feature data;

[0029] In this embodiment, a high-resolution camera is used to shoot the hardware samples at different angles to ensure that sufficient image data is collected. During the shooting process, ensure that the camera setting is fixed at a specific shooting angle to avoid image inconsistency caused by angle changes. The acquired image is geometrically corrected, and the geometric distortion in the image is corrected using the intrinsic parameter calibration model and the extrinsic parameter calibration model. Specifically, the camera calibration tool in OpenCV is used for distortion correction, and the calculated intrinsic and extrinsic parameter matrices are used for image reprojection correction. Then, the corrected image is subjected to feature enhancement processing, and the local binary pattern (LBP) and histogram equalization algorithms are used to enhance the image contrast and details and enhance the significance of the features. The hardware feature data generated by the feature-enhanced image data is further extracted with feature points through contrast enhancement and edge detection technology to facilitate the use of subsequent steps.

[0030] Step S2: performing shape fuzzy retrieval on a preset hardware image database based on hardware feature data to generate a shape-like hardware data set;

[0031] In this embodiment, the hardware feature data is feature extracted through feature descriptors (such as SIFT or ORB), and the matching degree between descriptors is calculated. In order to deal with the ambiguity of the shape, a fuzzy matching technology based on scale space is adopted, and different scales and fuzziness are applied to each image in the image database for feature matching. Specifically, the RANSAC algorithm is used to verify the robustness of the matching results to filter out mismatched data and generate a shape-like hardware data set that meets the fuzzy matching requirements. The matched data is clustered and classified into different shape categories based on feature similarity, thereby forming a shape-like hardware data set containing multiple shape categories for subsequent grating structure construction.

[0032] Step S3: constructing a surface grating structure for the shape-like hardware data set, and performing optical response simulation to obtain hardware grating data;

[0033] In the present embodiment, the three-dimensional model of each type of hardware is imported into an optical simulation software (such as COMSOL Multiphysics). The corresponding grating structure is constructed using the software, and the parameters of the grating such as period, width and depth are set. Afterwards, an optical response simulation is performed, and the electromagnetic response of the grating structure is calculated using the finite difference time domain method (FDTD). In the specific operation, the wavelength range of the simulated light source is set, and the electric field distribution and magnetic field distribution of the grating surface are calculated by an optical simulation tool to generate detailed grating surface field distribution data. High-performance computing resources are used in the simulation process to improve the calculation accuracy, and the simulation results are saved as grating data for subsequent light field simulation.

[0034] Step S4: simulating the light field of the multi-angle image of the hardware to be retrieved based on the hardware grating data, and collecting four-dimensional reflected light to generate a four-dimensional data tensor;

[0035] In this embodiment, an optical model of the hardware to be retrieved is established in the optical simulation software, and the hardware grating data generated above is imported. The simulated light source and light field conditions are set to generate light field data at multiple incident angles. The four-dimensional reflected light acquisition technology is applied to record the reflected light intensity data at each incident angle. The specific steps include setting multiple incident angles, recording the reflected light intensity at each angle, and generating a four-dimensional data tensor by collecting the intensity and wavelength information of the reflected light. Finally, the four-dimensional data tensor contains time, space coordinates, light intensity and wavelength information, providing comprehensive light field data support for subsequent steps.

[0036] Step S5: mapping the four-dimensional data tensor according to the preset hardware material properties to obtain the spectrum-material property relationship;

[0037] In this embodiment, material properties (such as refractive index and reflectivity) are numerically processed to generate material property label vectors. These label vectors are combined with four-dimensional data tensors, and the multi-layer perceptron (MLP) in deep learning is used for model training. During the training process, the gradient descent algorithm is used to optimize the network parameters, and unsupervised pre-training is performed to obtain the initial network parameters. The input of the network is a four-dimensional data tensor, and the output is the predicted result of the material property. Through the optimization process, a mapping model that can effectively map spectral data to material properties is obtained, which can accurately describe the relationship between the spectrum and the material properties.

[0038] Step S6: Based on the spectrum-material property relationship, perform feature matching and precise retrieval on the shape-like hardware data set to generate retrieved hardware data.

[0039] In this embodiment, each sample in the shape-like hardware data set is converted into spectral data through the mapping model generated in the previous step to obtain the spectrum-material property characteristics of each sample. Subsequently, these features are retrieved using a feature matching algorithm (such as a nearest neighbor algorithm). The specific operation includes calculating the similarity between the spectrum-material property characteristics of the hardware sample to be retrieved and the sample characteristics in the shape-like hardware data set, and screening out the most matching hardware samples by setting a similarity threshold. Finally, the retrieved hardware data is generated to provide accurate matching results for the final hardware identification and classification.

[0040] Preferably, step S1 comprises:

[0041] Step S11: Acquire multi-angle images of hardware to be retrieved;

[0042] Step S12: performing edge detection on the multi-angle image of the hardware to be retrieved to obtain the edge contour of the hardware;

[0043] Step S13: geometrically correct the edge contour of the hardware to generate corrected hardware data;

[0044] Step S14: extracting features from the corrected hardware data to obtain hardware feature data;

[0045] Step S15: Perform feature enhancement processing on the corrected hardware data based on the hardware feature data to obtain the hardware feature data.

[0046] The invention is beneficial in that multi-angle images of the hardware parts to be retrieved are obtained to fully reflect their comprehensive appearance features. Multi-angle images help improve the accuracy and robustness of retrieval, and can better cope with the complex appearance changes of hardware parts. The edge contour information of the hardware parts is obtained by edge detection to highlight its geometric shape features. The edge contour lays the foundation for subsequent geometric correction and feature extraction, which helps to improve the retrieval effect. Through feature extraction, effective visual features can be extracted from the hardware parts image to provide support for subsequent feature enhancement and retrieval matching. Feature enhancement processing based on hardware feature data can further highlight the key visual features of hardware parts. The enhanced feature data is more conducive to subsequent image retrieval and improves the accuracy and robustness of retrieval. By making full use of deep learning technology, the high-level semantic features of hardware parts images can be automatically learned. The deep learning model has strong feature expression ability and generalization, which helps to improve the performance of image retrieval. By making full use of multi-angle images, edge contours, geometric correction, feature extraction and enhancement technical means, the visual features of hardware parts can be better extracted and expressed, thereby greatly improving the accuracy and robustness of hardware parts image retrieval, and providing strong support for the intelligent management of the hardware industry.

[0047] In this embodiment, the acquisition process of the multi-angle image of the hardware to be retrieved uses a high-resolution camera to shoot the hardware samples at different angles. Ensure that the shooting equipment is fixed at each angle position to avoid data deviation caused by angle changes. Use an automated turntable and a high-precision camera control system to gradually shoot images of the hardware samples at various angles. During the image acquisition process, the camera is set to a fixed aperture and exposure time to ensure consistent lighting conditions. Finally, a multi-angle hardware image data set is obtained, which includes a comprehensive view of the hardware samples at different angles, providing basic data for subsequent image processing and analysis. When edge detection is performed on the multi-angle image of the hardware to be retrieved, the Canny edge detection algorithm is used, which smoothes the image through a Gaussian filter to reduce the impact of noise. After that, the gradient strength and direction of the image are calculated, the non-edge points in the gradient map are removed by non-maximum suppression, and the edge contour in the image is finally extracted by double threshold detection and edge connection. The specific operation includes converting the multi-angle image into a grayscale image, and then applying the Canny edge detection algorithm to generate a hardware edge contour map. When geometric correction is performed on the edge contour of the hardware, the distortion in the edge contour image is first corrected. Using camera calibration technology, the intrinsic and extrinsic matrices of the image are calculated, and these parameters are used to adjust the geometric deformation in the image. The correction parameters are applied to the edge contour image, and the image geometric transformation function is used to reproject and correct the distortion of the image. Specifically, the cv2.undistort function in the OpenCV library is used to correct the image and generate the corrected hardware data. When extracting features from the corrected hardware data, the corrected image data is first input into a feature extraction algorithm, such as the HOG (Histogram of Directed Gradients) feature extraction algorithm. The algorithm generates feature vectors by calculating the gradient direction and intensity of each pixel in the image. These feature vectors are used to describe the shape and texture information in the image. The specific operation includes using the cv2.HOGDescriptor tool in OpenCV to extract HOG features and save the extracted feature vectors as hardware feature data. When performing feature enhancement processing on the hardware feature data, data enhancement techniques such as histogram equalization and local contrast enhancement are used. First, the histogram of the image is equalized by the cv2.equalizeHist function to improve the contrast of the image. Subsequently, a local contrast enhancement algorithm is applied to enhance the details in the image and improve the recognizability of the features. In the specific operation, the enhanced feature data is input into the deep learning model for training to further extract and optimize the features. Finally, the hardware feature data with feature enhancement is generated.

[0048] Preferably, step S2 comprises:

[0049] Step S21: performing multi-scale shape decomposition processing on the hardware feature data to obtain a multi-scale shape decomposition set;

[0050] Step S22: performing shape Gaussian blur processing on the multi-scale shape decomposition set to generate a simulated shape feature set;

[0051] Step S23: performing deep feature clustering on the simulated shape feature set to obtain a clustered shape feature set;

[0052] Step S24: Based on the clustered shape feature set, a neural network fuzzy matching is performed on the preset hardware image database to generate a class shape hardware data set.

[0053] The invention is beneficial in that it can obtain multi-angle images of the hardware parts to be retrieved, comprehensively reflect their appearance features, help improve the accuracy and robustness of retrieval, and cope with complex appearance changes. Geometric shape information is obtained through edge detection, which lays the foundation for subsequent geometric correction and feature extraction and improves the retrieval effect. In the geometric correction and feature extraction stage, the image distortion caused by shooting factors can be eliminated, and effective visual features can be extracted to support subsequent feature enhancement and retrieval matching. In the feature enhancement processing link, enhancement based on hardware feature data can highlight key visual features, thereby improving the accuracy and robustness of retrieval. The subsequent multi-scale shape decomposition link can extract shape features at different scales and more comprehensively characterize the appearance information, thereby improving the accuracy of retrieval. The subsequent shape Gaussian blur processing simulates the fuzzy characteristics of the shape of hardware parts, enhancing the adaptability and robustness of the method to complex appearance changes. The deep feature clustering link automatically learns the potential shape categories of hardware parts, provides a more effective feature representation for subsequent neural network matching, and improves the performance of retrieval. Finally, using the clustered shape feature set to perform neural network fuzzy matching on the preset hardware image database can better capture the similarities between hardware parts, handle the ambiguity and variability of appearance in real scenes, and further improve the accuracy of retrieval results. The shape-like hardware dataset obtained based on neural network fuzzy matching provides rich training and test data support for subsequent hardware image retrieval tasks, which is conducive to the training and optimization of deep learning models. It makes full use of multi-angle image acquisition, edge contour extraction, geometric correction, feature extraction enhancement technical means, and combines the advantages of deep learning to achieve effective expression and retrieval of the appearance features of hardware parts, providing strong support for the intelligent management of the hardware industry.

[0054] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0055] Step S21: performing multi-scale shape decomposition processing on the hardware feature data to obtain a multi-scale shape decomposition set;

[0056] In this embodiment, when performing multi-scale shape decomposition processing on hardware feature data, the scale space decomposition technology is first applied. The hardware feature data image to be processed is subjected to multi-level Gaussian blur processing to generate image data at different scales. The specific operation includes setting a series of different Gaussian filters and convolving the original image to extract the features of the image at different scales. The gaussian_filter function in the scikit-image library in Python is used to perform scale space decomposition of the image to obtain image features at multiple scales. Subsequently, the image features of different scales are analyzed and compared layer by layer, and finally merged into a multi-scale shape decomposition set. This set contains the shape information of the image at different scales, providing basic data for subsequent shape fuzzy processing.

[0057] Step S22: performing shape Gaussian blur processing on the multi-scale shape decomposition set to generate a simulated shape feature set;

[0058] In this embodiment, when performing shape Gaussian blur processing on a multi-scale shape decomposition set, the Gaussian blur algorithm is first applied to each layer of image data in the decomposition set to simulate the blur effect existing in the real world. Specifically, the cv2.GaussianBlur function in OpenCV is used to blur the image of each scale. The kernel size and standard deviation of the Gaussian blur are set to control the degree of blur. After blurring each image scale, a simulated shape feature set is generated. This set contains feature data of shapes at different degrees of blur, which can better simulate the image blur existing in actual applications, thereby providing a data set with practical application value for deep feature clustering.

[0059] Step S23: performing deep feature clustering on the simulated shape feature set to obtain a clustered shape feature set;

[0060] In this embodiment, when deep feature clustering is performed on the simulated shape feature set, the simulated shape features are first input into a deep convolutional neural network for feature extraction. The image features are extracted using a ResNet or VGG network structure, and then these deep features are used for clustering. In the specific operation, the extracted deep features are grouped using the K-means clustering algorithm, and the K value is set as the number of clusters to determine the category to which each feature belongs. The KMeans function in the scikit-learn library is used for cluster analysis. After clustering processing, a clustered shape feature set is obtained, in which each cluster represents a group of image data with similar shape features. These clustered shape feature sets help to identify and classify similar hardware images, providing an accurate feature set for subsequent matching and retrieval.

[0061] Step S24: Based on the clustered shape feature set, a neural network fuzzy matching is performed on the preset hardware image database to generate a class shape hardware data set.

[0062] In this embodiment, when performing neural network fuzzy matching on a preset hardware image database based on a clustered shape feature set, the clustered shape feature set and the hardware image data in the image database are first input into a pre-trained deep neural network. The network should have fuzzy matching capabilities, such as using a combination of a convolutional neural network (CNN) and a long short-term memory network (LSTM). The specific operation includes extracting features from the images in the database and performing matching calculations with the clustered shape feature set. The images are fuzzily matched using a neural network model, and the similarity score of each image is calculated. Finally, a class-shaped hardware data set is generated based on the matching score, which contains the hardware image data that is most similar in shape to the image to be retrieved, providing an accurate reference for the final retrieval results.

[0063] Preferably, step S3 comprises:

[0064] Step S31: Surface morphology analysis is performed on the shape-like hardware data set to obtain microscopic features of the hardware surface;

[0065] Step S32: constructing a three-dimensional surface grating model according to the microscopic features of the hardware surface;

[0066] Step S33: discretizing the three-dimensional surface grating model to obtain grating unit data;

[0067] Step S34: Perform optical response simulation on the grating unit data to obtain hardware grating data.

[0068] The benefit of the present invention is that by performing surface morphology analysis on a shape-like hardware data set, the microscopic features of the hardware surface can be obtained. This lays the foundation for the subsequent construction of a more accurate three-dimensional surface grating model. According to the obtained hardware surface microscopic features, a three-dimensional surface grating model is constructed, which can more carefully characterize the surface geometric features of the hardware parts. The three-dimensional surface grating model is discretized to obtain grating unit data, which provides the necessary input data for the subsequent optical response simulation. Hardware grating data can be obtained by performing optical response simulation on the grating unit data. These grating data can more accurately reflect the surface characteristics of hardware parts, which helps to improve the accuracy of hardware image retrieval. It can provide more detailed and accurate surface feature data support for hardware image retrieval methods based on deep learning, thereby improving the performance and reliability of retrieval results. These rich surface optical feature information helps to better capture the subtle differences between hardware parts, thereby improving the accuracy and robustness of retrieval.

[0069] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0070] Step S31: Surface morphology analysis is performed on the shape-like hardware data set to obtain microscopic features of the hardware surface;

[0071] In this embodiment, when performing surface morphology analysis on a shape-like hardware data set, each hardware sample image is first photographed at high resolution using a scanning electron microscope (SEM). After the image is acquired, it is processed using an image processing tool to extract surface microscopic features such as roughness, texture, and microstructure. The specific operation includes applying the "roughness" analysis tool and the "texture" analysis function in ImageJ to extract features from the image. The extracted microscopic feature data is statistically analyzed to calculate the uniformity and complexity of the surface structure. The microscopic features of the hardware surface obtained through these processes include various surface irregularities and texture details, which provide detailed data support for the subsequent construction of a three-dimensional grating model.

[0072] Step S32: constructing a three-dimensional surface grating model according to the microscopic features of the hardware surface;

[0073] In this embodiment, when constructing a three-dimensional surface grating model based on the microscopic features of the hardware surface, the extracted microscopic feature data is first converted into a three-dimensional surface mesh model. Computer-aided design (CAD) software, such as SolidWorks or AutoCAD, is used to convert the surface features into three-dimensional coordinate points and construct a corresponding surface grating model. During the operation, the microscopic feature data is input into the CAD software in the form of a point cloud, and these point cloud data are connected into a complete three-dimensional surface through the function of generating a mesh model. The three-dimensional surface grating model accurately reflects the microscopic structural characteristics of the hardware surface, providing a basis for subsequent discretization and optical response simulation.

[0074] Step S33: discretizing the three-dimensional surface grating model to obtain grating unit data;

[0075] In this embodiment, when the three-dimensional surface grating model is discretized, the discretization algorithm in computer graphics is used to convert the continuous three-dimensional model data into discrete grating unit data. First, the three-dimensional model grid is divided into a number of small grid units and processed using the discretization tool in MeshLab. The coordinates and attributes of each small grid unit are extracted to generate a corresponding discrete data set. During the discretization process, the size and resolution of the grid unit should be considered to ensure that the discretization result can accurately describe the surface features of the grating model. The grating unit data finally obtained includes the geometric properties and surface features of each discrete unit, providing the necessary data basis for optical response simulation.

[0076] Step S34: Perform optical response simulation on the grating unit data to obtain hardware grating data.

[0077] In this embodiment, when simulating the optical response of the grating unit data, the discretized grating unit data is first input into the optical simulation software, such as COMSOL Multiphysics or Lumerical FDTD. The light source parameters and the wavelength of the incident light are set to perform simulation calculations of the optical response. The specific operation includes applying the finite difference time domain (FDTD) method to perform electromagnetic field calculations on each grating unit and analyzing the optical response of the grating surface. By post-processing the simulation results, the reflection spectrum and light intensity distribution data of the grating unit at different wavelengths are obtained. Finally, these optical response data are summarized to generate hardware grating data, including the response characteristics and reflection behavior of the grating surface to light, which provides detailed optical feature data for the subsequent image retrieval process.

[0078] Preferably, step S34 includes:

[0079] Step S341: performing electromagnetic field distribution calculation on the grating unit data to obtain surface electromagnetic field distribution data;

[0080] Step S342: performing dynamic optical response simulation on the surface electromagnetic field distribution data to generate grating time variation response data;

[0081] Step S343: performing inverse scattering processing on the grating time-varying response data to obtain grating scattering characteristics;

[0082] Step S344: Perform spectrum analysis on the grating scattering characteristics to generate hardware grating data.

[0083] The benefit of the present invention is that by calculating the electromagnetic field distribution of the grating unit data, detailed electromagnetic field distribution data of the hardware surface can be obtained. These electromagnetic characteristic information can better describe the microscopic physical characteristics of the surface of the hardware parts and provide support for the subsequent dynamic optical response simulation. Based on the surface electromagnetic field distribution data, dynamic optical response simulation is performed to generate grating time-varying response data. This time-domain optical characteristic data can more comprehensively reflect the real-time optical characteristics of the hardware surface under different lighting conditions. The grating time-varying response data is subjected to inverse scattering processing to obtain grating scattering characteristics. These scattering characteristics contain the microscopic structure information of the hardware surface, laying the foundation for the subsequent spectrum analysis. By performing spectrum analysis on the grating scattering characteristics, hardware grating data can be generated. This hardware surface feature data based on frequency domain analysis can more finely characterize the optical characteristics of hardware parts and provide more abundant information support for hardware image retrieval. It can further explore the electromagnetic and optical characteristics of the hardware surface and obtain more detailed and comprehensive hardware grating data. These microscopic physical feature information helps to more accurately describe and distinguish the surface details of different hardware parts, thereby improving the performance and reliability of hardware image retrieval methods based on deep learning.

[0084] In this embodiment, the grating unit data is input into the electromagnetic field simulation software, such as the "Electromagnetic Wave Module" or HFSS in COMSOL Multiphysics. The wavelength of the light source, the angle of incidence, and the geometric parameters of the grating are set. An appropriate calculation grid is selected, and the electromagnetic field distribution solution algorithm is applied to the grating surface, including the finite element method (FEM) or the finite difference method (FDTD). During the simulation process, the electric field and the magnetic field are numerically solved, and the electromagnetic field distribution on the grating surface is recorded. The calculated electromagnetic field distribution data includes the distribution of the electric field intensity and the magnetic field intensity on the grating surface, which accurately reflects the response characteristics of the grating to the electromagnetic wave and provides basic data for the subsequent optical response simulation. The electromagnetic field distribution data is imported into the optical response simulation tool, such as Lumerical FDTD or the "Time Domain Optics Module" in COMSOL Multiphysics. The simulation parameters are set, including the waveform of the light source (such as a pulse waveform), the time step, and the dynamic response time range of the grating. A time domain simulation is performed to calculate the change of the electromagnetic field in time by a numerical method. During the simulation process, the optical response of the grating at different time points is recorded to generate the grating time change response data. The data covers the response characteristics of the grating to the time-varying light signal and reveals the dynamic behavior of the grating. When the grating time-varying response data is inversely scattered, the inverse scattering algorithm is used to analyze the scattering characteristics of the grating. First, the time-varying response data is input into the inverse scattering processing tool, such as the inverse scattering toolbox in MATLAB or the inverse scattering module in COMSOL. The inversion algorithm is applied to extract the scattering characteristics of the grating by calculating the match between the reflection data and the known scattering model. The specific steps include setting the target scattering model, performing inverse scattering calculations, and obtaining the scattering characteristics of the grating. The final grating scattering characteristics contain the scattering intensity of the incident light and its spatial distribution information, providing scattering characteristic data for subsequent spectrum analysis. The grating scattering characteristic data is input into the spectrum analysis tool, such as the "Fourier Transform Toolbox" in MATLAB or the spectrum analysis module in LabVIEW. Perform frequency domain conversion, use Fourier transform to perform spectrum analysis on the scattering characteristic data, and extract the frequency domain information of the scattering characteristics. By analyzing the scattering spectrum, the spectral characteristics of the grating and the response behavior of the grating at different frequencies are identified. Finally, the hardware grating data is generated, which includes the scattering intensity and frequency distribution of the grating under different spectra, accurately reflects the optical characteristics of the hardware grating, and provides detailed data support for image retrieval and recognition.

[0085] Preferably, step S4 comprises:

[0086] Step S41: constructing a light field complex amplitude mapping for the hardware grating data to obtain a light field propagation model;

[0087] Step S42: performing light field simulation on the multi-angle image of the hardware to be retrieved according to the light field propagation model to obtain three-dimensional light field data;

[0088] Step S43: performing bidirectional reflection light intensity calculation on the three-dimensional light field data to obtain reflection light intensity distribution data;

[0089] Step S44: performing spherical harmonic function expansion processing on the reflected light intensity distribution data to obtain a spherical harmonic coefficient matrix;

[0090] Step S45: reconstruct the tensor features of the spherical harmonic coefficient matrix to generate a four-dimensional data tensor.

[0091] The benefit of the present invention lies in that a light field propagation model of the hardware items to be retrieved is constructed by performing light field complex amplitude mapping on the hardware grating data. The optical properties and propagation behavior of the surface of the hardware items can be accurately simulated, laying a solid foundation for the subsequent light field analysis. Based on the light field propagation model, the light field simulation is performed on the multi-angle image of the hardware to be retrieved, and three-dimensional light field data is obtained. These three-dimensional light field data contain rich optical information on the surface of the hardware items, providing more comprehensive data support for subsequent feature extraction and image matching. The three-dimensional light field data is bidirectionally reflected light intensity calculated to obtain reflected light intensity distribution data. These reflected light intensity distribution data can well describe the texture and roughness visual characteristics of the surface of the hardware items, and enhance the effectiveness of the image description. Then, the method performs spherical harmonic function expansion processing on the reflected light intensity distribution data to obtain a spherical harmonic coefficient matrix. The spherical harmonic coefficient matrix contains the frequency domain characteristics of the optical properties of the surface of the hardware items, providing a more robust feature representation for subsequent image matching and retrieval. Finally, by performing tensor feature reconstruction on the spherical harmonic coefficient matrix, the method generates a four-dimensional data tensor. This four-dimensional data tensor can better capture the multi-scale and multi-angle visual features of the surface of hardware items, providing richer input data for deep learning models. It fully utilizes the optical property information of the surface of hardware items to generate a more comprehensive and discriminative visual feature representation. These features not only help improve the accuracy and robustness of hardware image retrieval, but also have broad value in other practical applications involving hardware item recognition.

[0092] In this embodiment, digital holographic microscopy is used to obtain grating data on the surface of hardware, which includes spatial frequency information and reflection characteristics of the surface microstructure. Then, the amplitude and phase information of the light field are reconstructed by inputting the grating data into the complex amplitude mapping algorithm based on Fourier optics. In the complex amplitude mapping process, the interaction between the reflected light and the microstructure of the object surface is used to calculate the propagation path and amplitude change of the light field, and a complete light field propagation model is obtained. The multi-angle images of the hardware to be retrieved are simulated by light field simulation to obtain three-dimensional light field data. Using the ray tracing algorithm in computer graphics, the multi-angle images are used as input to simulate the propagation and reflection behavior of light in three-dimensional space. Under the guidance of the light field propagation model, the ray tracing algorithm gradually calculates the propagation path, energy attenuation and phase change of the light to generate three-dimensional light field data. The data contains the optical properties of the hardware surface at different angles, reflecting its appearance changes under different lighting conditions. Using the bidirectional reflectance distribution function (BRDF) model, the reflected light intensity of the hardware surface at different incident angles and observation angles is accurately calculated. First, based on the three-dimensional light field data, the incident direction and reflection direction of the light at different surface points are extracted. Then, the reflected light intensity of each surface point is calculated through the BRDF model, taking into account the reflection characteristics and microstructure of the surface material. Finally, a set of reflected light intensity distribution data is generated, which accurately describes the intensity distribution characteristics of the light after reflection on the hardware surface. Using the spherical harmonic function theory in mathematics, the reflected light intensity distribution data is represented as a linear combination of a series of spherical harmonic functions. First, the reflected light intensity data is mapped to the spherical coordinate system to characterize the light intensity distribution in different directions. Then, through the spherical harmonic function expansion technology, these light intensity data are expanded into a set of coefficients of spherical harmonic basis functions. The spherical harmonic coefficient matrix retains the information of the reflected light intensity in all directions and can describe the characteristics of the light intensity distribution concisely and accurately. The spherical harmonic coefficient matrix is ​​used as the basic data for tensor reconstruction, and the main characteristic dimensions in the matrix are extracted through tensor decomposition technology (such as high-order singular value decomposition, HOSVD). Through tensor reconstruction, these characteristic dimensions are recombined into a four-dimensional data tensor containing space, direction, frequency and time dimensions. This tensor not only covers the optical properties of the hardware surface, but also reflects the changing characteristics of its light field under different conditions.

[0093] Preferably, the performing bidirectional reflection light intensity modeling on the three-dimensional light field data comprises the following steps:

[0094] Directionally deconstruct the three-dimensional light field data to obtain the incident direction light vector field;

[0095] Calculate the surface reflection distribution of the incident light vector field to generate local reflection light field distribution data;

[0096] Performing reflection angle weighting processing according to the local reflection light field distribution data to obtain a weighted reflection light intensity matrix;

[0097] The weighted reflected light intensity matrix is ​​subjected to light intensity smoothing filtering to generate reflected light intensity distribution data.

[0098] The benefit of the present invention lies in that the three-dimensional light field data is directional deconstructed to obtain the incident light vector field information. It can more accurately depict the direction of the incident light in different areas of the surface of the hardware items, laying a solid foundation for the subsequent calculation of the reflected light intensity. Next, based on the incident light vector field information, the method calculates the reflection distribution of the surface of the hardware items and generates local reflection light field distribution data. This local reflection light field distribution data fully captures the reflection characteristics of different areas on the surface of the hardware items, and provides a more refined input for the subsequent light intensity weighting processing. The local reflection light field distribution data is subjected to reflection angle weighting processing to obtain a reflection light intensity matrix. This weighted processing can better simulate the relative importance of reflected light in different areas of the surface of the hardware items and enhance the discriminability of the light intensity data. Finally, the weighted reflection light intensity matrix is ​​subjected to light intensity smoothing filtering processing to generate smoother and continuous reflection light intensity distribution data. This smooth reflection light intensity distribution data can better describe the overall reflection characteristics of the surface of the hardware items and reduce the influence of noise on the subsequent feature extraction. The complex optical characteristics of the surface of the hardware items are fully explored, and more abundant and accurate reflection light intensity distribution data are generated. These data not only provide high-quality input for subsequent spherical harmonic function feature extraction, but also provide more effective visual feature representation for deep learning model construction, thereby further improving the performance of hardware image retrieval. This hardware image retrieval method based on light field analysis and spherical harmonic feature extraction has significant advantages in improving retrieval accuracy and robustness, and has wide value in various practical applications involving hardware item recognition.

[0099] In this embodiment, the three-dimensional light field data is separated according to the spatial position and direction information, and each light in the light field is vectorized. It is realized by the light vectorization algorithm in geometric optics. The specific steps include extracting the light direction at each surface point from the three-dimensional light field data and expressing it in the form of a three-dimensional vector. Subsequently, these vectors are classified and sorted according to the incident direction of the light to form an incident direction light vector field. The vector field contains the incident direction information of the light at each surface point, providing a directional basis for the subsequent reflection distribution calculation. Using a physically based ray tracing (PBR) model, the directional distribution of the reflected light is calculated for the incident light vector at each surface point. In the specific implementation, the incident light vector is first vectorized with the surface normal to determine the initial reflection direction. Then, the light intensity distribution in different reflection directions is calculated using a microfacet distribution model (such as the Cook-Torrance model), taking into account the surface roughness and material characteristics. The local reflection light field distribution data finally generated reflects the light reflection of the hardware surface under different incident directions, laying a data foundation for further weighted processing of reflected light intensity. For each reflection direction in the local reflection light field distribution data, a weighted calculation is performed according to its degree of deviation from the observation angle. This step uses a Gaussian function for weight distribution. The smaller the deviation angle, the greater the weight, and vice versa. The purpose of weighted processing is to highlight the reflected light intensity close to the observation direction, so as to more accurately reflect the light distribution under actual observation conditions. The weighted light intensity data is integrated into a weighted reflection light intensity matrix, which covers the reflected light intensity information of the hardware surface at different observation angles. A bilateral filtering algorithm is used, which eliminates high-frequency noise while maintaining the intensity gradient change, ensuring the smoothness and continuity of the intensity distribution. In the specific operation, the filter window size and the smoothing parameters of the space and intensity domains are first set, and then the weighted reflection light intensity matrix is ​​bilaterally filtered point by point. The filtered reflected light intensity distribution data is smoother, reducing the local light intensity fluctuation caused by the surface microstructure, and providing a clearer and continuous light intensity distribution result. This data provides an accurate and stable basis for the subsequent spherical harmonic function expansion and feature extraction.

[0100] Preferably, the spherical harmonic function expansion processing of the reflected light intensity distribution model comprises the following steps:

[0101] Perform angle-stratified analysis on the reflected light intensity distribution data to obtain a stratified light intensity feature matrix;

[0102] Perform spherical coordinate transformation on the layered light intensity characteristic matrix to obtain the spherical coordinates of the light intensity distribution;

[0103] Perform spherical harmonic function expansion on the spherical coordinates of the light intensity distribution to generate a set of spherical harmonic coefficients;

[0104] The spherical harmonic coefficient set is subjected to coefficient regularization to obtain a spherical harmonic coefficient matrix.

[0105] The benefit of the present invention is that by performing angle layered analysis on the reflected light intensity distribution data, a layered light intensity feature matrix can be obtained. This step can capture the reflected light intensity characteristics of different angle areas on the surface of hardware items, and more carefully describe the optical properties of the surface of the items. Next, the layered light intensity feature matrix is ​​transformed into a spherical coordinate system to obtain a spherical coordinate representation of the light intensity distribution. The light intensity distribution data in this spherical coordinate system can better reflect the three-dimensional geometric shape and surface reflection characteristics of hardware items. Spherical harmonic function expansion processing is used to generate a spherical harmonic coefficient set. As an orthogonal function system defined on a sphere, spherical harmonics can efficiently encode the surface features of three-dimensional objects. Through the spherical harmonic function expansion, multi-scale geometric texture features of the surface of hardware items can be extracted, providing a more discriminative feature representation for subsequent image retrieval tasks. The spherical harmonic coefficient set is subjected to coefficient regularization processing to obtain a spherical harmonic coefficient matrix. This regularization step can effectively suppress noise, improve the robustness of spherical harmonic features, and ensure the stability and reliability of subsequent features in image retrieval. By performing a fine spherical harmonic function analysis on the reflected light intensity distribution, multi-scale spherical harmonic features are extracted that can fully describe the geometric texture characteristics of the surface of hardware items. This hardware image retrieval method based on light field analysis and spherical harmonic feature extraction makes full use of the complex optical properties of the surface of hardware items and generates a richer and more accurate visual feature representation, thereby significantly improving the performance and robustness of image retrieval.

[0106] In this embodiment, the reflected light intensity distribution data is layered according to the incident angle and reflection angle of the light, and the light intensity data at different angles are extracted respectively to form a multi-level angle light intensity data set. In the specific operation, the angle information of the light is used to divide the data into several levels according to the predetermined angle interval, and each level represents the light intensity distribution at a specific angle. Then, the layered light intensity data are matrixed to generate a layered light intensity feature matrix. Each layer of the matrix corresponds to the light intensity characteristics at different angles, providing the light intensity distribution at different angles, which lays the foundation for the subsequent spherical coordinate transformation. The layered light intensity feature matrix is ​​used to perform spherical coordinate transformation to generate the spherical coordinates of the light intensity distribution. In the specific implementation, the spherical coordinate transformation formula is first used to map the light intensity value of each layer in the matrix and its corresponding angle information to the spherical coordinate system. By converting the incident angle and reflection angle of each light intensity value into two angles of spherical coordinates (such as azimuth and polar angle), the light intensity data is transferred from the plane coordinate system to the spherical coordinate system. Then, these converted light intensity values ​​and their corresponding spherical coordinates are organized into a complete light intensity distribution spherical coordinate data set. This data set accurately represents the position and intensity of the light intensity distribution in the spherical coordinate system, providing a spatial basis for the subsequent spherical harmonic expansion processing. Through the spherical harmonic expansion formula, the light intensity distribution data is decomposed into a linear combination of spherical harmonics. The specific steps include selecting a suitable spherical harmonic basis and calculating the projection coefficients of each basis in the spherical coordinate system. These coefficients reflect the characteristics of the light intensity distribution in the spherical harmonic space and can effectively describe the changes in the light intensity data in different directions. During the expansion process, the orthogonality of the spherical harmonic basis is ensured to ensure the independence and accuracy of the coefficients. Finally, the generated spherical harmonic coefficient set contains the light intensity distribution information in all directions, laying a solid data foundation for the subsequent coefficient regularization processing. For each coefficient in the spherical harmonic coefficient set, regularization processing is performed to ensure that all coefficients have consistent scales under the same dimension. This operation is performed through the L2 norm regularization method, which limits the sum of the squares of each coefficient to a fixed range to avoid feature deviations caused by excessive individual coefficients. Then, the regularized coefficients are arranged in the order of their corresponding spherical harmonic basis to form a two-dimensional matrix, namely the spherical harmonic coefficient matrix. This matrix not only retains the directional characteristics of light intensity distribution, but also enhances the robustness of the features through regularization processing, and can provide more stable input data for subsequent deep learning models.

[0107] Preferably, step S5 comprises:

[0108] Step S51: performing principal component analysis layer by layer on the four-dimensional data tensor to obtain dimension-reduced feature data;

[0109] Step S52: constructing a multi-layer perceptron neural network on the dimension-reduced feature data to generate feature-associated network parameters;

[0110] Step S53: performing association mapping driving processing on preset hardware material properties and characteristic association network parameters to obtain a multi-dimensional mapping model;

[0111] Step S54: Perform forward propagation simulation on the dimension-reduced feature data based on the multi-dimensional mapping model to generate a spectrum-material property relationship.

[0112] The invention is beneficial in that by performing spherical harmonic function analysis on the reflected light intensity distribution, multi-scale spherical harmonic features that can fully describe the geometric texture features of the surface of hardware items are extracted. This feature extraction method based on light field analysis makes full use of the complex optical properties of the surface of hardware items to generate a richer and more accurate visual feature representation. Compared with the traditional method based on color and texture low-level features, this feature is more discriminative and robust. A feature association network is constructed through a multi-layer perceptron neural network, and a multidimensional mapping model is established in combination with preset hardware material properties. This deep learning model based on feature association network and material properties can effectively learn the intrinsic relationship between the spectrum and material properties of hardware items. The spectrum-material property relationship generated by forward propagation simulation can not only be used for image retrieval, but also support material classification and component analysis scenarios. It can significantly improve the accuracy and robustness of hardware image retrieval. In particular, for some hardware items with subtle differences, this method can better capture their surface optical properties, thereby achieving more accurate retrieval and recognition. At the same time, by performing dimensionality reduction and regularization on feature data, the method also improves the robustness of the retrieval system and reduces the impact of noise and changes. It fully exploits the advantages of the association between multi-dimensional visual features and material properties, and has significant technical innovation and application value in improving retrieval accuracy and robustness.

[0113] In this embodiment, the four-dimensional data tensor is decomposed layer by layer according to the dimension, and the high-dimensional tensor is decomposed into multiple low-dimensional subspaces. Then the data in each subspace is processed by PCA, and the principal component direction and its corresponding principal component value are extracted by eigenvalue decomposition of the covariance matrix. This process finds the principal component in the direction of the maximum data change by solving the eigenvalue and eigenvector, retains the main data information and removes redundant dimensions. After PCA processing is performed on each subspace layer by layer, the principal component data of all subspaces are integrated to generate a feature data set after dimensionality reduction. The reduced dimension feature data finally obtained reduces the dimensional complexity, retains the most representative features in the data, and lays the foundation for the subsequent neural network construction. According to the dimension and quantity of the reduced dimension feature data, the network structure of the multilayer perceptron is designed, including the number of nodes and the number of layers of the input layer, hidden layer and output layer. Then, the back propagation algorithm is used to train the neural network layer by layer through the randomly initialized network weights and biases. During the training process, the gradient descent algorithm is used to minimize the loss function to optimize the network parameters. After multiple iterative training, the final network parameters converge to the global optimal point to obtain the feature association network parameters. These parameters include the weight matrix and bias vector of the neural network, which reflect the complex association between the reduced-dimensional feature data and provide the necessary model basis for the associated mapping driven processing. The hardware material properties are used as input variables and mapped layer by layer with the feature association network parameters. In this process, the neural network parameters obtained by the above training are used to generate the output of each layer of the network through forward propagation calculation, and propagate to the final output layer layer by layer. The specific operation includes processing the input data using linear combinations and nonlinear activation functions, and capturing complex attribute relationships in each intermediate hidden layer. Finally, after the mapping processing of all layers, a multidimensional mapping model is generated, which can map various attributes of hardware materials to the reduced-dimensional feature space and reflect the complex relationship between material attributes and feature data. Based on the multidimensional mapping model generated above, the reduced-dimensional feature data is simulated by forward propagation to generate the spectrum-material attribute relationship. In the specific implementation, the reduced-dimensional feature data is input into the multidimensional mapping model, and the forward propagation calculation is performed layer by layer. In the calculation process of each layer, the weight matrix and activation function in the network are applied in turn, the input data is nonlinearly transformed, and the result is passed to the next layer. Through multi-layer calculations, a spectrum-material property mapping result is finally generated in the output layer. This result describes the spectral response relationship between the reduced-dimensional feature data and the hardware material properties, which can be used to predict the performance of hardware materials under different spectral conditions and provide key spectral feature information for subsequent hardware image retrieval.

[0114] Preferably, step S6 comprises:

[0115] Step S61: constructing a kd tree index for the spectrum-material property relationship to obtain retrieval index data;

[0116] Step S62: dividing the storage space of the shape-based hardware data set based on the search index data to obtain a multi-dimensional index tree;

[0117] Step S63: performing nearest neighbor search on the shape-like hardware data set according to the multidimensional index tree to generate a candidate hardware data set;

[0118] Step S64: Perform feature matching and precise retrieval on the candidate hardware data set and the hardware feature data to generate retrieved hardware data.

[0119] The invention is beneficial in that a kd tree index is constructed by using the spectrum-material property relationship, and a multidimensional index tree is established based on this. This multidimensional index based on the association of material properties can effectively perform spatial division and organizational management on the hardware data set. Compared with the traditional single index method based on visual features, this multidimensional index can better utilize material property information and improve the accuracy and robustness of retrieval. Based on the multidimensional index tree, the nearest neighbor search is performed on the shape-like hardware data set to generate a candidate hardware data set. Then, the candidate data set and its feature data are accurately matched and retrieved. This hierarchical retrieval strategy greatly improves the efficiency of retrieval. First, the search scope is quickly narrowed down with the help of multidimensional index, and then fine feature matching is performed, so that the most relevant results can be quickly found in a large-scale hardware data set. The spectrum-material property relationship and geometric feature information are integrated, and the multidimensional properties and geometric information of hardware items can be fully utilized in the retrieval process. This joint retrieval method can not only improve the accuracy of retrieval, but also weigh and balance different attributes according to user needs. For example, more emphasis can be placed on material composition, or more attention can be paid to geometric shape. By constructing a multidimensional index of spectrum-material property association and adopting a hierarchical and efficient retrieval strategy, it has significant advantages in retrieval efficiency and retrieval accuracy. At the same time, it also supports the joint retrieval of multidimensional attributes and geometric information, providing users with a more flexible and intelligent hardware image retrieval service.

[0120] In this embodiment, the characteristic data in the spectrum-material property relationship is used to select a suitable kd tree algorithm to perform spatial division. The process recursively divides the data set into a hyperrectangle of a k-dimensional space, and selects the median of each dimension as a split point, thereby establishing a balanced kd tree structure. Each node represents a data point, and each split hyperplane represents the value of the data point in a certain dimension. Then, the segmentation result is stored as retrieval index data in the form of an index. This index data structure can quickly locate specific data points in the spectral feature space, greatly improving the retrieval efficiency, and is particularly suitable for rapid query and comparison of large-scale hardware material property data. Based on the retrieval index data generated before, the storage space of the shape-like hardware data set is divided to obtain a multidimensional index tree. During specific implementation, the characteristic data of the shape-like hardware data set is first input into the constructed kd tree index structure. By traversing the kd tree, each hardware data point is mapped to its corresponding index position, and the entire data set is multidimensionally spaced according to the spatial division rule of the kd tree. In this process, the node structure of the index tree is used to store the data points in order, ensuring that the hardware data points with similar shapes or spectral characteristics are arranged adjacently in the multidimensional space. The multidimensional index tree structure finally generated can quickly locate and retrieve hardware data points with similar characteristics to the target data, providing structured support for subsequent nearest neighbor retrieval. The multidimensional index tree is used to perform nearest neighbor retrieval on the shape-like hardware data set to generate a candidate hardware data set. In the specific operation, the nearest neighbor search algorithm is performed on the multidimensional index tree structure to quickly locate the approximate neighbors of the target data. The algorithm first starts from the root node of the multidimensional index tree, searches down layer by layer according to the tree structure, compares the nodes most similar to the target data and records the shortest distance. Then, by means of reverse path search, other branch nodes that have not been searched are checked to ensure that the global nearest neighbors are found. After this multi-step search process, a candidate hardware data set is generated, which includes a set of hardware data that is most similar to the target data, covering multiple candidate data points that are closest to it, laying the foundation for further precise retrieval. Feature matching is performed on the candidate hardware data set and hardware feature data to accurately retrieve, and finally generate the retrieved hardware data. In the specific operation, the features of each hardware data in the candidate data set are first compared in detail with the target hardware data, and the similarity between the two is calculated using a feature matching algorithm (such as Euclidean distance or cosine similarity). This process uses the feature vector of the candidate data set to match the target feature vector one by one, and calculates the difference value of each pair of features one by one. Then, all matching results are sorted by similarity, and the top few items that match the most are selected as the final retrieval results. The retrieved hardware data set finally generated contains the hardware data that is closest to the target data, and ensures the accuracy and relevance of the results through feature matching, providing high-precision result support for deep learning retrieval applications of hardware images.

[0121] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0122] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A hardware image retrieval method based on deep learning, characterized in that: The following steps are involved: Step S1: Acquire multi-angle images of hardware to be retrieved; perform geometric correction on the multi-angle images of hardware to be retrieved, and perform feature enhancement processing to obtain hardware feature data; Step S2: performing shape fuzzy retrieval on a preset hardware image database based on hardware feature data to generate a shape-like hardware data set; Step S3: constructing a surface grating structure for the shape-like hardware data set, and performing optical response simulation to obtain hardware grating data; Step S4: simulating the light field of the multi-angle image of the hardware to be retrieved based on the hardware grating data, and collecting four-dimensional reflected light to generate a four-dimensional data tensor; Step S5: mapping the four-dimensional data tensor according to the preset hardware material properties to obtain the spectrum-material property relationship; Step S6: Based on the spectrum-material property relationship, perform feature matching and precise retrieval on the shape-like hardware data set to generate retrieved hardware data.

2. The hardware image retrieval method based on deep learning according to claim 1 is characterized in that: Step S1 includes the following steps: Step S11: Acquire multi-angle images of hardware to be retrieved; Step S12: performing edge detection on the multi-angle image of the hardware to be retrieved to obtain the edge contour of the hardware; Step S13: geometrically correct the edge contour of the hardware to generate corrected hardware data; Step S14: extracting features from the corrected hardware data to obtain hardware feature data; Step S15: Perform feature enhancement processing on the corrected hardware data based on the hardware feature data to obtain the hardware feature data.

3. The hardware image retrieval method based on deep learning according to claim 1 is characterized in that: Step S2 includes the following steps: Step S21: performing multi-scale shape decomposition processing on the hardware feature data to obtain a multi-scale shape decomposition set; Step S22: performing shape Gaussian blur processing on the multi-scale shape decomposition set to generate a simulated shape feature set; Step S23: performing deep feature clustering on the simulated shape feature set to obtain a clustered shape feature set; Step S24: Based on the clustered shape feature set, a neural network fuzzy matching is performed on the preset hardware image database to generate a class shape hardware data set.

4. The hardware image retrieval method based on deep learning according to claim 1 is characterized in that: Step S3 includes the following steps: Step S31: Surface morphology analysis is performed on the shape-like hardware data set to obtain microscopic features of the hardware surface; Step S32: constructing a three-dimensional surface grating model according to the microscopic features of the hardware surface; Step S33: discretizing the three-dimensional surface grating model to obtain grating unit data; Step S34: Perform optical response simulation on the grating unit data to obtain hardware grating data.

5. The hardware image retrieval method based on deep learning according to claim 4 is characterized in that: Step S34 includes the following steps: Step S341: performing electromagnetic field distribution calculation on the grating unit data to obtain surface electromagnetic field distribution data; Step S342: performing dynamic optical response simulation on the surface electromagnetic field distribution data to generate grating time variation response data; Step S343: performing inverse scattering processing on the grating time-varying response data to obtain grating scattering characteristics; Step S344: Perform spectrum analysis on the grating scattering characteristics to generate hardware grating data.

6. The hardware image retrieval method based on deep learning according to claim 1 is characterized in that: Step S4 includes the following steps: Step S41: constructing a light field complex amplitude mapping for the hardware grating data to obtain a light field propagation model; Step S42: performing light field simulation on the multi-angle image of the hardware to be retrieved according to the light field propagation model to obtain three-dimensional light field data; Step S43: performing bidirectional reflection light intensity calculation on the three-dimensional light field data to obtain reflection light intensity distribution data; Step S44: performing spherical harmonic function expansion processing on the reflected light intensity distribution data to obtain a spherical harmonic coefficient matrix; Step S45: reconstruct the tensor features of the spherical harmonic coefficient matrix to generate a four-dimensional data tensor.

7. The hardware image retrieval method based on deep learning according to claim 6 is characterized in that: The bidirectional reflection light intensity modeling of the three-dimensional light field data comprises the following steps: Directionally deconstruct the three-dimensional light field data to obtain the incident direction light vector field; Calculate the surface reflection distribution of the incident light vector field to generate local reflection light field distribution data; Performing reflection angle weighting processing according to the local reflection light field distribution data to obtain a weighted reflection light intensity matrix; The weighted reflected light intensity matrix is ​​subjected to light intensity smoothing filtering to generate reflected light intensity distribution data.

8. The hardware image retrieval method based on deep learning according to claim 6 is characterized in that: The spherical harmonic expansion process of the reflected light intensity distribution model comprises the following steps: Perform angle-stratified analysis on the reflected light intensity distribution data to obtain a stratified light intensity feature matrix; Perform spherical coordinate transformation on the layered light intensity characteristic matrix to obtain the spherical coordinates of the light intensity distribution; Perform spherical harmonic function expansion on the spherical coordinates of the light intensity distribution to generate a set of spherical harmonic coefficients; The spherical harmonic coefficient set is subjected to coefficient regularization to obtain a spherical harmonic coefficient matrix.

9. The hardware image retrieval method based on deep learning according to claim 1 is characterized in that: Step S5 includes the following steps: Step S51: performing principal component analysis layer by layer on the four-dimensional data tensor to obtain dimension-reduced feature data; Step S52: constructing a multi-layer perceptron neural network on the dimension-reduced feature data to generate feature-associated network parameters; Step S53: performing association mapping driving processing on preset hardware material properties and characteristic association network parameters to obtain a multi-dimensional mapping model; Step S54: Perform forward propagation simulation on the dimension-reduced feature data based on the multi-dimensional mapping model to generate a spectrum-material property relationship.

10. The hardware image retrieval method based on deep learning according to claim 1 is characterized in that: Step S6 includes the following steps: Step S61: constructing a kd tree index for the spectrum-material property relationship to obtain retrieval index data; Step S62: dividing the storage space of the shape-based hardware data set based on the search index data to obtain a multi-dimensional index tree; Step S63: performing nearest neighbor search on the shape-like hardware data set according to the multidimensional index tree to generate a candidate hardware data set; Step S64: Perform feature matching and precise retrieval on the candidate hardware data set and the hardware feature data to generate retrieved hardware data.

Images