Foreign matter identification method and device, equipment and storage medium

Abstract

The invention provides a foreign matter recognition method, device and equipment and a storage medium, and relates to the field of image processing, and the method comprises the steps: obtaining the spectral data of a target object, extracting the spectral features and morphological features of the target object based on the spectral data, and carrying out the feature extraction of the spectral data because the spectral data comprises the rich spectral information of the target object, calculating to obtain a similarity feature between the spectral feature and a reference spectrum in a preset spectrum library; and obtaining category information of the target object based on the similarity feature, the spectral feature after dimension reduction processing and the morphological feature, and determining whether the target object belongs to the foreign matter or whether the target object comprises the foreign matter based on the category information. The known reference spectrum in the spectrum library is fully utilized in the process, interference can be eliminated, the difference between the target object and the reference spectrum can be focused more accurately, the essential characteristics of the target object can be better captured, and the efficiency and accuracy of foreign matter recognition are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a foreign body recognition method, device, equipment and storage medium. Background Art

[0002] During the production and transportation of goods, foreign objects other than goods are often mixed in, causing the overall quality of the goods to decline, and even bringing epidemic risks to imported goods, making it impossible to dock and unload normally. Therefore, it is very important to identify foreign objects in the goods.

[0003] There are two main existing methods of identifying foreign objects. The first is manual identification, and the second is identifying foreign objects through visible light image recognition algorithms.

[0004] However, the manual identification method has disadvantages such as large workload, strong subjectivity, low efficiency, high professional experience requirements, and harm to human health. Although the visible light image recognition algorithm can distinguish foreign objects with obviously different colors, if the foreign objects are similar in color to normal goods, the accuracy of recognition will be reduced and it is easy to miss reports. Summary of the invention

[0005] The embodiments of the present invention provide a foreign object recognition method, device, equipment and storage medium to solve the problem of low efficiency and low accuracy in foreign object recognition.

[0006] In a first aspect, an embodiment of the present invention provides a foreign body identification method, comprising: Acquiring spectral data of a target object, and extracting spectral features and morphological features of the target object based on the spectral data; Calculate and obtain the similarity feature between the spectral feature and the reference spectrum in a preset spectral library; wherein the preset spectral library includes reference spectra corresponding to samples of multiple categories; Performing feature dimensionality reduction processing on the spectral features, and obtaining category information of the target object according to the spectral features after the dimensionality reduction processing, the morphological features and the similarity features; Based on the category information, it is determined whether the target object is a foreign object, or based on the category information, it is determined whether the target object includes a foreign object.

[0007] In a possible implementation, obtaining the category information of the target object according to the spectral features, the morphological features and the similarity features after the dimensionality reduction process includes: Based on a preset weight matrix, the spectral features after the dimensionality reduction processing, the morphological features and the similarity features are weighted and fused to obtain a comprehensive feature; wherein the preset weight matrix is ​​used to indicate the weight proportion of the spectral features after the dimensionality reduction processing, the morphological features and the similarity features in the comprehensive feature; The comprehensive feature is input into a trained classification model to obtain category information of the target object; wherein the classification model is trained based on multiple comprehensive features and a category label corresponding to each comprehensive feature.

[0008] In a possible implementation, the calculating and obtaining a similarity feature between the spectral feature and a reference spectrum in a preset spectral library includes: The spectral features and the reference spectra corresponding to each category of samples in the preset spectral library are standardized to obtain the standardized spectral features and each reference spectrum; wherein the standardized spectral features and each reference spectrum are both in the form of probability distribution; Calculate the similarity information between the standardized spectral features and each standardized reference spectrum; The similarity feature is obtained according to each piece of similarity information.

[0009] In a possible implementation, the calculating similarity information between the standardized spectral feature and each standardized reference spectrum includes: according to , calculate the similarity information between the standardized spectral features and each standardized reference spectrum; in, is the similarity information between the standardized spectral feature and each standardized reference spectrum, is the spectral feature after standardization. is the i-th component of the spectral feature after normalization, is the reference spectrum after standardization processing, is the j-th component of the reference spectrum after standardization, i is a non-zero integer, i=1...n, n represents the total number of components included in the spectral characteristics after standardization; j is a non-zero integer, j=1...m, m represents the total number of components included in the reference spectrum after standardization.

[0010] In a possible implementation, extracting the spectral features and morphological features of the target object based on the spectral data includes: Performing data conversion processing on the spectral data to obtain a pseudo-color image corresponding to the target object; Preprocessing the pseudo-color image, and obtaining contour information of the target object based on the pseudo-color image after preprocessing; wherein the preprocessing includes one or more of grayscale processing, Gaussian filtering denoising processing, binarization processing and morphological processing; The spectral features and the morphological features are extracted based on the contour information; wherein the morphological features include area features, perimeter features, and shape factor information.

[0011] In a possible implementation, the acquiring spectral data of the target object includes: The spectral data is acquired by using a hyperspectral camera in an external push-scan manner; wherein the hyperspectral camera is installed on an unmanned aerial vehicle (UAV) and the target object is a fixed target object; Alternatively, the spectral data of the target object on the conveyor belt is acquired by using a hyperspectral camera in an external push-scan manner; wherein the hyperspectral camera is mounted on a linear stand; Alternatively, the spectral data of the target object is acquired by using a hyperspectral camera in a built-in push-scan manner; wherein the hyperspectral camera is installed on a fixed stand, and the target object is a fixed target object.

[0012] In a possible implementation manner, determining whether the target object is a foreign object based on the category information includes: Acquire visible light data of the target object and infrared data of the target object; According to the category information, the visible light data and the infrared data, it is determined whether the target object is a foreign object, or whether the target object includes a foreign object.

[0013] In a second aspect, an embodiment of the present invention provides a foreign body identification device, the device comprising: A first processing unit is used to obtain spectral data of a target object, and extract spectral features and morphological features of the target object based on the spectral data; A calculation unit, used for calculating the similarity characteristics between the spectral characteristics and the reference spectra in a preset spectral library; wherein the preset spectral library includes reference spectra corresponding to samples of multiple categories; A second processing unit is used to perform feature dimensionality reduction processing on the spectral features, and obtain category information of the target object according to the spectral features after the dimensionality reduction processing, the morphological features and the similarity features; A distinguishing unit is used to determine whether the target object is a foreign object based on the category information, or to determine whether the target object includes a foreign object based on the category information.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the method described in the first aspect or any possible implementation method of the first aspect.

[0016] The embodiment of the present invention provides a foreign body identification method, device, equipment and storage medium. First, the spectral data of the target object is obtained. This process does not require direct contact with the target object, which can avoid damage and contamination to the target object; then, the spectral characteristics and morphological characteristics of the target object are extracted based on the spectral data. Since the spectral data includes rich spectral information of the target object, rich and accurate spectral characteristics and morphological characteristics can be obtained by extracting features from the spectral data, thereby improving the accuracy of foreign body identification; then, the similarity characteristics between the spectral characteristics and the reference spectrum in the preset spectral library are calculated; then, based on the similarity characteristics, the spectral characteristics after dimensionality reduction processing, and the morphological characteristics, the category information of the target object is obtained, and then, based on the category information, it is determined whether the target object belongs to a foreign object or whether the target object includes a foreign object. This process makes full use of the known reference spectrum in the spectral library, which is conducive to eliminating interference, focusing more accurately on the difference between the target object and the reference spectrum, better capturing the essential characteristics of the target object, and improving the efficiency and accuracy of foreign body identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flow chart of a foreign body identification method provided by an embodiment of the present invention; Figure 2 is a flowchart of another foreign body identification method provided by an embodiment of the present invention; Figure 3 is a schematic structural diagram of a foreign body identification device provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention. SUMMARY OF THE INVENTION During the production and transportation of goods, foreign objects other than goods are often mixed in. For example, foreign objects mixed in grain, ore and other goods may cause the overall quality of the goods to decline. For imported goods, it may even cause the goods to have epidemic risks and fail to dock and unload normally. Therefore, it is very important to identify foreign objects in the goods.

[0021] There are two main ways to identify foreign objects. The first is manual identification. However, manual identification has disadvantages such as large workload, strong subjectivity, low efficiency, high professional experience requirements, and harm to human health. The second is to identify foreign objects through visible light image recognition algorithms. Although this method can distinguish foreign objects with obviously different colors, if there are foreign objects with similar colors to normal goods, the accuracy of identification will be reduced, and it is easy to miss reports.

[0022] In order to improve the efficiency and accuracy of foreign body identification, in the embodiment of the present invention, the spectral data of the target object is first obtained. This process does not require direct contact with the target object, which can avoid damage and contamination of the target object; then the spectral characteristics and morphological characteristics of the target object are extracted based on the spectral data. Since the spectral data includes rich spectral information of the target object, rich and accurate spectral characteristics and morphological characteristics can be obtained by extracting features from the spectral data, thereby improving the accuracy of foreign body identification; then the similarity characteristics between the spectral characteristics and the reference spectrum in the preset spectral library are calculated; then based on the similarity characteristics, the spectral characteristics after dimensionality reduction processing, and the morphological characteristics, the category information of the target object is obtained, and then based on the category information, it is determined whether the target object belongs to a foreign body or whether it includes a foreign body. This process makes full use of the known reference spectrum in the spectral library, which is conducive to eliminating interference, focusing more accurately on the difference between the target object and the reference spectrum, better capturing the essential characteristics of the target object, and improving the efficiency and accuracy of foreign body identification.

[0023] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0024] Figure 1 A flowchart of a foreign body identification method provided by an embodiment of the present invention is described in detail as follows: Step 101: Acquire spectral data of a target object, and extract spectral features and morphological features of the target object based on the spectral data.

[0025] Exemplarily, the present embodiment can acquire spectral data of the target object based on a spectral imaging device, including, for example, reflection spectral data, absorption spectral data, emission spectral data, and transmission spectral data; after obtaining the spectral data, the present embodiment performs feature extraction based on the spectral data to obtain spectral characteristics of the target object, including, for example, spectral peak characteristics, spectral valley characteristics, spectral slope characteristics, spectral band ratio characteristics, etc.; morphological characteristics, including, for example, shape characteristics, contour shape, size characteristics, and texture characteristics, etc.

[0026] This embodiment extracts and quantifies the shape, peak value, valley value and other features of the spectral data, and comprehensively analyzes the spectral information of different bands. The spectral information of these spectral features includes parameters such as reflectivity and radiation intensity of each spectral band, which can reflect the unique properties of the target object in the spectrum.

[0027] In one example, the target object may be a sample obtained by sampling goods, or may be goods in the process of production or transportation.

[0028] Step 102, calculating and obtaining a similarity feature between the spectral feature and a reference spectrum in a preset spectral library; wherein the preset spectral library includes reference spectra corresponding to samples of multiple categories.

[0029] In this embodiment, a preset spectrum library is pre-built, and each category of samples in the preset spectrum library has its corresponding reference spectrum. The reference spectrum can be obtained by measuring the average spectrum of multiple samples of the same type.

[0030] Illustratively, after obtaining the spectral features of the target object, this embodiment calculates the similarity features between the spectral features and all reference spectra in a preset spectral library; or calculates the similarity features between the spectral features and some reference spectra in a preset spectral library.

[0031] In a feasible implementation manner, this embodiment calculates the similarity between the spectral feature and each reference spectrum in the preset spectral library, and then fuses each similarity to obtain the similarity feature between the spectral feature and the reference spectrum in the preset spectral library.

[0032] In another feasible implementation, this embodiment can determine one or more category samples in a preset spectral library that have an associated relationship with the category information based on the reference category information of the target object, calculate the similarity between the spectral feature and the reference spectrum corresponding to the one or more category samples stored in the preset spectral library, and then fuse the various similarities to obtain the similarity feature between the spectral feature and the reference spectrum in the preset spectral library.

[0033] Step 103, performing feature dimension reduction processing on the spectral features, and obtaining category information of the target object based on the spectral features, morphological features and similarity features after the dimension reduction processing.

[0034] For example, this embodiment can use principal component analysis (PCA) and linear discriminant analysis to perform feature dimension reduction processing on spectral features, reduce the dimension of spectral features, remove redundancy, and obtain spectral features after dimension reduction processing. Then, the spectral features, morphological features, and similarity features after dimension reduction processing are subjected to feature fusion processing to obtain comprehensive features, and then feature recognition is performed based on the comprehensive features to obtain the category information of the target object.

[0035] In a feasible implementation, this embodiment performs feature addition processing on the spectral features, morphological features and similarity features after dimensionality reduction processing to obtain comprehensive features; or, performs weighted fusion processing on the spectral features, morphological features and similarity features after dimensionality reduction processing to obtain comprehensive features.

[0036] After obtaining the comprehensive features, this embodiment can input the comprehensive features into a pre-trained model to obtain the category information of the target object output by the model.

[0037] Step 104: determining whether the target object is a foreign object based on the category information, or determining whether the target object includes a foreign object based on the category information.

[0038] In one example, this embodiment determines whether the target object is a foreign object, or whether the target object includes a foreign object, based on the obtained category information, as well as the cargo information and cargo category to which the target object belongs.

[0039] For example, if the category information of the target object is inconsistent with the cargo category information of the cargo to which the target object belongs, the target object is determined to be a foreign object; if the category information of the target object is consistent with the cargo category information of the cargo to which the target object belongs, the target object is determined not to be a foreign object.

[0040] Alternatively, if the category information of the target object includes category information of items other than goods, it is determined that the target object includes foreign matter; if the category information of the target object does not include category information of items other than goods, it is determined that the target object does not include foreign matter.

[0041] In summary, the embodiments of the present invention provide a foreign body identification method, device, equipment and storage medium. First, the spectral data of the target object is obtained. This process does not require direct contact with the target object, which can avoid damage and contamination to the target object; then, the spectral characteristics and morphological characteristics of the target object are extracted based on the spectral data. Since the spectral data includes rich spectral information of the target object, rich and accurate spectral characteristics and morphological characteristics can be obtained by extracting features from the spectral data, thereby improving the accuracy of foreign body identification; then, the similarity characteristics between the spectral characteristics and the reference spectrum in the preset spectral library are calculated; then, based on the similarity characteristics, the spectral characteristics after dimensionality reduction processing, and the morphological characteristics, the category information of the target object is obtained, and then, based on the category information, it is determined whether the target object belongs to a foreign body or whether it includes a foreign body. This process makes full use of the known reference spectrum in the spectral library, which is conducive to eliminating interference, focusing more accurately on the difference between the target object and the reference spectrum, better capturing the essential characteristics of the target object, and improving the efficiency and accuracy of foreign body identification.

[0042] In order to achieve automatic and efficient acquisition of spectral data of the target object, this embodiment performs different installation settings on the hyperspectral camera for different scenarios, and then realizes the collection of spectral data of the target object based on the hyperspectral camera; in addition, this embodiment first performs data conversion and preprocessing on the spectral data to effectively improve the data quality, facilitate subsequent analysis and identification, and then obtain spectral features and morphological features based on the obtained contour information of the target object; in addition, this embodiment first performs standardization on the spectral features and the reference spectrum, and then performs similarity calculation, which can effectively improve the efficiency and accuracy of the similarity calculation; in addition, this embodiment also performs weighted fusion processing on the spectral features, morphological features and similarity features after dimensionality reduction processing based on a preset weight matrix to obtain comprehensive features. The comprehensive features can integrate different features, make up for the problem of insufficient information or noise interference that may exist in a single data source, and more realistically reflect the characteristics of the target object. Then, the comprehensive features are input into the trained classification model to achieve efficient, accurate and automatic foreign body recognition.

[0043] Figure 2 Another implementation flow chart of a foreign body identification method provided by an embodiment of the present invention is described in detail as follows: Step 201 : Acquire spectral data of a target object, and extract spectral features and morphological features of the target object based on the spectral data.

[0044] In one example, step 201 includes the following three implementations: The first implementation method is to use a hyperspectral camera to collect spectral data in an external push-scan manner; wherein the hyperspectral camera is installed on an unmanned aerial vehicle (UAV) and the target object is a fixed target object.

[0045] The second implementation method is to use a hyperspectral camera to collect spectral data of the target object on the conveyor belt by an external push-scan method; wherein the hyperspectral camera is mounted on a linear stand.

[0046] The third implementation method is to use a hyperspectral camera to collect spectral data of the target object in a built-in push-scan manner; wherein the hyperspectral camera is installed on a fixed stand and the target object is a fixed target object.

[0047] In a feasible implementation manner, step 201 includes the following steps: The spectral data is converted and processed to obtain a pseudo-color image corresponding to the target object.

[0048] The pseudo-color image is preprocessed, and contour information of the target object is obtained based on the pseudo-color image after preprocessing; wherein the preprocessing includes one or more of grayscale processing, Gaussian filtering denoising processing, binarization processing and morphological processing.

[0049] According to the contour information, spectral features and morphological features are extracted; wherein the morphological features include area features, perimeter features and shape factor information.

[0050] Exemplarily, in this embodiment, a drone equipped with a hyperspectral camera can be used to fly back and forth at a constant speed at a certain altitude to collect spectral data. The hyperspectral camera collects data in an external push-scan manner, and obtains spectral data of a column of pixels per frame. The drone equipped with a hyperspectral camera moves in a straight line to collect hyperspectral image data within an area equal to the product of the viewing angle range width and the flight distance. For example, in the detection of foreign matter in grain, a drone equipped with a hyperspectral camera can fly to a height of 5 meters (m) from the surface of the grain, perform high-resolution scanning of a 2m wide viewing angle range at a speed of 5 meters per second (m / s), fly 10m in a straight line along the long side of the rectangle, and complete data collection on a 2*10m rectangular surface.

[0051] In this embodiment, a stand can also be placed on the conveyor belt, and a hyperspectral camera can be installed on the stand to collect data of the goods transported on the conveyor belt. The hyperspectral camera collects data in an external push-scan manner, and obtains spectral data of a column of pixels in each frame. The hyperspectral camera only scans a line with the same width as the viewing angle range. The conveyor belt moves at a constant speed below, and the product of the conveyor belt movement speed and the acquisition time is the actual length of the image acquisition range. Therefore, when collecting image data, hyperspectral image data within the image acquisition range surrounded by the length and the viewing angle range width can be obtained. For example, in the detection of foreign matter in grain, by installing a hyperspectral camera 1m above the surface of the grain on the conveyor belt, the viewing angle range width on the conveyor belt is 1.4m, the conveyor belt speed is 5m / s, and each acquisition is 1s, data collection within a rectangular range of 5*1*1.4m can be achieved.

[0052] In this embodiment, a hyperspectral camera can also be mounted on a fixed stand. The hyperspectral camera collects data in a built-in push-scan mode, and obtains spectral data of a column of pixels in each frame. The internal lens of the hyperspectral camera moves at a constant speed under the control of the motor inside the camera to collect spectral data. The data collection range is related to the motor movement speed, camera height and lens focal length. For example, in the detection of foreign matter in grains, when the hyperspectral camera is placed on a fixed stand at a height of 30 centimeters (cm) from the data collection range, the data collection range is a rectangle of 8cm*7cm.

[0053] In addition, this embodiment uses a hyperspectral camera to obtain spectral data of the target object, capturing the spectral information of each pixel at multiple wavelengths. Each spectral data forms a three-dimensional data cube ,in and represents the spatial coordinates, The spectral data is then converted to a spectral RGB image, and the appropriate wavelength is selected to generate a pseudo-color image.

[0054] After obtaining the pseudo-color image corresponding to the target object, this embodiment Convert the pseudo-color image to a grayscale image to simplify the image information and reduce the computational complexity; represents the converted grayscale image, Represents the value of the red channel. Represents the value of the green channel. Represents the value of the blue channel.

[0055] Then perform Gaussian filtering on the grayscale image, using the Gaussian kernel , remove the noise in the image and enhance the smoothness of the image; among them, Represents the Gaussian filter in coordinates The value at is the standard deviation of the Gaussian function.

[0056] Binarize the denoised grayscale image using a threshold Convert image to binary image ; Then use morphological operations (such as dilation, erosion, etc.) to process the binary image, repair the small holes caused by noise, and enhance the connectivity of the target area. Among them, Indicates that the image after binarization is at coordinate The pixel value at is the threshold, which is the critical value for converting a grayscale image to a binary image.

[0057] After preprocessing, this embodiment performs contour extraction on the preprocessed image based on a boundary tracking algorithm, identifies contour information of the target object, and then extracts spectral features and morphological features based on the contour information; wherein the morphological features include area features, perimeter features, and shape factor information.

[0058] Step 202, standardize the spectral features and the reference spectra corresponding to each category of samples in the preset spectral library to obtain the standardized spectral features and each reference spectrum; wherein the standardized spectral features and each reference spectrum are both in the form of probability distribution, and the preset spectral library includes reference spectra corresponding to multiple categories of samples.

[0059] Exemplarily, this embodiment pre-builds a reference spectrum library, i.e., a preset spectrum library, in which each category of samples has its corresponding reference spectrum. The reference spectrum can be obtained by measuring the average spectrum of multiple samples of the same category. For example, categories, each category Reference spectrum By The spectra of the samples are averaged to obtain:

[0060] in, Indicates category Middle The spectral vector of samples.

[0061] This embodiment performs standardization processing on the spectral features and the reference spectrum corresponding to each category of samples in the preset spectral library. and reference spectrum Normalize to a probability distribution:

[0062] in, and are the spectral vectors Quantity, is the spectral feature after standardization. is the reference spectrum after standardization.

[0063] Step 203: Calculate similarity information between the standardized spectral feature and each standardized reference spectrum.

[0064] In a feasible implementation manner, step 203 includes the following steps: according to , calculate the similarity information between the standardized spectral features and each standardized reference spectrum.

[0065] in, is the similarity information between the standardized spectral feature and each standardized reference spectrum, is the spectral feature after standardization. is the i-th component of the spectral feature after normalization, is the reference spectrum after standardization processing, is the j-th component of the reference spectrum after standardization, i is a non-zero integer, i=1...n, n represents the total number of components included in the spectral characteristics after standardization; j is a non-zero integer, j=1...m, m represents the total number of components included in the reference spectrum after standardization.

[0066] Step 204: Obtain similarity features based on each piece of similarity information.

[0067] Exemplarily, this embodiment combines various similarity information to obtain similarity features : .

[0068] Step 205, perform feature dimensionality reduction processing on the spectral features, and based on a preset weight matrix, perform weighted fusion processing on the spectral features, morphological features and similarity features after the dimensionality reduction processing to obtain comprehensive features; wherein the preset weight matrix is ​​used to indicate the weight ratio of the spectral features, morphological features and similarity features after the dimensionality reduction processing in the comprehensive features.

[0069] Exemplarily, this embodiment first performs PCA processing on the spectral features to reduce the dimension and remove redundancy. PCA projects the data onto the principal component axis by calculating the eigenvectors and eigenvalues, retaining the main feature information. Assume that the matrix of the spectral features is , the calculation process of PCA is as follows:

[0070] in, is the spectral feature after dimension reduction, It is before The eigenvectors of the principal components, is the mean vector, through Calculate the average value by column, is the centralized data matrix, is the covariance matrix, is obtained by The result of eigendecomposition, eig represents eigendecomposition, is the eigenvector matrix, is a diagonal matrix of eigenvalues.

[0071] After obtaining the spectral features after dimensionality reduction processing, this embodiment converts the spectral features after PCA processing into , morphological characteristics, such as area ,perimeter , Form Factor Etc., and similarity features Perform feature fusion and combine them into a fused feature vector :

[0072] Among them, this embodiment can also balance the influence of different features on the classification results by adjusting the feature weights. The preset weight matrix is , the comprehensive feature vector representation can be: ,in Represents element-wise product.

[0073] Step 206, input the comprehensive features into the trained classification model to obtain the category information of the target object; wherein the classification model is trained based on multiple comprehensive features and the category label corresponding to each comprehensive feature.

[0074] Exemplarily, this embodiment selects a Linear Discriminant Analysis (LDA) classifier as a classification model. The comprehensive features are input into the trained classification model to obtain the category information of the target object.

[0075] Among them, LDA finds the optimal projection direction by maximizing the ratio of the between-class variance to the within-class variance. This embodiment uses the existing labeled data for supervised learning and combines multiple comprehensive features As an independent variable, the category label corresponding to each comprehensive feature Input into the model as the dependent variable and train the LDA classification model.

[0076] Step 207 : determining whether the target object is a foreign object based on the category information, or determining whether the target object includes a foreign object based on the category information.

[0077] In one example, step 207 includes the following steps: Obtain visible light data and infrared data of the target object.

[0078] According to the category information, the visible light data and the infrared data, it is determined whether the target object is a foreign object, or whether the target object includes a foreign object.

[0079] Exemplarily, this embodiment can determine whether the target object is a foreign object, or determine whether the target object includes a foreign object based on the category information output by the model.

[0080] In one example, this embodiment determines whether the target object is a foreign object or whether the target object includes a foreign object based on the category information output by the model and the cargo information to which the target object belongs.

[0081] In a feasible implementation, the present embodiment can also determine whether the target object is a foreign object, or whether the target object includes a foreign object, based on the category information obtained based on the spectral data, the visible light data, and the infrared data. For example, the category information output by the model is double-determined based on the visible light data and the infrared data.

[0082] In another feasible implementation manner, the present embodiment can also synchronously acquire visible light data and infrared data of the target object through a visible light camera and an infrared spectrometer, and then, based on the spectral data, visible light data and infrared data, analyze and obtain the category information of the target object, and then determine whether the target object is a foreign object, or whether the target object includes foreign objects.

[0083] The visible light camera is used to capture the morphological features of the target, such as shape, size, color, etc. These feature information are crucial for the preliminary screening and identification of foreign objects. In this embodiment, the visible light camera and infrared spectrometer can be used to extract the morphological features, infrared spectrum features, etc. of the target. Then, through image processing technology, the visible light image can be extracted to obtain key features such as the outline and texture of the target.

[0084] For example, this embodiment uses a high-resolution visible light camera to shoot the target object to ensure that the image is clear and detailed. The contour features of the target object are extracted through image processing algorithms, such as edge detection such as the Canny operator, contour tracking, etc. The extracted contour is smoothed to remove noise and redundant information, and improve the accuracy and readability of the contour. This embodiment can also extract texture features through methods such as grayscale co-occurrence matrix and local binary pattern. The extracted texture features may include contrast, roughness, directionality, etc., which can reflect subtle differences in the surface of the target object.

[0085] Secondly, this embodiment extracts the infrared spectrum characteristics of the target object through an infrared spectrometer to reflect the chemical composition and physical state of the target object. This embodiment can obtain the spectrum information of the target object in the infrared band through scanning by the infrared spectrometer, and then analyze its chemical structure and composition, wherein the infrared spectrometer can be replaced by a hyperspectral camera whose band includes the infrared band.

[0086] The preset spectral library can also pre-store feature information such as morphological features and infrared spectral features of various categories of samples, calculate the similarity between the morphological features, infrared spectral features and spectral features of the target object and the samples of each category in the preset spectral library, obtain the similarity features of the target object, and then fuse the similarity features, morphological features, infrared spectral features and spectral features to obtain comprehensive features, which are input into the classification model to obtain the category information of the target object output by the model. This achieves the effect of improving the accuracy of the output of the category information. Correspondingly, when training the classification model, the comprehensive features obtained by fusing the similarity features, morphological features, infrared spectral features and spectral features are also used to train the classification model.

[0087] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0088] The following is an embodiment of the device of the present invention. For details not described in detail, reference may be made to the corresponding method embodiments described above.

[0089] Figure 3 The structure diagram of the foreign body identification device provided by the embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows: like Figure 3 As shown, the foreign object identification device 300 includes: The first processing unit 301 is used to obtain spectral data of the target object, and extract spectral features and morphological features of the target object based on the spectral data.

[0090] The calculation unit 302 is used to calculate the similarity feature between the spectral feature and the reference spectrum in the preset spectral library; wherein the preset spectral library includes reference spectra corresponding to multiple categories of samples.

[0091] The second processing unit 303 is used to perform feature dimension reduction processing on the spectral features, and obtain category information of the target object according to the spectral features, morphological features and similarity features after the dimension reduction processing.

[0092] The determination unit 304 is used to determine whether the target object is a foreign object based on the category information, or to determine whether the target object includes a foreign object based on the category information.

[0093] In a possible implementation, the second processing unit 303 is specifically configured to: Based on the preset weight matrix, the spectral features, morphological features and similarity features after dimensionality reduction are weighted and fused to obtain comprehensive features; wherein the preset weight matrix is ​​used to indicate the weight ratio of the spectral features, morphological features and similarity features after dimensionality reduction in the comprehensive features.

[0094] The comprehensive features are input into a trained classification model to obtain category information of the target object; wherein the classification model is trained based on multiple comprehensive features and the category label corresponding to each comprehensive feature.

[0095] In a possible implementation, the calculation unit 302 is specifically configured to: The spectral features and the reference spectra corresponding to each category of samples in the preset spectral library are standardized to obtain the standardized spectral features and each reference spectrum; wherein the standardized spectral features and each reference spectrum are both in the form of probability distribution.

[0096] Calculate the similarity information between the standardized spectral features and each standardized reference spectrum.

[0097] According to each piece of similarity information, a similarity feature is obtained.

[0098] In a possible implementation, the calculation unit 302 is further configured to: according to , calculate the similarity information between the standardized spectral features and each standardized reference spectrum.

[0099] in, is the similarity information between the standardized spectral feature and each standardized reference spectrum, is the spectral feature after standardization. is the i-th component of the spectral feature after normalization, is the reference spectrum after standardization processing, is the j-th component of the reference spectrum after standardization, i is a non-zero integer, i=1...n, n represents the total number of components included in the spectral characteristics after standardization; j is a non-zero integer, j=1...m, m represents the total number of components included in the reference spectrum after standardization.

[0100] In a possible implementation, the first processing unit 301 is specifically configured to: The spectral data is converted and processed to obtain a pseudo-color image corresponding to the target object.

[0101] The pseudo-color image is preprocessed, and contour information of the target object is obtained based on the pseudo-color image after preprocessing; wherein the preprocessing includes one or more of grayscale processing, Gaussian filtering denoising processing, binarization processing and morphological processing.

[0102] According to the contour information, spectral features and morphological features are extracted; wherein the morphological features include area features, perimeter features and shape factor information.

[0103] In a possible implementation, the first processing unit 301 is further configured to: The spectral data is collected by using an external push-scan method using a hyperspectral camera; the hyperspectral camera is installed on the UAV equipment, and the target object is a fixed target.

[0104] Alternatively, a hyperspectral camera is used to collect spectral data of the target object on the conveyor belt in an external push-scan manner; wherein the hyperspectral camera is mounted on a linear stand.

[0105] Alternatively, a hyperspectral camera is used to collect spectral data of the target object in a built-in push-scan manner; wherein the hyperspectral camera is installed on a fixed stand and the target object is a fixed target object.

[0106] In a possible implementation, the determination unit 304 is specifically configured to: Obtain visible light data and infrared data of the target object.

[0107] According to the category information, the visible light data and the infrared data, it is determined whether the target object is a foreign object, or whether the target object includes a foreign object.

[0108] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0109] Figure 4 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 40 and a memory 41. The memory 41 is used to store a computer program 42, and the processor 40 is used to call and run the computer program 42 stored in the memory 41 to perform the steps in the above-mentioned foreign body identification method embodiments, such as Figure 1 Alternatively, the processor 40 is used to call and run the computer program 42 stored in the memory 41 to implement the functions of each module / unit in the above-mentioned device embodiments, such as Figure 3 The functions of each unit are shown.

[0110] Exemplarily, the computer program 42 may be divided into one or more modules / units, one or more modules / units are stored in the memory 41 and executed by the processor 40 to complete the present application. One or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 42 in the electronic device 4. For example, the computer program 42 may be divided into Figure 3 The individual units shown.

[0111] The electronic device 4 may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that Figure 4 It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0112] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0113] The memory 41 may be an internal storage unit of the electronic device 4, such as a hard disk or memory of the electronic device 4. The memory 41 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 41 is used to store computer programs and other programs and data required by the electronic device. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0114] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0115] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0117] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0120] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0121] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A foreign body identification method, characterized in that: The method comprises: Acquiring spectral data of a target object, and extracting spectral features and morphological features of the target object based on the spectral data; Calculate and obtain the similarity feature between the spectral feature and the reference spectrum in a preset spectral library; wherein the preset spectral library includes reference spectra corresponding to samples of multiple categories; Performing feature dimensionality reduction processing on the spectral features, and obtaining category information of the target object according to the spectral features after the dimensionality reduction processing, the morphological features and the similarity features; Based on the category information, it is determined whether the target object is a foreign object, or based on the category information, it is determined whether the target object includes a foreign object.

2. The foreign body identification method according to claim 1, characterized in that: The obtaining the category information of the target object according to the spectral features, the morphological features and the similarity features after the dimensionality reduction process includes: Based on a preset weight matrix, the spectral features after the dimensionality reduction processing, the morphological features and the similarity features are weighted and fused to obtain a comprehensive feature; wherein the preset weight matrix is ​​used to indicate the weight proportion of the spectral features after the dimensionality reduction processing, the morphological features and the similarity features in the comprehensive feature; The comprehensive feature is input into a trained classification model to obtain category information of the target object; wherein the classification model is trained based on multiple comprehensive features and a category label corresponding to each comprehensive feature.

3. The foreign body identification method according to claim 1, characterized in that: The calculating of the similarity feature between the spectral feature and the reference spectrum in the preset spectral library includes: The spectral features and the reference spectra corresponding to each category of samples in the preset spectral library are standardized to obtain the standardized spectral features and each reference spectrum; wherein the standardized spectral features and each reference spectrum are both in the form of probability distribution; Calculate the similarity information between the standardized spectral features and each standardized reference spectrum; The similarity feature is obtained according to each piece of similarity information.

4. The foreign body identification method according to claim 3, characterized in that: The calculating of similarity information between the standardized spectral feature and each standardized reference spectrum includes: according to , calculate the similarity information between the standardized spectral features and each standardized reference spectrum; in, is the similarity information between the standardized spectral feature and each standardized reference spectrum, is the spectral feature after standardization. is the i-th component of the spectral feature after normalization, is the reference spectrum after standardization processing, is the j-th component of the reference spectrum after standardization, i is a non-zero integer, i=1...n, n represents the total number of components included in the spectral characteristics after standardization; j is a non-zero integer, j=1...m, m represents the total number of components included in the reference spectrum after standardization.

5. The foreign body identification method according to any one of claims 1 to 4, characterized in that: The extracting of spectral features and morphological features of the target object based on the spectral data includes: Performing data conversion processing on the spectral data to obtain a pseudo-color image corresponding to the target object; Preprocessing the pseudo-color image, and obtaining contour information of the target object based on the pseudo-color image after preprocessing; wherein the preprocessing includes one or more of grayscale processing, Gaussian filtering denoising processing, binarization processing and morphological processing; The spectral features and the morphological features are extracted based on the contour information; wherein the morphological features include area features, perimeter features, and shape factor information.

6. The foreign body identification method according to any one of claims 1 to 4, characterized in that: The obtaining of spectral data of the target object comprises: The spectral data is acquired by using a hyperspectral camera in an external push-scan manner; wherein the hyperspectral camera is installed on an unmanned aerial vehicle (UAV) and the target object is a fixed target object; Alternatively, the spectral data of the target object on the conveyor belt is acquired by using a hyperspectral camera in an external push-scan manner; wherein the hyperspectral camera is mounted on a linear stand; Alternatively, the spectral data of the target object is acquired by using a hyperspectral camera in a built-in push-scan manner; wherein the hyperspectral camera is installed on a fixed stand, and the target object is a fixed target object.

7. The foreign body identification method according to any one of claims 1 to 4, characterized in that: Determining whether the target object is a foreign object based on the category information includes: Acquire visible light data of the target object and infrared data of the target object; According to the category information, the visible light data and the infrared data, it is determined whether the target object is a foreign object, or whether the target object includes a foreign object.

8. A foreign body identification device, characterized in that: The device comprises: A first processing unit is used to obtain spectral data of a target object, and extract spectral features and morphological features of the target object based on the spectral data; A calculation unit, used for calculating the similarity characteristics between the spectral characteristics and the reference spectra in a preset spectral library; wherein the preset spectral library includes reference spectra corresponding to samples of multiple categories; A second processing unit is used to perform feature dimensionality reduction processing on the spectral features, and obtain category information of the target object according to the spectral features after the dimensionality reduction processing, the morphological features and the similarity features; A distinguishing unit is used to determine whether the target object is a foreign object based on the category information, or to determine whether the target object includes a foreign object based on the category information.

9. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the foreign body identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the foreign object recognition method according to any one of claims 1 to 7 is implemented.

Images