Data processing method and system for quality monitoring of candy production line

Abstract

The invention provides a data processing method and system for quality monitoring of a candy production line, and the method comprises the steps: projecting detection data and X-ray detection data to a combined potential space, obtaining a prior probability of foreign matter pollution, and determining a receptive field parameter of X-ray depth feature extraction according to the prior probability; calculating a metal feature vector of the metal detection data and the confidence coefficient of each vector component, extracting an X-ray feature map by adopting a receptive field, carrying out weighted preprocessing on the X-ray feature map by utilizing the confidence coefficient, carrying out cross attention calculation on the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, and carrying out cross attention calculation on the activation map; iteratively optimizing receptive field parameters based on the geometric features and the activation intensity of the high activation region set; x-ray features are extracted by using receptive field parameters obtained through iterative optimization, cross-modal correlation intensity is obtained through activation atlas calculation, and the extracted X-ray features, the cross-modal correlation intensity and feature stability and saliency indexes are input into a classifier to obtain defect types and probabilities.

Description

Technical Field

[0001] This application belongs to the field of quality monitoring, and particularly relates to a data processing method and system for quality monitoring of a candy production line. Background Art

[0002] In a candy production line, due to various factors such as raw material sources, production processes, and equipment wear, various foreign objects may be mixed into the products, such as metal debris, glass shards, stones, etc. The existence of these foreign objects not only seriously threatens the food safety of consumers, but may also lead to damage to the brand image and economic compensation. Deploying a foreign object detection system on the candy production line to monitor the product quality is a key link to ensure product quality and safety. The commonly used foreign object detections mainly include metal detection and X-ray detection. Metal detection mainly uses the principle of electromagnetic induction. When a metal foreign object passes through the detection area, it will cause a change in the electromagnetic field and thus be identified. However, candies with high salt content and high moisture content may generate signals similar to metals, interfering with the detection results. Moreover, for metal foreign objects with irregular shapes and tiny sizes, the detection sensitivity will significantly decrease, and non-metal foreign objects such as glass, stones, plastics, etc. cannot be detected. X-ray detection uses the principle that when X-rays penetrate substances, attenuation differences occur due to different densities, and foreign objects are identified by analyzing the intensity distribution image of the transmitted X-rays. The X-ray detection system can not only effectively detect metal foreign objects, but also detect non-metal foreign objects such as glass and stones. However, the X-ray detection technology has weak detection ability for low-density foreign objects.

[0003] Combining metal detection and X-ray detection can utilize the complementarity of the two technologies to achieve more comprehensive quality monitoring. However, simply connecting the two detection devices in series cannot fully exert the advantages of combined detection, but may instead increase the false alarm rate or fail to effectively improve the detection ability for difficult foreign objects due to ineffective information fusion. How to effectively fuse the detection data from two different physical principles and extract more discriminative features is a technical problem currently faced. Summary of the Invention

[0004] In view of the above problems, this application proposes a data processing method for quality monitoring of a candy production line, including: Collecting metal detection data and X-ray detection data in candy production, projecting the metal detection data and X-ray detection data into a joint latent space, obtaining a prior probability of foreign object contamination based on the joint latent space, and determining parameters of the receptive field for X-ray depth feature extraction according to the prior probability; Calculate the metal feature vector of the metal detection data and the confidence of each vector component, extract the X-ray feature map using the receptive field, perform weighted preprocessing on the X-ray feature map using the confidence, perform cross-attention calculation on the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, identify the set of high-activation regions from the activation map, and iteratively optimize the receptive field parameters based on the geometric features and activation intensity of the high-activation region set; Extract the X-ray features using the receptive field parameters obtained by iterative optimization, calculate the cross-modal association intensity through the activation map, and input the extracted X-ray features, the cross-modal association intensity, and the feature stability and significance indicators of each round of attention maps during the iteration process into a preset classifier to obtain the defect type and probability.

[0005] Preferably, the projecting the metal detection data and the X-ray detection data into the joint latent space includes: After preprocessing the metal detection data, extract a feature set including signal amplitude, phase, and frequency response. After preprocessing the X-ray detection data, extract a feature set including grayscale statistics, texture descriptors, and edge gradients. Concatenate the two feature sets into a high-dimensional feature vector; Non-linearly map the high-dimensional feature vector to the joint latent space of a preset dimension through an autoencoder. The training objective of the autoencoder is to minimize the difference between the original features and the reconstructed features, and at the same time maximize the separability of the distributions of different types of known foreign object samples in the latent space.

[0006] Preferably, the obtaining the prior probability of foreign object contamination based on the joint latent space includes: In the joint latent space, pre-define N foreign object category prototypes, and each prototype represents the latent space position of a specific foreign object or no foreign object; Calculate the Euclidean distance between the projection point of the current candy sample in the joint latent space and each pre-defined foreign object category prototype, convert the Euclidean distance into an N-dimensional probability vector through the Softmax function. Each component of the probability vector represents the prior probability that the current sample belongs to the corresponding foreign object category, and the sum of all components is 1.

[0007] Preferably, the determining the parameters of the receptive field for X-ray depth feature extraction according to the prior probability includes: If the prior probability of a certain foreign object category exceeds the upper threshold, obtain the Gaussian kernel parameters for this foreign object type from a preset parameter library; If the prior probabilities of all categories are lower than the lower threshold, obtain the Gaussian kernel parameters applicable globally from a preset parameter library; Otherwise, perform weighted averaging on the Gaussian kernel parameters corresponding to each foreign object category to generate Gaussian kernel parameters.

[0008] Preferably, calculating the metal feature vector of the metal detection data and the confidence of each vector component includes: Performing time-frequency analysis on the original metal detection signal to obtain a time-frequency spectrogram, identifying the regions matching the known metal foreign object signal pattern through template matching, and extracting the center frequency, bandwidth, duration, and energy of the matching region as the components of the metal feature vector; Calculating the deviation degree of each extracted metal feature vector component from the expected value of the metal foreign object on this component and the signal-to-noise ratio of the signal. If the signal-to-noise ratio is greater than the preset value and the deviation degree is within the preset range, the confidence is 0.95. Otherwise, determine the reduction amplitude of the confidence according to the deviation degree and the signal-to-noise ratio.

[0009] Preferably, using the confidence to perform weighted preprocessing on the X-ray feature map includes: Normalizing the confidence of each component in the metal feature vector, and determining the corresponding region of interest on the X-ray feature map according to the spatial position of the metal foreign object indicated by the metal feature vector in the X-ray image; Weighting the eigenvalues of the X-ray feature map within the region of interest, and the calculation method of the weight W is: W = 1 + α * C avg , where C avg is the average value of the confidence of each component of the metal feature vector corresponding to this region of interest, and α is the enhancement coefficient.

[0010] Preferably, performing cross-attention calculation on the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, and identifying the set of high-activation regions from the activation map includes: Using the preprocessed X-ray feature map as the query, the metal feature vector as the key and value, calculating the dot product similarity between the query and the key, applying the Softmax function to obtain the attention weight, and then multiplying the attention weight by the value and summing to generate a cross-attention activation map; Performing image segmentation on the activation map, and taking the connected regions with activation values higher than the activation threshold as high-activation regions.

[0011] Preferably, iteratively optimizing the receptive field parameters based on the geometric features and activation intensity of the set of high-activation regions includes: Calculating the geometric features and average activation intensity of the high-activation regions, and the geometric features include the centroid coordinates, major axis direction, major axis length, and minor axis length of the high-activation regions; Move the center positions of the M Gaussian kernels closest to the centroid of the highly activated region towards the centroid by a predetermined step size; adjust the covariance matrices of the M Gaussian kernels so that the major axis directions of the Gaussian kernels are aligned with the major axis direction of the highly activated region, and the sizes of the Gaussian kernels are proportional to the major axis direction and the length of the major axis; adjust the weights of the M Gaussian kernels according to the average activation intensity, and the increase in weight is proportional to the average activation intensity; The iterative process is executed for a preset number of times, or until the change amount of the receptive field parameters is less than a preset convergence threshold.

[0012] Preferably, the feature stability and significance indicators of the attention maps in each round of the iterative process include: For each highly activated region in the current round of map, calculate the area change rate and position drift distance between each highly activated region in the current round of map and the corresponding highly activated region in the previous round of map; take the average value of the area change rates and the average value of the position drift distances of all corresponding highly activated regions as the stability indicator of the current round; Calculate the ratio of the average activation intensity of all highly activated regions in the current round of activation map to the average activation intensity of the entire map, and take the ratio as the significance indicator of the current round; Take the average value of the stability indicators of all rounds as the feature stability value, and take the average value of the significance indicators of all rounds as the significance indicator value.

[0013] In addition, the present application also proposes a data processing system for quality monitoring of a candy production line, including: An initial unit for collecting metal detection data and X-ray detection data in candy production, projecting the metal detection data and X-ray detection data into a joint latent space, obtaining a prior probability of foreign object contamination based on the joint latent space, and determining parameters of the receptive field for X-ray depth feature extraction according to the prior probability; An optimization unit for calculating the metal feature vector of the metal detection data and the confidence of each vector component, extracting an X-ray feature map using the receptive field, performing weighted preprocessing on the X-ray feature map using the confidence, performing cross-attention calculation on the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, identifying a set of highly activated regions from the activation map, and iteratively optimizing the receptive field parameters based on the geometric features and activation intensity of the set of highly activated regions; An identification unit for extracting X-ray features using the receptive field parameters obtained by iterative optimization, calculating the cross-modal association intensity through the activation map, and inputting the extracted X-ray features, the cross-modal association intensity, and the feature stability and significance indicators of the attention maps in each round of the iterative process into a preset classifier to obtain the defect type and probability.

[0014] In this application, by projecting two types of detection data into a joint latent space and analyzing to obtain the prior probability of foreign object contamination, and then determining the receptive field parameters for X-ray depth feature extraction based on the prior probability, the X-ray feature extraction becomes more targeted, capable of adjusting the observation scale according to the most likely foreign object type, and improving the sensitivity and efficiency of early detection of specific types of foreign objects. Moreover, this application uses the confidence of each component of the metal feature to adjust the weight of the metal feature in attention calculation and performs weighted preprocessing on the X-ray feature map, ensuring the reliability of information fusion and improving the accuracy of decision-making. In addition, this application enhances the detection ability for tiny and complex foreign objects similar to the background through an iterative receptive field parameter optimization method based on the feedback of high-activation regions in the cross-attention map. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flowchart of a specific embodiment; Figure 2 is a schematic diagram of metal detection data; Figure 3 is a candy X-ray scan image; Figure 4 is a schematic diagram of the joint latent space; Figure 5 is a schematic diagram of region of interest enhancement. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of the present application.

[0017] Specific embodiments are as follows Figure 1 As shown, the present application proposes a data processing method for quality monitoring of a candy production line, including: S1. Collect metal detection data and X-ray detection data in candy production, project the metal detection data and the X-ray detection data into a joint latent space, obtain the prior probability of foreign object contamination based on the joint latent space, and determine the parameters of the receptive field for X-ray depth feature extraction according to the prior probability; Synchronously obtain the metal detection signal and the X-ray scan image generated by the candies flowing through the production line. The metal detection data is as shown in Figure 2 and the X-ray scan data is as shown in Figure 3As shown, they are bound according to the speed of the conveyor belt. For example, a metal detection signal is obtained at time t1, and an X-ray scan image is obtained at time t2. According to the speed of the conveyor belt, the two can be corresponding. Feature extraction is performed on the obtained metal detection data and X-ray detection data. For example, for metal data, the energy envelope and specific frequency components of the signal are extracted, and for X-ray data, the average density and local contrast information of the image are extracted. Those skilled in the art should know that preprocessing can be performed before feature extraction, which will not be elaborated here. The extracted features are mapped to a low-dimensional joint latent feature space by means of dimensionality reduction. The dimensionality reduction methods include but are not limited to principal component analysis or non-linearity, etc. The joint latent feature space contains the internal connection and complementary information between the two types of sensor data. In the joint latent space, the prior probability of foreign object contamination is estimated by analyzing the proximity of the projection point of the current sample to the known contamination category distribution in the historical sample data. The prior probability reflects the preliminary judgment on the possible type or size of the foreign object, and further guides the initial setting of the receptive field in X-ray image analysis. For example, if the prior probability indicates that there may be a slender foreign object, the shape of the receptive field will be adjusted accordingly to a more directional ellipse.

[0018] S2, calculate the metal feature vector of the metal detection data and the confidence of each vector component, extract the X-ray feature map using the receptive field, perform weighted preprocessing on the X-ray feature map using the confidence, perform cross-attention calculation on the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, identify the high-activation region set from the activation map, and iteratively optimize the receptive field parameters based on the geometric features and activation intensity of the high-activation region set; Further analyze the metal detection data to obtain a metal feature vector containing multiple specific indicators, and assign a confidence to each indicator. In one embodiment, the confidence is calculated based on the stability and clarity of the signal. Using the determined receptive field parameters, extract the X-ray feature map from the X-ray image. The confidence of the metal feature is used to adjust the processing method of the X-ray feature map. For example, in the area where the metal signal indicates suspicion and the confidence is high, certain features of the corresponding part of the X-ray feature map will be given higher weights or the extraction process will be enhanced. The X-ray feature map and the metal feature vector are jointly input into a cross-attention module. The cross-attention module calculates the mutual correlation degree between the two modal features to generate a global activation map. The highlighted areas on the map represent the potential abnormal positions jointly pointed to by the metal and X-ray evidence. After identifying the high-activation regions, use image processing and other methods to obtain information such as their shape, size, and activation intensity, and then iteratively adjust the parameters of the X-ray receptive field through a feedback mechanism. For example, gradually focus the center of the receptive field on the centroid of the high-activation region, or adjust the receptive field size to more closely cover the region.

[0019] S3. Extract the X-ray features using the receptive field parameters obtained through iterative optimization, calculate the cross-modal correlation intensity through the activation map, and input the extracted X-ray features, the cross-modal correlation intensity, and the feature stability and significance indicators of each round of attention maps during the iterative process into a preset classifier to obtain the defect type and probability.

[0020] After the iterative optimization process converges, extract the X-ray features from the X-ray data using the finally determined receptive field parameters. At the same time, the cross-attention activation map itself provides an overall cross-modal correlation intensity, which is an index that can quantify the degree of association between metal detection features and X-ray image features and is extracted from the activation map generated by cross-attention calculation. This intensity can be one or more numerical values. For example, calculate the average activation value, the highest activation value of all high-activation regions in the activation map, or the sum of the energies of high-activation regions, etc. These numerical values together constitute the cross-modal correlation intensity features input to the subsequent classifier.

[0021] Furthermore, calculate the change situation of the high-activation regions on the activation map, such as the stability of the positions of these regions and the convergence speed of the activation intensity. In one embodiment, the change situation is specifically the feature stability and significance indicators. The X-ray features, the cross-modal correlation intensity score, and the dynamic indicators during the iterative process are used as a comprehensive feature set and input into a pre-trained classification model, which includes but is not limited to decision trees or gradient boosting machines. The classification model makes a final judgment on the quality status of the candies and outputs a specific defect type judgment and the confidence probability of this judgment. Among them, the greater the cross-modal correlation intensity, the stronger the consistency between metal detection and X-ray detection, while the feature stability and significance indicate the reliability of the features, which are then combined into the X-ray features to improve the recognition accuracy.

[0022] In an alternative embodiment, the projecting the metal detection data and the X-ray detection data into the joint latent space includes: After preprocessing the metal detection data, extract a feature set including signal amplitude, phase, and frequency response. After preprocessing the X-ray detection data, extract a feature set including gray-scale statistics, texture descriptors, and edge gradients. Concatenate the two feature sets into a high-dimensional feature vector. Non-linearly map the high-dimensional feature vector into a joint latent space of a preset dimension through an autoencoder. The training objective of the autoencoder is to minimize the difference between the original features and the reconstructed features, and at the same time maximize the separability of the distributions of different types of known foreign object samples in the latent space.

[0023] The key information is extracted from the data collected by the metal detector after processing such as signal denoising. The key information includes, but is not limited to, the intensity of the metal signal, the phase of the signal, and the main frequency of the signal. At the same time, the image data obtained from the X-ray scan after preprocessing is extracted. The image features include, but are not limited to, the average gray value such as 180, a set of numerical values describing the texture characteristics of the image such as 0.2, 0.5, 0.8, and the gradient value representing the edge sharpness of the image such as 25. The numerical information from the metal detection and the numerical information from the X-ray image are combined to obtain a high-dimensional feature vector. If there are 3 numerical values for the metal data and 5 numerical values for the X-ray data, then the dimension of the high-dimensional feature vector is 8. The high-dimensional feature vector containing 8 numerical values is input into an autoencoder, and the autoencoder can transform or compress this 8-dimensional feature vector into a space with a lower dimension, such as a 3-dimensional joint latent space, and output a vector containing 3 new numerical values. Among them, the autoencoder has two learning objectives: one is to ensure that the original 8-dimensional feature vector can be restored as accurately as possible from this 3-dimensional latent space vector to ensure that not too much information is lost; the other is to make the candy samples representing different types of foreign objects such as small iron filings or small stones, their corresponding 3-dimensional latent space vectors can automatically gather in different regions in the space during the learning process, or in other words, the distributions of different category samples can be clearly separated. For example, all the latent vectors of the samples containing iron filings are close to point A, and the latent vectors of the samples containing stones are close to point B, and points A and B are far apart in the latent space, as Figure 4 shown.

[0024] In an alternative embodiment, obtaining the prior probability of foreign object contamination based on the joint latent space includes: In the joint latent space, N foreign object category prototypes are predefined, and each prototype represents the latent space position of a specific foreign object or no foreign object; Calculate the Euclidean distance between the projection point of the current candy sample in the joint latent space and each predefined foreign object category prototype, and convert the Euclidean distance into an N-dimensional probability vector through the Softmax function. Each component of the probability vector represents the prior probability that the current sample belongs to the corresponding foreign object category, and the sum of all components is 1.

[0025] To determine the possibility of foreign object contamination, i.e., the prior probability, in candies, several representative class centroids, e.g., 3, are preset in the joint latent space, which are the foreign object class prototypes. The first prototype may represent the ideal position of foreign objects such as metal debris in the space, the second prototype represents the position of foreign objects such as stones, and the third prototype represents the position without any foreign objects. In one embodiment, the prototypes are learned based on a large amount of known sample data. They are the standard reference points for each category, which are the cluster centers of each category obtained by performing cluster analysis on the projection points of a large number of labeled training samples in the joint latent space, or are manually set as representative points in the latent space by domain experts according to prior knowledge and sample distribution characteristics. When a new candy sample enters the detection, its metal detection data and X-ray detection data are integrated and processed and mapped into the joint latent space to form a new data point. Calculate the distance between this new sample data point and each predefined foreign object class prototype. Preferably, the Euclidean distance is used. For example, the distance from the new sample point to the metal prototype is 2, the distance to the stone prototype is 10, and the distance to the no foreign object prototype is 5. Further, Softmax is used to convert 2, 10, and 5 into a three-dimensional probability list. Since the new sample point is closest to the metal prototype, its corresponding probability will be the highest. For example, the calculation result may be 0.8 for metal, 0.05 for stone, and 0.15 for no foreign object. Each value represents the prior probability that the candy sample belongs to the corresponding foreign object category.

[0026] In an alternative embodiment, the parameters for determining the receptive field of X-ray depth feature extraction according to the prior probability include: If the prior probability of a certain foreign object category exceeds the upper threshold, obtain the Gaussian kernel parameters for this foreign object type from a preset parameter library; If the prior probabilities of all categories are lower than the lower threshold, obtain the Gaussian kernel parameters applicable globally from a preset parameter library; Otherwise, perform weighted averaging on the Gaussian kernel parameters corresponding to each foreign object category to generate Gaussian kernel parameters.

[0027] Specifically, assume that three possible foreign object categories are identified: metal, stone, and glass, and the prior probabilities are 0.8 for metal, 0.1 for stone, and 0.05 for glass. If the upper threshold is set to 0.7, since the prior probability of metal, 0.8, exceeds this upper threshold, Gaussian kernel parameters optimized specifically for detecting metal foreign objects are retrieved from the parameter library, such as a set of parameters describing small size and high detail attention, for X-ray depth feature extraction to accurately identify tiny metal debris. If the prior probability of metal is 0.1, stone is 0.08, glass is 0.06, and the lower threshold is 0.2, and the prior probabilities of all foreign object categories are lower than this lower threshold, then there is no clear high-risk foreign object type currently, and a set of general Gaussian kernel parameters applicable to global scanning is selected from the parameter library. This set of parameters describes a receptive field with a larger range and balanced observation for general screening. If the situation is between the above two cases, for example, the probability of metal is 0.5, the probability of stone is 0.3, and the probability of glass is 0.1, then the preset metal detection parameters and stone detection parameters are retrieved, and a new set of Gaussian kernel parameters is generated by weighted averaging according to their respective prior probabilities of 0.5 and 0.3 as weights. In one embodiment, the Gaussian kernel parameters define a set of initial receptive field characteristics, including but not limited to the size, standard deviation, mean, and / or covariance matrix of the Gaussian kernel matrix, such as the initial means, covariances, and weights for M Gaussian kernels. These parameters will serve as the starting point for the subsequent iterative optimization process and be fine-tuned through actual data-driven means.

[0028] In an alternative embodiment, calculating the metal feature vector of the metal detection data and the confidence of each vector component includes: Performing time-frequency analysis on the original metal detection signal to obtain a time-frequency spectrogram, identifying the regions matching the known metal foreign object signal patterns through template matching, and extracting the center frequency, bandwidth, duration, and energy of the matching regions as the components of the metal feature vector; Calculating the degree of deviation of each extracted metal feature vector component from the expected value of the metal foreign object on this component and the signal-to-noise ratio of the signal. If the signal-to-noise ratio is greater than the preset value and the degree of deviation is within the preset range, the confidence is 0.95; otherwise, the confidence reduction amplitude is determined according to the degree of deviation and the signal-to-noise ratio.

[0029] Specifically, the original signal from the metal detector is subjected to time-frequency analysis, such as wavelet transform, to be converted into a time-frequency spectrogram, which can display the intensity of the signal at different times and different frequencies simultaneously. By matching the template with the generated time-frequency spectrogram, the area similar to the fingerprint of the known metal foreign object is found, and then key data points are extracted from it. For example, the center frequency of the matching area is 100 kHz, the signal bandwidth is 10 kHz, the signal lasts for 5 mm, and its energy intensity is 0.2. These four values form the feature vector describing this potential metal foreign object. For each component in the metal feature vector, such as 100 kHz, the deviation is calculated by comparing this component with the expected value of the typical center frequency of the small steel ball. Moreover, the clarity of the current signal, that is, the signal-to-noise ratio, is also calculated. Suppose the preset signal-to-noise ratio threshold is 15 dB, and the allowable deviation range of the expected frequency is plus or minus 5 kHz. If the signal-to-noise ratio of the current signal is 20 dB, higher than 15 dB, and the measured center frequency of 100 kHz differs from the expected 105 kHz by 5 kHz, within the allowable deviation range, then the confidence level of this center frequency component is very high. In one embodiment, it is set to 0.95. If the signal-to-noise ratio is only 10 dB, or the frequency deviation reaches 10 kHz, exceeding the range, then the confidence level will be adjusted downward accordingly. The specific reduction amplitude is determined according to the actual deviation size and signal-to-noise ratio value, and the present invention does not make specific limitations on this.

[0030] In an alternative embodiment, the weighted preprocessing of the X-ray feature map using the confidence level includes: Normalize the confidence level of each component in the metal feature vector, and determine the corresponding region of interest on the X-ray feature map according to the spatial position of the metal foreign object indicated by the metal feature vector in the X-ray image; Weight the eigenvalues of the X-ray feature map within the region of interest, and the calculation method of the weight W is: W = 1 + α * C avg , where C avg is the average value of the confidence levels of each component of the metal feature vector corresponding to this region of interest, and α is the enhancement coefficient.

[0031] The determination of spatial position depends on the physical calibration between the metal detector and the X-ray scanner on the production line, the speed of the candy on the conveyor belt, and the synchronized timestamp of the sensor data acquisition. The time point detected by the metal detector is then converted into the corresponding spatial coordinate range when the X-ray image captures the candy or its surrounding area, thereby defining the area of interest. Suppose the metal detector detects a signal suspected to be a small metal wire and generates a metal feature vector containing three components, such as length estimation, material response, and shape regularity. The original confidence scores given to these three components are 0.9, 0.7, and 0.8 respectively. The confidence is normalized to the range of 0 to 1. If they are already in this range, they remain unchanged. At the same time, the metal detection data indicates that the suspected metal wire is located near the 50th row and 100th column of the candy image. On the corresponding X-ray feature map, a rectangular area of, for example, 10 pixels by 10 pixels is delineated around this position as the area of interest. The X-ray feature values within the area of interest are weighted. Specifically, C is calculated. avg , that is, the average value of the normalized confidence of each component of the metal feature vector corresponding to the area. Taking the above data as an example, the average value of the confidence is 0.8. If the enhancement coefficient α is 0.5, the weight is 1.4. In one embodiment, within the above 10 times 10 pixel area of interest, each X-ray feature value, such as the grayscale value or texture feature value of the point, will be multiplied by a weight of 1.4. If the metal detector is more confident that there is metal at a certain location, then the X-ray system will pay more attention to the signal there when analyzing the location, such as Figure 5 shown.

[0032] In an optional embodiment, performing cross-attention calculation on the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, and identifying a set of highly activated regions from the activation map, includes: The preprocessed X-ray feature map is used as the query, and the metal feature vector is used as the key and value. The dot product similarity between the query and the key is calculated, and the Softmax function is applied to obtain the attention weight. The attention weight is then multiplied by the value and summed to generate a cross-attention activation map. Image segmentation is performed on the activation map, and connected areas with activation values higher than an activation threshold are regarded as high activation areas.

[0033] Specifically, if the preprocessed X-ray feature map is a 100x100 image representing the internal structure of a candy, where the value of each pixel represents the density at that point, for example, 0.2 represents a low-density area and 0.9 represents a high-density area. At the same time, the metal detector gives a metal feature vector, such as containing three values: [5, 75, 2], which represent that the detected metal signal indicates a size of 5, a signal intensity of 75, and a material type of 2. When calculating the cross-attention, each pixel or small area of the X-ray feature map is used as a query to compare with this metal feature vector as the key. If the feature of a small area in the X-ray image is high density and the size is close to 5, then its dot product similarity with the metal feature vector will be relatively high. The similarity score is normalized by the Softmax function to obtain the attention weight. For example, the attention weight of a certain pixel on the X-ray image is 0.8, indicating that this pixel area is highly correlated with the metal feature. In one embodiment, to ensure dimension matching for dot product similarity calculation, before the metal feature vector is input into the cross-attention module, it is dimensionally mapped to the same dimension as the feature dimension of the X-ray feature map through a learnable linear projection layer. Similarly, the X-ray feature map can also be appropriately transformed, such as flattened or adjusted in the channel dimension, before being used as a query to meet the input requirements of the attention mechanism. The attention weight is multiplied by the metal feature vector as the value and summed up to generate a new 100x100 cross-attention activation map. On this map, the higher the value of a region, for example, the average activation value of a region reaches 0.85, it indicates a strong correspondence between the features of this X-ray region and the features detected by the metal detector. Identify the set of high-activation regions from this activation map. If the activation threshold is 0.7, search for all connected regions composed of pixels with continuously higher values than 0.7. For example, if the activation values of all 12 pixels within a 3x4 pixel rectangular region on the map are greater than 0.7, then this 3x4 region is identified and marked as a high-activation region. If there are three such regions on the map, then the set of high-activation regions includes these three independent regions.

[0034] In an alternative embodiment, the iterative optimization of the receptive field parameters based on the geometric features and activation intensity of the high-activation region set includes: Calculate the geometric features and average activation intensity of the high-activation region, where the geometric features include the centroid coordinates, major axis direction, major axis length, and minor axis length of the high-activation region; Move the center positions of M Gaussian kernels closest to the centroid of the high-activation region by a predetermined step length; adjust the covariance matrix of the M Gaussian kernels so that the major axis direction of the Gaussian kernels aligns with the major axis direction of the high-activation region, and the size of the Gaussian kernels is proportional to the major axis direction and major axis length; adjust the weights of the M Gaussian kernels according to the average activation intensity, and the increase in weight is proportional to the average activation intensity; The iterative process is executed for a preset number of times or until the change amount of the receptive field parameter is less than a preset convergence threshold.

[0035] Specifically, calculate the geometric characteristics and signal intensity of the high-activation region. For example, the center point of the high-activation region is located at the 150th row and 220th column of the image coordinates. The overall shape of this region is slender, with its main direction making an angle of 30 degrees with the horizontal line. The length along this main direction is 25 pixel units, and the width perpendicular to the main direction is 10 pixel units. At the same time, the average activation signal intensity within this region is calculated to be 0.85. Select, for example, two Gaussian kernels that are closest to the center of this high-activation region, i.e., the coordinates (150, 220). The center positions of these two Gaussian kernels will be moved a small fixed distance, such as 3 pixels, in the direction of (150, 220). At the same time, the shape parameters of the two Gaussian kernels, that is, the covariance matrices, are also adjusted so that their main axis directions also become 30 degrees, and their sizes are adjusted proportionally according to the major axis length of 25 pixels and minor axis length of 10 pixels of the high-activation region to better match the shape of this region. Since the average activation intensity of this region is relatively high at 0.85, the weights of these two Gaussian kernels will also increase accordingly, and the increase amplitude is proportional to the intensity value of 0.85. The above geometric features and the process of adjusting the Gaussian kernel parameters are repeated, for example, fixed to execute 5 rounds, or until the change amount of the Gaussian kernel parameters between two consecutive rounds is very small, such as the overall change of the parameter values is less than five ten-thousandths, then the iteration stops.

[0036] In an optional embodiment, the feature stability and significance indicators of the attention maps in each round during the iterative process include: For each high-activation region in the current round map, calculate the area change rate and position drift distance between each high-activation region in the current round map and the corresponding high-activation region in the previous round map; take the average value of the area change rates and the average value of the position drift distances of all corresponding high-activation regions as the stability indicator for the current round; Calculate the ratio of the average activation intensity of all high-activation regions in the current round activation map to the average activation intensity of the entire map, and take this ratio as the significance indicator for the current round; Take the average value of the stability indicators of all rounds as the feature stability value, and take the average value of the significance indicators of all rounds as the significance indicator value.

[0037] During the iterative process, for the atlas generated in each round, the stability index and the significance index are calculated. Suppose in the second round of iteration, a highly activated region, compared with its corresponding region in the first round, has an area that changes from 100 pixels to 105 pixels, and its area change rate is five percent. At the same time, its center point moves from position (10, 10) to (11, 10), a movement of 1 pixel. If another highly activated region has an area that decreases from 50 pixels to 48 pixels, with a change rate of 0.04, and its center also moves 1 pixel, then the average area change rate in the second round of iteration is 0.5%, and the average position drift is 1 pixel. These two numbers together constitute the stability index for this round. In one embodiment, the corresponding relationship between highly activated regions in successive rounds is determined by calculating the overlap degree between regions or the Euclidean distance between centroids. For example, a certain highly activated region in the current round will be considered as the corresponding region for subsequent stability calculations with the region in the previous round that has the largest IoU and exceeds a preset threshold, or the region with the closest centroid distance. The average value of the area change rates and the average value of the position drift distances of all corresponding highly activated regions are used as the stability index for the current round.

[0038] Then the significance index for this round is calculated. If the average activation intensity of these highly activated regions is 0.8, while the average activation intensity of the entire atlas is 0.2, then the significance index is 4. When all iterations, for example, three iterations, are completed, these metrics calculated round by round are aggregated. It is obtained by separately calculating the average value of the average area change rates of all rounds and the average value of the average position drifts of all rounds. If the average area change rates for the three rounds are 1%, 0.5%, and 0.2% respectively, then the area change part in the stability value is the average of these three. Similarly, if the average position drifts for the three rounds are 0.5 pixels, 1 pixel, and 0.3 pixels respectively, then the position drift part in the stability value is 0.6 pixels. The significance index value is the average after summing up the significance index values obtained in all rounds. For example, the average of the significance indices 3.5, 4.0, and 4.2 for the three rounds is 3.9.

[0039] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention. In addition, any combination can be made between various different embodiments of the embodiments of the present invention, as long as it does not violate the idea of the embodiments of the present invention, it should also be regarded as the content disclosed in the embodiments of the present invention.

Claims

1. A data processing method for quality monitoring of a candy production line, characterized in that, Including: Collecting metal detection data and X-ray detection data in candy production, projecting the metal detection data and X-ray detection data into a joint latent space, obtaining a prior probability of foreign object contamination based on the joint latent space, and determining parameters of a receptive field for X-ray depth feature extraction according to the prior probability; Calculating a metal feature vector of the metal detection data and the confidence of each vector component, extracting an X-ray feature map using the receptive field, performing weighted preprocessing on the X-ray feature map using the confidence, performing cross-attention calculation on the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, identifying a set of highly activated regions from the activation map, and iteratively optimizing the receptive field parameters based on the geometric features and activation intensity of the set of highly activated regions; Extracting X-ray features using the receptive field parameters obtained by iterative optimization, calculating a cross-modal correlation intensity through the activation map, and inputting the extracted X-ray features, the cross-modal correlation intensity, and the feature stability and significance indicators of each round of attention maps during the iterative process into a preset classifier to obtain the defect type and probability.

2. The method according to claim 1, wherein The projecting the metal detection data and the X-ray detection data into the joint latent space includes: Preprocessing the metal detection data and then extracting a feature set including signal amplitude, phase, and frequency response, preprocessing the X-ray detection data and then extracting a feature set including gray-scale statistics, texture descriptors, and edge gradients, and splicing the two feature sets into a high-dimensional feature vector; Nonlinearly mapping the high-dimensional feature vector into a joint latent space of a preset dimension through an autoencoder, and the training objective of the autoencoder is to minimize the difference between the original feature and the reconstructed feature while maximizing the separability of the distributions of different types of known foreign object samples in the latent space.

3. The method according to claim 1, characterized in that, The obtaining the prior probability of foreign object contamination based on the joint latent space includes: In the joint latent space, N foreign object category prototypes are predefined, and each prototype represents a potential space position of a specific foreign object or no foreign object; Calculating the Euclidean distance between the projection point of the current candy sample in the joint latent space and each predefined foreign object category prototype, converting the Euclidean distance into an N-dimensional probability vector through a Softmax function, and each component of the probability vector represents the prior probability that the current sample belongs to the corresponding foreign object category, and the sum of all components is 1.

4. The method according to claim 1, wherein The determining the parameters of the receptive field for X-ray depth feature extraction according to the prior probability includes: If the prior probability of a certain foreign object category exceeds an upper threshold, obtain the Gaussian kernel parameters for this foreign object type from a preset parameter library; If the prior probabilities of all categories are lower than a lower threshold, obtain the Gaussian kernel parameters applicable globally from a preset parameter library; Otherwise, perform weighted averaging on the Gaussian kernel parameters corresponding to each foreign object category to generate Gaussian kernel parameters.

5. The method according to claim 1, characterized in that The calculating the metal feature vector of the metal detection data and the confidence of each vector component includes: Perform time-frequency analysis on the original metal detection signal to obtain a time-frequency spectrogram, identify the regions that match the known metal foreign object signal pattern through template matching, and extract the center frequency, bandwidth, duration, and energy of the matching regions as the components of the metal feature vector; Calculate the deviation degree of each extracted metal feature vector component from the expected value of the metal foreign object on the component and the signal-to-noise ratio of the signal. If the signal-to-noise ratio is greater than the preset value and the deviation degree is within the preset range, the confidence level is 0.

95. Otherwise, determine the reduction amplitude of the confidence level based on the deviation degree and the signal-to-noise ratio.

6. The method according to claim 1, characterized in that The weighted preprocessing of the X-ray feature map using the confidence level includes: Normalize the confidence level of each component in the metal feature vector, and determine the corresponding region of interest on the X-ray feature map according to the spatial position of the metal foreign object indicated by the metal feature vector in the X-ray image; Weight the eigenvalues of the X-ray feature map within the region of interest, and the calculation method of the weight W is: W = 1 + α * C avg , where C avg is the average of the confidence levels of the components of the metal feature vector corresponding to the region of interest, and α is the enhancement coefficient.

7. The method according to claim 1, wherein The cross-attention calculation of the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, and identify the set of high-activation regions from the activation map, including: Use the preprocessed X-ray feature map as the query, the metal feature vector as the key and value, calculate the dot product similarity between the query and the key, apply the Softmax function to obtain the attention weights, and then multiply the attention weights by the values and sum them to generate a cross-attention activation map; Perform image segmentation on the activation map, and use the connected regions with activation values higher than the activation threshold as high-activation regions.

8. The method according to claim 1, characterized in that, Iteratively optimize the receptive field parameters based on the geometric features and activation intensity of the set of high-activation regions, including: Calculate the geometric features and average activation intensity of the high-activation regions. The geometric features include the centroid coordinates, major axis direction, major axis length, and minor axis length of the high-activation regions; Move the center positions of the M Gaussian kernels closest to the centroid of the high-activation region by a predetermined step; adjust the covariance matrix of the M Gaussian kernels to align the major axis direction of the Gaussian kernels with the major axis direction of the high-activation region, and make the size of the Gaussian kernels proportional to the major axis direction and major axis length; adjust the weights of the M Gaussian kernels according to the average activation intensity, and the increase in weight is proportional to the average activation intensity; The iteration process is executed for a preset number of times, or until the change amount of the receptive field parameters is less than the preset convergence threshold.

9. The method according to claim 1, characterized in that The feature stability and significance indicators of the attention maps in each round of the iteration process, including: For each high-activation region in the current round of the map, calculate the area change rate and position drift distance between each high-activation region in the current round of the map and the corresponding high-activation region in the previous round of the map; use the average value of the area change rates and the average value of the position drift distances of all corresponding high-activation regions as the stability indicator of the current round; Calculate the ratio of the average activation intensity of all high-activation regions in the current round of the activation map to the average activation intensity of the entire map, and use the ratio as the significance indicator of the current round; Use the average value of the stability indicators of all rounds as the feature stability value, and use the average value of the significance indicators of all rounds as the significance indicator value.

10. A data processing system for quality monitoring of a candy production line, characterized in that, Include: An initial unit for collecting metal detection data and X-ray detection data in candy production, projecting the metal detection data and X-ray detection data into a joint latent space, obtaining a prior probability of foreign object contamination based on the joint latent space, and determining parameters of a receptive field for X-ray depth feature extraction according to the prior probability; An optimization unit for calculating a metal feature vector of the metal detection data and the confidence of each vector component, extracting an X-ray feature map using the receptive field, performing weighted preprocessing on the X-ray feature map using the confidence, performing cross-attention calculation on the preprocessed X-ray feature map and the metal feature vector to obtain an activation map, identifying a set of highly activated regions from the activation map, and iteratively optimizing the receptive field parameters based on the geometric features and activation intensity of the set of highly activated regions; An identification unit for extracting X-ray features using the receptive field parameters obtained by iterative optimization, calculating a cross-modal association intensity through the activation map, and inputting the extracted X-ray features, the cross-modal association intensity, and the feature stability and significance indexes of each round of attention maps in the iterative process into a preset classifier to obtain the defect type and probability.

Images