Product detection system and method

Abstract

The present application provides a product detection system and method, which relates to the field of intelligent production. It uses artificial intelligence-based image recognition and processing technology to perform image segmentation and color histogram feature extraction of screw packing state detection images and reference state images that have passed screw packing state quality inspection, so as to automatically determine whether there is a screw product missing according to the product state semantic coefficient between the color histogram semantic aggregation representation features of the screw packing state detection image and the reference state image. In this way, the detection speed can be significantly improved to meet the needs of large-scale production lines. At the same time, the use of artificial intelligence algorithms to process the screw packing state not only reduces errors caused by human factors, but also detects tiny differences, thereby greatly improving the accuracy and intelligence level of detection.

Description

Technical Field

[0001] The present application relates to the field of intelligent production, and more specifically, to a product detection system and method. Background Art

[0002] In the field of precision manufacturing, such as the assembly of electronic equipment or mechanical parts, every detail may affect the performance and reliability of the final product. As a connector, the correct installation position, quantity and tightening degree of screws directly affect the structural integrity and functional stability of the product. Any missing or improper installation of a screw may cause product failure or even cause safety hazards. Therefore, the assembly quality inspection of screws has become an important link that cannot be ignored on the production line.

[0003] However, traditional screw assembly inspection usually relies on manual visual inspection, which is intuitive but has many shortcomings in practical applications. Specifically, manual inspection is inefficient, especially in large-scale production line environments. Manpower alone cannot meet the rapid production pace, leading to bottlenecks in the entire production process. In addition, manual inspection is easily affected by factors such as ambient light, work pressure, and personal emotions. Changes in these external conditions will directly affect the inspection results. For example, in low light conditions, even the most experienced operators may miss small problems; in a high-pressure work environment, operators may ignore certain details in their rush to complete the task, thus affecting the quality of inspection.

[0004] Therefore, an optimized product detection solution is needed. Summary of the invention

[0005] In order to solve the above-mentioned technical problems, the present application is proposed. The embodiment of the present application provides a product detection system and method, which adopts artificial intelligence-based image recognition and processing technology to perform image segmentation and color histogram feature extraction of screw packing state detection images and reference state images of screw packing state quality inspection, so as to automatically judge whether there is a screw product missing according to the product state semantic coefficient between the color histogram semantic aggregation representation features of the screw packing state detection image and the reference state image. In this way, the detection speed can be significantly improved to meet the needs of large-scale production lines. At the same time, the artificial intelligence algorithm is used to process the screw packing state, which not only reduces the errors caused by human factors, but also can detect small differences, thereby greatly improving the accuracy and intelligence level of detection.

[0006] According to one aspect of the present application, there is provided a product detection system, comprising:

[0007] The screw packing state detection image acquisition module is used to obtain the screw packing state detection image acquired by the camera;

[0008] The screw packing state reference image acquisition module is used to extract the screw packing state quality inspection qualified reference state image;

[0009] A detection reference state image block division module is used to divide the screw packing state detection image and the reference state image into blocks to obtain a sequence of detection state image blocks and a sequence of reference state image blocks;

[0010] A detection reference state color histogram calculation module, used to calculate the color histograms of the sequence of the detection state image blocks and the sequence of the reference state image blocks respectively to obtain a sequence of detection state color histograms and a sequence of reference state color histograms;

[0011] A detection reference state color semantic feature extraction module is used to pass the sequence of the detection state color histogram and the sequence of the reference state color histogram through a screw packing state feature extractor to obtain a sequence of detection state color histogram semantic feature vectors and a sequence of reference state color histogram semantic feature vectors;

[0012] A detection reference state color feature semantic aggregation module is used to obtain a detection state color histogram semantic aggregation representation vector and a reference state color histogram semantic aggregation representation vector by passing the sequence of the detection state color histogram semantic feature vector and the sequence of the reference state color histogram semantic feature vector through a node aggregation network guided by feature significance;

[0013] A product status semantic Bhattacharyya coefficient calculation module, used to calculate the product status semantic Bhattacharyya coefficient between the detection status color histogram semantic aggregation representation vector and the reference status color histogram semantic aggregation representation vector;

[0014] The screw packing status detection result generating module is used to determine the screw packing status detection result based on the product status semantic Bhattacharyya coefficient, and the screw packing status detection result is used to indicate whether there is any screw product missing.

[0015] According to another aspect of the present application, a product detection method is provided, comprising:

[0016] Obtain the screw packing status detection image collected by the camera;

[0017] Extract the reference state image of the screw packaging state that has passed the quality inspection;

[0018] Separately dividing the screw packing state detection image and the reference state image into image blocks to obtain a sequence of detection state image blocks and a sequence of reference state image blocks;

[0019] respectively calculating the color histograms of the sequence of the detection state image blocks and the sequence of the reference state image blocks to obtain a sequence of detection state color histograms and a sequence of reference state color histograms;

[0020] Passing the sequence of the detection state color histogram and the sequence of the reference state color histogram through a screw packing state feature extractor to obtain a sequence of detection state color histogram semantic feature vectors and a sequence of reference state color histogram semantic feature vectors;

[0021] Passing the sequence of the detection state color histogram semantic feature vectors and the sequence of the reference state color histogram semantic feature vectors through a node aggregation network guided by feature significance to obtain a detection state color histogram semantic aggregation representation vector and a reference state color histogram semantic aggregation representation vector;

[0022] Calculating the product state semantic Bhattacharyya coefficient between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector;

[0023] Based on the semantic Bhattacharyya coefficient of the product status, a screw packing status detection result is determined, and the screw packing status detection result is used to indicate whether there is any screw product missing.

[0024] Compared with the prior art, the product detection system and method provided by the present application adopts the image recognition and processing technology based on artificial intelligence to perform image segmentation and color histogram feature extraction of the screw packing state detection image and the reference state image of the screw packing state quality inspection, so as to automatically judge whether there is a screw product missing according to the product state semantic coefficient between the color histogram semantic aggregation representation features of the screw packing state detection image and the reference state image. In this way, the detection speed can be significantly improved to meet the needs of large-scale production lines. At the same time, the use of artificial intelligence algorithm for screw packing state processing not only reduces errors caused by human factors, but also detects tiny differences, thereby greatly improving the accuracy and intelligence level of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0026] Figure 1 4 is a system block diagram of a product detection system according to an embodiment of the present application.

[0027] Figure 2 Schematic diagram of data flow of a product detection system according to an embodiment of the present application.

[0028] Figure 3 It is a block diagram of a detection reference state color feature semantic aggregation module in a product detection system according to an embodiment of the present application.

[0029] Figure 4 Flow chart of a product detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0031] In the field of precision manufacturing, such as the assembly of electronic equipment and mechanical parts, every detail plays a key role in the performance and reliability of the final product. Among them, the screws as connectors, whether their installation position is correct, the number is accurate, and the degree of tightening are directly related to the structural integrity and functional stability of the product. Once the screws are missing or improperly installed, it is very likely to cause product failure and even bring safety hazards. Therefore, the assembly quality of screws is a crucial link in the production line and cannot be ignored.

[0032] However, traditional screw assembly inspection often relies on manual visual inspection. Although this inspection method is relatively intuitive, it has many disadvantages in actual application. Specifically, manual inspection is not efficient, especially in the environment of large-scale production lines. Relying solely on manpower cannot meet the requirements of rapid production, and thus becomes a bottleneck in the entire production process. Moreover, manual inspection is easily affected by factors such as ambient light, work pressure, and personal emotions. Once these external conditions change, they will directly affect the inspection effect. For example, when the light is poor, even the most experienced operator may overlook some minor problems; and in a high-pressure working environment, the operator may ignore certain details in a hurry to complete the task, which ultimately affects the quality of the inspection.

[0033] Therefore, in view of the above technical problems, the technical concept of the present application is to collect the screw packing state detection image by a camera, extract the reference state image of the screw packing state that has passed the quality inspection, and use the image recognition and processing technology based on artificial intelligence to perform image segmentation and color histogram feature extraction of the screw packing state detection image and the reference state image, so as to automatically judge whether there is a screw product missing according to the product state semantic coefficient between the color histogram semantic aggregation representation features of the screw packing state detection image and the reference state image. In this way, a large number of screw packing states can be quickly detected, significantly improving the detection speed and meeting the needs of large-scale production lines. At the same time, the use of artificial intelligence algorithms to process the screw packing state not only reduces errors caused by human factors, but also detects small differences, thereby greatly improving the accuracy and intelligence level of detection.

[0034] Figure 1 4 is a system block diagram of a product detection system according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of a product detection system according to an embodiment of the present application. Figure 1 and Figure 2As shown, in the product inspection system 100, it includes: a screw packing state detection image acquisition module 110, which is used to obtain the screw packing state detection image collected by the camera; a screw packing state reference image acquisition module 120, which is used to extract the reference state image of the screw packing state that passes the quality inspection; a detection reference state image blocking module 130, which is used to respectively block the screw packing state detection image and the reference state image to obtain a sequence of detection state image blocks and a sequence of reference state image blocks; a detection reference state color histogram calculation module 140, which is used to respectively calculate the color histogram of the sequence of the detection state image blocks and the sequence of the reference state image blocks to obtain a sequence of detection state color histograms and a sequence of reference state color histograms; a detection reference state color semantic feature extraction module 150, which is used to extract the sequence of the detection state color histograms and the sequence of the reference state color histograms through the screw packing state A feature extractor is used to obtain a sequence of detection state color histogram semantic feature vectors and a sequence of reference state color histogram semantic feature vectors; a detection reference state color feature semantic aggregation module 160 is used to obtain a detection state color histogram semantic aggregation representation vector and a reference state color histogram semantic aggregation representation vector by passing the sequence of detection state color histogram semantic feature vectors and the sequence of reference state color histogram semantic feature vectors through a node aggregation network guided by feature significance; a product state semantic Bhattacharyya coefficient calculation module 170 is used to calculate the product state semantic Bhattacharyya coefficient between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector; a screw packing state detection result generation module 180 is used to determine the screw packing state detection result based on the product state semantic Bhattacharyya coefficient, and the screw packing state detection result is used to indicate whether there is any screw product missing.

[0035] In an embodiment of the present application, the screw packing state detection image acquisition module 110 and the screw packing state reference image acquisition module 120 are respectively used to obtain the screw packing state detection image acquired by the camera and the reference state image for extracting the screw packing state quality inspection. It should be understood that the screw packing state detection image usually contains information such as the number, position, arrangement state, and integrity of screws in actual production, while the reference state image of the screw packing state quality inspection contains information such as the number, position, arrangement state, and integrity of standard screws. By extracting and analyzing the information in the screw packing state detection image and the reference state image of the screw packing state quality inspection, the missing of screw products can be effectively identified, thereby improving the quality control of the production line.

[0036] In an embodiment of the present application, the detection reference state image block module 130 is used to perform image block division on the screw packing state detection image and the reference state image respectively to obtain a sequence of detection state image blocks and a sequence of reference state image blocks. Specifically, in an embodiment of the present application, the detection reference state image block module is used to: pass the screw packing state detection image and the reference state image through an image blocker to obtain a sequence of detection state image blocks and a sequence of reference state image blocks. It should be understood that different local features in the screw packing state detection image and the reference state image have different state feature information. Therefore, in order to more accurately identify and analyze the state of each area, so as to more carefully analyze and process specific parts in the image, in the technical solution of the present application, the screw packing state detection image and the reference state image are passed through an image blocker to obtain a sequence of detection state image blocks and a sequence of reference state image blocks. In this way, a large image can be decomposed into multiple small blocks, and each image block can represent an area in the image, thereby helping to extract local feature details and improve the accuracy of image recognition.

[0037] In an embodiment of the present application, the detection reference state color histogram calculation module 140 is used to calculate the color histogram of the sequence of the detection state image blocks and the sequence of the reference state image blocks respectively to obtain the sequence of the detection state color histogram and the sequence of the reference state color histogram. Accordingly, considering that in the screw packing state detection, the color information of the screws in each area is crucial for the judgment of the packing state, and the color histogram is a common image feature representation method, which can effectively extract the color distribution information of the image. Based on this, in the technical solution of the present application, the color histogram of the sequence of the detection state image blocks and the sequence of the reference state image blocks are calculated respectively to obtain the sequence of the detection state color histogram and the sequence of the reference state color histogram. That is, the color histogram can help distinguish screws from the background, or distinguish screws of different colors, so as to judge whether the packing state meets the requirements.

[0038] In an embodiment of the present application, the detection reference state color semantic feature extraction module 150 is used to pass the sequence of the detection state color histograms and the sequence of the reference state color histograms through a screw packing state feature extractor to obtain a sequence of detection state color histogram semantic feature vectors and a sequence of reference state color histogram semantic feature vectors. Specifically, in an embodiment of the present application, the detection reference state color semantic feature extraction module is used to: pass the sequence of the detection state color histograms and the sequence of the reference state color histograms through a screw packing state feature extractor based on a deep convolutional neural network model to obtain a sequence of the detection state color histogram semantic feature vectors and a sequence of the reference state color histogram semantic feature vectors. It should be understood that in order to further extract and mine the color feature information of each detection state color histogram and the reference state color histogram, in the technical solution of the present application, the sequence of the detection state color histogram and the sequence of the reference state color histogram are passed through a screw packing state feature extractor based on a hole convolutional neural network model to obtain a sequence of detection state color histogram semantic feature vectors and a sequence of reference state color histogram semantic feature vectors. That is, the hole convolution allows the network to capture a wider range of contextual color information in the detection and reference state histograms without increasing the number of parameters, thereby improving the accuracy of subsequent processing and analysis.

[0039] In an embodiment of the present application, the detection reference state color feature semantic aggregation module 160 is used to obtain a detection state color histogram semantic aggregation representation vector and a reference state color histogram semantic aggregation representation vector by passing the sequence of the detection state color histogram semantic feature vector and the sequence of the reference state color histogram semantic feature vector through a node aggregation network guided by feature significance. Accordingly, considering that the importance of the sequence of the detection state color histogram semantic feature vector and the sequence of the reference state color histogram semantic feature vector in different local areas is different, and in order to help the model focus on the most informative part of the image, so as to identify and extract the most critical color semantic features of the task, and integrate them into a global feature representation, in the technical solution of the present application, the sequence of the detection state color histogram semantic feature vector and the sequence of the reference state color histogram semantic feature vector are extracted for significance and aggregated in the whole image domain through a node aggregation network guided by feature significance to obtain a detection state color histogram semantic aggregation representation vector and a reference state color histogram semantic aggregation representation vector.

[0040] Here, the sequence of the semantic feature vectors of the color histogram of the detection state is used for explanation. Specifically, first, the feature significance description factor of each semantic feature of the color histogram of the detection state is calculated. In particular, the feature significance description factor is calculated based on the mean and standard deviation of each feature, which can identify and highlight the most critical color semantic features, and provide a basis for subsequent feature significance enhancement and aggregation. Then, the feature significance attenuation factor of each feature vector is calculated based on the distance span between each semantic feature vector of the color histogram of the detection state and the current semantic feature vector of the color histogram of the detection state, so that the contribution of each feature vector in the model can be dynamically adjusted, so that the model can pay more attention to those features with a closer semantic distance in space to the current feature and obtain a higher weight value. Subsequently, the feature significance attenuation factor of each feature is weighted modulated to the feature significance description factor to capture the key color semantic changes, so that the features with more consistent or close features of the current node will be given a greater weight, while other features will be weakened accordingly, and the semantic feature significance attenuation description factor of the color histogram of the detection state of each feature is obtained. Then, the saliency decay descriptor of each modulated detection state color histogram semantic feature is input into the gated mask module to screen the modulated saliency decay descriptor, so as to dynamically adjust the weight of the feature, retain those features that are most useful for the current task, and suppress or ignore those unimportant features. Finally, the weighted sum of the sequence of detection state color histogram semantic feature vectors is calculated with the masked saliency descriptor as the weight to obtain the detection state color histogram semantic aggregate representation vector, so that the features that are considered more important by the mask module can contribute more to the final aggregate representation vector, thereby highlighting the importance of these features in the detection state, so as to more comprehensively reflect the color histogram semantic information of the detection state.

[0041] Specifically, Figure 3 FIG. 4 is a block diagram of a detection reference state color feature semantic aggregation module in a product detection system according to an embodiment of the present application. Figure 3As shown, the detection reference state color feature semantic aggregation module 160 includes: a detection state color histogram semantic feature significance description factor calculation unit 161, which is used to calculate the feature significance description factor of each detection state color histogram semantic feature vector in the sequence of the detection state color histogram semantic feature vector, wherein the feature significance description factor is related to the mean and variance of each detection state color histogram semantic feature vector; a detection state color histogram semantic feature significance attenuation factor construction unit 162, which is used to construct the feature significance attenuation factor of each detection state color histogram semantic feature vector based on the distance span between each detection state color histogram semantic feature vector in the sequence of the detection state color histogram semantic feature vector and the current detection state color histogram semantic feature vector; a detection state color histogram semantic feature significance description factor modulation unit 161; Element 163 is used to modulate the feature significance description factor of each detection state color histogram semantic feature vector based on the feature significance attenuation factor of each detection state color histogram semantic feature vector to obtain a sequence of detection state color histogram semantic feature significance attenuation description factors; a detection state color histogram semantic feature significance attenuation factor masking unit 164 is used to input the sequence of detection state color histogram semantic feature significance attenuation description factors into a gated masking module to obtain a sequence of detection state color histogram semantic feature significance attenuation weight factors; a detection state color histogram semantic aggregate representation generating unit 165 is used to calculate the weighted sum of the sequence of detection state color histogram semantic feature vectors using the sequence of detection state color histogram semantic feature significance attenuation weight factors as a sequence of weights to obtain the detection state color histogram semantic aggregate representation vector.

[0042] More specifically, in an embodiment of the present application, the detection state color histogram semantic feature significance description factor calculation unit is used to: respectively calculate the mean and variance of the detection state color histogram semantic feature vector to obtain the detection state color histogram semantic feature mean and the detection state color histogram semantic feature variance; calculate the positional difference between the detection state color histogram semantic feature vector and the detection state color histogram semantic feature mean to obtain the detection state color histogram semantic difference vector; calculate the fourth power of each eigenvalue in the detection state color histogram semantic difference vector to obtain the detection state color histogram semantic modulation difference vector; calculate the expected value of the detection state color histogram semantic modulation difference vector to obtain the detection state color histogram expected value; divide the detection state color histogram expected value by the square of the detection state color histogram semantic feature variance to obtain the feature significance description factor corresponding to the detection state color histogram semantic feature vector.

[0043] More specifically, in an embodiment of the present application, the detection state color histogram semantic feature significance attenuation factor construction unit is used to: extract the maximum value in each detection state color histogram semantic feature vector to obtain a sequence of detection state color histogram semantic feature maximum values; extract the maximum value of the current detection state color histogram semantic feature vector to obtain the current detection state color histogram semantic coding feature maximum value; calculate the positional subtraction between the sequence of the detection state color histogram semantic feature maximum values ​​and the current detection state color histogram semantic coding feature maximum value to obtain the detection state color histogram semantic feature offset value. sequence; counting the distance span values ​​between the semantic feature vectors of each detection state color histogram and the semantic feature vector of the current detection state color histogram to obtain a sequence of semantic distance span values ​​of the detection state color histogram, and calculating the squares of each value in the sequence of semantic distance span values ​​of the detection state color histogram to obtain a sequence of semantic distance span modulation values ​​of the detection state color histogram; dividing the sequence of semantic feature offset values ​​of the detection state color histogram by the sequence of semantic distance span modulation values ​​of the detection state color histogram by position to obtain feature significance attenuation factors of the semantic feature vectors of each detection state color histogram.

[0044] More specifically, in an embodiment of the present application, the detection state color histogram semantic feature significance attenuation factor mask unit is used to: perform normalization processing based on the Sigmoid function on each detection state color histogram semantic feature significance attenuation description factor in the sequence of the detection state color histogram semantic feature significance attenuation description factors to obtain a set of normalized detection state color histogram semantic feature significance attenuation description factors; input each normalized detection state color histogram semantic feature significance attenuation description factor in the set of normalized detection state color histogram semantic feature significance attenuation description factors into a gating function for masking processing to obtain a sequence of detection state color histogram semantic feature significance attenuation weight factors.

[0045] In the embodiment of the present application, specifically, the detection reference state color feature semantic aggregation module is used to: process the sequence of the detection state color histogram semantic feature vector through a node aggregation network guided by feature significance, using the following feature aggregation formula to obtain the detection state color histogram semantic aggregation representation vector; wherein the feature aggregation formula is: ;in, represents a sequence of semantic feature vectors of the detection state color histogram, , and are the first , and detection state color histogram semantic feature vector, To extract the maximum value of a vector, express and The distance span between For the The feature significance attenuation factor of the semantic feature vector of the color histogram of the detection state, It is The semantic feature vector of the color histogram of the detection state The eigenvalues ​​at the positions, and They are The mean and square of the variance of the semantic feature vector of the color histogram of each detection state, To calculate the expected value, It is The detection state color histogram semantic feature vector corresponding to the detection state color histogram semantic feature significance attenuation description factor, yes function, is the first in the set of semantic feature significance attenuation description factors of the normalized detection state color histogram Normalized detection state color histogram semantic feature significant attenuation description factor, For masking, is the first in the sequence of the semantic feature significance attenuation weight factors of the detection state color histogram The detection state color histogram semantic feature significance attenuation weight factor, is the predetermined threshold, is the detection state color histogram semantic aggregation representation vector. Similarly, the encoding method of the sequence of the reference state color histogram semantic feature vector is also as shown in the above formula.

[0046] In an embodiment of the present application, the product status semantic Bhattacharyya coefficient calculation module 170 is used to calculate the product status semantic Bhattacharyya coefficient between the detection status color histogram semantic aggregation representation vector and the reference status color histogram semantic aggregation representation vector. It should be understood that in order to compare and analyze the detection status color features based on the color features of the reference state, so as to measure the color semantic similarity between the two, in the technical solution of the present application, the product status semantic Bhattacharyya coefficient between the detection status color histogram semantic aggregation representation vector and the reference status color histogram semantic aggregation representation vector is calculated. It should be understood that the Bhattacharyya coefficient can quantify the similarity between two probability distributions. By calculating the Bhattacharyya coefficient of the color histogram semantic aggregation representation vector of the detection state and the reference state, the color feature difference between the two can be quantitatively evaluated, which provides a quantitative basis for judging the packing state of the screws.

[0047] In an embodiment of the present application, the screw packing state detection result generation module 180 is used to determine the screw packing state detection result based on the product state semantic Barthel coefficient, and the screw packing state detection result is used to indicate whether there is a screw product missing. That is, the product state semantic Barthel coefficient calculated between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector is used to automatically determine whether there is a screw product missing. In this way, a large number of screw packing states can be quickly detected, significantly improving the detection speed and meeting the needs of large-scale production lines. At the same time, the screw packing state processing is performed using an artificial intelligence algorithm, which not only reduces errors caused by human factors, but also detects minor differences, thereby greatly improving the accuracy and intelligence level of the detection. In particular, the detection result can be obtained based on the comparison between the product state semantic Barthel coefficient and a predetermined threshold.

[0048] Preferably, considering that the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector respectively represent the global image semantic space domain aggregated image semantic features of the screw packing state detection image and the reference state image based on the local image semantic feature space domain distribution energy significance, therefore, when calculating the product state semantic Bhattacharyya coefficient therebetween to map to a common image color feature similarity metric space, there will also be image semantic feature aggregation alignment conflicts caused by the feature distribution differences of the source image semantics in the local-global image semantic space domain, thereby causing the loss of aggregation key information and affecting the calculation accuracy of the product state semantic Bhattacharyya coefficient between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector.

[0049] Based on this, in the preferred example, when calculating the product state semantic Bhattacharyya coefficient between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector, the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector are jointly optimized. Specifically, the optimization process includes the following steps: cascading the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector into a detection state-reference state color histogram semantic aggregation cascade vector; performing an optimization on all eigenvalues ​​of the detection state-reference state color histogram semantic aggregation cascade vector based on the distance between eigenvalues, such as the absolute value distance difference. The method comprises the steps of: dividing the first detection state-reference state color histogram semantic aggregation clustering vector into the first power value of the first detection state-reference state color histogram semantic aggregation clustering vector, dividing the first power value of the first detection state-reference state color histogram semantic aggregation clustering vector by the second power value of the first detection state-reference state color histogram semantic aggregation clustering vector; and dividing the first power value of the first detection state-reference state color histogram semantic aggregation clustering vector by the first power value of the first detection state-reference state color histogram semantic aggregation clustering vector. a norm of which is raised to a second power value with the ratio value as an exponent to obtain a second detection state-reference state color histogram semantic aggregation clustering related weight value; for each eigenvalue of the detection state-reference state color histogram semantic aggregation clustering vector, multiply it by the inverse of the difference between the first detection state-reference state color histogram semantic aggregation clustering related weight value and the second detection state-reference state color histogram semantic aggregation clustering related weight value to obtain the optimized eigenvalue of the detection state-reference state color histogram semantic aggregation clustering vector; for each eigenvalue other than the cluster in the detection state-reference state color histogram semantic aggregation cascade vector, multiply it by the first detection state-reference state color histogram semantic aggregation clustering related weight value The inverse of the sum of the weight value related to the aggregation clustering and the weight value related to the semantic aggregation clustering of the second detection state-reference state color histogram is obtained to obtain the optimized out-of-class feature value of the detection state-reference state color histogram semantic aggregation cascade vector; the optimized feature value of the detection state-reference state color histogram semantic aggregation clustering vector and the optimized out-of-class feature value of the detection state-reference state color histogram semantic aggregation cascade vector are combined into an optimized detection state-reference state color histogram semantic aggregation cascade vector; the optimized detection state-reference state color histogram semantic aggregation cascade vector is split into an optimized detection state color histogram semantic aggregation representation vector and an optimized reference state color histogram semantic aggregation representation vector.

[0050] That is, the optimization process of the detection state-reference state color histogram semantic aggregation cascade vector is expressed as: ;in, For each eigenvalue of the detection state-reference state color histogram semantic aggregation clustering vector, is the detection state-reference state color histogram semantic aggregation cascade vector, is the detection state-reference state color histogram semantic aggregation clustering vector, is the number of features of the detection state-reference state color histogram semantic aggregation cascade vector, is the number of features of the semantic aggregation clustering vector of the detection state-reference state color histogram, is the ratio of the number of features of the detection state-reference state color histogram semantic aggregation clustering vector to the number of features of the detection state-reference state color histogram semantic aggregation cascade vector, represents the clustering feature set corresponding to the semantic aggregation clustering vector of the detection state-reference state color histogram, and Represents the binary norm and the uninorm of the vector respectively. Power, is the optimized detection state-reference state color histogram semantic aggregation cascade vector.

[0051] Here, in order to avoid the loss of key suffix semantic information relative to the original feature set as a whole due to aggregation conflict between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector based on aggregation features, the clustering ratio of the number of features of the detection state-reference state color histogram semantic aggregation clustering vector relative to the number of features of the detection state-reference state color histogram semantic aggregation cascade vector is used as a decision function to perform an adversarial decision on the set absolute representation of the one-norm of the detection state-reference state color histogram semantic aggregation clustering vector and the detection state-reference state color histogram semantic aggregation cascade vector, and the clustering ratio is used with the detection state-reference state color histogram semantic aggregation clustering vector as a decision function. The clustering intrinsic conflict representation of the two norms of the graph semantic aggregation clustering vector and the detection state-reference state color histogram semantic aggregation cascade vector is positively and negatively interacted to construct the optimized detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector based on the aggregation feature and the solid alignment guardrail of the original feature set as a whole, so as to achieve the mitigation of the intention of harmful information loss based on the aggregation risk transferability of the optimized detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector, and improve the calculation accuracy of the product state semantic Bhattacharyya coefficient between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector. In this way, a large number of screw packing states can be quickly detected, the detection speed can be significantly improved, and the needs of large-scale production lines can be met. At the same time, the artificial intelligence algorithm is used to process the screw packing state, which not only reduces the errors caused by human factors, but also detects small differences, thereby greatly improving the accuracy and intelligence level of detection.

[0052] In summary, the product detection system 100 based on the embodiment of the present application is explained, which uses artificial intelligence-based image recognition and processing technology to perform image segmentation and color histogram feature extraction of the screw packing state detection image and the reference state image of the screw packing state quality inspection, so as to automatically judge whether there is a screw product missing according to the product state semantic coefficient between the color histogram semantic aggregation representation features of the screw packing state detection image and the reference state image. In this way, the detection speed can be significantly improved to meet the needs of large-scale production lines. At the same time, the use of artificial intelligence algorithms for screw packing state processing not only reduces errors caused by human factors, but also detects small differences, thereby greatly improving the accuracy and intelligence level of detection.

[0053] As described above, the product detection system 100 according to the embodiment of the present application can be implemented in various terminal devices, such as a server for product detection. In one example, the product detection system 100 according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the product detection system 100 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the product detection system 100 can also be one of the many hardware modules of the terminal device.

[0054] Alternatively, in another example, the product detection system 100 and the terminal device may also be separate devices, and the product detection system 100 may be connected to the terminal device via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0055] In other embodiments of the present application, a product detection system is also provided, which is used to detect the placement and direction of parts. In addition, it is also used to detect the verticality of screw installation and whether it is missing. Specifically, the placement position and angle of the part housing can be detected by color extraction and polygon comparison; the verticality of the screw installation and whether it is missing can be detected by a 90-degree vertical light source. In a specific implementation of other embodiments, the polygon comparison function is used to locate the terminal block, and then multiple area functions are used to detect whether iron parts exist. It is worth noting that in order to maximize the luminescence of iron parts, a 90-degree vertical light source must be used. When using three area functions, when there are iron parts in the terminal block, a certain degree of white area can be detected; when there are no iron parts in the terminal block, the three area functions can only detect a small amount of white area, thereby realizing the detection of iron parts.

[0056] Figure 4 FIG. 1 is a flow chart of a product detection method according to an embodiment of the present application. Figure 4As shown, in the product detection method, it includes: S110, obtaining a screw packing state detection image collected by a camera; S120, extracting a reference state image of the screw packing state that has passed the quality inspection; S130, respectively performing image segmentation on the screw packing state detection image and the reference state image to obtain a sequence of detection state image blocks and a sequence of reference state image blocks; S140, respectively calculating the color histograms of the sequence of the detection state image blocks and the sequence of the reference state image blocks to obtain a sequence of detection state color histograms and a sequence of reference state color histograms; S150, passing the sequence of the detection state color histograms and the sequence of the reference state color histograms through a screw packing state feature extractor to obtain a detection state color histogram. S160, passing the sequence of detection state color histogram semantic feature vectors and the sequence of reference state color histogram semantic feature vectors through a node aggregation network guided by feature significance to obtain a detection state color histogram semantic aggregation representation vector and a reference state color histogram semantic aggregation representation vector; S170, calculating the product state semantic Bhattacharyya coefficient between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector; S180, determining the screw packing state detection result based on the product state semantic Bhattacharyya coefficient, and the screw packing state detection result is used to indicate whether there is any screw product missing.

[0057] Here, those skilled in the art will appreciate that the specific operations of each step in the above product detection method have been described in detail above. Figures 1 to 3 The product detection system has been described in detail, and therefore, its repeated description will be omitted.

[0058] In summary, the product detection method based on the embodiment of the present application is explained, which uses artificial intelligence-based image recognition and processing technology to perform image segmentation and color histogram feature extraction of the screw packing state detection image and the reference state image of the screw packing state quality inspection, so as to automatically judge whether there is a screw product missing according to the product state semantic coefficient between the color histogram semantic aggregation representation features of the screw packing state detection image and the reference state image. In this way, the detection speed can be significantly improved to meet the needs of large-scale production lines. At the same time, the use of artificial intelligence algorithms for screw packing state processing not only reduces errors caused by human factors, but also detects small differences, thereby greatly improving the accuracy and intelligence level of detection.

[0059] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details of the above application are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present application to the use of the above specific details to be implemented.

Claims

1. A product detection system, characterized in that: include: The screw packing state detection image acquisition module is used to obtain the screw packing state detection image acquired by the camera; The screw packing state reference image acquisition module is used to extract the screw packing state quality inspection qualified reference state image; A detection reference state image block division module is used to divide the screw packing state detection image and the reference state image into blocks to obtain a sequence of detection state image blocks and a sequence of reference state image blocks; A detection reference state color histogram calculation module, used to calculate the color histograms of the sequence of the detection state image blocks and the sequence of the reference state image blocks respectively to obtain a sequence of detection state color histograms and a sequence of reference state color histograms; A detection reference state color semantic feature extraction module is used to pass the sequence of the detection state color histogram and the sequence of the reference state color histogram through a screw packing state feature extractor to obtain a sequence of detection state color histogram semantic feature vectors and a sequence of reference state color histogram semantic feature vectors; A detection reference state color feature semantic aggregation module is used to obtain a detection state color histogram semantic aggregation representation vector and a reference state color histogram semantic aggregation representation vector by passing the sequence of the detection state color histogram semantic feature vector and the sequence of the reference state color histogram semantic feature vector through a node aggregation network guided by feature significance; A product status semantic Bhattacharyya coefficient calculation module, used to calculate the product status semantic Bhattacharyya coefficient between the detection status color histogram semantic aggregation representation vector and the reference status color histogram semantic aggregation representation vector; A screw packing status detection result generating module, used for determining a screw packing status detection result based on the product status semantic Barthel coefficient, wherein the screw packing status detection result is used for indicating whether there is a screw product missing; The detection reference state color feature semantic aggregation module includes: A detection state color histogram semantic feature significance description factor calculation unit, used to calculate a feature significance description factor of each detection state color histogram semantic feature vector in the sequence of the detection state color histogram semantic feature vectors, wherein the feature significance description factor is related to the mean and variance of each detection state color histogram semantic feature vector; A detection state color histogram semantic feature significance attenuation factor construction unit, used to construct a feature significance attenuation factor of each detection state color histogram semantic feature vector based on the distance span between each detection state color histogram semantic feature vector in the sequence of the detection state color histogram semantic feature vector and the current detection state color histogram semantic feature vector; A detection state color histogram semantic feature significance description factor modulation unit, used for modulating the feature significance description factor of each detection state color histogram semantic feature vector based on the feature significance attenuation factor of each detection state color histogram semantic feature vector to obtain a sequence of detection state color histogram semantic feature significance attenuation description factors; A detection state color histogram semantic feature significance attenuation factor mask unit, used for inputting the sequence of detection state color histogram semantic feature significance attenuation description factors into a gated mask module to obtain a sequence of detection state color histogram semantic feature significance attenuation weight factors; The detection state color histogram semantic aggregation representation generation unit is used to calculate the weighted sum of the sequence of the detection state color histogram semantic feature vectors using the sequence of the detection state color histogram semantic feature significance attenuation weight factors as the sequence of weights to obtain the detection state color histogram semantic aggregation representation vector.

2. The product detection system according to claim 1, characterized in that: The detection reference state image block division module is used to: pass the screw packing state detection image and the reference state image through an image block divider to obtain a sequence of the detection state image blocks and a sequence of the reference state image blocks.

3. The product detection system according to claim 2, characterized in that: The detection reference state color semantic feature extraction module is used to: pass the sequence of the detection state color histogram and the sequence of the reference state color histogram through a screw packing state feature extractor based on a deep convolutional neural network model to obtain a sequence of the detection state color histogram semantic feature vectors and a sequence of the reference state color histogram semantic feature vectors.

4. The product detection system according to claim 3, characterized in that: The screw packing state feature extractor based on the deep convolutional neural network model is a screw packing state feature extractor based on the void convolutional neural network model.

5. The product detection system according to claim 4, characterized in that: The detection state color histogram semantic feature significance description factor calculation unit is used to: Respectively calculating the mean and variance of the detection state color histogram semantic feature vector to obtain the detection state color histogram semantic feature mean and the detection state color histogram semantic feature variance; Calculating the positional difference between the detection state color histogram semantic feature vector and the detection state color histogram semantic feature mean to obtain a detection state color histogram semantic difference vector; Calculating the fourth power of each eigenvalue in the detection state color histogram semantic difference vector to obtain a detection state color histogram semantic modulation difference vector; Calculating the expected value of the semantic modulation difference vector of the detection state color histogram to obtain the expected value of the detection state color histogram; The expected value of the detection state color histogram is divided by the square of the semantic feature variance of the detection state color histogram to obtain a feature significance description factor corresponding to the semantic feature vector of the detection state color histogram.

6. The product detection system according to claim 5, characterized in that: The detection state color histogram semantic feature significance attenuation factor construction unit is used to: Extracting the maximum values ​​in the semantic feature vectors of the color histograms of the detection states to obtain a sequence of maximum values ​​of the semantic features of the color histograms of the detection states; Extracting the maximum value of the semantic feature vector of the current detection state color histogram to obtain the maximum value of the semantic coding feature of the current detection state color histogram; Calculate the positional subtraction between the sequence of maximum values ​​of the semantic features of the detection state color histogram and the maximum value of the semantic coding feature of the current detection state color histogram to obtain a sequence of offset values ​​of the semantic features of the detection state color histogram; Counting the distance span values ​​between each detection state color histogram semantic feature vector and the current detection state color histogram semantic feature vector to obtain a sequence of detection state color histogram semantic distance span values, and calculating the square of each value in the sequence of detection state color histogram semantic distance span values ​​to obtain a sequence of detection state color histogram semantic distance span modulation values; The sequence of the detection state color histogram semantic feature offset values ​​and the sequence of the detection state color histogram semantic distance span modulation values ​​are divided by position to obtain feature significance attenuation factors of the semantic feature vectors of each detection state color histogram.

7. The product detection system according to claim 6, characterized in that: The detection state color histogram semantic feature significance attenuation factor mask unit is used to: Performing a normalization process based on a Sigmoid function on each detection state color histogram semantic feature significant attenuation description factor in the sequence of the detection state color histogram semantic feature significant attenuation description factors to obtain a set of normalized detection state color histogram semantic feature significant attenuation description factors; Each normalized detection state color histogram semantic feature significance attenuation description factor in the set of normalized detection state color histogram semantic feature significance attenuation description factors is input into a gating function for masking processing to obtain a sequence of detection state color histogram semantic feature significance attenuation weight factors.

8. A product detection method, using the product detection system according to claim 1, characterized in that: include: Obtain the screw packing status detection image collected by the camera; Extract the reference state image of the screw packaging state that has passed the quality inspection; Separately dividing the screw packing state detection image and the reference state image into image blocks to obtain a sequence of detection state image blocks and a sequence of reference state image blocks; respectively calculating the color histograms of the sequence of the detection state image blocks and the sequence of the reference state image blocks to obtain a sequence of detection state color histograms and a sequence of reference state color histograms; Passing the sequence of the detection state color histogram and the sequence of the reference state color histogram through a screw packing state feature extractor to obtain a sequence of detection state color histogram semantic feature vectors and a sequence of reference state color histogram semantic feature vectors; Passing the sequence of the detection state color histogram semantic feature vectors and the sequence of the reference state color histogram semantic feature vectors through a node aggregation network guided by feature significance to obtain a detection state color histogram semantic aggregation representation vector and a reference state color histogram semantic aggregation representation vector; Calculating the product state semantic Bhattacharyya coefficient between the detection state color histogram semantic aggregation representation vector and the reference state color histogram semantic aggregation representation vector; Based on the semantic Bhattacharyya coefficient of the product status, a screw packing status detection result is determined, and the screw packing status detection result is used to indicate whether there is any screw product missing.

9. The product detection method according to claim 8, characterized in that: The screw packing state detection image and the reference state image are respectively divided into blocks to obtain a sequence of detection state image blocks and a sequence of reference state image blocks, including: passing the screw packing state detection image and the reference state image through an image divider to obtain a sequence of detection state image blocks and a sequence of reference state image blocks.

Images