AI visual inspection method for bagged objects
The AI visual detection method for bagged items uses image preprocessing and a CNN model to address the challenge of dual training for bagged and unbagged items, achieving efficient and accurate identification through a single training process.
Patent Information
- Application Number
- CN202411191273.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-12-06
AI Technical Summary
Existing AI recognition systems for smart scales struggle with accurately identifying bagged items due to changes in appearance characteristics and color distribution, necessitating separate and time-consuming training processes for bagged and unbagged items, increasing complexity and cost.
An AI visual detection method involving image preprocessing techniques such as color stripping and bag shape processing, combined with a convolutional neural network (CNN) model, to extract and enhance image features, effectively handling bagged items in a single training process.
This method reduces training time and cost, improves recognition accuracy and efficiency, and enhances the stability of smart scales by maintaining high recognition accuracy even with varying bag shapes, colors, and lighting conditions.
Smart Images

Figure CN119090845A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of visual inspection technology, and in particular to an AI visual inspection method for bagged objects. Background Art
[0002] In the rapid development of the modern business environment, AI recognition equipment is gradually penetrating into various industries, especially in retail and supply chain management, showing great potential and application value. As an important place for daily consumption, supermarkets have become the focus of industry attention in terms of operational efficiency and customer experience. Among them, smart scales, as an important part of the supermarket self-checkout system, integrate advanced AI recognition technology to achieve fast and accurate identification of goods, and automatically print price tags, greatly improving checkout speed and accuracy.
[0003] The core technology of the smart scale lies in its built-in AI recognition system, which relies on high-precision cameras and powerful image processing capabilities. When a customer places an item on the scale, the camera quickly captures the image of the item and performs in-depth analysis of the image through complex image processing algorithms. This process involves taking pictures of the item from all angles to ensure that the most comprehensive and accurate product information is captured. The system then extracts feature points from the image, which include key information such as the shape, color, and texture of the item, which together constitute an important basis for identifying the identity of the item.
[0004] After extracting enough feature points, the system will integrate these feature point information and generate a vector library, which is actually a database containing the feature information of various products. When a new product needs to be identified, the system will convert its image into a feature vector and match it with the feature vector in the vector library to determine the type and price of the product by comparing the similarity.
[0005] However, although this technology has achieved remarkable results in practical applications, there are still problems that cannot be ignored. In particular, when the goods are bagged, their appearance characteristics and color distribution will change significantly, which poses a huge challenge to the AI recognition system. Since the image features of bagged items are significantly different from those of unbagged items, existing AI recognition algorithms often have difficulty accurately identifying bagged items, resulting in two independent training processes in practical applications: one is for recognition training of unbagged items, and the other is for recognition training of bagged items. This dual training mode is not only time-consuming and costly, but also increases the complexity and maintenance difficulty of the system.
[0006] Therefore, a more efficient and accurate AI visual inspection method is urgently needed to solve the problem of inaccurate identification of bagged items. By improving image processing technology, optimizing feature extraction algorithms, and introducing advanced deep learning models, it is expected to achieve one-time training and identification of bagged items, thereby greatly improving the recognition efficiency and accuracy of smart scales, and providing more convenient and efficient automated services for commercial scenarios such as supermarkets. Summary of the invention
[0007] In order to solve the problems existing in the background technology, the present invention provides an AI visual detection method for bagged objects, which comprises the following steps:
[0008] S1. Image acquisition and conversion:
[0009] S11, acquiring a first image of a bagged object to be detected;
[0010] S12, converting the first image from the RGB color space to the HSV color space;
[0011] S13, acquiring a second image containing only bagged materials without any object, and converting the second image from an RGB color space to an HSV color space;
[0012] S2, color stripping:
[0013] S21, performing a subtraction operation on corresponding pixel values of the first image and the second image in the HSV color space to generate a color-stripped image to obtain a third image;
[0014] S22, converting the third image from the HSV color space back to the RGB color space;
[0015] S3. Bag shape processing:
[0016] S31, performing gray value processing on the image after color stripping, identifying and deleting the discontinuous area with the farthest distance by calculating the center point distance of each area in the gray image;
[0017] S32, deeply screening and deleting the remaining discontinuous areas, and finally retaining and extracting the largest continuous area of the object to form an object image without the influence of the bag shape;
[0018] S4. Image detection:
[0019] The feature information of the bag-free object image obtained in step S32 is extracted, and the final image features are output in the fully connected layer. The feature information is used to identify the object and obtain the name of the object.
[0020] In a preferred solution, in step S4, the bag-free object image is processed using a convolutional neural network (CNN) model, and the feature information of the image is extracted through alternating processing of convolutional layers, activation functions, and pooling layers in the convolutional neural network (CNN) model.
[0021] In the preferred solution, the convolutional neural network CNN model includes:
[0022] Input layer, which receives the preprocessed object image;
[0023] Convolutional layer 1-convolutional layer 3, each layer extracts features of different levels of the image through convolution operations;
[0024] Activation function 1-3, located after each convolutional layer, is used to introduce nonlinear factors and improve the expressiveness of the model;
[0025] Pooling layer 1-pooling layer 3, through pooling operation to reduce the number of parameters and calculation amount, to prevent overfitting;
[0026] The fully connected layer nonlinearly combines the features learned by the convolutional layer and the pooling layer to output the final image features.
[0027] In the preferred solution, the processing process in step S4 is as follows:
[0028] S41, the input layer receives the preprocessed object image; the preprocessed image is passed to the input layer of the CNN for further processing;
[0029] S42, each convolution layer extracts features of different levels of the image through convolution operation; the convolution operation slides the convolution kernel on the image and performs weighted summation on each local area;
[0030] Convolutional layer 1 applies multiple different convolution kernels to perform convolution operations on the input image, extracting the edge and texture shallow features of the image, and the generated feature map is used as the input of the next layer; Convolutional layer 2 further performs convolution operations on the feature map of the previous layer to extract deep features; Convolutional layer 3 continues to extract high-level image features for object recognition and classification;
[0031] S43, activation function 1 to activation function 3 are located after each convolution layer, introduce nonlinear factors, handle complex nonlinear relationships, and generate new activation values by applying the activation function to the output of the convolution layer, which are used as the input of the next layer;
[0032] S44, using the maximum pooling or average pooling method through pooling layer 1 to pooling layer 3, pooling layer 1 performs a pooling operation on the output of convolution layer 1 to reduce the data dimension and extract important features; pooling layer 2 continues to perform a pooling operation on the output of the previous layer to further reduce the data dimension; pooling layer 3 is the final pooling layer, which retains key information, reduces the amount of calculation, and improves the generalization ability of the model;
[0033] S45. The fully connected layer nonlinearly combines the features learned by the convolutional layer and the pooling layer to output the final image features or classification results; each neuron in the fully connected layer is connected to all neurons in the previous layer; the fully connected layer flattens the output of the last pooling layer and converts it into a one-dimensional vector; the vector is input into the fully connected layer, and the final output result is obtained through weighted summation and activation function operations; for image recognition tasks, the predicted probability of each category is output; for feature extraction tasks, the output is the extracted image feature vector.
[0034] In a preferred solution, in step S32, expansion and carving image processing techniques are used for processing.
[0035] In a preferred solution, the specific process of step S32 includes:
[0036] S321, performing a dilation operation on the bright area of the foreground object in the extended image. During the processing, the brightness value of each pixel is expanded to its surrounding neighborhood. If there is a sufficiently bright pixel in the neighborhood, the pixel is regarded as part of the foreground.
[0037] S322, using engraving processing to reduce the bright area in the image; removing edge pixels to reduce noise and small independent areas in the image;
[0038] S323, alternately use dilation and carving to perform morphological opening operations to further remove discontinuous areas caused by bag wrinkles and uneven light;
[0039] S324, screening and deleting discontinuous regions. After morphological processing, if there are still some small, discontinuous regions, the area, shape, and position features of each continuous region are calculated to screen out small regions that do not meet the conditions. These small regions are deleted, and only the largest continuous region, i.e., the main body of the object, is retained.
[0040] S325. After deleting all discontinuous areas, what remains is the largest continuous area of the object; this area is extracted as the final image processing result.
[0041] In the preferred solution, the specific operation process of S321 is:
[0042] S3211, select a small rectangular or circular structure element to define the neighborhood range of the dilation operation;
[0043] S3212, sliding the structure element on the original image, and for each position, calculating the maximum value of pixels in the area covered by the structure element;
[0044] S3213, assigning the maximum value to the pixel point corresponding to the center of the structure element;
[0045] S3214. After the dilation process, the edges of the object are expanded outward, and small holes and discontinuous areas are filled.
[0046] In the preferred solution, the specific operation process of S322 is:
[0047] S3221, slide the structure element on the original image;
[0048] S3222. For each position, calculate the minimum value of pixels in the area covered by the structure element;
[0049] S3223, assigning the minimum value to the pixel point corresponding to the center of the structure element;
[0050] S3224. After the engraving operation, the edges of the object are retracted inwards, and some small protrusions and unimportant details are removed.
[0051] The beneficial effects achieved by the present invention are:
[0052] First, the present invention uses the processing steps of color stripping and bag shape processing, so that only one training is required for unbagged items to accurately identify the same bagged and unbagged items, which greatly reduces training time and cost and improves the use efficiency and user experience of equipment such as smart scales.
[0053] Second, the present invention uses the HSV color space for color stripping, effectively removing the influence of the bag color on object recognition. Through morphological operations such as grayscale value processing, expansion, and engraving, the discontinuous areas caused by factors such as bag wrinkles and uneven light are further removed, and the main features of the object are retained. These preprocessing steps significantly improve the image quality and provide clearer and more accurate image data for subsequent AI recognition.
[0054] Third, the present invention adopts the alternating processing of multi-layer convolutional layers, activation functions and pooling layers of the convolutional neural network (CNN) model to extract different levels of image features, and performs nonlinear combination through the fully connected layer to output the final image features. This not only improves the expressiveness of the model, but also reduces the number of parameters and calculation amount through operations such as pooling, prevents overfitting, and ensures the efficiency and accuracy of the model.
[0055] Fourth, the present invention further removes noise and small independent areas in the image by alternately using dilation and carving for morphological opening operations. At the same time, by calculating the features of each continuous area and screening and deleting small areas that do not meet the conditions, the largest continuous area of the object is finally retained. This step effectively improves the accuracy and stability of image recognition.
[0056] Fifth, the present invention uses efficient image preprocessing technology and optimized CNN models, which greatly shortens the time of the entire visual inspection process and significantly improves the efficiency of computing power. Even in the face of bags of different shapes, colors, and materials and complex ambient lighting conditions, it can maintain high recognition stability and accuracy. The present invention not only reduces the cost of equipment use, but also improves the response speed and stability of the overall system, significantly improves the recognition accuracy of bagged objects, and enhances the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 It is a flow chart of AI visual inspection of bagged objects of the present invention;
[0058] Figure 2 It is a flowchart of AI visual inspection of bagged objects in the prior art;
[0059] Figure 3 It is an image shape diagram of the bag shape processing process. DETAILED DESCRIPTION
[0060] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures recorded in the following embodiments are merely illustrative, and the present invention is not limited to the various structures recorded in the following embodiments. All other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.
[0061] Reference Figure 2In the prior art, AI model recognition requires feature point extraction of bagged and unbagged images. The changes in color and shape after bagging will affect the feature points of the image, so this is the root cause of the need for both bagged and unbagged algorithms for recognition. The solution of this technical solution is to perform image processing on the bagged object, perform color stripping, and then strip the bag shape through continuous area changes to obtain the unbagged image object for AI recognition. Therefore, only the color of the object needs to be deleted, and the RGB data of the bagged object photo is converted to HSV. Then take a photo of the bag and convert it to HSV. The two values are subtracted to obtain new HSV data, and then the HSV data is converted to RGB. After color processing, there will be a dark background formed by local wrinkles or light influence. By obtaining the grayscale value, discontinuous areas will be formed at this time, such as Figure 3 shown.
[0062] The present invention simultaneously calculates the distance between the center points of each area, deletes the discontinuous area with the farthest distance, and then screens out the discontinuous area by dilation and carving, deletes it, and retains only the largest continuous area. In this way, the image of the object without the bag can be obtained. The pre-processed object is sent to the model for image detection, and the name of the identified object can be obtained. After the body is marked and recognized without bagging, it can meet the requirements of one-time training recognition of bagged and unbagged objects.
[0063] Combine the following Figure 1 The above process is described in detail. An AI visual inspection method for bagged objects comprises the following steps:
[0064] S1. Image acquisition and conversion:
[0065] S11, acquiring a first image of a bagged object to be detected;
[0066] S12, converting the first image from the RGB color space to the HSV color space;
[0067] S13, acquiring a second image containing only bagged materials without any object, and converting the second image from an RGB color space to an HSV color space;
[0068] S2, color stripping:
[0069] S21, performing a subtraction operation on corresponding pixel values of the first image and the second image in the HSV color space to generate a color-stripped image to obtain a third image;
[0070] S22, converting the third image from the HSV color space back to the RGB color space; in step S32, processing the image using dilation and engraving image processing techniques. The specific process includes:
[0071] S321, a dilation operation is performed on the bright area of the foreground object in the extended image. During the processing, the brightness value of each pixel is expanded to its surrounding neighborhood. If there is a sufficiently bright pixel in the neighborhood, the pixel is regarded as part of the foreground. The specific operation process of S321 is as follows:
[0072] S3211, select a small rectangular or circular structure element to define the neighborhood range of the dilation operation;
[0073] S3212, sliding the structure element on the original image, and for each position, calculating the maximum value of pixels in the area covered by the structure element;
[0074] S3213, assigning the maximum value to the pixel point corresponding to the center of the structure element;
[0075] S3214. After the dilation process, the edges of the object are expanded outward, and small holes and discontinuous areas are filled.
[0076] S322, using engraving processing operation to reduce the bright area in the image; removing edge pixels to reduce noise and small independent areas in the image; the specific operation process of S322 is:
[0077] S3221, slide the structure element on the original image;
[0078] S3222. For each position, calculate the minimum value of pixels in the area covered by the structure element;
[0079] S3223, assigning the minimum value to the pixel point corresponding to the center of the structure element;
[0080] S3224. After the engraving operation, the edges of the object are retracted inwards, and some small protrusions and unimportant details are removed.
[0081] S323, alternately use dilation and carving to perform morphological opening operations to further remove discontinuous areas caused by bag wrinkles and uneven light;
[0082] S324, screening and deleting discontinuous regions. After morphological processing, if there are still some small, discontinuous regions, the area, shape, and position features of each continuous region are calculated to screen out small regions that do not meet the conditions. These small regions are deleted, and only the largest continuous region, i.e., the main body of the object, is retained.
[0083] S325. After deleting all discontinuous areas, what remains is the largest continuous area of the object; this area is extracted as the final image processing result.
[0084] S3. Bag shape processing:
[0085] S31, performing gray value processing on the image after color stripping, identifying and deleting the discontinuous area with the farthest distance by calculating the center point distance of each area in the gray image;
[0086] S32, deeply screening and deleting the remaining discontinuous areas, and finally retaining and extracting the largest continuous area of the object to form an object image without the influence of the bag shape;
[0087] S4. Image detection:
[0088] The feature information of the bag-free object image obtained in step S32 is extracted, and the final image features are output in the fully connected layer. The feature information is used to identify the object and obtain the name of the object.
[0089] In step S4, the bag-free object image is processed using a convolutional neural network (CNN) model. In the convolutional neural network (CNN) model, feature information of the image is extracted through alternating processing of convolutional layers, activation functions, and pooling layers.
[0090] In the preferred solution, the convolutional neural network CNN model includes:
[0091] Input layer, which receives the preprocessed object image;
[0092] Convolutional layer 1-convolutional layer 3, each layer extracts features of different levels of the image through convolution operations;
[0093] Activation function 1-3, located after each convolutional layer, is used to introduce nonlinear factors and improve the expressiveness of the model;
[0094] Pooling layer 1-pooling layer 3, through pooling operation to reduce the number of parameters and calculation amount, to prevent overfitting;
[0095] The fully connected layer nonlinearly combines the features learned by the convolutional layer and the pooling layer to output the final image features.
[0096] The processing process is as follows:
[0097] S41, the input layer receives the preprocessed object image; the preprocessed image is passed to the input layer of the CNN for further processing;
[0098] S42, each convolution layer extracts features of different levels of the image through convolution operation; the convolution operation slides the convolution kernel on the image and performs weighted summation on each local area;
[0099] Convolutional layer 1 applies multiple different convolution kernels to perform convolution operations on the input image, extracting the edge and texture shallow features of the image, and the generated feature map is used as the input of the next layer; Convolutional layer 2 further performs convolution operations on the feature map of the previous layer to extract deep features; Convolutional layer 3 continues to extract high-level image features for object recognition and classification;
[0100] S43, activation function 1 to activation function 3 are located after each convolution layer, introduce nonlinear factors, handle complex nonlinear relationships, and generate new activation values by applying the activation function to the output of the convolution layer, which are used as the input of the next layer;
[0101] S44, using the maximum pooling or average pooling method through pooling layer 1 to pooling layer 3, pooling layer 1 performs a pooling operation on the output of convolution layer 1 to reduce the data dimension and extract important features; pooling layer 2 continues to perform a pooling operation on the output of the previous layer to further reduce the data dimension; pooling layer 3 is the final pooling layer, which retains key information, reduces the amount of calculation, and improves the generalization ability of the model;
[0102] S45. The fully connected layer nonlinearly combines the features learned by the convolutional layer and the pooling layer to output the final image features or classification results; each neuron in the fully connected layer is connected to all neurons in the previous layer; the fully connected layer flattens the output of the last pooling layer and converts it into a one-dimensional vector; the vector is input into the fully connected layer, and the final output result is obtained through weighted summation and activation function operations; for image recognition tasks, the predicted probability of each category is output; for feature extraction tasks, the output is the extracted image feature vector.
[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. An AI visual inspection method for bagged objects, characterized in that: It includes the following steps: S1. Image acquisition and conversion: S11, acquiring a first image of a bagged object to be detected; S12, converting the first image from the RGB color space to the HSV color space; S13, acquiring a second image containing only bagged materials without any object, and converting the second image from an RGB color space to an HSV color space; S2, color stripping: S21, performing a subtraction operation on corresponding pixel values of the first image and the second image in the HSV color space to generate a color-stripped image to obtain a third image; S22, converting the third image from the HSV color space back to the RGB color space; S3. Bag shape processing: S31, performing gray value processing on the image after color stripping, identifying and deleting the discontinuous area with the farthest distance by calculating the center point distance of each area in the gray image; S32, deeply screening and deleting the remaining discontinuous areas, and finally retaining and extracting the largest continuous area of the object to form an object image without the influence of the bag shape; S4. Image detection: The feature information of the bag-free object image obtained in step S32 is extracted, and the final image features are output in the fully connected layer. The feature information is used to identify the object and obtain the name of the object.
2. The AI visual inspection method for bagged objects according to claim 1, characterized in that: In step S4, the bag-free object image is processed using a convolutional neural network (CNN) model. In the convolutional neural network (CNN) model, feature information of the image is extracted through alternating processing of convolutional layers, activation functions, and pooling layers.
3. The AI visual inspection method for bagged objects according to claim 2, characterized in that: The convolutional neural network CNN model includes: Input layer, which receives the preprocessed object image; Convolutional layer 1-convolutional layer 3, each layer extracts features of different levels of the image through convolution operations; Activation function 1-3, located after each convolutional layer, is used to introduce nonlinear factors and improve the expressiveness of the model; Pooling layer 1-pooling layer 3, through pooling operation to reduce the number of parameters and calculation amount, to prevent overfitting; The fully connected layer nonlinearly combines the features learned by the convolutional layer and the pooling layer to output the final image features.
4. The AI visual inspection method for bagged objects according to claim 3, characterized in that: The processing in step S4 is as follows: S41, the input layer receives the preprocessed object image; the preprocessed image is passed to the input layer of the CNN for further processing; S42, each convolution layer extracts features of different levels of the image through convolution operation; the convolution operation slides the convolution kernel on the image and performs weighted summation on each local area; Convolutional layer 1 applies multiple different convolution kernels to perform convolution operations on the input image, extracting the edge and texture shallow features of the image, and the generated feature map is used as the input of the next layer; Convolutional layer 2 further performs convolution operations on the feature map of the previous layer to extract deep features; Convolutional layer 3 continues to extract high-level image features for object recognition and classification; S43, activation function 1 to activation function 3 are located after each convolution layer, introduce nonlinear factors, handle complex nonlinear relationships, and generate new activation values by applying the activation function to the output of the convolution layer, which are used as the input of the next layer; S44, using the maximum pooling or average pooling method through pooling layer 1 to pooling layer 3, pooling layer 1 performs a pooling operation on the output of convolution layer 1 to reduce the data dimension and extract important features; pooling layer 2 continues to perform a pooling operation on the output of the previous layer to further reduce the data dimension; pooling layer 3 is the final pooling layer, which retains key information, reduces the amount of calculation, and improves the generalization ability of the model; S45. The fully connected layer nonlinearly combines the features learned by the convolutional layer and the pooling layer to output the final image features or classification results; each neuron in the fully connected layer is connected to all neurons in the previous layer; the fully connected layer flattens the output of the last pooling layer and converts it into a one-dimensional vector; the vector is input into the fully connected layer, and the final output result is obtained through weighted summation and activation function operations; for image recognition tasks, the predicted probability of each category is output; for feature extraction tasks, the output is the extracted image feature vector.
5. The AI visual inspection method for bagged objects according to claim 1, characterized in that: In step S32, expansion and carving image processing techniques are used for processing.
6. The AI visual inspection method for bagged objects according to claim 5, characterized in that: The specific process of step S32 includes: S321, performing a dilation operation on the bright area of the foreground object in the extended image. During the processing, the brightness value of each pixel is expanded to its surrounding neighborhood. If there is a sufficiently bright pixel in the neighborhood, the pixel is regarded as part of the foreground. S322, using engraving processing to reduce the bright area in the image; removing edge pixels to reduce noise and small independent areas in the image; S323, alternately use dilation and carving to perform morphological opening operations to further remove discontinuous areas caused by bag wrinkles and uneven light; S324, screening and deleting discontinuous regions. After morphological processing, if there are still some small, discontinuous regions, the area, shape, and position features of each continuous region are calculated to screen out small regions that do not meet the conditions. These small regions are deleted, and only the largest continuous region, i.e., the main body of the object, is retained. S325. After deleting all discontinuous areas, what remains is the largest continuous area of the object; this area is extracted as the final image processing result.
7. The AI visual inspection method for bagged objects according to claim 6, characterized in that: The specific operation process of S321 is as follows: S3211, select a small rectangular or circular structure element to define the neighborhood range of the dilation operation; S3212, sliding the structure element on the original image, and for each position, calculating the maximum value of pixels in the area covered by the structure element; S3213, assigning the maximum value to the pixel point corresponding to the center of the structure element; S3214. After the dilation process, the edges of the object are expanded outward, and small holes and discontinuous areas are filled.
8. The AI visual inspection method for bagged objects according to claim 6, characterized in that: The specific operation process of S322 is as follows: S3221, slide the structure element on the original image; S3222. For each position, calculate the minimum value of pixels in the area covered by the structure element; S3223, assigning the minimum value to the pixel point corresponding to the center of the structure element; S3224. After the engraving operation, the edges of the object are retracted inwards, and some small protrusions and unimportant details are removed.
