Tableware appearance detection method, device and equipment

Abstract

The invention relates to the technical field of tableware detection, in particular to a tableware appearance detection method, device and equipment. The method comprises the steps that various light sources are arranged on a tableware conveying path, and the light sources are aligned to different positions of tableware; acquiring image data of each position of the tableware irradiated by the light source; and a pre-trained neural network model obtains a detection result of the tableware appearance according to the image data. The multiple light sources at different angles are installed at the key positions of the sorting path, it is ensured that tableware is evenly and fully illuminated in all directions when passing through the sorting path, the quality of subsequent shot images is improved, potential flaws are more obvious, and the detection precision is effectively improved. In addition, the collected images are automatically analyzed based on a deep learning algorithm, the stain and damage conditions on the tableware are recognized, and accurate and efficient defect detection is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of tableware detection equipment, and in particular to a tableware appearance detection method, device and equipment. Background Art

[0002] In the catering service industry, tableware packs are an important part of providing customers with convenient dining. Traditional tableware packs usually contain at least basic tableware such as plates, bowls and cups, and are efficiently sorted, stacked and packaged through automated equipment. If the tableware in the tableware pack is stained or damaged, it will greatly affect the user experience. Therefore, it is necessary to screen out stained and damaged tableware in the tableware pack in a timely manner.

[0003] For example, Chinese patent CN109570068B discloses a fully automatic tableware sorting, stacking and packaging machine. In order to ensure the quality of the tableware, suspended 2D visual sensors are set on the teacup collection belt, bowl collection belt and plate collection belt. These sensors are connected to the push-down cylinders. When damaged or dirty tableware is detected, the sensor will trigger the corresponding cylinder action to push the unqualified tableware from the collection belt to the collection area below.

[0004] However, the existing detection schemes have low detection accuracy and cannot perform comprehensive and effective identification, thus affecting the hygiene standards of the final tableware package. Summary of the invention

[0005] In view of this, it is necessary to provide a tableware appearance detection method, device and equipment to solve the problem of low detection accuracy in the prior art, inability to perform comprehensive and effective identification, and resulting in damaged and unclean tableware being packaged.

[0006] A tableware appearance detection method, comprising: Arrange a plurality of light sources on the tableware conveying path, and aim the light sources at different positions of the tableware; Acquiring image data of each position of the tableware after being irradiated by the light source; Based on the pre-trained neural network model, the detection result of the tableware appearance is obtained according to the image data.

[0007] Preferably, it includes: determining tableware information according to the image data; wherein the tableware information includes the material, color and shape of the tableware; The image data is compared with a preset standard sample corresponding to the tableware information, and the acquisition conditions are adjusted in real time according to the comparison result; wherein the real-time adjustment of the acquisition conditions includes adjusting the light source intensity, light source color and camera parameters.

[0008] Preferably, it includes: acquiring image data of each position of the tableware after being irradiated by the light source through a multi-spectral imaging camera.

[0009] Preferably, it includes: recording the image data of unqualified tableware according to the detection result; The neural network model is trained using the unqualified tableware image data.

[0010] Preferably, it includes: when unqualified tableware is detected, an alarm message is issued and displayed, and image data of the unqualified tableware is fed back to the cleaning process.

[0011] A tableware appearance detection device, comprising: A light source module is deployed on the tableware conveying path, and the light source is aimed at different positions of the tableware; An image acquisition module, used to acquire image data of each position of the tableware after being irradiated by the light source; The visual recognition processing module is used to obtain the detection result of the appearance of the tableware according to the image data based on a deep learning algorithm.

[0012] Preferably, it comprises: a visual recognition processing module, further used to determine tableware information according to the image data; wherein the tableware information includes the material, color and shape of the tableware; An adjustment module is used to obtain the tableware information and determine the corresponding preset standard sample; compare the image data with the preset standard sample; and adjust the acquisition conditions in real time according to the comparison result; wherein the real-time adjustment of the acquisition conditions includes adjusting the light source intensity, light source color and camera parameters.

[0013] Preferably, it comprises: a multi-spectral imaging camera for acquiring image data of each position of the tableware after being irradiated by the light source; A storage module, used for recording image data of unqualified tableware according to the detection results; A training module, used for training the deep learning algorithm using the unqualified tableware image data; The alarm module is used to issue and display an alarm message when unqualified tableware is detected.

[0014] A tableware appearance detection device comprises the tableware appearance detection device as described above, a sorting device, and a conveyor belt; the sorting device is used to sort different tableware to corresponding conveyor belts; The conveyor belt is used to convey the corresponding tableware; The tableware appearance detection device is deployed on the conveying path of the conveyor belt and is used to detect the appearance of the tableware.

[0015] Preferably, it includes: a real-time push-down cylinder, which is connected to the signal of the tableware appearance detection device, and is used to push unqualified tableware out of the conveyor belt according to the detection results of the tableware appearance detection device. Compared with the prior art, the beneficial effect of the present application is that: multiple light sources at different angles are installed at key positions of the sorting path to ensure that the tableware is evenly and fully illuminated from all directions when passing through, thereby improving the quality of subsequent captured images, making potential defects more obvious, and effectively improving the detection accuracy. In addition, the collected images are automatically analyzed based on a deep learning algorithm to identify stains and damage on the tableware, thereby achieving accurate and efficient defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the tableware appearance detection method provided in the embodiment of the present application.

[0017] Figure 2 It is a schematic diagram of the structure of a tableware appearance detection device provided in an embodiment of the present application.

[0018] Figure 3 It is a structural schematic diagram of a tableware appearance detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] The tableware appearance inspection solution provided in this application is suitable for tableware sorting, stacking and packaging combined machines, especially for inspecting the tableware therein, and timely discovering damaged and stained tableware. Figure 1 , Figure 1 : is a flow chart of a tableware appearance detection method provided in an embodiment of the present application, the tableware appearance detection method comprises the following steps: S101: deploying a plurality of light sources on a tableware conveying path, wherein the light sources are directed at different positions of the tableware.

[0021] In the prior art, additional light sources are usually not used, or the light sources used may not provide sufficient light coverage, making it difficult to detect defects in certain parts. This embodiment deploys multiple light sources and accurately aligns them with different positions of the tableware to ensure that the surface of the tableware is evenly illuminated from multiple perspectives, thereby improving the quality of subsequent images and thus increasing the possibility of capturing subtle defects. The multiple light sources can be the same light source at multiple different angles, or light sources with different light intensities, different color temperatures, and different frequencies.

[0022] For example, suppose that in a fully automatic sorting, stacking and packaging machine, three groups of light sources, such as LED lights, are installed above the collection belt of tea cups, bowls and plates. The first group irradiates the tableware vertically from top to bottom, and is used to illuminate the bottom of the bowl, the bottom of the plate, etc.; the second group irradiates the inner side wall of the tableware at 45 degrees to the horizontal direction; and the third group irradiates the outer side wall of the tableware in the horizontal direction. In this embodiment, each group of light sources can include multiple LED lights, which can be evenly arranged along the circumference when irradiating the side wall of the tableware to fully illuminate every part of the tableware.

[0023] Furthermore, each group of light sources may include different types of light sources, for example, each group of light sources may include white LEDs for basic lighting; red LEDs, which are particularly helpful in highlighting edges or dark stains; and UV (ultraviolet) LEDs, which can help identify organic residues such as grease that are invisible to the naked eye. These light sources are precisely aimed at different positions of the tableware to ensure that the surface of the tableware is evenly illuminated from multiple angles, thereby improving the quality of subsequent image data.

[0024] S102: Acquire image data of each position of the tableware after being irradiated by the light source.

[0025] The image data of tableware can be obtained through the camera. For light sources at different angles, a camera can be set at the corresponding position. For example, a high-resolution industrial camera is used to capture the image of tableware after being illuminated by the above light source at a speed of 30 frames per second. For a standard-sized dinner plate, at least 6 high-definition pictures from different angles can be captured when it passes completely. These high-quality images at different angles and different lighting conditions provide a rich information basis for subsequent analysis, so that even tiny defects can be clearly recorded.

[0026] Furthermore, the camera can be a multi-spectral imaging camera, which can simultaneously capture images in multiple bands such as visible light, near infrared (NIR), and ultraviolet (UV) by acquiring image data of each position of the tableware after being irradiated by the light source. For example, if there are organic pollutants such as grease, release agents, etc. remaining on the surface of the tableware, these substances may not be easily detected under visible light, but will emit fluorescence under ultraviolet light, thus becoming clearly visible; and for the glaze on porcelain, near-infrared light has a specific absorption band for the glaze on porcelain. When there are cracks or unevenness in the glaze layer, it will change the reflection pattern of the near-infrared light. By analyzing the changes in these reflection patterns, potential problem areas can be identified. In addition, industrial cameras and multi-spectral imaging cameras can be set up on the inspection line for combination to further expand the types of images captured.

[0027] S103: Based on the pre-trained neural network model, a detection result of the tableware appearance is obtained according to the image data.

[0028] In this embodiment, the pre-trained neural network model can use a convolutional neural network (CNN). The model has been trained on several labeled tableware defect samples. When a new image is input, the pre-trained neural network model can automatically extract features and determine whether there are stains, cracks or other types of damage. This not only improves the detection accuracy, but also reduces the need for manual intervention, and realizes an efficient automated detection process.

[0029] For example, when pre-training a neural network model, you first need to build a dataset containing a large number of annotated tableware images. These images should cover different types of tableware, and include samples in clean and intact conditions as well as in various damaged and stained conditions. Each image should be labeled to indicate its corresponding category, such as "intact", "cracked", "chipped", "water-stained", and "oil-stained". Then the collected images are pre-processed to ensure that they have the same size and color mode (grayscale or color), and may also need to be enhanced to increase data diversity, such as rotation, flipping, or adjusting brightness and contrast. Then a deep learning architecture suitable for image classification tasks is selected to build a CNN model and trained using the previously prepared dataset. During the training process, the model will learn how to distinguish different types of tableware and their surface features, especially water stains, oil stains, and damage. Then the model can be verified and tuned, and the model performance can be tested on an independent validation set, and the model effect can be evaluated based on accuracy, recall, and other indicators. If necessary, the model performance can be improved by adjusting hyperparameters, introducing regularization techniques, or adding more training data.

[0030] Once the model is trained and verified, it can be deployed in the actual tableware inspection environment. For example, when the tableware passes through the inspection area, the multispectral imaging camera captures a series of high-definition pictures taken from different angles. The pictures (image data) are fed into the deployed CNN model for real-time analysis. The model quickly scans each picture, looking for any areas that do not meet the normal tableware appearance standards, such as water stains, oil stains, cracks, gaps, or other abnormal shapes.

[0031] Based on the model's predictions, the system determines the status of the tableware. "Qualified" tableware will continue along the conveyor belt to the packaging area; while tableware marked as "unqualified" will trigger an alarm mechanism and be removed from the production line by a push-off cylinder for re-cleaning or disposal. All detected breakage cases will be recorded and fed back into a new data point. Periodically, this new data will be used to further train and optimize the existing CNN model so that it can more accurately identify future breakages.

[0032] Furthermore, based on the image data, the tableware information is determined; wherein the tableware information includes the material, color and shape of the tableware; the image data is compared with a preset standard sample corresponding to the tableware information, and the acquisition conditions are adjusted in real time based on the comparison result; wherein the real-time adjustment of the acquisition conditions includes adjusting the light source intensity, light source color and camera parameters.

[0033] In this embodiment, machine vision technology can be used, such as training in deep learning algorithms, to identify tableware information based on the image data of tableware. Tableware is first preliminarily scanned when it enters the detection area. For example, ceramic bowls and glass cups can be distinguished through color recognition, spectral analysis, etc., and shape analysis can distinguish different types of tableware.

[0034] The actual image is compared with the pre-stored standard sample. If it is found that the surface reflection of a batch of ceramic bowls is strong, affecting the image quality, the light intensity will be automatically reduced and the camera exposure time will be adjusted to the optimal value. In this way, the best imaging state can be maintained, ensuring the consistency and accuracy of the inspection, and it can work stably even when facing tableware with large changes in material or color.

[0035] Furthermore, according to the detection results, the image data of unqualified tableware is recorded; and the deep learning algorithm is trained using the image data of unqualified tableware. For example, whenever defective tableware is detected, the system will automatically save one or more related images and attach a detailed description, such as "there are obvious cracks on the bottom of the porcelain bowl numbered #12345". This information will be archived and can also be used to train and improve the neural network model, such as regularly using the latest batch of unqualified product image data as a new batch of training sets to update the existing neural network model. As more different types of defects are learned, the model becomes more intelligent and can better adapt to various complex situations in practice. The continuously optimized algorithm makes the system more and more accurate and efficient, reduces the false alarm rate, and can quickly respond to the emergence of new defects. This embodiment not only facilitates tracking historical records, but also provides valuable data support for algorithm improvement.

[0036] Furthermore, when unqualified tableware is detected, an alarm message is issued and displayed. Once unqualified tableware is detected, an alarm is immediately triggered, and the corresponding tableware image and words such as "unqualified" and "NG" pop up on the display device such as the workstation screen. In addition, the specific image of the problematic tableware and the specific defects it has can also be displayed, such as "The glass cup No. #67890 is found to have a chip on the rim."

[0037] In addition, statistical analysis can be performed regularly on relevant unqualified data. For example, when the probability of stains appearing on a certain part of the tableware is higher than a preset value, it can be considered that there is a problem in the cleaning process. The image data of the unqualified tableware can then be fed back to the cleaning process so that the various processes and equipment in the cleaning process such as the dishwasher can be inspected and repaired.

[0038] The above-mentioned embodiment of the present application solves the problem of insufficient detection accuracy in the prior art through a multi-dimensional data collection method, and enhances the detection capability of the system. In addition, it also solves the shadow problem that may be caused by a traditional single light source, and the problem of a single type of defect detection caused by a traditional single type of light source.

[0039] Based on the same inventive concept, the present application also provides a tableware appearance detection device 200, such as Figure 2 The tableware appearance detection device shown in the figure comprises: The light source module 201 is deployed on the tableware conveying path, and the light source is aimed at different positions of the tableware; An image acquisition module 202 is used to acquire image data of each position of the tableware after being irradiated by the light source; The visual recognition processing module 203 is used to obtain the detection result of the tableware appearance according to the image data based on the deep learning algorithm.

[0040] The visual recognition processing module 203 is also used to determine the tableware information according to the image data; wherein the tableware information includes the material, color and shape of the tableware; The adjustment module 204 is used to obtain the tableware information and determine the corresponding preset standard sample; compare the image data with the preset standard sample; and adjust the acquisition conditions in real time according to the comparison result; wherein the real-time adjustment of the acquisition conditions includes adjusting the light source intensity, light source color and camera parameters.

[0041] Furthermore, the image acquisition module 202 may be a multi-spectral imaging camera, which is used to acquire image data of each position of the tableware after being irradiated by the light source; Furthermore, it also includes a storage module for recording the image data of unqualified tableware according to the detection results; a training module for training the deep learning algorithm using the unqualified tableware image data; and an alarm module for issuing and displaying an alarm message when unqualified tableware is detected.

[0042] For the implementation process of each module and unit in this embodiment, please refer to steps S101-S103 and any optional implementation methods thereof, which will not be described in detail in this embodiment.

[0043] Based on the same inventive concept, the present application also provides a tableware appearance detection device 300, such as Figure 3 The structural schematic diagram of the tableware appearance detection equipment shown in the figure, the tableware appearance detection equipment includes the tableware appearance detection device 200 as described above, as well as a sorting device and a conveyor belt; the sorting device is used to sort different tableware to the corresponding conveyor belt; the conveyor belt is used to convey the corresponding tableware; the tableware appearance detection device 200 is deployed on the conveying path of the conveyor belt, and is used to detect the appearance of the tableware. Figure 3 In the figure, A, B, and C represent three conveyor belts respectively; D represents a sorting device; A1, B1, and C1 represent tableware appearance detection devices 200 deployed on the three conveyor belts respectively.

[0044] Assuming that a complete tableware package includes teacups, bowls and plates, the washed teacups, bowls and plates are first sent to their respective conveyor belts by a sorting device. The tableware appearance detection device 200 mentioned above is equipped above each conveyor belt, which is responsible for real-time detection of the tableware passing through.

[0045] The qualified tableware continues to the packaging area, while the problematic tableware is marked out. This realizes the automatic management of the whole process, improves the work efficiency, reduces the labor cost, and ensures the high quality of the final product.

[0046] Furthermore, a real-time push-down cylinder is installed at a key position of each conveyor belt, such as Figure 3 A2, B2, and C2 in the tableware appearance detection device 200 are connected to the tableware appearance detection device 200. Once unqualified tableware is detected, the system will immediately send instructions to the corresponding cylinder, and the cylinder will push the problematic tableware out of the conveyor belt for collection or directly enter the re-cleaning cycle. This instant response mechanism ensures that only qualified tableware can enter the next process, effectively preventing defective products from being mixed into the finished product, ensuring product quality, and also simplifying the operation process of the production line and reducing the need for manual intervention.

[0047] Furthermore, the sorting device may be a height-limiting rod that is adjustable in height and tilted, for example Figure 3 D in the figure is used to allow the tableware corresponding to the current conveyor belt to pass according to the height of the tableware, and to block the tableware that needs to be sorted and enter the target conveyor belt along the inclined direction.

[0048] Furthermore, a tableware limiting baffle may be included to keep the tableware in the shooting field of view. That is, when the tableware is on the conveyor belt, it will be blocked by the tableware limiting baffle and move, so that when taking pictures, the tableware is in the center of the camera's field of view.

Claims

1. A tableware appearance detection method, characterized in that: include: Arrange a plurality of light sources on the tableware conveying path, and aim the light sources at different positions of the tableware; Acquiring image data of each position of the tableware after being irradiated by the light source; Based on the pre-trained neural network model, the detection result of the tableware appearance is obtained according to the image data.

2. The tableware appearance detection method according to claim 1, characterized in that: include: Determine tableware information according to the image data; wherein the tableware information includes the material, color and shape of the tableware; The image data is compared with a preset standard sample corresponding to the tableware information, and the acquisition conditions are adjusted in real time according to the comparison result; wherein the real-time adjustment of the acquisition conditions includes adjusting the light source intensity, light source color and camera parameters.

3. The tableware appearance detection method according to any one of claims 1 to 2, characterized in that: Also includes: The image data of each position of the tableware after being irradiated by the light source is obtained by a multi-spectral imaging camera.

4. The tableware appearance detection method according to claim 1, characterized in that: include: According to the detection results, recording the image data of unqualified tableware; The neural network model is optimized using the unqualified tableware image data.

5. The tableware appearance detection method according to claim 1, characterized in that: include: When unqualified tableware is detected, an alarm message is issued and displayed.

6. A tableware appearance detection device, characterized in that: include: A light source module is deployed on the tableware conveying path, and the light source is aimed at different positions of the tableware; An image acquisition module, used to acquire image data of each position of the tableware after being irradiated by the light source; The visual recognition processing module is used to obtain the detection result of the appearance of the tableware according to the image data based on a deep learning algorithm.

7. The tableware appearance detection device according to claim 6, characterized in that: include: The visual recognition processing module is further used to determine the tableware information according to the image data; wherein the tableware information includes the material, color and shape of the tableware; An adjustment module is used to obtain the tableware information and determine the corresponding preset standard sample; compare the image data with the preset standard sample; and adjust the acquisition conditions in real time according to the comparison result; wherein the real-time adjustment of the acquisition conditions includes adjusting the light source intensity, light source color and camera parameters.

8. The tableware appearance detection device according to claim 7, characterized in that: Also includes: A multi-spectral imaging camera, used to obtain image data of each position of the tableware after being irradiated by the light source; A storage module, used for recording image data of unqualified tableware according to the detection results; A training module, used for training the deep learning algorithm using the unqualified tableware image data; The alarm module is used to issue and display an alarm message when unqualified tableware is detected.

9. A tableware appearance inspection device, characterized in that: The device comprises a tableware appearance detection device as claimed in any one of claims 6 to 8, a sorting device and a conveyor belt; the sorting device is used to sort different tableware to corresponding conveyor belts; The conveyor belt is used to convey the corresponding tableware; The tableware appearance detection device is deployed on the conveying path of the conveyor belt and is used to detect the appearance of the tableware.

10. The tableware appearance inspection device according to claim 9, characterized in that: Also includes: The real-time push-down cylinder is connected to the tableware appearance detection device signal and is used to push unqualified tableware out of the conveyor belt according to the detection result of the tableware appearance detection device.

Images