Canned pork rinsing and hair picking automatic detection system and device based on image recognition

Abstract

The invention discloses an automatic detection system and device for rinsing and hair picking of canned pork based on image recognition, and relates to the technical field of automatic detection. Comprising a physical sensing module for acquiring a pork can image in a pork can conveying process, and sorting defective pork cans according to an image recognition result; the virtual generation module is used for taking the image containing the defect as a source domain image, taking the image of the current batch as a target domain image, and taking the source domain image and the target domain image as input and output of a generative adversarial network model to obtain a synthetic defect image; and the model decision module is used for acquiring a detection and identification model and a detection result of the current batch according to the high-fidelity simulation image and the real image of the current batch. Precise and automatic sorting of cans containing hair and other defects is achieved, the detection efficiency and accuracy are effectively improved, and the problems of false detection and missing detection caused by material differences are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic detection technology, in particular to a pig can rinsing and picking hair automatic detection system and device based on image recognition. BACKGROUND

[0002] As an important instant food, pig cans have a wide consumer demand in the domestic and foreign markets. In the production process of pig cans, the pretreatment of raw meat is one of the key links to determine the quality of the final product. The rinsing process aims to remove blood stains, impurities and residual pig hair and other foreign matter on the surface of the raw meat, while the subsequent picking process is used to manually or semi-manually check and remove small hairs still attached to the meat pieces after rinsing, which directly relates to the hygiene and safety of the product and the consumer experience.

[0003] At present, most domestic pig can production enterprises still mainly rely on manual visual inspection in the picking process after rinsing. The operator needs to observe the rinsed meat pieces one by one beside the assembly line and manually pick out residual hairs or other visible foreign matter. This traditional method has many drawbacks: first, manual detection is inefficient and difficult to match the pace of modern high-speed production lines, becoming a bottleneck to the overall capacity improvement; second, the detection result is highly dependent on the experience, attention and fatigue level of the workers, which is prone to missed detection, false detection and other problems, resulting in unstable product quality; third, long-term high-intensity visual work can easily cause visual fatigue of workers, increasing the risk of occupational health; finally, the labor cost is rising year by year, and the problems of difficult recruitment and management are becoming increasingly prominent, which is not conducive to the sustainable development of enterprises.

[0004] Chinese invention patent with publication number CN115147363A discloses an image defect detection and classification method and system based on deep learning algorithm, which includes: image acquisition to obtain training images, including qualified samples and unqualified samples; classification of training images to obtain classification types, including qualified sample types and unqualified sample types, and unqualified sample types including defect types; pre-processing of training images, obtaining image data sets according to the image data and classification types of the pre-processed training images; training through image data sets and deep learning algorithm convolutional neural network to obtain image defect detection and classification model; when the detection image is obtained, the image defect detection and classification model is used to process the detection image to obtain the defect detection and classification result of the detection image. It reduces the difficulty of defect detection and classification, and improves the accuracy of defect detection and classification.

[0005] Although the above technical solution can identify the surface defects of pork, there are natural differences among different pig breeds, different meat sources and different batches, so there is inevitable diversity among pork raw materials, which causes the color, texture and optical properties of the surface of the inspected object to present significant time-varying characteristics. Therefore, during the detection of the surface defects of pork, segmentation deviation is easy to occur, and false detection and missed detection phenomena occur, which seriously restricts the tolerance ability of the detection system to material changes. SUMMARY

[0006] The purpose of the present application is to provide an image recognition-based automatic detection system and device for rinsing and picking up hair of canned pork, to solve the problems raised in the background art.

[0007] To achieve the above purpose, the present application provides the following technical solution: an image recognition-based automatic detection system for rinsing and picking up hair of canned pork, characterized in that it comprises:

[0008] The physical perception module acquires the image of the canned pork during transmission, and according to the recognition result of the image of the canned pork, it sorts out the canned pork with defects;

[0009] The virtual generation module takes the image of the canned pork with defects as the source domain image, takes the image of the canned pork of the current batch as the target domain image, and takes both the source domain image and the target domain image as the input of the generative adversarial network model, and outputs the corresponding synthetic defect image, comprising:

[0010] SA1: image processing: collect and acquire the image of the inspected defective canned pork, take the image of the canned pork with defects as the source domain image, extract and take a preset number of defect-free canned pork in the current batch production line, acquire the image of the defect-free canned pork, take the preset number of defect-free canned pork images as the target domain image, and pre-process the source domain image and the target domain image;

[0011] SA2: synthetic image screening: take the pre-processed source domain image and target domain image as the input of the generative adversarial network model, output the corresponding synthetic simulation image, and perform blur detection, reasonableness check and artificial sampling inspection on the synthetic simulation image to obtain the final synthetic simulation image;

[0012] SA3: image enhancement: according to the imaging conditions of the current batch production line, set the interference noise of the simulated production line, and through the interference noise of the simulated production line, perform noise processing on the final synthetic simulation image to obtain a high-fidelity simulation image;

[0013] The model decision module: according to the high-fidelity simulation image and the real image of the current batch, the pre-training meta-learning model is trained, and the detection and recognition model of the current batch is obtained, and the detection result of the current batch is obtained through the detection and recognition model.

[0014] Further, the physical perception module acquires pork can image through the main vision system and the auxiliary sensor system, the main vision system includes an industrial camera, the auxiliary sensor system includes a 3D depth camera and / or a spectral camera, and the main vision system and the auxiliary sensor system are arranged side by side on a rigid support, and the rigid support is fixed above the transmission belt.

[0015] Further, the lens of the industrial camera is provided with a ring-shaped LED shadowless lamp, the illumination mode of the ring-shaped LED shadowless lamp is set as coaxial illumination or low-angle illumination, and the ring-shaped LED shadowless lamp is powered by a constant-current power supply.

[0016] Further, according to the comparison between the Laplacian variance of the synthetic simulation image and the preset variance threshold, the blur degree is detected, specifically:

[0017] When the Laplacian variance is lower than the preset variance threshold, the corresponding synthetic simulation image is deleted; otherwise, the corresponding synthetic simulation image is retained;

[0018] The synthetic simulation images retained in the blur degree detection process are reasonably checked by the impurity classification model, the synthetic simulation images with unreasonable impurity positions are deleted, and the synthetic simulation images with reasonable impurity positions are retained;

[0019] In the process of reasonable inspection, a preset number of image samples are extracted from the retained synthetic simulation images, and according to the comparison between the sampling qualified rate of the image samples and the preset sampling rate, artificial sampling is carried out, specifically:

[0020] When the sampling qualified rate is greater than the preset sampling rate, the retained synthetic simulation image is the final synthetic simulation image; otherwise, the synthetic simulation image is reacquired through the generative adversarial network model until the corresponding final synthetic simulation image is obtained.

[0021] Further, the noise processing of the final synthetic simulation image includes illumination change processing, occlusion and interference simulation processing, motion blur processing and salt and pepper noise processing.

[0022] Further, the detection result of the current batch is obtained, including:

[0023] SB1: Construct a recognition model: according to the high-fidelity simulation image, the pre-training meta-learning model is trained to obtain the meta-learning model weight, and the current batch of high-fidelity simulation images and real images are cross mixed to obtain the training data, and the training data is input into the pre-training meta-learning model with the meta-learning model weight for iterative training to obtain the detection recognition model of the current batch;

[0024] SB2: Modal decision: the pork can images collected by the main visual system and the auxiliary sensor system are respectively input into the detection recognition model, the current pork can image confidence corresponding to the industrial camera, the 3D depth camera and the spectral camera is output, and the recognition result of the pork can is determined according to the comparison result between the current pork can image confidence and the preset confidence threshold, specifically:

[0025] When the current pork can image confidence is greater than the preset confidence threshold, the current pork can image is defect-free, that is, the corresponding pork can is defect-free; otherwise, the current pork can image has defects, that is, the corresponding pork can has defects.

[0026] Further, the current pork can image confidence corresponding to the industrial camera is compared with the preset confidence threshold, and the recognition result of the pork can is determined according to the comparison result, specifically:

[0027] When the current pork can image confidence corresponding to the industrial camera is greater than the preset confidence threshold, the current pork can image is defect-free, that is, the corresponding pork can is defect-free; otherwise, the recognition result of the pork can is determined through the current pork can image confidence corresponding to the 3D depth camera and the spectral camera;

[0028] According to the current pork can image confidence corresponding to the 3D depth camera and the spectral camera, the auxiliary image confidence is obtained, and the auxiliary image confidence is compared with the preset confidence threshold, and the recognition result of the pork can is determined according to the comparison result, specifically:

[0029] When the auxiliary image confidence is greater than the preset confidence threshold, the current pork can image is defect-free, that is, the corresponding pork can is defect-free; otherwise, the current pork can image has defects, that is, the corresponding pork can has defects.

[0030] Further, the auxiliary image confidence is determined according to the current pork can image confidence corresponding to the 3D depth camera and the spectral camera, specifically:

[0031] SB2.1: determining the confidence weight: according to the current pork can image confidence corresponding to the 3D depth camera and the spectral camera, the sum of the current pork can image confidence corresponding to the auxiliary sensor system is obtained, and the confidence weight of the main vision system and the confidence weight of the auxiliary sensor system are determined through the current pork can image confidence corresponding to the industrial camera, specifically:

[0032]

[0033] Among them: The confidence weight of the main vision system is The confidence weight of the auxiliary sensor system is The current pork can image confidence corresponding to the industrial camera is The current pork can image confidence corresponding to the 3D depth camera is The current pork can image confidence corresponding to the spectral camera is

[0034] SB2.2: determining the auxiliary image confidence: according to the confidence weight of the main vision system and the current pork can image confidence corresponding to the industrial camera, the main vision system image confidence is determined, according to the confidence weight of the auxiliary sensor system and the sum of the current pork can image confidence corresponding to the auxiliary sensor system, the auxiliary sensor system image confidence is determined, and the main vision system image confidence and the auxiliary sensor system image confidence are combined to obtain the final fusion confidence, specifically:

[0035]

[0036] Among them: The final fusion confidence is The confidence weight of the main vision system is The confidence weight of the auxiliary sensor system is The current pork can image confidence corresponding to the industrial camera is The current pork can image confidence corresponding to the 3D depth camera is The current pork can image confidence corresponding to the spectral camera is

[0037] A pork can rinsing and picking automatic detection device based on image recognition uses any one of the pork can rinsing and picking automatic detection system based on image recognition.

[0038] Compared with the prior art, the beneficial effects of the present application are:

[0039] One: the application obtains high-fidelity simulation defect images that are highly consistent with the color and texture of the current batch of pork by generating a generative adversarial network model, and then obtains the detection and recognition model of the current batch through the training of the meta-learning model, so that the material diversity problem caused by natural differences between different pig breeds, parts and batches can be overcome, false positives and false negatives are reduced, and the stability and consistency of product quality are ensured.

[0040] Secondly, the application simulates various interferences such as real light changes, water stains, steam obstructions, motion blurs and sensor noises in the image enhancement stage, so that the trained recognition model has strong fault tolerance and adaptability to complex industrial production environments, thereby ensuring the detection reliability in real harsh working conditions.

[0041] Thirdly, the application combines a multi-sensor system composed of industrial cameras, 3D depth cameras and spectral cameras with a fusion algorithm based on confidence weight, so that it can integrate the judgment results of different sensors, that is, when the confidence of the main vision system is insufficient, the auxiliary sensor system provides key verification, so that the advantages are complementary, and the accuracy and reliability of the final decision are further improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The system block diagram of the automatic detection system for rinsing and picking up the pork cans in the application;

[0043] Figure 2 The acquisition process diagram of the high-fidelity simulation image in the application;

[0044] Figure 3 The flowchart of the recognition result of the pork cans in the application;

[0045] Figure 4 The acquisition process diagram of the final fusion confidence in the application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0047] Due to the natural differences among different pig breeds, different parts of meat sources and different batches, there is also an inevitable diversity among pork raw materials, which will cause the color, texture and optical properties of the inspected object surface to present significant time-varying characteristics. Therefore, in the process of detecting the surface defects of pork, segmentation deviation is easy to occur, and false detection and missed detection phenomenon will occur, which seriously restricts the tolerance ability of the detection system to material changes. The technical scheme of the present application collects the can image through the physical perception module, and combines the generated adversarial network model to combine the historical defect image with the current batch of non-defect image, and obtains the corresponding high-fidelity simulation defect image. At the same time, through the model decision module based on the meta-learning model training, the detection and recognition model suitable for the current batch is obtained, and through the multi-modal confidence fusion decision, the precise and automatic sorting of the can containing hair and other defects is realized, which effectively improves the detection efficiency and accuracy, and overcomes the false detection and missed detection problem caused by material difference.

[0048] Embodiment 1

[0049] Reference Figures 1-3 The embodiment provides a kind of based on image recognition's pork can rinse and pick up hair automatic detection system, which includes physical perception module, virtual generation module and model decision module. Wherein physical perception module is used to obtain corresponding pork can image in the process of pork can transmission, and according to the identification result of pork can image, the pork can containing defect is sorted out. Virtual generation module will contain various defect annotations pork can image as magazine defect sample (including but not limited to containing hair, metal, plastic and other multiple magazine types), i.e. source domain image, current batch pork can image as target domain image, and source domain image and target domain image as the input of generated adversarial network model, output obtains corresponding synthetic defect image. At the same time, according to the synthetic defect image obtained, it is combined with the dynamic environment noise of simulation production line, so that corresponding simulation image data is obtained. Model decision module is used to train pre-training meta-learning model by obtaining current batch simulation image data and real image, to obtain the detection and recognition model of current batch, and detect current batch pork can by the detection and recognition model of current batch, to obtain corresponding detection result.

[0050] In the embodiment, the physical perception module identifies and obtains the image of the pork cans through the set industrial camera during the transmission of the pork cans by the conveying mechanism, and when the defect signal is obtained, the pork cans containing defects such as hair are removed from the conveying mechanism through the pneumatic / mechanical arm device. Further, in the embodiment, the 3D depth camera and / or the spectral camera (such as a multispectral camera or a hyperspectral camera) are set to collect the three-dimensional height information of the surface of the pork cans and / or the material information of the pork cans, so as to determine the defect condition of the pork cans through the collected three-dimensional height information and / or material information.

[0051] Further, a light-receiving type photoelectric sensor is arranged 10-15 cm in front of the detection station to trigger the photoelectric sensor signal during the transmission of the pork cans, and the industrial camera and the auxiliary sensor are fixed above the transmission belt through a rigid support, and the field of view of the industrial camera and the auxiliary sensor can completely cover the pork area in the can opening. Meanwhile, the actuator is arranged 50-100 cm downstream of the detection station to remove the pork cans containing defects such as hair from the transmission belt.

[0052] Further, a ring-shaped LED shadowless lamp is arranged around the lens of the industrial camera to provide uniform and shadowless illumination, so as to reduce the shadow caused by unevenness and make the defect features such as hair more prominent. It is worth noting that the ring-shaped LED shadowless lamp in the embodiment adopts coaxial illumination or low-angle illumination, and is powered by a constant-current power supply, so as to ensure the stability of the brightness of the light source and improve the contrast of the image.

[0053] Further, the auxiliary sensor system in the embodiment includes a 3D depth camera and / or a spectral camera, that is, the 3D depth camera and the spectral camera can be used as the auxiliary sensor system respectively, or can be combined as the auxiliary sensor system, and are installed in parallel with the industrial camera on the rigid support.

[0054] Further, the actuator in the embodiment is set as a pneumatic push rod controlled by a two-way electromagnetic valve, so as to remove the pork cans containing defects such as hair from the conveying belt through the pneumatic push rod according to the trigger signal of the pork cans containing defects such as hair. That is, the pork cans are screened through the pneumatic push rod.

[0055] In the embodiment, the virtual generation module obtains the current batch of canned pork images by industrial camera shooting in the physical perception module, so as to take the current batch of canned pork images without defects as the target domain images. Meanwhile, a preset number of canned pork images containing defects are obtained from the impurity defect sample library, and are taken as the source domain images, so as to obtain the corresponding synthetic defect images through the source domain images, the target domain images and the generative adversarial network model. Meanwhile, the dynamic environmental noise of the simulated production line is set according to the current batch of production line environmental noise, so as to process the synthetic defect images obtained by the generative adversarial network model according to the dynamic environmental noise of the simulated production line, thereby obtaining the corresponding simulation image data. Specifically as follows:

[0056] Step SA1: image processing. That is, in the production detection record of the canned pork production line, the canned pork images with defects detected are collected, or the defect sample images retained by shooting from the artificial re-inspection station, that is, the canned pork images containing various hair defect annotations, are collected, so as to take the obtained canned pork images containing various hair defect annotations as the source domain images. Meanwhile, when the current new batch of canned pork production line is initially running, a preset number of defect-free canned pork are artificially extracted, and the corresponding defect-free canned pork images are shot and obtained at the detection station, so as to take the preset number of defect-free canned pork images as the target domain images.

[0057] Further, the obtained canned pork images containing various hair defect annotations are annotated by setting the boundary box, that is, the defects in the defect canned pork image are framed by the rectangular frame in the boundary box, and the frame body is close to the defect edge. Meanwhile, the framed defect image and the target domain image are subjected to data preprocessing, that is, scale normalization processing, color space unification processing and data cleaning processing, so as to obtain the preprocessed defect image, that is, the source domain image and the preprocessed target domain image. It should be noted that the scale normalization processing, the color space unification processing and the data cleaning processing in the embodiment are all conventional technical means, so they will not be described in detail in the embodiment.

[0058] Step SA2: synthetic image screening. That is, the preprocessed source domain image and the target domain image obtained in step SA1 are taken as the input of the generative adversarial network model (such as StarGAN v2 model), so as to extract the impurity contour and shape of the preprocessed source domain image, and then combine the extracted impurity contour and shape with the current batch of pork color, fat texture and light feeling according to the current batch of pork color, fat texture and light feeling in the target domain image through the generative adversarial network model, so as to obtain the corresponding synthetic simulation image.

[0059] Further, the synthetic simulation images obtained by the generative adversarial network model are subjected to blurring degree detection, reasonableness check and artificial sampling inspection to filter out blurred images, images with unreasonable impurity positions and images with poor quality. Specifically, according to the Laplacian variance of each synthetic simulation image, the Laplacian variance of each synthetic simulation image is compared with a preset variance threshold to delete the synthetic simulation images lower than the preset variance threshold, i.e. remove blurred images. That is, synthetic simulation images not lower than the preset variance threshold are retained, and the retained synthetic simulation images are subjected to reasonableness check by the impurity classification model to delete synthetic simulation images with unreasonable impurity positions by the impurity classification model and retain synthetic simulation images with reasonable impurity positions. At the same time, 1-5% of the image samples of the retained synthetic simulation images are randomly sampled, and the sampled image samples are subjected to quality inspection by quality inspectors to obtain the corresponding sampling qualified rate. That is, when the obtained sampling qualified rate is greater than a preset sampling rate, the finally retained synthetic simulation images are the final synthetic simulation images. Otherwise, the generative adversarial network model is used to reacquire synthetic simulation images, and the corresponding final synthetic simulation images are obtained. It is worth noting that the Laplacian variance of each synthetic simulation image and the obtained sampling qualified rate in this embodiment are conventional technical solutions, so they are not described in detail in this embodiment. At the same time, the preset variance threshold and the preset sampling rate in this embodiment are set according to actual data requirements, so they are not described in detail in this embodiment.

[0060] Step SA3: image enhancement. That is, according to the imaging conditions in the current batch production line, the interference noise in the simulation production line is set to perform noise processing on the final synthetic simulation images obtained in step SA2 by the set interference noise, thereby generating corresponding high-fidelity simulation images. In this embodiment, a random number generator is used to generate configuration parameters corresponding to each final synthetic simulation image. That is, according to the set configuration parameters, each final synthetic simulation image is added with corresponding noise processing, thereby obtaining simulation images under different interference noise environments.

[0061] Specifically, the noise enhancement processing manner in this embodiment includes illumination change enhancement processing, occlusion and interference simulation processing, motion blur processing and sensor noise processing. Among them, according to the set configuration parameters, the corresponding configuration parameters are multiplied on the image brightness and contrast to simulate the light aging, voltage fluctuation or reflectivity difference of the can surface caused by the light difference, so as to perform random adjustment of the brightness / contrast of the image. At the same time, through the adjustment of the Gamma value, the non-linear change of the camera response curve is simulated, so as to perform Gamma correction adjustment of the image. That is, through the random adjustment of brightness / contrast and Gamma correction adjustment, the illumination change processing of the image is realized.

[0062] Further, a white highlight area with irregular shape and semi-transparency is superimposed on the synthetic simulation image at a random position, and its brightness and transparency can be randomly set, that is, the specular reflection caused by water droplets or oil stains is simulated, so as to perform water stain reflection processing of the image. At the same time, a white fog-like texture with semi-transparency and edge blur is superimposed on the synthetic simulation image, and its transparency can be randomly changed, that is, the contrast reduction and detail blur caused by steam are simulated, so as to perform steam occlusion processing of the image. That is, through the water stain reflection processing and steam occlusion processing, the occlusion and interference simulation processing of the image is realized.

[0063] Further, according to the running direction of the conveyor belt, a linear motion blur in the corresponding direction is applied to the synthetic simulation image, and the kernel length of the linear motion blur can be randomly set according to the speed range of the conveyor belt, so as to simulate the smear caused by the relative motion between the pork can and the camera, thereby performing motion blur processing of the image.

[0064] Further, a weak and Gaussian-distributed random value is added to all image pixels of the synthetic simulation image to simulate the electronic noise generated by the camera sensor in the signal acquisition and amplification process, thereby performing Gaussian noise processing of the image. At the same time, a small number of pixel points in the image are converted to pure white or pure black to simulate the bad points on the sensor or the occasional errors in data transmission, thereby performing salt and pepper noise processing of the image.

[0065] It is worth noting that in this embodiment, when the image is subjected to illumination change processing, occlusion and interference simulation processing, motion blur processing and salt and pepper noise processing, they are all conventional technical means processing methods, so this embodiment does not perform specific description and illustration.

[0066] In this embodiment, the model decision module constructs the corresponding detection and recognition model through the training of the pre-trained meta-learning model, and detects the current batch of pork cans through the set detection and recognition model to obtain the corresponding detection result. Specifically as follows:

[0067] Step SB1: Constructing a recognition model. That is, according to the simulation images processed by noise obtained in step SA3, the pre-trained meta-learning model (such as Model-Agnostic Meta-Learning model) is trained to obtain the corresponding meta-learning model weight, that is, the initial weight of the pre-trained meta-learning model, according to the simulation images processed by noise.

[0068] Further, the current batch of simulation image data and real images are cross-mixed as training data, and the obtained training data is input into the pre-trained meta-learning model with initial weight, and the pre-trained meta-learning model is iteratively trained, so that the trained meta-learning model can be obtained. That is, the obtained trained meta-learning model is the detection and recognition model of the current batch.

[0069] Step SB2: Modal decision. That is, the detection and recognition model of the current batch obtained in step SB1 is used to analyze the current batch of pork can images, so as to determine the defect condition of the pork can according to the analysis result. And in the case that the pork can has defects, the execution mechanism in the physical perception module is used to remove the pork can with defects from the transmission belt.

[0070] Specifically, the pork can image obtained by the industrial camera is input into the detection and recognition model, and the confidence of the current pork can image is output. At the same time, the confidence of the obtained pork can image is compared with the pre-set confidence threshold (which can be set according to the actual production line image data, so it is not specifically described in this embodiment, for example, 0.95), and the corresponding pork can recognition result is determined according to the comparison result. Specifically:

[0071] When the confidence of the current pork can image is greater than the pre-set confidence threshold, the current pork can image has no defects, that is, the corresponding pork can has no defects. On the contrary, when the confidence of the current pork can image is not greater than the pre-set confidence threshold, the current pork can image obtained by the auxiliary sensor system is input into the detection and recognition model, and the confidence of the auxiliary image is output. That is, the current pork can image is obtained by the 3D depth camera and / or the spectral camera, and the confidence of the auxiliary image is obtained again according to the pork can image obtained by the auxiliary sensor system. That is, according to the comparison result between the confidence of the auxiliary image and the pre-set confidence threshold (which can be set according to the actual production line image data, so it is not specifically described in this embodiment, for example, 0.95), the corresponding pork can recognition result is further determined. Specifically:

[0072] When the auxiliary image confidence is greater than the preset confidence threshold, then the current pork can image is defect-free, that is, the corresponding pork can is defect-free. Conversely, when the auxiliary image confidence is not greater than the preset confidence threshold, then the current pork can image has defects, that is, the corresponding pork can has defects, and the pork can containing defects is removed from the transmission belt by the actuator at this time.

[0073] The embodiment also provides a pork can rinsing and picking automatic detection device based on image recognition, which uses the pork can rinsing and picking automatic detection system based on image recognition.

[0074] Embodiment 2

[0075] The embodiment provides a pork can rinsing and picking automatic detection system based on image recognition, and the specific implementation method is the same as that of embodiment 1, and the difference lies in that the auxiliary sensor system can be separately provided as a 3D depth camera or a spectrum camera, or can be provided as a combination of a 3D depth camera and a spectrum camera. Specifically, the auxiliary sensor system in the embodiment is provided as a combination of a 3D depth camera and a spectrum camera. That is, the pork can images obtained by the 3D depth camera and the spectrum camera respectively, and the corresponding confidence sizes thereof are combined, so that the corresponding auxiliary image confidence can be obtained. The application is exemplified by combining the specific implementation of the embodiment.

[0076] Reference Figure 4 In the embodiment, the corresponding auxiliary image confidence is determined according to the pork can image confidences corresponding to the 3D depth camera and the spectrum camera, and specifically as follows:

[0077] Step SB2.1: Determine the confidence weight. That is, the pork can image obtained by the 3D depth camera is taken as the input of the detection and recognition model, and the current pork can image confidence corresponding to the 3D depth camera is output. The pork can image obtained by the spectrum camera is taken as the input of the detection and recognition model, and the current pork can image confidence corresponding to the spectrum camera is output. At the same time, the current pork can image confidences corresponding to the 3D depth camera and the spectrum camera are combined, and the sum of the current pork can image confidences corresponding to the auxiliary sensor system is obtained.

[0078] Further, the confidence weight of the industrial camera main vision system and the confidence weight of the auxiliary sensor system are determined according to the sum of the current pork can image confidences corresponding to the auxiliary sensor system and the current pork can image confidence corresponding to the industrial camera, and specifically as follows:

[0079]

[0080] wherein: is the confidence weight of the primary vision system, is the confidence weight of the auxiliary sensor system, is the current pork can image confidence corresponding to the industrial camera, is the current pork can image confidence corresponding to the 3D depth camera, is the current pork can image confidence corresponding to the spectral camera.

[0081] In the process of specific implementation, the current pork can image confidence corresponding to the industrial camera is 0.85, the current pork can image confidence corresponding to the 3D depth camera is 0.25, and the current pork can image confidence corresponding to the spectral camera is 0.3, so the confidence weight of the corresponding primary vision system is about 0.61, and the confidence weight of the corresponding auxiliary sensor system is about 0.39.

[0082] Step SB2.2: Determine the auxiliary image confidence. That is, the confidence weight of the primary vision system obtained is combined with the current pork can image confidence corresponding to the industrial camera to determine the corresponding primary vision system image confidence. At the same time, the confidence weight of the auxiliary sensor system obtained is combined with the sum of the current pork can image confidence corresponding to the auxiliary sensor system to determine the corresponding auxiliary sensor system image confidence.

[0083] Further, the primary vision system image confidence and the auxiliary sensor system image confidence are combined to obtain the corresponding auxiliary image confidence, that is, the final fusion confidence is obtained, which is specifically:

[0084]

[0085] wherein: is the final fusion confidence, is the confidence weight of the primary vision system, is the confidence weight of the auxiliary sensor system, is the current pork can image confidence corresponding to the industrial camera, is the current pork can image confidence corresponding to the 3D depth camera, is the current pork can image confidence corresponding to the spectral camera.

[0086] In the process of specific implementation, the current pork can image confidence of the industrial camera is 0.85, the confidence weight of the main vision system is about 0.61, and the corresponding image confidence of the main vision system is 0.5185. At the same time, the current pork can image confidence of the 3D depth camera is 0.25, the current pork can image confidence of the spectral camera is 0.3, the sum of the current pork can image confidence of the auxiliary sensor system is 0.55, and the confidence weight of the auxiliary sensor system is about 0.39, and the corresponding image confidence of the auxiliary sensor system is 0.2145. That is, the corresponding final fusion confidence is 0.733.

[0087] Although the embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An image recognition-based automatic detection system for rinsing and picking up hairs of canned pork, characterized by, The method comprises the following steps: A physical perception module: obtains images of pork cans in the transmission process, and sorts pork cans containing defects according to the recognition result of the pork can images; A virtual generation module: takes the pork can images containing defects as source domain images, takes the current batch of pork can images as target domain images, and takes both the source domain images and the target domain images as inputs of a generative adversarial network model, and outputs corresponding synthetic defect images, comprising: SA1: image processing: collect the pork can images detected with defects, and take the pork can images containing defects as source domain images, extract and photograph a predetermined number of defect-free pork cans in the current batch production line, obtain defect-free pork can images, take the predetermined number of defect-free pork can images as target domain images, and pre-process the source domain images and the target domain images; SA2: synthetic image screening: take the pre-processed source domain images and target domain images as inputs of a generative adversarial network model, output corresponding synthetic simulation images, and perform blur detection, reasonableness check and artificial sampling on the synthetic simulation images to obtain final synthetic simulation images; SA3: image enhancement: according to the imaging conditions of the current batch production line, set the interference noise of the simulation line, and perform noise processing on the final synthetic simulation images through the interference noise of the simulation line to obtain high-fidelity simulation images; A model decision module: trains a pre-trained meta-learning model according to the high-fidelity simulation images and real images of the current batch, obtains a detection and recognition model for the current batch, and obtains the detection result of the current batch through the detection and recognition model.

2. The image recognition-based automatic detection system for rinsing and picking up hair of pork cans according to claim 1, characterized in that, The physical perception module acquires pork can images through a main vision system and an auxiliary sensor system, the main vision system comprises an industrial camera, the auxiliary sensor system comprises a 3D depth camera and / or a spectral camera, the main vision system and the auxiliary sensor system are arranged side by side on a rigid support, and the rigid support is fixed above a transmission belt.

3. The image recognition-based automatic detection system for rinsing and picking up hair of pork cans according to claim 2, characterized in that, The lens of the industrial camera is provided with a ring-shaped LED shadowless lamp, the illumination mode of the ring-shaped LED shadowless lamp is set to coaxial illumination or low-angle illumination, and the ring-shaped LED shadowless lamp is powered by a constant-current power supply.

4. The image recognition-based automatic detection system for rinsing and picking up hair of pork cans according to claim 1, characterized in that, According to the comparison between the Laplacian variance of the synthetic simulation image and the preset variance threshold, the blur detection is performed, specifically: When the Laplacian variance is lower than the preset variance threshold, the corresponding synthetic simulation image is deleted; otherwise, the corresponding synthetic simulation image is retained; The synthetic simulation images retained in the blur detection process are subjected to reasonableness check through an impurity classification model, the synthetic simulation images with unreasonable impurity positions are deleted, and the synthetic simulation images with reasonable impurity positions are retained; In the process of reasonableness check, a predetermined number of image samples are extracted from the synthetic simulation images retained in the process of reasonableness check, and according to the comparison between the sampling qualified rate of the image samples and the preset sampling rate, artificial sampling is performed, specifically: When the sampling inspection qualified rate is greater than the preset sampling inspection rate, the reserved synthetic simulation image is the final synthetic simulation image; otherwise, the synthetic simulation image is reacquired through the generative adversarial network model until the corresponding final synthetic simulation image is acquired.

5. The image recognition-based automatic detection system for rinsing and picking up hair of pork cans according to claim 1, characterized in that, The noise processing on the final synthetic simulation image includes illumination change processing, occlusion and interference simulation processing, motion blur processing and salt and pepper noise processing.

6. The image recognition-based automatic detection system for rinsing and picking up hair of pork cans according to claim 1, characterized in that, The detection result of the current batch is acquired, including: SB1: constructing a recognition model: training the pre-training meta-learning model according to the high-fidelity simulation image, acquiring the meta-learning model weight, and cross-mixing the current batch of high-fidelity simulation images and real images to acquire training data, and inputting the training data into the pre-training meta-learning model with the meta-learning model weight as an input to perform iterative training to acquire the detection recognition model of the current batch; SB2: modal decision: the pork can image collected by the main visual system and the auxiliary sensor system is respectively input into the detection recognition model, the current pork can image confidence of the industrial camera, the 3D depth camera and the spectral camera is output, and the recognition result of the pork can is determined according to the comparison result between the current pork can image confidence and the preset confidence threshold, specifically: When the current pork can image confidence is greater than the preset confidence threshold, the current pork can image is defect-free, that is, the corresponding pork can is defect-free; otherwise, the current pork can image has defects, that is, the corresponding pork can has defects.

7. The image recognition-based automatic detection system for rinsing and picking up hair of pork cans according to claim 6, characterized in that, The current pork can image confidence of the industrial camera is compared with the preset confidence threshold, and the recognition result of the pork can is determined according to the comparison result, specifically: When the current pork can image confidence of the industrial camera is greater than the preset confidence threshold, the current pork can image is defect-free, that is, the corresponding pork can is defect-free; Otherwise, the recognition result of the pork can is determined through the current pork can image confidence of the 3D depth camera and the spectral camera; The auxiliary image confidence is acquired according to the current pork can image confidence of the 3D depth camera and the spectral camera, the auxiliary image confidence is compared with the preset confidence threshold, and the recognition result of the pork can is determined according to the comparison result, specifically: When the auxiliary image confidence is greater than the preset confidence threshold, the current pork can image is defect-free, that is, the corresponding pork can is defect-free; otherwise, the current pork can image has defects, that is, the corresponding pork can has defects.

8. The image recognition-based automatic detection system for rinsing and picking up hair of pork cans according to claim 6 or 7, characterized in that, The auxiliary image confidence is determined according to the current pork can image confidence of the 3D depth camera and the spectral camera, specifically: SB2.1: determining the confidence weight: the sum of the current pork can image confidence of the auxiliary sensor system is acquired according to the current pork can image confidence of the 3D depth camera and the spectral camera, and the confidence weight of the main visual system and the confidence weight of the auxiliary sensor system are determined through the current pork can image confidence of the industrial camera, specifically: wherein: is a confidence weight for the primary vision system, is a confidence weight for the auxiliary sensor system, is a current pork can image confidence corresponding to the industrial camera, is a current pork can image confidence corresponding to the 3D depth camera, is a current pork can image confidence corresponding to the spectral camera; SB2.2: determining the auxiliary image confidence: determining the main vision system image confidence according to the confidence weight of the main vision system and the current pork can image confidence corresponding to the industrial camera, determining the auxiliary sensor system image confidence according to the sum of the confidence weight of the auxiliary sensor system and the current pork can image confidence corresponding to the auxiliary sensor system, and combining the main vision system image confidence and the auxiliary sensor system image confidence to obtain the final fusion confidence, specifically: wherein: is the final fusion confidence, is the confidence weight of the primary vision system, is the confidence weight of the secondary sensor system, is the current pork can image confidence corresponding to the industrial camera, is the current pork can image confidence corresponding to the 3D depth camera, is the current pork can image confidence corresponding to the spectral camera.

9. An image recognition-based automatic detection device for rinsing and picking up hairs of pork cans, characterized by, The pork can rinsing and picking automatic detection system based on image recognition in any one of claims 1-8 is used.

Images