Can detection method and system based on fusion of machine vision and intelligent sensor
By using the fusion technology of machine vision and intelligent sensors for canned testing on the tomato sauce canned production line, the problem of low manual verification efficiency is solved, and the appearance, quantity and weight of canned food is accurately detected, and product quality and production efficiency are improved.
Patent Information
- Application Number
- CN202410974093.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The verification and inspection of the existing tomato sauce canned production lines mainly relies on manual labor, resulting in low detection efficiency and low accuracy, and the inability to accurately detect the appearance defects of the canned food, affecting product quality and production efficiency.
The fusion technology based on machine vision and intelligent sensors is adopted to detect the appearance, quantity and weight of cans before and after packing through machine vision algorithms and intelligent sensors to achieve accurate detection and improve detection efficiency and automation.
It improves the accuracy and production efficiency of appearance inspection after canning, and improves the quality of tomato sauce canned products and the degree of automation of production lines.
Smart Images

Figure CN118904753A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of canned food detection, and in particular to a canned food detection method and system based on the fusion of machine vision and intelligent sensors. Background Art
[0002] At present, the production line of canned ketchup products has been basically automated, that is, the cleaning, selection, crushing, heating, canning, and packaging of raw tomato materials are completed through program-controlled automated equipment. However, in addition to the above production processes, the inspection of intermediate products in the production line of canned ketchup products will also directly affect the quality of the canned ketchup products produced. The inspection work on the existing canned ketchup product production line is all done manually;
[0003] However, for the sake of production efficiency of the production line, in the canning and boxing process that directly affects the quality of the canned tomato sauce products that finally come out of the line, manual inspection can only perform a rough inspection of the appearance after canning, or inspect the number of boxes after boxing. Manual inspection not only has low inspection efficiency, resulting in insufficient production efficiency of canned tomato sauce products, but also cannot accurately inspect the appearance defects of canned tomato sauce, which in turn leads to low inspection accuracy.
[0004] Therefore, the present invention proposes a canned food detection method and system based on the fusion of machine vision and intelligent sensor. Summary of the invention
[0005] The present invention provides a can detection method and system based on the fusion of machine vision and intelligent sensors, which are used to realize accurate detection of the appearance and weight of cans before canning and the appearance and quantity after canning based on machine vision algorithms and intelligent sensors, thereby improving the accuracy of appearance detection after canning and boxing, and also improving the detection efficiency of cans, thereby improving the production efficiency of canned ketchup products, and also improving the degree of automation of the canned ketchup production line.
[0006] The present invention provides a canned food detection method based on the fusion of machine vision and intelligent sensor, comprising:
[0007] S1: Perform pre-packing inspection on each can passing through the entrance of the packing process workshop based on machine vision algorithms and intelligent sensors to obtain pre-packing inspection results for each can passing through the entrance of the packing process workshop;
[0008] S2: Based on the pre-packing inspection results of each can, all cans that have passed the pre-packing inspection are screened out from all the cans that have passed the entrance of the packing process workshop, and all cans that have passed the pre-packing inspection are controlled to pass through the packing process;
[0009] S3: Based on the machine vision algorithm, detect whether the quantity of cans in all can boxes passing through the exit of the packing process workshop is qualified, and obtain the packing quantity detection result of each can box. At the same time, based on the machine vision algorithm, detect whether the appearance of all cans that have passed the pre-packing inspection in all can boxes passing through the exit of the packing process workshop is qualified, and obtain the post-packing appearance detection result of each can that has passed the pre-packing inspection in each can box;
[0010] S4: Based on the packing quantity inspection results of all canned food boxes passing through the packing process workshop outlet and the post-packing appearance inspection results of all qualified cans in the corresponding canned food boxes before packing, all the final qualified canned food boxes are screened out from all the canned food boxes passing through the packing process workshop outlet.
[0011] Preferably, the canned food inspection method based on the fusion of machine vision and intelligent sensor, S1: based on the machine vision algorithm and the intelligent sensor, each canned food passing through the entrance of the packing process workshop is inspected before packing, and the pre-packing inspection result of each canned food passing through the entrance of the packing process workshop is obtained, including:
[0012] Based on the machine vision algorithm, the appearance of all cans passing through the entrance of the packing process workshop is detected to see if they are qualified, and the appearance inspection results of each can before packing are obtained;
[0013] Based on the intelligent sensor, the weight of all cans passing through the entrance of the packing process workshop is detected to see if they are qualified, and the weight test results of each can before packing are obtained;
[0014] The pre-packing inspection result of each can includes the pre-packing appearance inspection result and the pre-packing weight inspection result of each can.
[0015] Preferably, the canned food inspection method based on the fusion of machine vision and intelligent sensor detects whether the appearance of all canned food passing through the entrance of the packing process workshop is qualified based on the machine vision algorithm, and obtains the appearance inspection result of each canned food before packing, including:
[0016] Extracting the image outer contour of each unidirectional appearance image of each can passing through the entrance of the packing process workshop, and dividing each image outer contour into a plurality of contour segments without inflection points based on all inflection points in each image outer contour;
[0017] Combining any contour segment without inflection point in the image outer contour of each unidirectional appearance image of each can with any standard contour segment without inflection point in the corresponding unidirectional standard image outer contour, to obtain a combination result of multiple combination methods of the image outer contour of each unidirectional appearance image of each can;
[0018] Determine a matching contour combination of the image outer contour of the corresponding unidirectional appearance image of the corresponding can from the combination results of all combinations of the image outer contours of each unidirectional appearance image of each can;
[0019] Based on the matching contour combination of the image outer contours of each unidirectional appearance image of all cans passing through the entrance of the packing process workshop, all cans that have passed the first step deformation detection are screened out from all cans that have passed the entrance of the packing process workshop, and the pre-packing appearance inspection results of all cans that have passed the first step deformation detection among all cans that have passed the entrance of the packing process workshop are set as unqualified pre-packing appearance;
[0020] Determine the pre-packing appearance inspection results of each can that passes the first step deformation inspection at the entrance of the packing process workshop.
[0021] Preferably, the can detection method based on the fusion of machine vision and intelligent sensor, in the combination results of all combination modes of the image outer contours of each unidirectional appearance image of each can, determines the matching contour combination of the image outer contours of the corresponding unidirectional appearance image of the corresponding can, including:
[0022] Calculate the overall matching degree of the combination results of each combination method, including:
[0023]
[0024] In the formula, σ all is the overall matching degree of the combination result of the currently calculated combination method, p is the total number of contour segment combinations contained in the combination result of the currently calculated combination method, and m t is the total number of pixels contained in the contour segment without inflection point in the tth contour segment combination contained in the combination result of the currently calculated combination method, n t is the total number of pixels contained in the standard contour segment without inflection point in the tth contour segment combination contained in the combination result of the currently calculated combination mode, min(m t , n t ) is to take m t and n t The minimum value in max(m t , n t ) is to take m t and n t The maximum value in Pv ti1 Pv is the pixel value of the i-th pixel point in the contour segment without inflection point in the t-th contour segment combination included in the combination result of the currently calculated combination method, sorted in a preset manner, ti2 is the pixel value of the i-th pixel point in the standard contour segment without inflection point in the t-th contour segment combination included in the combination result of the currently calculated combination method, which is sorted in a preset manner;
[0025] Among all combination results of the image outer contours of each unidirectional appearance image of each can, all contour segment combinations contained in the combination result with the greatest specific matching degree are regarded as the matching contour combination of the image outer contour of the corresponding unidirectional appearance image of the corresponding can.
[0026] Preferably, the can detection method based on the fusion of machine vision and intelligent sensor, based on the matching contour combination of the image outer contour of each unidirectional appearance image of all cans passing through the entrance of the packing process workshop, screens out all cans that pass the first step of deformation detection from all cans passing through the entrance of the packing process workshop, including:
[0027] Calculate the specific matching degree of each matching profile combination, including:
[0028]
[0029] σ detail is the specific matching degree of the currently calculated matching contour combination, m is the total number of pixels contained in the contour segment without inflection points in the currently calculated matching contour combination, n is the total number of pixels contained in the standard contour segment without inflection points in the currently calculated matching contour combination, min(m, n) is the minimum value of m and n, Pv i1 Pv is the pixel value of the i-th pixel point in the contour segment without inflection point in the currently calculated matching contour combination, sorted in a preset manner, i2 is the pixel value of the i-th pixel point in the standard contour segment without inflection point in the matching contour combination currently calculated and sorted in a preset manner, and max(m, n) is the maximum value of m and n;
[0030] Among all the cans that have passed the entrance of the packing workshop, all cans whose specific matching degrees of all matching contour combinations in the combination results of all combinations of the image outer contours of all the corresponding unidirectional appearance images are not less than a specific matching degree threshold are regarded as all the cans that have passed the first step of deformation detection.
[0031] Preferably, the can inspection method based on the fusion of machine vision and intelligent sensor determines the appearance inspection results before packing of each can that passes the first step deformation inspection at the entrance of the packing process workshop, including:
[0032] All unidirectional appearance pictures of each can that has passed the first step deformation test at the entrance of the packing process workshop are spliced to obtain a full-view appearance picture of each can that has passed the first step deformation test at the entrance of the packing process workshop;
[0033] Based on the all-round appearance picture of each can that passes the first step deformation test at the entrance of the packing process workshop, a second step deformation test and a dirt test are performed on each can that passes the first step deformation test at the entrance of the packing process workshop to obtain the pre-packing appearance test results of each can that passes the first step deformation test at the entrance of the packing process workshop.
[0034] Preferably, the can inspection method based on the fusion of machine vision and intelligent sensor is based on the omnidirectional appearance image of each can that has passed the first step deformation inspection at the entrance of the packing process workshop, and the second step deformation inspection and dirt inspection are performed on each can that has passed the first step deformation inspection at the entrance of the packing process workshop, and the pre-packing appearance inspection results of each can that has passed the first step deformation inspection at the entrance of the packing process workshop are obtained, including:
[0035] Inputting the omnidirectional appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop into the deformation detection model, obtaining the second step deformation detection result of each can that has passed the first step deformation test at the entrance of the packing process workshop;
[0036] Inputting the omnidirectional appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop into the dirt detection model, obtaining the dirt detection result of each can that has passed the first step deformation test at the entrance of the packing process workshop;
[0037] When the corresponding second step deformation detection result is deformation or the corresponding dirt detection result is dirt, the pre-packing appearance detection result of the can that passed the first step deformation detection is set as unqualified pre-packing appearance;
[0038] When the corresponding second-step deformation detection result is that there is no deformation and the corresponding dirt detection result is that there is no dirt, the pre-packing appearance detection result of the can that passed the first-step deformation detection is set as qualified pre-packing appearance.
[0039] Preferably, the canned food inspection method based on the fusion of machine vision and intelligent sensor, S2: based on the pre-packing inspection result of each canned food, all the canned food that have passed the pre-packing inspection are screened out from all the canned food that have passed the entrance of the packing process workshop, and all the canned food that have passed the pre-packing inspection are controlled to pass the packing process, including:
[0040] Obtaining the pre-packing appearance inspection result and the pre-packing weight inspection result of each can from the pre-packing inspection result of each can;
[0041] Among all the cans that pass the entrance of the packing process workshop, all cans that have passed the pre-packing appearance inspection and the pre-packing weight inspection shall be regarded as all the cans that have passed the pre-packing inspection, and all the cans that have passed the pre-packing inspection shall be controlled to pass the packing process.
[0042] Preferably, the canned food inspection method based on the fusion of machine vision and intelligent sensor, S4: based on the packing quantity inspection results of all canned food boxes passing through the packing process workshop exit and the post-packing appearance inspection results of all cans that have passed the pre-packing inspection in the corresponding canned food boxes, all the final qualified canned food boxes are screened out from all the canned food boxes that have passed through the packing process workshop exit, including:
[0043] Among all the cans exported from the packing workshop, all the cans with qualified can quantity in the packing quantity test results are selected as preliminary qualified cans;
[0044] Among all the preliminary qualified canned food boxes, all the preliminary qualified canned food boxes whose post-packing appearance inspection results of all the qualified cans before packing in the corresponding preliminary qualified canned food boxes are qualified after packing appearance are screened out as all the final qualified canned food boxes.
[0045] The present invention provides a canned food detection system based on the fusion of machine vision and intelligent sensors, which is used to execute the canned food detection method based on the fusion of machine vision and intelligent sensors described in any one of Embodiments 1 to 9, including:
[0046] A pre-packing inspection module is used to perform pre-packing inspection on each can passing through the entrance of the packing process workshop based on a machine vision algorithm and an intelligent sensor, and obtain the pre-packing inspection result of each can passing through the entrance of the packing process workshop;
[0047] A packing control module is used to screen out all cans that have passed the pre-packing inspection from all cans that have passed the entrance of the packing process workshop based on the pre-packing inspection results of each can, and control all cans that have passed the pre-packing inspection to pass the packing process;
[0048] The post-packing inspection module is used to detect whether the quantity of cans in all can boxes passing through the packing process workshop exit is qualified based on the machine vision algorithm, and obtain the packing quantity inspection result of each can box. At the same time, based on the machine vision algorithm, it is used to detect whether the appearance of all cans that have passed the pre-packing inspection in all can boxes passing through the packing process workshop exit is qualified, and obtain the post-packing appearance inspection result of each can that has passed the pre-packing inspection in each can box;
[0049] The factory inspection module is used to screen out all the final qualified can boxes from all the can boxes passing through the exit of the packing process workshop based on the packing quantity inspection results of all the can boxes passing through the exit of the packing process workshop and the post-packing appearance inspection results of all the qualified cans in the corresponding can boxes before packing.
[0050] Compared with the prior art, the beneficial effects of the present invention are as follows: based on machine vision algorithms and intelligent sensors, accurate detection of the appearance and weight of cans before canning and the appearance and quantity after canning is achieved, thereby improving the accuracy of appearance detection after canning and boxing, and also improving the detection efficiency of cans, thereby improving the production efficiency of canned ketchup products, and also improving the degree of automation of the canned ketchup production line.
[0051] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.
[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0054] Figure 1 This is a flow chart of a canned food detection method based on the fusion of machine vision and intelligent sensors in an embodiment of the present invention;
[0055] Figure 2 Schematic diagram of a canned food inspection system based on the fusion of machine vision and intelligent sensors in an embodiment of the present invention. DETAILED DESCRIPTION
[0056] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0057] Embodiment 1:
[0058] The present invention provides a canned food detection method based on the fusion of machine vision and intelligent sensor. Figure 1 ,include:
[0059] S1: Perform pre-packing inspection on each can passing through the entrance of the packing process workshop based on machine vision algorithms and intelligent sensors to obtain pre-packing inspection results for each can passing through the entrance of the packing process workshop;
[0060] S2: Based on the pre-packing test results of each can, all cans that have passed the pre-packing test are screened out from all the cans that have passed the entrance of the packing process workshop, and all cans that have passed the pre-packing test are controlled to pass through the packing process;
[0061] S3: Based on the machine vision algorithm, detect whether the quantity of cans in all can boxes passing through the exit of the packing process workshop is qualified, and obtain the packing quantity detection result of each can box. At the same time, based on the machine vision algorithm, detect whether the appearance of all cans that have passed the pre-packing inspection in all can boxes passing through the exit of the packing process workshop is qualified, and obtain the post-packing appearance detection result of each can that has passed the pre-packing inspection in each can box;
[0062] S4: Based on the packing quantity inspection results of all canned food boxes passing through the packing process workshop outlet and the post-packing appearance inspection results of all qualified cans in the corresponding canned food boxes before packing, all the final qualified canned food boxes are screened out from all the canned food boxes passing through the packing process workshop outlet.
[0063] The can in this embodiment mainly refers to an iron can containing ketchup.
[0064] In this embodiment, the box packing workshop is a process workshop for box packing the filled and sealed cans.
[0065] In this embodiment, the pre-packing inspection includes a pre-packing appearance inspection and a pre-packing weight inspection.
[0066] In this embodiment, controlling all the qualified cans before packing inspection to pass the packing process means that all the qualified cans before packing inspection enter the packing process for packing processing.
[0067] In this embodiment, the machine vision algorithm is used to detect whether the number of cans in all can boxes passing through the box packing process workshop is qualified, and the packing quantity detection result of each can box is obtained as follows:
[0068] Based on a shooting device arranged directly above the exit of the packing process workshop, with the camera plane parallel to the shooting angle of the conveyor belt, an image of each can of can passing through the exit of the packing process workshop is obtained;
[0069] The can box image is input into a preset can quantity statistical model to obtain the number of cans loaded in each can box as the packing quantity detection result of the corresponding can box;
[0070] Among them, the preset can quantity statistical model is a model obtained after training the neural network model by using a large number of can box images as the model input of training samples in advance, and taking the number of cans loaded in the can boxes corresponding to the large number of can box images as the model output.
[0071] In this embodiment, the method of detecting whether the appearance of all cans that have passed the pre-packing inspection in all can boxes passing the exit of the packing process workshop is qualified based on a machine vision algorithm, and obtaining the post-packing appearance inspection result of each can that has passed the pre-packing inspection in each can box is the same as the method steps and principles of "detecting whether the appearance of all cans that have passed the entrance of the packing process workshop is qualified based on a machine vision algorithm, and obtaining the pre-packing appearance inspection result of each can".
[0072] The beneficial effects of the above technology are: based on machine vision algorithms and intelligent sensors, accurate detection of the appearance and weight of cans before packing and the appearance and quantity after packing can be achieved, which improves the accuracy of appearance detection after canning and packing, and also improves the detection efficiency of cans, thereby improving the production efficiency of canned ketchup products and the degree of automation of the canned ketchup production line.
[0073] Embodiment 2:
[0074] Based on Example 1, a canned food inspection method based on the fusion of machine vision and intelligent sensors, S1: Based on the machine vision algorithm and the intelligent sensor, each can passing through the entrance of the packing process workshop is inspected before packing, and the pre-packing inspection result of each can passing through the entrance of the packing process workshop is obtained, including:
[0075] Based on the machine vision algorithm, the appearance of all cans passing through the entrance of the packing process workshop is detected to see if they are qualified, and the appearance inspection results of each can before packing are obtained;
[0076] Based on the intelligent sensor, the weight of all cans passing through the entrance of the packing process workshop is detected to see if they are qualified, and the weight test results of each can before packing are obtained;
[0077] The pre-packing inspection result of each can includes the pre-packing appearance inspection result and the pre-packing weight inspection result of each can.
[0078] In this embodiment, based on the intelligent sensor (i.e., intelligent weighing sensor), the weight of all cans passing through the entrance of the packing process workshop is detected to determine whether they are qualified, and the weight detection result of each can before packing is obtained, including:
[0079] Based on the intelligent weighing sensor, the weight of each can passing through the entrance of the packing process workshop is detected, and it is determined whether the difference between the weight of each can and the standard weight is within the threshold (for example, 10 grams). If so, the pre-packing weight detection result of the corresponding can is set to
[0080] In this embodiment, the pre-packing appearance inspection result includes two situations: the pre-packing appearance is qualified and the pre-packing appearance is unqualified.
[0081] In this embodiment, the weight detection result before packing includes two situations: qualified weight and unqualified weight.
[0082] The beneficial effects of the above technology are: based on the machine vision algorithm, the appearance and weight of all cans passing through the entrance of the packing process workshop are inspected to see if they are qualified, and the packing inspection of each can passing through the entrance of the packing process workshop is completed.
[0083] Embodiment 3:
[0084] On the basis of Example 2, a canned food inspection method based on the fusion of machine vision and intelligent sensor is used to detect whether the appearance of all canned food passing through the entrance of the packing process workshop is qualified based on the machine vision algorithm, and obtain the appearance inspection result of each canned food before packing, including:
[0085] Extracting the image outer contour of each unidirectional appearance image of each can passing through the entrance of the packing process workshop, and dividing each image outer contour into a plurality of contour segments without inflection points based on all inflection points in each image outer contour;
[0086] Combining any contour segment without inflection point in the image outer contour of each unidirectional appearance image of each can with any standard contour segment without inflection point in the corresponding unidirectional standard image outer contour, to obtain combination results of multiple combination methods of the image outer contour of each unidirectional appearance image of each can, wherein the combination result of each combination method includes multiple contour segment combinations covering all contour segments without inflection point in the corresponding image outer contour and all standard contour segments without inflection point in the corresponding standard image outer contour, and each contour segment combination includes one contour segment without inflection point in the corresponding image outer contour and one standard contour segment without inflection point in the corresponding standard image outer contour;
[0087] Determine a matching contour combination of the image outer contour of the corresponding unidirectional appearance image of the corresponding can from the combination results of all combinations of the image outer contours of each unidirectional appearance image of each can;
[0088] Based on the matching contour combination of the image outer contours of each unidirectional appearance image of all cans passing through the entrance of the packing process workshop, all cans that have passed the first step deformation detection are screened out from all cans that have passed the entrance of the packing process workshop, and the pre-packing appearance inspection results of all cans that have passed the first step deformation detection among all cans that have passed the entrance of the packing process workshop are set as unqualified pre-packing appearance;
[0089] Determine the pre-packing appearance inspection results of each can that passes the first step deformation inspection at the entrance of the packing process workshop.
[0090] In this embodiment, the one-way appearance image is an appearance image of the can obtained by photographing from a preset single shooting angle.
[0091] In this embodiment, the image outer contour of the one-way appearance image is a contour formed by the outermost circle of pixel points in the one-way appearance image.
[0092] In this embodiment, all the inflection points in the outer contour of the image are determined by screening out inflection points from all position points of the contour in the prior art.
[0093] In this embodiment, the standard contour segment without inflection points in the standard image outer contour is a partial contour segment obtained by dividing the standard image outer contour using all inflection points in the standard image outer contour.
[0094] The beneficial effect of the above technology is: by arbitrarily combining and matching the contour segments without inflection points in the image outer contour of each unidirectional appearance image of each can passing through the entrance of the packing process workshop with the standard contour segments without inflection points in the corresponding unidirectional standard image outer contour, and screening out the matching contour combination of the image outer contour of the corresponding unidirectional appearance image of each can, the matching contour combination of the image outer contour of the corresponding unidirectional appearance image of each can is used to realize the first step of deformation detection of all cans passing through the entrance of the packing process workshop, and on this basis, accurate pre-packing appearance detection of each can passing through the entrance of the packing process workshop is realized.
[0095] Embodiment 4:
[0096] On the basis of Example 3, the can detection method based on the fusion of machine vision and intelligent sensor, in the combination results of all combination methods of the image outer contours of each unidirectional appearance image of each can, determines the matching contour combination of the image outer contours of the corresponding unidirectional appearance image of the corresponding can, including:
[0097] Calculate the overall matching degree of the combination results of each combination method, including:
[0098]
[0099] In the formula, σ all is the overall matching degree of the combination result of the currently calculated combination method, p is the total number of contour segment combinations contained in the combination result of the currently calculated combination method, and m t is the total number of pixels contained in the contour segment without inflection point in the tth contour segment combination contained in the combination result of the currently calculated combination method, n t is the total number of pixels contained in the standard contour segment without inflection point in the tth contour segment combination contained in the combination result of the currently calculated combination mode, min(m t , n t ) is to take m t and n t The minimum value in max(m t , nt ) is to take m t and n t The maximum value in Pv ti1 Pv is the pixel value of the i-th pixel point in the contour segment without inflection point in the t-th contour segment combination included in the combination result of the currently calculated combination method, sorted in a preset manner, ti2 is the pixel value of the i-th pixel point in the standard contour segment without inflection point in the t-th contour segment combination included in the combination result of the currently calculated combination method, which is sorted in a preset manner;
[0100] Among all combination results of the image outer contours of each unidirectional appearance image of each can, all contour segment combinations contained in the combination result with the greatest specific matching degree are regarded as the matching contour combination of the image outer contour of the corresponding unidirectional appearance image of the corresponding can.
[0101] In this embodiment, the overall matching degree of the combination result of each combination mode represents the matching degree between the contour segments without inflection points and the standard contour segments without inflection points included in all contour segment combinations in the combination result of each combination mode.
[0102] The beneficial effect of the above technology is: by accurately calculating the overall matching degree of the combination results of each combination method, and based on the principle of screening out all contour segment combinations contained in the combination results with the largest specific matching degree, the matching contour combination of the image outer contour of each unidirectional appearance image of each can is accurately determined.
[0103] Embodiment 5:
[0104] On the basis of Example 3, the canned food inspection method based on the fusion of machine vision and intelligent sensor, based on the matching contour combination of the image outer contour of each unidirectional appearance image of all canned food passing through the entrance of the packing process workshop, screens out all canned food passing through the entrance of the packing process workshop that have passed the first step of deformation inspection, including:
[0105] Calculate the specific matching degree of each matching profile combination, including:
[0106]
[0107] σ detail is the specific matching degree of the currently calculated matching contour combination, m is the total number of pixels contained in the contour segment without inflection points in the currently calculated matching contour combination, n is the total number of pixels contained in the standard contour segment without inflection points in the currently calculated matching contour combination, min(m, n) is the minimum value of m and n, Pv i1 Pv is the pixel value of the i-th pixel point in the contour segment without inflection point in the currently calculated matching contour combination, sorted in a preset manner, i2is the pixel value of the i-th pixel point in the standard contour segment without inflection point in the matching contour combination currently calculated and sorted in a preset manner, and max(m, n) is the maximum value of m and n;
[0108] Among all the cans that have passed the entrance of the packing workshop, all cans whose specific matching degrees of all matching contour combinations in the combination results of all combinations of the image outer contours of all the corresponding unidirectional appearance images are not less than a specific matching degree threshold are regarded as all the cans that have passed the first step of deformation detection.
[0109] In this embodiment, the specific matching degree of the matching contour combination indicates the matching degree between the contour segment without inflection point included in the contour segment combination and the standard contour segment without inflection point.
[0110] In this embodiment, the specific matching degree threshold is a preset value that the specific matching degree of the qualified cans in the first step of deformation detection cannot be less than.
[0111] The beneficial effect of the above technology is: through the accurate calculation of the specific matching degree of each matching contour combination, and based on the principle that the specific matching degree is not less than the specific matching degree threshold, all cans that pass the first step deformation detection are accurately screened out from all the cans passing through the entrance of the packing process workshop.
[0112] Embodiment 6:
[0113] On the basis of Example 3, a canned food inspection method based on the fusion of machine vision and intelligent sensors is used to determine the pre-packaging appearance inspection results of each canned food that has passed the first step of deformation inspection at the entrance of the packing process workshop, including:
[0114] All unidirectional appearance images of each can that has passed the first step deformation test at the entrance of the packing process workshop are stitched together (this can be achieved by using the image stitching technology based on ORB or SURF in the prior art) to obtain a 360-degree appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop;
[0115] Based on the all-round appearance picture of each can that passes the first step deformation test at the entrance of the packing process workshop, a second step deformation test and a dirt test are performed on each can that passes the first step deformation test at the entrance of the packing process workshop to obtain the pre-packing appearance test results of each can that passes the first step deformation test at the entrance of the packing process workshop.
[0116] In this embodiment, the omnidirectional appearance image is an image including all appearance details of the cans that have passed the first step of deformation detection.
[0117] The beneficial effects of the above technology are as follows: by splicing all the unidirectional appearance images of each can that has passed the first step deformation test at the entrance of the packing process workshop, an accurate all-round appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop is obtained; and based on the all-round appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop, accurate second-step deformation detection and dirt detection of each can that has passed the first step deformation test at the entrance of the packing process workshop is achieved.
[0118] Embodiment 7:
[0119] On the basis of Example 6, a can inspection method based on the fusion of machine vision and intelligent sensors is used. Based on the omnidirectional appearance image of each can that has passed the first step deformation inspection at the entrance of the packing process workshop, a second step deformation inspection and a dirt inspection are performed on each can that has passed the first step deformation inspection at the entrance of the packing process workshop, and the pre-packing appearance inspection results of each can that has passed the first step deformation inspection at the entrance of the packing process workshop are obtained, including:
[0120] Inputting the omnidirectional appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop into the deformation detection model, obtaining the second step deformation detection result of each can that has passed the first step deformation test at the entrance of the packing process workshop;
[0121] Inputting the omnidirectional appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop into the dirt detection model, obtaining the dirt detection result of each can that has passed the first step deformation test at the entrance of the packing process workshop;
[0122] When the corresponding second step deformation detection result is deformation or the corresponding dirt detection result is dirt, the pre-packing appearance detection result of the can that passed the first step deformation detection is set as unqualified pre-packing appearance;
[0123] When the corresponding second-step deformation detection result is that there is no deformation and the corresponding dirt detection result is that there is no dirt, the pre-packing appearance detection result of the can that passed the first-step deformation detection is set as qualified pre-packing appearance.
[0124] In this embodiment, the deformation detection model is a model obtained by pre-training a neural network model using a large number of all-round appearance images of cans with deformed appearance as positive samples and a large number of all-round appearance images of cans without deformed appearance as negative samples. The model can determine whether the appearance of the corresponding can is deformed based on the input all-round appearance image of the can.
[0125] In this embodiment, the second step deformation detection results of the cans that have passed the first step deformation detection include two situations: no deformation or deformation.
[0126] In this embodiment, the dirt detection model is a model obtained by pre-training a neural network model using a large number of all-round appearance images of cans with dirt on their appearance as positive samples, and a large number of all-round appearance images of cans without dirt on their appearance as negative samples. The model can determine whether there is dirt on the appearance of the corresponding can through the input all-round appearance image of the can.
[0127] In this embodiment, the dirt detection result of the cans that pass the first step of deformation detection includes two situations: the presence of dirt or the absence of dirt.
[0128] The beneficial effects of the above technology are: based on the pre-trained deformation detection model and dirt detection model and the comprehensive appearance image of the cans that passed the first step deformation detection, accurate second-step deformation detection and accurate dirt detection are achieved for the cans that passed the first step deformation detection.
[0129] Embodiment 8:
[0130] Based on Example 2, a canned food inspection method based on the fusion of machine vision and intelligent sensors, S2: based on the pre-packing inspection result of each can, all canned food that have passed the pre-packing inspection are screened out from all canned food that have passed the entrance of the packing process workshop, and all canned food that have passed the pre-packing inspection are controlled to pass the packing process, including:
[0131] Obtaining the pre-packing appearance inspection result and the pre-packing weight inspection result of each can from the pre-packing inspection result of each can;
[0132] Among all the cans that pass the entrance of the packing process workshop, all cans that have passed the pre-packing appearance inspection and the pre-packing weight inspection shall be regarded as all the cans that have passed the pre-packing inspection, and all the cans that have passed the pre-packing inspection shall be controlled to pass the packing process.
[0133] The beneficial effect of the above technology is: based on the pre-packing appearance inspection results and pre-packing weight inspection results of each can, the appearance and weight of the cans are checked before the packing process to ensure that the appearance and weight of the cans undergoing the packing process are qualified.
[0134] Embodiment 9:
[0135] Based on Example 1, the canned food inspection method based on the fusion of machine vision and intelligent sensor, S4: based on the packing quantity inspection results of all canned food boxes passing through the packing process workshop exit and the post-packing appearance inspection results of all cans that have passed the pre-packing inspection in the corresponding canned food boxes, all the final qualified canned food boxes are screened out from all the canned food boxes that have passed through the packing process workshop exit, including:
[0136] Among all the cans exported from the packing workshop, all the cans with qualified can quantity in the packing quantity test results are selected as preliminary qualified cans;
[0137] Among all the preliminary qualified canned food boxes, all the preliminary qualified canned food boxes whose post-packing appearance inspection results of all the qualified cans before packing in the corresponding preliminary qualified canned food boxes are qualified after packing appearance are screened out as all the final qualified canned food boxes.
[0138] The beneficial effects of the above technology are: based on the packing quantity detection results of all can boxes passing the packing process workshop exit and the post-packing appearance detection results of all qualified cans in the corresponding can boxes before packing, accurate appearance detection of the cans after packing and accurate detection of the number of cans in the box after packing are achieved, thereby excluding cans whose appearance is deformed or damaged during packing, and also achieving accurate verification of the number of cans in the can box after packing.
[0139] Embodiment 10:
[0140] The present invention provides a canned food detection system based on the fusion of machine vision and intelligent sensors, which is used to execute the canned food detection method based on the fusion of machine vision and intelligent sensors described in any one of embodiments 1 to 9, referring to Figure 2 ,include:
[0141] A pre-packing inspection module is used to perform pre-packing inspection on each can passing through the entrance of the packing process workshop based on a machine vision algorithm and an intelligent sensor, and obtain the pre-packing inspection result of each can passing through the entrance of the packing process workshop;
[0142] A packing control module is used to screen out all cans that have passed the pre-packing inspection from all cans that have passed the entrance of the packing process workshop based on the pre-packing inspection results of each can, and control all cans that have passed the pre-packing inspection to pass the packing process;
[0143] The post-packing inspection module is used to detect whether the quantity of cans in all can boxes passing through the packing process workshop exit is qualified based on the machine vision algorithm, and obtain the packing quantity inspection result of each can box. At the same time, based on the machine vision algorithm, it is used to detect whether the appearance of all cans that have passed the pre-packing inspection in all can boxes passing through the packing process workshop exit is qualified, and obtain the post-packing appearance inspection result of each can that has passed the pre-packing inspection in each can box;
[0144] The factory inspection module is used to screen out all the final qualified can boxes from all the can boxes passing through the exit of the packing process workshop based on the packing quantity inspection results of all the can boxes passing through the exit of the packing process workshop and the post-packing appearance inspection results of all the qualified cans in the corresponding can boxes before packing.
[0145] The beneficial effects of the above technology are: based on machine vision algorithms and intelligent sensors, accurate detection of the appearance and weight of cans before packing and the appearance and quantity after packing can be achieved, which improves the accuracy of appearance detection after canning and packing, and also improves the detection efficiency of cans, thereby improving the production efficiency of canned ketchup products and the degree of automation of the canned ketchup production line.
[0146] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. Canned food detection method based on machine vision and intelligent sensor fusion, It is characterized in that include: S1: Perform pre-packing inspection on each can passing through the entrance of the packing process workshop based on machine vision algorithms and intelligent sensors to obtain pre-packing inspection results for each can passing through the entrance of the packing process workshop; S2: Based on the pre-packing inspection results of each can, all cans that have passed the pre-packing inspection are screened out from all the cans that have passed the entrance of the packing process workshop, and all cans that have passed the pre-packing inspection are controlled to pass through the packing process; S3: Based on the machine vision algorithm, detect whether the quantity of cans in all can boxes passing through the exit of the packing process workshop is qualified, and obtain the packing quantity detection result of each can box. At the same time, based on the machine vision algorithm, detect whether the appearance of all cans that have passed the pre-packing inspection in all can boxes passing through the exit of the packing process workshop is qualified, and obtain the post-packing appearance detection result of each can that has passed the pre-packing inspection in each can box; S4: Based on the packing quantity inspection results of all canned food boxes passing through the packing process workshop outlet and the post-packing appearance inspection results of all qualified cans in the corresponding canned food boxes before packing, all the final qualified canned food boxes are screened out from all the canned food boxes passing through the packing process workshop outlet.
2. According to claim 1, the canned food detection method based on the fusion of machine vision and intelligent sensor, It is characterized in that S1: Based on machine vision algorithms and intelligent sensors, each can passing through the entrance of the packing process workshop is inspected before packing, and the pre-packing inspection results of each can passing through the entrance of the packing process workshop are obtained, including: Based on the machine vision algorithm, the appearance of all cans passing through the entrance of the packing process workshop is detected to see if they are qualified, and the appearance inspection results of each can before packing are obtained; Based on the intelligent sensor, the weight of all cans passing through the entrance of the packing process workshop is detected to see if they are qualified, and the weight test results of each can before packing are obtained; The pre-packing inspection result of each can includes the pre-packing appearance inspection result and the pre-packing weight inspection result of each can.
3. According to claim 2, the canned food detection method based on the fusion of machine vision and intelligent sensor, It is characterized in that Based on the machine vision algorithm, the appearance of all cans passing through the entrance of the packing process workshop is detected to see if they are qualified, and the appearance inspection results of each can before packing are obtained, including: Extracting the image outer contour of each unidirectional appearance image of each can passing through the entrance of the packing process workshop, and dividing each image outer contour into a plurality of contour segments without inflection points based on all inflection points in each image outer contour; Combining any contour segment without inflection point in the image outer contour of each unidirectional appearance image of each can with any standard contour segment without inflection point in the corresponding unidirectional standard image outer contour, to obtain a combination result of multiple combination methods of the image outer contour of each unidirectional appearance image of each can; Determine a matching contour combination of the image outer contour of the corresponding unidirectional appearance image of the corresponding can from the combination results of all combinations of the image outer contours of each unidirectional appearance image of each can; Based on the matching contour combination of the image outer contours of each unidirectional appearance image of all cans passing through the entrance of the packing process workshop, all cans that have passed the first step deformation detection are screened out from all cans that have passed the entrance of the packing process workshop, and the pre-packing appearance inspection results of all cans that have passed the first step deformation detection among all cans that have passed the entrance of the packing process workshop are set as unqualified pre-packing appearance; Determine the pre-packing appearance inspection results of each can that passes the first step deformation inspection at the entrance of the packing process workshop.
4. According to claim 3, the canned food detection method based on the fusion of machine vision and intelligent sensor, It is characterized in that Determining a matching contour combination of the image outer contours of the corresponding one-way appearance images of the corresponding can from the combination results of all combination methods of the image outer contours of each one-way appearance image of each can, including: Calculate the overall matching degree of the combination results of each combination method, including: In the formula, σ all is the overall matching degree of the combination result of the currently calculated combination method, p is the total number of contour segment combinations contained in the combination result of the currently calculated combination method, and m t is the total number of pixels contained in the contour segment without inflection point in the tth contour segment combination contained in the combination result of the currently calculated combination method, n t is the total number of pixels contained in the standard contour segment without inflection point in the tth contour segment combination contained in the combination result of the currently calculated combination mode, min(m t , n t ) is to take m t and n t The minimum value in max(m t , n t ) is to take m t and n t The maximum value in Pv ti1 Pv is the pixel value of the i-th pixel point in the contour segment without inflection point in the t-th contour segment combination included in the combination result of the currently calculated combination method, sorted in a preset manner, ti2 is the pixel value of the i-th pixel point in the standard contour segment without inflection point in the t-th contour segment combination included in the combination result of the currently calculated combination method, which is sorted in a preset manner; Among all combination results of the image outer contours of each unidirectional appearance image of each can, all contour segment combinations contained in the combination result with the greatest specific matching degree are regarded as the matching contour combination of the image outer contour of the corresponding unidirectional appearance image of the corresponding can.
5. According to claim 3, the canned food detection method based on the fusion of machine vision and intelligent sensor, It is characterized in that Based on the matching contour combination of the image outer contours of each single-direction appearance image of all cans passing through the entrance of the packing process workshop, all cans that pass the first step of deformation detection are screened out from all cans passing through the entrance of the packing process workshop, including: Calculate the specific matching degree of each matching profile combination, including: σ detail is the specific matching degree of the currently calculated matching contour combination, m is the total number of pixels contained in the contour segment without inflection points in the currently calculated matching contour combination, n is the total number of pixels contained in the standard contour segment without inflection points in the currently calculated matching contour combination, min(m, n) is the minimum value of m and n, Pv i1 Pv is the pixel value of the i-th pixel point in the contour segment without inflection point in the currently calculated matching contour combination, sorted in a preset manner, i2 is the pixel value of the i-th pixel point in the standard contour segment without inflection point in the currently calculated matching contour combination, sorted in a preset manner, and max(m, n) is the maximum value of m and n; Among all the cans that have passed the entrance of the packing workshop, all cans whose specific matching degrees of all matching contour combinations in the combination results of all combinations of the image outer contours of all the corresponding unidirectional appearance images are not less than a specific matching degree threshold are regarded as all the cans that have passed the first step of deformation detection.
6. The canned food detection method based on the fusion of machine vision and intelligent sensor according to claim 3, It is characterized in that Determine the appearance inspection results before packing of each can that has passed the first step of deformation inspection at the entrance of the packing process workshop, including: All unidirectional appearance pictures of each can that has passed the first step deformation test at the entrance of the packing process workshop are spliced to obtain a full-view appearance picture of each can that has passed the first step deformation test at the entrance of the packing process workshop; Based on the all-round appearance picture of each can that passes the first step deformation test at the entrance of the packing process workshop, a second step deformation test and a dirt test are performed on each can that passes the first step deformation test at the entrance of the packing process workshop to obtain the pre-packing appearance test results of each can that passes the first step deformation test at the entrance of the packing process workshop.
7. The canned food detection method based on the fusion of machine vision and intelligent sensor according to claim 6, It is characterized in that Based on the all-round appearance picture of each can that has passed the first step deformation test at the entrance of the packing process workshop, the second step deformation test and dirt test are performed on each can that has passed the first step deformation test at the entrance of the packing process workshop, and the pre-packing appearance test results of each can that has passed the first step deformation test at the entrance of the packing process workshop are obtained, including: Inputting the omnidirectional appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop into the deformation detection model, obtaining the second step deformation detection result of each can that has passed the first step deformation test at the entrance of the packing process workshop; Inputting the omnidirectional appearance image of each can that has passed the first step deformation test at the entrance of the packing process workshop into the dirt detection model, obtaining the dirt detection result of each can that has passed the first step deformation test at the entrance of the packing process workshop; When the corresponding second step deformation detection result is deformation or the corresponding dirt detection result is dirt, the pre-packing appearance detection result of the can that passed the first step deformation detection is set as unqualified pre-packing appearance; When the corresponding second-step deformation detection result is that there is no deformation and the corresponding dirt detection result is that there is no dirt, the pre-packing appearance detection result of the can that passed the first-step deformation detection is set as qualified pre-packing appearance.
8. The canned food detection method based on the fusion of machine vision and intelligent sensor according to claim 2, It is characterized in that S2: Based on the pre-packing test results of each can, all cans that have passed the pre-packing test are screened out from all cans that have passed the packing process workshop entrance, and all cans that have passed the pre-packing test are controlled to pass the packing process, including: Obtaining the pre-packing appearance inspection result and the pre-packing weight inspection result of each can from the pre-packing inspection result of each can; Among all the cans that pass the entrance of the packing process workshop, all cans that have passed the pre-packing appearance inspection and the pre-packing weight inspection shall be regarded as all the cans that have passed the pre-packing inspection, and all the cans that have passed the pre-packing inspection shall be controlled to pass the packing process.
9. The canned food detection method based on the fusion of machine vision and intelligent sensor according to claim 1, It is characterized in that S4: Based on the packing quantity test results of all cans passing through the packing process workshop outlet and the post-packing appearance test results of all qualified cans in the corresponding cans before packing, all the final qualified cans are screened out from all the cans passing through the packing process workshop outlet, including: Among all the cans exported from the packing process workshop, all the cans with qualified can quantity in the packing quantity test results are selected as preliminary qualified cans; Among all the preliminary qualified canned food boxes, all the preliminary qualified canned food boxes whose post-packing appearance inspection results of all the qualified cans before packing in the corresponding preliminary qualified canned food boxes are qualified after packing appearance are screened out as all the final qualified canned food boxes.
10. Canned food inspection system based on machine vision and intelligent sensor fusion, It is characterized in that The method for detecting canned food based on the fusion of machine vision and intelligent sensor according to any one of claims 1 to 9 comprises: A pre-packing inspection module is used to perform pre-packing inspection on each can passing through the entrance of the packing process workshop based on a machine vision algorithm and an intelligent sensor, and obtain the pre-packing inspection result of each can passing through the entrance of the packing process workshop; A packing control module is used to screen out all cans that have passed the pre-packing inspection from all cans that have passed the entrance of the packing process workshop based on the pre-packing inspection results of each can, and control all cans that have passed the pre-packing inspection to pass the packing process; The post-packing inspection module is used to detect whether the quantity of cans in all can boxes passing through the packing process workshop exit is qualified based on the machine vision algorithm, and obtain the packing quantity inspection result of each can box. At the same time, based on the machine vision algorithm, it is used to detect whether the appearance of all cans that have passed the pre-packing inspection in all can boxes passing through the packing process workshop exit is qualified, and obtain the post-packing appearance inspection result of each can that has passed the pre-packing inspection in each can box; The factory inspection module is used to screen out all the final qualified can boxes from all the can boxes passing through the exit of the packing process workshop based on the packing quantity inspection results of all the can boxes passing through the exit of the packing process workshop and the post-packing appearance inspection results of all the qualified cans in the corresponding can boxes before packing.
