Noodle processing method and system based on multi-modal detection

Abstract

The application provides a noodle processing method and system based on multi-modal detection, relates to the technical field of data processing, and performs steam heating treatment on to-be-detected noodles, and obtains a production attribute of the to-be-detected noodles; when the production attribute is a single attribute, the corresponding to-be-detected noodles are taken as single noodles, overhead detection data of the single noodles after heating is continuously collected, single appearance detection is performed on the overhead detection data, and a first detection result is obtained; when the production attribute is a layered attribute, the corresponding to-be-detected noodles are taken as layered noodles, a circumferential detection graph of the layered noodles after heating is collected, layered appearance detection is performed on the circumferential detection graph, and a second detection result is obtained; and the first detection result or the second detection result is sent to a production end, so that customized detection can be performed on noodles with different production forms, defects such as the shape, filling displacement and structure damage of the noodles can be effectively identified, and the quality control reliability of noodle production is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for processing pasta based on multimodal detection. Background Technology

[0002] As a traditional staple food, pasta requires processes such as kneading, shaping, fermentation, and steaming in industrial production. Quality control after the pasta comes out of the oven is a key link to ensure the product qualification rate. It mainly involves testing aspects such as appearance, skin thickness, filling, and structural integrity. In particular, after the pasta has been cooked by heat treatment such as steaming, the stability of the product's forming quality directly affects the final taste experience. Therefore, quality testing of pasta after it comes out of the oven is an essential and crucial link in the production process.

[0003] Currently, the main method for testing pasta is through manual sampling. However, this method is inefficient, subjective, and difficult to meet the real-time monitoring needs of mass production. Furthermore, it usually uses a uniform testing method for all pasta, which means it cannot differentiate between pasta based on their actual production form, resulting in low testing quality and affecting user experience.

[0004] Therefore, how to conduct customized testing on freshly baked pasta based on its production process in order to effectively identify its condition and improve testing quality has become an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a method and system for processing pasta based on multimodal detection, which can perform customized detection on the pasta after it is made, so as to effectively identify the state of the pasta and thus improve the detection quality.

[0006] A first aspect of the present invention provides a pasta processing method based on multimodal detection, comprising: The pasta to be tested is subjected to steam heating treatment, and the production attributes of the pasta to be tested are obtained, including single attributes and layered attributes; When the production attribute is determined to be a single attribute, the corresponding pasta to be tested is treated as a single pasta, and the top-view detection data of the single pasta after heating is continuously collected. The top-view detection data is used to perform individual appearance detection to obtain the first detection result of the single pasta. When the production attribute is determined to be a layered attribute, the corresponding pasta to be tested is regarded as a layered pasta. A circumferential detection image of the heated layered pasta is collected, and a layered appearance detection is performed on the circumferential detection image to obtain a second detection result of the layered pasta. Send the first detection result or the second detection result to the production end.

[0007] Optionally, in one possible implementation of the first aspect, the step of performing individual appearance inspection on the top-view inspection data to generate a first inspection result for a single piece of pasta includes: Select any top view from the top view detection data as the detection top view, perform contour detection on the detection top view, and generate the contour detection result of a single piece of pasta. Water film detection is performed on the top-view inspection data to generate surface inspection results for a single piece of pasta. Based on the contour detection results and the skin detection results, the first detection result of a single piece of pasta is obtained.

[0008] Optionally, in one possible implementation of the first aspect, the step of performing contour detection on the top view to generate a contour detection result for a single piece of pasta includes: Based on OpenCV, the top view contour of a single piece of pasta in the top view is identified and detected as the actual production contour, and the center point of the actual production contour is obtained as the actual midpoint. The center point of the standard fabricated outline is retrieved as the standard midpoint. The standard midpoint is aligned with the actual midpoint to obtain the intersection of the regions of the standard fabricated outline and the actual fabricated outline, thus obtaining the common area. The actual deviation area is obtained by the difference between the contour area and the common area of ​​the actual created contour, and the actual deviation area is filled with correction pixel values. Based on the difference between the contour region and the common area of ​​the standard contour, the standard deviation region is obtained, and the standard deviation region is filled with supplementary pixel values. Based on the actual deviation area and the standard deviation area, the contour detection results of a single piece of pasta are obtained.

[0009] Optionally, in one possible implementation of the first aspect, the step of performing water film detection on the top-view inspection data to generate a surface inspection result for a single piece of pasta includes: The top-view detection data is extracted according to a preset waiting time to obtain the epidermal detection data; Based on a preset time interval, the water film area within the top-view outline of a single piece of pasta in the skin detection data is obtained; The water film regions are sorted in ascending order according to time sequence to obtain a water film sequence. The difference between two adjacent water film regions in the water film sequence is obtained in turn to obtain multiple thickness-increasing regions. The thickness-increasing regions are sorted sequentially to obtain a thickness-increasing sequence, and the water film region at the end of the water film sequence is placed at the end of the thickness-increasing sequence to obtain the epidermal thickness sequence. The preset gradient pixel values ​​in the preset gradient sequence are retrieved sequentially and filled into the area in the skin thickness sequence to generate the skin detection result of a single piece of pasta.

[0010] Optionally, in one possible implementation of the first aspect, it also includes: The infrared acquisition device is controlled to continuously acquire infrared data from the heated single piece of pasta, thus obtaining top-view infrared data. Obtain the infrared pixel value of the infrared pixel point within the top-view outline of a single piece of pasta in the top-view infrared data; Infrared pixels with infrared pixel values ​​greater than a preset pixel value are selected as heat-insulating pixels, and the number of heat-insulating pixels is counted to obtain the heat-insulating quantity. When it is determined that the insulation quantity is less than or equal to the preset insulation quantity, the corresponding acquisition time of the infrared acquisition device is retrieved as the end acquisition time, and the start acquisition time of the infrared acquisition device is also retrieved. The heat preservation time is obtained based on the difference between the cutoff time and the start time of the data collection. The theoretical filling range is obtained based on the heat preservation time and the preset filling reference table. The preset filling reference table includes a one-to-one correspondence between the preset time range and the preset filling range.

[0011] Optionally, in one possible implementation of the first aspect, the step of performing layered appearance detection on the circumferential detection map to obtain a second detection result of the layered pasta includes: Interlayer detection is performed on the circumferential detection map to obtain interlayer detection results; Perform floor height detection on the circumferential detection map to obtain the floor height detection results; Based on the interlayer detection results and the layer height detection results, a second detection result for the layered pasta is obtained.

[0012] Optionally, in one possible implementation of the first aspect, performing inter-layer detection on the circumferential detection map to obtain inter-layer detection results includes: Based on OpenCV, the splicing contours of each spliced ​​food in the circumferential detection image are identified, and the gap contours between adjacent splicing contours are obtained. The area of ​​the gap contour corresponding to each splicing layer in the circumferential detection image is counted to obtain the area of ​​the interlayer gap, and the number of gap contours corresponding to each splicing layer is obtained. Based on the interlayer gap area and the number of gap contours, the average gap area corresponding to each layer of spliced ​​pasta is obtained; The gap area is obtained by calculating the absolute value of the difference between the outline area of ​​each gap profile and the average gap area in the corresponding splicing layer. The gap profile with a gap area greater than a preset allowable area is selected as the interlayer detection result.

[0013] Optionally, in one possible implementation of the first aspect, the step of performing floor height detection on the circumferential detection map to obtain the floor height detection result includes: The circumferential detection image is processed into coordinates to obtain the maximum and minimum ordinates of the corresponding pixels of each spliced ​​food in the circumferential detection image. The actual layer height of the layered pasta is obtained based on the difference between the maximum and minimum ordinates. The preset packaging layer height of the packaging device corresponding to the layered pasta is retrieved, and the layered pasta with an actual layer height greater than the preset packaging layer height is used as the layer height detection result.

[0014] Optionally, in one possible implementation of the first aspect, it also includes: The infrared acquisition device is controlled to continuously acquire data on the heated layered pasta to obtain top-view layered data. Starting from the initial acquisition time of the infrared acquisition device, the top-view layered map in the top-view layered data is retrieved as the layered verification map based on the preset overall heat preservation time. Based on OpenCV, the outer contour of the layered pasta in the layered verification image is identified, and the infrared pixel values ​​of the infrared pixels within the outer contour of the layered pasta are obtained. Infrared pixels with values ​​less than a preset pixel value are selected as heat loss pixels. Adjacent heat loss pixels are counted to obtain multiple heat loss regions. Identify the splicing contours of each spliced ​​pasta dish within the layered outer contour, and send the splicing contours that intersect with the heat loss area as heat loss contours to the production end.

[0015] A second aspect of the present invention provides a pasta processing system based on multimodal detection, comprising: The acquisition module is used to perform steam heating treatment on the pasta to be tested and acquire the production attributes of the pasta to be tested, including single attributes and layered attributes. The individual detection module is used to determine that when the production attribute is a single attribute, treat the corresponding pasta to be detected as an individual pasta, continuously collect the top-view detection data of the individual pasta after heating, perform individual appearance detection on the top-view detection data, and obtain the first detection result of the individual pasta. The layered detection module is used to determine that when the production attribute is a layered attribute, to treat the corresponding pasta to be detected as a layered pasta, to collect a circumferential detection image of the heated layered pasta, to perform layered appearance detection on the circumferential detection image, and to obtain a second detection result of the layered pasta. The sending module is used to send the first detection result or the second detection result to the production end.

[0016] A third aspect of the present invention provides an electronic device comprising: a memory, a processor, and a computer program, the computer program being stored in the memory, and the processor executing the computer program to perform the methods described in the first aspect of the present invention and various possible methods related to the first aspect.

[0017] A fourth aspect of the present invention provides a storage medium storing a computer program, which, when executed by a processor, is used to implement the first aspect of the present invention and various methods possibly involved in the first aspect.

[0018] The beneficial effects of this invention are as follows: 1. This invention distinguishes the production attributes of pasta into individual attributes and layered attributes, performing individual appearance inspection on single pieces of pasta and layered appearance inspection on layered pasta separately, thus matching the inspection strategy with the actual production form of the pasta. Customized inspections can be performed for different forms of pasta, effectively balancing the inspection requirements for individual forming quality and batch structural stability, and significantly improving the targeting and reliability of pasta production quality control.

[0019] 2. This invention determines the crust thickness distribution for single-piece pasta by utilizing the physical characteristics of surface water film dissipation after baking, and infers the fullness of the filling by using infrared heat preservation time; for layered pasta, it checks whether the gap between layers and the layer height exceed the standard, and uses infrared heat loss area to locate pasta with structural damage, thereby effectively distinguishing qualified products with normal shape and sufficient filling from defective products with damage, and significantly reducing the false negative rate of defect identification. Attached Figure Description

[0020] Figure 1 The flowchart shows the pasta processing method based on multimodal detection provided by the present invention. Figure 2 This is a schematic diagram of the public area provided by the present invention; Figure 3 This is a schematic diagram of the structure of the pasta processing system based on multimodal detection provided by the present invention; Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0023] It should be understood that in the various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0024] It should be understood that in this invention, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0025] It should be understood that in this invention, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0026] It should be understood that in this invention, "B corresponding to A", "B corresponding to A", "A and B correspond", or "B and A correspond" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0027] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0028] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0029] This invention provides a pasta processing method based on multimodal detection, such as... Figure 1 As shown, it includes: S1. The pasta to be tested is subjected to steam heating treatment, and the production attributes of the pasta to be tested are obtained. The production attributes include single attributes and layered attributes.

[0030] The food items to be tested can be randomly selected items, and the production attributes can be pre-defined food type labels based on the production line configuration, including single attributes and layered attributes. Single attributes correspond to individual food items placed independently on the conveyor belt, such as steamed buns or dumplings, while layered attributes correspond to layered food items stacked in a steamer, such as longevity peach-shaped food items stacked on a birthday tray.

[0031] S2. When the production attribute is determined to be a single attribute, the corresponding pasta to be tested is treated as a single pasta, and the top-view detection data of the single pasta after heating is continuously collected. The top-view detection data is used to perform individual appearance detection to obtain the first detection result of the single pasta.

[0032] The top-view inspection data can be a sequence of images continuously captured from directly above a single piece of pasta, or it can be directly acquired by a camera. Individual appearance inspection involves inspecting the appearance of a single piece of pasta, specifically to determine the shape distortion and surface thickness distribution.

[0033] In some embodiments, step S2 (performing individual appearance inspection on the top-view inspection data to generate a first inspection result for a single piece of pasta) includes S21-S23: S21. Select any top view from the top view detection data as the detection top view, perform contour detection on the detection top view, and generate the contour detection result of a single piece of pasta.

[0034] In some embodiments, step S21 (performing contour detection on the top view and generating contour detection results for a single piece of pasta) includes S211-S215: S211. Based on OpenCV, the top view outline of a single piece of pasta in the top view is identified and detected as the actual production outline, and the center point of the actual production outline is obtained as the actual midpoint.

[0035] The actual production outline can be the actual outer boundary of a single piece of pasta as presented in a top view, such as the actual shape outline of a steamed bun in an image.

[0036] S212. Take the center point of the standard fabrication contour as the standard midpoint, align the standard midpoint with the actual midpoint, obtain the intersection of the regions of the standard fabrication contour and the actual fabrication contour, and obtain the common area.

[0037] The standard production outline can be a pre-stored standard outline for pasta, such as the round or oval outline that a steamed bun should have. See also Figure 2 After aligning the standard midpoint with the actual midpoint, the overlapping area of ​​the two contours is the common area, representing the part of the pasta shape that matches the standard shape.

[0038] S213. Based on the difference between the contour area and the common area of ​​the actual created contour, the actual deviation area is obtained, and the actual deviation area is filled with correction pixel values.

[0039] It should be noted that the actual deviation area is the part of the actual production outline that exceeds the standard production outline. By marking it with the corrected pixel value, the deviation of the pasta can be seen intuitively, making it easier for the subsequent production staff to make corrections.

[0040] S214. Based on the difference between the contour region and the common region of the standard contour, a standard deviation region is obtained, and the standard deviation region is filled with supplementary pixel values.

[0041] It's easy to understand that the standard deviation area is the part where the standard production outline exceeds the actual production outline, indicating that the surface has collapsed or is missing a corner. It is marked by supplementary pixel values.

[0042] S215. Based on the actual deviation area and the standard deviation area, obtain the contour detection results of a single piece of pasta.

[0043] By performing a difference operation between the actual production outline and the standard production outline and filling them with different pixel values, the location and degree of shape distortion of the pasta can be presented intuitively, so that the determination of shape defects no longer depends on manual visual inspection.

[0044] S22. Perform water film detection on the top-view inspection data to generate surface inspection results for a single piece of pasta.

[0045] In some embodiments, step S22 (performing water film detection on the top-view detection data to generate surface detection results for a single piece of pasta) includes S221-S225: S221. The top-view detection data is extracted according to the preset waiting time to obtain the epidermal detection data.

[0046] It should be noted that after the preset waiting time, the surface temperature of the thinner skin area has risen rapidly due to heat conduction, and the water film has begun to enter the critical evaporation state, while the heat conduction of the thicker skin area is still delayed. At this time, the detection data can be captured better to capture the difference in skin thickness.

[0047] The preset waiting time can be a pre-set time period for the water film to begin to dissipate after the pasta is taken out of the oven, such as starting to collect images 3 seconds after taking out of the oven, in order to capture the image sequence of the water film dissipation stage.

[0048] S222. Based on a preset interval, obtain the water film area within the top-view outline of a single piece of pasta in the surface detection data.

[0049] The preset interval duration can be a pre-set acquisition interval, such as retrieving one image every 2 seconds. The water film area can be the pixel area on the surface of a single piece of pasta that still has a water film attached. Freshly baked pasta will have a water film formed on its surface due to steam condensation. When acquiring white light images, supplementary lighting can be applied to facilitate the acquisition of the water film area. This is existing technology and will not be elaborated here.

[0050] S223. Sort the water film regions in ascending order according to time sequence to obtain a water film sequence. Sequentially obtain the difference between two adjacent water film regions in the water film sequence to obtain multiple thickness-increasing regions.

[0051] It's important to note that the piping hot filling inside a freshly baked pasta dish acts as a continuous heat source. This heat needs to penetrate the dough to reach the surface, causing the surface water film to evaporate. Where the dough is thin, the thermal resistance is low, heat conduction is rapid, and the surface temperature is high, causing the water film to evaporate quickly. Conversely, where the dough is thick, the thermal resistance is high, heat conduction is slow, and the surface temperature is relatively lower, causing the water film to dissipate more slowly. Therefore, subtracting the water film area from the area in the next second gives the region that evaporated during that time difference—the area of ​​increasing thickness. The later the water film disappears, the thicker the dough.

[0052] S224. Sort the thickness increasing regions sequentially to obtain a thickness increasing sequence, and place the water film region at the end of the water film sequence at the end of the thickness increasing sequence to obtain an epidermal thickness sequence.

[0053] It is not difficult to understand that the increasing thickness regions are arranged in order of disappearance, which is the epidermis from thin to thick. The water film region at the end is the last to disappear, which is the thickest part of the skin. Placing it at the end of the sequence forms a complete epidermal thickness sequence.

[0054] It is understandable that by sequentially obtaining the difference set of two adjacent water film regions in the water film sequence, and then sorting them according to the order in which the thickness-increasing regions are generated, a thickness-increasing sequence is obtained.

[0055] S225. Sequentially retrieve the preset gradient pixel values ​​from the preset gradient sequence and fill them into the area in the skin thickness sequence to generate the skin detection result of a single piece of pasta.

[0056] The preset gradient sequence can be a pre-set set of pixel values ​​arranged from light to dark, corresponding to the thickness of the outer skin from thin to thick. It's easy to understand that a larger number of values ​​indicates a thicker skin, and the number of colors retrieved subsequently will also be different; that is, different types of pasta will have different color depths depending on the number of colors retrieved. If the thickest part of the color in the outer skin detection result is not in the center but off-center, it indicates a shift in the filling distribution.

[0057] Through the above implementation method, the surface thickness distribution is calculated by utilizing the physical characteristics of the dissipation of the water film on the surface of the pasta after it comes out of the oven, and an intuitive surface thickness heat map is generated with gradient pixel values. This allows for the determination of surface uniformity and filling offset without touching the pasta.

[0058] S23. Based on the contour detection result and the skin detection result, the first detection result of a single piece of pasta is obtained.

[0059] It is easy to understand that the contour detection result reflects whether the shape is distorted, and the skin detection result reflects the skin thickness and filling distribution. The combination of the two is the first detection result of a single piece of pasta, which is used to determine whether the single piece of pasta is qualified.

[0060] Based on the above embodiments, it also includes: The infrared acquisition device is controlled to continuously acquire data from the heated single piece of pasta, obtaining top-view infrared data.

[0061] It should be noted that different physical materials have different specific heat capacities and heat storage capacities. For example, dense meat fillings have strong heat storage capacity and a long heat retention time, while loose vegetable fillings or fillings with insufficient amount of filling will lose heat quickly, mainly due to the air inside the dough. Therefore, the amount of filling can be inferred by continuously monitoring the temperature changes of the pasta using an infrared camera, eliminating the need to break open the dough for inspection as required by current technologies.

[0062] Obtain the infrared pixel value of the infrared pixel point corresponding to the top-view outline of a single piece of pasta in the top-view infrared data.

[0063] Infrared pixels with values ​​greater than a preset pixel value are selected as heat-insulating pixels, and the number of heat-insulating pixels is counted to obtain the heat-insulating quantity.

[0064] The preset pixel value can be a pre-defined high-temperature threshold. Pixels with infrared pixel values ​​higher than this threshold are considered to be in a high-temperature zone that is still retaining heat. The number of heat-retaining pixels reflects how much heat is still retained inside the pasta.

[0065] When the insulation quantity is determined to be less than or equal to the preset insulation quantity, the corresponding acquisition time of the infrared acquisition device is retrieved as the end acquisition time, and the start acquisition time of the infrared acquisition device is also retrieved.

[0066] The preset heat preservation quantity can be a pre-set lower limit threshold for the heat preservation area. When the number of heat preservation pixels drops below this threshold, it indicates that the pasta has lost most of its heat, and the recording ends at this time.

[0067] The heat preservation time is obtained based on the difference between the cutoff time and the start time of the data collection. The theoretical filling range is obtained based on the heat preservation time and the preset filling reference table. The preset filling reference table includes a one-to-one correspondence between the preset time range and the preset filling range.

[0068] The preset filling reference table can be a table established in advance based on preset time intervals and preset filling intervals, including a one-to-one correspondence between preset time intervals and preset filling intervals. By referring to the table with the measured heat preservation time, the amount of filling in the meat bun can be determined remotely without cutting the dough, thus facilitating adjustments by staff.

[0069] By utilizing the above-described methods and taking advantage of the differences in heat retention capacity of different amounts of filling, the amount of filling can be detected without damaging the pasta.

[0070] S3. When the production attribute is determined to be a layered attribute, the corresponding pasta to be tested is taken as a layered pasta, a circumferential detection image of the heated layered pasta is collected, and a layered appearance detection is performed on the circumferential detection image to obtain the second detection result of the layered pasta.

[0071] The circumferential inspection image can be an image taken from the side of the layered pasta, used to observe the gaps between layers, the overall layer height, and the splicing status between the pasta pieces. It can generally be an image acquired after rotating 360 degrees, for example, an image acquired and spliced ​​together after rotation. Layered pasta is usually a multi-layered pasta dish stacked in a steamer, and the focus is on testing the stability of its physical structure rather than individual fillings.

[0072] In some embodiments, step S3 (performing layered appearance detection on the circumferential detection map to obtain a second detection result of the layered pasta) includes S31-S33: S31. Perform interlayer detection on the circumferential detection map to obtain interlayer detection results.

[0073] In some embodiments, step S31 (performing interlayer detection on the circumferential detection map to obtain interlayer detection results) includes S311-S315: S311. Based on OpenCV, identify the splicing contours of each spliced ​​pasta in the circumferential detection image, and obtain the gap contours located between adjacent splicing contours.

[0074] In this context, "assembled pasta" can refer to individual pasta items that make up a layered pasta dish. For example, a peach-shaped layered pasta dish can be composed of five individual peaches pieced together. Each individual peach is an assembled pasta item, and the assembly outline can be the outer boundary of each assembled pasta item as shown in a side view. The gap outline can be the seam area between adjacent pasta items. It's easy to understand that each layer, such as the first layer and the second layer, has different gaps. For example, the gap between two adjacent peaches at the bottom of the first layer.

[0075] S312. Calculate the outline area of ​​the gap contour corresponding to each splicing layer in the circumferential detection diagram, obtain the interlayer gap area, and obtain the number of gap contours corresponding to each splicing layer.

[0076] S313. Based on the interlayer gap area and the number of gap contours, obtain the average gap area corresponding to each layer of spliced ​​pasta.

[0077] It's easy to understand that the average area of ​​the gaps is the average of the gap areas of each layer. Normally steamed dough expands evenly, so the gaps should be relatively consistent.

[0078] S314. Obtain the gap area based on the absolute value of the difference between the outline area of ​​each gap contour and the average gap area in the corresponding splicing layer.

[0079] S315. Select the gap profile with a gap area greater than the preset allowable area as the interlayer detection result.

[0080] The preset allowable area can be a pre-set allowable threshold for gap deviation. It should be noted that if the gap area between two pasta items differs significantly from the average, the gap may be too large or too small. Physically, this means that they either expanded excessively and stuck together during steaming, or were too far apart. These unqualified areas are directly selected as the interlayer detection results.

[0081] By implementing the above methods, the average value of the gaps between layers can be statistically analyzed and the deviation area of ​​each gap can be compared to automatically identify situations such as food sticking together.

[0082] S32. Perform floor height detection on the circumferential detection map to obtain the floor height detection results.

[0083] In some embodiments, step S32 (performing floor height detection on the circumferential detection map to obtain floor height detection results) includes S321-S323: S321. Perform coordinate processing on the circumferential detection image to obtain the maximum and minimum ordinates of the corresponding pixel points of each spliced ​​pasta in the circumferential detection image.

[0084] S322. Based on the difference between the maximum and minimum ordinates, the actual layer height of the layered pasta is obtained.

[0085] S323. Retrieve the preset packaging layer height of the packaging device corresponding to the layered pasta, and take the layered pasta with the actual layer height greater than the preset packaging layer height as the layer height detection result.

[0086] The preset packaging layer height can be the maximum height that the packaging box can accommodate. It should be noted that the next step in production is automatic boxing, and the food will be heated directly inside the packaging before being taken out for consumption. If the actual layer height exceeds the maximum size of the packaging box, it will hit the packaging lid and cause sticking.

[0087] By implementing the above methods, the actual layer height of the layered pasta is calculated through coordinate processing and compared with the packaging layer height, which can intercept unqualified products that are too tall before packaging.

[0088] S33. Based on the interlayer detection results and the layer height detection results, a second detection result for the layered pasta is obtained.

[0089] It is easy to understand that the interlayer test results reflect whether there is adhesion between the layers of pasta, while the layer height test results reflect whether the overall height exceeds the standard. The combination of the two results constitutes the second test result of the layered pasta, which is used to determine whether the structure of the layered pasta is qualified.

[0090] Based on the above embodiments, it also includes: The infrared acquisition device is controlled to continuously acquire data on the heated layered pasta to obtain top-view layered data.

[0091] Starting from the initial acquisition time of the infrared acquisition device, the top-view layered map in the top-view layered data is retrieved as the layered verification map based on the preset overall heat preservation time.

[0092] It should be noted that the heat preservation time of each layered pasta dish should be relatively consistent, at least to maintain the preset overall heat preservation time. Therefore, this solution will identify the layered pasta dishes that do not meet the requirements.

[0093] The preset overall heat preservation time can be a pre-set waiting time from the time the product comes out of the oven to the start of the test. The image retrieved at this time is used for heat loss detection.

[0094] Based on OpenCV, the outer contour of the layered pasta in the layered verification image is identified, and the infrared pixel values ​​of the infrared pixels within the outer contour of the layered pasta are obtained.

[0095] Infrared pixels with values ​​less than a preset pixel value are selected as heat loss pixels, and adjacent heat loss pixels are counted to obtain multiple heat loss regions.

[0096] It should be noted that if heat is rapidly lost from a certain area in a whole steaming basket of pasta, it often means that the physical structure of that area has been damaged, such as the dough breaking and causing internal steam to leak out. These areas with abnormally low temperatures are called heat loss areas. This is because layered pasta has internal blind spots that cannot be identified with current technology.

[0097] Identify the splicing contours of each spliced ​​pasta dish within the layered outer contour, and send the splicing contours that intersect with the heat loss area as heat loss contours to the production end.

[0098] It's easy to understand that by comparing the intersection of the temperature-loss area with the outlines of each piece of pasta, the system can better pinpoint the problematic piece of pasta in that particular steamer.

[0099] By utilizing the intersection of the infrared thermal loss area and the outline of the pasta to locate structurally damaged pasta, it is possible to identify individuals with hidden damage or leaks without disassembling the layered pasta, thus significantly improving the accuracy of defect detection in layered pasta.

[0100] S4. Send the first detection result or the second detection result to the production end.

[0101] It should be noted that sending the test results to the production end helps the production line to sort and remove substandard pasta in a timely manner, and also provides feedback data for adjusting production parameters.

[0102] Through the above implementation method, the test results are reported to the production end, realizing a closed-loop linkage between testing and production.

[0103] See Figure 3 This is a schematic diagram of the structure of a pasta processing system based on multimodal detection provided in an embodiment of the present invention. The pasta processing system based on multimodal detection includes: The acquisition module is used to perform steam heating treatment on the pasta to be tested and acquire the production attributes of the pasta to be tested, including single attributes and layered attributes. The individual detection module is used to determine that when the production attribute is a single attribute, treat the corresponding pasta to be detected as an individual pasta, continuously collect the top-view detection data of the individual pasta after heating, perform individual appearance detection on the top-view detection data, and obtain the first detection result of the individual pasta. The layered detection module is used to determine that when the production attribute is a layered attribute, to treat the corresponding pasta to be detected as a layered pasta, to collect a circumferential detection image of the heated layered pasta, to perform layered appearance detection on the circumferential detection image, and to obtain a second detection result of the layered pasta. The sending module is used to send the first detection result or the second detection result to the production end.

[0104] See Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. The electronic device 40 includes: a processor 41, a memory 42, and a computer program; wherein... The memory 42 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0105] The processor 41 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0106] Alternatively, the memory 42 can be either standalone or integrated with the processor 41.

[0107] When the memory 42 is a device independent of the processor 41, the device may further include: Bus 43 is used to connect the memory 42 and the processor 41.

[0108] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0109] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0110] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0111] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing pasta based on multimodal detection, characterized in that, include: The pasta to be tested is subjected to steam heating treatment, and the production attributes of the pasta to be tested are obtained, including single attributes and layered attributes; When the production attribute is determined to be a single attribute, the corresponding pasta to be tested is treated as a single pasta, and the top-view detection data of the single pasta after heating is continuously collected. The top-view detection data is used to perform individual appearance detection to obtain the first detection result of the single pasta. When the production attribute is determined to be a layered attribute, the corresponding pasta to be tested is regarded as a layered pasta. A circumferential detection image of the heated layered pasta is collected, and a layered appearance detection is performed on the circumferential detection image to obtain a second detection result of the layered pasta. Send the first detection result or the second detection result to the production end.

2. The method according to claim 1, characterized in that, The step of performing individual appearance inspection on the top-view inspection data to generate a first inspection result for a single piece of pasta includes: Select any top view from the top view detection data as the detection top view, perform contour detection on the detection top view, and generate the contour detection result of a single piece of pasta. Water film detection is performed on the top-view inspection data to generate surface inspection results for a single piece of pasta. Based on the contour detection results and the skin detection results, the first detection result of a single piece of pasta is obtained.

3. The method according to claim 2, characterized in that, The process of performing contour detection on the top view of the detection device to generate contour detection results for a single piece of pasta includes: Based on OpenCV, the top view contour of a single piece of pasta in the top view is identified and detected as the actual production contour, and the center point of the actual production contour is obtained as the actual midpoint. The center point of the standard fabricated outline is retrieved as the standard midpoint. The standard midpoint is aligned with the actual midpoint to obtain the intersection of the regions of the standard fabricated outline and the actual fabricated outline, thus obtaining the common area. The actual deviation area is obtained by the difference between the contour area and the common area of ​​the actual created contour, and the actual deviation area is filled with correction pixel values. Based on the difference between the contour region and the common area of ​​the standard contour, the standard deviation region is obtained, and the standard deviation region is filled with supplementary pixel values. Based on the actual deviation area and the standard deviation area, the contour detection results of a single piece of pasta are obtained.

4. The method according to claim 2, characterized in that, The process of performing water film detection on the top-view inspection data to generate surface inspection results for a single piece of pasta includes: The top-view detection data is extracted according to a preset waiting time to obtain the epidermal detection data; Based on a preset time interval, the water film area within the top-view outline of a single piece of pasta in the skin detection data is obtained; The water film regions are sorted in ascending order according to time sequence to obtain a water film sequence. The difference between two adjacent water film regions in the water film sequence is obtained in turn to obtain multiple thickness-increasing regions. The thickness-increasing regions are sorted sequentially to obtain a thickness-increasing sequence, and the water film region at the end of the water film sequence is placed at the end of the thickness-increasing sequence to obtain the epidermal thickness sequence. The preset gradient pixel values ​​in the preset gradient sequence are retrieved sequentially and filled into the area in the skin thickness sequence to generate the skin detection result of a single piece of pasta.

5. The method according to claim 2, characterized in that, Also includes: The infrared acquisition device is controlled to continuously acquire infrared data from the heated single piece of pasta, thus obtaining top-view infrared data. Obtain the infrared pixel value of the infrared pixel point within the top-view outline of a single piece of pasta in the top-view infrared data; Infrared pixels with infrared pixel values ​​greater than a preset pixel value are selected as heat-insulating pixels, and the number of heat-insulating pixels is counted to obtain the heat-insulating quantity. When it is determined that the insulation quantity is less than or equal to the preset insulation quantity, the corresponding acquisition time of the infrared acquisition device is retrieved as the end acquisition time, and the start acquisition time of the infrared acquisition device is also retrieved. The heat preservation time is obtained based on the difference between the cutoff time and the start time of the data collection. The theoretical filling range is obtained based on the heat preservation time and the preset filling reference table. The preset filling reference table includes a one-to-one correspondence between the preset time range and the preset filling range.

6. The method according to claim 1, characterized in that, The step of performing layered appearance inspection on the circumferential inspection map to obtain a second inspection result for the layered pasta includes: Interlayer detection is performed on the circumferential detection map to obtain interlayer detection results; Perform floor height detection on the circumferential detection map to obtain the floor height detection results; Based on the interlayer detection results and the layer height detection results, a second detection result for the layered pasta is obtained.

7. The method according to claim 6, characterized in that, The step of performing interlayer detection on the circumferential detection map to obtain interlayer detection results includes: Based on OpenCV, the splicing contours of each spliced ​​food in the circumferential detection image are identified, and the gap contours between adjacent splicing contours are obtained. The area of ​​the gap contour corresponding to each splicing layer in the circumferential detection image is counted to obtain the area of ​​the interlayer gap, and the number of gap contours corresponding to each splicing layer is obtained. Based on the interlayer gap area and the number of gap contours, the average gap area corresponding to each layer of spliced ​​pasta is obtained; The gap area is obtained by calculating the absolute value of the difference between the outline area of ​​each gap profile and the average gap area in the corresponding splicing layer. The gap profile with a gap area greater than a preset allowable area is selected as the interlayer detection result.

8. The method according to claim 6, characterized in that, The process of performing floor height detection on the circumferential detection map to obtain floor height detection results includes: The circumferential detection image is processed into coordinates to obtain the maximum and minimum ordinates of the corresponding pixels of each spliced ​​food in the circumferential detection image. The actual layer height of the layered pasta is obtained based on the difference between the maximum and minimum ordinates. The preset packaging layer height of the packaging device corresponding to the layered pasta is retrieved, and the layered pasta with an actual layer height greater than the preset packaging layer height is used as the layer height detection result.

9. The method according to claim 1, characterized in that, Also includes: The infrared acquisition device is controlled to continuously acquire data on the heated layered pasta to obtain top-view layered data. Starting from the initial acquisition time of the infrared acquisition device, the top-view layered map in the top-view layered data is retrieved as the layered verification map based on the preset overall heat preservation time. Based on OpenCV, the outer contour of the layered pasta in the layered verification image is identified, and the infrared pixel values ​​of the infrared pixels within the outer contour of the layered pasta are obtained. Infrared pixels with values ​​less than a preset pixel value are selected as heat loss pixels. Adjacent heat loss pixels are counted to obtain multiple heat loss regions. Identify the splicing contours of each spliced ​​pasta dish within the layered outer contour, and send the splicing contours that intersect with the heat loss area as heat loss contours to the production end.

10. A pasta processing system based on multimodal detection, characterized in that, include: The acquisition module is used to perform steam heating treatment on the pasta to be tested and acquire the production attributes of the pasta to be tested, including single attributes and layered attributes. The individual detection module is used to determine that when the production attribute is a single attribute, treat the corresponding pasta to be detected as an individual pasta, continuously collect the top-view detection data of the individual pasta after heating, perform individual appearance detection on the top-view detection data, and obtain the first detection result of the individual pasta. The layered detection module is used to determine that when the production attribute is a layered attribute, to treat the corresponding pasta to be detected as a layered pasta, to collect a circumferential detection image of the heated layered pasta, to perform layered appearance detection on the circumferential detection image, and to obtain a second detection result of the layered pasta. The sending module is used to send the first detection result or the second detection result to the production end.

Images