Smart factory-based bakery product defect detection system and method therefor

Abstract

The present invention relates to a technology for identifying defects in bakery products based on a smart factory, and more specifically, aims to eliminate subjective errors by introducing a multi-sensor-based automatic quality inspection system to perform objective and quantitative quality evaluations, evaluate quality according to consistent standards, enable more precise quality evaluation by constructing a system capable of comprehensively evaluating complex quality elements by combining weight sensors, image sensors, multi-spectral sensors, and freshness analysis sensors, and provide an automatic quality management system capable of inspecting all products in real time in a smart factory environment and immediately identifying and classifying defective products, thereby ensuring improved productivity and quality consistency, and applying optimal quality evaluation criteria by adjusting the scoring weights of each sensor according to product type.

Description

Technology Field

[0001] The present invention relates to a smart factory system and a quality determination method for evaluating the quality of bakery products. More specifically, it relates to a system and method for automatically determining the quality and defect status of bakery products by fusing and analyzing data collected using a plurality of sensors, including a weight sensor, an image sensor, a multispectral sensor, and a freshness analysis sensor. Background Technology

[0002] Generally, the quality of bakery products is determined by various factors during the manufacturing process, and typically, weight, surface color, shape, texture, internal ingredient content, and freshness serve as key quality evaluation criteria.

[0003] However, existing quality evaluation methods have the following limitations.

[0004] (1) Relying on manual inspection

[0005] Visual inspection methods lack consistency due to the subjective judgment of workers, and real-time quality control is difficult in mass production environments.

[0006] (2) Limited quality evaluation criteria

[0007] Existing methods are primarily evaluated based on weight and size, making it difficult to perform precise analysis of surface defects or internal component content.

[0008] (3) Single sensor inspection

[0009] Individual inspection methods exist, such as weight inspection, image inspection, and chemical analysis, but it is difficult to determine overall quality based on each inspection alone.

[0010] (4) Freshness inspection limit

[0011] Destructive testing is required to evaluate the moisture content, carbohydrate, and protein components of a product, and it is difficult to analyze components using non-destructive methods. The problem to be solved

[0012] The present invention relates to a smart factory-based automatic quality inspection system for evaluating the quality of bakery products. By introducing a multi-sensor-based automatic quality inspection system, objective and quantitative quality evaluation can be performed, thereby eliminating subjective errors and evaluating quality according to consistent standards. Furthermore, by constructing a system capable of comprehensively evaluating complex quality elements by combining weight sensors, image sensors, multi-spectral sensors, and freshness analysis sensors, more precise quality evaluation is possible. Additionally, by providing an automatic quality management system capable of inspecting all products in real-time in a smart factory environment and immediately identifying and classifying defective products, the invention aims to ensure improved productivity and quality consistency, and to apply optimal quality evaluation criteria by adjusting the scoring weights of each sensor according to product type. means of solving the problem

[0013] According to one embodiment of the present invention, a bakery product defect determination system may include: a weight sensor for measuring the weight of a bakery product; an image sensor for collecting data on the appearance, color, and texture of a bakery product; a multispectral sensor for collecting reflectance and absorption rates of a plurality of wavelength regions by irradiating a plurality of preset wavelength regions; a freshness analysis sensor for collecting absorbance of a specific component by measuring the intensity of reflected light by irradiating a near-infrared wavelength region; a scoring unit for individually scoring the data collected from the weight sensor, image sensor, multispectral sensor, and freshness analysis sensor through a respective scoring method and determining whether each sensor is normal; and a final determination unit for finally determining whether a bakery product is defective by applying weights to each sensor's score according to the importance of the sensed data based on the scores for each sensor output from the scoring unit.

[0014] According to one embodiment of the present invention, the scoring unit may further include a weight scoring unit that calculates a weight score by calculating a relative error with respect to a weight reference value based on weight data collected from the weight sensor, and determines whether it is normal using the calculated weight score.

[0015] According to one embodiment of the present invention, the scoring unit may further include an image scoring unit that calculates color, shape, and texture scores by comparing color, shape, and texture data collected from the image sensor with reference color, shape, and texture, respectively, calculates an image score by applying a pre-set weight for each bakery product type to the calculated color, shape, and texture scores, and determines whether the image is normal using the calculated image score.

[0016] According to one embodiment of the present invention, the scoring unit may further include a spectrum scoring unit that calculates a spectrum score based on reflectance data measured in a plurality of wavelength regions through the multispectral sensor and determines whether it is normal using the calculated spectrum score.

[0017] According to one embodiment of the present invention, the scoring unit may further include a freshness scoring unit that analyzes near-infrared spectroscopic data collected through the freshness analysis sensor to calculate the moisture, carbohydrate, and protein content of the product, calculates a freshness score by comparing the calculated moisture, carbohydrate, and protein content of the product with a preset reference value, and determines whether the product is normal using the calculated freshness score.

[0018] According to one embodiment of the present invention, the final determination unit applies different weights to score items for each sensor depending on the type of bakery product, and can optimize the scoring formula for each sensor by referring to a database storing quality standards for each product.

[0019] According to one embodiment of the present invention, the final determination unit can automatically adjust the score weight for each sensor according to the characteristics of the product and dynamically adjust the optimal defect determination threshold for each product by learning previous determination data based on a machine learning algorithm by the scoring execution unit. Effects of the invention

[0020] This invention provides a system and method for automatically determining the quality of bakery products in a smart factory environment, thereby overcoming the limitations of existing manual quality inspection methods and enabling more precise and efficient quality evaluation. Through quality analysis using multiple sensors, weight, color, shape, surface texture, internal ingredients, and freshness can be comprehensively evaluated, allowing for more sophisticated defect detection than existing methods utilizing a single criterion. Furthermore, this invention can inspect all products in real time to automatically detect and classify defective products, thereby enabling improved productivity and the maintenance of quality consistency.

[0021] Furthermore, the present invention enables objective quality determination by applying consistent quality standards based on quantitative sensor data, and allows for the setting of optimized weights for each product type to apply differentiated quality standards to bakery products, thereby enabling detailed quality evaluation tailored to the characteristics of each product, such as sliced ​​bread, croissants, bagels, and scones. In particular, by applying a machine learning-based quality determination algorithm, it moves away from conventional fixed threshold determination methods and can automatically learn quality deviations caused by changes in the production environment or differences in raw materials, and dynamically adjust optimal quality standards.

[0022] Consequently, the present invention can secure higher accuracy and reliability than existing quality inspection methods, and has the effect of simultaneously realizing increased productivity and reduced manufacturing costs through real-time automatic inspection and machine learning-based quality optimization. Brief explanation of the drawing

[0023] FIG. 1 is a configuration diagram of a smart factory-based bakery product defect detection system implemented according to an embodiment of the present invention. Figure 2 is a detailed configuration diagram of the scoring unit illustrated in Figure 1. FIG. 3 is a flowchart of a smart factory-based bakery product defect detection method according to one embodiment of the present invention. Specific details for implementing the invention

[0024] Below, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein.

[0025] The terms used in this invention are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0026] In the present invention, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not excluding in advance the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0027] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this invention pertains.

[0028] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the present invention.

[0029] In addition, it will be understood that each block of the drawings and combinations of the flowchart drawings can be executed by computer program instructions, and since these computer program instructions can be loaded into the processor of a general-purpose computer, a computer for special purposes, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create a means to perform the functions described in the flowchart block(s).

[0030] Since these computer program instructions may be stored in computer-available or computer-readable memory that can be directed toward a computer or other programmable data processing equipment to implement a function in a specific way, the instructions stored in said computer-available or computer-readable memory may also produce a manufactured item containing instruction means that performs the function described in the flowchart block(s).

[0031] Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that perform a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in the flowchart block(s).

[0032] Additionally, each block may represent a module, segment, or part of code containing one or more executable instructions for executing a specific logical function(s).

[0033] And it should be noted that in some alternative embodiments, the functions mentioned in the blocks may occur out of order. For example, two blocks illustrated in succession may actually be performed substantially simultaneously, or the blocks may be performed in reverse order according to the corresponding functions.

[0034] In this embodiment, the term '~part' refers to a hardware component such as software, FPGA (field-Programmable Gate Array), or ASIC (Application Specific Integrated Circuit), and the '~part' performs certain roles.

[0035] However, the term '~part' is not limited to software or hardware. The '~part' may be configured to reside in an addressable storage medium or may be configured to operate one or more processors.

[0036] Accordingly, as an example, 'parts' include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'. Furthermore, the components and 'parts' may be implemented to utilize one or more CPUs within a device or secure multimedia card.

[0037] In describing the embodiments of the present invention in detail, the primary focus will be on examples of specific systems, but the main point claimed in this specification is applicable to other communication systems and services having a similar technical background without significantly departing from the scope disclosed in this specification, and this will be possible at the judgment of a person with skilled technical knowledge in the relevant technical field.

[0038] Hereinafter, a smart factory-based bakery product defect detection system and method according to an embodiment of the present invention will be described with reference to the drawings.

[0039] FIG. 1 is a configuration diagram of a smart factory-based bakery product defect detection system implemented according to an embodiment of the present invention.

[0040] According to one embodiment of the present invention, a smart factory-based bakery product defect detection system may include a weight sensor (100), an image sensor (200), a multispectral sensor (300), a freshness analysis sensor (400), a scoring execution unit (500), and a final determination unit (600).

[0041] The weight sensor (100) can measure the weight of the bakery product.

[0042] According to one embodiment of the present invention, the weight sensor is a sensor that measures the weight of a bakery product and can perform the function of determining whether the manufacturing process of the product has been carried out correctly through accurate weighing.

[0043] The weight of a bakery product is an important quality factor in the manufacturing process, and since the weight of the final product may vary depending on the amount of dough, the fermentation state, the moisture content, and the degree of moisture evaporation during the oven baking process, according to one embodiment of the present invention, a weight sensor determines whether the product is underweight or overweight by comparing it with a standard weight, and can classify it as defective if it deviates from a certain weight range.

[0044] According to one embodiment of the present invention, the weight sensor is an essential sensor for measuring the weight of bakery products to evaluate quality, and quality standards can be maintained by setting an allowable error range for each product. Depending on the type of bakery product, the effects of fermentation, moisture evaporation, and additional ingredients (butter, nuts, etc.) must be considered, and it is important to set individual weight allowable ranges based on this, which will be explained in more detail in the scoring execution unit.

[0045] According to one embodiment of the present invention, the weight sensor can not only apply a fixed allowable range but can also dynamically adjust an optimal reference value by analyzing weight changes based on machine learning.

[0046] According to one embodiment of the present invention, the weight sensor can measure weight using a load cell-based strain gauge method, wherein the load cell converts the force applied to the product into an electrical signal, and the signal can be input into the system of the present invention after being converted into digital data through an A / D converter.

[0047] According to one embodiment of the present invention, the weight sensor can use a continuous measurement method to measure products moving on a conveyor belt in real time, thereby preventing a decrease in production speed in an automated process.

[0048] According to one embodiment of the present invention, since the weight sensor may be affected by the surrounding environment (e.g., vibration, airflow), accuracy can be improved by applying a filtering algorithm, and abnormal measurement values ​​can be corrected by applying an outlier removal algorithm as the filtering algorithm.

[0049] According to one embodiment of the present invention, a weight sensor may be located at the bottom of a conveyor belt and may automatically measure the weight when a product passes over the weight sensor.

[0050] According to one embodiment of the present invention, the weight sensor can measure weight in units of grams (g) or kilograms (kg), and can be set to precision: 0.1g and maximum tolerance: within ±0.5%.

[0051] The image sensor (200) can collect data on the appearance, color, and texture of the bakery product.

[0052] According to one embodiment of the present invention, an image sensor can determine whether a bakery product is defective by analyzing its appearance, color, surface texture, and shape (size, ratio), etc.

[0053] According to one embodiment of the present invention, an image sensor can perform precise inspection of the color uniformity, surface defects (cracks, contamination, degree of baking), size, and shape of a bakery product, and can maintain the quality of the product and ensure visual consistency.

[0054] According to one embodiment of the present invention, the image sensor may include an RGB camera and is designed to enable high resolution (HD or higher) and multi-spectral imaging.

[0055] According to one embodiment of the present invention, since the surface color of a bakery product is directly related to the quality of the product, the final quality can be evaluated by calculating a Color Score, Shape Score, and Texture Score through an image sensor.

[0056] According to one embodiment of the present invention, the image sensor can acquire an image of a product using a high-resolution RGB camera and analyze the following elements by applying an image analysis algorithm. More specifically, through color analysis, the color error can be calculated by comparing the standard color (RGB) of the bakery product with the actual captured color, and the actual color difference can be analyzed by correcting for differences in lighting and environment.

[0057] In addition, shape analysis can detect the outline of bakery products and compare them with a reference shape, and the precision of the shape can be evaluated by measuring the product's length, width, curvature, etc.

[0058] Finally, texture analysis can be used to analyze the surface pattern of bakery products to determine the presence of defects, and the uniformity of the bread crust can be evaluated by comparing it with a standard texture.

[0059] According to one embodiment of the present invention, the image sensor may use a line scan method that continuously photographs products on a conveyor belt and performs real-time inspection, thereby having the effect of not reducing production speed.

[0060] According to one embodiment of the present invention, the sensing conditions for an image sensor to produce an optimal result are recommended as follows: for the lighting environment, a light source color temperature of 6500K (daylight) and an illuminance of 1000 to 2000 lux (consistent lighting must be maintained to improve color analysis accuracy), and for the improvement of texture analysis, a polarizing filter is used to remove surface reflected light.

[0061] In addition, regarding shooting distance and angle, a distance of 30 to 50 cm between the camera and the product, and simultaneous shooting at 90° in the front and 45° to the side is recommended.

[0062] According to one embodiment of the present invention, the image sensor may use ΔE (CIE Lab ΔE value, color difference standard) for color error in units of sensing values, mm or percentage (%) for shape difference, and a texture matching rate (%) relative to the standard for texture uniformity.

[0063] The multispectral sensor (300) can collect the reflectance and absorption rates of the plurality of wavelength regions by investigating a plurality of preset wavelength regions.

[0064] According to one embodiment of the present invention, the multispectral sensor can perform the role of an optical sensor that performs quality evaluation by photographing the surface and internal components of a bakery product in a specific wavelength band.

[0065] According to one embodiment of the present invention, a multispectral sensor can perform color uniformity and surface and internal defect detection by analyzing reflectance and absorption rate data of a product by irradiating various wavelengths (Visible, Near-Infrared, Shortwave Infrared, etc.).

[0066] According to one embodiment of the present invention, a multispectral sensor can detect reflected light after illuminating a product with light of various wavelengths using an LED or a laser, and can evaluate the internal components and surface characteristics of the product by analyzing the sensed and measured reflectance and absorption rates.

[0067] According to one embodiment of the present invention, the multispectral sensor may use pre-set wavelength bands (Wavelength Bands) including visible light (VIS, 400~700nm, for color and surface uniformity analysis) and short-wave infrared (SWIR, 1000~2500nm, for detecting proteins, fats, and foreign substances), and as a measurement method, at least one of the following may be used: reflectance analysis (detecting component concentration and product surface changes by analyzing light reflected at a specific wavelength), absorption analysis (measuring how much light a specific component (moisture, carbohydrates, proteins, etc.) absorbs at a specific wavelength), and image-based spectrum analysis (capturing the entire surface of a product at multiple wavelengths to detect defects and discoloration).

[0068] According to one embodiment of the present invention, the optimal sensing conditions of a multispectral sensor are

[0069] It is recommended to use a multispectral light source (LED, halogen, laser), minimize the influence of ambient light in a closed measurement environment, have a camera-product distance of 20 to 40 cm, and shoot at 90° in the front and 45° to the side.

[0070] According to one embodiment of the present invention, the measurement units of the multispectral sensor may include Reflectance (0 to 1 or percent (%)) and Absorption (A).

[0071] The freshness analysis sensor (400) can collect the absorbance of a specific component by measuring the intensity of the reflected light by irradiating the near-infrared wavelength region.

[0072] According to one embodiment of the present invention, a freshness analysis sensor is a sensor that evaluates the quality retention period of a bakery product by measuring its freshness in a non-destructive manner, and the sensor can analyze the chemical composition of the product and quantify its freshness using near-infrared spectroscopy or electronic nose (E-nose) technology.

[0073] According to one embodiment of the present invention, the freshness analysis sensor can analyze items as follows.

[0074] (1) Water Content: Evaluates the moisture content and potential for spoilage of the product

[0075] (2) Digestible Carbohydrates: Predicting freshness by measuring the degree of sugar breakdown

[0076] (3) Protein Breakdown: Analysis of protein denaturation and spoilage

[0077] (4) Volatile Organic Compounds (VOCs) Detection: Detects specific gases produced as the product ages to measure the degree of spoilage

[0078] According to one embodiment of the present invention, a freshness analysis sensor quantifies the characteristics of a product that change over time and can predict the optimal consumption period based on the manufacturing date.

[0079] According to one embodiment of the present invention, a freshness analysis sensor can perform sensing through near-infrared (NIR) spectroscopy and electronic nose (E-Nose)-based volatile compound analysis.

[0080] According to one embodiment of the present invention, a freshness analysis sensor can analyze reflectance and absorption rate data by irradiating a product with near-infrared (NIR) in the 900-2500 nm band.

[0081] According to one embodiment of the present invention, a freshness analysis sensor can collect moisture content, carbohydrate denaturation rate, protein degradation rate, and volatile compound concentration by utilizing the principle that the absorbance of specific components (moisture, sugar, protein, etc.) contained in a bakery product changes at a specific wavelength using near-infrared spectroscopy.

[0082] According to one embodiment of the present invention, freshness can be estimated by applying to data collected through a freshness analysis sensor the characteristic that the fresher a bakery product is, the more constant its reflectance is at a specific wavelength, and the reflectance changes as spoilage progresses.

[0083] In addition, according to one embodiment of the present invention, a freshness analysis sensor can detect volatile organic compounds (VOCs) emitted from bakery products using electronic nose (E-Nose)-based volatile compound analysis and analyze the concentration of volatile gases (e.g., alcohols, acetones, acetic acid, etc.) to calculate a Freshness Score.

[0084] According to one embodiment of the present invention, the optimal sensing condition of the freshness analysis sensor is a light source condition.

[0085] The NIR wavelength range can be set to 900~2500nm, and a laser-based near-infrared irradiation device can be used.

[0086] In addition, environmental conditions recommended include a measurement temperature of 15–25°C and a humidity of 40–60%.

[0087] According to one embodiment of the present invention, it is recommended that the freshness analysis sensor be positioned at the top and side of the product for measurement (since freshness is important for both surface and internal components), and the units of sensing values ​​are moisture content: percentage (%), carbohydrate denaturation rate: percentage (%), protein degradation rate: percentage (%),

[0088] Volatile compound concentration: ppm (Parts per Million), Freshness Score: 0~1 (0: spoiled, 1: fresh) can be used.

[0089] The scoring unit (500) can individually score the data collected from the weight sensor, image sensor, multispectral sensor, and freshness analysis sensor through each scoring method and determine whether each sensor is normal.

[0090] According to one embodiment of the present invention, the scoring unit can calculate a score for each sensor based on data collected from a weight sensor, an image sensor, a multispectral sensor, and a freshness analysis sensor, and transmit this to a final determination unit to determine whether the bakery product is ultimately defective.

[0091] According to one embodiment of the present invention, the scoring unit applies an individual scoring method by considering the characteristics of the data measured at each sensor, can perform scoring for each sensor, and can determine whether each sensor is normal using the score for each sensor.

[0092] According to one embodiment of the present invention, the scoring unit can perform scoring for each sensor using the scoring formula below, and a detailed explanation of each formula will be provided with reference to FIG. 2.

[0093] According to one embodiment of the present invention, the scoring unit can determine normality based on a score per sensor rather than a sensing value in order to integrate multiple sensor data and enable consistent defect determination.

[0094] For example, since the measurement values ​​of weight sensors are g (grams), the color values ​​of image sensors are RGB values, and the data of freshness sensors are %, direct comparison is difficult. Therefore, if all data is normalized to a range of 0 to 1 through scoring, it is possible to evaluate whether a product is defective based on consistent criteria.

[0095] According to one embodiment of the present invention, the scoring unit (500) may further include a weight scoring unit, an image scoring unit, a spectrum scoring unit, and a freshness scoring unit as detailed components, which will be explained in more detail with reference to FIG. 2.

[0096] Figure 2 is a detailed configuration diagram of the scoring unit illustrated in Figure 1.

[0097] According to one embodiment of the present invention, the scoring unit may further include a weight scoring unit (510), an image scoring unit (520), a spectrum scoring unit (530), and a freshness scoring unit (540).

[0098] The weight scoring unit (510) calculates a weight score by calculating a relative error with respect to a weight reference value based on weight data collected from a weight sensor, and can determine whether it is normal using the calculated weight score.

[0099] According to one embodiment of the present invention, the scoring method of the weight sensor of the weight scoring unit is designed such that a penalty is applied as the weight deviates from the reference weight as shown in Equation 1, and an optimal determination criterion can be set by considering the allowable error for each product. Accordingly, if the weight deviation is within the allowable range, a score close to 1 is assigned to evaluate it as a normal product, and if it exceeds the allowable range, a score close to 0 is assigned to determine it as a defective product.

[0100]

[0101] Mathematical formula 1 is the actual measured weight of the product ( ) and standard weight( Weight score based on the difference between ) It shows that ) is being calculated.

[0102] According to one embodiment of the present invention, the actual measured weight ( ) and standard weight( Scoring can be performed by calculating the difference (deviation) as an absolute value, such that the score approaches 1 as the weight approaches the ideal value, and decreases as it deviates from the reference value, and to this end You can convert the error into an absolute value using .

[0103] According to one embodiment of the present invention, the deviation is based on the reference weight ( By dividing by ) and calculating the relative error rate, normalization can be performed so that it can be determined at a constant ratio regardless of the product size (weight) (for example, correcting so that the 5g error between bread (450g) and croissant (75g) does not have the same effect).

[0104] According to one embodiment of the present invention, a score can be calculated by subtracting the relative error rate from the maximum score (1.0), and if the weight is equal to the reference value, the weight score can be judged as 1, indicating the highest quality. If the deviation increases, the score can be gradually reduced so that the lowest score becomes 0. At this time, as the quality decreases, it may also be possible to limit the score so that it does not fall below 0 (set to a minimum of 0).

[0105] According to one embodiment of the present invention, the weight scoring unit learns weight changes in the production process based on machine learning and can adjust the weight score threshold in real time, thereby correcting weight errors caused by changes in the manufacturing environment and enabling more precise quality evaluation.

[0106] According to one embodiment of the present invention, the weight scoring unit (510) can determine whether it is normal using the calculated weight score. The weight score ( The method of determining whether it is normal based on ) can be determined by utilizing a preset threshold.

[0107] According to one embodiment of the present invention If this is equal to or greater than 0.9, it is determined to be a normal product, and If this is greater than 0.7 and less than 0.9, it is determined as a caution product, and If this is less than 0.7, it can be identified as a defective product.

[0108] The image scoring unit (520) calculates color, shape, and texture scores by comparing color, shape, and texture data collected from the image sensor with reference color, shape, and texture, and calculates an image score by applying a pre-set weight for each bakery product type to the calculated color, shape, and texture scores, and can determine whether it is normal using the calculated image score.

[0109] According to one embodiment of the present invention, the image scoring unit (520) analyzes the color, shape, and texture data of a bakery product collected by an image sensor to calculate color, shape, and texture scores, and uses the calculated color, shape, and texture scores to produce an image score as in Equation 2 ( ) can be produced.

[0110]

[0111]

[0112] : Weights for color, shape, and texture scores

[0113] According to one embodiment of the present invention, the image scoring unit (520) individually evaluates quality elements (color, shape, texture) measured through an image sensor and applies weights to obtain an image score ( It is possible to produce ).

[0114] According to one embodiment of the present invention, the image scoring unit (520) has a color score ( ) can be derived as in mathematical formula 3.

[0115]

[0116] According to one embodiment of the present invention, color score ( ) is the difference between the standard color of the bakery product and the measured color actually measured by the image sensor ( It can be determined by comparing ).

[0117] According to one embodiment of the present invention, color difference ( ) represents the difference in distance between two colors in the CIE LAB color space, and the greater the difference between the reference value and the actual measured value, the lower the quality can be judged, and As the value increases, the color score approaches 0 and matches the reference color As the value decreases, the color score can be calculated to approach 1, and here It can be set to the maximum allowable color difference value.

[0118] According to one embodiment of the present invention, the image scoring unit (520) has a shape score ( ) can be derived as in mathematical formula 4.

[0119]

[0120] According to one embodiment of the present invention, shape score ( ) is the area of ​​the bakery product measured through the image sensor ( ) and standard area( It can be calculated by comparing ).

[0121] According to one embodiment of the present invention, the area of ​​a measured bakery product ( ) and standard area( The larger the difference, the lower the quality score becomes, and the shape score is calculated to be close to 0; the smaller the difference, the higher the quality score becomes, and the shape score is calculated to be close to 1. Through this, it can be normalized to enable fair evaluation of all bakery product sizes using relative error relative to the reference area.

[0122] According to one embodiment of the present invention, the image scoring unit (520) has a texture score ( ) can be derived as in mathematical formula 5.

[0123]

[0124] According to one embodiment of the present invention, shape score ( ) can be calculated by calculating the matching rate between the surface pattern of the bakery product measured through the image sensor and the reference pattern, and in Equation 5 represents the number of patterns matching the reference texture, and can mean the total number of standard texture patterns.

[0125] According to one embodiment of the present invention, the more the number of matching patterns between the texture of the measured bakery product and the reference texture is relatively large, the closer the texture score is to 1, and the more the number of matching patterns is relatively large, the more surface defects are recognized, so the texture score can be calculated to be close to 0.

[0126] According to one embodiment of the present invention, the image scoring unit (520) may use a method of calculating similarity based on filtered edge pixels as a pattern matching method for calculating the number of matching patterns between the texture of the measured bakery product and the reference texture.

[0127] According to one embodiment of the present invention, the image scoring unit (520) can calculate an image score as shown in Equation 6 by applying a pre-set weight for each bakery product type to the calculated color, shape, and texture scores.

[0128]

[0129] According to one embodiment of the present invention, the weight for each item of mathematical formula 6 ( The optimal result can be derived by adjusting ) according to preset criteria.

[0130] According to one embodiment of the present invention, higher defect detection accuracy can be achieved by varying the weights for each item depending on the bakery product. According to the above embodiment, for products where color is important, such as bread and tiramisu, the weight for the color score can be set relatively higher than other weights; for products where shape is important, such as croissants, bagels, and scones, the weight for the shape score can be set relatively higher than other weights; and for products where texture is important, such as yokan, the weight for the texture score can be set relatively higher than other weights.

[0131] According to one embodiment of the present invention, the weight for each score for each bakery product can show the highest defect detection accuracy relatively when following Table 1 below.

[0132]

[0133] According to one embodiment of the present invention, the image scoring unit (520) can determine whether it is normal using the calculated image score, and the weight score ( The method of determining whether it is normal based on ) can be determined by utilizing a preset threshold.

[0134] According to one embodiment of the present invention If this is equal to or greater than 0.85, it is determined to be a normal product, and If this is greater than or equal to 0.7 and less than 0.0.85, it is determined as a caution product, and If this is less than 0.7, it can be identified as a defective product.

[0135] The spectrum scoring unit (530) scores the spectrum score based on reflectance data measured in multiple wavelength regions through a multispectral sensor. It is possible to calculate ) and determine whether it is normal using the calculated spectrum score.

[0136] According to one embodiment of the present invention, the spectrum scoring unit (530) has a spectrum score (as in Equation 7) ) can be produced.

[0137]

[0138] According to one embodiment of the present invention, the spectrum scoring unit (530) is the intensity of light reflected at a measured wavelength (λ) from the surface of a bakery product collected by a multispectral sensor. Spectrum score using ) ) can be produced.

[0139] The intensity of the light reflected here ( ) is an important factor reflecting the surface characteristics, moisture content, color consistency, density, etc. of bakery products, and the reference reflectance ( The score is calculated by normalizing the difference with ), and the smaller this difference, the better the quality can be judged.

[0140] According to one embodiment of the present invention, the intensity of light ( ) and reference reflectance( The smaller the difference from ), the greater the spectrum score ( As ) relatively approaches 1, the strength of ( ) and reference reflectance( The smaller the difference from ), the greater the spectrum score ( ) can become relatively close to 0.

[0141] According to one embodiment of the present invention, the spectrum scoring unit (530) has a spectrum score ( If ) is 0.9 or higher, it is determined to be a normal product, if 0.75 or higher or less than 0.9, it is determined to be a caution product, and if less than 0.75, it may be determined to be a defective product.

[0142] The freshness scoring unit (540) analyzes near-infrared spectroscopic data collected through a freshness analysis sensor to calculate the moisture, carbohydrate, and protein content of the product, calculates a freshness score by comparing the calculated moisture, carbohydrate, and protein content of the product with a preset reference value, and can determine whether it is normal using the calculated freshness score.

[0143] According to one embodiment of the present invention, a freshness score (as shown in Equation 8) ) can be produced.

[0144]

[0145] According to one embodiment of the present invention, the moisture content-based score, the carbohydrate content-based score, and the protein content-based score can be calculated according to Equation 9.

[0146]

[0147] Here is the measured moisture content, represents the standard moisture content, and is the measured carbohydrate content represents the standard carbohydrate content, and is the measured protein content can mean the standard protein content.

[0148] According to one embodiment of the present invention, moisture, carbohydrate, and protein weights may have different values, and moisture weight ( A relatively high weight (0.7~0.8) may be applied to products where moisture is important (e.g., tiramisu, yokan, bagel), and a relatively low weight (0.5~0.6) may be applied to products where crispness is required (e.g., croissant, scone).

[0149] Also, carbohydrate weight ( In the case of ), a relatively medium weight (0.2 to 0.4) may be applied to products where the crystallization of sugar and starch is important (e.g., yokan, scones), and a relatively low weight (0.2 to 0.3) may be applied to bread products where moisture loss is a major variable.

[0150] Finally, protein weighting ( In the case of products where protein degradation does not occur quickly, a low weight can be set (mostly at the level of 0.05 to 0.1), but exceptionally, for products with a high cream content (e.g., tiramisu), the weight can be set to 0.05 to minimize reflection.

[0151] According to one embodiment of the present invention, the freshness scoring unit (540) calculates a freshness score (through mathematical formula 8) If ) is 0.9 or higher, it can be determined as a normal product, if 0.75 or higher but less than 0.9, it can be determined as a caution product, and if less than 0.75, it can be determined as a defective product.

[0152] The final determination unit (600) can determine whether the bakery product is defective by applying weights to each sensor's score based on the importance of the sensed data, based on each sensor's score output from the scoring unit.

[0153] According to one embodiment of the present invention, based on the score for each sensor output from the scoring execution unit, a weight is applied to the score for each sensor according to the importance of the sensed data to obtain a final determination score ( ) can be produced.

[0154] According to one embodiment of the present invention, the final determination unit (600) applies different weights to each sensor score item according to the type of bakery product, and can optimize the scoring formula for each sensor by referring to a database storing product quality standards.

[0155] According to one embodiment of the present invention, the final determination unit (600) obtains a final determination score (through mathematical formula 10) It calculates ) and through this, can finally determine whether the bakery product is defective.

[0156]

[0157] According to one embodiment of the present invention, the final determination unit can calculate a final defect determination score by assigning weights to the scores for each sensor, taking into account the importance of each sensor.

[0158] Here, the criteria for assigning weights based on the importance of each sensor are as follows.

[0159] (1) Weight ( )

[0160] For bakery products where the weight must be consistent (e.g., sliced ​​bread, bagel), a relatively high weight (0.3) can be applied, and for products where weight variation is relatively acceptable (e.g., croissant), a low weight (0.1) can be applied.

[0161] (2) Image weights ( )

[0162] For bakery products where the shape and color are important (e.g., croissant, tiramisu), a relatively high weight (0.4) can be applied, whereas for products where internal quality is more important than appearance (e.g., yokan, bread), a relatively low weight (0.2) can be applied.

[0163] (3) Multispectral weights ( )

[0164] For bakery products where surface reflectance and compositional changes are important (e.g., scones, yokan), a relatively high weight (0.3) can be applied, whereas for products with generally constant spectral characteristics (e.g., croissants), a relatively low weight (0.2) can be applied.

[0165] (4) Freshness weight ( )

[0166] For bakery products where moisture content is important (e.g., tiramisu, yokan, bread), a relatively high weight (0.4) can be applied, whereas for products where moisture content is relatively less important (e.g., croissant, scone), a relatively low weight (0.2) can be applied.

[0167] According to one embodiment of the present invention, the optimal weights for each score for each bakery product are as shown in Table 2 below.

[0168]

[0169] According to one embodiment of the present invention, the final determination unit (600) has a final determination score (calculated through mathematical formula 10) If ) is 0.85 or higher, it can be determined as a final normal product; if 0.7 or higher but less than 0.85, it can be determined as a caution product; and if 0.7 or lower, it can be finally determined as a defective product.

[0170] According to one embodiment of the present invention, a scoring unit learns previous discrimination data based on a machine learning algorithm, automatically adjusts score weights for each sensor according to product characteristics, and can dynamically adjust the optimal defect discrimination threshold for each product.

[0171] According to one embodiment of the present invention, the final determination unit can automatically adjust the score weight through a score weight automatic adjustment algorithm such as Equation 11 below.

[0172]

[0173]

[0174] According to one embodiment of the present invention, the weight adjustment principle of Equation 11 may be based on increasing the weight of a specific sensor when it consistently shows high accuracy, and decreasing the weight of a specific sensor when frequent errors occur in that sensor.

[0175] For example, weights can be adjusted such that in the case of a croissant, the image sensor weight is increased if the discrimination accuracy of the image sensor (color and surface analysis) is high, and in the case of a bagel, the multispectral sensor weight is increased if the accuracy of the multispectral sensor (internal density analysis) is higher than that of the weight sensor.

[0176] According to one embodiment of the present invention, the final determination unit can dynamically adjust the optimal defect determination threshold through a defect determination threshold adjustment algorithm such as Equation 12 below.

[0177]

[0178]

[0179] According to one embodiment of the present invention, the threshold adjustment principle of Equation 12 may be based on strictly adjusting the threshold to strengthen quality standards when the defect rate is higher than the target, and relaxing the threshold to reduce unnecessary waste rates when the defect rate is lower than the target.

[0180] For example, the standard for color deviation (ΔE) of a croissant is 15, but if it is determined that the highest quality is maintained when ΔE ≤ 13 in actual production data, the threshold can be adjusted to 13; and the standard for moisture content of a bagel is 30%, but if it is determined that the highest quality is maintained in the 38% range, the threshold can be adjusted to 38%.

[0181] FIG. 3 is a flowchart of a smart factory-based bakery product defect detection method according to one embodiment of the present invention.

[0182] Data is collected from bakery products using a weight sensor, an image sensor, a multispectral sensor, and a freshness analysis sensor (S10).

[0183] According to one embodiment of the present invention, a weight sensor for measuring the weight of a bakery product,

[0184] Data can be collected from bakery products using an image sensor that collects appearance, color, and texture data of bakery products, a multispectral sensor that collects reflectance and absorption rates of multiple wavelength regions by irradiating multiple preset wavelength regions, and a freshness analysis sensor that collects absorbance of specific components by measuring the intensity of reflected light by irradiating a near-infrared wavelength region.

[0185] Data collected from the weight sensor, image sensor, multispectral sensor, and freshness analysis sensor are individually scored using their respective scoring methods, and the normality of each sensor is determined (S20).

[0186] According to one embodiment of the present invention, data collected from a weight sensor, an image sensor, a multispectral sensor, and a freshness analysis sensor are individually scored through a respective scoring method, and the normality of each sensor can be determined. A weight score is calculated by calculating a relative error with respect to a weight reference value based on weight data collected from the weight sensor. Color, shape, and texture data collected from the image sensor are compared with reference color, shape, and texture to calculate color, shape, and texture scores, respectively. An image score is calculated by applying a pre-set weight for each bakery product type to the calculated color, shape, and texture scores.

[0187] In addition, a spectrum score can be calculated based on reflectance data measured in multiple wavelength ranges through a multispectral sensor, and the moisture, carbohydrate, and protein content of a product can be calculated by analyzing near-infrared spectroscopic data collected through a freshness analysis sensor, and a freshness score can be calculated by comparing the calculated moisture, carbohydrate, and protein content of the product with preset reference values.

[0188] Based on the score for each sensor output from the scoring execution unit, a weight is applied to the score for each sensor according to the importance of the sensed data to finally determine whether the bakery product is defective (S30).

[0189] According to one embodiment of the present invention, based on the score for each sensor output from the scoring unit, a weight is applied to the score for each sensor according to the importance of the sensed data to finally determine whether the bakery product is defective.

[0190] The embodiments of the present invention are not implemented only through the apparatus and / or methods described above. Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention.

Claims

Claim 1 A smart factory-based bakery product defect determination system comprising: a weight sensor for measuring the weight of a bakery product; a image sensor for collecting data on the appearance, color, and texture of a bakery product; a multispectral sensor for irradiating a plurality of preset wavelength regions to collect the reflectance and absorption rates of the plurality of wavelength regions; a freshness analysis sensor for irradiating a near-infrared wavelength region to measure the intensity of reflected light and collect the absorbance of a specific component; a scoring unit for individually scoring the data collected from the weight sensor, image sensor, multispectral sensor, and freshness analysis sensor through respective scoring methods and determining the normality of each sensor; and a final determination unit for finally determining whether a bakery product is defective by applying weights to the scores of each sensor according to the importance of the sensed data based on the scores of each sensor output from the scoring unit. Claim 2 A smart factory-based bakery product defect determination system according to claim 1, wherein the scoring performing unit further comprises a weight scoring unit that calculates a weight score by calculating a relative error with respect to a weight reference value based on weight data collected from the weight sensor, and determines whether it is normal using the calculated weight score. Claim 3 A smart factory-based bakery product defect determination system according to claim 1, wherein the scoring unit further comprises an image scoring unit that calculates color, shape, and texture scores by comparing color, shape, and texture data collected from the image sensor with reference color, shape, and texture, respectively, calculates an image score by applying a pre-set weight for each bakery product type to the calculated color, shape, and texture scores, and determines whether the product is normal using the calculated image score. Claim 4 A smart factory-based bakery product defect determination system according to claim 1, wherein the scoring performing unit further comprises a spectrum scoring unit that calculates a spectrum score based on reflectance data measured in a plurality of wavelength regions through the multi-spectral sensor and determines whether it is normal using the calculated spectrum score. Claim 5 A smart factory-based bakery product defect determination system according to claim 1, wherein the scoring unit further comprises a freshness scoring unit that analyzes near-infrared spectroscopic data collected through the freshness analysis sensor to calculate the moisture, carbohydrate, and protein content of the product, calculates a freshness score by comparing the calculated moisture, carbohydrate, and protein content of the product with a preset reference value, and determines whether the product is normal using the calculated freshness score. Claim 6 A smart factory-based bakery product defect detection system according to claim 1, wherein the final determination unit applies different weights to score items for each sensor according to the type of bakery product, and optimizes the scoring formula for each sensor by referring to a database storing quality standards for each product. Claim 7 A smart factory-based bakery product defect detection system according to claim 1, wherein the final detection unit is characterized by the scoring execution unit learning previous detection data based on a machine learning algorithm, automatically adjusting score weights for each sensor according to product characteristics, and dynamically adjusting the optimal defect detection threshold for each product.

Images