QA systems and methods

Abstract

The method performed by a computer conducting a quality assurance (QA) analysis of a QA sample may include: acquiring image sensor data of the QA sample using an image sensor assembly; generating at least one of a 2D model and a 3D model of the QA sample using the image sensor data with a computing device; and performing a QA analysis of the QA sample by comparing data points of at least one of the 2D model and 3D model of the QA sample with the corresponding 2D model and 3D model from the QA specification of the QA sample with the computing device.

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 481,113, filed on January 23, 2023, the entire content of which is incorporated herein by reference.

Background Art

[0002] Background Processed pieces, including food, are divided by a processor or cut into smaller pieces in other ways according to customer needs. Also, excess fat, bones, and other foreign or undesirable materials are routinely trimmed from food. For example, for steaks served in restaurants or chicken cuts used in frozen foods or chicken burgers, it is often highly desirable to divide and / or trim the food into uniform sizes.

[0003] Many of the division / trimming of processed pieces, especially food, are currently done using high - speed dividers. These machines use various scanning techniques to check the size and shape of the food as it moves forward on a moving conveyor. This information is analyzed with the aid of a computer to determine the most efficient way to divide the food into the optimal size. For example, a customer may desire to divide chicken breasts into two different weight sizes, but without fat or with a limited amount of acceptable fat. The chicken breasts are scanned as they move on the feed conveyor belt, and a computer is used to determine the best way to divide the chicken breasts into the weight desired by the customer, without fat or with a limited amount of fat, in order to use the chicken breasts most effectively.

[0004] The splitting and / or trimming of processed pieces can be performed after the food has been transferred from the feed conveyor to the cutting conveyor, using a variety of cutting devices including high-speed liquid jet cutters (the liquid may include, for example, water or liquid nitrogen) or rotary or reciprocating blades. In many high-speed splitting systems, several high-speed water jet cutters are positioned along the length of the conveyor to achieve high throughput of split / cut processed pieces. Once splitting / trimming is performed, the resulting pieces are removed from the cutting conveyor and placed on a pick-up conveyor for further processing, or possibly in storage bins.

[0005] While the high-speed splitting machines referenced herein are highly sophisticated for analyzing processed pieces and determining the optimal way to split or cut such pieces at high production rates (e.g., typically exceeding 200 pieces per minute), they cannot always account for variations in the shape, dimensions, weight, density, color, and texture of incoming raw, unprocessed food products. Furthermore, even if the splitting machine is frequently recalibrated, the machine can quickly become out of sync (e.g., due to component wear, timing issues, etc.).

[0006] Therefore, the difference between an accurate, intended split cut and an incorrect split cut can be minute and frequent on the one hand, and dramatic and rare on the other. In either situation, the splitting error can be "hidden" among the thousands of food pieces being split per hour. Even trained, observer operators monitoring for specific problems, such as pieces that are too heavy or too light, can find it difficult to spot "outliers," especially since larger pieces pass along the conveyor belt at a rate of 2-3 pieces per second, or smaller pieces at about 10 pieces per second. Furthermore, splitting inaccuracies often develop slowly over time, making them difficult for operators to notice. [Overview of the project] [Problems that the invention aims to solve]

[0007] overview In some embodiments, the techniques described herein are computer-operated methods for performing quality assurance (QA) analysis on a QA sample, which is at least one of a quality assurance (QA) sample processed by a processing system and a QA sample to be processed by the processing system, wherein the processing system has a controller configured to manage the modes for which the processing system processes a workpiece in response to an analysis of the workpiece, separate from the QA analysis, and the method is a computer-operated method that includes: acquiring QA image sensor data of the QA sample using an image sensor assembly of the QA system; generating at least one of a 2D model and a 3D model of the QA sample using the QA image sensor data using a computing device; and performing a QA analysis of the QA sample by comparing the QA data of at least one of the 2D model and 3D model of the QA sample with the QA specifications of the QA sample using a computing device.

[0008] In some embodiments, the techniques described herein relate to a quality assurance (QA) system for performing a QA analysis of a QA sample which is at least one of a QA sample which is processed by a processing system and a QA sample which will be processed by a processing system, wherein the processing system comprises a controller configured to manage the modes for which the processing system processes a workpiece in response to an analysis of a workpiece separate from the QA analysis, an image sensor assembly of the QA system configured to acquire QA image sensor data of the QA sample, a processor, and a memory storing instructions, wherein when the instructions are executed by the processor, the instructions cause the computing device of the QA system to generate at least one of a 2D model and a 3D model of the QA sample, and to perform a QA analysis of the QA sample by comparing the QA data of at least one of the 2D model and 3D model of the QA sample with the QA specifications of the QA sample.

[0009] In some embodiments, the technique described herein is a method performed by a computer performing a quality assurance (QA) analysis of a QA sample which is processed by a processing system and at least one of the QA samples which will be processed by the processing system, wherein the processing system has a controller configured to manage the modes for which the processing system processes the workpiece in response to analysis of the workpiece, separate from the QA analysis, and the method is to acquire QA image sensor data of the QA sample using an image sensor assembly of the QA system and to generate at least one of a 2D model and a 3D model of the QA sample using the QA image sensor data using a computing device The present invention relates to a method performed by a computer, which includes: obtaining a QA weight measurement of a QA sample using a weight measurement assembly of a QA system; and performing a QA analysis of the QA sample using a computing device, the method comprising: comparing QA image data of at least one of a 2D model and a 3D model of the QA sample with the QA specification of the QA sample; and comparing at least one of the QA weight measurement and the calculated density of the QA sample based on the QA weight measurement with at least one of the weight value and density value determined from at least one of the QA specification of the QA sample and scanning of the QA sample by a processing system.

[0010] This summary is provided to introduce, in a simplified form, the selection of concepts that will be further described in the detailed explanation below. This summary is not intended to identify the main features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0011] Description of the drawing The aforementioned aspects of the present invention and many associated advantages will be more readily understood by referring to the following detailed description, when considered in conjunction with the accompanying drawings. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram of a non-limiting example of a food processing management system in various aspects of the present disclosure. [Figure 2] This is a schematic diagram of non-limiting examples of processing systems according to various aspects of the present disclosure. [Figure 3] This is a block diagram of non-limiting examples of processor computing devices in various aspects of the present disclosure. [Figure 4A] This is a diagram illustrating an example of a QA scanning station in various aspects of this disclosure. [Figure 4B] This is a diagram illustrating an example of a QA scanning station in various aspects of this disclosure. [Figure 4C] This is a diagram illustrating an example of a QA scanning station in various aspects of this disclosure. [Figure 4D] This is a diagram illustrating an example of a QA scanning station in various aspects of this disclosure. [Figure 5A] This is a top isometric view of alternative examples of QA scanning stations according to various embodiments of the present disclosure. [Figure 5B] Figure 5A is an isometric view of the bottom of the QA scanning station. [Figure 6] This is an isometric view of alternative examples of QA scanning stations in various aspects of the present disclosure. [Figure 7] This is a block diagram of non-limiting examples of quality assurance (QA) computing devices in various aspects of the present disclosure. [Figure 8] This flowchart illustrates a non-limiting example of a method for conducting a QA analysis of processed parts. [Figure 9] This is a block diagram illustrating a non-limiting example of a computing device suitable for use as a computing device having the examples of this disclosure. [Modes for carrying out the invention]

[0013] Detailed explanation Aspects of the present disclosure relate to systems and methods for ensuring the quality of processed pieces such as food products and machine parts.

[0014] Food processing lines are generally monitored by quality assurance (QA) personnel to ensure that processed products and machine parts are within specifications ("specs").

[0015] With respect to food, QA personnel often take small samples of the product being processed and check that they are being sliced, cut, trimmed, etc. according to the required specs. For example, a QA technician may take a sample of 10 sliced / trimmed chicken breasts (from thousands of incoming chicken breasts or chicken cuts) and measure and / or weigh the chicken breasts to determine whether the chicken breasts or breast portions are of the correct size, weight, shape, etc.

[0016] Measurements for evaluating product size can include taking one measurement each for height, length, and width. The QA technician may also overlay the sample on a laminated sheet showing the required product outline to evaluate whether the sample is of the correct shape. Further, the QA technician may weigh the sample on a bench scale to obtain the exact weight of the sample and determine whether the sample is of the correct weight. Other aspects may also be evaluated depending on the product specifications. For example, certain food suppliers may require that their beef patties have a specific "bands coating" area or a specific shape. Other suppliers may require that the sliced meat has no blood spots or blemishes.

[0017] Although QA technicians are often highly skilled, there is always variation among technicians, and manual measurements are prone to inaccuracies and inconsistencies over time. Further, there is a large variation in skills and experience among technicians, which can lead to data variation. In that regard, QA measurements are not standardized across the entire food processing line or production process.

[0018] Moreover, considering the time required to obtain measurement values, the number of food samples that can be measured remains small. In that regard, data regarding samples may not always accurately represent all processed products in the production process. Nevertheless, without the ability to measure all processed foods, if the threshold level of the sample (e.g., over 5%) is out of spec, the production manager may reprocess all the food (e.g., pass all the food through the machine again), downgrade the food, and / or discard the entire process.

[0019] When conducting measurements, QA technicians can refer to the spec sheet to determine whether the food is within spec or out of spec. However, QA technicians often do not have the time or bandwidth for any further analysis, such as how much the food is out of spec (e.g., the percentage out of spec) and what adjustments can be made to the machine to bring the food into spec. Instead, technicians often make guesses at on-the-fly adjustments to the machine in an attempt to account for out-of-spec processing. For example, the technician can adjust the density value of the incoming food to change the split size, weight, shape, etc.

[0020] In that connection, even if the measurement is carried out accurately, the data may not always be used efficiently and effectively to improve the quality of the processed product. For example, if some analysis is desired, the data must be manually entered into a computing device. Moreover, if a significant portion (or all) of the incoming product has already been processed, the analysis, if performed, will occur after that fact.

[0021] As mentioned above, food processing machinery typically uses various scanning techniques to determine the size, shape, and quantity of food as it moves along a moving conveyor belt. This information is then analyzed with computer assistance to determine, for example, how to most efficiently and accurately divide the food into optimal sizes, how to trim the product (e.g., locating fat for trimming), and how to collect the product (e.g., sorting and / or picking up products of various sizes for further processing or packaging).

[0022] Regarding machine components, proactive QA analysis is often difficult because components are part of a complex system. In this regard, high-level aspects of the machine, such as belt speed and oven temperature, may be monitored. However, it may be beneficial to monitor or perform QA on individual machine components, such as conveying system components (e.g., pins, belt pickets or rods, links, chains, mesh components, etc.), cutting components, or other high-wear components. Proactively addressing machine component issues can be used to evaluate belt slack, belt wear, blade wear, or other problems before they affect machining accuracy.

[0023] Examples of the present disclosure relate to quality assurance (QA) systems and methods that can be used to evaluate the attributes of a workpiece, such as the characteristics of the workpiece before it is processed by a processing machine and / or the quality of the workpiece after it has been processed. In some examples, the exemplary QA systems and methods disclosed herein may be used to obtain the precise dimensions of a workpiece, for example, by using data from one or more sensors configured to capture image data of the processed workpiece. In some examples, the exemplary QA systems and methods disclosed herein may be used to obtain the precise weight measurement of a workpiece, the weight measurement may be obtained substantially simultaneously with any image data or other data.

[0024] Image data, weight data, and any other data ("QA data") (such as machine learning model output data, as described below) relating to a workpiece collected by a QA system can be processed by the QA system's computing device to evaluate one or more attributes of the workpiece. For example, weight measurements, along with dimensional data, may be used to determine additional parameters of the processed workpiece, such as density. Knowing the density value of a workpiece (or multiple workpieces) allows the density setting of the processing machine to be automatically adjusted to accurately divide the remaining workpiece. In this regard, the QA systems and methods disclosed herein may also be used to determine the density of a workpiece before it is processed by a processing machine, and as a result, the density setting in the machine can be automatically adjusted to accurately divide the incoming workpiece. In other embodiments, the QA systems and methods disclosed herein may also be used to determine the density of a workpiece after it has been processed by a processing machine, and as a result, the density setting in the machine can be automatically adjusted to accurately divide the remaining workpiece.

[0025] QA data may be processed by a QA system or another computing device to generate additional data for use in evaluating a workpiece or the behavior of a workpiece. For example, in some cases, the QA systems and methods disclosed herein can be used to simply determine whether a workpiece is within or outside of specifications and to inform the QA personnel of this. In some cases, if a workpiece is outside of specifications, the QA systems and methods disclosed herein may be used to determine how far outside the specifications the workpiece is, to provide a list of possible corrective actions that the QA personnel can take, and / or to automatically change the machine settings (or, in the case of machine parts, to automatically order or suggest an order for a replacement part or repair of the part).

[0026] In some examples, the QA systems and methods disclosed herein may be used to evaluate production processes, such as by transmitting QA data to a monitoring system for further processing. In some examples, the QA systems and methods disclosed herein may be used to identify one or more additional processing steps of processed pieces, such as trimming, meat tenderizing, sorting, picking, and packaging. In some examples, the QA systems and methods disclosed herein may be used to implement global optimization for assigning processed pieces to package configurations based on evaluations.

[0027] As can be understood, the QA systems and methods disclosed herein enable users to obtain QA data with significantly higher accuracy compared to data obtained from manual measurements using calipers and scales. Furthermore, the QA data obtained using the QA systems and methods disclosed herein can be obtained in a fraction of the time it would take to perform manual measurements. In this regard, the QA systems and methods disclosed herein can be used to obtain highly accurate QA data in a shorter time compared to prior art methods.

[0028] In addition, the QA systems and methods disclosed herein are configured to directly process and use QA data without requiring any manual input, thereby reducing or eliminating data transmission errors. The QA data and any related data can be processed to determine appropriate machine adjustments or repairs / replacements of parts, which may be done manually or automatically.

[0029] In this regard, the QA systems and methods disclosed herein can be used to automatically record and store data for historical review. At least one of short-term and long-term historical data can be used to control the automation of a processing apparatus (e.g., automatically changing settings within the machine, automatically ordering or suggesting replacement parts or repairs of parts). For example, a short-term moving average of historical QA data can be used for control / automation rather than using instantaneous QA values. In other embodiments, artificial intelligence, such as one or more machine learning models, can use short-term and / or long-term historical QA data for baseline data and for training one or more machine learning models.

[0030] In this regard, the QA systems and methods disclosed herein can be used to continuously train and update various aspects of the QA systems and methods during routine monitoring. For example, QA data generated and processed by the QA systems and methods disclosed herein can be used to train one or more machine learning models, which can be used to perform at least one QA analysis, provide a list of possible corrective actions that a QA person can take, automatically change settings within a machine, and, in the case of machine parts, automatically order or suggest replacement parts or repairs of parts.

[0031] The QA systems and methods disclosed herein provide a comprehensive approach to assist QA personnel in routine monitoring of production by providing faster and more accurate identification of defects or specification levels, along with the ability to continuously train and update the systems during routine monitoring. The aforementioned and other benefits will be further understood from the following description.

[0032] In this disclosure, references to “food,” “foodstuffs,” “foodstuff pieces,” “foodstuff items,” “pieces,” and “parts” are used interchangeably and mean to include all kinds of food. Such food may include meat, fish, poultry, plant-based products, fruits, vegetables, nuts, or other types of food. The QA systems and methods also cover raw foods, as well as partially and / or fully processed or cooked foods.

[0033] Furthermore, while the exemplary QA systems and methods disclosed herein may be described with specific applicability to food or food products, they may also be used outside the food sector. For example, the exemplary QA systems and methods disclosed herein may be applicable to machine parts or other processed pieces. Accordingly, this disclosure may refer to “processed pieces,” “products,” “components,” “samples,” etc., and these terms are synonymous with each other. It should be understood that references to “processed pieces,” “products,” “components,” “samples,” etc., also include food, food products, food pieces, food products, etc. Furthermore, references to “food,” “food products,” “food pieces,” “food products,” “pieces,” “parts,” etc., also include “processed pieces,” products, components, samples, etc.

[0034] Furthermore, “QA sample” may be used to generally refer to any processed piece, food, etc., that is analyzed using the QA systems and methods disclosed herein. In this regard, a QA sample may include processed pieces that have already been at least partially processed by the processing system, incoming processed pieces that have not yet been processed, used components, new components, etc. Furthermore, when referring to “processed piece,” etc., it may also include a QA sample.

[0035] Figure 1 shows a schematic diagram of a non-limiting example of a workpiece processing management system 102 that can be used to collect and process QA data for evaluating one or more attributes of a workpiece (or "QA sample") processed by a workpiece processing machine or system. The workpiece processing management system 102 may include various network-connected computing devices configured to perform the actions of collecting and processing QA data for evaluating one or more attributes of a QA sample, as well as other actions of processing the same type of QA sample and / or workpiece (e.g., conveying, slicing, splitting, sorting, packaging, etc.).

[0036] In the illustrated example, the workpiece processing management system 102 includes a processing system 104, a QA station 106 having an integrated QA scanning system 108 and a weight measuring assembly 110, a QA computing device 111, an optional monitoring system 112, and a model management computing device 113, all of which are connected to each other via a network 114 for communication. The network 114 can be any type of network that can enable communication between the various components of the workpiece processing management system 102. For example, the network can be a Wi-Fi network.

[0037] First, the processing system 104 will be described with reference to Figures 1 and 2. The processing system 104 is generally configured to perform processing on a workpiece before it is designated as a QA sample to be scanned and / or weighed by the QA station 106. In this way, the quality of the workpiece processed by the processing system 104 (e.g., whether the workpiece is within specifications) can be analyzed using data collected from the QA station 106. However, in some examples, the QA station 106, including a QA scanning system 108 and / or a weighing assembly 110, may be located upstream of the processing system 104 to collect QA data related to incoming workpieces, either additionally or alternatively.

[0038] The processing system 104 includes a transport system 116 or another moving device configured to transport workpieces WP or workpieces between various parts of the processing system 104. For example, the transport system 116 may transport workpieces between one or more of the slicer 118, scanning station 120, cutter station 122, pickup station 124, sorter 126, and packager 128. Various components of the processing system 104 may be controlled by a processor computing device 130.

[0039] The conveying system 116 may include a powered belt 115 supported by a series of rollers (not indicated), one of which is a drive roller that drives the belt 115 in a standard manner. Encoders may be used for the support roller or end roller to determine the position of the workpiece on the conveying belt and the progress or movement of the workpiece in the conveying direction. Although a single belt 115 is shown, the conveying system 116 may consist of one or more belts, for example, a flat solid belt that supports the workpiece while scanning under a portion of the scanning station 120. Such a belt is typically a flat non-metallic belt. The workpiece can be transferred from a first belt to a second belt that supports the workpiece during the splitting or trimming process at the cutter station 122. If a waterjet cutter is used to split or trim the workpiece, it is advantageous to use an open-mesh metal belt so that the waterjet can pass downwards and the belt has sufficient structural integrity to withstand the impact from the waterjet. Such open-mesh metal belts are commercially available.

[0040] The slicer 118 may be used to slice primal cuts (e.g., cuts of meat initially separated from the animal carcass during meat butchering or processing, such as pork loin) into subprimal cuts (e.g., if it is pork loin, sirloin chops, center loin chops, center rib chops, rib end chops, etc.) before further processing by the processing system 104. In this regard, the slicer 118 may be positioned downstream of a cutter (not shown) used to cut the carcass into primal cuts. The slicer 118 may also be used to cut subprimal cuts, such as pork chops or chicken breast, into slices.

[0041] Various types of slicers can be used to slice the workpiece into one or more cuts or slices of desired thickness. The slicer 118 may be configured to cut muscle and optionally bone, and may be oriented vertically or horizontally. For example, the slicer 118 may be in the form of a high-speed water jet, laser, rotary saw, hacksaw, or band saw.

[0042] The slicer 118 may be adjustable to obtain each cut or slice of a desired thickness. Such adjustments may be under the control of a processor, such as a processor computing device 130. For example, the slicer 118 may be adjusted based on data transmitted from the QA scanning system 108 and processed by the processor computing device 130 and / or QA computing device 111, such as taking into account different density values ​​of the workpiece (for example, if a higher density is measured by the QA scanning system 108, smaller slices may be made to achieve slices within the weight specification). In an example where the slicer 118 is oriented to slice horizontally, the thickness of each cut or slice required to produce a desired target weight may depend on any workpiece undercuts, voids, or other irregularities. In that case, measured QA data from a sliced ​​QA sample may be used to adjust the slicer settings (e.g., horizontal slice height offset) to account for the irregularities of the workpiece. In some examples, the processing system 104 receives cut or sliced ​​products from another machine or location, and the slicer 118 is excluded. In general, terms such as “slicing,” “dividing,” “cutting,” and “trimming” may include any type or any combination of product cutting (e.g., slicing only, dividing only, or any other type of product cutting, as well as any combination of slicing, dividing, and other types of product cutting).

[0043] The workpiece WP is inspected at the scanning station 120 to confirm physical parameters or properties of the workpiece, for example, relating to the size and / or shape of the workpiece. Such properties may include, for example, length, width, length / width aspect ratio, thickness, thickness profile, contour, outer contour configuration, outer taper, flatness, outer perimeter configuration, outer perimeter size and shape, volume, weight, and whether the workpiece contains any undesirable materials such as bone, fat, cartilage, metal, glass, plastic, etc., as well as the location of undesirable materials within the workpiece.

[0044] Such physical parameters may include the maximum value, mean value, arithmetic mean value, and / or intermediate value of such parameters. With respect to the thickness profile of the workpiece, such a profile may be along the length of the workpiece, across the width of the workpiece, and both across / along the width and length of the workpiece.

[0045] The parameter called "perimeter" of a workpiece refers to the boundary or distance around the workpiece. Therefore, the terms perimeter, perimeter configuration, perimeter size, and perimeter shape relate to the distance around the configuration, the size and shape of the outermost boundary or edge of the workpiece, etc.

[0046] The size and / or shape parameters / characteristics listed above are not intended to be limiting or comprehensive. Other size and / or shape parameters / characteristics may be confirmed, monitored, measured, etc., by the systems and methods disclosed herein. Furthermore, the definitions or descriptions of any particular size and / or shape parameters / characteristics described above are not intended to be limiting or comprehensive.

[0047] The scanning station 120 may include any suitable scanner, such as one or more of the scanners and / or systems and methods for processing scanner data described in U.S. Patent No. 1,0721947, titled "Apparatus for acquiring and analyzing product-specific data for products of the food processing industry as well as a system comprising such an apparatus and a method for processing products of the food processing industry," which is incorporated herein by reference in its entirety.

[0048] In the illustrated example, the scanning station 120 may utilize the X-ray apparatus 119 to determine the physical properties of the workpiece, including its shape, mass, and weight. X-rays can pass through the object in the direction of the X-ray detector (unlabeled). Such X-rays are attenuated by the workpiece in proportion to its mass. The X-ray detector can measure the intensity of the X-rays received after they have passed through the workpiece. This information may be used to determine physical parameters relating to the size and / or shape of the workpiece, including, for example, length, width, aspect ratio, thickness, thickness profile, contour, outer contour configuration, perimeter, outer perimeter configuration, outer perimeter size and / or shape, volume, weight, and other aspects of the physical parameters / properties of the workpiece. With respect to the outer perimeter configuration of the workpiece, the X-ray detector can determine its location along the outer perimeter of the workpiece based on an XY coordinate system or other coordinate system. Examples of such X-ray scanning devices are disclosed in U.S. Patent No. 5,585,605, entitled “Optical-scanning system employing laser and laser safety control,” U.S. Patent No. 1,0654185, entitled “Cutting / portioning using combined X-ray and optical scanning,” U.S. Patent No. 5,585,603, entitled “Method and system for weighing objects using X-rays,” and U.S. Patent No. 1,0721947 (referenced above), which are incorporated herein by reference in their entirety.

[0049] The scanning station 120 may also include an optical scanner 121 for generating at least one of the following: a visible light (e.g., grayscale) image, a laser light scattering image, a height map, a hyperspectral image, or a multispectral image of the workpiece, to show one or more of the overall shape / size of the workpiece, the composition of the workpiece (e.g., fat vs. lean meat), or the height or thickness over an area of ​​the workpiece. Scanning by the scanning station 120 can be performed using a variety of techniques, such as those shown and described in U.S. Patent No. 1,0654185, U.S. Patent No. 1,0721947, and U.S. Patent No. 1,1570998, all of which are incorporated in whole by reference.

[0050] The optical scanner 121 may include a video camera (not shown) for viewing a workpiece illuminated by one or more light sources. Light from the light sources extends across the moving conveyor belt 115 to define sharp shadows or light stripes, such that the area in front of the transverse beam is darkened. When the workpiece is not being transported by the conveyor belt 115, the shadow lines / light stripes form straight lines across the belt. However, when the workpiece passes over the shadow lines / light stripes, the irregular upper surface of the workpiece generates irregular shadow lines / light stripes when viewed by a video camera (not shown) pointed diagonally downwards over the workpiece and the shadow lines / light stripes. The video camera detects the displacement of the shadow lines / light stripes from the position they would occupy if the workpiece were not on the conveyor belt. This displacement represents the thickness of the workpiece along the shadow lines / light stripes.

[0051] The length of the workpiece is determined by the distance the belt travels over which the shadow lines / light stripes are generated by the workpiece. In this regard, an encoder incorporated into the conveyor 115 generates pulses at fixed distance intervals corresponding to the forward movement of the conveyor.

[0052] In some examples, the scanning station 120 uses a single SICK® camera with a single laser light source suitable for acquiring optical data and generating two or more images / views based on the optical data. For example, the single camera may communicate with a separate processor and / or processor computing device 130 (having one or more feature recognition modules, etc.) to generate one or more views from the acquired optical data, such as a fat recognition (FRS) object view, a laser scattering object view, and a height-mode object view.

[0053] In some examples, at least two optical cameras are used, each optionally equipped with a different imaging processor. For example, a simple optical camera, such as a grayscale camera, and / or an RGB camera, and / or an IR and / or UV camera, and / or a charge-coupled device (CCD) can be used to acquire and / or generate one or more complete images of a workpiece to detect specific characteristics, such as the outer contour of the workpiece. Furthermore, a second special camera, such as a multispectral or hyperspectral camera, can be used to acquire image / data of specific areas or characteristics of the workpiece, such as blood spots or fat streaks. Alternatively, it should be understood that a single camera / scanner may be used to capture all the data necessary to generate various images through various imaging processes, etc.

[0054] The results of the scan performed at the scanning station 120 are transmitted to the processor computing device 130.

[0055] Here, an exemplary embodiment of the processor computing device 130 will be described with reference to Figure 3. As described above, the processor computing device 130 may generally be configured to control components of the machining system 104 in response to scanning data transmitted from the scanning station 120 and other data or inputs (such as those from the QA computing device 111). In the exemplary block diagram of Figure 3, the processor computing device 130 includes a processor 302, a communication interface 304, a computer-readable medium 306, and at least one data store 316. As shown, the computer-readable medium 306 stores logic that causes the processor computing device 130 to provide a sensor data processing engine 308, a model generation engine 310, and a workpiece machining engine 312 in response to execution by one or more processors 302.

[0056] The processor computing device 130 may be implemented by any computing device or set of computing devices, including, but not limited to, desktop computing devices, laptop computing devices, mobile computing devices, edge computing devices, programmable logic controllers (PLCs), server computing devices, computing devices for cloud computing systems, and / or combinations thereof. In some examples, the processor 302 may include any suitable type of general-purpose computer processor. In some examples, the processor 302 may include, but not limited to, one or more dedicated computer processors or AI accelerators optimized for a particular computing task, including a graphics processing unit (GPU), a vision processing unit (VPT), ​​and a tensor processing unit (TPU).

[0057] In some examples, the communication interface 304 includes one or more hardware and / or software interfaces suitable for providing communication links between components. The communication interface 304 may support one or more wired communication technologies (including, but not limited to, Ethernet®, FireWire, and USB), one or more wireless communication technologies (including, but not limited to, Wi-Fi, WiMAX, Bluetooth, 2G, 3G, 4G, 5G, and LTE), and / or combinations thereof.

[0058] As used herein, “computer-readable media” means, but is not limited to, any removable or non-removable device that implements any technology capable of storing information in a volatile or non-volatile manner that will be read by the processor of a computing device, including, but not limited to, hard drives, flash memory, solid-state drives, random-access memory (RAM), read-only memory (ROM), CD-ROMs, DVDs, or other disk storage devices, magnetic cassettes, magnetic tapes, and magnetic disk storage devices.

[0059] As used herein, “engine” means logic embodied in hardware or software instructions that can be written in one or more programming languages, including, but not limited to, C, C++, C#, COBOL, JAVA®, PHP, Perl, HTML, CSS, JavaScript, VBScript, ASPX, Go, and Python. An engine may be compiled into an executable program or written in an interpreted programming language. A software engine may be callable from other engines or from itself. Generally, an engine as described herein refers to a logic module that can be merged with other engines or divided into sub-engines. An engine may be implemented by logic stored in any type of computer-readable medium or computer storage device, stored in and executed on one or more general-purpose computers, and thus create a dedicated computer configured to provide the engine or its functionality. An engine may be implemented by logic programmed into an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or another hardware device.

[0060] As used herein, “datastore” means any suitable device configured to store data for access by computing devices. An example of a datastore is a highly reliable, high-speed relational database management system (DBMS) running on one or more computing devices and accessible over a high-speed network. Another example of a datastore is a key-value store. However, any other suitable storage technique and / or device capable of quickly and reliably providing the stored data in response to queries may be used, and the computing device may be locally accessible rather than over a network, or provided as a cloud-based service. A datastore may also include data stored in an organized manner on computer-readable storage media such as hard disk drives, flash memory, RAM, ROM, or any other type of computer-readable storage medium. Those skilled in the art will recognize, without departing from the scope of this disclosure, that the separate datastores described herein may be combined into a single datastore, and / or that the single datastore described herein may be separated into multiple datastores.

[0061] The sensor data processing engine 308 of the processor computing device 130 may be configured to process incoming sensor data of the workpiece. The sensor data may include one or more images captured by the scanning station 120. For example, the sensor data may include one or more images generated by the X-ray device 119 and the optical scanner 121.

[0062] The sensor data processing engine 308 may be configured to run one or more feature recognition modules for generating views / images from scanning data and / or processing data from different views. For example, the sensor data processing engine 308 may be configured to generate at least one of the following from data acquired by the optical scanner 121: a fat recognition (FRS) object view, a laser scattering object view, and a height-mode object view.

[0063] Before executing one or more feature recognition modules, the sensor data processing engine 308 may first analyze the data from the X-ray apparatus 119 and the optical scanner 121 to determine whether the workpiece scanned by the optical scanner 121 is the same as the workpiece previously scanned by the X-ray apparatus 119, and / or whether the workpiece has moved or shifted during transfer between conveyors, as described in U.S. Patents 1,0654185 and 1,0721947 (referenced above), which are incorporated herein by reference. In this regard, the comparison of X-ray data and optical data may be processed by the processor computing device 130.

[0064] The sensor data processing engine 308 may be configured to generate a registered scan of a workpiece, including a first scan of a first scanning type (e.g., X-ray) and a second scan of a second scanning type (e.g., optical image). For example, a registered scan of a workpiece can be generated by the sensor data preprocessing engine 308, which maps the X-ray image of the workpiece scanned in the X-ray apparatus 119 to the (optionally transformed) optical image of the workpiece as scanned by the optical scanner 121. In one example, the registered scan is generated by the sensor data preprocessing engine 308 using a system and method described in U.S. Patent No. 1,0654185, which is incorporated herein by reference in its entirety. For example, the X-ray data may optionally be mapped to the optical data using one or more image / data transformations or translations to account for any movement / shift of the workpiece on the conveyor.

[0065] The data processed by the sensor data processing engine 308 may be sent to or retrieved by the model generation engine 310 to generate one or more 2D and 3D models of the scanned workpiece. The model generation engine 310 may include software modules suitable for processing the scanning data and generating 3D models (showing contours, shape, volume, texture, etc.), 2D models (showing height and outline, for example), or other images. For example, the model generation engine 310 may run proprietary DSI Q-LINK® segmentation software developed by Design Systems, Inc. of Redmond, Washington.

[0066] The model generation engine 310 may execute one or more feature recognition modules to generate a 2D view / image / model from the scanning data. One or more feature recognition modules may be executed by the model generation engine 310 in addition to, or instead of, by the sensor data processing engine 308. For example, the model generation engine 310 may be configured to generate at least one of the following: fat recognition (FRS) object view, laser scattering object view, and height-mode object view. The 2D model of the workpiece may be used to identify the contour of the workpiece, the shape irregularities of the workpiece, the height, etc.

[0067] The model generation engine 310 may also be configured to generate a 3D model of the scanned workpiece. The 3D model of the workpiece may be used to determine how to cut the workpiece into desired portions and / or how to trim the workpiece to a desired overall shape.

[0068] Cutting, splitting, trimming, etc., of the workpiece can be performed by the workpiece processing engine 312. The workpiece processing engine 312 can analyze data received from the sensor data processing engine 308 and / or the model generation engine 310 to determine the cutting path and other processing steps (e.g., sorting, picking, retrieval, etc.) for the cutter station 122. The workpiece processing engine 312 may also receive / process instructions from the QA computing device 111 to make any necessary cuts or processing adjustments to ensure that the workpiece to be processed is within the required specifications, such as optimizing the use of the processed workpiece.

[0069] For example, if the piece processing engine 312 receives instructions from the QA computing device 111 regarding process adjustments, the processing system 104 may perform further scans at the scanning station 120 and analyze the scan data to determine how to cut, split, trim, or otherwise process at least some of the remaining or raw pieces for its production process. In some examples, if the piece processing engine 312 receives instructions from the QA computing device 111, the piece processing engine 312 may be able to perform all necessary cutting or other processing of the piece without any further scanning or analysis.

[0070] Returning to Figure 2, after any processing (e.g., cutting, splitting, trimming, etc.), the processed piece (and / or any material removed from the processed piece) may be transported to other locations such as an extraction conveyor (e.g., the singulator of the transport system 116 as shown in Figure 6), a storage bin, a sorter 126, a packager 128, or a pickup station 124. The pickup station 124, sorter 126, and packager 128 can receive instructions from the processor computing device 130 based on the processed scanning data from the scanning station 120.

[0071] For example, if a processed piece is divided into pieces, the processor computing device 130 may instruct the pickup station 124 and / or sorter 126 to remove trim pieces or other unwanted pieces from the conveyor or to process them further. Trim pieces may be removed or to process them further based on, for example, a known location on the conveyor resulting from a cutting command, scanning data from the scanning station 120 indicating that the incoming product was not the correct shape / size / type to produce a particular piece, or QA data indicating that the divided pieces are out of specification. In another example, the processor computing device 130 may instruct the pickup station 124 and / or sorter 126 to transfer all pieces of a particular type to a designated conveyor, bin, etc., for packaging together, based on information received by the QA computing device 111, etc.

[0072] Figures 1, 2, and 3 show specific components and subassemblies of the machining system, but it should be understood that any other suitable configuration of the machining components may be used. For example, the machining system 104 may incorporate embodiments of the systems shown and described in U.S. Patent No. 7,651388, entitled "Portioning apparatus and method", U.S. Patent No. 7,672752, entitled "Sorting workpieces to be portioned into various end products to optimally meet overall production goals", and U.S. Patent No. 8,688267, entitled "Classifying workpieces to be portioned into various end products to optimally meet overall production goals", which are incorporated herein by reference in their entirety.

[0073] Referring again to Figure 1, an exemplary embodiment of the QA station 106 is described here. As described above, the QA station 106 is configured to collect and process QA data for evaluating one or more attributes of a QA sample processed or to be processed by the processing system 104. The QA station 106 may also include an integrated QA scanning system 108 and a weight measurement assembly 110. A QA computing device 111, although shown separately, may define an integral part of the QA station 106. Generally, the QA station 106 may be configured to take in and package QA image data, QA weight data, and any other relevant data about the QA sample ("QA data") and transmit it to a computing device (such as the QA computing device 111, the monitoring system 112, the model management computing device 113, and / or the processor computing device 130) for performing a QA analysis or otherwise processing the QA data. The QA analysis may include determining whether the QA sample is within specifications and, optionally, determining any corresponding steps to be taken.

[0074] First, an exemplary embodiment of the QA scanning system 108 will be described. Figure 1 shows a block diagram illustrating a non-limiting example of a QA scanning system 108 according to various aspects of the present disclosure. In the illustrated example, the QA scanning system 108 includes an image sensor assembly 132 having at least one image sensor for capturing QA image data of a QA sample being observed. The QA scanning system 108 may further include an illumination assembly 136 configured to adequately illuminate the QA sample being captured by the image sensor assembly 132. The QA scanning system 108 may further include an image processor 134, which may be used to receive, process, and package the QA image data captured by the image sensor assembly 132 for transmission to a QA computing device 111 or another computing device (such as a processor computing device 130 of the processing system 104). The QA scanning system 108 may include any other components that are necessary or appropriate for the application and / or environment, such as a heater, a transport system, etc. Furthermore, the QA scanning system 108 used in the systems and methods described herein excludes any type of scanning that may be performed by human observation that does not support the required processing speed and accuracy of the workpiece processing management system 102.

[0075] First, the image sensor assembly 132 of the QA scanning system 108 will be described in detail. The image sensor assembly 132 may include any suitable image sensor or combination of image sensors for acquiring and / or generating QA image data related to evaluating the quality of an intended QA sample. In some examples, the image sensor assembly 132 may include one or more cameras for acquiring still images of the QA sample and the support surface on which it is placed. For example, the illumination assembly 136 may include one or more optical still cameras, such as a grayscale camera, an RGB camera, an infrared (IR) and / or UV camera, a thermal imaging device, a thermal camera, a charge-coupled device (CCD), etc. The optical still cameras may be used to acquire and / or generate one or more complete still images of the QA sample to detect specific characteristics such as the outer contour or periphery of the QA sample, or the depth or height of the QA sample. Such QR still image data showing the outer contour of the QA sample may be used to determine whether the product meets the shape and / or size specifications of the product, such as chicken breast with the required length / width and / or a specific bun coverage rate.

[0076] The image sensor assembly 132 may also include one or more additional stationary cameras, such as multispectral or hyperspectral cameras. Additional stationary cameras, such as multispectral or hyperspectral cameras, may be used to acquire images / data of specific areas or characteristics of a QA sample, such as blood spots, fat streaks, or lignified chicken muscle striae.

[0077] In some examples, the image sensor assembly 132 may be movable relative to a stationary or fixed QA sample. For example, in one example, the image sensor assembly 132 may include one or more image sensors positioned at the end of a robotic arm so that numerous stationary images of the QA sample and / or its surroundings can be taken using the image sensors. In such an example, the image sensor assembly 132 may include one or more stationary cameras or other cameras (e.g., stereo cameras). In such an example, the robot-controlled image sensor assembly 132 may capture enough images of the QA sample to generate a 3D model of the QA sample. The support surface on which the QA sample is placed may also be captured by the image sensors to determine the distance of the sensors from the surface, which can be used when calculating the height of the product. In addition to or instead, the image sensor assembly 132 may include one or more image sensors mounted on a linear actuator, as described below.

[0078] It can be understood that capturing images from various angles can generate more reliable and complete image data of QA samples. In other words, using a stationary camera or another camera that can move relative to the imaging support surface allows for observation of QA samples from more / better angles, thereby eliminating gaps in QA image data and improving the accuracy of QA images.

[0079] In the examples described herein, the image sensor assembly 132 may also include, or instead of, image sensor technology suitable for capturing image data necessary to generate a 3D model of the QA sample and / or a 2D representation of the height or altitude of the scene. In some examples, the image sensor assembly 132 includes at least one of the following: a 3D vision system or LiDAR (Light Detection and Ranging), structured optical scanning, or 3D laser scanning technology such as photogrammetry, a stereo depth camera, a time-of-flight (ToF) stereo camera, or a combination thereof.

[0080] In some examples, the image sensor assembly 132 includes a stereo depth (e.g., stereoscopic) camera configured to generate a 3D depth image or height map of the QA sample. For example, at least one Intel RealSense Depth camera (e.g., D405) may be used. When one or more stereo depth cameras are used, or in a similar image sensor assembly configuration, each camera may be configured to optimize the contrast of the captured image (e.g., minimizing white pixels) to optimize object segmentation, such as separating the QA sample from the background and reducing clutter noise in the image. In this regard, each camera may also be configured to optimize resolution, exposure, frame rate, etc., to maximize the accuracy of the still camera image. In some examples, multiple still images of the same QA sample may be acquired to define the QA image, and a time filter may be used to filter out temporal noise from the image.

[0081] In one example, the image sensor assembly 132 includes a structured light source and scanner (also referred to herein as a “3D laser scanner,” etc.) configured to capture QA sample depth and surface information to generate a height map or 3D model of a product and / or a 2D representation of the height or elevation of a scene. In one example, the structured light source / scanner is a SICK® TriSpector1000 3D laser scanner. 3D laser scanners, such as the SICK® TriSpector1000 3D laser scanner, digitally capture the size and shape of a physical object using a laser beam. Generally, in a 3D laser scanner, a laser probe projects a laser beam onto the surface supporting the object to be scanned, while a sensor camera continuously records the changing distance and shape of the laser beam in three dimensions (XYZ) as the laser beam sweeps along the object. The shape of the object appears on a computer monitor as millions of points called a “point cloud” as the laser moves around capturing the entire surface shape of the object.

[0082] In order for the laser to sweep along the QA sample, the structured optical scanner may be movable relative to the imaging support surface on which the QA sample is placed, and / or the imaging support surface may be movable relative to the structured optical scanner. For example, the structured optical scanner may be movable relative to the imaging support surface by a robot or actuator. In this way, when the structured optical scanner is moved by a robot / actuator, the laser beam emitted from the scanner can sweep along the QA sample and the imaging support surface.

[0083] In one example, the structured optical scanner is a SICK® TriSpector1000 3D laser scanner mounted on a linear actuator, such as a belt-driven linear actuator available from Festo Corporation in Islandia, New York. In one specific example, a Festo® toothed belt-axis ELGE-TB actuator is used. The linear actuator may include an integrated encoder that tracks the scanner's movement to accurately capture image data relative to the laser beam's sweep distance. In addition, the linear actuator and scanner may be I / O link-enabled to facilitate integration.

[0084] In other examples, the QA sample may be moved relative to a fixed structured optical scanner using a transport system such as an endless conveyor belt. In such examples, the endless conveyor belt may define the imaging support surface. An encoder can track the movement of the belt to accurately capture image data relative to the sweep distance of the laser beam.

[0085] In some examples, the image sensor assembly 132 may further include a still optical camera, a stereo camera, etc., mounted on a linear actuator such as one of the cameras described above. In such examples, the image sensor assembly 132 can capture still images of the QA sample at various angles in addition to capturing structured optical scanning. It can be seen that by capturing both still images at various angles and structured optical scanning, it is possible to generate more reliable and complete image data of the QA sample.

[0086] In one example, the image sensor assembly 132 may include a robot capable of identifying and picking up (e.g., picking) one of the QA samples being transported within the processing system 104. For example, a 3D laser scanner (such as the SICK® TriSpector1000 3D laser scanner) may be fixed to the end of a robot arm that also has a gripper, and the robot may move the 3D laser scanner over the QA sample for scanning, pick up the QA sample from the conveyor, and place the QA sample on a scale at the QA station 106 to acquire weight data. The 3D laser scanning data may be used to generate a 3D model for determining the height and volume of the QA sample, and the height and volume of the QA sample may be combined with the weight data to obtain density. Such an example can simplify the QA station as only a scale is required and can avoid scanning during weighing. The robot may be a relatively inexpensive robot capable of identifying and picking up several samples per minute, for example, which will be placed on the QA station platform. For example, the robot used could be a VIM-303 robot available from Visual Robotic Systems, Inc. in Eugene, Oregon. In another example, the robot could be a Universal Robots UR series robot available from Buchanan Automation in Snohomish, Washington.

[0087] As described above, the QA scanning system 108 may include an illumination assembly 136 configured to adequately illuminate the QA sample being captured by the camera of the image sensor assembly 132. When a stationary camera is used, the illumination assembly 136 may include a light source configured to project a flash of light onto the QA sample and the support surface on which it rests when the image is captured by the camera. In other examples, the light source may generate a constant low level of light projected onto the QA sample / support surface. The light source may be one or more of the following: strobe illumination mounted near the camera lens (e.g., camera flash), ring illumination surrounding the camera lens, ambient light, etc. The illumination assembly 136 may further include a shroud assembly or other devices (e.g., air knife, mist discharge system, etc.) to adequately control the illumination conditions regardless of the environment.

[0088] The QA scanning system 108 may further include an image processor 134, which may be used to receive, process, package, and / or transmit QA image data captured by the image sensor assembly 132 for transmission to the QA computing device 111 or another computing device (such as the monitoring system 112, the model management computing device 113, and / or the processor computing device 130 of the processing system 104). In this regard, the image processor 134 may receive still images, 3D laser scan images, etc., and may perform any pre-processing or post-processing for transmission of the QA image data to the QA computing device 111 or another computing device. For example, the image processor 134 may transmit the QA image data to the model management computing device 113 for training one or more machine learning models, as described herein. Any image data generated by the cameras and / or scanners of the QA station 106, whether pre-processed or post-processed, may be simply referred to herein as “QA image data”.

[0089] Preprocessing or postprocessing may include generating views from image data, formatting image data and / or views generated from image data, adding metadata to image data, and packaging / compressing / converting image data for transmission to QA computing device 111 or another computing device. For example, one or more imaging and / or calibration methods described in U.S. Patents 8,839,949, 1,047,1619, 1,0654,185, 1,072,1947, 1,147,5977, 1,042,7882, 1,086,9489, 1,126,6156, and 1,157,0998, which are incorporated in their entirety by reference, may be used for preprocessing. It should be understood that some or all of the preprocessing or postprocessing may instead be performed by QA computing device 111 or another computing device (such as model management computing device 113).

[0090] In some examples, the image processor 134 may include one or more formatting modules configured to format image data for optimal transfer to and / or processing by the QA computing device 111 or another computing device. For example, the formatting modules of the image processor 134 may perform at least one of the following: image data transformation, image data resizing, image data labeling, image data expansion, etc. In certain examples of still images, the formatting modules may perform at least one of the following: image data grayscale conversion, image data translation, image data rotation, image data scaling / resizing, image data contrast adjustment, image data contrast modification, image data adaptation to specific model constraints, etc.

[0091] In some examples, the image processor 134 may process 3D laser scanning data to generate a view / image from the 3D scanning data. For example, the image processor 134 may receive 3D laser scanning sensor data from the image sensor assembly 132 and generate a 3D point cloud of the scene. The image processor 134 may also package the 3D point cloud data in a format suitable for processing by a computing device, such as to create a 3D model of the QA sample and / or a 2D representation of the height or elevation of the scene. In this regard, the image processor 134 may package the 3D point cloud data in a format suitable for transmission and / or further processing, such as CSV, .las, .ply, .png, .pdf, etc. Alternatively, the 3D model of the QA sample and / or the 2D representation of the height or elevation of the scene may be generated by another computing device, such as the QA computing device 111.

[0092] The image processor 134 may include, or otherwise incorporate, access to an appropriate structured optical / 3D laser scanning software program for configuring and setting up the scanner. The 3D scanning software program may be integrated into the 3D laser scanner or may be accessible on a remote or cloud-based server. If the 3D laser scanner is a SICK® TriSpector1000 3D laser scanner, integrated SOPAS software may be used to configure the scanner.

[0093] A 3D laser scanning configuration may include defining parameters such as the object, plane, region of interest, field of view, blobbing (what is being blobbed, size, minimum width rectangle, etc.), pixel size (e.g., based on encoder count), sensor, scan trigger point, data preprocessing, and data output transmission. For example, the imaging support surface on which the QA sample is placed during scanning (and optionally during weighing) may be defined as the plane. The plane may be used as a reference from which all height measurements can be determined.

[0094] Using a task module within integrated SOPAS software or a similar software tool, a plane may be defined by using a predetermined percentile (e.g., top 6%) of points that are flat when scanned, ignoring the area where the QA sample typically lies on the surface (e.g., a circle substantially centered on the surface). A plane can be constructed from these points. The plane may be verified for each 3D scan, with or without the QA sample placed on the imaging support surface. In this way, any height measurement of the QA sample can be calculated for each scanned image by referring to the imaging support surface plane, and is therefore always effectively calibrated.

[0095] In other examples, the plane representing the imaging support surface may be calculated as parallel to the background of the still camera depth image using one or more plane equations, for example, after the black background of the imaging surface is defined as the region of interest (ROI) of the image. A part or feature of the background in the depth image may be used to obtain a pixel sample of the background, for example, by defining the corners or portions of the background. The plane can be calculated to be substantially parallel to the corners or portions of the background (for example, the average of pixels may be used to estimate the plane if it is not flat). After defining the plane, a 3D affine transformation matrix may be calculated to allow the QA sample image to be rotated and translated as a 3D object on the background plane.

[0096] After scanning, the image processor 134 may run one or more modules (e.g., within SOPAS) to blob the QA samples and separate them from the plane. In one example, an image segmentation machine learning model may be used to separate the QA samples (e.g., shown as a color image) from the plane (e.g., shown as a black background). For example, the image segmentation machine learning model may incorporate the Segment Anything Model (SAM) available from Meta AI, FastSAM from Ultralytics, or another suitable image segmentation model that uses an image segmentation technique.

[0097] Each QA project may be identified as a "job" by associating scanned and still image data with the QA sample plant number, system line / machine number, processing system sub-lane number, QA bin number, product SKU or serial number, QA machine number, date / time, etc. In this regard, the image processor 134 may add metadata to any QA image data to indicate the origin of the QA sample being scanned / imaged.

[0098] The image processor 134 may format or package QA image data for transmission to the QA computing device 111 or another computing device in response to a request and / or triggered when other QA data (e.g., weight data) for a particular QA sample or job is ready for transmission, such as when acquired by a camera, scanner, etc. For example, for each scan and still image, the QA image data may be exported to the QA computing device 111 or another computing device via a command channel. In this regard, the image processor 134 may be configured with an output interface for transmitting the QA image data to the QA computing device 111 or another computing device such as ftp. In one example, the image processor 134 may output a .png file containing image data (e.g., FRS (reflectance) data, height data (peak height, minimum height, etc.), encoder data, etc.) along with any command sequence and any other data to the QA computing device 111 or another computing device.

[0099] As described above, the QA station 106 may further include a gravimetric assembly 110 configured to collect QA weight data of a QA sample. Generally, the gravimetric assembly 110 is configured to capture QA weight data of a QA sample, which may be used to evaluate one or more attributes of the QA sample, such as weight, density, and / or volume, for QA analysis. The QA computing device 111 may then use the QA weight data to determine, for example, whether the QA sample is within specifications.

[0100] Herein, an example of a weight measurement assembly 110 for use in a QA station 106 is described. Figure 1 shows a block diagram illustrating a non-limiting example of a weight measurement assembly 110 according to various aspects of the present disclosure. In the illustrated example, the weight measurement assembly 110 includes a weight input device 138 for acquiring QA weight data and an output device 140 for outputting the QA weight data to a processing device for QA analysis. For example, the output device 140 may output the QA weight data to a QA computing device 111 via appropriate wired or wireless means, such as via Ethernet, serial, or USB connection, Bluetooth, etc. In some examples, the QA weight data may be manually output (or input) to the QA computing device 111 by a technician.

[0101] The weight input device 138 may be any suitable weighing device that is suitable for capturing a substantially accurate weight of the QA sample, whether the QA sample is stationary or being moved by a conveying system. In one example, the weight input device 138 is a simple bench scale, such as an analog or digital scale (e.g., iScale), configured to capture an analog or digital weight reading of the QA sample. In one example, the weight input device 138 is a high-precision platform or bench scale having an integrated load cell and controller of food-grade stainless steel, optionally. The load cell may have an accuracy of up to one-tenth of a gram, depending on the application. Preferably, the scale is I / O-enabled for ease of installation and use.

[0102] The scale platform may also define an imaging support surface on which the QA sample is placed for imaging by the image sensor assembly 132. In this regard, the QA sample can be weighed while simultaneously being imaged / scanned by the image sensor assembly 132. To support accurate imaging / scanning, the scale platform may preferably remain substantially stationary (not moving vertically) when supporting the QA sample, especially if the scale platform is calibrated before weighing and imaging of the QA sample as described above. In other examples, the scale platform calibration may be performed while the QA sample is on the platform, such as by defining the platform as a plane in the scanner configuration as described above. In such cases, movement of the scale platform after receiving the QA sample does not adversely affect simultaneous imaging.

[0103] In some examples, the weight input device 138 may be a support device that moves in accordance with the weight of the QA sample, and the degree of movement of the support device is measured by the image sensor assembly 132. For example, the imaging support surface may be supported by a high-precision spring that allows vertical movement of the QA sample under increased load / weight. The vertical movement captured by the image sensor assembly 132 can be converted into a weight measurement of the QA sample by the image processor 134 and / or the QA computing device 111.

[0104] In some examples, the weight input device 138 may be incorporated into a transport system having a conveyor belt that moves products on the weight input device 138. For example, the weight input device 138 may be configured as a load cell or weighing cell positioned below a portion of the conveyor belt so that weight measurements of QA samples can be obtained as the QA samples pass through the weighing cell. In this regard, the weight input device 138 may be configured as a weighing deck. In situations where a weighing deck is already installed in a food processing plant for weighing / sorting products processed by processing machines, the data captured by the weighing deck may be sent to a QA computing device 111 or another computing device (such as a model management computing device 113 for training one or more machine learning models and / or a monitoring system 112 for QA weight analysis and reporting) for QA analysis and processing.

[0105] In some examples, the weight input device 138 may be defined by the conveyor belt itself, and the vertical displacement of the belt caused by the weight of the QA sample may be measured to determine the product weight. For example, the vertical displacement of the conveyor belt may be captured in an image by an image sensor assembly 132 and then processed by an image processor 134 or a QA computing device 111 to determine the product weight.

[0106] If the image sensor assembly is positioned above an imaging support surface that is vertically displaceable for measuring weight, and the vertical displacement of the surface is not used for measuring weight, the QA station 106 may be configured to accommodate such vertical movement when processing QA image data. For example, in some examples, the image sensor assembly may move vertically with the imaging support surface such that the distance between the image sensor assembly and the imaging support surface remains substantially constant. In other examples, the image processor 134 and / or QA computing device 111 may take into account such vertical displacement of the imaging support surface (for example, by running a dynamic calibration module). In the examples described herein, the height of the imaging support surface or platform may be calibrated each time a QA sample is weighed / imported / scanned by defining the surface as a plane in the scanner configuration and / or image, for example.

[0107] Herein, with reference to Figures 4A to 4D, an example of a QA station 406 formed according to the systems and methods disclosed herein is described. Parts of QA station 406 similar to those used in QA station 106 are given the same reference number, except that they are in the 400s. Generally, QA station 406 includes an integrated QA scanning system 408, a weight measuring assembly 410, and at least one QA computing device (not shown).

[0108] Using an integrated scanning system, a weighting assembly, and a computing device, the QA station 406 can accurately acquire image data and weight data substantially simultaneously. In this way, rapid and accurate QA data can be obtained for performing QA analysis of QA samples. For example, a QA station having an integrated QA scanning system, a weighting assembly, and a computing device, such as the QA station 406 shown and described herein, avoids error-prone "double handling" of QA samples. More specifically, if QA weighting of a QA sample is performed in a separate assembly or system from the QA scanning system, correlation of data between the weighting assembly and the scanning system is necessary to ensure that the weight and scanning data relate to the same QA sample. Any mismatch in data order can cause errors and result in invalid and unusable QA data.

[0109] The QA station 406 shown and described herein is an example of a QA station having an integrated QA scanning system, a weight measurement assembly, and a computing device. It should be understood that other configurations are possible.

[0110] First, the QA scanning system 408 of the QA station 406 will be described. The QA scanning system 408 may include an image sensor assembly 432, an illumination assembly (not shown), and at least one image processor. A housing 464 may enclose at least some of the components of the QA station 406 for hygiene, ease of cleaning, ease of use / storage, etc.

[0111] The image sensor assembly 432 may include at least one still camera 450 and at least a structured optical scanner 452 positioned above an imaging support surface 453 on which a QA sample may be placed during imaging. The at least one still camera 450 may be mounted inside the housing 464 or on a structure within the housing. The at least one still camera 450 may be any optical still camera suitable for the intended application, such as one or more of a grayscale camera, RGB camera, IR camera, UV camera, CCD, multispectral or hyperspectral camera, etc.

[0112] Illumination assemblies, which may also be mounted inside the housing 464 or on structures within the housing, may be any suitable illumination assemblies configured to provide sufficient illumination of the QA sample and the imaging support surface 453 for image acquisition by at least one static camera 450. For example, the illumination assembly may include strobe illumination configured to illuminate the imaging support surface 453 when a snapshot is taken by at least one static camera 450. In some examples, at least one static camera 450 may be omitted, in which case the illumination assembly may also be omitted.

[0113] The structured optical scanner 452 may also be mounted inside the housing 464 or on a structure within the housing. More specifically, the structured optical scanner 452 is mounted on a linear actuator 460 which is mounted inside the housing 464 or on a structure within the housing. In this way, the structured optical scanner 452 is movable relative to the imaging support surface 453 so that the scanner's laser beam can sweep over the length of the imaging support surface 453. The linear actuator 460 may include an integrated encoder that tracks the movement of the structured optical scanner 452 to accurately capture image data with respect to the sweep distance of the laser beam. In one example, the structured optical scanner 452 is a SICK® TriSpector1000 3D laser scanner, and the linear actuator is a belt-driven linear actuator such as a Festo® toothed belt-axis ELGE-TB actuator with an encoder.

[0114] The structured optical scanner 452 may include an integrated image processor configured to run scanner software configurable to suit the intended application. The scanner software may be used to configure the structured optical scanner 452 for the intended application by defining parameters such as objects, planes, regions of interest, fields of view, blobbing, pixel size, sensors, and scan trigger points. The configuration may include defining the plane of the upper surface of the imaging support surface 453 using task modules within the software so that arbitrary height data and measurements can be determined by reference to the plane, as described above. The configuration may also include using task modules within the software to set up arbitrary data preprocessing, data output transmission commands, etc. When the SICK® TriSpector1000 3D laser scanner is used, the integrated SOPAS software may be used to configure the scanner as described above.

[0115] Herein, the weight measurement assembly 410 of the QA station 406 is described. The weight measurement assembly 410 may generally include a weight input device 438 configured as a high-accuracy bench scale or load cell for acquiring QA weight data. In the illustrated example, the load cell defining the platform 454 is positioned on adjustable legs and is removable from the internal compartment of the housing 464. However, in other examples, the weight input device 438 includes a load cell integrated into the bottom of the housing 464. In any case, if the weight input device 438 is intended for use in an industrial food processing setting, it may be washable and made from food-grade material. To aid in cleaning, the scale platform 454 and any internal horizontal parts of the housing 464 may be at a slight angle to facilitate discharge from the front opening of the housing 464.

[0116] The weight measurement assembly 410 further includes an output device (not shown) for outputting QA weight data to a processing device for QA analysis. The output device may be configured as a controller separate from or integrated with the weight input device 438. The output device may be configured to perform any pre-processing or post-processing of the weight data and may transmit it to the QA computing device 111 and / or processor computing device 130 via appropriate wired or wireless means, such as via Ethernet, serial, or USB connection, Bluetooth, etc.

[0117] As described above, the QA station 406 is configured to acquire image and weight data substantially simultaneously. In other words, scanning and still images of the QA sample may be acquired by the image sensor assembly 432 while the QA sample is being weighed or while the QA sample is on the platform 454. In this regard, the platform 454 of the weight input device 438 also defines an imaging support surface 453. Thus, the platform 454 of the weight input device 438 may be located below the image sensor assembly 432, such as below a window defined in the upper part of the housing 464 through which imaging may be performed. In this way, once the QA sample is placed on the platform 454 for weighing, at least one still camera 450 and structured optical scanner 452 may acquire still images and 3D laser scanning data. Acquiring image and weight data substantially simultaneously speeds up the QA process compared to, for example, a system including separate weight and imaging assemblies.

[0118] Herein, we describe at least one QA computing device of the QA station 406. The at least one QA computing device may be defined by a single device or a set of devices. For example, the at least one QA computing device may incorporate one or more functional embodiments of the QA computing device 111 and / or processor computing device 130, which are further described below. In addition, the at least one QA computing device may include any necessary controllers, adapters, etc., of any of the components of the QA scanning system 408 and the weighing assembly 410. Notably, at least one static camera 450, structured optical scanner 452, linear actuator 460, and weighing assembly 410 may be I / O link compatible to facilitate installation and control. Any wired connections between components may be sealed inside the housing 464 or through appropriate seals housed in the housing wall so that the QA station 406 has washdown compatibility.

[0119] The QA station 406 may include a display unit 468 fixed within the upper portion of the housing 464. The display unit 468 may be configured to display information to the user through a GUI, such as a graphical user interface (GUI) associated with scanner software (e.g., SOPAS) or another computing device (e.g., QA computing device 411 or processor computing device 130). The display unit 468 may be coverable / sealable by a cover to ensure washdown capability.

[0120] Alternative examples of QA stations 506 and 606 are shown in Figures 5A-5B and 6, respectively. Referring to Figures 5A and 5B, parts similar to those used in QA station 106 are given the same reference numbers, except that they are in the 500s, and QA station 506 may include a platform 554 on which QA samples to be analyzed for QA can be placed. The image sensor assembly 532 may be positioned above the platform 554 such that its lens 533 is directed downward toward the platform 554 to capture a plan view of the QA sample. The distance between the image sensor assembly 532 and the platform 554 may be adjustable by any suitable height adjustment assembly. A suitable illumination assembly 536 may be positioned relative to the image sensor assembly 532 to adequately illuminate the camera's field of view. For example, a ring illumination may surround the downward-facing lens of the image sensor assembly 532 so as to adequately illuminate the camera's field of view including the QA sample.

[0121] In the examples shown in Figures 5A and 5B, a single image sensor, such as an optical still camera or stereo camera, may be used to capture an image of a stationary product. However, it should be understood that in some examples, two or more image sensors may be required. For example, a second and third image sensor may be placed near the first image sensor to capture an alternative view of the QA sample, for example, to create a more accurate 3D model of the product. If a single camera is used to look down on the QA sample, voids and undercuts beneath the product may not be detected in the image. Additional sensors may be used to view the QA sample at an oblique angle, thereby capturing a view of the product that may reveal, for example, any voids or undercuts beneath the product.

[0122] In addition, or in an alternative configuration, a single image sensor, such as an optical static camera or stereo camera, may be movable relative to the QA sample to capture multiple views of the QA sample. As described above, the image sensor may be mounted on the end of a robot arm, a linear actuator, or the like.

[0123] Figure 6 shows QA station 606 (similar parts are given similar reference numbers, except that they are in the 600s), which includes a moving platform or transport system for supporting the QA sample as it passes through the image sensor assembly 632 and the weighing assembly 610. The image sensor assembly 632 may include a suitable number of image sensors fixed to the transport system to capture sufficient images of the QA sample as it moves through the image sensor assembly 632. In another example, the image sensor assembly 632 may include at least one image sensor positioned at the end of a robotic arm so that a number of still images of the QA sample and / or its surroundings can be taken as the QA sample is transported.

[0124] The weight input device of the weight measurement assembly 610 may be positioned below the image sensor assembly 632 so that an image of the QA sample can be captured during and / or before and / or after the weight measurement is performed. In one example, the weight input device is configured as a load cell or weighing cell positioned below a belt portion of a conveyor belt, and as a result, the weight of the QA sample can be measured as it passes through the weighing cell. In this regard, the weight input device may be configured as a weighing deck. In some examples, the weight input device 638 may be defined by the conveyor belt itself, and the vertical displacement of the belt caused by the weight of the QA sample can be measured (through images, etc.) to determine the product weight.

[0125] As described above, the processing system 104 may be configured to perform processing of the QA samples before they are scanned / weighed by the QA station. In this way, the quality of the processed pieces (such as whether the pieces are within specifications) can be analyzed using data collected from the QA station. For example, referring to Figures 2 and 6, some or all of the QA samples processed by the processing system 104 may be sorted by the sorter 126 or picked by the pickup station 124 and led to the QA station 606, for example, through the singulator 620. The singulator 620 may be configured to sequentially position multiple QA samples, such as some or all of the processed pieces, for analysis by the QA station 606. In this regard, the QA station described herein may be incorporated into a food processing line so that some or all of the processed pieces processed by the processing system 104 can pass through the QA station for QA analysis after processing.

[0126] In some examples, a QA station may be located upstream of the processing system 104 to collect QA data related to incoming workpieces, either additionally or alternatively. For example, in the example of Figure 6, some or all of the workpieces to be processed by the processing system 104 may be loaded into the singulator 620 for imaging / weighing by the QA station 606 (hence the "QA sample"). After passing through the QA station 606, the QA sample is loaded manually or automatically into the transport system 116 of the processing system 104 for processing. In other examples, such as those with fixed QA stations like QA stations 406 and 506, the QA sample may be manually placed into the QA station and then loaded manually or automatically into the transport system 116 of the processing system 104 for processing.

[0127] In some examples, the QA station may be located upstream and / or downstream of a heat treatment system or cooking line that includes ovens, fryers, steamers, roasters, etc. The QA station may be used to collect QA image data and QA weight data to evaluate food safety (e.g., whether the QA sample is thoroughly cooked), food quality (e.g., whether the color of the QA sample indicates a lack of browning, charring, etc.), or other attributes. For example, the QA station may be located upstream of an oven, and images of incoming QA samples may be captured by an image sensor assembly to generate a height map or 3D model of the product. In some cases, incoming QA samples may also be weighed by a weighing assembly to obtain weight data. In other examples, incoming QA samples are weighed only if they are estimated to have a certain minimum volume, as determined from the height map / 3D model.

[0128] Based on the analysis of QA image data and / or QA weight data, the temperature of the QA sample may be obtained during and / or after the cooking process. For example, QA samples exceeding a certain volume and / or weight may be evaluated during and / or after the cooking process to determine whether the QA sample has been properly cooked. By knowing the volume and optionally the weight of the heat-treated QA sample, adjustments to the cooking process may be made based on the measured temperature. In addition, or in an alternative form, adjustments to the cooking process may be made based on the color of the product detected in a color image generated by an image processor.

[0129] The method for selecting the QA sample to be measured, and the temperature measurements and analyses performed on the heat-treated QA sample, may be carried out in accordance with the systems and methods described in U.S. Patent No. 9,366579, entitled “Thermal process control,” U.S. Patent No. 9,366580, entitled “Thermal measurement and process control,” and U.S. Provisional Patent Application No. 63 / 517204, entitled “Programmed Food Equilibration System And Method For Real Time Process Yield In A Thermal Process,” the entire disclosure of which is incorporated herein by reference. Image data from the image sensor assembly 132 (and optionally weight data from the gravimetric assembly 110) may be used to select the QA sample to be measured (in contrast to, or in addition to, using, machine scanning data as described in, for example, U.S. Patent Nos. 9,366579 and U.S. Patent Nos. 9,366580).

[0130] If the QA station is located downstream of the heat treatment line, in addition to the image sensor assembly and optionally the gravimetric assembly, the QA station may be configured to include a temperature probe assembly configured to measure and output the temperature of the heat-treated QA sample. For example, a temperature probe assembly such as that described in U.S. Provisional Patent Application No. 63 / 517204 may be used. Temperature data may be transmitted to the QA computing device 111, the computing device of the heat treatment system, and / or another device communicating with it, so that appropriate heat treatment adjustments can be made to ensure proper cooking. In some cases, temperature data may also be transmitted to a monitoring system 112 for heat treatment QA analysis and reporting, either in addition or instead.

[0131] In some cases, QA stations may be located upstream and / or downstream of the heat treatment line to identify QA samples or pieces that are suitable and / or unsuitable for heat treatment. For example, pieces that are too large or too thick may not cook properly during heat treatment. Similarly, if pieces are stacked together, the stacked pieces may define an overall combined piece size that does not cook properly during the process.

[0132] In this regard, the image sensor assembly of the QA station may be positioned relative to a conveying system that moves pieces in and out of a heat treatment system such as an oven or fryer, in order to identify pieces unsuitable for heat treatment (e.g., too large). The QA image data captured by the image sensor assembly may be used, for example, to locate any pieces larger than the maximum size suitable for heat treatment (e.g., they will not be cooked sufficiently during the process). In some examples, the QA image data generated for some or all of the incoming pieces may be used to generate a height map of the imaged products, and based on the height map analysis (e.g., generated by the QA computing device 111 and / or the processor computing device 130), pieces exceeding a certain volume are picked off the line and / or moved from heat treatment to other processing. In some cases, the QA image data generated for some or all of the incoming pieces may exclude data below a certain height or distance from the image sensor assembly, for example, based on the settings of the image processor. In this way, only products that may be larger than a given volume are analyzed based on the minimum height of the product.

[0133] Figure 7 shows a block diagram illustrating an embodiment of a non-limiting example of a QA computing device 111 configured to perform some or all of the functions of the QA systems and methods described herein. It should be understood that certain functions may instead be performed by other computing devices such as the image processor 134 of the QA scanning system 108, the processor computing device 130 of the processing system 104, or both, and / or other computing devices such as the monitoring system 112 and / or the model management computing device 113. The QA computing device 111 may network with any of the other devices of the QA sample processing management system 102, such as the image processor 134 of the QA scanning system 108, the output device 140 of the weight measurement assembly 110, the processor computing device 130, the monitoring system 112, and the model management computing device 113. Although the QA computing device 111 is described with reference to the QA station 106, it should be understood that the functionality of the QA computing device 111 may be incorporated into any suitable QA station, such as QA station 406, QA station 506, and / or QA station 606.

[0134] Generally, the QA computing device 111 includes a processor 704, a communication interface 706, a computer-readable medium 708, and one or more data stores (e.g., a QA data store 720, a training data store 722, and a QA model data store 724).

[0135] The QA computing device 111 may be implemented by any computing device or set of computing devices, including, but not limited to, desktop computing devices, laptop computing devices, mobile computing devices, edge computing devices, PLCs, server computing devices, computing devices in cloud computing systems, and / or combinations thereof. In some examples, the processor 704 may include any suitable type of general-purpose computer processor. In some examples, the processor 504 may include, but not limited to, one or more dedicated computer processors or AI accelerators optimized for a particular computing task, including graphics processing units (GPUs), visual processing units (VPTs), and tensor processing units (TPUs).

[0136] For example, the QA computing device 111 may be configured as an NVIDIA Jetson Orin package such as the Advantech MIC-711-OX. A TCP / IP connection may be used to transfer data between the QA computing device 111 and the processor computing device 130 and / or other computing devices (e.g., the monitoring system 112 or the model management computing device 113).

[0137] In some examples, the data rate between the QA computing device 111 and the processor computing device 130 and / or other computing devices (e.g., the monitoring system 112 or the model management computing device 113) can be increased by using PCI Firewire or other communication bridges. In other examples, the data rate between the QA computing device 111 and other computing devices can continue to use a common network protocol connection such as TCP / IP, but in the meantime, the processing power of the QA computing device 111 can be increased.

[0138] A communication protocol may be used to reliably and efficiently transmit data between the QA computing device 111 and other computing devices. The communication protocol may be configured as a platform-independent, high-level protocol enabling the use of simple commands. For example, the communication protocol may enable bidirectional communication between the QA computing device 111 and other computing devices. A protocol buffer (Protobuf) may be used to optimize the efficiency of data transfer. This protocol may support both synchronous and asynchronous communication. In some examples, a high-level, restricted API implemented on the QA computing device 111 and / or the processor computing device 130 may be used to validate the transmitted sensor data.

[0139] In some examples, the communication interface 706 includes one or more hardware and / or software interfaces suitable for providing communication links between components. The communication interface 706 may support one or more wired communication technologies (including, but not limited to, Ethernet, FireWire, and USB), one or more wireless communication technologies (including, but not limited to, Wi-Fi, WiMAX, Bluetooth, 2G, 3G, 4G, 5G, and LTE), and / or a combination thereof.

[0140] As shown in the figure, the computer-readable medium 708 stores logic that, in response to execution by one or more processors 704, can cause the QA computing device 111 to provide an image data processing engine 710, a model generation engine 712, a QA analysis engine 714, a machine adjustment engine 716, a data normalization engine 717, and a package optimization engine 718.

[0141] First, an exemplary embodiment of the image data processing engine 710 will be described. The image data processing engine 710 may generally be configured to process, format, store, and / or transmit any image sensor data received from and / or retrieved from the image sensor assembly 132 and / or the image processor 134 of the QA scanning system 108. As described above, the QA scanning system 108 may include an image processor 134 that can receive still image and 3D image sensor data from the image sensor assembly 132. The image processor 134 may package the image sensor data ("QA image data") and send it to the image data processing engine 710, which may process the QA image data, store the QA image data in the QA data store 720 for later retrieval, and / or transmit the QA image data to another device or engine (e.g., a model generation engine 712).

[0142] In some examples, the image data processing engine 710 may include circuitry for generating views / images from scan data and / or processing data from different views (in addition to or instead of the image processor 134). For example, the image data processing engine 710 may be configured to generate at least one of the following from QA image data (e.g., a 2D representation of scene height or elevation): a fat recognition (FRS) object view of a QA sample, a laser scattering object view of a QA sample, and a height-mode object view of a QA sample. The image data processing engine 710 may also include circuitry for running one or more feature recognition modules to identify features of a QA sample, such as object outline, shape, color defects (e.g., bloodstains), striae, etc. In this regard, metadata (e.g., tags, descriptions, etc.) to describe any identified features may be added to the file. The metadata may be used to identify fabricated pieces within the dataset and to train one or more machine learning models used for QA analysis and / or fabricated piece processing, etc. The image data processing engine 710 may also process the QA image data to generate height maps for generating 3D models of the QA samples.

[0143] The image data processing engine 710 may be configured to exclude any background image data when generating QA image data. In this regard, the image data processing engine 710 may be programmed to exclude image data of a specific color (e.g., if the surface on which the product is imaged is a specific color) and / or image data of a specific depth (e.g., if the distance of the surface to the camera is known). Background data or other data may be excluded from data processing by any other appropriate means instead. For example, in addition or in an alternative form, the imaging support surface (see, for example, the imaging support surface 453 shown in Figures 4B and 4C) may be calibrated per scan (e.g., defined as a reference plane) by defining the imaging support surface as a plane using 3D laser scanner integration software (e.g., SOPAS) or a plane calculation method as described above. In this way, when processing QA image data, anything below that plane can be excluded (e.g., the plane is defined per scan or otherwise calibrated).

[0144] QA sample separation image segmentation machine learning models, such as those described in U.S. Provisional Patent Application No. 63 / 588,917, entitled "Edge Computing Device System And Method," may also be used to separate QA samples on the imaging support surface 453. Using a QA sample separation image segmentation machine learning model may be useful when the imaging support surface is non-uniform. In this regard, a QA sample separation image segmentation machine learning model can improve upon known methods that account for non-uniform undersides (e.g., voids) of QA samples, such as those described in U.S. Patent No. 1,1570,998, incorporated herein by reference. The method described in U.S. Patent No. 1,1570,998 can be used to account for non-uniform undersides of a product when calculating the height or weight / mass of a QA sample. However, such a method does not account for non-uniform belt surfaces. Therefore, the method described in U.S. Patent No. 1,1570,998 can be improved by using the QA sample separation image segmentation machine learning model described herein to account for differences in belt height.

[0145] The image data processing engine 710 may be configured to generate images of QA samples having multiple channels or layers. For example, the image data processing engine 710 may combine image files into files having one or more corresponding channels or layers. For instance, an FRS image and a height map image may be combined into a single file having two channels. By combining images into a single file having multiple channels, all necessary image data can be sent to the model generation engine 712 or another computing device in a single file rather than separate files. Thus, the QA computing device 111 or another computing device can process the file at optimal speed and with higher accuracy.

[0146] In some examples, the image data processing engine 710 may include one or more formatting modules configured to format QA image data for optimal transfer to and / or processing by another engine of the QA computing device 111 or another computing device, such as another engine of the QA computing device 111 or another processor computing device 130, a monitoring system 112, and a model management computing device 113. For example, the formatting modules of the image data processing engine 710 may perform at least one of the following: conversion of QA image data, resizing of QA image data, labeling of QA image data, or expansion of QA image data. In a particular example of an image, the formatting module may perform at least one of the following: grayscale conversion of the image, translation of the image, rotation of the image, scaling / resizing of the image, adjustment of the image contrast, modification of the image data contrast, or adaptation of the image to specific model constraints. Any suitable image processing library available to the QA computing device 111 (e.g., Python) may be used to process or format the image data. The processed / formatted QA image data may be stored in the QA data store 720, transmitted to another engine of the QA computing device 111, and / or transmitted to another computing device such as one or more of the processor computing device 130, the monitoring system 112, and the model management computing device 113.

[0147] In one example, the image data processing engine 710 sends the processed / formatted QA image data of the QA sample to the model management computing device 113, or to a computing device (e.g., a cloud-based computing device) that communicates with the model management computing device 113. The QA image data may be used to train one or more machine learning models that can be run by the QA computing device 111 for QA analysis and / or by the processor computing device 130 for workpiece processing and / or machine management. In this regard, for optimal consistency, reliability, and speed, the same or substantially similar processing / formatting may be performed on any data used for both training and using machine learning models.

[0148] In one example, the image data processing engine 710 (and / or image processor 134) may use one or more image processing modules and / or machine learning models trained to identify QA samples for QA analysis. For example, if various split or cut pieces are being transported on a belt, the image data processing engine 710 may be configured to identify the pieces required for QA analysis.

[0149] In some examples, the image data processing engine 710 may run an image data optimization module to select conflicting sensor data (e.g., 3D point cloud data from them) originating from different image sensors and / or sensor sources. As described above, the image sensor assembly 132 may include two or more types of sensor assemblies and / or multiple sensors of the same type, for example, to capture images showing various views of the product and help generate accurate 3D or 2D models. For example, the image sensor assembly 132 may include at least one of the following: a 3D vision system or LiDAR (Light Detection and Ranging), structured optical scanning, or 3D laser scanning technology such as photogrammetry, a stereo camera, a time-of-flight (ToF) stereo camera, or a combination thereof. In this regard, if two or more sensors or two or more types of sensors are used, conflicting sensor data (e.g., sensor data relating to the same 3D point cloud data of the product) may be captured for the QA sample.

[0150] In this regard, the image data processing engine 710 may run an image data optimization module to select conflicting sensor data. The image data optimization module may select 3D point cloud data based on, for example, the resolution of the image sensor data, the processing power required to use the image sensor data, and the compatibility of the image sensor data with other modules.

[0151] The selected 3D point cloud data may also depend on the intended use of the 3D or 2D model. For example, if a 3D model is generated to determine the height and / or volume of a QA sample, such as to be combined with weight measurements to determine the density of the product, the selected 3D point cloud data may include 3D laser scanning data. If a 3D model is generated to determine the surface texture of a QA sample, the selected 3D point cloud data may include stereo camera data (which may produce a sharper image of the texture). In this regard, different types of sensor data may be selected to generate one or more 3D or 2D models that show different attributes of the QA sample.

[0152] In some examples, the image data optimization module of the image data processing engine 710 uses one or more machine learning models (e.g., stored in the QA model data store 724) to identify 3D point cloud data as output from conflicting / competitive sensor data based on the accuracy / efficiency of the models generated for use in QA analysis as input. For example, if a sensor data type is selected for the generation of a particular type of model (e.g., a 3D model showing volume), the sensor data used for that model is stored in the QA data store 720 and can be classified based on the accuracy / efficiency of the models for use in QA analysis (such as being verified by secondary manual testing or other criteria). The machine learning model may be trained to select 3D point cloud data for each model using data such as model accuracy / efficiency or other criteria as training data. The selected 3D point cloud data and / or 2D or 3D models generated using the 3D point cloud data may be stored in the training data store 722 and / or sent to or retrieved from the model generation engine 712 to train the machine learning model.

[0153] It should be noted that any aspect of the image data processing engine 710 may be performed by the model generation engine 712, either by alternatives or additionally.

[0154] Herein, an exemplary embodiment of the model generation engine 712 is described. The model generation engine 712 may generally be configured to generate a 2D or 3D model of a QA sample using image data and / or 3D laser scanning data (e.g., 3D point cloud data) from the image processor 134, the image data processing engine 710, and / or the QA data store 720. In this regard, the model generation engine 712 may include software modules suitable for processing the image data and 3D point cloud data to generate a 3D model (showing contours, shape, volume, texture, etc.), a 2D model (e.g., showing height and outline), or other images. For example, the model generation engine 712 may run proprietary DSI Q-LINK® partitioning software developed by Design Systems, Inc. of Redmond, Washington.

[0155] In the examples described herein, the QA weight data of the QA sample may be verified using a weight measurement assembly 110 (and in a preferred embodiment, the weight is obtained substantially simultaneously with the QA image data as described above). In this regard, the model generation engine 712 and / or the image data processing engine 710 may be configured to associate the QA weight data with the QA image data of the QA sample. The QA weight data may be specified in the metadata of the image data and / or in any 2D or 3D model generated from the QA image data. In this way, when performing a QA analysis of the QA sample, the weight of the QA sample can be considered together with the data of the 2D / 3D model. Any other data may also be considered. As described above, the QA data may include image data, weight data, and any other data about the QA sample collected by the QA system to evaluate one or more attributes of the QA sample. For example, the QA data may include measurements of the QA sample and / or conveyor belt components generated by a high-speed optical micrometer, as described in U.S. Provisional Patent Application No. 63 / 588,917 incorporated herein.

[0156] In some examples, the weight measurement assembly 110 may be configured to capture QA weight data for multiple related QA samples, such as divided pieces of processed meat. It may be beneficial to obtain weight data for each divided processed piece to ensure that the processing system 104 divides each processed meat properly. For example, if the processing system 104 is dividing chicken breast fillets to make chicken nuggets, the quality analysis may include determining whether each nugget has substantially the same shape, weight, and / or size, and / or meets the specifications. In another example, if a butterfly cut of chicken is divided into chicken breast fillets, the quality analysis may include determining whether each chicken breast fillet has substantially the same weight and / or size.

[0157] For example, the weighing assembly 110 may be configured to capture QA weight data for each of a group of related QA samples as each of those samples is added to the fixed or displaceable platform of the weighing assembly 110. For example, the weighing assembly 110 may generate weight data for each of a group of related QA samples when each of the group of related QA samples is placed on a weighting platform such as a bench-scale platform, on a conveyor system with load cells, on a vertically displaceable surface imaged by a QA station 106, etc. The controller of the weighing assembly 110 and / or the QA computing device 111 or another computing device may determine the individual weight of each of the group of related QA samples in real time using a first total weight with N QA samples and a second subsequent total weight with N+1 QA samples. For example, the weight of the N+1 QA sample may be determined by subtracting the first total weight from the second subsequent total weight. In this regard, each weight may be stored in the data store of the controller / computing device for use in weight calculations.

[0158] In addition, or in an alternative form, the weight measurement assembly 110 may be configured such that the controller / computing device of the weight measurement assembly can generate QA weight data for each of a plurality of related QA samples after receiving an input indicating that a sample is to be added to the weight platform. For example, QA samples of a plurality of related QA samples may be added to the weight platform after an operator presses an "add" button, etc. A weight is captured each time after the "add" input (etc.), and the controller of the weight measurement assembly 110 and / or the QA computing device 111 or another computing device may determine the individual weight of each of the plurality of related QA samples in real time. Similarly, the individual weight of each of the plurality of related QA samples may be determined by using a first total weight with N QA samples and a second subsequent total weight with N+1 QA samples (prompted by an "add" input, etc.), and subtracting the first total weight from the second subsequent total weight to determine the weight of the N+1 QA sample. In this regard, each of the weights may be stored again in the data store of the controller / computing device for use in weight calculations. In other examples, the weight may be reset to zero each time an "additional" input is received, and the weight of the added QA sample may be retrieved and recorded.

[0159] In another embodiment, any of the above techniques for obtaining QA weight data for each of a group of related QA samples may be performed by performing the technique in reverse. For example, instead of taking a weight measurement each time a QA sample is added to a weight platform, the weight measurements may be obtained by first adding all related QA samples to the platform, and then taking individual measurements as each piece is removed from the platform.

[0160] In another example, the weighing assembly 110 may be configured to capture the total weight of all of a plurality of related QA samples, and QA image data (optionally captured simultaneously) may be used with the total weight to determine the QA weight data for each of the plurality of related QA samples. For example, all of the plurality of related QA samples may be placed on a weighing platform (e.g., on a bench scale platform, on a conveyor system with load cells, on a vertically displaceable surface imaged by the QA station 106, etc.), and the total weight measurement of all samples may be obtained. The total volume of all of the plurality of related QA samples and the volume of each piece of each of the plurality of related QA samples may be obtained using the QA image data, such as in accordance with one of the techniques described herein. By knowing the total volume of all samples and the volume of each piece, the ratio of the volume of each of the plurality of related QA samples to the total volume of all samples can be obtained. This volume ratio may then be used with the total weight of all of the plurality of related QA samples to determine the weight of each of the plurality of related QA samples. In other words, each of the multiple related QA samples has an individual weight to total weight ratio that is substantially the same as the individual volume to total volume ratio.

[0161] In any case, all QA weight data from multiple related QA samples, as well as QA weight data for each individual piece of multiple related QA samples, may be correlated with QA image data that incorporates all of the multiple related QA samples. For example, the QA weight data may be specified in the metadata of the image data and / or in any 2D or 3D model generated from all of the QA image data of multiple related QA samples. In this way, all of the QA weight data from multiple related QA samples, as well as QA weight data for each individual piece of multiple related QA samples, can be considered together with the data in all of the 2D / 3D models of the multiple related QA samples when performing QA analysis on the QA samples.

[0162] To help illustrate this point, when a QA analysis is performed on chicken nuggets divided from chicken breast fillets, the QA weight data of each chicken nugget produced from the chicken breast fillets, as well as the total weight of the chicken breast fillets divided into nuggets, may be stored and / or correlated with the QA image data of the chicken breast fillets divided into nuggets. Such correlated data may be used to perform a QA analysis on each divided nugget, such as determining whether each nugget is of the correct size, shape, and weight. Such correlated data may also be used to perform a QA analysis on incoming chicken breast fillets that will be divided into nuggets. For example, the correlated data may be used to understand discrepancies between predicted and measured values ​​of incoming chicken breast fillets that will be divided into nuggets. Further embodiments will be better understood in the description below.

[0163] In some examples, the model generation engine 712 and / or the image data processing engine 710 may include one or more formatting modules configured to format the QA data for optimal transfer to the QA computing device 111 or another computing device and / or for processing by the QA computing device 111 or another computing device. For example, the formatting module of the model generation engine 712 may perform at least one of the following: transformation of the QA data, resizing of the QA data, labeling of the QA data, or expansion of the QA data. Any suitable image processing library available to the QA computing device 111 (e.g., Python) may be used to process or format the QA data. The processed / formatted QA data may be stored in the QA data store 720, sent to another engine in the QA computing device 111, and / or sent to another computing device such as one or more of the processor computing device 130, the monitoring system 112, and the model management computing device 113.

[0164] In one example, the model generation engine 712 optionally stores the processed / formatted QA data of the QA sample in the training data store 722 and then sends it to the model management computing device 113, or to a computing device (e.g., a cloud-based computing device) that communicates with the model management computing device 113. The QA data may be used to train one or more machine learning models that can be run by the QA computing device 111 and / or the processor computing device 130. In this regard, for optimal consistency, reliability, and speed, the same or substantially similar processing / formatting may be applied to any data used for both training and using machine learning models. In some examples, one or more machine learning models may be run by the QA analysis engine 714 of the QA computing device 111.

[0165] Herein, an exemplary embodiment of the QA analysis engine 714 is described. The QA analysis engine 714 of the QA computing device 111 may generally be configured to analyze QA data including 2D and / or 3D models or other image data generated by the model generation engine 712, as well as any associated weight data for performing QA analysis of QA samples.

[0166] In an example where QA analysis is performed on a QA sample to perform one or more subsequent actions on that QA sample, the QA analysis engine 714 may first perform a model matching process to ensure that the model or other image data generated by the model generation engine 712 matches the QA sample being analyzed or processed. Such a model matching process may be used, for example, when an incoming product passes through the QA station 106 before being scanned and processed by the processing system 104, so that automatic adjustments can be made to the machine as the product is processed (e.g., adjustment of density settings for accurate division). Such a model matching process may also be used, for example, when a processed product passes through the QA station 106 after being processed by the processing system 104, so that the QA image data can be analyzed together with any scanning data generated by the processing system 104.

[0167] In this regard, models or other image data generated by the model generation engine 712 can be matched with scanning data from the scanning station 120 of the processing system 104. Matching may be necessary to verify that the QA sample scanned at the scanning station 120 is the same as the QA sample scanned at the QA station 106, and / or whether the QA sample has moved or shifted during transfer between conveyors, as described in U.S. Patents No. 1,0654185 and No. 1,0721947 (see above), which are incorporated herein by reference. In this regard, the comparison of image data may be processed by the QA analysis engine 714.

[0168] The matching itself can be performed, for example, by determining discrete positions along the periphery of the QA sample with respect to an XY coordinate system or another coordinate system. The QA analysis engine 714 can compare data identifying coordinates along the periphery of the workpiece determined by the scanning station 120 with corresponding data obtained in the image data generated by the model generation engine 712. Image data matching may include superimposing images to determine a match, and if necessary, the image data may be transformed by, for example, XY translation, rotation, XY shear, XY displacement, etc. If the datasets match within a fixed threshold level, confirmation can be provided that the QA sample scanned at the QA station 106 is the same as the workpiece scanned at the scanning station 120. If a suitable match cannot be achieved, the workpiece may be skipped in the machining process and transferred to, for example, manual machining.

[0169] After selectively matching image data, the QA analysis engine 714 may analyze the QA data, including the model generated by the model generation engine 712 (and / or image data processing engine 710), as well as any weight data and / or other image data for performing a QA analysis of the QA sample. The QA analysis may include comparing the physical parameters / characteristics of the QA sample with the specifications of the QA sample. The QA sample specifications may include required values ​​for the parameters / characteristics (e.g., minimum, maximum, mean, maximum standard deviation, etc.). The QA analysis may be performed to compare any of the physical parameters / characteristics of the QA sample described herein, or other relevant physical parameters / characteristics of the QA sample, with the specifications of the QA sample.

[0170] The specifications of the QA sample may be presented as tabular reference data, as a 3D model, as a drawing, or in any other suitable format. The specifications may be stored, for example, in the QA data store 720 and / or in the data store of the processor computing device 130 or another computing device. The QA analysis engine 714 may retrieve relevant processing specification information stored in the QA data store 720 or another data store in order to run various modules configured to analyze various aspects of the QA sample.

[0171] Based on the results of the QA analysis engine 714, the machine adjustment engine 716 may be used to make or suggest adjustments to the machining equipment, to send QA samples or any machined pieces to other processes for further machining, and to provide information to technicians or managers (for example, via a tablet or monitor located near the QA station 106, through a user interface provided by the communication interface 706). For example, if a QA sample that does not meet specifications is detected, a notification and / or alarm may be generated.

[0172] In one embodiment, the QA analysis engine 714 may perform a specification module which may include comparing QA image data with specification information of a QA sample. Performing the specification module may include identifying coordinates along the perimeter of a 2D or 3D model and comparing those coordinates with the specifications of the QA sample to evaluate the shape, size, specific length / width, etc., of the QA sample. If the QA image dataset matches within a fixed threshold level, the specification module may indicate that the QA sample conforms to the product specifications. If the QA image dataset does not match within a fixed threshold level, the specification module may indicate that the QA sample is out of specification and, in some cases, how far it is from the specifications (e.g., percentage of shape mismatch, percentage of length difference, etc.).

[0173] In some cases, the specification module may include using QA weight data (e.g., acquired by the weight measurement assembly 110) to determine whether the weight of the QA sample falls within the required minimum or maximum threshold weight range. For example, to determine whether a processed product is within the weight specification range, a product that has been divided or trimmed after processing by the processing system 104 may be analyzed. If the weight of the QA sample is not within the required weight range, the specification module may indicate that the QA sample is out of specification and, in some cases, how far it is from the specification (e.g., a percentage difference in weight).

[0174] If the QA analysis engine 714 indicates that a QA sample is outside of specifications, the machine adjustment engine 716 may send instructions to the processor computing device 130 to automatically adjust the settings of the machining system 104. For example, if the QA sample has an incorrect shape, size, or weight, the processor computing device 130 may be instructed to adjust its cutting path as necessary to bring the shape, size, or weight of the remaining workpiece to be machined back within specifications. In this regard, adjustments to the machine may be made after analyzing a sample of already machined products (for example, if the QA station 106 is located downstream of the machining system 104) or after analyzing some or all of the incoming products before they are machined (for example, if the QA station 106 is located upstream of the machining system 104).

[0175] In some cases, the specification module may include using weight data (e.g., acquired by a weight measurement assembly 110) to determine the density of the product and comparing the density value with a density value calculated by a processor computing device 130 using, for example, scanning data from a scanning station 120. In this regard, mechanical density adjustment may be performed automatically by, for example, a mechanical adjustment engine 716. The mechanical density setting may also be adjusted based on the average density of QA samples analyzed by a QA analysis engine 714. In some cases, manual adjustments to the machine may be made based on the average density of QA samples provided by the QA analysis engine 714 (e.g., through a communication interface 706). In some cases, a notification may be generated indicating a mismatch between the mechanical density setting and the calculated QA sample density, which may result in appropriate action being taken (e.g., determining whether the workpiece has voids or undercuts, a large amount of fat or bone, troubleshooting mechanical sensors or other components, etc.).

[0176] In some cases, data generated by the specification module or other modules of the QA analysis engine 714 may be used to train one or more machine learning models (e.g., stored in the QA model data store 724) to provide adjustments to parameters confirmed by the processed scan data of the scan station 120 as output, based on the data analysis results of the QA analysis engine 714 as input. For example, if the QA analysis indicates that the QA sample has a different density than that determined by analyzing the scan data of the scan station 120, at least one of the parameters / settings of the scan station 120, the processing instructions generated by the processor computing device 130 based on the results of the scan station 120, or other machine settings may be automatically adjusted to account for any difference. In some examples, if the QA analysis indicates that the QA sample has a different density than that determined by analyzing the scan data of the scan station 120, the operator may be provided with a notification.

[0177] In this regard, the machine learning model may be trained using training data relating to any parameter calculation adjustments, machine adjustments, notifications, etc., that are performed or generated in response to a comparison between the data of the QA analysis engine 714 (stored in the training data store 722) and the data of the scanning station 120. Such training data may be used to train one or more machine learning models to output parameter calculation adjustments, machine adjustments, notifications, etc., based on the QA data as input.

[0178] The QA analysis engine 714 can run various other modules for performing QA analysis on QA samples. For example, in some cases, the QA analysis engine 714 may run a product output module which can be used to estimate, for example, the total volume and / or weight of a production process (production process "output"), the total volume and / or weight of a specific part of a production process (e.g., a divided piece of a specific size), etc. For example, a sensor may be used to count pieces or QA samples passing through the sensor on a conveyor belt to generate a piece count rate, which may be multiplied by the average weight of each piece (determined by the generated QA data for at least some of the pieces) to provide a production rate in pounds. In another example, the average area of ​​each piece (determined by the generated QA data for at least some of the pieces) may be combined with the belt speed and / or piece count to provide belt coverage data. The data generated by the product output module may be used to determine any relevant production output data.

[0179] In some examples, the QA analysis engine 714 may execute a texture analysis module which may include comparing texture data obtained from a model or other image data generated by the model generation engine 712 with texture specification information of a QA sample (for example, stored in the QA data store 720 of the processing system 104 or another data store). For example, the texture analysis module may include comparing texture data obtained from a color image generated by the model generation engine 712 with a color scanned image obtained by the scanning station 120 and processed by the processor computing device 130. For example, the texture analysis module may execute a feature recognition subroutine configured to match and / or compare each feature of the image.

[0180] Based on the comparison, the texture analysis module may generate output indicating whether the QA sample is within specifications (e.g., sufficient browning, sufficient charmarks, minimum percentage of blood spots / blemishes / fat or stripes), whether the QA sample tends toward a particular consistency (e.g., striping or grayish coloration may indicate the consistency of lignified chicken), or other aspects relating to texture or appearance. In the latter case, the texture analysis module may retrieve data relating to the volume, shape, size, and density of the QA sample (from the QA data store 720 and / or the data store of the processor computing device 130 containing scan data) to help determine whether the QA sample has a particular consistency. In the specific example of lignified chicken, image data relating to shape, contour, color, stripes, etc., may indicate the consistency of lignified chicken, which can then be verified to determine whether it has a density value exceeding a certain density threshold. In other examples, a separate probe to assist in measuring the consistency of the product (e.g., its elasticity) may be included in the QA station 106 either before or after being flagged by the QA analysis engine 714.

[0181] The data generated by the texture analysis module may be used by the machine adjustment engine 716 to send commands to the processor computing device 130, for example, to automatically adjust the settings of the processing system 104 as needed and to provide notifications to the operator. For example, if a QA sample has a color that indicates the product is outside of the heat treatment specifications (e.g., insufficient browning or carbonization), the processor computing device 130 may be instructed to adjust its heat processor settings to ensure proper heat treatment. In such cases, adjustments may be made to the machine after analyzing a sample of an already processed product (e.g., the QA station 106 is located downstream of the processing system 104).

[0182] In another example, if a QA sample has an out-of-spec color or appearance combined with an abnormal density or elasticity value (e.g., indicating the consistency of potentially woody chicken meat), the processor computing device 130 may be instructed to move the QA sample, processed pieces, and / or incoming pieces to other processing for different applications and / or additional processing (e.g., massage). In such cases, adjustments may be made to the machine after analyzing a sample of already processed products (e.g., the QA station 106 is located downstream of the processing system 104) and / or after analyzing some or all of the incoming products before processing (e.g., the QA station 106 is located upstream of the processing system 104).

[0183] In some cases, data generated by the texture analysis module of the QA analysis engine 714 may be used to train one or more machine learning models (stored, for example, in the QA model data store 724) to provide, as output, adjustments to machine settings, instructions for machine adjustments, notifications, etc., based on texture image data as input. For example, if the QA analysis indicates that a QA sample has a color or texture that indicates it is outside of heat treatment specifications, the model may be used to automatically adjust the processing instructions to ensure proper heat treatment. In this regard, the machine learning model may be trained using, for example, training data generated from the correlation of processing instruction adjustments made in response to different QA texture results (stored in the training data store 722). In another case, the machine learning model may be trained using, for example, training data generated from the correlation of a confirmed QA sample consistency type (e.g., woody chicken meat) with QA data such as color image data, density data, and / or elastic data. The machine learning model may be used to estimate product consistency, automatically adjust machine settings, provide instructions for machine adjustments, send notifications, etc., based on the QA data as input. For example, a machine may be instructed to send QA samples, processed pieces, and / or incoming pieces to other processes for different applications and / or additional processing (such as massage).

[0184] In some examples, the QA analysis engine 714 may train and / or run various other machine learning modules suitable for performing QA analysis on QA samples, such as one or more of the machine learning modules described in U.S. Provisional Patent Application No. 63 / 588,917 incorporated herein. In this regard, the QA computing device 111 may be configured to store and run machine learning models necessary for processing QA data.

[0185] Machine learning models typically require considerable processing power and capacity. Furthermore, as processing needs change or machine learning models improve, the ability to easily access, update, and / or upgrade a separate computing device for use with the processing system 104 and optionally one or more additional processing systems within the facility may be beneficial. In this regard, it may be beneficial to configure embodiments of the systems and methods described herein as including a QA computing device 111, which is a separate local, high-power, or edge computing device from the processor computing device 130, such as the data processing computing device described in U.S. Provisional Patent Application No. 63 / 588,917. In some examples, the QA computing device 111 is an integrated component of the QA station 106 that communicates via wired connections with components of the QA station 106 (e.g., an image sensor assembly 132, an image processor 134, and a weight measurement assembly 110) via an I / O link master or the like.

[0186] In some examples, the QA computing device 111 may run one or more machine learning models that use QA data as input and output QA analysis information to the machine adjustment engine 716 and / or processor computing device 130. This information may be used to verify or adjust the machining of workpieces of the same or similar type as the QA sample. For example, the QA analysis engine 714 of the QA computing device 111 may output information such as the location of QA sample features (e.g., bone, sciatic nerve, cross-section, outline, fat or red meat region), the outline of the QA sample and any features therein (e.g., bone, fat / red meat, foreign body, etc.), the region of interest of the QA sample (e.g., the region including the maximum nominal height of the QA sample), the classification of the QA sample (e.g., sirloin pork chop, center loin pork chop, etc.), and the location of machine components (e.g., conveyor belt components) relative to a coordinate system.

[0187] In some examples, the QA computing device 111 may run one or more machine learning models that output information to the processor computing device 130 or another computing device, including information realized by the machine learning models based on measurements in the QA data. For example, the QA computing device 111 may output information about the slack, elongation, wear, etc., of the conveyor belt based on conveyor belt measurements in the QA data.

[0188] Here, we describe exemplary machine learning models configured to be executed by the QA analysis engine 714. Some exemplary machine learning models may be substantially similar to those described in U.S. Provisional Patent Application No. 63 / 588,917, which is incorporated herein. Therefore, such exemplary machine learning models are described only briefly for the sake of brevity. Furthermore, it should be understood that the machine learning models described herein are illustrative only, and other variations and / or additional models of the described models may also be used.

[0189] For example, a classification machine learning model may be configured to classify QA samples as types of processed pieces, such as types of subprimal cuts, to determine whether the processed pieces have been properly processed, sorted, packaged, etc. The processing of subprimal cuts may vary depending on the type of subprimal cut, as described in U.S. Patent Application No. 18 / 462776, which is incorporated herein by reference in its entirety. For example, a particular type of subprimal cut may be divided or trimmed according to customer specifications or other requirements specific to that cutting type. Furthermore, a particular type of subprimal cut may be used in a particular final product, for example, depending on the supply and demand for that type of subprimal cut.

[0190] A classification machine learning model may be configured to identify a QA sample as a certain type of subprime cut, for example, to verify or adjust the value sorting and / or value optimization of processed or incoming subprime cuts, and to classify the subprime cut type into one of at least two categories. For example, a classification machine learning model may be configured to identify a subprime cut or "chop" of a whole bone-in pork loin, as shown and described in U.S. Patent Application No. 18 / 462776 incorporated herein.

[0191] A classification machine learning model for identifying / categorizing subprimal cuts (e.g., "chops" of a whole bone-in pork loin) into at least one of two categories may be configured to provide at least one classification probability score for the subprimal cut based on QA image data of the subprimal cut. Based on the information in the QA image data, the classification machine learning model may output a classification probability score (percentage) for one of several different pork chop types.

[0192] A classification machine learning model may be trained on QA image data of QA samples, each image may be labeled with one or more classification types. Such annotated QA image data of QA samples and other image data of QA samples of interest (e.g., data obtained by using the system and method disclosed in U.S. Provisional Patent Application No. 63 / 588,917 incorporated herein) may be sent to a model management computing device 113 to train the classification machine learning model. The classification machine learning model can learn to provide classification probability scores for QA samples based on features recognized in the QA images compared to the training data. Further details of the classification machine learning model are provided in U.S. Provisional Patent Application No. 63 / 588,917 incorporated herein.

[0193] In other examples, a QA sample 3D generation machine learning model may be configured to receive top and bottom image data of the QA sample from an image sensor assembly 132 as input, and then generate a 3D model of the QA sample as output. For example, an image processor 134 and / or an image data processing engine 710 may generate image data (e.g., grayscale, height, etc.) of both the top and bottom of a QA sample (e.g., a pork chop) by using a previous top image of a section of the QA sample as a mirror image of the bottom image of the target QA sample (e.g., a sliced ​​pork loin chop). In other examples, the image sensor assembly 132 may include a scanner below the imaging support surface to capture image data of the bottom of the QA sample. In yet another example, the QA sample may be inverted so that top and bottom images of the QA sample can be captured. As a non-limiting example, an operator may manually invert a QA sample at a fixed station (for example, using QA station 406 or QA station 506), and the QA sample may be inverted when being transported between conveyors located beneath the first and second scanners to scan the first and second sides. In any case, an image matching process as described above may be performed to match or correlate the image with the QA sample.

[0194] If top and bottom images of a QA sample cannot be obtained, such as the first chop sliced ​​from a pork loin, a 3D generative machine learning model can predict a 3D model based on training data collected for the first piece when training the QA sample 3D generative machine learning model, using QA image data and / or data obtained by using the systems and methods disclosed in U.S. Provisional Patent Application No. 63 / 588,917 incorporated herein.

[0195] In some examples, a QA sample 3D generation machine learning model may generate a 3D model of a QA sample as output by extrapolating features identified from images of opposing surfaces (e.g., top and bottom) of the QA sample through the body of the QA sample. For example, the 3D model output may be constructed to account for internal features, such as by assuming straight lines between features on the opposing surfaces. In other examples, the 3D model output may be generated by extrapolating density data from the top to the bottom to estimate the shape of the bottom surface, including arbitrary voids, by using a technique described in U.S. Patent No. 11570998 incorporated herein.

[0196] The 3D model output of the QA sample 3D generation machine learning model may be used by the QA analysis engine 714 to manage various aspects of QA analysis. In one example, the 3D model output may be used to provide classification probability scores for each face of the QA sample (e.g., top and bottom). In this regard, the overall or final assigned classification of the QA sample and / or workpiece may be adjusted based on the higher probability score of the two faces from the package optimization engine 718 for the workpiece and / or supply or demand information. The final assigned classification of the QA sample may be used by the QA analysis engine 714 to determine whether the processed or incoming workpiece is properly classified by the processor computing device 130. In a related example, the top or bottom of the processed workpiece may be selected for display in the package based on the confirmed classification of that side of the QA sample (e.g., the higher value classification from the package optimization engine 718, etc., may be selected for display in the package).

[0197] In another example, the 3D model output of the QA sample may be used to verify or adjust the cutting path of the workpiece. For example, the cutting path of the workpiece may be based on aligning the fat and red lines of the QA sample from top to bottom by comparing the same top and bottom attributes. The 3D model output data of the QA sample can be used in the 2D cutting module of the processor computing device 130 to make adjustments necessary for cutting the workpiece (e.g., using the cutter station 122) according to specific specifications (e.g., fat removal, bone resection, no red trim, composite fat region, etc.). The 3D model output data of the QA sample can also be used to adjust partial cutting and / or trimming of the workpiece to the desired overall shape. Adjustments may be made by the machine adjustment engine 716 by comparing the cutting path defined by the QA analysis engine 714 with the cutting path defined by the processor computing device 130.

[0198] In some cases, the 3D model output data of the QA sample can be used to adjust the angled cutting path of the workpiece using the waterjet cutter. An angled cutting path may be necessary to precisely remove features of the workpiece. For example, fat, bone, or other undesirable material may pass through the workpiece at a certain angle. Angled cutting is often required to cut off undesirable material such as fat or bone without cutting off the valuable lean meat of the workpiece. In some cases, the 3D model output data of the QA sample may contain information about the angles of internal features, and the 3D model output data can be used to adjust the cutting path of the workpiece at the cutter station 122 based on the estimated location of angled features inside the QA sample. Adjustment may be made by the machine adjustment engine 716 by comparing the cutting path defined by the QA analysis engine 714 with the cutting path defined by the processor computing device 130.

[0199] Angled cutting paths may also be required to optimize downstream machining processes of the workpiece. For example, angled edges / faces can produce workpieces with a higher surface area per unit weight, allowing for more crumbs to be applied and / or improving appearance. 3D model output of the QA sample may be used to refine the edges / faces of the target angled workpiece based on the thickness / height, internal features, external shape, classification, etc., of the QA sample.

[0200] In some cases, 3D model output data of QA samples can be used to predict voids, undercuts, or other irregularities in a machined piece. In this regard, one or more machined piece anomaly machine learning models may be used to output predicted machined piece shape, machined piece contour, or substrate absences including voids, undercuts, or other irregularities, based on the measured weight and volume (per QA image data) of one or more QA samples as input. In this regard, one or more machined piece anomaly machine learning models may be trained using QA data such as the weight and volume of QA samples that correlate with observed or measured voids, undercuts, or other irregularities of the QA samples.

[0201] The output of the machine learning model for workpiece anomalies may be used by the machine adjustment engine 716 to adjust any processing mode of workpiece processing. For example, if a QA sample is measured to have a lower weight and volume than expected based on the density settings of the processing system, one or more parameters or settings of the processing system, such as its density setting, slice thickness, or piece size, may be adjusted to account for the discrepancy.

[0202] In another example, the conveyor 3D generation machine learning model may be configured to generate 3D models of one or more components of the conveying system 116, such as a powered conveyor belt 115. As described above, images of the conveying system 116 may be used to evaluate belt slack, belt wear, or other issues or information that may affect food processing accuracy. The component 3D models may be compared to the specifications of the component by the QA analysis engine 714. The component specifications may include CAD images of the component or CAD images of the system containing the component (e.g., the entire conveying system 116), optical images of the component / system, and / or measurements of the component / system.

[0203] For example, QA analysis of mechanical components such as conveyor system components may include comparing measurements of belt components with a 3D model of the conveying system 116. If the distance between belt pickets or rods in the 3D model is measured to be greater than the specified distance, the QA analysis engine 714 may output such information to the machine adjustment engine 716, which may adjust the machining to account for elongation in the conveyor belt. In other examples, such comparisons may be used to track gaps in the belt between pickets or rods and compare the gaps with a model to determine whether / how the gaps are changing over time. In other examples, measurements of belt links in the 3D model may be compared with previous measurements or known dimensions to account for belt slack / wear.

[0204] In other examples, an image segmentation machine learning model may be configured to identify features of a QA sample, identify distinct parts of a sample, etc., by segmenting or "extracting" objects, features, etc., within the image as output, based on QA image data as input. For example, an image segmentation machine learning model may use still camera images to identify features of a QA sample. The image segmentation machine learning model may incorporate Segment Anything Model (SAM) available from Meta AI, FastSAM from Ultralytics, or another suitable image segmentation model using image segmentation techniques.

[0205] In one example, the segmented image output identifies each of several related QA samples within the QA image data. For example, the segmented image output may include information (e.g., outline in the image) for identifying each part of a segmented processed piece (e.g., each chicken nugget from a chicken breast cut). The identified related QA samples or parts of the processed piece may be correlated to the individual weight of each part. As described above, the QA weight of each of several related QA samples may be obtained by adding or removing each of the related QA samples on a weighing platform, weighing all related QA samples together to obtain a total weight, and then allocating a portion of that total weight to each individual related QA sample (e.g., using the total volume, which is determined from the individual volumes in the QA image data). The individual QA weights of each of several related QA samples may be used together with the segmented image output to perform a QA analysis on each of the several related QA samples.

[0206] As a related note, the QA analysis may include correlating or analyzing the characteristics of the processed pieces used to create the relevant QA samples using the QA analysis results of each individual relevant QA sample. For example, based on the QA analysis of chicken nuggets separated from chicken breast fillets (e.g., whether each nugget is within the specifications for shape, weight, and / or size), specific characteristics of the chicken breast fillets may also be determined. For example, if it is determined that the nuggets have a higher density than expected (e.g., determined by weight and size), the density setting of the machine for separating the chicken breasts may be adjusted, and incoming chicken breasts may be subjected to pre-separation massage, etc.

[0207] In some examples, a feature recognition image segmentation machine learning model may provide bone outlines as output based on still images from an image sensor assembly 132 as input. The output may be a binary image or map showing bone locations, where every pixel indicates the presence or absence of bone. Accurate data on bone locations in QA samples can be used to adjust trimming, cutting, etc., of workpieces to cut closer to the bones, minimizing product waste or yield loss. Furthermore, bone locations may also be used by the QA analysis engine 714 to classify QA samples / workpieces. Such bone location data may be used alone or in combination with classification probability scores, as described above.

[0208] Fat / red boundary image segmentation machine learning models may also be used to identify fat / red boundaries in QA samples. For example, a fat / red boundary image segmentation machine learning model may, based on an optical image as input, provide as output an image having the outlines of fat and / or red regions within the QA sample. The model output may be, for example, a marked-up version of the input QA image with computer-generated annotations indicating the outlines of fat and / or red regions within the QA sample.

[0209] While image segmentation models can be used without training, in some cases, the reliability and efficiency of the image segmentation machine learning model can be optimized by supplying training data to the model management computing device 113. For example, annotated QA images or other QA sample images showing feature outlines, cutting lines, etc., may be used to further train the image segmentation machine learning model.

[0210] In another example, a Region of Interest (ROI) machine learning model may be configured to generate ROIs of a QA sample as output, based on a QA image as input. The ROI may be a proposed portion or outline of a region / object in the QA sample. The ROI may be represented as a binary mask image (for example, in the mask image, pixels belonging to the ROI are set to 1 and pixels outside the ROI are set to 0) or in another format usable by the QA analysis engine 714. The model output may further include symbolic (text) labels added to the ROI, for example, to compactly describe its contents and individual points of interest (POIs) within the ROI.

[0211] ROIs in a QA image may be used to locate features within a QA sample, to specify areas within the QA sample for measurement (e.g., height measurement, temperature measurement, etc.), or for any other purpose. In some examples, the ROI output of an ROI machine learning model is used to define areas on a QA sample that are likely to define the peak thickness / height of the QA sample. For example, a chicken breast fillet does not have uniform thickness / height across its width / length. Rather, the peak thickness / height of the chicken breast is typically at the rounded end of the breast, with the thinner parts of the breast near the pointed end. If some parts of the chicken breast are thicker than others, the thicker parts will take longer to reach a safe temperature during the cooking process. Once the thicker parts reach a safe temperature, the thinner parts will dry out. Therefore, in response to the output of the QA analysis engine 714 indicating a discrepancy that causes a difference in cooking temperature, the mechanical adjustment engine 716 may make adjustments to the chicken breast processing (e.g., splitting, trimming, sorting, etc.). To manage this type of processing, accurate peak height measurement may be crucial.

[0212] The ROI representing the peak height region of the chicken breast may include the rounded edge region of the chicken breast, which is substantially horizontal (the thickest / highest region of the breast). By finding the flattest spot in the peak thickness region of the chicken breast, the ROI is likely to exclude any bulges and meat protrusions. The ROI output may also include POIs representing the precise peak height of the chicken breast. The QA analysis engine 714 may use the ROI / POI output of the ROI machine learning model to determine whether or not adjustments should be made to the processing of the chicken breast.

[0213] In some examples, the ROI output of an ROI machine learning model can define a region of chicken breast to measure the height and / or slope of the tail ridge of the chicken breast or butterfly to check for lignified chicken. As is known in the industry, lignified chicken, or chicken with an unpleasant texture (e.g., hard to touch, tough, more complex consistency, coarse fibrous texture, etc.), can often be recognized by a prominent tail ridge. If an ROI is identified in a QA image of the chicken associated with the relevant tail ridge region, the relevant height / slope of the tail ridge can be determined for grading / evaluating the chicken. For example, a piece of chicken that does not show lignification may be used for premium sandwich portions, while a slightly lignified piece may be used for lower-value slice portions, and a more extremely lignified piece may be repurposed for products often made from trimmings, such as pet food or marinades.

[0214] As described above, the ROI output in the QA image may be used to locate features within the QA sample. In some examples, an ROI machine learning model may be used to locate an ROI in a steak piece that is likely to contain the sciatic nerve. The sciatic nerve, which can be seen in filet mignon or other steak cuts, is typically located within the fat layer of the steak. Moreover, the sciatic nerve is often located within the center of the largest portion of a particular fat region in the steak. In this regard, in some examples, the ROI output of a steak piece may be defined by the largest inscribed circle that can be superimposed on the fat region of the QA image.

[0215] A POI within an ROI may be substantially at the center of the ROI and identifies a possible location of the sciatic nerve. The ROI / POI output may be sent to the QA analysis engine 714 for steak QA analysis. For example, the QA analysis may include verification of sciatic nerve removal, verification of sciatic nerve retention in divided steak pieces, etc. If, based on the QA analysis, processing a steak with a sciatic nerve is deemed outside the specifications of the steak as determined by the QA analysis engine 714, adjustments may be made to the steak processing (e.g., through the machine adjustment engine 716).

[0216] The ROI machine learning model may be trained using QA image data and other QA sample image data that identify regions of interest using annotations, labels, etc. The ROI machine learning model learns to identify ROIs based on features recognized in the images and the locations of ROIs / POIs relative to those features, compared to the training data. For example, if a human operator is measuring chicken breast at a QA station, the operator may indicate the peak height location in the QA image using a touchscreen, etc. In the case of the sciatic nerve, the image of the steak piece may be annotated to include the largest inscribed circle in a specific layer of fat containing the nerve. Such annotated QA image data may be sent to a model management computing device 113 to train the ROI machine learning model.

[0217] Other machine learning models may be run by the QA analysis engine 714 using the QA data of the QA sample as input to provide information such as the carcass side of the QA sample, the skin side of the QA sample, the left and / or right side of the cut, the tenderloin cut location, the rib location, etc.

[0218] Any suitable type of machine learning model, including convolutional neural networks, may be used, though not limited to them. Any suitable technique, including one or more of gradient descent, data augmentation, hyperparameter tuning, and locking / unlocking model architecture layers, may be used to train the machine learning model, though not limited to them. In some examples, annotated raw images, and possibly weight data, are used as training input. In some examples, instead of, or in addition to, annotated raw images, one or more features derived from the image may be used to train the machine learning model, including, but not limited to, a version of the image in a transformed color space, a set of edges detected in the image, one or more statistical calculations about the overall content of the image, or other features derived from the image.

[0219] The QA analysis engine 714 is configured to generate machine learning model output data by running one or more of the machine learning models described herein or other suitable models. The QA analysis engine 714 may perform any necessary post-processing of the output used by the machine tuning engine 716, the processor computing device 130, and / or other computing devices.

[0220] For example, the QA analysis engine 714 may include one or more formatting modules configured to perform, for example, any of the preprocessing steps described above, or any other steps necessary for using the output when managing the processing of the QA sample (e.g., matching the format of the output data with the original QA data, formatting the output data for compatibility with one or more modules of the machine adjustment engine 716). In one example, the formatting module may be configured to convert the pixel locations related to the aspect of the output image to the coordinate system of the machine adjustment engine 716 and / or the processor computing device 130. Postprocessing may also include digitizing or reducing the data for efficient data transfer between the QA computing device 111 and another computing device.

[0221] The QA analysis engine 714 may also include one or more modules configured to select one or more outputs from a plurality of outputs generated by a machine learning model. For example, if the machine learning model outputs three possible classification labels for a QA sample (e.g., subprimal cut types such as pork chops) each with varying probabilities, the QA analysis engine 714 may categorize the QA sample as a specific type based on information sent from the package optimization engine 718.

[0222] For example, if the package optimization engine 718 transmits information to the QA analysis engine 714 indicating that the supply of QA samples, such as pork chops, is likely to contain more of a particular type, then the QA samples, which could be classified as one of several types (by product specifications), may be classified as the chop type with a lower supply quantity. Alternatively, or in addition to the above, if the package optimization engine 718 transmits information to the QA analysis engine 714 indicating that there is high demand for a particular processed piece, such as sirloin pork chops, then the QA samples, which could be classified as one of several types (by product specifications), may be classified as the chop type with higher demand. This can maximize production operating profits.

[0223] The QA analysis engine 714 may also include one or more modules configured to extract information from machine learning model output data or other QA data for transmission to the machine adjustment engine 716 and / or processor computing device 130. For example, the QA analysis engine 714 may receive segmented images of QA samples as output of a machine learning model, and the QA analysis engine 714 may extract various parameters from the segmented images (e.g., position, size, aspect ratio, shape, etc.). For example, as described above, the segmented image output may identify each of several related QA samples in the QA image data (e.g., chicken breast nuggets), and the identified related QA samples may be correlated to individual weights to perform a QA analysis on each of the related QA samples.

[0224] Please understand that, in addition to QA images, QA weight data, and other QA analysis data, machine learning model outputs and data extracted from machine learning model outputs may also be considered "QA data" as used herein.

[0225] The QA analysis engine 714 transmits (optionally post-processed) QA data to the machine adjustment engine 716 and / or the workpiece processing engine 312 of the processor computing device 130, and / or may store any QA data in the QA data store 720 for retrieval by the machine adjustment engine 716. The machine adjustment engine 716 and / or the workpiece processing engine 312 of the processor computing device 130 use the information in the post-processed output data to determine the next steps to adjust the machining of the QA sample, if any.

[0226] Herein, an exemplary embodiment of the machine adjustment engine 716 is described. As described above, the QA analysis engine 714 may transmit the post-processed output data to the machine adjustment engine 716 and / or the workpiece processing engine 312 of the processor computing device 130. It should be understood that the machine adjustment engine 716 may be incorporated into the workpiece processing engine 312 of the processor computing device 130, and therefore, when describing embodiments of the machine adjustment engine 716, it should be understood that any function may be performed instead by the workpiece processing engine 312 of the processor computing device 130.

[0227] The machine adjustment engine 716 may be configured to execute one or more machine adjustment modules to process QA data and determine what adjustments need to be made to any machining steps or components. Generally, the machine adjustment modules may be configured to provide the workpiece machining system with information about recommended or required adjustments based on the QA analysis of the QA analysis engine 714. The information may include commands for display or retrieval by the operator of the machining system, commands to automatically or semi-automatically change settings within the machining system 104 for machining the workpiece in response to execution by the controller of the machining system (e.g., the processor 302 of the processor computing device 130), or other information related to adjusting or verifying the configuration of the system or process used to machine the workpiece.

[0228] In some examples, the machine adjustment engine 716 may run a density adjustment module configured to automatically or semi-automatically adjust the density setting on the processing system 104 if the measured density of the QA sample is outside the specifications. For example, the density value on the machine may be adjusted to process the workpiece based on the actual density of the QA sample rather than based on the estimated density of the incoming workpiece (e.g., changing the density setting from 1.0 to 1.2). Adjusting the density setting on the processing system 104 may also automatically adjust the system's processing settings, such as how the cutter station 122 cuts, slices, or trims the workpiece to achieve a specific part size, thickness, etc.

[0229] In other cases, the machine's processing settings may be adjusted to take density differences into account. For example, the cutting command of a waterjet cutter may be adjusted to adjust the size or shape of the workpieces divided based on density differences. In another example, a slicer may be adjusted to take different density values ​​of workpieces (for example, if a higher density is measured by the QA scanning system 108, smaller slices may be created to achieve slices within the weight specification).

[0230] In some examples, the machine adjustment engine 716 may run a heat treatment adjustment module configured to automatically or semi-automatically adjust the heat treatment settings on the heat treatment system if the measured size, height, etc., of the QA sample is outside of specifications. For example, the machine temperature setting (and / or other heat treatment settings such as humidity setting and / or belt speed) may be adjusted to ensure that workpieces that are larger or smaller than expected are properly cooked, frozen, etc.

[0231] In some examples, the machine adjustment engine 716 may run a preventive maintenance module configured to determine the repair / replacement of appropriate machine parts, such as conveyor belt components, if the measured QA samples are out of specification. For example, if QA analysis by the QA analysis engine 714 shows that machine parts are out of specification by less than a certain percentage, the machine adjustment engine 716 may indicate that repair is needed. If QA analysis by the QA analysis engine 714 shows that machine parts are out of specification by more than a certain percentage, the machine adjustment engine 716 may indicate that replacement is needed, and an automated order and / or notification to a technician may be issued.

[0232] In some examples, the machine adjustment engine 716 may run a simulation module configured to simulate the machining of a workpiece after adjustments, repairs, etc., have been made to the machining system. For example, the machine adjustment engine 716 may output a simulation command to the processor or controller of the machining system (e.g., the processor 302 of the processor computing device 130), and the machining system may run the simulation and display the simulation results, for example, on the display of the computing device.

[0233] In some examples, the machine adjustment engine 716 may be configured to output one of the following: information regarding adjustments to the machining system; commands to automatically or semi-automatically change settings within the machining system in response to actions by the machining system's controller; or other relevant information to adjust or verify the configuration of the system or process used to machine a workpiece based on QA data as input; for example, by running one or more machine learning models stored in the QA model data store 724.

[0234] For example, one or more of the density adjustment module, heat treatment adjustment module, and preventive maintenance module may run one or more machine learning models that provide information or instructions for adjusting density settings or related settings, heat treatment settings, or for repairing or replacing machine components as output based on QA data as input. Such machine learning models may be trained using QA analysis data from the QA analysis engine 714 and information about machine adjustments made to the machining system based on the QA analysis data.

[0235] The machine adjustment engine 716 may, instead or additionally, run any suitable module and / or machine learning model to adjust the machining system to conform the workpiece to the desired specifications in response to the analysis of the QA analysis engine 714.

[0236] Herein, the data normalization engine 717 of the QA computing device 111 is described. The data normalization engine 717 may be performed to correlate or normalize workpiece data across the platform of the workpiece processing management system 102, computing devices, etc. Data normalization may include correlating QA data with processing system data (e.g., X-ray data, optical scanning data, temperature data, etc.) so that the processing system of the workpiece processing management system 102, such as the processing system 104, can adjust machine settings and / or take corrective actions based solely on the processing system data. In other words, using only X-ray images, workpiece mass data may be generated based on previous correlations of mass data to workpieces having specific attributes in the X-ray images.

[0237] In this regard, one or more normalized data machine learning models may be used to normalize the data by providing QA data as output based on processing system data as input. For example, the machine learning model may process an X-ray scan from the processing system 104 as input, and based on the attributes of the X-ray scan (e.g., an analog signal associated with the scan, the outer circumference of the workpiece determined from the scan, etc.), the machine learning model may output the mass of the workpiece. The machine learning model may be trained using the correlation between the QA data and the processing system data.

[0238] Data normalization allows QA data to be correlated with processing system data for faster and more optimal processing of workpieces. For example, adjustments to the processing system may be made based solely on processing system data, without the need to acquire QA data. In this regard, the main purpose of the QA station 106 may be used to train one or more machine learning models, including a normalized data machine learning model.

[0239] Herein, the package optimization engine 718 of the QA computing device 111 is described. The package optimization engine 718 may be run to analyze models or other image data generated by the model generation engine 712 in order to determine the most optimal way to use the QA samples / processed pieces. For example, if a segmented piece is within a certain degree of specification, as determined by the QA analysis engine 714, such a segmented piece may be considered a "higher value" piece that can be instructed by the package optimization engine 718 to be packaged together to maximize the package value. In other cases, a piece that is within a certain degree of specification may be assigned to a package for a particular customer with stricter requirements (more stringent specifications), while other pieces may be assigned to other packages.

[0240] The package optimization engine 718 may perform global optimization to assign each piece or a specific quantity of pieces to a package configuration based on the QA analysis results from the QA analysis engine 714. The package configuration assigned to each QA sample / processed piece, piece, or a certain quantity of pieces may be based on information regarding the supply of raw incoming processed pieces, the requirements for finished QA processed pieces, or other information from other sources. The finished QA sample data may identify at least one of the monetary value and demand for each packaging configuration. One or more machine learning models may be trained to identify package configurations as outputs based on a QA analysis of QA samples (e.g., percentages within a specification) as input. In this regard, performing global optimization may include using one or more machine learning models to identify package configurations for QA samples / processed pieces.

[0241] Referring back to Figure 1, the QA sample processing management system 102 further includes a monitoring system 112, which may be embodied in any computing device that network communicates with the processing system 104 and / or the QA station 106. The monitoring system 112 includes a processor (not shown) and a computer-readable medium storing logic that causes the monitoring system 112 to provide a QA data processing engine 142 and a QA analysis reporting engine 144 in response to execution by the processor.

[0242] The QA data processing engine 142 is generally configured to receive and analyze QA analysis data (from the QA analysis engine 714, the machine adjustment engine 716, and / or the package optimization engine 718, etc.). Analysis of QA analysis data may include compiling information on a specific machine or production process (e.g., the percentage of out-of-spec samples, the degree of out-of-spec, heat treatment data, etc.), comparing QA data across production processes, and comparing QA data with machine scan data. The QA analysis reporting engine 144 may be configured to report the QA analysis data by organizing the QA analysis data and displaying the data to a viewer. In this regard, the monitoring system 112 is generally configured to monitor the QA process and provide insights into the QA process for purposes such as optimizing the QA process, identifying any gaps in the QA process, and troubleshooting machine equipment or settings based on the QA process results.

[0243] Figure 8 is a flowchart illustrating a non-limiting example of method 800 for performing a quality assurance (QA) analysis of a QA sample, which may be performed by one or more engines of the QA computing device 111 or any other computing device. Method 800 may also be performed on a workpiece that is being processed or will be processed by a processing system such as the processing system 104.

[0244] In block 802, method 800 may include acquiring image sensor data (e.g., one or more images) of a QA sample using an image sensor assembly. In some examples, the image sensor assembly is configured to generate at least one still color camera image and / or 3D point cloud data. For example, an image of a QA sample may be acquired using the above-described image sensor assembly 132, which may include at least one still camera and a structural optical scanner system. Acquiring image sensor data may include moving at least one image sensor of the image sensor assembly relative to the QA sample to acquire an image of the QA sample at an oblique angle. Image sensor data may be acquired by the image sensor assembly when the QA sample is stationary or when the QA sample is moving through the image sensor assembly (e.g., using a transport system).

[0245] In block 804, method 800 may include generating at least one of a 2D model and a 3D model of the QA sample using a computing device. For example, the image processor 134 may send still camera image data and / or 3D point cloud data generated from image sensor data to the image data processing engine 710 of the QA computing device 111. The image data processing engine 710 may use the 3D point cloud data to generate a height map in order to create at least one of a 2D model and a 3D model of the QA sample. The image data processing engine 710 may run an image data optimization module to select 3D point cloud data from conflicting and / or competing sensor data originating from different image sensors and / or different sensor sources of the image sensor assembly. In some examples, one or more machine learning models may be used to identify 3D point cloud data from conflicting / competing sensor data as output, based on at least one of the accuracy and efficiency with which they generate models using the selected data as input.

[0246] In block 806, method 800 may include performing a QA analysis of a QA sample by using a computing device to compare data points of at least one of the 2D and 3D models of the QA sample with data points of the corresponding 2D and 3D models from the specifications of the QA sample. For example, the QA analysis engine 714 may execute a specification module which may include comparing model data or other image data generated by the model generation engine 712 with the manufacturing specification information of the QA sample. For example, executing a specification module may include identifying coordinates along the perimeter of the model and comparing those coordinates with the specifications of the QA sample to evaluate the shape, size, specific length / width, etc. of the QA sample. If the dataset matches within a fixed threshold level, the specification module may indicate that the QA sample conforms to the product specifications. If the dataset does not match within a fixed threshold level, the specification module may indicate that the QA sample is out of specification and, in some cases, how far it is from the specifications (e.g., percentage of shape mismatch, percentage of length difference, etc.).

[0247] In some examples, Method 800 may further include acquiring color sensor data of a QA sample using an image sensor assembly and performing a QA analysis of the QA sample by comparing the color sensor data of the QA sample acquired using the image sensor assembly with color data values ​​from at least one of the specifications of the QA sample and the scan of the QA sample using a computing device (such as the QA analysis engine 714 of the QA computing device 111).

[0248] In some examples, Method 800 may further include obtaining a weight measurement of the QA sample using a gravimetric assembly such as a gravimetric assembly 110, and performing a QA analysis of the QA sample by using a computing device to compare at least one of the measured weight and the calculated density of the QA sample based on the measured weight with at least one of the weight and density values ​​determined from at least one of the specifications of the QA sample and scanning of similar type workpieces.

[0249] In some examples, Method 800 may further include acquiring color sensor data of a QA sample using an image sensor assembly and using one or more machine learning models to identify the consistency of the QA sample as output, using one or more of the color sensor data as input and the calculated density of the QA sample based on the measured weight of the QA sample.

[0250] In some examples, Method 800 may further include positioning a QA sample on a vertically displaceable surface, capturing one or more images showing the vertical displacement of the vertically displaceable surface caused by the weight of the QA sample using an image sensor assembly (such as image sensor assembly 132), and processing the image data showing the vertical displacement of the vertically displaceable surface to obtain a weight measurement of the QA sample.

[0251] In some examples, method 800 may further include running one or more machine learning models to output a QA analysis using QA data as input. For example, one of the exemplary machine learning models described herein as configured to be run by the QA analysis engine 714 may be used.

[0252] In some examples, method 800 may further include using a computing device (such as the machine adjustment engine 716 of the QA computing device 111) to adjust the settings of a machine (such as the machining system 104) configured to machine a QA sample / workpiece based on a QA analysis of the QA sample.

[0253] In some examples, Method 800 may further include using a computing device (such as the package optimization engine 718 of the QA computing device 111) to perform a global optimization for assigning QA samples / processed pieces to package configurations based on a QA analysis of the QA samples.

[0254] While the exemplary Method 800 described above illustrates a specific operation, the order and / or combination of operations may be modified without departing from the scope of this disclosure. For example, some of the operations described may be performed in parallel or in a different order that does not substantially affect the functionality of Method 800. Furthermore, in some examples, some of the operations described may be omitted. In other examples, Method 800 may be carried out using different components of the exemplary device or system. Method 800 may be performed using any of the embodiments disclosed herein.

[0255] Figure 9 is a block diagram illustrating an exemplary computing device 900 suitable for use as a computing device of the present disclosure. Although several different types of computing devices have been described above, the exemplary computing device 900 illustrates various elements common to many different types of computing devices. Although Figure 9 is described with reference to a computing device implemented as a device on a network, the following description is applicable to servers, personal computers, mobile phones, smartphones, tablet computers, embedded computing devices, and other devices that may be used to implement some of the examples of the present disclosure. Some examples of computing devices may be implemented in or include application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other customized devices. Furthermore, those skilled in the art will recognize that computing device 900 may be any number of currently available or yet-to-be-developed devices.

[0256] In its most basic configuration, the computing device 900 includes at least one processor 902 and system memory 910 connected by a communication bus 908. Depending on the exact configuration and type of the device, the system memory 910 may be volatile or non-volatile memory such as read-only memory ("ROM"), random access memory ("RAM"), EEPROM, flash memory, or similar memory technology. Those skilled in the art will recognize that the system memory 910 typically stores data and / or program modules that are immediately accessible and / or currently being manipulated by the processor 902. In this regard, the processor 902 may serve as the computing center of the computing device 900 by supporting the execution of instructions.

[0257] As further illustrated in Figure 9, the computing device 900 may include a network interface 906 having one or more components for communicating with other devices over a network. An example of the present disclosure may provide access to basic services that utilize the network interface 906 to perform communication using common network protocols. The network interface 906 may also include a wireless network interface configured to communicate over one or more wireless communication protocols such as Wi-Fi, 2G, 3G, LTE, WiMAX, Bluetooth®, Bluetooth Low Energy®, etc. As will be understood by those skilled in the art, the network interface 906 illustrated in Figure 9 may represent one or more wireless or physical communication interfaces described and illustrated above with respect to a particular component of the computing device 900.

[0258] In the example shown in Figure 9, the computing device 900 also includes a storage medium 904. However, the service may be accessed using a computing device that does not include means for persisting data to a local storage medium. Therefore, the storage medium 904 shown in Figure 9 is represented by a dashed line to indicate that the storage medium 904 is optional. In any case, the storage medium 904 may be volatile or non-volatile, removable or non-removable, and may be implemented using any technology capable of storing information, such as a hard drive, solid-state drive, CD-ROM, DVD, or other disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device, etc.

[0259] Suitable embodiments of a computing device including a processor 902, system memory 910, communication bus 908, storage medium 904, and network interface 906 are known and commercially available. For the sake of clarity and because they are not essential to understanding the claimed subject matter, Figure 9 does not show some of the typical components of many computing devices. In that regard, the computing device 900 may include input devices such as a keyboard, keypad, mouse, microphone, touch input device, touchscreen, tablet, etc. Such input devices may be coupled to the computing device 900 by wired or wireless connections including RF, infrared, serial, parallel, Bluetooth, Bluetooth Low Energy, USB, or other suitable connection protocols using wireless or physical connections. Similarly, the computing device 900 may also include output devices such as a display, speaker, printer, etc. These devices are well known in the art and are not further illustrated or described herein.

[0260] The concept of the principle of this disclosure is subject to various modifications and alternative forms, specific embodiments of the concept are shown in the drawings as examples and are described in detail herein. However, it should be understood that there is no intention to limit the concept of this disclosure to any particular form disclosed, but rather to cover all modifications, equivalents, and alternatives that are consistent with this disclosure and the appended claims.

[0261] References in this specification such as “one example” or “an example” indicate that the described example may include a particular feature, structure, or characteristic, but not all examples necessarily include that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same example. Moreover, when a particular feature, structure, or characteristic is described in relation to an example, it is considered within the knowledge of those skilled in the art to apply that feature, structure, or characteristic in relation to other examples, whether explicitly stated or not. In addition, it should be understood that items in a list of the form “at least one of A, B, and C” may mean (A), (B), (C), (A and B), (B and C), (A and C), or (A, B, and C). Similarly, items in a list of the form “at least one of A, B, or C” may mean (A), (B), (C), (A and B), (B and C), (A and C), or (A, B, and C).

[0262] Words such as “top,” “bottom,” “left,” “right,” “first,” and “second” in this disclosure are intended to provide the reader with an orientation by referring to the drawings and are not intended to indicate a required orientation of a component or graphic image, or to limit the scope of the claims.

[0263] In drawings, certain structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order is not required. Rather, in some examples, such features may be arranged in a different way and / or order than those shown in the illustrative drawings. Furthermore, the inclusion of a structural or method feature in a particular drawing is not intended to mean that such feature is required in all examples; in some examples, it may not be included, or may be combined with other features.

[0264] This application may include modifiers such as “generally,” “approximately,” “about,” or “substantially.” These terms serve to indicate, for example, that the “dimensions,” “shape,” “temperature,” “time,” or other physical parameters in question do not need to be precise, but can vary as long as the function to be performed is still viable.

[0265] Where used herein, terms such as “about” and “approximately” are used to include numbers that are within 10%, 5%, or 1% in either direction (greater or less) unless otherwise specified or otherwise evident from the context (except when such a number is greater than 100% of a possible value).

[0266] Where an electronic or software component is described as being configured to perform a certain operation, such a configuration can be achieved, for example, by designing electronic circuits or other hardware to perform the operation, by programming programmable electronic circuits (e.g., a microprocessor or other suitable electronic circuits) to perform the operation, or by any combination thereof.

[0267] The phrase "connected to" refers to any component that is physically connected, directly or indirectly, to another component, and / or communicates directly or indirectly with another component (e.g., connected to another component via a wired or wireless connection, and / or other appropriate communication interface).

[0268] The section headings and title of this patent application provided herein are for convenience only and should not be construed as limiting the disclosure.

[0269] Preferred embodiments of the present invention are shown and described herein, but it will be apparent to those skilled in the art that such embodiments are provided only as examples. Many variations, modifications, and substitutions will be conceivable to those skilled in the art without departing from the present invention. Various alternatives to the embodiments of the present invention described herein may be employed in carrying out the present invention. The following claims define the scope of the present invention and are intended to cover methods and structures within the scope of these claims, as well as their equivalents.

[0270] List of inventions Item 1. A computer-operated method for performing a quality assurance (QA) analysis on a QA sample, which is at least one of a quality assurance (QA) sample processed by a processing system and a QA sample to be processed by a processing system, wherein the processing system has a controller configured to manage the manner in which the processing system processes a workpiece in response to an analysis of the workpiece, separate from the QA analysis, and the method comprises: acquiring QA image sensor data of the QA sample using an image sensor assembly of the QA system; generating at least one of a 2D model and a 3D model of the QA sample using the QA image sensor data using a computing device; and performing a QA analysis of the QA sample by comparing the QA data of at least one of the 2D model and 3D model of the QA sample with the QA specifications of the QA sample using a computing device.

[0271] Item 2. The computer-based method described in Item 1, wherein the image sensor assembly includes at least one of a stereo camera, a structural optical scanner system, and a still camera.

[0272] Item 3. A method performed by the computer described in Item 1, further comprising moving at least one image sensor of the image sensor assembly relative to the QA sample in order to capture an image of the QA sample at an oblique angle.

[0273] Section 4. The QA specification for the QA sample is the method of execution of the computer as described in Section 1, which is reference data, 3D models, drawings, or any combination thereof.

[0274] 5. The method performed by the computer described in 1, further comprising using a computing device to run an image data optimization module to select QA image sensor data from at least one of conflicting and competing sensor data originating from at least one of different image sensors and different sensor data sources of an image sensor assembly.

[0275] 6. A method performed by a computer as described in 5, further comprising using a computing device to select QA image sensor data according to at least one of the following: the resolution of the image sensor data, the processing power required to use the image sensor data, the compatibility of the image sensor data with the modules of the computing device, and the accuracy and / or efficiency of generating a model using the selected data as input.

[0276] 7. The method performed by a computer as described in 1, further comprising using one or more machine learning models to select QA image sensor data from at least one of conflicting and competing sensor data as output, based on at least one of accuracy and efficiency of generating a model with selected image sensor data as input.

[0277] Item 8. The method performed by the computer described in Item 1, further comprising transporting a QA sample under an image sensor assembly using a transport system.

[0278] The computer-based method described in Section 8, further comprising: transporting a QA sample on a weighing cell to obtain a QA weight measurement of the QA sample; and performing a QA analysis of the QA sample using a computing device, by comparing at least one of the QA weight measurement and the calculated density of the QA sample based on the QA weight measurement with at least one of the weight and density values ​​determined from at least one of the QA specification of the QA sample and scanning of the QA sample by a processing system.

[0279] Item 10. A computer-based method according to Item 1, further comprising obtaining a QA weight measurement of a QA sample using a gravimetric assembly, and performing a QA analysis of a QA sample by using a computing device to compare at least one of the QA weight measurement and the calculated density of the QA sample based on the QA weight measurement with at least one of the weight and density values ​​determined from at least one of the QA specification of the QA sample and scanning of the QA sample by a processing system.

[0280] Item 11. If the QA sample has a calculated density different from the QA specifications of the QA sample and the scanning of the QA sample by the processing system, the method performed by the computer in Item 10 further includes using a computing device to adjust at least one of the following: density setting on the processing system, cutting setting on the processing system, sorting setting on the processing system, packaging setting on the processing system, temperature setting on the processing system, and cooking time on the processing system.

[0281] Item 12. If the QA sample or workpiece has a weight or volume exceeding a threshold weight or volume, the computer method described in Item 10 further includes obtaining the temperature of the QA sample or workpiece during or after the heat treatment.

[0282] The method executed by a computer according to item 10, further comprising turning at least one of the processed piece and the QA sample from heat treatment to another treatment when the QA sample has a weight or volume exceeding the threshold weight or volume.

[0283] The method executed by a computer according to item 10, further comprising capturing color sensor data of the QA sample using an image sensor assembly and using one or more of the calculated densities of the QA sample based on the color sensor data as input and the QA weight measurement of the QA sample to identify the consistency of the QA sample as output by using one or more machine learning models.

[0284] The method executed by a computer according to item 1, further comprising capturing color sensor data of the QA sample using an image sensor assembly and performing QA analysis of the QA sample by comparing the color sensor data of the QA sample captured using the image sensor assembly with color data values from at least one of the QA specifications of the QA sample and the scanning of the QA sample by the processing system using a computing device.

[0285] The method executed by a computer according to item 1, further comprising positioning the QA sample on a vertically displaceable surface, capturing one or more images showing the vertical displacement of the vertically displaceable surface caused by the weight of the QA sample using an image sensor assembly, and processing the image data showing the vertical displacement of the vertically displaceable surface to obtain the QA weight measurement of the QA sample.

[0286] Item 17. Further including scanning a QA sample using a scanning station of a processing system to generate scanning data, generating at least one of a 2D model and a 3D model of the QA sample based on the scanning data, and comparing data points of a first data set including at least one of the 2D model and the 3D model generated from the QA image sensor data of the QA sample with data points of a second data set including corresponding 2D and 3D models generated from the scanning data of the QA sample to perform a QA analysis of the QA sample, the method executed by a computer according to Item 1.

[0287] Item 18. The method executed by a computer according to Item 17, further including performing a model matching process to determine whether the QA sample used to create the first data set is the same as the QA sample used to create the second data set.

[0288] Item 19. The method executed by a computer according to Item 18, wherein the model matching process includes performing a conversion of the first data set to the second data set, and performing the conversion includes one or more of a direction conversion of the QA sample, a rotation conversion of the QA sample, a scaling of the size of the QA sample, and a shear distortion of the QA sample.

[0289] Item 20. The method executed by a computer according to Item 1, further including using a computing device to adjust the settings of the processing system based on the QA analysis of the QA sample.

[0290] Item 21. The method executed by a computer according to Item 1, further including using a computing device to perform global optimization to assign a processed piece processed by the processing system to a package configuration based on the QA analysis of the QA sample.

[0291] Item 22. A method performed by the computer described in Item 1, further comprising using a computing device to output at least one machining system setting adjustment as a list of possible adjustments based on the QA analysis of a QA sample.

[0292] Item 23. The method performed by a computer as described in Item 1, further comprising using a computing device to run one or more machine learning models to output a QA analysis of a QA sample, using at least one of a 2D model and a 3D model of the QA sample as input.

[0293] The computer-operated method described in Section 23, wherein one or more machine learning models are configured to perform at least one of the following after receiving at least one of a 2D model and a 3D model of the QA sample as input: generate a 3D model of the QA sample; generate classification probability scores for at least one possible type of processed piece for the QA sample; generate a region of interest in an image of the QA sample; and generate an outline of at least one object or feature of the processed piece in an image of the QA sample.

[0294] Section 25. The method according to Section 24, wherein generating a 3D model of a QA sample as output is in response to receiving images of a first surface and a second opposing surface of the QA sample as input.

[0295] Item 26. The method according to Item 25, wherein the image of the first surface of the QA sample is an image of the top surface of the QA sample, and the image of the second surface of the QA sample is an image of the top surface of the previously cut QA sample.

[0296] The method according to item 26, wherein generating a 3D model of a QA sample comprises at least one of extrapolating identified features from images of the top and bottom surfaces of the QA sample through the thickness of the QA sample, and extrapolating density data from the top surface of the QA sample to the bottom surface of the QA sample to estimate the shape of the bottom surface including any voids.

[0297] Section 28. The top and bottom images of the QA sample are height maps, as described in Section 26 or 27.

[0298] Item 29. The method according to item 26, 27, or 28, further comprising defining a cutting path for a workpiece based on features identified in a 3D model of a QA sample for the processing system.

[0299] 30. The method according to 30.5, for generating classification probability scores for at least one possible type of processed piece of a QA sample, based on at least one of the top and bottom images of the QA sample.

[0300] The method according to paragraph 30, wherein a computing device receives and processes the output of classification probability scores for at least one possible type of processed piece of a QA sample, comprising categorizing the QA sample based on at least one of a first classification probability score and a second classification probability score of the QA sample, using the top and bottom surfaces of the QA sample, respectively, and using the demand for a first type of processed piece corresponding to the first classification probability score and a second type of processed piece corresponding to the second classification probability score.

[0301] The method according to paragraph 32. Generating classification probability scores for at least one possible type of processed piece of a QA sample, comprising providing a label for at least one possible type of processed piece of a QA sample if the classification probability score exceeds a minimum threshold; providing a list of first and second possible types of processed pieces of a QA sample based on a first and second highest classification probability score; and providing a list of all possible types of processed pieces of a QA sample and the corresponding classification probability scores for each type.

[0302] 33. The method according to 32, wherein a computing device receives and processes the output of classification probability scores for at least one possible type of processed piece of a QA sample, and categorizes the QA sample based on at least one of the classification probability scores and the demand for at least one possible type of processed piece.

[0303] The method according to item 33, further comprising using a processing system to perform at least one of cutting, splitting, trimming, sorting, and packaging of a processed piece, based on the categorized type of the QA sample.

[0304] The method of paragraph 35, wherein generating a region of interest in an image of a QA sample comprises at least one of superimposing the largest inscribed circle onto an image of a QA sample within a steak fat region that is likely to contain the sciatic nerve, and superimposing the outline onto an image of a QA sample that defines a likely peak height portion of the QA sample.

[0305] The method according to item 36. The method according to item 24, wherein the QA sample is a piece of chicken, and generating a region of interest in an image of the piece of chicken includes at least one of superimposing the outline onto an image of the piece of chicken surrounding a substantially flat peak height portion of chicken breast, and superimposing the outline onto an image of the piece of chicken surrounding a portion of the caudal ridge of the piece of chicken in order to measure the height and / or slope of the caudal ridge relevant to the evaluation of lignified chicken.

[0306] The method according to paragraph 37, wherein generating an outline in an image of the QA sample of at least one object or feature of the QA sample comprises drawing the outline of at least one of each of the multiple pieces of the QA sample, bone, fat / red boundary, edge of the QA sample, periphery of the QA sample, underside of the QA sample, and cutting line of the QA sample.

[0307] Item 38. The method executed by a computer according to item 1, further comprising adjusting at least one setting on a processing system using a computing device when a QA sample has at least one of physical parameters, characteristics, and attributes different from the corresponding physical parameters, characteristics, and attributes of the specifications of the QA sample.

[0308] Item 39. The method executed by a computer according to item 38, further comprising executing one or more machine learning models to output a machine adjustment instruction for adjusting processing settings for processing a workpiece using QA analysis as an input using a computing device.

[0309] Item 40. The method executed by a computer according to item 39, further comprising normalizing QA data and workpiece processing system data using a computing device.

[0310] Item 41. The method executed by a computer according to item 40, further comprising executing one or more machine learning models to output QA data as an output based on workpiece processing system data as an input using a computing device.

[0311] Item 42. The method executed by a computer according to item 1, further comprising defining an imaging support surface plane substantially parallel to the imaging support surface on which a QA sample is placed during imaging, and the imaging support surface plane can be used as a reference from which all height measurement values of the QA sample can be determined.

[0312] The computer-based method described in paragraph 43, further comprising obtaining a QA weight measurement of the QA sample using a bench scale having a platform defining an imaging support surface, and performing a QA analysis of the QA sample using a computing device by comparing at least one of the QA weight measurement and the calculated density of the QA sample based on the QA weight measurement with at least one of the weight and density values ​​determined from at least one of the specifications of the QA sample and scanning of the QA sample by a processing system.

[0313] 44. A computer-based method as described in 43, further comprising: using a weight measurement assembly to obtain QA weight measurements for each of a plurality of related QA samples; using an image sensor assembly to acquire QA image sensor data of the plurality of related QA samples; using a computing device to identify each of the plurality of related QA samples in the QA image sensor data; using a computing device to correlate the QA weight measurements of each of the plurality of related QA samples with each of the identified plurality of related QA samples in the QA image sensor data; and using a computing device to perform a QA analysis on each of the plurality of related QA samples by comparing at least one of the QA weight measurements and QA image sensor data with the specifications of the QA sample.

[0314] 45. The computer-based method described in 44, further comprising using a computing device to perform a QA analysis on each of a plurality of related QA samples by comparing at least one of a QA weight measurement and the calculated density of each of a plurality of related QA samples based on the QA weight measurement with at least one of a weight value and a density value determined from at least one of the QA specifications for each of the plurality of related QA samples and each scan of the plurality of related QA samples by a processing system.

[0315] Section 46. The method performed by a computer as described in Section 44, which involves using a computing device to perform a QA analysis on each of several related QA samples, and including generating at least one of a 2D model and a 3D model for each of the several related QA samples.

[0316] Item 47. The method performed by a computer as described in Item 44, wherein each of several related QA samples is part of a workpiece.

[0317] Item 48. A quality assurance (QA) system for performing a QA analysis of a QA sample which is processed by a processing system and at least one of the QA samples which will be processed by the processing system, wherein the processing system has a controller configured to manage the manner in which the processing system processes a workpiece in response to an analysis of a workpiece separate from the QA analysis, and comprises an image sensor assembly of the QA system configured to acquire QA image sensor data of the QA sample, a processor, and a memory storing instructions, wherein when the instructions are executed by the processor, the instructions cause the computing device of the QA system to generate at least one of a 2D model and a 3D model of the QA sample, and to perform a QA analysis of the QA sample by comparing the QA data of at least one of the 2D model and 3D model of the QA sample with the QA specifications of the QA sample.

[0318] 49. The QA system according to 48, further comprising an image processor configured to generate at least one of 3D point cloud data and color data from captured QA image sensor data of a QA sample.

[0319] 50. The image sensor assembly of the QA system includes at least one of a stereo camera, a structural optical scanner system, and a still camera, as described in section 48 or 49.

[0320] The QA system according to item 51. The QA system further comprising a gravimetric assembly configured to obtain a weight measurement of a QA sample by measuring the vertical displacement of the imaging support surface on which the QA sample is placed.

[0321] Item 52. The QA system as described in Item 48, further comprising a weighing deck of the QA system configured to obtain weight measurements of QA samples and to transport the QA samples past an image sensor assembly.

[0322] 53. The QA system according to 48, wherein the processing system has a scanning station configured to acquire one or more scans of a workpiece to generate at least one of a 2D model and a 3D model of the workpiece, and the memory of the computing device of the QA system further stores instructions that, when executed by the processor, cause the computing device of the QA system to perform a QA analysis of the QA sample by comparing QA data of at least one of the 2D model and 3D model generated from QA image sensor data of the QA sample with the corresponding 2D model and 3D model generated from scan data of the workpiece.

[0323] Section 54. The QA specification for the QA sample is the QA system described in Section 48, which is reference data, a 3D model, a drawing, or any combination thereof.

[0324] Item 55. The QA system according to item 48, further comprising a transport system for transporting QA samples under an image sensor assembly.

[0325] The QA system described in Section 48, further comprising a weighing assembly of the QA system configured to take QA weight measurements of QA samples.

[0326] Item 57. The weighing assembly is a bench scale having a platform, and the imaging support surface of the image sensor assembly is defined by the upper surface of the platform of the weighing assembly, as described in Item 56 of the QA system.

[0327] Section 58. The weight measurement assembly is defined by a vertically displaceable surface and an image sensor assembly, and one or more images of the QA sample may be captured using the image sensor assembly showing the vertical displacement of the vertically displaceable surface caused by the weight of the QA sample, and as a result, the QA weight measurement of the QA sample may be obtained using the vertical displacement of the vertically displaceable surface, as described in Section 56.

[0328] Item 59. The QA system described in Item 48, further comprising a temperature measurement assembly of the QA system configured to take temperature measurements of a QA sample.

[0329] Item 60. A method performed by a computer for performing a quality assurance (QA) analysis of a QA sample which is processed by a processing system and at least one of the QA samples which will be processed by the processing system, wherein the processing system has a controller configured to manage the manner in which the processing system processes the workpiece in response to an analysis of the workpiece separate from the QA analysis, and the method is to acquire QA image sensor data of the QA sample using an image sensor assembly of the QA system, and to generate at least one of a 2D model and a 3D model of the QA sample using the QA image sensor data using a computing device, and QA system A computer-based method comprising: obtaining QA weight measurements of a QA sample using a stem weight measurement assembly; and performing a QA analysis of the QA sample using a computing device, the method comprising: comparing QA image data of at least one of a 2D model and a 3D model of the QA sample with the QA specifications of the QA sample; and comparing at least one of the QA weight measurements and the calculated density of the QA sample based on the QA weight measurements with at least one of the weight and density values ​​determined from at least one of the QA specifications of the QA sample and scanning of the QA sample by a processing system.

[0330] Item 61. A method performed by a computer for conducting a quality assurance (QA) analysis of a QA sample, comprising: acquiring image sensor data of the QA sample using an image sensor assembly; generating at least one of a 2D model and a 3D model of the QA sample using the image sensor data using a computing device; and performing a QA analysis of the QA sample by comparing the QA data of at least one of the 2D model and 3D model of the QA sample with the specifications of the QA sample using a computing device.

[0331] Item 62. A quality assurance (QA) system for processing a QA sample, comprising an image sensor assembly configured to acquire image sensor data of the QA sample, a processor, and a memory storing instructions, wherein, when executed by the processor, the instructions cause the computing device of the QA system to generate at least one of a 2D model and a 3D model of the QA sample, and to perform a QA analysis of the QA sample by comparing the QA data of at least one of the 2D model and 3D model of the QA sample with the specifications of the QA sample.

[0332] An example of the present invention in which exclusive property or privileges are claimed is defined as follows:

Claims

1. A computer method for performing a quality assurance (QA) analysis on a QA sample which is at least one of a quality assurance (QA) sample processed by a processing system and a QA sample which will be processed by a processing system, wherein the processing system has a controller configured to manage the manner in which the processing system processes the processed piece in response to an analysis of the processed piece separate from the QA analysis, and the method is: The QA system's image sensor assembly is used to acquire QA image sensor data from a QA sample, and Using a computing device, generate at least one of a 2D model and a 3D model of the QA sample using the QA image sensor data, The QA analysis of the QA sample is performed by comparing the QA data of at least one of the 2D model and 3D model of the QA sample with the QA specifications of the QA sample using a computing device. The way a computer performs actions, including [specific actions].

2. The computer-based method according to claim 1, wherein the image sensor assembly includes at least one of a stereo camera, a structural optical scanner system, and a still camera.

3. The method performed by a computer according to claim 1, further comprising moving at least one image sensor of the image sensor assembly relative to the QA sample in order to capture an image of the QA sample at an oblique angle.

4. The method performed by a computer according to claim 1, further comprising using a computing device to run an image data optimization module to select QA image sensor data from at least one of conflicting and competing sensor data originating from at least one of different image sensors and different sensor data sources of the image sensor assembly.

5. Obtaining the QA weight measurement value of the QA sample using a weight measurement assembly, The QA analysis of the QA sample is performed by using a computing device to compare at least one of the QA weight measurement and the calculated density of the QA sample based on the QA weight measurement with at least one of the weight and density values ​​determined from at least one of the QA specifications of the QA sample and scanning of the QA sample by the processing system. A method performed by a computer according to claim 1, further comprising:

6. If the QA sample has a calculated density different from the QA specifications of the QA sample and the scanning of the QA sample by the processing system, the computer method according to claim 5 further includes using a computing device to adjust at least one of the density setting on the processing system, the cutting setting on the processing system, the sorting setting on the processing system, the packaging setting on the processing system, the temperature setting on the processing system, and the cooking time on the processing system.

7. The image sensor assembly is used to acquire color sensor data from the QA sample, To identify the consistency of the QA sample as output, one or more machine learning models are used, using one or more of the color sensor data and the calculated density of the QA sample based on the QA weight measurement of the QA sample as inputs. A method performed by a computer according to claim 5, further comprising:

8. The image sensor assembly is used to acquire color sensor data from the QA sample, The QA analysis of the QA sample is performed by using a computing device to compare the color sensor data of the QA sample acquired using the image sensor assembly with the color data values ​​from at least one of the QA specifications of the QA sample and the scanning of the QA sample by the processing system. A method performed by a computer according to claim 1, further comprising:

9. The aforementioned QA sample is positioned on a surface that can be displaced in the vertical direction, Using the image sensor assembly, one or more images are taken showing the vertical displacement of the vertically displaceable surface caused by the weight of the QA sample. To obtain the QA weight measurement of the QA sample, image data showing the vertical displacement of the vertically displaceable surface is processed. A method performed by a computer according to claim 1, further comprising:

10. The computer-based method according to claim 1, further comprising using a computing device to run one or more machine learning models to output a QA analysis of the QA sample, using at least one of the 2D model and 3D model of the QA sample as input.

11. The one or more machine learning models, after receiving at least one of the 2D model and 3D model of the QA sample as input, To generate a 3D model of the aforementioned QA sample, To generate classification probability scores for at least one possible type of processed piece for the aforementioned QA sample, To generate a region of interest within the image of the aforementioned QA sample, and To generate the outline of at least one object or feature of the processed piece within the image of the QA sample. A computer method according to claim 10, configured to perform at least one of the following.

12. The method performed by a computer according to claim 1, further comprising adjusting at least one setting on the processing system using a computing device if, based on the QA analysis, the QA sample has at least one physical parameter, characteristic, and attribute that is different from the corresponding physical parameter, characteristic, and attribute of the specification of the QA sample.

13. The computer-based method according to claim 12, further comprising using a computing device to run one or more machine learning models to output machine adjustment commands for adjusting machining settings for machining a workpiece using the QA analysis as input.

14. The definition of an imaging support plane substantially parallel to the imaging support surface on which the QA sample is placed during imaging, using a computing device, wherein the imaging support plane can be used as a reference from which all height measurements of the QA sample can be determined, The QA weight measurement of the QA sample is obtained using a weight measurement assembly defined by a bench scale having a platform that defines the imaging support surface, The QA analysis of the QA sample is performed by using a computing device to compare at least one of the QA weight measurement and the calculated density of the QA sample based on the QA weight measurement with at least one of the weight value and density value determined from at least one of the specifications of the QA sample and the scanning of the QA sample by the processing system. A method performed by the computer according to claim 1, including the following:

15. Using the aforementioned weight measurement assembly, QA weight measurements are obtained for each of several related QA samples. Using an image sensor assembly, QA image sensor data of the multiple related QA samples is acquired, Using a computing device, identify each of the multiple related QA samples in the QA image sensor data, Using a computing device, the QA weight measurement of each of the plurality of related QA samples is correlated with each of the identified plurality of related QA samples in the QA image sensor data, The QA analysis of each of the multiple related QA samples is performed by comparing at least one of the QA weight measurement values ​​and QA image sensor data with the specifications of the QA sample using a computing device. A method performed by a computer according to claim 14, further comprising:

16. The computer-operated method according to claim 15, further comprising performing a QA analysis on each of the plurality of related QA samples by using a computing device to compare at least one of the QA weight measurement and the calculated density of each of the plurality of related QA samples based on the QA weight measurement with at least one of the weight and density values ​​determined from at least one of the QA specifications of each of the plurality of related QA samples and the scanning of each of the plurality of related QA samples by the processing system.

17. The method performed by a computer according to claim 15, wherein performing a QA analysis on each of the plurality of related QA samples using a computing device includes generating at least one of the 2D model and the 3D model of each of the plurality of related QA samples.

18. A quality assurance (QA) system for performing QA analysis on a QA sample which is processed by a processing system, and at least one of the QA samples which will be processed by the processing system, wherein the processing system has a controller configured to manage the manner in which the processing system processes the processed piece in response to an analysis of the processed piece separate from the QA analysis, A QA system image sensor assembly configured to acquire QA image sensor data from a QA sample, Processor and The system includes a memory that stores instructions, and when an instruction is executed by the processor, it is transmitted to the computing device of the QA system. To generate at least one of the 2D model and 3D model of the aforementioned QA sample, The QA analysis of the QA sample is performed by comparing the QA data of at least one of the 2D model and 3D model of the QA sample with the QA specifications of the QA sample. A QA system that enables this process.

19. The QA system according to claim 18, further comprising a weight measurement assembly of the QA system configured to take QA weight measurements of the QA sample.

20. A method performed by a computer for performing a quality assurance (QA) analysis of a QA sample which is processed by a processing system and at least one of the QA samples which will be processed by the processing system, wherein the processing system has a controller configured to manage the manner in which the processing system processes the processed piece in response to an analysis of the processed piece separate from the QA analysis, and the method is The QA system's image sensor assembly is used to acquire QA image sensor data from a QA sample, and Using a computing device, generate at least one of a 2D model and a 3D model of the QA sample using the QA image sensor data, The QA weight measurement of the QA sample is obtained using the weight measurement assembly of the QA system, Performing a QA analysis on the QA sample using a computing device, The QA image data of at least one of the 2D model and 3D model of the QA sample is compared with the QA specifications of the QA sample. The QA weight measurement and the calculated density of the QA sample based on the QA weight measurement are compared with at least one of the weight and density values ​​determined from at least one of the QA specifications of the QA sample and the scanning of the QA sample by the processing system. This includes performing a QA analysis on the aforementioned QA sample. The way a computer performs actions, including [specific actions].

Images