Color transformation matrix and image capture

Abstract

Methods, apparatuses, and systems related to image capture and transformation with a color transformation matrix. A method includes receiving a first pin having a first reference color at a junction of at least two mirrors and capturing a first image that includes at least one view of the first pin reflected in one or more of the at least two mirrors. The method includes receiving a second pin having a second reference color at the junction of the at least two mirrors and capturing a second image that includes at least one view of the second pin reflected in one or more of the at least two mirrors. The method includes generating a color transformation matrix that is based at least in part on the first image with the first reference color and the second image with the second reference color. Disclosed apparatuses and systems may perform such method.

Description

COLOR TRANSFORMATION MATRIX AND IMAGE CAPTURECROSS REFERENCES

[0001] The present application for Patent claims priority to U.S. Provisional Patent Application No. 63 / 761,190 by Persson, entitled “COLOR TRANSFORMATION MATRIX AND IMAGE CAPTURE,” filed February 21 , 2025, which is assigned to the assignee hereof and which is expressly incorporated in its entirety herein.TECHNICAL FIELD

[0002] The present disclosure relates to a method, apparatus, and system for generating a color transformation matrix and for transforming images based on the color transformation matrix.BACKGROUND

[0003] Image analysis often relies on known reference information, including reference colors, to facilitate computer vision efforts. Creating information for reference colors, including a table or matrix representation of reference colors to facilitate image analysis may rely on data collected remotely from where it is used. Further, multiple similar systems may be calibrated in a manner such that different machines interpret image data differently. That is, some machines may be configured differently and may thus draw different inferences from common training data.SUMMARY

[0004] Accordingly, there is a need for systems, apparatuses, and methods for generating a color transformation matrix and for transforming images based on the color transformation matrix, which mitigate, alleviate or address the shortcomings existing and provide color transformation matrix generation that is consistent across multiple machines, use common reference data to support similar results for similar inputs, and which allows for color calibration without altering other aspects of the system.

[0005] A method is disclosed. The method may include receiving, such as at a junction of at least two mirrors, a first pin, such as a first pin having a first reference color. The at least two mirrors may be configured with respect to a camera to capture images, such asto capture images that include a plurality of views of an object positioned at the junction. The method may include capturing, such as with the camera, a first image, such as a first image that includes at least one view of the first pin reflected in one or more of the at least two mirrors. The method may include receiving a second pin, such as a second pin having a second reference color, at the junction of the at least two mirrors. The method may include capturing with the camera a second image, such as a second image that includes at least one view of the second pin, reflected in one or more of the at least two mirrors. The method may include generating a color transformation matrix, such as a color transformation matrix that is based on, such as based at least in part on, the first image with the first reference color and / or the second image with the second reference color.

[0006] Further, an apparatus, which may be referred to as a device, is disclosed. The apparatus may include a camera, which may be configured to capture images, such as images that comprise at least one view reflected in a set of mirrors. The set of mirrors may be configured to reflect an object, such as an object traveling in a path of the set of mirrors. The apparatus may include a receiver for a pin having a reference color; the receiver may be configured to position the pin in the path of the set of mirrors. The apparatus may include a processor coupled with memory, which may be configured to generate a color transformation matrix, such as a color transformation matrix that is based on, such as based at least in part on, one or more images, such as images captured of respective reference colors of pins received by the receiver.

[0007] Further, a system is disclosed. The system may include means for receiving pins, such as pins each having a respective reference color. The means for receiving may be configured to position the pins to reflect in a plurality of mirrors, which may be configured with respect to a camera to capture images that include one or more views corresponding to the plurality of mirrors. The system may include means for capturing with the camera an image, such as an image that includes at least one view of a received pin reflected in one or more of the mirrors. The system may include means for generating a color transformation matrix, such as a color transformation matrix that is based on, such as based at least in part on, the first image with the first reference color and / or the second image with the second reference color.

[0008] It is an advantage of the present disclosure that a color transformation matrix, which may be used to evaluate and / or analyze an image, may be created from known or controlled reference colors.

[0009] Further, it is an advantage of the present disclosure that a color transformation matrix may be created using a same imaging system, including a set of mirrors, camera, optics (e.g., a prism) and lighting that is used to capture subsequent images to be analyzed.

[0010] It is a further advantage of the present disclosure that a color transformation matrix may be generated at a device that will use the color transformation matrix.

[0011] Further, it is an advantage of the present disclosure that multiple, separate devices may be individually calibrated by generating a respective color transformation matrix locally using common reference colors.

[0012] It is a further advantage of the present disclosure that a color transformation matrix may be generated from separate individual reference colors introduced sequentially, which may avoid a need for calibration or color transformation matrix generation from a set or array of colors.

[0013] Further, it is an advantage of the present disclosure that pins supplying or including the reference colors may be received and analyzed in an imaging path without altering the imaging path, and thus without disrupting the optics or other configuration of the system.

[0014] It is a further advantage of the present disclosure that images may be captured and transformed or analyzed based on a color transformation matrix generated from a discrete number of reference colors.BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features and advantages of the present disclosure will become readily apparent to those skilled in the art by the following detailed description of example embodiments thereof with reference to the attached drawings, which are briefly described in the following.

[0016] Fig. 1 is a diagram illustrating an example system configured to generate a color transformation matrix and to capture and transform an image(s) according to this disclosure.

[0017] Figs. 2A-2C are diagrams illustrating an example set of mirrors according to this disclosure.

[0018] Figs. 3A-3B are a flow-chart illustrating an example method for generating a color transformation matrix and / or transforming an image with the color transformation matrix, according to this disclosure.

[0019] Fig. 4 is a block diagram illustrating an example apparatus (or device) according to this disclosure.

[0020] Figs. 5A-5B are diagrams illustrating an example color correction map according to this disclosure.DETAILED DESCRIPTION

[0021] Instruments that rely on or employ image analysis or computer vision techniques may be sensitive to changes or differences in perceived color of objects. Instruments of the same type and design may, for example, vary in results because of differences, such as internal differences, between cameras, lighting, mirrors, image background colors, and / or environmental conditions. Such differences may result in different colors of captured images between devices, which may affect analysis for objects appearing in the images.

[0022] Furthermore, calibration efforts may depend on a particular configuration of such instruments. Some example image analysis tools described herein, for instance, may have imaging paths that present challenges for calibration.

[0023] Accordingly, the disclosure herein provides for generation of a color transformation matrix that may employ calibration pins for color calibration of an image analysis system, which may provide for more reliable calibration of image analysis devices or instruments than other solutions. As disclosed, pins with standardized colors may be measured (e.g., images may be captured) on the same place in each instrument to allow for consistentand reliable results from instrument to instrument. The resulting values of measured colors (e.g., captured images of reference colors) may be used to calculate a color transformation matrix. The matrix is then used to transform images taken from that specific instrument.

[0024] The figures are schematic and simplified for clarity and intend to show details that aid understanding the disclosure, while other details may have been left out. Throughout, the same reference numerals are used for identical or corresponding parts.

[0025] Fig. 1 is a diagram illustrating an example system 100 which includes image analysis device 102, which may also be referred to as apparatus 102, configured to generate a color transformation matrix and to capture and transform one or more image(s) 118 according to this disclosure. Device 102 (e.g., apparatus 102) may be configured to analyze granular objects 114 deposited in device 102 via hopper 112, which may be a bowl or receptacle configured to hold and transport granular objects 114 to a path such that camera 104 may capture an image of such granular objects 114. Granular objects 114 may be or may include any of a wide variety of natural and / or artificial material.

[0026] Granular objects 114 may be kernels of grain, such as wheat, barley, oats, rye, rice, corn, maize, sorghum, millet, quinoa, buckwheat, or amaranth, or any combination thereof. Granular objects 114 may be vegetable seeds, such as cucumber, lettuce, carrot, pumpkin, pepper, eggplant, or cruciferous vegetables (e.g., broccoli, cabbage, kale, cauliflower, sprouts, or the like), or any combination thereof. Granular objects 114 may be fruit seeds, such as apple, pear, citrus (e.g., oranges, lemons, limes, or the like), tomatoes, avocados, stonefruit (e.g., peaches, apricots, plums, cherries, or the like), melons (e.g., watermelon, cantaloupe, honeydew, or the like), grape, pomegranates, passion fruit, or papayas, or any combination thereof. Granular objects 114 may be oilseeds, such as rapeseed (canola), cotton seed, sunflower seed, sesame see, linseed (flaxseed), or chia seed, or any combination thereof. Granular objects 114 may be legumes, such as soybean, chickpea, lentils, pinto beans, black beans, or peas, or any combination thereof. Granular objects 114 may be weed seeds, such as couch grass, cleavers, chickweed, or foxtail, or any combination thereof. Granular objects 114 may be or include other seeds from plants such as tobacco, hemp, and / or trees, such as pine seeds.

[0027] Granular objects 114 may, additionally or alternatively, be or include nonfoodstuffs, such as hardware (e.g., bolts, screws, nuts, nails, tacks, pins, staples, rivets, or the like), notions (e.g., buttons, beads, rivets, tags, needles, pins, or the like), and / or sundries (e.g., paperclips, staples, tacks, or the like).

[0028] In one or more examples, granular objects 114 may be or include other natural or non-natural materials, such as stones, dirt, insect shells, mollusk shells (e.g., snail shells), animal waste, non-seed plant matter, glass (e.g., glass shards), and / or heavy metals (e.g., lead, including lead shot).

[0029] In one or more examples, granular objects 114 may include combinations of natural and artificial material, such as kernels mixed with non-foodstuffs or grains mixed with seeds. Indeed, an advantage of the present disclosure is that such diverse materials, which may be seen as contaminants or impurities in some examples, may be identified and documented.

[0030] By way of example, device 102 may be a configured to capture one or more images per granular object 114 (e.g., one or more images of a kernel of grain) and to use advanced image analysis (e.g., machine-learning based image processing and / or computer vision) to detect details of such granular object that may be undetectable by a human. Device 102 may thus perform a thorough visual inspection, which may include identify impurities, foreign or different substances, and / or various defects. Device 102 may contemporaneously or simultaneously detect, identify, and / or document other characteristics, including, for example, physical measurements, color, size, and / or weight (e.g., thousand kernel weight (TKW)) for each of the deposited granular objects 114. The device 102 may achieve high accuracy in measurement analysis through its configuration, including mirror design, that may capture more than 90% of each kernel with only one image per kernel, combined with its effective calibration, including color calibration using a color transformation matrix.

[0031] Device 102 may include a camera 104 or other image capture device that includes, for example, an image sensor and image processor. An image sensor may be seen as a semiconductor sensor array, such as CMOS or CCD, configured to convert incident light into electrical signals through a photoelectric effect. Such signals may then be processed through an analog-to-digital converter and image signal processor, whichmay be integral to camera 104, to create a digital file containing luminance and chrominance data for each pixel in an image, such as image(s) 118. Image 118 may be seen as a digital image which may comprise a two-dimensional matrix of discrete pixel values, where each pixel contains numerical data representing color and brightness information, which may be encoded in RGB color space with 8 or more bits per channel. Image 118 may be a file (e.g., a set of data) that can be compressed using various algorithms like JPEG or stored uncompressed in formats such as RAW or TIFF.

[0032] Device 102 may include a set of mirrors 202, which may include at least two separate mirrors or mirrored surfaces connected, coupled, or joined at a junction 204. Junction 204 may be seen as a joint between mirrors and / or a place or line where such mirrors are joined. Junction 204 may be seen a region or characteristic of the set of mirrors 202. The camera 104 may be configured with respect to the set of mirrors 202 to capture images that include one or more views, such as a plurality of views, of an object, such as a granular object 114a, which may pass by the mirrors (e.g., over the junction 204). Device 102 may include a prism 108 between camera 104 and the set of mirrors 202, which may be allow for or support image capture by camera 104 of objects passing over or in front of the set of mirrors 202. Camera 104 may thus capture images 118, which may include a first image, a second image, and / or one or more additional images, and each of such images may include different objects. An image 118 may include several views of an object, which may include a direct view (e.g., an unobstructed view of an object) as well as one or more views of an object reflected in the set of mirrors 202.

[0033] Device 102 may include a user interface 110. User interface 110 may be seen as a point of interaction between a human and system 100 (or an aspect of system 100, such as device 102). User interface 110 may comprise both visual elements, such as buttons, menus, icons, and windows, and interaction methods, such as mouse clicks, keyboard inputs, touch gestures, and voice commands, that allow a user or users to control and interact with digital content or functionality of system 100. User interface 110 may comprise a graphical user interface (GUI), which may include a touch screen configured to display information to a user and / or to receive inputs from a user. The user interface 110 may display, for example, one or more prompts 116, such as a first prompt, a second prompt, and / or one or more additional prompts. A prompt 116 may be seen as a windowor other visual element with verbal (such as written) and / or audible data displayed or communicated via user interface 110.

[0034] In one or more examples, prompt(s) 116 may include instructions to a user for calibrating device 102. Such instructions may be instructions for generating a color transformation matrix, which may be used in the analysis of images of granular objects 114. For example, user interface 110 may display first prompt 116 that instructs a user to insert a pin 206 in device 102; pin 206 may be inserted (e.g., positioned in or on) the set of mirrors 202, such as in or on, or otherwise positioned at, the junction 204. Pin 206 may be, for example, a steel rod coupled with or affixed to a handle. Pin 206 may have a variety of shapes, such cylindrical, flat, or rectangular. A portion of pin 206, such as the steel rod, may be painted or otherwise colored such that it has a known or objectively defined reference color. Pin 206 may be one of several pins 206 that each include a different reference color. Pin 206 may be a first pin having a first reference color, pin 206a may be a second pin having a second reference color, and pins 206b may be additional pins each having different reference colors. For example, the pins 206 may include a red pin, a green pin, a blue pin, a black pin, a white pine, and / or a beige pin.

[0035] The specific colors of the various pins may be different (e.g., some examples may include a different quantity of colors). But the colors of a set of pins may be defined to enable calibration. For example, certain calibration instructions for device 102 may be based on a particular set of reference colors, and device 102 may display prompts (e.g., instructions) accordingly. In other examples, device 102 may be configured with different calibration instructions, and the set of reference colors associated with such calibration may be tailored accordingly.

[0036] The device 102 may include a computer system 400, which may include processing circuitry, such as processor 402, and memory circuitry, such as memory 401. Computer system 400 may be referred to as device 400, electronic device 400, or apparatus 400 in various examples. Computer system 400 may include an interface or several interfaces to enable communication with and / or control of various aspects of device 102.

[0037] Figs. 2A-2C are diagrams 200a, 200b, and 200c (collectively diagrams 200) illustrating an example set of mirrors 202 according to this disclosure. The set of mirrors202 may be examples of the set of mirrors 202 described with reference to Fig. 1. Set of mirrors 202 may include a junction 204, which may be an example of the junction 204 described with reference to Fig. 1. The set of mirrors 202 may be configured to receive a pin 206, which may be an example of a pin 206 described with reference to Fig. 1. Pin 206 may include a rod or shaft 208, which may be made of (e.g., may comprise) a ferrous material, such as steel or other material that comprises iron. In some examples, pin 206, such as shaft 208, includes other material that may be attracted to a magnet (e.g., magnetic material), which may be non-ferrous. Pin 206 may include handle 210, which may be made of the same material as the shaft 206, or another material, such as plastic, rubber, composite, or the like.

[0038] The set of mirrors 202 may include several (e.g., two or more) mirrors. The mirrors may be seen as individual sheets of flat glass with a metallic coating on one side. Such metallic coating may be aluminum, for example, though it could be silver or other metal. The metallic backing may be protected by a layer of paint or other protective coating. The glass of the mirrors may be float glass. Although other materials for the glass and / or metallic portions may be used and may support specular reflection. The set of mirrors 202 may thus reflect an object, such as pin 206, positioned in front or above the mirrors 202. For example, a pin 206 positioned at the junction 204 may create one or more reflection(s) 214, such as a reflection in each mirror. Accordingly, a single image of the pin 206 may be captured, such as by camera 104 as described with reference to Fig. 1 , and such image may include at least three views of the pin 206 (e.g., a direct view of the pin 206 and two views of reflections 214).

[0039] The set of mirrors 202 may include a magnet 220, which may be configured to receive and / or hold metallic objects positioned at the junction of the set of mirrors 202. In one or more examples, the set of mirrors 202 comprises a body 222 or frame, which may be made of plastic, metal, or the like, configured to hold and support the mirrors. The set of mirrors 202 may also include the magnet 220. In various examples, the magnet 220 may have different sizes. In one or more examples, magnet 220 may occupy a relatively small area under the set of mirrors 202; for instance, magnet 220 may extend under a portion the mirrors 220. In one or more examples, magnet 220 may extend along the length of the set of mirrors 202 in parallel to the junction 204. The set of mirrors 204 may thus be configured to receive a pin or pins 206 made of steel (or other ferrous material)because such steel pins may be held in place at the junction 204 by magnet 220. Junction 204 may thus be referred to as a receiver, such as a receiver configured to receive a pin or pins 206. Magnet 220 may be a permanent magnet, a temporary magnet, or an electromagnet. Magnet 220 may comprise magnetic material or magnet may be another material that may be magnetized or may have magnetic properties due to, for example, an application of electric current. In one or more examples, magnet 220 may be or may comprise steel, iron, cobalt, ceramic, ferrite, or the like.

[0040] Once a pin 206 is positioned upon (e.g., received by) the set of mirrors 202, camera 104 may capture an image 118 of the pin 206. This process may be repeated for several pins 206 (e.g., following instructions by prompts 116). Because each pin 206 may comprise a different color, which may be used a reference, a matrix of colors, which may be referred to as a color transformation matrix may be generated. For example, computer system 400 may include instructions stored in memory 401 and executable by processor 402 to generate a color transformation matrix based on images 118. Device 102 may thus be configured to generate a color transformation matrix based on images 118 captured from pins 206 of various reference colors.

[0041] Accordingly, as shown in and described with reference to Figs. 1 and 2A-2C, system 100 may be seen as a system for generating a color transformation matrix and for capturing and transforming one or more images based on the color transformation matrix.

[0042] In one or more examples, system 100 comprises means for receiving pins each having a respective reference color. The means for receiving may be configured to position each pin to reflect in a plurality of mirrors configured with respect to a camera to capture images that include one or more views corresponding to the plurality of mirrors.

[0043] In one or more examples, system 100 comprises means for capturing with the camera an image. Such image may include at least one view of a received pin reflected in one or more of the mirrors.

[0044] In one or more examples, system 100 comprises means for generating a color transformation matrix. Such color transformation matrix may be based on, such as based at least in part on, the first image with the first reference color and / or the second image with the second reference color.

[0045] In one or more examples, system 100 comprises means for displaying a prompt via a user interface. The means for receiving may be configured based on, such as based at least in part on, the prompt.

[0046] In one or more examples, system 100 comprises means for determining a sequence of reference colors for the color transformation matrix generation. The means for displaying may be configured to display prompts in an order based on, such as based at least in part on, the sequence.

[0047] In one or more examples, system 100 comprises means for transforming an image of a granular unit based on, such as based at least in part on, the color transformation matrix.

[0048] Further, as shown in and described with reference to Figs. 1 and 2A-2C, device 102 may be seen as an apparatus configured for generating a color transformation matrix and for capturing and transforming one or more images based on the color transformation matrix.

[0049] In one or more examples, apparatus 102 comprises a camera configured to capture images that comprise at least one view reflected in a set of mirrors. The set of mirrors may be configured to reflect an object traveling in a path of the set of mirrors.

[0050] In one or more examples, apparatus 102 comprises a receiver for a pin having a reference color. The receiver may be configured to position the pin in the path of the set of mirrors. The reference color may comprise at least one of red, green, blue, black, white, or beige.

[0051] In one or more examples, apparatus 102 comprises a processor coupled with memory and configured to generate a color transformation matrix that is based on, such as based at least in part on, images captured of respective reference colors of pins received by the receiver.

[0052] In one or more examples of the apparatus 102, the receiver is configured to position the pin with respect to an intersection of two mirrors of the set of mirrors.

[0053] In one or more examples of the apparatus 102, the receiver comprises a magnet configured to hold the pin at the junction of the set of mirrors.

[0054] In one or more examples, the apparatus 102 comprises a prism between the set of mirrors and the camera. The camera may be configured to capture the image via the prism and / or the receiver is configured to position the pin with respect to the prism.

[0055] In one or more examples of the apparatus 102, the pin comprises a handle portion and a color portion. The receiver may be configured to receive the color portion.

[0056] In one or more examples of the apparatus 102, the pin comprises a ferrous material. For example, the color portion may comprise a ferrous material, such as steel.

[0057] In one or more examples of the apparatus 102, the processor coupled with memory is configured to determine a sequence of reference colors for the color transformation matrix. Such determination may be, for example, based on instructions (e.g., software or firmware) received via an interface and communicated to the processor and / or stored in memory.

[0058] In one or more examples, the apparatus 102 comprises a user interface. The processor coupled with memory may be configured to cause the user interface to display prompts for pins of different reference colors based on, such as based at least in part on, the sequence of reference colors for the color transformation matrix.

[0059] In one or more examples of the apparatus 102, the processor coupled with memory is configured to transform an image of the object based on, such as based at least in part on, the color transformation matrix. Transforming an image may be seen as changing data associated with an image based on data from the color transformation matrix. Additionally or alternatively, transforming an image may be seen as capturing an image (e.g., writing data for the image) based on or with reference to data from the color transformation matrix.

[0060] Figs. 3A-3B is a flow-chart illustrating an example method 300 for generating a color transformation matrix and / or transformation an image with the color transformation matrix, according to this disclosure. Method 300 may be performed, for example, by system 100 and / or device 102, as described with reference to Fig. 1.

[0061] In one or more examples, the method 300 comprises determining S302 a sequence of reference colors for a color transformation matrix generation. The determining may be performed by processor 402 of Fig. 1.

[0062] In one or more examples, the method 300 comprises displaying S304 a first prompt via a user interface. The first prompt may be displayed based on, such as based at least in part on, the sequence of reference colors for the color transformation matrix. The displaying may be performed by user interface 110 of Fig. 1.

[0063] The method 300 comprises receiving S306, such as at a junction of at least two mirrors, a first pin, such as a first pin having a first reference color. The first pin may be received by junction 204 of Figs. 1 and 2A-2C. The first pin may be received based on, such as based at least in part on, the first prompt. In one or more examples, the at least two mirrors are configured with respect to a camera to capture images that include a plurality of views of an object positioned at the junction. In one or more examples, the junction may be seen as comprising a magnet; for example, a magnet may be located within or in the proximity of the junction. As described elsewhere herein, the mirrors may be held by a body or other apparatus, and the magnet may be in the body below the junction. The junction may thus receive a pin because of the magnet field associated with the magnet.

[0064] The method 300 comprises capturing S308 with the camera a first image that includes at least one view of the first pin reflected in one or more of the at least two mirrors. The capturing may be performed by camera 104.

[0065] In one or more examples, the method 300 comprises displaying S310 a second prompt via the user interface, the second prompt may be displayed based on, such as based at least in part on, the sequence of reference colors for the color transformation matrix. The second prompt may be displayed based on, such as based at least in part on, capturing the first image. Such displaying may be performed by user interface 110 of Fig.1.

[0066] The method 300 comprises receiving S312 a second pin, such as a second pin having a second reference color at the junction of the at least two mirrors. The second pinmay be received based on, such as based at least in part on, the second prompt. Such receiving may be performed by junction 204 of Figs. 1 and 2A-2C.

[0067] The method 300 comprises capturing S314 with the camera a second image that includes at least one view of the second pin reflected in one or more of the at least two mirrors. Such capturing may be performed by camera 104 of Fig. 1.

[0068] The method 300 comprises generating S318 a color transformation matrix, such as a color transformation matrix that is based on, such as based at least in part on, the first image with the first reference color and / or the second image with the second reference color. Such generating may be performed by processor 402 and / or memory 401 of Fig. 1.

[0069] In one or more examples of the method 300, the first reference color of the first pin comprises one color and the second reference color of the second pin comprises another color that is different from the first reference color. In one or more examples, the first pin and / or the second pin comprise a ferrous material.

[0070] In one or more examples, the method 300 comprises receiving S316 one or more additional pins, such as one, two, three, or more additional pins that may each have a respective reference color. Each of the additional pins may be received based on, such as based at least in part on, the determined sequence. Such receiving may be performed by junction 204 of Fig. 1.

[0071] In one or more examples of the method 300, the first reference color of the first pin, the second reference color of the second pin, and / or the respective reference colors of the one or more additional pins each comprise a different color. The respective reference colors may be, for example, red, green, blue, black, white, beige, or tan. In one or more examples, the respective reference colors, such as the reference color of the first pin, the reference color of the second pin, and the respective reference color of each of the one or more additional pins are selected from the group consisting of: red, green, blue, black, white, and beige.

[0072] In one or more examples, the method 300 comprises capturing S320 a third image of a granular unit, such as a unit or kernel of grain as described herein. Such capturing may be performed by camera 104 of Fig. 1.

[0073] In one or more examples, the method 300 comprises transforming S322 the third image based on, such as based at least in part on, the color transformation matrix. Such transforming may be performed by processor 402 and / or memory 401 of Fig. 1.

[0074] Fig. 4 is a block diagram illustrating an example device 400 (e.g., apparatus 400) according to this disclosure. The device 400 comprises memory 401 , processor 402, and, in some examples, an interface 403. Memory 401 may be referred to, and may comprise, memory circuitry. Processor 402 may be referred to, and may comprise, processor circuitry. The electronic device 400 may be configured to perform any of the methods disclosed herein, or aspects of such methods, such as those described with reference to Figs. 3A-3B. In other words, the device 400 may be configured for generating a color transformation matrix and / or transformation of an image with or using the color transformation matrix. Device 400 may be an example of computer system 400 described with reference to Fig. 1.

[0075] The device 400 may be configured to communicate with other aspects of a system or other device, such as system 100 and / or aspects of device 102 described with reference to Fig. 1. Such communication may be via interface 403, which may include a wired or wireless interface configured according to one or more standard communication protocols.

[0076] The electronic device 400 is optionally configured to perform one or more of the operations disclosed herein and described with reference to Figs. 3A-3B (such as any one or more of S302, S304, S306, S308, S310, S312, S314, S316, S318, S320, and / or S322). The operations of the device 400 may be embodied in the form of executable logic routines (for example, lines of code, software programs, etc.) that are stored on a non-transitory computer readable medium (for example, memory circuitry 401) and are executed by processor circuitry 402.

[0077] Furthermore, the operations of the device 400 may be considered a method that the device 400 is configured to carry out. Also, while the described functions and operations may be implemented in software, such functionality may as well be carried out via dedicated hardware or firmware, or some combination of hardware, firmware and / or software.

[0078] Memory 401 may be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, a random access memory (RAM), or another suitable device. In a typical arrangement, memory circuitry 401 may include a non-volatile memory for long term data storage and a volatile memory that functions as system memory for processor 402. Memory 401 may exchange data with processor 402 over a data bus. Control lines and an address bus between memory 401 and processor 402 also may be present (not shown in Fig. 4). Memory 401 is considered a non-transitory computer readable medium.

[0079] Memory 401 may be configured to store information (such as information for generating a color transformation matrix, or information indicative of color transformation matrix) in a part of the memory.

[0080] Figs. 5A-5B is a diagram illustrating an example color correction map 500a, 500b according to this disclosure. Color correction map 500, 500b may illustrate an outcome of or calibration resulting from generation and / or use of a color transformation matrix described with reference to the preceding Figs. 1-4. The system 100 (such as device 102) may be seen as generating and / or displaying color correction map 500a, 500b. A color transformation matrix can be generated by capturing images, such as images 118 of Fig.1 , of color reference targets, such as reference colors of pins 206 of Figs. 1-2C, under controlled lighting conditions present within device 102. Device 102, such as with computer system 400 (e.g., device 400) of Figs. 1 and 4, may compute a matrix.

[0081] A color transformation matrix may map colors from images captured by device 102, such as captured red, green, and blue (RGB) values to a standardized color space (such as CIE XYZ or sRGB). A pixel of an image, such as image 118 of Fig. 1, may include RGB values, the calibration of which may affect output or use of such image in analysis. As described above, device 102 may capture a series of images. Device 102, such as via computer system 400, may employ, for example a least squares optimization or singular value decomposition for mapping colors from such images to a standardized color space. The matrix coefficients may minimize the color difference between the measured values and reference values across sampled colors. Color transformation matrix may then be used by device 102 in analysis of images 118 captured from granular objects 114.

[0082] By way of example, color transformation may include values for each of three color channels, such as RGB, for three views per image, and it may include a bias value (or correction factor) for each image view. Such an example would result in a 4x9 matrix. In such an example, for each view captured, blue, green, and red color channels, plus a bias value, may be documented. A descriptive example matrix, including a resulting change to color channels is depicted in Table 1.Table 1. Example Color Transformation Matrix with Description

[0083] Other matrix dimensions are also possible, such as: 3x9, which would include 3 colors for 3 views per image; 4x12, which would include 3 colors for 4 views per image plus a bias value; 3x12, which would include 3 colors for 4 views per image; 4x6, which would include 3 colors for 2 views per image (such as if only one mirror were used) plus a bias value; 3x6, which would include 3 colors for 2 views per image (such as if only one mirror were used); 4x3, which would include 3 colors for 1 view per image plus a bias value; 3x3, which would include 3 colors for 1 view per image; 5x12, which would include 4 colors for 3 views per image plus a bias.

[0084] Color correction map 500a, 500b may be generated from or based on application of a color transformation matrix. For example, color correction map 500a may represent color transformation for each color of pin captured (e.g., images of six pins results in six colors). As discussed above, an image of each pin may include multiple views of the pin, such as a view of the pin captured directly and two views captured from reflections of pins. Color group 502a may thus represent color calibration for each reference color of directly captured views (e.g., a direct view and reflected views, as described with reference to Figs. 2A-3C) of the pins, while color groups 502b and 502c may represent color calibration of images of pins captured from reflections.

[0085] Each color block 503 of a color group 502 may include a first section 504 that represents the reference color of a pin, a second section 506 that represents a color captured in the image, and a third section 508 that represents an adjusted color (e.g., following application of a color transformation matrix). Color correction map 500a may thus indicate the differences or relative changes between reference colors, captured colors, and calibrated colors for each color pin in each of several views. Color correction map 500b depicts the color groups 502 in a consolidated format, which may be displayed to a user (such as via user interface 110 of Fig. 1). The color correction map 500a, 500b includes a schematic depiction of colors for illustrative purposes.

[0086] The skilled person will appreciate that a color correction map 500a, 500b may include actual colors instead of the illustrative patterns depicted herein. For example, for a system that employs pins with reference colors red, green, blue, white, black, and beige, each color group 502 would include a color block for each of those respective colors, which are represented in Figs. 5A-5B with patterns.

[0087] A color transformation matrix and / or color correction map 500 may be stored as data in memory 401 of Figs. 1 and 4 and may be generated and / or utilized by processor 402 as described herein.

[0088] It should be noted that the figures may or may not be drawn to scale and that elements of similar structures or functions are represented by like reference numerals throughout the figures. It should also be noted that the figures are only intended to facilitate the description of the embodiments. They are not intended as an exhaustive description of the disclosure or as a limitation on the scope of the disclosure. In addition, an illustrated embodiment needs not have all the aspects or advantages shown. An aspect or an advantage described in conjunction with a particular embodiment is not necessarily limited to that embodiment and can be practiced in any other embodiments even if not so illustrated, or if not so explicitly described.

[0089] Embodiments of methods, apparatuses, and systems according to the disclosure are set out in the following items:Item 1. A method, comprising:receiving, at a junction of at least two mirrors, a first pin having a first reference color, the at least two mirrors configured with respect to a camera to capture images that include a plurality of views of an object positioned at the junction;capturing with the camera a first image that includes at least one view of the first pin reflected in one or more of the at least two mirrors;receiving a second pin having a second reference color at the junction of the at least two mirrors;capturing with the camera a second image that includes at least one view of the second pin reflected in one or more of the at least two mirrors; andgenerating a color transformation matrix that is based at least in part on the first image with the first reference color and the second image with the second reference color.Item 2. The method of item 1 , wherein the first reference color of the first pin comprises one color and the second reference color of the second pin comprises another color that is different from the first reference color.Item 3. The method of any of items 1-2, further comprising:displaying a first prompt via a user interface, wherein the first pin is received based at least in part on the first prompt; anddisplaying a second prompt via the user interface, wherein the second pin is received based at least in part on the second prompt.Item 4. The method of item 3, wherein the second prompt is displayed based at least in part on capturing the first image.Item 5. The method of item 3, further comprising:determining a sequence of reference colors for the color transformation matrix generation, wherein the first prompt and / or the second prompt is / are displayed based at least in part on the sequence.Item 6. The method of item 5, further comprising:receiving one or more additional pins that each have a respective reference color, wherein each of the additional pins is received based at least in part on the determined sequence.Item 7. The method of any of items 1-6, wherein the first reference color of the first pin, the second reference color of the second pin, and the respective reference colors of the one or more additional pins each comprise a different color selected from the group consisting of: red, green, blue, black, white, and beige.Item 8. The method of any of items 1-7, further comprising:capturing a third image of a granular unit; andtransforming the third image based at least in part on the color transformation matrix.Item 9. The method of any of items 1-8, wherein the junction comprises a magnet.Item 10. The method of any of claims 1-9, wherein the first pin and the second comprise a ferrous material.Item 11. A system, comprising:means for receiving pins each having a respective reference color, the means for receiving configured to position each pin to reflect in a plurality of mirrors configured with respect to a camera to capture images that include one or more views corresponding to the plurality of mirrors;means for capturing with the camera an image that includes at least one view of a received pin reflected in one or more of the mirrors; andmeans for generating a color transformation matrix that is based at least in part on the first image with the first reference color and the second image with the second reference color.Item 12. The system of item 11 , further comprising:means for displaying a prompt via a user interface, wherein the means for receiving is configured based at least in part on the prompt.Item 13. The system of item 12, further comprising:means for determining a sequence of reference colors for the color transformation matrix generation, wherein the means for displaying is configured to display prompts in an order based at least in part on the sequence.Item 14. The system of any of item 11-13, further comprising:means for transforming an image of a granular unit based at least in part on the color transformation matrix.Item 15. An apparatus, comprising:a camera configured to capture images that comprise at least one view reflected in a set of mirrors;the set of mirrors configured to reflect an object traveling in a path of the set of mirrors;a receiver for a pin having a reference color, the receiver configured to position the pin in the path of the set of mirrors; anda processor coupled with memory and configured to generate a color transformation matrix that is based at least in part on images captured of respective reference colors of pins received by the receiver.Item 16. The apparatus of item 15, wherein the receiver is configured to position the pin with respect to an intersection of two mirrors of the set of mirrors.Item 17. The apparatus of any of items 15-16, wherein the receiver comprises a magnet configured to hold the pin at the junction of the set of mirrors.Item 18. The apparatus of any of items 15-17, further comprising:a prism between the set of mirrors and the camera, wherein the camera is configured to capture the image via the prism and the receiver is configured to position the pin with respect to the prism.Item 19. The apparatus of any of items 15-18, wherein the pin comprises a handle portion and a color portion, and the receiver is configured to receive the color portion.Item 20. The apparatus of any of items 15-19, wherein the pin comprises a ferrous material.Item 21. The apparatus of any of items 15-20, wherein the pin comprises steel.Item 22. The apparatus of any of items 15-21, wherein the processor coupled with memory is configured to determine a sequence of reference colors for the color transformation matrix.Item 23. The apparatus of any of items 15-22, further comprising:a user interface, wherein the processor coupled with memory is configured to cause the user interface to display prompts for pins of different reference colors based at least in part on the sequence of reference colors for the color transformation matrix.Item 24. The apparatus of any of items 15-23, wherein the processor coupled with memory is configured to transform an image of the object based at least in part on the color transformation matrix.Item 25. The apparatus of any of items 15-24, wherein the reference color comprises at least one of red, green, blue, black, white, or beige.

[0090] The use of the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. does not imply any particular order, but are included to identify individual elements. Moreover, the use of the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. does not denote any order or importance, but rather the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. are used to distinguish one element from another. Note that the words “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. are used here and elsewhere for labelling purposes only and are not intended to denote any specific spatial or temporal ordering. Furthermore, the labelling of a first element does not imply the presence of a second element and vice versa.

[0091] It may be appreciated that the Figures, such as Figs. 1, 2A-2C, 3A-3B, and 4, comprise some features, circuitries, or operations which are illustrated with a solid line and some features, circuitries, or operations which are illustrated with a dashed line. Features, circuitries, or operations which are comprised in a solid line are features, circuitries, or operations which are comprised in at least one of the broadest example embodiments. Features, circuitries, or operations which are comprised in a dashed line are example embodiments which may be comprised in, or a part of, or are further features, circuitries, or operations which may be taken in addition to the solid line example embodiments. It should be appreciated that these operations need not be performed in order presented. Furthermore, it should be appreciated that not all of the operations need to be performed. The example operations may be performed in any order and in any combination. Further, the claims may not recite all features or combinations disclosed herein. This disclosure, taken as a whole, describes examples and embodiments that may not be claimed herein but to which the applicant is entitled, and which may be the subject of related applications.

[0092] It is to be noted that the word "comprising" does not necessarily exclude the presence of other elements or steps than those listed.

[0093] It is to be noted that the words "a" or "an" preceding an element do not exclude the presence of a plurality of such elements.

[0094] It is to be noted that a list in the form of “at least one of A, B, or C, or any combination thereof” should be understood to mean “A”, or “B”, or “C”, or “A and B”, or “A and C”, or “B and C”, or “A and B and C”.

[0095] It is to be noted that the term "indicative of" may be seen as “associated with”, “related to”, “descriptive of’, “characterizing”, and / or “defining”. The terms “indicative of’, “associated with”, “related to”, “descriptive of”, “characterizing”, and “defining” can be used interchangeably. The term “indicative of’ can be seen as indicating a relation. For example, weight data indicative of weight may comprise one or more weight parameters.

[0096] It is to be noted that the word "based on" may be seen as “as a function of” and / or “derived from”. The terms “based on” and “as a function of” can be used interchangeably. For example, a parameter determined “based on” a data set can be seen as a parameter determined “as a function of’ the data set. A parameter or action may be determined or performed “based on” multiple inputs, items, or predicates. For example, a parameter or action may be “based at least in part on” a first input and “based at least in part on” a second input, which may or may not be expressly recited or claimed. In other words, the parameter may be an output of one or more functions with multiple inputs, including the data set as an input.

[0097] A function may be characterizing a relation between an input and an output, such as mathematical relation, a database relation, a hardware relation, logical relation, and / or other suitable relations.

[0098] It should further be noted that any reference signs do not limit the scope of the claims, that the example embodiments may be implemented at least in part by means of both hardware and software, and that several "means", "units" or "devices" may be represented by the same item of hardware.

[0099] The various example methods, devices, nodes and systems described herein are described in the general context of method steps or processes, which may be implemented in one aspect by a computer program product, embodied in a computer-readable medium, including computer-executable instructions, such as program code,executed by computers in networked environments. A computer-readable medium may include removable and non-removable storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory (RAM), compact discs (CDs), digital versatile discs (DVD), etc. Generally, program circuitries may include routines, programs, objects, components, data structures, etc. that perform specified tasks or implement specific abstract data types. Computer-executable instructions, associated data structures, and program circuitries represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes.

[0100] Although features have been shown and described, it will be understood that they are not intended to limit the claimed disclosure, and it will be made obvious to those skilled in the art that various changes and modifications may be made without departing from the scope of the claimed disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense. The claimed disclosure is intended to cover all alternatives, modifications, and equivalents.

Claims

27CLAIMSWhat is claimed is:

1. A method, comprising:receiving, at a junction of at least two mirrors, a first pin having a first reference color, the at least two mirrors configured with respect to a camera to capture images that include a plurality of views of an object positioned at the junction;capturing with the camera a first image that includes at least one view of the first pin reflected in one or more of the at least two mirrors;receiving a second pin having a second reference color at the junction of the at least two mirrors;capturing with the camera a second image that includes at least one view of the second pin reflected in one or more of the at least two mirrors; andgenerating a color transformation matrix that is based at least in part on the first image with the first reference color and the second image with the second reference color.

2. The method of claim 1 , wherein the first reference color of the first pin comprises one color and the second reference color of the second pin comprises another color that is different from the first reference color.

3. The method of any of claims 1-2, further comprising:displaying a first prompt via a user interface, wherein the first pin is received based at least in part on the first prompt; anddisplaying a second prompt via the user interface, wherein the second pin is received based at least in part on the second prompt.

4. The method of claim 3, wherein the second prompt is displayed based at least in part on capturing the first image.

5. The method of claim 3, further comprising:determining a sequence of reference colors for the color transformation matrix generation, wherein the first prompt and / or the second prompt is / are displayed based at least in part on the sequence.

6. The method of claim 5, further comprising:receiving one or more additional pins that each have a respective reference color, wherein each of the additional pins is received based at least in part on the determined sequence.

7. The method of any of claims 1-6, wherein the first reference color of the first pin, the second reference color of the second pin, and the respective reference colors of the one or more additional pins each comprise a different color selected from the group consisting of: red, green, blue, black, white, and beige.

8. The method of any of claims 1-7, further comprising:capturing a third image of a granular unit; andtransforming the third image based at least in part on the color transformation matrix.

9. The method of any of claims 1-8, wherein the junction comprises a magnet.

10. The method of any of claims 1 -9, wherein the first pin and the second comprise a ferrous material.

11. A system, comprising:means for receiving pins each having a respective reference color, the means for receiving configured to position the pins to reflect in a plurality of mirrors configured with respect to a camera to capture images that include one or more views corresponding to the plurality of mirrors;means for capturing with the camera an image that includes at least one view of a received pin reflected in one or more of the mirrors; andmeans for generating a color transformation matrix that is based at least in part on the first image with the first reference color and the second image with the second reference color.

12. The system of claim 11 , further comprising:means for displaying a prompt via a user interface, wherein the means for receiving is configured based at least in part on the prompt.

13. The system of claim 12, further comprising:means for determining a sequence of reference colors for the color transformation matrix generation, wherein the means for displaying is configured to display prompts in an order based at least in part on the sequence.

14. The system of any of claims 11-13, further comprising:means for transforming an image of a granular unit based at least in part on the color transformation matrix.

15. An apparatus, comprising:a camera configured to capture images that comprise at least one view reflected in a set of mirrors;the set of mirrors configured to reflect an object traveling in a path of the set of mirrors;a receiver for a pin having a reference color, the receiver configured to position the pin in the path of the set of mirrors; anda processor coupled with memory and configured to generate a color transformation matrix that is based at least in part on images captured of respective reference colors of pins received by the receiver.

16. The apparatus of claim 15, wherein the receiver is configured to position the pin with respect to an intersection of two mirrors of the set of mirrors.

17. The apparatus of any of claims 15-16, wherein the receiver comprises a magnet configured to hold the pin at the junction of the set of mirrors.

18. The apparatus of any of claims 15-17, further comprising:a prism between the set of mirrors and the camera, wherein the camera is configured to capture the image via the prism and the receiver is configured to position the pin with respect to the prism.

19. The apparatus of any of claims 15-18, wherein the pin comprises a handle portion and a color portion, and the receiver is configured to receive the color portion.

20. The apparatus of any of claims 15-19, wherein the pin comprises a ferrous material.

21. The apparatus of any of claims 15-20, wherein the pin comprises steel.

22. The apparatus of any of claims 15-21 , wherein the processor coupled with memory is configured to determine a sequence of reference colors for the color transformation matrix.

23. The apparatus of any of claims 15-22, further comprising:a user interface, wherein the processor coupled with memory is configured to cause the user interface to display prompts for pins of different reference colors based at least in part on the sequence of reference colors for the color transformation matrix.

24. The apparatus of any of claims 15-23, wherein the processor coupled with memory is configured to transform an image of the object based at least in part on the color transformation matrix.

25. The apparatus of any of claims 15-24, wherein the reference color comprises at least one of red, green, blue, black, white, or beige.

Images