Ranked Adaptive ROI for Vision Cameras

Abstract

There is a need to improve efficiency in machine vision systems. [Solution] Disclosed herein is a machine vision technique for determining a region of interest (ROI). An exemplary implementation includes a computing device for executing an application operable to: (1) capture a first plurality of images over a field of view (FOV); (2) identify, from the first plurality of images, a plurality of regions of interest (ROIs) within the FOV, where each of the plurality of ROIs is ranked based on a frequency of occurrence of a visual feature within a predetermined proximity range for each of the ROIs in the plurality of images; and (3) sequentially identify candidate locations for the visual feature in a second image based on the rank of each of the ROIs until a visual feature is identified at the candidate location.

Description

Technical Field

[0001] Finding visual features such as barcodes on an object is often useful. Current vision systems achieve this by first capturing an image of the object and then searching the field of view (FOV) of the image (e.g., searching the entire image for a barcode or other object). However, searching the entire image is inefficient. Therefore, there is a need to improve efficiency in these types of machine vision systems.

Summary of the Invention

[0002] In embodiments, the present invention is a method for operating a machine vision system, the machine vision system comprising a computing device for running an application and a fixed imaging device communicatively coupled to the computing device. The method includes (a) acquiring a first image on the field of view (FOV) via the fixed imaging device; (b) analyzing at least a portion of the first image via the application to detect visual features in the first image; (c) determining the location of the visual features in the first image via the application; and (d) determining a first region of interest (ROI) in the first image based on the location of the visual features via the application. (i) ranking the first and second ROIs; (j) acquiring a third image via a fixed imaging device; (k) analyzing at least a portion of the second image via an application to detect visual features in the second image; (g) determining the location of the visual features in the second image via an application; (h) determining a second ROI in the second image based on the location of the visual features via an application; (i) ranking the first and second ROIs; (j) acquiring a third image via a fixed imaging device; (k) analyzing a third ROI in the third image via an application to detect visual features, wherein the third ROI in the third image is ranked; and (l) transmitting data associated with the visual features in the third image to a host processor in response to the detection of the visual features in the third image.

[0003] In a variation of this embodiment, analyzing the third ROI includes setting the third ROI to be a higher-ranked ROI than the first and second ROIs, and updating the third ROI to be a lower-ranked ROI than the first and second ROIs if it is determined that the visual features are not within the set third ROI.

[0004] A further variation of this embodiment further includes, if analyzing the third ROI is determined not to be within the updated third ROI, setting the third ROI to be the FOV of the third image.

[0005] A further variation of this embodiment further includes analyzing a third ROI by analyzing a third ROI set in the FOV of a third image to detect a visual feature; determining the location of the visual feature in the third image via an application; determining a new third ROI in the third image based on the location of the visual feature via an application; determining, via an application, that the new third ROI is within a default tolerance range of the first ROI; and incrementing the weighting coefficient of the first ROI via an application in response to the determination that the new third ROI is within a default tolerance range of the first ROI.

[0006] In a further variation of this embodiment, the method further includes iteratively acquiring images after the analysis of the ROIs of a third image, and in each iteration, incrementing the weighting coefficient of the first ROI if it is determined that the visual feature is within the first ROI, incrementing the weighting coefficient of the second ROI if it is determined that the visual feature is within the second ROI, and re-ranking the first and second ROIs based on the weighting coefficients.

[0007] In a further variation of this embodiment, detecting visual features by analyzing at least a portion of a first image includes determining a bounding box for the visual features via an application, and determining a first ROI within the first image includes applying a scaling factor to the bounding box.

[0008] In a further variation of this embodiment, the method further includes transmitting data associated with a visual feature in a first image to a host processor in response to the detection of a visual feature in a first image, and transmitting data associated with a visual feature in a second image to a host processor in response to the detection of a visual feature in a second image.

[0009] In a further variation of this embodiment, ranking the first and second ROIs includes presenting representations of the first and second ROIs to the user, receiving a user selection from the user for either the first or second ROI, and setting ranks for the first and second ROIs based on this user selection.

[0010] In another embodiment, the present invention is a method for operating a machine vision system, the machine vision system comprising a computing device for running an application and a fixed imaging device communicatively coupled to the computing device. The method may include: acquiring a first plurality of images on a field of view (FOV) via the fixed imaging device; identifying a plurality of regions of interest (ROIs) in the FOV from the first plurality of images via the application, each of which ROIs is associated with a visual feature, and each of the ROIs is ranked based on the frequency of recurrence of the visual feature within a predetermined proximity range to each of the ROIs in the plurality of images; acquiring a second image via the fixed imaging device; sequentially identifying candidate locations of visual features in the second image via the application, based on the rank of each ROI, until a visual feature is identified at a candidate location; and providing data related to the visual features from the second image to a host processor.

[0011] In this variant of the embodiment, the multiple regions of interest are limited to n or fewer ROIs.

[0012] In further variations of this embodiment, n is specified by the user.

[0013] In a further variation of this embodiment, the method further includes receiving an input from a user indicating that the ROI function should be disabled, acquiring a third image via a fixed imaging device, and, in response to receiving this input, setting the ROI in the third image to become the FOV of the third image.

[0014] In a further variation of this embodiment, the method further includes receiving an input from a user indicating that the ROI function should be disabled; acquiring a third image via a fixed imaging device; setting an ROI in the third image based on the highest-ranked ROI among a plurality of ROIs; determining that the feature is not within the ROI in the third image; and, in response to receiving this input and determining that the visual feature is not within the ROI in the third image, setting the ROI in the third image to be within the FOV of the third image.

[0015] In a further variation of this embodiment, the method further includes receiving input from a user indicating that ranking updates should be disabled, and in response to this input, not updating any recurrence frequency of any of the multiple ROIs when identifying visual features in a second image.

[0016] In a further variation of this embodiment, the visual feature is identified at a candidate location based on the fact that the entire visual feature lies within the candidate location.

[0017] In a further variation of this embodiment, visual features are identified at candidate locations based on the fact that more than 50% of the visual features are located within the candidate locations.

[0018] In a further variation of this embodiment, the method further includes creating a ranking table that includes the rank of each of the multiple ROIs, the frequency of recurrence of visual features within a predetermined proximity range for each of the multiple ROIs, and the coordinates of each of the multiple ROIs.

[0019] In a further variation of this embodiment, the visual feature is a barcode.

[0020] In a further variation of this embodiment, the visual feature is a crack on the surface of the structure.

[0021] Similar reference numerals refer to identical or functionally similar elements throughout separate figures. The attached figures, along with the detailed description below, are incorporated into the specification to form part of it and serve to further illustrate conceptual embodiments containing the claimed invention and to illustrate the various principles and advantages of those embodiments. [Brief explanation of the drawing]

[0022] [Figure 1] This is a diagram illustrating an exemplary system for optimizing one or more imaging settings for a machine vision job, according to embodiments described herein. [Figure 2] This is a perspective view of the imaging device shown in Figure 1, according to an embodiment described herein. [Figure 3] This diagram shows an exemplary application interface that includes the identification of a single barcode, where the area of ​​interest (ROI) is the field of view (FOV). [Figure 4] This figure shows an exemplary interface, including the identification and setting of a single barcode, according to the embodiments described herein. [Figure 5] This diagram shows an exemplary interface for a case where the ROI is FOV and the identification of multiple barcodes is included. [Figure 6] This figure shows an exemplary interface, including the identification and setting of multiple barcodes, according to embodiments described herein. [Figure 7] This figure shows an exemplary method for identifying visual features in a barcode, including capturing a first set of images, according to an embodiment. [Figure 8]It is a diagram showing an exemplary ranking table. [Figure 9] It is a diagram showing an exemplary method for identifying visual features in a barcode according to an embodiment.

Best Mode for Carrying Out the Invention

[0023] Those skilled in the art will understand that the elements of the figures are illustrated for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some of the elements of the figures may be exaggerated relative to other elements to help improve the understanding of the embodiments of the present invention.

[0024] The components of the apparatus and method are represented in the drawings by conventional symbols where appropriate, showing only the specific details relevant to understanding the embodiments of the present invention so as not to obscure the disclosure with details that would be readily apparent to those skilled in the art having the benefit of the description herein.

[0025] Broadly speaking, some embodiments described herein use ranking-based adaptive ROI techniques to find visual features in an image. For example, referring to Figure 1, an imaging device 104 may acquire an image and transmit it to a user computing device 102. The imaging application 116 may then use the image to rank the ROIs of the image (for example, based on the frequency of recurrence of visual features within the ROIs, or based on any other suitable technique). The imaging application 116 may then receive additional images and search those additional images for the visual feature in the highest-ranked ROI. If the visual feature is not found in the highest-ranked ROI, the imaging application 116 may search for the visual feature in the second-highest-ranked ROI, and so on, until the visual feature is found. If the visual feature is not found in any of the ROIs, the imaging application 116 may set the ROI to be the FOV of a new image (for example, as a fallback mechanism) so that the imaging application 116 can effectively search the entire new image (for example, at maximum resolution) for the visual feature.

[0026] More specifically, Figure 1 shows an exemplary imaging system 100 configured to locate barcodes or other objects. In the embodiment of Figure 1, the imaging system 100 includes a user computing device 102 and an imaging device 104 that is communicably coupled to the user computing device 102 via a network 106. Generally speaking, the user computing device 102 is configured to allow a user / operator to create machine vision jobs to run on the imaging device 104. Once created, the user / operator may then send / upload the machine vision job to the imaging device 104 via the network 106, and the machine vision job is then interpreted and executed. The user computing device 102 may include one or more operator workstations and may include one or more processors 108, one or more memories 110, a network interface 112, an input / output interface 114, and an imaging application 116.

[0027] The imaging device 104 is connected to the user computing device 102 via a network 106 and is configured to interpret and execute machine vision jobs received from the user computing device 102. Typically, the imaging device 104 may obtain a job file containing one or more job scripts from the user computing device 102 via the network 106, the job scripts may define machine vision jobs, and the imaging device 104 may be configured to acquire and / or analyze images according to the machine vision jobs. For example, the imaging device 104 may include flash memory used to determine, store, or otherwise process the acquisition data / dataset and / or post-acquisition data. The imaging device 104 may then receive, recognize, and / or otherwise interpret triggers to cause the imaging device 104 to acquire an image of a target object, according to a configuration established via one or more job scripts. After acquisition and / or analysis, the imaging device 104 may transmit the image and any associated data to the user computing device 102 via the network 106 for further analysis and / or storage. In various embodiments, the imaging device 104 may be a “smart” camera for acquiring, interpreting, and executing job scripts that define machine vision jobs, such as any one or more job scripts contained in one or more job files, for example, acquired from a user computing device 102, and / or otherwise, may be configured to automatically perform a full range of functions of the imaging device 104.

[0028] The job file may be, broadly speaking, a JSON representation / data format of one or more job scripts transferable from the user computing device 102 to the imaging device 104. The job file may also be readable by a C++ runtime engine or other suitable runtime engine running on the imaging device 104. Furthermore, the imaging device 104 may run a server (not shown) configured to listen for and receive job files from the user computing device 102 via the network 106. Alternatively, the server configured to listen for and receive job files may be one or more cloud-based servers, such as a cloud-based computing platform. For example, the server may be any one or more cloud-based platforms, such as Microsoft Azure or Amazon AWS.

[0029] In any case, the imaging device 104 may include one or more processors 118, one or more memories 120, a network interface 122, and an input / output interface 124, and an imaging assembly 126. The imaging assembly 126 may include a digital camera and / or digital video camera for capturing or shooting digital images and / or frames. Each digital image may include pixel data that can be analyzed by one or more tools, each configured to perform an image analysis task, as described herein. For example, the digital camera and / or digital video camera of the imaging assembly 126 may be configured to shoot, capture, or otherwise generate digital images, and in at least some embodiments, such images may be stored in the memory (e.g., one or more memories 110, 120) of the respective device (e.g., user computing device 102, imaging device 104).

[0030] For example, the imaging assembly 126 may include a photorealistic camera (not shown) for capturing, sensing, or scanning 2D image data. The photorealistic camera may be an RGB (red, green, blue) based camera for capturing 2D images containing RGB-based pixel data. In various embodiments, the imaging assembly may further include a three-dimensional (3D) camera (not shown) for capturing, sensing, or scanning 3D image data. The 3D camera may include an infrared (IR) projector and associated IR camera for capturing, sensing, or scanning 3D image data / data sets. In some embodiments, the photorealistic camera of the imaging assembly 126 may capture 2D images and associated 2D image data at the same or similar time point as the 3D camera of the imaging assembly 126, so that the imaging device 104 can have both sets of 3D and 2D image data available for a particular surface, object, area, or scene at the same or similar time point. In various embodiments, the imaging assembly 126 may include a 3D camera and a photorealistic camera as a single imaging device configured to acquire 3D depth image data simultaneously with 2D image data. As a result, the acquired 2D images and corresponding 2D image data may be depth-aligned with the 3D images and 3D depth image data.

[0031] The entire area that the imaging device 104 can image may be called the field of view (FOV). In this regard, the FOV is determined by the optics of the imaging device 104. Typically, the imaging device 104 may capture images of the environment that appear within the FOV. When analyzing the image, a portion of the image may be identified with respect to pixel values ​​and / or coordinates of the image (e.g., horizontal and vertical pixel values, a Cartesian coordinate system including the pixel values, etc.).

[0032] As mentioned above, pixel data may be analyzed by one or more tools, each configured to perform an image analysis task. For example, a barcode scanning / reading tool may be executed to detect or read a barcode. For this purpose, it may be useful to define a region of interest (ROI) corresponding to a specific area or object captured by the imaging assembly 126 and execute a tool (e.g., execute a barcode scanning / reading tool) within the ROI. In this regard, as will become clear from the following description, the determination of the ROI may be performed as part of the execution of a tool (e.g., the execution of a barcode scanning / reading tool). Furthermore, or alternatively, the determination of the ROI may be specifically performed using an ROI designation / determination tool.

[0033] To further describe jobs and the corresponding tools for jobs, the imaging application 116 may typically present the user with a set of menus for creating a new job or editing an existing job. In creating a new job, the user can select from a variety of tools that make up a particular job. Such tools may include, but are not limited to, (i) barcode scanning / reading tools, (ii) pattern matching tools, (iii) edge detection tools, (iv) semantic segmentation tools, (v) object detection tools, (vi) object tracking tools, (vii) region of interest designation / determination tools, etc. For example, if a first type of product is moving along an assembly line, the user may configure a first job that includes selecting and configuring specific tools to be performed (e.g., continuously or iteratively) while the first type of product is moving along the assembly line. Then, when the assembly line switches to a second type of product, the user may terminate the first job and start a second job by configuring a second job that includes selecting and configuring different tools that are more useful for analyzing the second type of product.

[0034] The imaging device 104 may also process 2D image data / datasets and / or 3D image datasets for use by other devices (e.g., a user computing device 102, an external server). For example, one or more processors 118 may process scanned or sensed image data or datasets acquired by the imaging assembly 126. Processing of the image data may generate post-imaging data, which may include metadata, simplified data, normalized data, result data, status data, or warning data, as determined from the original scanned or sensed image data. The image data and / or post-imaging data may be sent to a user computing device 102 running an imaging application 116 for display, operation, and / or other information exchange. In other embodiments, the image data and / or post-imaging data may be sent to a server for storage or further operation. As described herein, the user computing device 102, the imaging device 104, and / or an external server or other centralized processing unit and / or storage may store such data and / or transmit the image data and / or post-imaging data to another application running on a user device such as a mobile device, tablet, handheld device, or desktop device.

[0035] Each of the one or more memories 110, 120 may include one or more forms of volatile and / or non-volatile, fixed and / or removable memory, such as read-only memory (ROM), electronic programmable read-only memory (EPROM), random access memory (RAM), erasable electronic programmable read-only memory (EEPROM), and / or other hard drives, flash memory, microSD cards, etc. In general, computer programs or computer-based products, applications, or code (e.g., imaging application 116 or other computational instructions described herein) may be stored on computer-usable storage media or tangible non-temporary computer-readable media (e.g., standard random-access memory (RAM), optical disks, universal serial bus (USB) drives, etc.) on which such computer-readable program code or computer instructions are embodied, in order to facilitate, implement, or execute machine-readable instructions, methods, processes, elements, or limitations, such as those illustrated, depicted, or described in relation to the various flowcharts, examples, diagrams, figures, and / or other disclosures herein. The computer-readable program code or computer instructions may be installed on one or more processors 108, 118 or otherwise adapted to be executed by one or more processors 108, 118 (e.g., operating in relation to their respective operating systems in one or more memories 110, 120).In this regard, the program code may be implemented in any preferred programming language, and may be implemented as machine code, assembly code, bytecode, interpretable source code, etc. (for example, by Golang, Python, C, C++, C#, Objective-C, Java, Scala, ActionScript, JavaScript, HTML, CSS, XML, etc.).

[0036] One or more memories 110, 120 may store an operating system (OS) (e.g., Microsoft Windows, Linux®, Unix, etc.) that can facilitate functions, apps, methods, or other software as described herein. One or more memories 110 may also store an imaging application 116 which may be configured to enable the construction of machine vision jobs as further described herein. Furthermore, or alternatively, the imaging application 116 may also be stored in one or more memories 120 of the imaging device 104 and / or an external database (not shown) which is accessible by a user computing device 102 via a network 106 or otherwise connected to communicate with the user computing device 102. One or more memories 110, 120 may also store machine-readable instructions, including one or more applications, one or more software components, and / or one or more application programming interfaces (APIs), which may be implemented to facilitate or perform any features, functions, or other disclosures described herein, such as any methods, processes, elements, or limitations, as illustrated, depicted, or described in relation to the various flowcharts, examples, diagrams, figures, and / or other disclosures herein. For example, at least part of an application, software component, or API may be a machine vision-based imaging application, such as imaging application 116, or include an imaging application, or otherwise be part of an imaging application, each of which may be configured to facilitate the various functions described herein. One or more other applications may be assumed, and it should be understood that they are executed by one or more processors 108, 118.

[0037] One or more processors 108, 118 may be connected to one or more memories 110, 120 via a computer bus that is responsible for transmitting electronic data, data packets, or other electronic signals between one or more processors 108, 118 and one or more memories 110, 120 in order to implement or perform machine-readable instructions, methods, processes, elements, or limitations as illustrated, depicted, or described in relation to various flowcharts, examples, diagrams, figures, and / or other disclosures herein.

[0038] One or more processors 108, 118 may interface with one or more memories 110, 120 via a computer bus to execute an operating system (OS). One or more processors 108, 118 may also interface with one or more memories 110, 120 via a computer bus to create, read, update, delete, or otherwise access or interact with data stored in one or more memories 110, 120 and / or an external database (e.g., a relational database such as Oracle, DB2, MySQL®, or a NoSQL-based database such as MongoDB). The data stored in one or more memories 110, 120 and / or the external database may include all or part of any of the data or information described herein, including, for example, machine vision job images (e.g., images captured by the imaging device 104 in response to the execution of a job script) and / or other appropriate information.

[0039] The network interfaces 112, 122 may be configured to transmit (e.g., send and receive) data to one or more networks or local terminals, such as network 106 described herein, via one or more external ports / network ports. In some embodiments, the network interfaces 112, 122 may include client-server platform technologies such as ASP.NET, Java J2EE, Ruby on Rails, Node.js, web services, or online APIs, which are responsible for receiving and responding to electronic requests. The network interfaces 112, 122 may implement client-server platform technologies that exchange information with one or more memories 110, 120 (including stored applications, components, APIs, data, etc.) via a computer bus in order to implement or execute machine-readable instructions, methods, processes, elements, or limitations, such as those illustrated, depicted, or described in relation to the various flowcharts, examples, diagrams, figures, and / or other disclosures herein.

[0040] In some embodiments, network interfaces 112, 122 may include, or exchange information with, one or more transceivers (e.g., WWAN, WLAN, and / or WPAN transceivers) that function in accordance with IEEE standards, 3GPP® standards, or other standards and may be used in receiving and transmitting data via external ports / network ports connected to network 106. In some embodiments, network 106 may include a private network or a local area network (LAN). Furthermore, or alternatively, network 106 may include a public network such as the Internet. In some embodiments, network 106 may include, in non-limiting examples, a router, wireless switch, or other such wireless connection point that communicates with the user computing device 102 and imaging device 104 via wireless communication based on any one or more of various wireless standards, including IEEE 802.11a / b / c / g (WIFI), BLUETOOTH® standards, etc.

[0041] The input / output interfaces 114, 124 may include or implement an operator interface configured to present information to an administrator or operator and / or to receive input from an administrator or operator. The operator interface may provide a display screen (e.g., via the user computing device 102 and / or imaging device 104) that the user / operator may use to visualize any images, graphics, text, data, features, pixels, and / or other appropriate visualizations or information. For example, the user computing device 102 and / or imaging device 104 may at least partially include, implement, access, draw, or otherwise expose a graphical user interface (GUI) for displaying images, graphics, text, data, features, pixels, and / or other appropriate visualizations or information on the display screen.

[0042] The input / output interfaces 114, 124 may also include input / output components (e.g., ports, capacitive or resistive touch sensor input panels, keys, buttons, lights, LEDs) that can be directly or indirectly accessed via the user computing device 102 and / or the imaging device 104, or attached to the user computing device 102 and / or the imaging device 104. According to some embodiments, an administrator or user / operator may access the user computing device 102 and / or the imaging device 104 to build jobs, review images or other information, make changes, enter responses and / or selections, and / or perform other functions.

[0043] As described herein, in some embodiments, the user computing device 102 may perform functions as described herein as part of a “cloud” network, or otherwise communicate with other hardware or software components in the cloud to transmit, retrieve, or otherwise analyze the data or information described herein.

[0044] Figure 2 is a perspective view of the imaging device 104 of Figure 1 according to an embodiment described herein. The imaging device 104 includes a housing 202, an imaging aperture 204, a user interface label 206, a dome-shaped switch / button 208, one or more light-emitting diodes (LEDs) 210, and a mounting point 212. As previously stated, the imaging device 104 obtains job files from a user computing device (e.g., user computing device 102), and then the imaging device 104 interprets and executes these job files. The instructions contained in the job files may include device configuration settings (also referred to herein as “imaging settings”) that can be manipulated to adjust the configuration of the imaging device 104 before capturing an image of a target object.

[0045] For example, device configuration settings may include instructions for adjusting one or more settings related to the imaging aperture 204. As an example, suppose at least part of the intended analysis corresponding to a machine vision job requires the imaging device 104 to maximize the brightness of any captured image. To meet this requirement, the job file may include device configuration settings for increasing the aperture size of the imaging aperture 204. The imaging device 104 may interpret these instructions (e.g., by one or more processors 118) and increase the aperture size of the imaging aperture 204 accordingly. Thus, the imaging device 104 may be configured to automatically adjust its own configuration to best suit a particular machine vision job. Furthermore, the imaging device 104 may include, or be adaptable to include, one or more bandpass filters, one or more polarizers, one or more DPM diffusers, one or more C-mount lenses, and / or one or more C-mount liquid lenses, for example, affecting illumination on or otherwise received through the imaging aperture 204.

[0046] The user interface label 206 may include a dome-shaped switch / button 208 and one or more LEDs 210, thereby enabling a variety of interactive and / or directive functions. Typically, the user interface label 206 may enable the user to trigger and / or adjust the imaging device 104 (e.g., by the dome-shaped switch / button 208), and to recognize (e.g., by one or more LEDs 210) when one or more functions, errors, and / or other actions have been performed or are being performed with respect to the imaging device 104. For example, the trigger function of the dome-shaped switch / button (e.g., dome-shaped switch / button 208) may enable the user to acquire images using the imaging device 104 and / or display a trigger configuration screen of a user application (e.g., imaging application 116). The trigger configuration screen may enable the user to configure one or more triggers for the imaging device 104, which can be stored in memory (e.g., one or more memories 110, 120) for use in machine vision jobs developed later, as described herein.

[0047] As another example, the adjustment function of a dome-shaped switch / button (e.g., dome-shaped switch / button 208) may allow the user to automatically and / or manually adjust the configuration of the imaging device 104 according to a preferred / default configuration, and / or display an imaging configuration screen of a user application (e.g., imaging application 116). The imaging configuration screen may allow the user to configure one or more configurations of the imaging device 104 (e.g., aperture size, exposure length, etc.) which can be stored in memory (e.g., one or more memories 110, 120) for use in machine vision jobs developed later, as described herein.

[0048] To further this example, as will be further described herein, a user may use an imaging configuration screen (or, more generally, imaging applications 116, 156) to establish two or more configurations of imaging settings for the imaging device 104. The user may then save these two or more configurations of imaging settings as part of a machine vision job, and these configurations are then sent to the imaging device 104 in a job file containing one or more job scripts. One or more job scripts may then instruct the processor of the imaging device 104 (e.g., one or more processors 118) to automatically and sequentially adjust the imaging settings of the imaging device according to one or more of the two or more configurations of imaging settings after each successive image acquisition.

[0049] The mounting point 212 may allow the user to attach the imaging device 104 to mounting devices (e.g., imaging tripod, camera mount, etc.), structural surfaces (e.g., warehouse walls, warehouse ceilings, structural support beams, etc.), other accessory items, and / or any other suitable connecting devices, structures, or surfaces, and / or to detachably attach them. For example, the imaging device 104 may be optimally positioned on mounting devices in distribution centers, manufacturing plants, warehouses, and / or other facilities to image and thereby monitor the quality / consistency of products, packages, and / or other items as they pass through the imaging device 104's FOV. Furthermore, the mounting point 212 may allow the user to attach the imaging device 104 to a number of accessory items, including, but not limited to, one or more external lighting devices, one or more mounting devices / brackets, etc.

[0050] In addition, the imaging device 104 may include several hardware components contained within the enclosure 202 that enable connection to a computer network (e.g., network 106). For example, the imaging device 104 may include a network interface (e.g., network interface 122) that enables the imaging device 104 to connect to a network such as a Gigabit Ethernet connection and / or a dual Gigabit Ethernet connection. Furthermore, the imaging device 104 may include transceivers and / or other communication components as part of the network interface for communicating with other devices (e.g., user computing device 102) via, for example, Ethernet / IP, PROFINET, Modbus TCP, CC-Link, USB 3.0, RS-232, and / or any other suitable communication protocol, or a combination thereof.

[0051] One application of the aforementioned technology is the identification of one or more barcodes (or other visual features such as cracks on the surface of a structure) within an image. Current techniques for identifying barcodes in an image involve searching the entire field of view (FOV) of the image. However, this technique is inefficient because barcodes may only be located in a portion of the image. Therefore, some embodiments set the ROI to be only a portion of the FOV and search only the ROI.

[0052] It should be understood that the term “image” as used herein can refer to a single image or a video frame. Therefore, the techniques described herein apply equally to single images and videos.

[0053] Figures 3 to 6 show exemplary application interfaces according to embodiments described herein. These exemplary application interfaces 300, 400, 500, and 600 can represent interfaces for imaging applications (e.g., imaging application 116) that can be accessed by a user via a user computing device (e.g., user computing device 102).

[0054] In Figure 3, an exemplary application interface 300 shows the identification of a single barcode 310, where the region of interest (ROI) 330 is the field of view (FOV) 320 (e.g., the entire image). In this regard, in some embodiments, the default ROI is the FOV. Furthermore, in this example, the imaging application 116 determines the bounding box 340 of the barcode 310.

[0055] In Figure 4, the exemplary application interface 400 also demonstrates the identification of a single barcode 310. However, in contrast to the example in Figure 3, the ROI 330 is set to a smaller portion of the image (e.g., not the entire FOV 320), thus saving time and processing power in searching the image.

[0056] In Figure 5, an exemplary application interface 500 shows the identification of barcode 310 and the identification of additional barcodes 510. In the example in Figure 5, the ROI 330 is set to the FOV 320 (e.g., the entire image), and a bounding box 540 is drawn around the barcode 510.

[0057] In Figure 6, an exemplary application interface 600 also shows the identification of barcode 310 and additional barcode 510. However, in contrast to the example in Figure 5, the ROI 330 is set in a smaller portion of the image (e.g., not the entire FOV 320), thus saving time and processing power in searching the image. The example in Figure 6 also shows a bounding box 540 drawn around barcode 510.

[0058] Furthermore, regarding the examples in Figures 3 to 6, although the example ROI 330 shown is a rectangle, the ROI can be any shape. For example, the ROI can be a rectangle, a triangle, a pentagon, or a polygon with n sides. Alternatively, the ROI can be specified to be a percentage of the FOV (e.g., 50% of the FOV). Alternatively, the user can manually set the ROI by drawing a specific shape (e.g., via the user computing device 102) or by specifying a set of parameters.

[0059] Generally speaking, some embodiments described herein help find visual features in an image using ranking-based adaptive ROI techniques. For example, the imaging application 116 may have data on multiple ROIs used in past iterations of a job or in past jobs. The imaging application 116 may rank the ROIs (for example, based on the recurrence frequency of visual features within the ROI, or based on any other suitable technique). The imaging application 116 receives a new image and then searches for the visual feature in the highest-ranked ROI. If the visual feature is not found in the highest-ranked ROI, the imaging application 116 may search for the visual feature in the second-highest-ranked ROI, and so on, until the visual feature is found. If the visual feature is not found in any of the ROIs, the imaging application 116 may set the ROI to be the FOV of the new image so that the imaging application 116 can effectively search the entire new image for the visual feature.

[0060] Figure 7 shows an exemplary method 700 for identifying visual features in a barcode. In block 705, the imaging device 104 acquires a first set of images. The images may be sequential, or there may be intervening images between them. For example, the images may be in a video stream and separated by a predefined interval. It should be understood that the user can customize how the images are acquired (for example, via a user computing device 102) (for example, by setting the interval between image acquisitions).

[0061] In block 710, multiple ROIs are identified from the first set of images. Each of the multiple ROIs may be associated with a visual feature (e.g., a barcode, a crack on the surface of a structure, etc.).

[0062] In some embodiments, one or more ROIs (e.g., ROIs in the first received image) may be identified by searching the image's FOV for a visual feature. Once a visual feature is found, a bounding box (e.g., 340) may be drawn around the visual feature (e.g., 310, 510). Next, a scaling factor (e.g., 10%, 20%, 50%, 100%, 500%, 1,000%, etc.) may be applied to the bounding box to determine the ROI. For example, if the scaling factor is 100%, each side of the bounding box may be enlarged by 100% to form the ROI. The user may set the scaling factor in the system. Furthermore, with respect to the bounding box, in some embodiments, the bounding box precisely encompasses the visual feature. In other embodiments, the bounding box encompasses the visual feature plus a smaller area outside the visual feature. For example, the bounding box may encompass the visual feature plus an area that extends each side of the visual feature by 10%.

[0063] Furthermore, each ROI may be ranked. For example, a particular ROI may be applied to and evaluated across multiple images to determine whether a visual feature is found within that ROI, thereby generating the number of recurrences and / or frequency of recurrences of the visual feature within that ROI. Thus, a ranking may be created based on the number of recurrences and / or frequency of recurrences. For example, the ROI with the highest frequency of recurrence may have the highest rank, the ROI with the second highest frequency of recurrence may have the second highest rank, and so on. If there is a tie in rank between recurrences, any appropriate technique may be used to resolve the tie. For example, to resolve the tie, the ROI in which the most recent visual feature was found may be ranked higher. In another example, two ROIs with the same frequency of recurrence may be given the same rank (e.g., rank 5 out of 10 ROIs), but when searching for new images, the ROI with the more recent detection of the visual feature may be applied first (e.g., as further explained with respect to the last two rows of the exemplary ranking table 800 in Figure 8).

[0064] In some embodiments, the recurrence rate is based on the entire visual feature being within the ROI. However, in other embodiments, the recurrence rate is based on the visual feature being within the ROI above a certain percentage (e.g., 50%, 75%, 90%), or on the visual feature being within a specific margin of the ROI. In this regard, the recurrence rate may be based on the visual feature being within a predetermined proximity range to the ROI. The percentage by which the visual feature must be within the ROI, or the specific margin by which the visual feature must be within the ROI, may be set by the user.

[0065] In addition, the frequency of recurrence may be within the same job or across different jobs. For example, when starting a new job, the user may optionally choose to import data from the previous job (e.g., a ranking table such as the exemplary ranking table 800) so that the ranking system starts the new job with the ranked ROIs from the previous job already included.

[0066] Furthermore, or alternatively, the recall rate may relate to the number of previous images analyzed (e.g., the last 1000 images analyzed) or to the number of previous triggers (e.g., a trigger is the occurrence of a visual feature detected within an image).

[0067] The ranking may be recorded in a ranking table, such as the example ranking table 800 in Figure 8. In the example shown, column 810 identifies ROIs according to coordinates. In the example shown, only one coordinate is given to identify an ROI. This coordinate may be any suitable coordinate to identify an ROI, such as the coordinate of a corner of the ROI (e.g., the top-left corner) or the coordinate of the center of the ROI. However, in other examples, the ranking table may list two or more coordinates to identify an ROI (e.g., the coordinates of all four corners).

[0068] Column 820 shows the number of recurrences (e.g., the number of visual features that are within the ROI or within a default proximity range to the ROI). It should be understood that the recurrence rate is simply the number of recurrences divided by the total number of last triggers; for example, in the example in Figure 8, for an ROI with coordinates X1, Y1, the recurrence rate is 600 / 1000. Column 830 shows the ranking of the ROIs.

[0069] To facilitate the ranking process, some embodiments include weighting factors, as in the example in column 840. For example, ROIs may be ranked whole or partially based on weighting factors. In some implementations, the weighting factor may be incremented each time a visual feature is detected within the ROI or within a predetermined proximity range to the ROI. In other examples, the weighting factor may be incremented when a visual feature is detected a predetermined number of times (e.g., 5 times) within the ROI or within a predetermined proximity range to the ROI. In one such example, the rank is determined first based on the weighting factor, and then ties between weighting factors are resolved using the recurrence count.

[0070] Furthermore, in some cases, a visual feature may be detected more than once within each ROI. For example, in the examples in Figures 5 and 6, two or more barcodes 510 may be detected (e.g., within the ROI or within a predetermined proximity range to the ROI). In some embodiments, when two or more visual features are detected within the ROI, the recurrence count (e.g., in column 820) is still incremented as if the visual feature had been detected only once (e.g., the number in column 820 is incremented by a maximum of one per image, regardless of how many times the visual feature is detected). However, in other embodiments, the recurrence count is incremented based on the number of times the visual feature has been detected within the ROI (e.g., by one each time a visual feature is detected). In addition, in some embodiments, the weighting coefficient may be incremented based on the number of times the visual feature has been detected within the ROI (e.g., the weighting coefficient is incremented by one if the visual feature is detected twice within the ROI).

[0071] Returning to the example in Figure 7, in block 715, the imaging device 104 acquires a second image. Note that the second image may be one of the first set of images, or an image that is not part of the first set of images.

[0072] In block 720, the imaging application 116 sequentially identifies candidate locations (e.g., ROIs) of a visual feature in the second image based on the rank of each ROI until a visual feature is identified at a candidate location. For example, the first candidate location may be the highest-ranked ROI. If no visual feature is found in the first ROI, the next candidate location (e.g., the next highest-ranked ROI) may be searched until a visual feature is found. In some embodiments, the number n of candidate locations (e.g., ROIs) to be searched may be set by the user.

[0073] In block 725, data related to the visual feature may be provided to a host processor such as processor 108. The data related to the visual feature may be arbitrary data. Examples of data include decoded barcode data and image data of the ROI where the visual feature was located.

[0074] Following block 725, the process may optionally continue iterating. For example, the process may acquire a third image and apply the ranked ROI technique described herein to identify visual features in the third image. However, further or alternatively, in the optional block 730, the imaging application 116 may receive input indicating that an ROI function should be disabled (e.g., continue running the job, but a specific ROI function will be disabled). In one example, the user may choose to disable all adaptive ROI functions. In this example, after receiving the user input, the imaging application 116 may analyze the FOV rather than ROIs from ranked ROIs to detect visual features. In another example, the user may partially disable the adaptive ROI functions. In such an example, the imaging application 116 may first search for the highest-ranked ROI for visual features, but if no visual features are found within the highest-ranked ROI, the imaging application then searches the FOV rather than iterating through the remaining ranked ROIs.

[0075] In yet another example, the user may disable ranking updates (for example, they may pause ranking updates). For instance, a third received image may be analyzed according to the ranked ROI, but no updates may be made to any ranking table, such as the example ranking table 800.

[0076] Figure 9 shows a further exemplary method 900, which includes receiving a first image. The exemplary method 900 begins in block 905 when the imaging device 104 acquires a first image on the FOV. The imaging application 166 may then analyze at least a portion of the first image in block 910 to detect visual features within the first image.

[0077] In some embodiments, if the visual feature is a barcode, the imaging application 116 detects the barcode by being able to decode it. However, it should be noted that the barcode does not necessarily need to be decoded for its presence to be detected. Furthermore, it should be understood that the term "barcode" as used herein also refers to valid barcode equivalents such as quick response (QR) codes or other information markers.

[0078] Furthermore, in an embodiment where the system can access previous data such as a ranking table in block 910, at least a portion of the analyzed first image may be the highest-ranked ROI. However, in an embodiment where the system cannot access such previous data, at least a portion of the analyzed first image may be the FOV of the first image, and the first ROI may be constructed as described below, particularly with respect to block 920.

[0079] In block 915, the imaging application 116 may determine the location of a visual feature in the first image. In block 920, the imaging application 116 may then determine a first ROI in the first image. In some embodiments, this determination is based on the location of the visual feature. For example, the imaging application 116 may place a bounding box around the visual feature and then apply a scaling factor (e.g., 10%, 20%, 50%, 100%, 500%, 1,000%, etc.) to the bounding box to create / determine the ROI.

[0080] In some embodiments, instead of the imaging application 116 automatically determining the first ROI, the user may manually set the first ROI by drawing a specific shape (e.g., on a display showing the first image) (e.g., via a user computing device 102) or by specifying a set of parameters such as the shape of the ROI (e.g., a rectangle, triangle, pentagon, or polygon with n sides) or the size of the ROI (e.g., the ROI is 50% of the FOV).

[0081] In block 925, the imaging device 104 acquires a second image. In block 930, the imaging application 116 analyzes at least a portion of the second image to detect visual features within the second image. In block 935, the imaging application 116 determines the location of the visual features within the second image. In block 940, the imaging application 116 may then determine a second ROI within the second image. In some embodiments, this determination is based on the location of the visual features. For example, the imaging application 116 may place a bounding box around the visual features and then apply a scaling factor (e.g., either the same scaling factor as or a different scaling factor that may have been applied to the visual features in the first image) to the bounding box to create / determine the ROI.

[0082] However, in some embodiments, optionally in block 930, at least a portion of the analyzed second image includes a first ROI (determined, for example, in block 920) (not, for example, the FOV of the second image). In some embodiments, if no visual features are found within the first ROI, the imaging application 116 may search for other ROIs (if any from previous data) until visual features are found, for example, according to ranking techniques described herein.

[0083] In block 945, the imaging application 116 ranks the first and second ROIs. The first and second ROIs may be ranked by any suitable technique, such as based on recurrence frequency and / or weighting factors, as previously described. For example, if previously acquired data exists, the imaging application may compare the first and / or second ROIs with ROIs already known (e.g., listed in ranking table 800) (e.g., a perfect match or within a predetermined tolerance), update the ranking in ranking table 800, and thereby rank the first and second ROIs. In another example, if no previously acquired data exists and the visual feature is detected once in each of the first and second images, ties in rank may exist between recurrence frequencies, in which case the ties may be resolved as described above. In yet another example, ranking may be achieved by presenting a depiction of the first and second ROIs to the user and receiving ROI selections from the user.

[0084] In block 950, the imaging device 104 acquires a third image. In block 955, the imaging application 116 searches for the higher-ranked of the first and second ROI detections (e.g., detection within the ROI or detection within a predetermined proximity range to the ROI) for the visual feature. If no visual feature is found, in determination block 960, the imaging application 116 searches for the lower-ranked of the first and second ROI for the visual feature. If no visual feature is found in block 960, the imaging application 116 searches for the FOV of the third image for the visual feature in block 965.

[0085] As an alternative, if no visual features are found, following block 960, the imaging application 116 may instead continue iterating through lower-ranked ROIs until a visual feature is found, or until no visual features are found in any ROI, so that the FOV is then searched. In some such embodiments, the user may control (e.g., by a setting) the number of ROIs iterated through before switching to the FOV if no visual features are found in any of the ROIs.

[0086] After a visual feature is detected, data associated with the visual feature may be sent to one or more host processors, such as processor 118 (for example, in block 970). However, it should be noted that any data may be sent to the host processor at any time. For example, data associated with a visual feature in the first image may be sent (for example, following block 910, etc.), and data associated with a visual feature in the second image may be sent (for example, following block 930, etc.).

[0087] Furthermore, it should be understood that the exemplary method 900 may be continued in accordance with the techniques described herein. For example, following block 970, the method may continue its iteration (e.g., by taking in a fourth image, searching for the best weighted ROI in the fourth image for a visual feature, etc.).

[0088] Furthermore, it should be understood that each of the actions described in exemplary methods 700 and 900 may be performed in any order, number of times, or in any other suitable combination. For example, some or all of the blocks of methods 700 and 900 may be performed once or more times in their entirety. In some exemplary implementations, some parts of the blocks may not be performed while still causing the actions described herein. Moreover, methods 700 and 900 are not mutually exclusive, and the blocks may be performed in relation to and / or in combination with each other in any suitable manner.

[0089] In addition, in some embodiments, machine learning (e.g., convolutional neural networks, deep learning algorithms, etc.) may be used to analyze the ranking data collected by the imaging system 100. In particular, there are two advantageous implementations of this.

[0090] In the first implementation, there exists a process plant containing multiple imaging devices 104. Each imaging device 104 may collect data and generate a ranking table (e.g., ranking table 800). The collected data and / or ranking table may be analyzed by a machine learning algorithm to create a default ranking table or a default ranked set of ROIs to be used for future deployment of imaging devices 104 across the entire process plant. Advantageously, having such a default ranked set of ROIs improves the technical performance of new imaging devices 104 deployed in the processing plant. In particular, using a default ranked set of ROIs improves processing time compared to new imaging devices 104 deployed in a processing plant without adaptive ROI capabilities.

[0091] In the second implementation, data from the imaging devices 104 of the processing plant is analyzed, and a ranking table is developed based on machine learning analysis that can be used for multiple imaging devices 104 (e.g., all imaging devices 104 in the plant, only imaging devices in a specific section of the plant, only imaging devices that capture information on a specific product, etc.).

[0092] (Additional considerations) The above description refers to the block diagram in the attached drawings. Alternative implementations of the examples represented by the block diagrams include one or more additional or alternative elements, processes, and / or devices. Furthermore, or alternatively, one or more of the exemplary blocks in the figures may be combined, split, rearranged, or omitted. The components represented by the blocks in the figures are implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. In some examples, at least one of the components represented by the blocks is implemented by a logic circuit. As used herein, the term “logic circuit” is expressly defined as a physical device comprising at least one hardware component configured to control one or more machines and / or to perform the operations of one or more machines (e.g., by operation by default configuration and / or by execution of stored machine-readable instructions). Examples of logic circuits include one or more processors, one or more coprocessors, one or more microprocessors, one or more controllers, one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more microcontroller units (MCUs), one or more hardware accelerators, one or more special-purpose computer chips, and one or more system-on-a-chip (SoC) devices. Some exemplary logic circuits, such as ASICs or FPGAs, are hardware specifically configured to perform operations (for example, the operations described herein and, if any, one or more of the operations represented by the flowcharts of this disclosure).Some exemplary logic circuits are hardware that executes machine-readable instructions to perform operations (e.g., operations described herein, and, if one or more of the operations represented by the flowcharts of this disclosure exist, those operations). Some exemplary logic circuits include combinations of specially configured hardware and hardware that executes machine-readable instructions. The above description refers to the various operations described herein and the flowcharts that may be attached herein to illustrate the flow of those operations. Any such flowchart represents an exemplary method disclosed herein. In some examples, the method represented by the flowchart implements the apparatus represented by the block diagram. Alternative implementations of the exemplary methods disclosed herein may include additional or alternative operations. Furthermore, the operations of alternative implementations of the methods disclosed herein may be combined, separated, rearranged, or omitted. In some examples, the operations described herein are performed by machine-readable instructions (e.g., software and / or firmware) stored in a medium (e.g., a tangible machine-readable medium) for execution by one or more logic circuits (e.g., processors). In some examples, the operations described herein are performed by one or more configurations of one or more specially designed logic circuits (e.g., ASICs). In some examples, the operations described herein are performed by a combination of a specially designed logic circuit and machine-readable instructions stored in a medium (e.g., a tangible machine-readable medium) for execution by the logic circuit.

[0093] When used herein, each of the terms “tangible machine-readable medium,” “non-temporary machine-readable medium,” and “machine-readable storage device” is explicitly defined as a storage medium (e.g., a hard disk drive platter, a digital versatile disk, a compact disk, flash memory, read-only memory, random access memory, etc.) on which machine-readable instructions (e.g., program code in the form of software and / or firmware) are stored for any appropriate period of time (e.g., permanently, for a long period (e.g., while a program associated with a machine-readable instruction is running), and / or for a short period (e.g., while a machine-readable instruction is cached and / or during a buffering process)). Furthermore, when used herein, each of the terms “tangible machine-readable medium,” “non-temporary machine-readable medium,” and “machine-readable storage device” is explicitly defined to exclude propagating signals. That is, when used in any claim of this patent, the terms “tangible machine-readable medium,” “non-temporary machine-readable medium,” and “machine-readable storage device” cannot be read if they are carried out by propagating signals.

[0094] The above specification describes specific embodiments. However, those skilled in the art will understand that various modifications and changes can be made without departing from the scope of the invention as set forth in the claims below. Accordingly, this specification and the drawings should be considered illustrative, not restrictive, and all such changes are intended to be within the scope of this teaching. Furthermore, the described embodiments / examples / implements should not be construed as mutually exclusive, but rather understood as potentially combinatorial if their combination is permissible in any way. In other words, any feature disclosed in any of the above embodiments / examples / implements may be included in any of the other above embodiments / examples / implements.

[0095] No advantage, superiority, solution to a problem, or any element that can produce or make more prominent any advantage, superiority, or solution should be construed as any material feature or element, essential feature or element, or intrinsic feature or element of any part or all of the claims. The claimed invention is defined solely by the attached claims, including any amendments made during the pendency of this application and all equivalents of the issued claims.

[0096] Furthermore, in this document, terms indicating relationships such as first and second, upper and lower may be used simply to distinguish one entity or action from another entity or action, without necessarily requiring or implying such an actual relationship or order between entities or actions. The terms “equip,” “have,” “possess,” “include,” “contain,” “contain,” or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that equips, equips, equips, has, equips, or contains a list of elements may not contain only those elements, but may also contain other elements that are not expressly listed or that are inherent to such process, method, article, or apparatus. The element (~) in “...equips…a,” “...has…a,” “...includes…a,” and “...contains…a” does not, without further constraint, exclude the presence of additional identical elements in the process, method, article, or apparatus that equips, equips, has, equips, or contains this element. The terms “a” and “an” are defined as one or more unless otherwise expressly stated herein. The terms “substantially,” “basically,” “approximately,” “about,” or any other variation thereof are defined as “near,” as understood by those skilled in the art, and this term is defined as “within 10% in one non-limiting embodiment, within 5% in another embodiment, within 1% in another embodiment, and within 0.5% in another embodiment. The term “combined,” as used herein, is defined as “connected,” but not necessarily directly or mechanically. A device or structure “configured” in a certain way is configured at least in that way, but may also be configured in ways not listed.

[0097] To enable readers to quickly grasp the nature of the technical disclosure, an abstract of this disclosure is provided. This abstract is presented with the understanding that it is not used to interpret or limit the claims or their meaning. In addition, it can be understood that in the aforementioned “Modes for Carrying Out the Invention,” various features are grouped together in various embodiments in order to simplify the disclosure. The method of this disclosure should not be interpreted as reflecting an intention that the claimed embodiments require more features than those explicitly enumerated in each claim. Rather, as reflected in the following claims, the subject matter of the invention may consist of fewer features than all the features of a single disclosed embodiment combined. Accordingly, the following claims are incorporated herein into the “Modes for Carrying Out the Invention,” and each claim asserts itself as separately claimed subject matter.

Claims

1. A method for operating a machine vision system, wherein the machine vision system includes a computing device for running an application and a fixed imaging device communicatively coupled to the computing device, and the method is (a) Acquiring a first image in the field of view (FOV) via the fixed imaging device, (b) Analyzing at least a portion of the first image via the application in order to detect visual features in the first image, (c) Determining the position of the visual feature in the first image via the application, (d) Determining a first region of interest (ROI) in the first image based on the position of the visual feature via the application, (e) Acquiring a second image via the fixed imaging device, (f) Analyzing at least a portion of the second image via the application in order to detect the visual features in the second image, (g) Determining the position of the visual feature in the second image via the application, (h) Determining a second ROI in the second image based on the position of the visual feature via the application, (i) Ranking the first and second ROIs, (j) Acquiring a third image via the fixed imaging device, (k) To detect the visual features, to analyze the third ROI in the third image via the application, wherein the third ROI in the third image is analyzed based on the ranking, (l) A method comprising transmitting data associated with the visual feature in the third image to a host processor in response to the detection of the visual feature in the third image.

2. Analyzing the third ROI as described above is Setting the third ROI to be a higher ranked ROI than the first ROI and the second ROI, The method according to claim 1, further comprising updating the third ROI to a lower-ranked ROI among the first ROI and the second ROI if it is determined that the visual feature is not within the set third ROI.

3. Analyzing the third ROI as described above is The method of claim 2, further comprising setting the third ROI to be the FOV of the third image if it is determined that the visual feature is not within the updated third ROI.

4. Analyzing the third ROI as described above is The third ROI set in the FOV of the third image is analyzed to detect the visual features, The application is used to determine the position of the visual features in the third image, Through the application, a new third ROI in the third image is determined based on the position of the visual feature, The application determines that the new third ROI is within the predetermined tolerance range of the first ROI, The method of claim 3, further comprising incrementing the weighting factor of the first ROI via the application in response to the determination that the new third ROI is within a predetermined tolerance range of the first ROI.

5. After the analysis of the third ROI of the third image, images are acquired iteratively, and in each iteration, If it is determined that the aforementioned visual feature is within the first ROI, the weighting coefficient of the first ROI is incremented. If it is determined that the aforementioned visual feature is within the second ROI, the weighting coefficient of the second ROI is incremented. The method according to claim 1, further comprising re-ranking the first and second ROIs based on the weighting coefficients.

6. The process of analyzing at least a portion of the first image to detect the visual features includes determining the bounding box of the visual features via the application, The method according to claim 1, wherein determining the first ROI in the first image includes applying a scaling factor to the bounding box.

7. In response to detecting the visual features in the first image, data associated with the visual features in the first image is transmitted to the host processor. The method according to claim 1, further comprising transmitting data associated with the visual feature in the second image to the host processor in response to the detection of the visual feature in the second image.

8. The ranking of the first and second ROIs is to present the representation of the first and second ROIs to the user, The user selects either the first ROI or the second ROI from the user, The method according to claim 1, further comprising setting the ranks of the first ROI and the second ROI based on the user selection.

9. A method for operating a machine vision system, wherein the machine vision system includes a computing device for running an application and a fixed imaging device communicatively coupled to the computing device, and the method is The process involves acquiring a first set of multiple images in the field of view (FOV) via the aforementioned fixed imaging device, Identifying multiple regions of interest (ROIs) within the FOV from the first multiple images via the application, wherein each of the multiple ROIs is associated with a visual feature, and each of the multiple ROIs is ranked based on the frequency of recurrence of the visual feature within a predetermined proximity range to each of the multiple ROIs in the multiple images. The second image is acquired via the aforementioned fixed imaging device, Through the application, candidate positions of the visual features in the second image are continuously identified based on the rank of each of the plurality of ROIs until the visual features are identified at the candidate positions. A method comprising providing a host processor with data relating to the visual features from the second image.

10. The method according to claim 9, wherein the plurality of regions of interest are limited to n or fewer ROIs.

11. The method according to claim 10, wherein n is specified by the user.

12. Receiving input from the user indicating that the ROI function should be disabled, The third image is acquired via the aforementioned fixed imaging device, The method according to claim 9, further comprising setting the ROI in the third image to be the FOV of the third image in response to the reception of the input.

13. Receiving input from the user indicating that the ROI function should be disabled, The third image is acquired via the aforementioned fixed imaging device, The ROI in the third image is set based on the highest-ranked ROI among the aforementioned multiple ROIs, It is determined that the aforementioned visual features are not within the ROI in the third image, The method of claim 9, further comprising receiving the input and determining that the visual feature is not within the ROI in the third image, and setting the ROI in the third image to be the FOV of the third image.

14. Receiving input from the user indicating that they wish to disable ranking updates, The method according to claim 9, further comprising not updating any recurrence frequency of any of the plurality of ROIs when identifying the visual feature in the second image in response to the input.

15. The method according to claim 9, wherein the visual feature is identified at the candidate position based on the fact that the entirety of the visual feature lies within the candidate position.

16. The method according to claim 9, wherein the visual feature is identified at the candidate position based on the fact that 50% or more of the visual feature is located within the candidate position.

17. Each of the aforementioned multiple ROIs has a rank, The frequency of recurrence of the visual features within a predetermined proximity range for each of the plurality of ROIs, The method according to claim 9, further comprising creating a ranking table that includes the coordinates of each of the plurality of ROIs.

18. The method according to claim 9, wherein the visual feature is a barcode.

19. The method according to claim 9, wherein the aforementioned visual feature is a crack on the surface of the structure.

Images