Method, system, computer program, and computer-readable storage medium for removing foreign matter

Abstract

To provide techniques for removing an anomaly from a collection of material.SOLUTION: A processor may receive an image of a collection of material having a plurality of objects. The processor may identify an anomaly from the plurality of objects. The processor may generate a bounding box for the anomaly. The processor may generate one or more picking points on the anomaly. The one or more picking points may be configured on at least one balance points of the anomaly. The processor may remove the anomaly from the collection of material via the one or more picking points.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure generally relates to the field of material sorting, and more specifically, to removing irregular materials from a population of materials.

Background Art

[0002] In many industries, it is necessary to sort raw materials during the manufacture of products. Although there are various sorting methods, such methods often cannot remove all of the irregularly shaped materials that need to be removed. If all irregularly shaped materials cannot be removed, the value of the final product may decrease, and ultimately, it may lead to equipment failure when the irregularly shaped materials get caught in the equipment.

Summary of the Invention

Problems to be Solved by the Invention

[0003] An object of the present invention is to provide a technique for removing foreign matter from a population of materials.

Means for Solving the Problems

[0004] Embodiments of the present disclosure include a method, a computer program product, and a system for removing foreign matter from a population of materials. A processor can receive an image of a population of materials having a plurality of objects. The processor can identify foreign matter from the plurality of objects. The processor can generate a bounding box for the foreign matter. The processor can generate one or more picking points on the foreign matter. The one or more picking points can be configured on at least one balance point of the foreign matter. The processor can remove the foreign matter from the population of materials based on the one or more picking points.

[0005] The above summary is not intended to describe every illustrated embodiment or every implementation of the present disclosure.

Brief Description of the Drawings

[0006] The drawings included in this disclosure are incorporated herein and constitute a part of this specification. These drawings illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure. The drawings are merely illustrative of specific embodiments and do not limit the disclosure.

[0007]

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3A

Figure 3B

Figure 4

[0008] The embodiments described herein are amenable to various modifications and alternative forms, and the specific details thereof will be shown by way of example in the drawings and will be described in detail. However, it should be understood that the specific embodiments described should not be taken in a limiting sense. On the contrary, it is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure.

Modes for Carrying Out the Invention

[0009] Aspects of the present disclosure generally relate to the field of removing foreign matter from materials, and more particularly to identifying picking points associated with foreign matter to enable proper removal. The present disclosure is not necessarily limited to such applications, but various aspects of the present disclosure can be understood through discussion of various examples using this context.

[0010] In conventional removal systems, robotic means (such as robotic arms) are often used to remove impurities from various types of materials (e.g., tea leaves and crude drug components). In these conventional systems, impurities can be visually identified, and picking points can be identified using a combination of bounding boxes and segmentation calculations. In these methods, picking points may be generated that are not located on the impurities (e.g., anomalies or unwanted materials), or the picking points are often located on the distal portions of the impurities. When a picking point is generated that is not located on the impurity, that picking point is a failure. Depending on the situation, such picking point failures are due to the unique shape of the impurity or anomaly and the inability to correctly estimate where the picking point will be successful during removal of the impurity. If the impurity / anomaly is not removed, the unwanted material may result in an unwanted consumer product having one or more impurities / anomalies. Therefore, there is a need for a method to more accurately define successful picking points.

[0011] In an embodiment, the picking point system can be configured to identify whether an object within a material is an impurity or an anomaly. When an impurity / anomaly is identified, the picking point system can detect the bounding box of the object (e.g., the impurity / anomaly) and determine the overall size of the bounding box. In embodiments where the size of the bounding box is smaller than a threshold area, the picking point system can generate a picking point (e.g., at the center of the bounding box) and configure robotic means to remove the impurity / anomaly. In embodiments where the size of the bounding box exceeds the threshold size, the picking point system can be configured to define the segmentation of the impurity / anomaly (e.g., using unsupervised segmentation).

[0012] In embodiments where the segmentation of the impurity / anomaly is represented by points, the picking point system can collect the obtained points into one or more connected regions (e.g., using a statistical method such as DBSCAN). In embodiments where the segmentation of the impurity / anomaly is represented by another shape (e.g., a polygon), each drawn shape can be configured into one connected region. In these embodiments, the picking point system can be configured to calculate the number of picking points using machine learning principles (e.g., k - centroid clustering).

[0013] The embodiments discussed in this specification can be used in various implementations. For example, one exemplary embodiment can include removing impurities (e.g., foreign substances) when processing tea leaves for consumption. In many places, tea trees are cultivated in fields and tea leaves are harvested from the tea trees. Despite various efforts (e.g., filtering), often, impurities (e.g., any unwanted plant substances other than tea leaves) or foreign objects (e.g., stones and fibers) or both may be collected together with the tea leaves. Therefore, in order to enable further processing of the tea leaves, it is necessary to remove the impurities or foreign objects or both contained in the tea leaves before processing. In many cases, if such impurities or foreign objects (e.g., abnormal objects) or both are not removed from the collected material, such abnormal objects may cause poor products or production delays if the impurities or abnormal objects affect the processing equipment. For example, if stones or pieces of wire are mixed in the tea leaves and such substances are not removed in a timely manner, as a result, the stones or wire may be damaged or clog the tea processing equipment. Therefore, a method for accurately detecting such abnormal objects (e.g., impurities and foreign objects) is needed.

[0014] Turning now to the figures, noting that like reference numerals are used in the accompanying drawings to indicate like parts, FIG. 1A shows an exemplary embodiment of a picking point system 100 according to an embodiment of the present disclosure, and a population 102 of materials composed of abnormal objects 104A, 104B, and one or more materials 106. The picking point system 100 can be used to generate one or more picking points and remove one or more abnormal objects (e.g., abnormal object 104A and abnormal object 104B) from the population 102 of materials. FIG. 1A provides only an illustration of one implementation and does not imply any limitation with respect to the environments in which different embodiments can be implemented. Many changes to the depicted environment can be made by those skilled in the art without departing from the scope of the invention as recited in the claims.

[0015] In an embodiment, the picking point system 100 can be configured to remove one or more foreign objects (e.g., foreign object 104A and foreign object 104B) from the material population 102 by generating one or more picking points. In an embodiment, a picking point can be defined as a specific, defined location where robotic means, such as a robotic arm, can pick (lift), grasp, swat away, etc., one or more foreign objects (e.g., foreign object 104A and foreign object 104B). Thus, the picking point system 100 can be configured to generate one or more picking points located on the foreign objects. The material population 102 can include any type or amount of material, such as organic materials (e.g., tea leaves and other plant materials), inorganic materials (e.g., metal components), or any combination thereof. Further, the material population 102 can have two or more material types.

[0016] For example, the material population 102 can include a combination of components such as tea leaves and various herbal medicines that can be included in a specific final product. Such components may be of various sizes and shapes within the material population 102. The material population 102 may also include one or more foreign objects (e.g., foreign object 104A and foreign object 104B).

[0017] As discussed herein, an anomaly can refer to any object that is identified for removal by the picking point system 100. In some embodiments, the anomaly may be a desired object that has been collected with other materials (e.g., refuse materials). For example, the population of materials 102 may be composed of one type of material but of different sizes. In this example, the picking point system 100 can be configured to identify a material of a particular size as an anomaly and pick or remove that material from the population of materials 102 for use elsewhere. With such embodiments, the picking point system 100 can sort through the population of materials 102 and identify objects of interest or anomalies without removing each piece of material within the population of materials 102. In other embodiments, the anomalies (e.g., anomaly 104A and anomaly 104B) can be taken to refer to undesired objects (e.g., impurities or foreign objects or both) that are to be removed from the population of materials 102.

[0018] Returning to the exemplary embodiment shown in FIG. 1A, the population of materials 102 can include a plurality of objects including, but not limited to, foreign objects 104A and 104B, and one or more materials 106. The one or more materials 106 can be of similar size or of diverse sizes, and can include any number of different materials. The foreign objects 104A and 104B contemplated herein can have any kind of shape or configuration. As shown in FIG. 1A, the foreign object 104A can be an object that does not have a uniform shape. The foreign object 104A can be any kind of material having various properties. For example, the foreign object 104A can be malleable, brittle, or fibrous. Using a conventional picking point system can result in the generation of a failed picking point and the foreign object 104A may not be removed from the population of materials 102. The foreign object 104B has a more uniform shape and represents a foreign object having a generally small area. The material 106 is depicted as a leaf-shaped material in FIG. 1A, but such a depiction is only for providing a clear distinction between the material 106 and the foreign objects 104A and 104B, and should thus not be regarded as limiting.

[0019] FIG. 1B shows an exemplary embodiment of a picking point system 100 according to an embodiment of the present disclosure. The picking point system 100 can generate / identify one or more picking points and remove one or more foreign objects (e.g., foreign object 104A, and foreign object 104B) from the population of materials 102. In an embodiment, the picking point system 100 can be configured to capture or receive, or capture and receive, one or more images 108 of the population of materials 102 as referenced in FIG. 1A. FIG. 1B provides an illustration of only one implementation and does not imply any limitations with respect to the environment in which different embodiments can be implemented.

[0020] In an embodiment, one or more images 108 can be generated using any type of imaging device configured to capture information about the population of materials 102. In some embodiments, a conventional image recognition camera can be used, but in other embodiments, other imaging techniques can be used or combined with a conventional image recognition method. For example, in some embodiments, the population of materials 102 can be a thick layer in which a material and an anomaly are mixed with low or almost no opacity. In such embodiments, in one or more images 108 captured using a conventional image recognition camera, the picking point system 100 may not be able to determine the parameters (e.g., the bounding boxes or segmentations or both of the anomalies 104A, 104B, and the material 106) necessary to be associated with the plurality of objects. As a result, a secondary imaging method can be used. One such secondary imaging method can include, but is not limited to, infrared imaging. In an embodiment, especially when there is a layer of material overlapping the population of materials 102, infrared imaging can enable the picking point system 100 to distinguish between the intricate layers of material and generate one or more images 108 that can define the parameters of one or more anomalies (e.g., anomalies 104A and 104B).

[0021] In an embodiment, the picking point system 100 can be configured to use one or more images 108 of the population of materials 102 to identify whether one or more anomalies (e.g., anomaly 104A and anomaly 104B) are present within the population of materials 102. In an embodiment, the picking point system 100 is configured to use a segmentation analysis method on one or more images 108 to identify whether one or more anomalies are present within the population of materials 102. In these embodiments, the picking point system 100 can be configured to perform patch and feature extraction of one or more images 108 of the population of materials 102.

[0022] In an embodiment, using the extracted features, the picking point system 100 can be configured to perform various statistical analyses (e.g., principal component analysis (PCA) and k-means clustering). In an embodiment, the picking point system 100 can determine whether the resulting information represents normal objects / materials (e.g., material 106) and / or abnormal objects / materials (e.g., anomalies 104A-104B) within one or more images 108 of the population of materials 102. In an embodiment, such determination can be performed using a machine learning function that enables comparison of the obtained information with a corpus or dictionary. In these embodiments, the corpus or dictionary can include a historical repository having data associated with what was previously identified as an anomaly 104 or material 106, or a database of provided data (e.g., data provided by an administrator) that can be used to identify aspects that distinguish between material 106 and anomalies 104A and / or 104B, or both.

[0023] FIG. 1C shows an exemplary embodiment of a picking point system 100 configured to generate bounding boxes 110A and 110B for each of one or more anomalies (e.g., anomaly 104A and anomaly 104B) according to an embodiment of the present disclosure.

[0024] In an embodiment, the picking point system 100 can generate a bounding box for each identified anomaly and be configured to determine a threshold area. As shown in FIG. 1B, the picking point system 100 can generate a bounding box 110A around the anomaly 104A and generate a bounding box 110B around the anomaly 104B. In some embodiments, the bounding boxes 110A and 110B can be configured to enclose the entirety of each anomaly 104A and 104B. When comparing the total area associated with the anomaly 104A and the total area associated with the anomaly 104B, the total areas may not be significantly different. However, when comparing the area associated with the bounding box 110A and the area associated with the bounding box 110B, there are significant differences due to the composition and spread of the anomaly 104A.

[0025] In an embodiment, when a bounding box for each identified anomaly (e.g., 104A and 104B) is generated, the picking point system 100 can compare the area of the bounding box associated with each identified anomaly with a threshold area. In some embodiments, the threshold area can be configured to vary according to the type of anomaly, while in other embodiments, one threshold area can be used for each identified anomaly. In an embodiment, if the area of the bounding box of a particular anomaly is less than the area of the threshold area, the picking point system 100 can be configured to generate a picking point at the center of the bounding box of that particular anomaly.

[0026] In an embodiment, the threshold area can depend on various factors including, but not limited to, the following factors: (i) the type of material within the material population 102 and previously identified anomalies (e.g., the threshold area can be increased / decreased according to the average size of the objects / materials found within the material population 102), (ii) the likelihood of errors associated with the removal means (e.g., when determining the threshold area, the percentage error associated with the robotic means can be considered), (iii) defining a specific amount of area that serves as an indicator that the anomaly is located at the center of the bounding box, or (iv) any combination thereof.

[0027] Furthermore, in some embodiments, the threshold area can be set over time based on the principles of machine learning, and is sized to ensure that a picking point at the center of the bounding box is successful when the size of the bounding box is smaller than the threshold area. In embodiments where one threshold area is used for each identified anomaly, the threshold area is often a small area. In these embodiments, by determining that the bounding box of the anomaly has an area smaller than the threshold area, the picking point system 100 can generate a picking point at the center of the bounding box, and it is guaranteed that the picking point is on the anomaly.

[0028] Returning to FIG. 1C, the picking point system 100 compares the area of the bounding box 110B of the foreign object 104B with a threshold area and determines that the bounding box 110B has an area smaller than the defined threshold area. Accordingly, the picking point system 100 can generate a picking point 112 at the center of the foreign object 104B. In an embodiment, the picking point generated as a result of the area of the bounding box being below the threshold area is not only located on the foreign object, but is often generated at the barycenter or balancing point of the foreign object. By generating the picking point 112 at the barycenter or balancing point, it can be ensured that the weight of the foreign object 104B is evenly distributed during removal. By evenly or substantially evenly distributing the weight of the foreign object 104B, it can be ensured that most or all of the foreign object 104 is removed from the population of materials 102. If the picking point is generated at a position other than the balancing point of the foreign object, such positioning may cause stress on the structure of the foreign object, and as a result, the foreign object may be damaged. If the foreign object is damaged during the removal process, the removal becomes more difficult and the efficiency associated with the picking point system 100 decreases.

[0029] In an embodiment, if the area of the bounding box of a particular foreign object is larger than the area of the threshold area, the picking point system 100 can be configured to generate one or more picking points on the foreign object (as depicted by the small dashed box within 110A in FIG. 1C). In these embodiments, the picking point system 100 can generate one or more picking points on the foreign object by calculating one or more segmentation analyses of one or more of the images 108. In some embodiments, the segmentation analysis performed during foreign object detection can be reused to generate picking points on the foreign object, while in other embodiments, a secondary segmentation analysis can be performed to generate picking points.

[0030] In an embodiment, the segmentation analysis can provide information in the form of associated segments that identify the location parameters of the anomalies. In an embodiment, the segmentation analysis can include that one or more segments of the anomalies are composed of points. In these embodiments, the picking point system 100 can observe the points and determine whether there are clusters of points. The picking point system 100 can be configured to classify / identify the clusters of points as connected regions.

[0031] In an embodiment, the segmentation analysis can include one or more segments of the anomalies that are composed of one or more polygons. In embodiments where one or more segments include one or more polygons, the picking point system 100 can be configured to classify / identify that each of the one or more polygons is a connected region (such as each of the small dashed boxes within / of 110A). In some embodiments, each polygon is a connected region, but in other embodiments, two or more polygons can be identified as one connected region. In an embodiment, the segmentation analysis can be composed of either points or polygons, and in some embodiments, it can be composed of both points and polygons.

[0032] As a result of the area of the bounding box 110A of the anomaly 104A being larger than the threshold area, the picking point system 100 can be configured to perform segmentation analysis on the anomaly 104A and determine where on the anomaly 104A one or more connected regions should be identified. In the exemplary embodiment presented in FIG. 1C, the connected regions associated with the anomaly 104A are represented by various small dashed boxes located on the anomaly 104A. In an embodiment, the picking point system 100 can generate one or more picking points on the identified anomaly in each of the identified connected regions. In some embodiments, only one picking point is generated, but in other embodiments, the picking point system 100 can calculate any number of picking points necessary to remove the anomaly. In an embodiment, one or more picking points can be generated to ensure that the structural integrity of the anomaly 104A is maintained through the removal process. For example, the picking point system 100 can ensure that the weight of the anomaly 104A is evenly distributed by considering various centers of balance of the anomaly 104A when generating the picking points.

[0033] In an embodiment, the picking point system 100 can be configured to remove anomalies from the population of materials 102 via one or more generated picking points found from one or more images 108. The picking point system 100 can configure various types of removal means for the removal. In some embodiments, the removal means (e.g., any type of robotic means) can be configured to simultaneously pick or grip each of the one or more picking points generated for a particular anomaly. The removal means can include any type of mechanical device capable of removing one or more anomalies, but in many embodiments, the anomalies are removed via robotic means (e.g., robotic arms) configured by the picking point system 100 to remove one or more anomalies at the generated picking points.

[0034] Referring now to FIG. 2, it is a flowchart showing an exemplary method 200 for generating picking points for anomalies according to an embodiment of the present disclosure. In some embodiments, the method 200 can be used to identify anomalies and generate picking points that enable accurate and complete removal of the anomalies from the population of materials.

[0035] In some embodiments, method 200 begins at operation 202 where a processor receives an image of a population of materials having a plurality of objects / materials. Method 200 proceeds to operation 204. At operation 204, the processor identifies anomalies from the plurality of objects. Method 200 proceeds to operation 206. At operation 206, the processor generates a bounding box for the anomalies. Method 200 proceeds to operation 208. At operation 208, the processor generates one or more picking points on the anomalies. Method 200 proceeds to operation 210. At operation 210, the processor removes the anomalies from the population of materials via one or more picking points on the anomalies. In some embodiments, as shown in FIG. 2, after operation 210, method 200 can end.

[0036] In some embodiments described below, there are one or more operations of method 200 that are not depicted for the sake of brevity, and the operations / steps are further executed by the processor.

[0037] Thus, in an embodiment, the processor can compare the area of the bounding box with a threshold area. In an embodiment, in response to comparing the area of the bounding box with the threshold area, the processor may determine that the area of the bounding box is less than the threshold area. In these embodiments, the processor can generate one or more picking points at the center of the bounding box.

[0038] In an embodiment, in response to comparing the area of the bounding box with a threshold value, the processor may determine that the area of the bounding box exceeds the threshold area. In an embodiment, the processor can calculate one or more segmentations of the foreign object and analyze the one or more segmentations. In an embodiment, in response to analyzing the one or more segmentations, the processor can identify that the one or more segmentations are composed of points. In these embodiments, the processor can also determine whether the cluster of points is a connected region.

[0039] In an embodiment, in response to analyzing the one or more segmentations, the processor can identify that the one or more segmentations are composed of one or more polygons and can determine that each of the one or more polygons is a connected region. In an embodiment, while generating one or more picking points on the foreign object, the processor can identify one or more connected regions on the foreign object and calculate the number of the one or more picking points for the one or more connected regions. In an embodiment, while removing the foreign object from the population of materials via the one or more picking points, the robotic means can pick or grip simultaneously at the one or more picking points.

[0040] Although the present disclosure includes a detailed description of cloud computing, it should be understood that the implementation of the teachings described herein is not limited to a cloud computing environment. Rather, the embodiments of the present disclosure can be implemented with any other type of computing environment that is currently known or will be developed later.

[0041] Cloud computing is a service delivery model that enables convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0042] The characteristics are as follows.

[0043] On-demand self-service: Cloud consumers can, as needed, automatically and unilaterally provision computing capabilities such as server time and network storage without the need for human interaction with the service provider.

[0044] Broad network access: The capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0045] Resource pooling: The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to demand. Consumers are generally location-independent in that they have no control or knowledge of the exact location of the provided resources, although they may be able to specify a higher level of abstraction (e.g., country, state, or data center).

[0046] Rapid elasticity: The function can provision and quickly scale out rapidly and elastically, and in some cases automatically, and can quickly release and scale in quickly. For consumers, the functions available for provisioning often appear to be unlimited and can be purchased in any quantity at any time.

[0047] Measured services: The cloud system automatically controls and optimizes resource usage by using a metering function at some level of abstraction suitable for the type of service (e.g., storage, processing, bandwidth, and active user accounts). It can monitor, control, and report resource usage to provide transparency to both the provider and consumer of the services being utilized.

[0048] The service model is as follows.

[0049] Software as a Service (SaaS): The function provided to the consumer is to use the provider's applications running on the cloud infrastructure. These applications are accessible from various client devices through a thin-client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, server, operating system, storage, or individual application functions, with the exception of limited user-specific application configuration settings.

[0050] Platform as a Service (PaaS): The function provided to the consumer is to deploy the applications generated or obtained by the consumer, which are generated using the programming languages and tools supported by the provider, onto the cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure such as the network, server, operating system, or storage, but has control over the deployed applications and, in some cases, the configuration of the application hosting environment.

[0051] Infrastructure as a Service (IaaS): The function provided to the consumer is to provision processing, storage, network, and other basic computing resources on which the consumer can deploy and run any software that may include an operating system and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating system, storage, the deployed applications, and, in some cases, limited control over the selection of network components (e.g., the host firewall).

[0052] The deployment model is as follows.

[0053] Private cloud: The cloud infrastructure is operated solely for a certain organization. This cloud infrastructure can be managed by that organization or a third party and can exist on-premises or off-premises.

[0054] Community Cloud: The cloud infrastructure is shared by several organizations and supports a specific community with common concerns (e.g., mission, security requirements, policies, and compliance considerations). This cloud infrastructure can be managed by those organizations or a third party and can exist on-premises or off-premises.

[0055] Public Cloud: The cloud infrastructure is available to the general public or a large industry group and is owned by an organization that sells cloud services.

[0056] Hybrid Cloud: The cloud infrastructure remains a distinct entity but is a hybrid of two or more clouds (private, community, or public) connected to each other by standardized or proprietary technologies (e.g., cloud bursting for load distribution between clouds) that enable data and application portability.

[0057] Cloud computing environments are service-oriented, focusing on statelessness, low coupling, modularity, and semantic interoperability. The core of cloud computing is the infrastructure that includes a network of interconnected nodes.

[0058] Referring now to FIG. 3A, an exemplary cloud computing environment 310 is depicted. As shown, cloud computing environment 310 includes one or more cloud computing nodes 300 that can communicate with local computing devices used by cloud consumers such as, for example, a personal digital assistant (PDA) or cellular telephone 300A, desktop computer 300B, laptop computer 300C, or in-vehicle computer system 300N or combinations thereof. Nodes 300 can communicate with one another. These nodes can be physically or virtually grouped (not shown) to form one or more networks such as the private cloud, community cloud, public cloud, or hybrid cloud as described above, or combinations thereof. This enables cloud computing environment 310 to provide infrastructure as a service, platform as a service, or software as a service or combinations thereof where cloud consumers do not need to maintain resources on local computing devices. It is intended that the types of computing devices 300A - N shown in FIG. 3A are merely exemplary, and that computing nodes 300 and cloud computing environment 310 can communicate with any type of computerized device on or across any type of network (e.g., using a web browser) or both.

[0059] Referring now to FIG. 3B, a set of functional abstractions provided by cloud computing environment 310 (FIG. 3A) is shown. It should be understood in advance that the components, layers, and functions shown in FIG. 3B are merely exemplary and that embodiments of the present disclosure are not limited thereto. As shown, the following layers and corresponding functions are provided.

[0060] The hardware and software layer 315 includes hardware and software components. Examples of hardware components include mainframe 302, RISC (Reduced Instruction Set Computer) architecture-based server 304, server 306, blade server 308, storage device 311, and network and network components 312. In some embodiments, the software components include network application server software 314 and database software 316.

[0061] The virtualization layer 320 provides an abstraction layer that can provide the following examples of virtual entities, namely, virtual server 322, virtual storage 324, virtual network 326 including a virtual private network, virtual applications and operating systems 328, and virtual clients 330.

[0062] In one example, the management layer 340 can provide the functions described below. Resource provisioning 342 provides for the dynamic procurement of computing resources and other resources utilized to execute tasks within a cloud computing environment. Metering and pricing 344 provides for cost tracking when resources are utilized within a cloud computing environment and for billing or charging for the consumption of these resources. In one example, these resources can include application software licenses. Security provides for the verification of identification information about cloud consumers and tasks and for the protection of data and other resources. The user portal 346 provides access to the cloud computing environment for consumers and system administrators. Service level management 348 provides for the allocation and management of cloud computing resources such that the required service levels are met. Planning and fulfillment of service level agreements (SLAs) 350 provides for the pre-placement and procurement of cloud computing resources whose future needs are predicted in accordance with the SLA.

[0063] The workload layer 360 provides examples of functions that can utilize a cloud computing environment. Examples of workloads and functions that can be provided from this layer include mapping and navigation 362, software development and life cycle management 364, virtual classroom education delivery 366, data analysis processing 368, transaction processing 370, and pick point generation 372.

[0064] FIG. 4 is a high-level block diagram of an exemplary computer system 401 that can be used to implement one or more of the methods, tools, and modules described herein, and any associated functionality, according to an embodiment of the present invention (e.g., using one or more processor circuits or computer processors of a computer). In some embodiments, the main components of computer system 401 can include one or more processors 402, a memory subsystem 404, a terminal interface 412, a storage interface 416, an I / O (input / output) device interface 414, and a network interface 418, all of which can be communicatively coupled directly or indirectly for component-to-component communication via a memory bus 403, an I / O bus 408, and an I / O bus interface unit 410.

[0065] Computer system 401 can include one or more general-purpose programmable central processing units (CPUs) 402A, 402B, 402C, and 402D, which are generally referred to herein as CPU 402. In some embodiments, computer system 401 can include multiple processors typical of relatively large-scale systems, but in other embodiments, computer system 401 can alternatively be a single CPU system. Each CPU 402 can execute instructions stored in memory subsystem 404 and can include one or more levels of on-board cache.

[0066] System memory 404 can include a computer system readable medium in the form of volatile memory, such as random access memory (RAM) 422 or cache memory 424. Computer system 401 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 426 can be provided for reading from and writing to a non-removable non-volatile magnetic medium, such as a “hard drive”. Although not shown, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a “floppy disk”), or an optical disk drive for reading from and writing to a removable non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media can be provided. Further, memory 404 can include flash memory, such as a flash memory stick drive or flash drive. The memory devices can be connected to memory bus 403 by one or more data media interfaces. Memory 404 can include at least one program product having a set of program modules (e.g., at least one) configured to execute the functions of various embodiments.

[0067] One or more programs / utilities 428, each having a set of at least one program module 430, can be stored in memory 404. Programs / utilities 428 can include a hypervisor (also called a virtual machine monitor), one or more operating / systems, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or some combination thereof, can include an implementation of a network environment. Program 428 or program module 430 or both generally execute the functions or methodologies of various embodiments.

[0068] Memory bus 403 is shown in FIG. 4 as a single bus structure that provides a direct communication path between CPU 402, memory subsystem 404, and I / O bus interface 410. However, in some embodiments, memory bus 403 can include multiple different buses or communication paths, which can be arranged in various forms, such as hierarchical point-to-point links, star or web configurations, multi-layer buses, parallel and redundant paths, or any other suitable type of configuration. Further, although I / O bus interface 410 and I / O bus 408 are shown as single respective units, computer system 401 can include, in some embodiments, multiple I / O bus interface units 410, multiple I / O buses 408, or both. Further, multiple I / O interface units are shown, which separate I / O bus 408 from various communication paths extending to various I / O devices. However, in other embodiments, some or all of the I / O devices can be directly connected to one or more system I / O buses.

[0069] In some embodiments, computer system 401 can be a multi-user mainframe computer system, a single-user system, or a server computer or similar device that has little or no direct user interface but receives requests from other computer systems (clients). Further, in some embodiments, computer system 401 can be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a telephone, a smartphone, a network switch or router, or any other suitable type of electronic device.

[0070] Note that FIG. 4 is intended to depict representative main components of an exemplary computer system 401. However, in some embodiments, individual components may be of higher or lower complexity than those shown in FIG. 4, there may be components other than or in addition to those shown in FIG. 4, and the number, type, and configuration of such components may vary.

[0071] As discussed in more detail herein, some or all of the operations of some embodiments of the methods described herein may be performed in an alternative order or not at all, and further, it is contemplated that multiple operations may be performed simultaneously or as part of a larger process.

[0072] The present invention can be a system, method, or computer program product or a combination thereof at any possible technical detail level of integration. The computer program product can include one or more computer-readable storage media having computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0073] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile discs (DVDs), memory sticks, floppy disks, punch cards, or mechanical coding devices such as raised structures in grooves in which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed as being a transient signal per se, such as a radio wave, or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0074] The computer-readable program instructions described herein can be downloaded to each computing / processing device from a computer-readable storage medium or can be downloaded from an external computer or an external storage device via a network, such as, for example, the Internet, a local area network, a wide area network, or a wireless network or a combination thereof. The network can include a copper transmission cable, an optical transmission fiber, wireless transmission, a router, a firewall, a switch, a gateway computer, or an edge server, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0075] Computer-readable program instructions for carrying out the operations of the present invention may be any combination of one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuits, or source code or object code written in any combination of object-oriented programming languages such as Smalltalk, C++, and procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly as a stand-alone software package on the user's computer, partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or an external computer connection may be made (e.g., through the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to customize the electronic circuit to implement aspects of the present invention.

[0076] Aspects of the invention will be described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0077] These computer readable program instructions may be provided to a processor of a computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram(s). These computer program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, or other device to function in a particular manner, such that the medium in which the instructions are stored comprises an article of manufacture including instructions which implement the function / act specified in one or more blocks of the flowchart and / or block diagram(s).

[0078] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram(s).

[0079] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, segment, or portion of one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions shown within the block may be performed in an order different from that shown in the figures. For example, two blocks shown in succession may, depending on the functions involved, actually be performed as one step, executed simultaneously, substantially simultaneously, in a partially or fully overlapping manner in time, or these blocks may sometimes be executed in reverse order. It should be noted that each block of the block diagram or flowchart diagram, or both, and combinations of blocks in the block diagram or flowchart diagram, or both, can also be implemented by a dedicated hardware-based system that executes the specified function or operation, or by a combination of dedicated hardware and computer instructions.

[0080] The descriptions of the various embodiments of the present disclosure are presented for purposes of illustration, but are not intended to be exhaustive or to limit the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are chosen to best explain the principles of the embodiments, the practical application, or a technical improvement over technologies found in the marketplace, or to enable one of ordinary skill in the art to understand the embodiments disclosed herein.

[0081] Although the present invention has been described based on specific embodiments, it is expected that changes and modifications will be apparent to those of ordinary skill in the art. Accordingly, the following claims are intended to be construed to cover all such changes and modifications that fall within the true spirit and scope of the present disclosure.

Claims

1. A method for removing foreign objects within a population of materials, comprising: receiving, by a processor, an image of the population of materials having a plurality of objects; identifying the foreign objects from the plurality of objects; generating a bounding box for the foreign objects; comparing an area of the bounding box with a threshold area; generating one or more picking points on the foreign objects based on a comparison result between the area of the bounding box and the threshold area, wherein the one or more picking points are configured on at least one balance point of the foreign objects; removing the foreign objects from the population of materials based on the one or more picking points. A method comprising the above steps.

2. In response to comparing the area of the bounding box with the threshold area, determining that the area of the bounding box is less than the threshold area; generating the one or more picking points at the center of the bounding box. The method according to claim 1, further comprising the above steps.

3. In response to comparing the area of the bounding box with the threshold area, determining that the area of the bounding box is greater than the threshold area. The method according to claim 1, further comprising the above step.

4. Generating the one or more picking points on the foreign objects further comprises: calculating one or more segmentations of the foreign objects; analyzing the one or more segmentations. The method according to claim 3, further comprising the above steps.

5. In response to analyzing the one or more segmentations, identifying that the one or more segmentations are composed of points; determining that a cluster of points is a connected region. The method according to claim 4, further comprising the above steps.

6. In response to analyzing the one or more segmentations, identifying that the one or more segmentations are composed of one or more polygons; determining that each of the one or more polygons is a connected region. The method according to claim 4, further comprising the above steps.

7. Generating the one or more picking points on the foreign objects further comprises: identifying one or more connected regions on the foreign objects. Calculating the number of the one or more picking points for the one or more connection regions The method according to claim 1, further comprising

8. Removing the foreign matter from the population of the material based on the one or more picking points Collecting the foreign matter via robotic means at the one or more picking points The method according to claim 7, further comprising

9. A system for removing foreign matter in a population of material, comprising A memory, and A processor communicating with the memory, the processor being configured to Receive an image of the population of the material having a plurality of objects Identify the foreign matter from the plurality of objects Generate a bounding box for the foreign matter Compare the area of the bounding box with a threshold area Generating one or more picking points on the foreign matter based on a comparison result between the area of the bounding box and the threshold area, wherein the one or more picking points are configured on at least one balance point of the foreign matter Removing the foreign matter from the population of the material based on the one or more picking points Configured to execute an operation including System

10. In response to comparing the area of the bounding box with the threshold area, the operation is configured to Determine that the area of the bounding box is less than the threshold area Generate the one or more picking points at the center of the bounding box The system according to claim 9, further comprising

11. In response to comparing the area of the bounding box with the threshold area, the operation is configured to Determine that the area of the bounding box is greater than the threshold area The system according to claim 9, further comprising

12. Generating the one or more picking points on the foreign matter may include Calculating one or more segmentations of the foreign matter Analyzing the one or more segmentations The system according to claim 11, further comprising

13. In response to analyzing the one or more segmentations, the operation is configured to identifying that the one or more segmentations are composed of points, determining that clusters of points are connected regions The system according to claim 12, further comprising.

14. In response to analyzing the one or more segmentations, the operation is identifying that the one or more segmentations are composed of one or more polygons, determining that each of the one or more polygons is a connected region The system according to claim 12, further comprising.

15. Generating the one or more picking points on the foreign object is identifying one or more connected regions on the foreign object, calculating the number of the one or more picking points for the one or more connected regions The system according to claim 9, further comprising.

16. Removing the foreign object from the population of materials based on the one or more picking points is collecting the foreign object via robotic means at the one or more picking points The system according to claim 15, further comprising.

17. A computer program for removing foreign objects in a population of materials, the computer program including program instructions executable by a processor, the program instructions causing the processor to perform the method according to any one of claims 1 to 8.

18. A computer-readable storage medium storing the computer program according to claim 17.

Images