Material sorting method and material sorting apparatus

Abstract

A material sorting method, which relates to the technical field of raw coal sorting. The method is applied to a material sorting apparatus, wherein the material sorting apparatus comprises: a conveying device for conveying materials, a radiation device arranged above the conveying device, and a color sorting camera arranged above the conveying device and located downstream of the radiation device. The sorting method comprises: acquiring a radiation image of a material by means of a radiation device, and acquiring a color sorting image of the material by means of a color sorting camera; on the basis of a radiation contour of the material in the radiation image, determining a target color sorting contour of the material in the color sorting image; on the basis of the color sorting image and the target color sorting contour, determining a material category of the material; and on the basis of the material category, sorting the material. Thus, the limitations of relying on a single feature can be avoided, the robustness of material identification against environmental interference is enhanced, and the accuracy and efficiency of material identification are ensured, thereby ensuring the material sorting performance.

Description

Material sorting methods and material sorting equipment Technical Field

[0001] This disclosure relates to the field of raw coal sorting technology, specifically to a material sorting method and material sorting equipment. Background Technology

[0002] In the field of raw coal sorting, accurate identification and separation of coal and gangue are core steps in improving coal quality, increasing sorting efficiency, and reducing resource waste. Related technologies primarily rely on X-ray detection results to identify and sort the transported materials.

[0003] However, in actual processing, X-rays are not sensitive to materials with small density differences, which can easily lead to misjudgment and thus affect the sorting accuracy. Summary of the Invention

[0004] To overcome the problems existing in related technologies, an exemplary embodiment of this disclosure provides a material sorting method applied to a material sorting device. The material sorting device includes: a conveying device for conveying materials, a ray device disposed above the conveying device, and a color sorting camera disposed above the conveying device and downstream of the ray device. The method includes: acquiring a ray image of the material through the ray device and acquiring a color sorting image of the material through the color sorting camera; determining a target color sorting contour of the material in the color sorting image based on the ray contour of the material in the ray image; determining the material category of the material based on the color sorting image and the target color sorting contour; and sorting the material based on the material category.

[0005] In some embodiments, determining the target color sorting profile of the material in the color sorting image based on the ray profile of the material in the ray image includes: determining the ray profile of the material based on the ray image, and determining the initial color sorting profile of the material based on the color sorting image; determining the motion state of the material relative to the conveying device based on the ray profile and the initial color sorting profile; if the motion state is motion, adjusting the candidate profile corresponding to the ray profile in the color sorting image based on the ray profile to obtain the target color sorting profile of the material in the color sorting image.

[0006] In some embodiments, adjusting the candidate contour corresponding to the ray contour in the color sorting image based on the ray contour to obtain the target color sorting contour of the material in the color sorting image includes: determining the candidate contour corresponding to the ray contour in the color sorting image based on the ray contour; determining the background region in the candidate contour based on the position of the candidate contour and the background image in the color sorting image; determining the first centroid based on the candidate contour; determining the second centroid corresponding to the non-background region in the candidate contour based on the background region; determining the adjustment displacement based on the first centroid and the second centroid; and adjusting the candidate contour based on the adjustment displacement to obtain the target color sorting contour.

[0007] In some embodiments, adjusting the candidate contour based on the adjustment displacement to obtain the target color selection contour includes: obtaining the candidate position of the candidate contour based on the adjustment displacement; adjusting the candidate position and / or the size of the candidate contour to obtain multiple intermediate contours; and selecting the intermediate contour with the smallest area as the target color selection contour based on the area of ​​the background image within each intermediate contour.

[0008] In some embodiments, determining the target color sorting profile of the material in the color sorting image based on the ray profile of the material in the ray image includes: determining a first profile corresponding to the target based on the ray image; determining a second profile in the color sorting image based on the first profile; determining a foreground region and a background region in the color sorting image based on the second profile; determining a reference value for the background grayscale based on the color sorting image; determining candidate sub-regions in the background region based on the reference value; determining a target region of the color sorting image based on the candidate sub-regions and the foreground region; and determining the target color sorting profile of the material in the color sorting image through the target region.

[0009] In some embodiments, determining the target region of a color-sorted image based on candidate sub-regions and foreground regions includes: if a candidate sub-region is adjacent to a foreground region, then fusing the candidate sub-region and the foreground region to obtain the target region; if any one or more target conditions are met, then removing the candidate sub-region, wherein the target conditions include: the candidate sub-region is not adjacent to the foreground region, the difference between the average gray value of the candidate sub-region and the reference value is less than a first approximate threshold, and the difference between the gray value of the offset position in the foreground region corresponding to the candidate sub-region and the reference value is greater than a second approximate threshold, wherein the offset position is determined based on the first centroid position of the candidate sub-region and the second centroid position of the foreground region.

[0010] In some embodiments, determining the material category of a material based on a color sorting image and a target color sorting profile includes: inputting the color sorting image into a target recognition model to determine multiple recognition boxes included in the color sorting image and their corresponding material types; and determining the type of the material based on the profile, the multiple recognition boxes, and their corresponding material types.

[0011] In some embodiments, determining the material type based on the contour, multiple recognition boxes, and corresponding material types includes: if a first recognition box exists among the multiple recognition boxes, the material type is determined to be a first type, wherein the first recognition box is a recognition box that includes the entire area of ​​the contour and whose corresponding material type is the first type; if no first recognition box exists among the multiple recognition boxes, the material type is determined based on the area of ​​a second recognition box, wherein the second recognition box is a recognition box that includes a partial area of ​​the contour and whose corresponding material type is the first type; if the material types corresponding to the multiple recognition boxes are all second types, the material type is determined to be a second type.

[0012] In some embodiments, the training process of the target recognition model includes: acquiring multiple image samples and multiple recognition box information corresponding to each image sample, wherein the material type corresponding to the multiple recognition box information includes a first type and / or a second type; and training an initial recognition model based on the multiple image samples and the multiple recognition box information corresponding to each image sample to obtain the target recognition model.

[0013] In some embodiments, sorting materials based on material category includes: determining the relative motion state between the material and the conveying device based on the X-ray image, the color sorting image, and the mapping relationship between the X-ray image and the color sorting image; if it is determined that the material and the conveying device are in a relatively stationary state, then determining a material sorting strategy based on the X-ray image, the distance between the color sorting camera and the X-ray device, and the operating speed of the conveying device, wherein the sorting strategy includes sorting time and sorting position; if it is determined that the material and the conveying device are in a relative motion state, then determining a sorting strategy based on the motion blur of the material in the color sorting image; and sorting the material according to the sorting strategy and the material category.

[0014] In some embodiments, determining a sorting strategy based on the motion blur of materials in a color sorting image includes: determining the degree of motion blur of materials in the color sorting image; if the degree of motion blur is less than a motion blur threshold, determining a material sorting strategy based on the ray image, the distance between the color sorting camera and the sorting mechanism, and the operating speed of the conveying device; if the degree of motion blur is greater than or equal to the motion blur threshold, determining a material sorting time based on the motion blur of the materials.

[0015] Secondly, this disclosure also provides a material sorting device, wherein the device includes: a conveying device for conveying materials; an X-ray device disposed above the conveying device; a color sorting camera disposed above the conveying device and downstream of the X-ray device; a material sorting device connected to the X-ray device and the color sorting camera respectively, for determining the material category of the materials by the material sorting method provided in either aspect; and a sorting device disposed downstream of the conveying device for sorting materials based on the material category.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0017] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: According to the material sorting method provided by this disclosure, the target color sorting contour of the material in the color sorting image is determined with reference to the ray contour of the material in the ray image. This can effectively avoid the limitations of a single feature. In the process of material identification, not only can the influence of environmental interference and impurity areas on material category identification be effectively eliminated, but the material identification process can also focus on the material itself, enhance the anti-environmental interference capability of material identification, and ensure the accuracy and efficiency of material identification. Thus, the material is sorted according to the identified material category, which can effectively improve the material sorting quality, shorten the sorting process, and ensure the material sorting performance. Attached Figure Description

[0018] This disclosure can be better understood by describing exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which:

[0019] Figure 1 is a schematic flowchart illustrating a material sorting method according to an exemplary embodiment of a disclosed invention;

[0020] Figure 2 is a flowchart illustrating a material sorting method according to an exemplary embodiment of a disclosed invention;

[0021] Figure 3 is a schematic diagram illustrating a candidate contour adjustment process according to an exemplary embodiment of a disclosed invention;

[0022] Figure 4 is a flowchart illustrating a material sorting method according to an exemplary embodiment of a disclosed invention;

[0023] Figure 5 is a schematic diagram illustrating a target region identification according to an exemplary embodiment of a disclosed document;

[0024] Figure 6 is a flowchart illustrating a material sorting method according to an exemplary embodiment of a disclosed invention;

[0025] Figure 7 is a schematic diagram illustrating a labeling according to an exemplary embodiment of a disclosed document;

[0026] Figure 8 is a schematic diagram illustrating an image of a mineral according to an exemplary embodiment disclosed in a publication;

[0027] Figure 9 is a schematic diagram illustrating an image of a mineral according to an exemplary embodiment disclosed in a publication;

[0028] Figure 10 is a schematic flowchart illustrating a material sorting method according to an exemplary embodiment of a disclosure;

[0029] Figure 11 is a schematic flowchart illustrating a material sorting method according to an exemplary embodiment of a publication. Detailed Implementation

[0030] The following describes specific embodiments of this disclosure. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. It should be understood that, in the actual implementation of any embodiment, just as in any engineering or design project, various specific decisions are often made to achieve the developer's specific goals and to meet system-related or business-related constraints, and this can change from one embodiment to another. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this disclosure, changes in design, manufacturing, or production based on the technical content disclosed in this disclosure are merely conventional technical means and should not be construed as insufficient content of this disclosure.

[0031] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. The terms “a” or “one,” etc., do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising,” “including,” etc., mean that the element or object preceding “comprising” or “including” encompasses the elements or objects listed following “comprising” or “including” and their equivalents, and do not exclude other elements or objects. The terms “connected,” “linked,” etc., are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0032] In related technologies, the identification and sorting of transmitted materials is mainly based on X-ray detection results. X-ray detection primarily relies on differences in X-ray absorption and attenuation among materials. However, since absorption attenuation is related to factors such as material density, effective atomic number, and thickness, when the density difference between materials is minimal and there is no significant distinction in atomic number and thickness, the signal differences after X-rays penetrate the material can be masked by interference from equipment detection accuracy and environmental noise. This ultimately leads to blurred material feature identification, making misjudgments more likely and affecting sorting accuracy.

[0033] To address the aforementioned problems, this disclosure provides a material sorting method applied to a material sorting equipment. The material sorting equipment may include: a conveying device for transporting materials, an X-ray device disposed above the conveying device, and a color sorting camera disposed above the conveying device and downstream of the X-ray device. The conveying device of the material sorting equipment can be a belt conveyor, chain conveyor, or roller conveyor, etc., and the materials can be conveyed to the sorting device of the material sorting equipment as the conveying device moves, where different types of materials are separated and collected. Both the X-ray device and the color sorting camera can be disposed above the conveying device, thereby enabling them to photograph the materials located on the top surface of the conveying device and moving with the movement of the conveying device. Both the X-ray device and the color sorting camera are disposed above the conveying device, and the color sorting camera can be located downstream of the X-ray device.

[0034] As shown in Figure 1, this material sorting method may include the following steps:

[0035] Step S110: Obtain X-ray images of the material using an X-ray device and color sort images of the material using a color sorting camera.

[0036] The X-ray device is located above the conveyor and emits rays, such as X-rays, onto the surface of the conveyor. As the material moves with the conveyor to the image acquisition area corresponding to the X-ray device, the X-ray device emits rays onto the surface of the conveyor, acquiring data from the image acquisition area and thus obtaining a X-ray image of that area. The X-ray image can at least include the outline information of the material. The color sorting camera can be an industrial camera, such as a CCD camera, area scan camera, or line scan camera. The color sorting camera can also be positioned above the conveyor and downstream of the X-ray device. Therefore, after passing through the image acquisition area corresponding to the X-ray device, the material moves with the conveyor to the image acquisition area corresponding to the color sorting camera, which then captures an image of that area as a color sorting image. The color sorting image can include information about the material, specifically the material captured in the corresponding X-ray image. Furthermore, the color sorting image may also include information such as the color of the material's surface.

[0037] Step S120: Based on the ray profile of the material in the ray image, determine the target color sorting profile of the material in the color sorting image.

[0038] Based on the ray image, the ray contour of the material collected in the ray image can be determined by methods such as contour extraction algorithms, deep learning models, or edge detection.

[0039] During the color sorting camera's capture process, environmental factors such as dust and the brightness of the lighting inside the material sorting equipment may cause deviations between the captured color sorting image and the actual material transport. Since both the X-ray image and the color sorting image capture the same batch of material, there is a certain mapping relationship between them. Therefore, the X-ray outline of the material can be used as a reference to correct the material's outline in the color sorting image, thus obtaining a reliable target color sorting outline.

[0040] Step S130: Determine the material category of the material based on the color sorting image and the target color sorting outline.

[0041] Color sorting images can determine the surface features of materials, such as color and texture. By identifying the target color sorting contour of the material in the color sorting image, the corresponding local image of the material and its actual spatial location can be determined. Identifying the material category based on the target color sorting contour in the local image ensures focused material category identification, effectively reduces interference from impurity areas, and guarantees the reliability and accuracy of material type determination.

[0042] Step S140: Sort materials based on material category.

[0043] With a clear understanding of the material category, targeted sorting of the materials can ensure the rationality and accuracy of the sorting process, thereby shortening the sorting process, improving sorting efficiency, and ultimately enhancing the performance of the material sorting equipment.

[0044] According to the material sorting method provided in this disclosure, the target color sorting contour of the material in the color sorting image is determined with reference to the ray contour of the material in the ray image. This can effectively avoid the limitations of a single feature. In the process of material identification, it can not only effectively eliminate the influence of environmental interference and impurity areas on material category identification, but also enable the material identification process to focus on the material itself, enhance the anti-environmental interference capability of material identification, and ensure the accuracy and efficiency of material identification. Thus, the material is sorted according to the identified material category, which can effectively improve the material sorting quality, shorten the sorting process, and ensure the material sorting performance.

[0045] The following examples will illustrate the process of determining the target color selection profile.

[0046] In some embodiments, as shown in FIG2, step S120 above may include the following steps:

[0047] Step S121: Determine the ray profile of the material based on the ray image, and determine the initial color sorting profile of the material based on the color sorting image.

[0048] Based on the ray image, the ray contour of the material collected in the ray image can be determined using methods such as contour extraction algorithms, deep learning models, or edge detection. Based on the color sorting image, the initial color sorting contour corresponding to the material in the color sorting image can be determined using methods such as contour extraction algorithms, deep learning models, or edge detection.

[0049] Step S122: Based on the ray profile and the initial color sorting profile, determine the motion state of the material relative to the conveying device.

[0050] Since the distance between the X-ray device and the color sorting camera is constant, and the time interval between each X-ray image and the color sorting image is the same, for materials on the conveyor, when they remain relatively stationary relative to the conveyor, the material's pose does not change relative to the conveyor, and its X-ray profile should have the same shape and position as the initial color sorting profile. However, when relative movement occurs between the material and the conveyor, the material's pose may change, resulting in a difference in the shape and / or position of the material's X-ray profile from the initial color sorting profile.

[0051] Specifically, relative motion between the material and the conveying device may occur because the material has a certain acceleration, causing its position on the conveying device to change as it moves, resulting in a difference between the ray profile and the initial color sorting profile. Relative motion between the material and the conveying device may also occur because the material rotates, causing a change in its posture on the conveying device, resulting in a difference between the ray profile and the initial color sorting profile. Therefore, the relative motion state between the material and the conveying device—whether it is stationary or in motion—can be determined based on the ray profile and the initial color sorting profile.

[0052] Step S123: If the motion state is motion, then based on the ray profile, adjust the candidate profile corresponding to the ray profile in the color sorting image to obtain the target color sorting profile of the material in the color sorting image.

[0053] Because the material and the conveying device are in motion relative to each other, the candidate contours corresponding to the ray contours in the color sorting image differ from the actual contours of the material in the color sorting image. Therefore, it is necessary to adjust the candidate contours based on their positions in the color sorting image to determine the target color sorting contour. Adjusting the candidate contours can involve directly translating, rotating, or scaling them, or adjusting the position of each contour point. In some examples, the target color sorting contour may be the actual contour of the material in the color sorting image.

[0054] According to the method for determining the target color sorting profile provided in this disclosure, by comparing and analyzing the ray image and the color sorting image of the material, the target color sorting profile obtained in the color sorting image can be closer to the actual profile of the material. Therefore, in the subsequent process of extracting the color sorting image from the target color sorting profile, determining the material classification, and performing sorting, the problem of excessive difference between the color sorting image extracted from the target profile and the actual color sorting image corresponding to the material caused by the relative movement between the material and the conveying device can be avoided. Even if the material moves or rotates during the transmission process, the actual profile of the material in the color sorting image can be accurately restored. This allows for more accurate acquisition of material characteristics, resulting in higher material identification accuracy of the material sorting equipment and avoiding misidentification and missed identification.

[0055] In some embodiments, step S123 above may include the following steps:

[0056] Step a1: Based on the ray contour, determine the candidate contours in the color-sorted image that correspond to the ray contour.

[0057] Since the initial color sorting profile is obtained by extracting the profile from the color sorting image, while the ray profile is obtained by extracting the profile from the ray image, it is difficult to compare the initial color sorting profile and the ray profile in the same dimension. Therefore, the candidate profile corresponding to the ray profile in the color sorting image can be determined based on the information of the ray profile, the distance between the ray device and the color sorting camera, and the conveying speed of the material conveying device.

[0058] a2, based on the position of the candidate contour and the background image in the color-sorted image, determines the background region in the candidate contour.

[0059] A color sorting image can include a background image and the area where the material is located. The background image is the area outside the material captured in the color sorting image, i.e., the upper surface of the conveyor. Since the color and shape of the area containing the material in the color sorting image are not necessarily uniform, while the top surface of the conveyor has uniform color and occupies a large proportion of the image, making it easier to detect, the background image in the color sorting image can be determined first. Then, by combining this with the position of the candidate contour in the color sorting image, the background area within the region enclosed by the candidate contour can be determined.

[0060] Step a3: Determine the first centroid based on the candidate contour.

[0061] Based on the position of the candidate contour, the positional information of each contour point on the candidate contour in the color sorting image can be determined. Then, by directly calculating the geometric center, the first centroid of the candidate contour can be directly determined by calculating the positional information of each contour point on the candidate contour, thus determining the position of the first centroid in the color sorting image. Alternatively, the first centroid can be determined using image moments based on the region in the color sorting image enclosed by the candidate contour. Furthermore, the candidate contour can be represented by a parametric equation, and the first centroid can be calculated using methods such as boundary integrals, thereby determining the positional information of the first centroid of the candidate contour.

[0062] Step a4: Based on the background region, determine the second centroid corresponding to the non-background region in the candidate contour.

[0063] Based on the background region within the candidate contour, the difference between the region enclosed by the candidate contour and the background region can be calculated to determine the non-background region within the candidate contour. Because the material moves relative to the conveyor during its movement, the overlap between the actual material location in the color sorting image and the region enclosed by the candidate contour is small. Therefore, the non-background region within the candidate contour is only a portion of the actual material location captured in the color sorting image. The non-background region within the candidate contour can be determined based on the background region, and a second centroid can be determined for this non-background region. For example, the second centroid can be determined based on the contour points of the non-background region, identifying its location, and then combining this location information. Alternatively, the second centroid can be obtained by binarizing the non-background region and calculating it using image moments.

[0064] Step a5: Determine the adjustment displacement based on the first and second centroids.

[0065] Since the position of the candidate contour in the color sorting image is the theoretical position of the ray contour in the ray image, and the non-background area is the area occupied by the material in the candidate contour, when the material is stationary relative to the conveying device, the color sorting image enclosed by the candidate contour should consist entirely of non-background areas, or mostly of non-background areas, and the first centroid of the candidate contour should be close to or coincide with the second centroid of the non-background area. However, when the material moves relative to the conveying device, the area corresponding to the material in the color sorting image shifts, resulting in a smaller proportion of non-background areas in the color sorting image enclosed by the candidate contour, and a larger shift of the second centroid relative to the first centroid.

[0066] Therefore, based on the position information of the first and second centroids in the color sorting image, the direction of material movement relative to the candidate contour can be determined according to the direction from the first centroid to the second centroid. The distance of material movement relative to the candidate contour is determined based on the distance from the first centroid to the second centroid. The adjustment displacement includes both the direction and distance of movement of the candidate contour.

[0067] Step a6: Based on the adjustment of displacement, adjust the candidate contour to obtain the target color selection contour.

[0068] The candidate contour is adjusted by adjusting the direction and distance of the displacement. This allows the coordinates of each contour point to move along the direction of the displacement, with the moving distance being the displacement distance itself, thus obtaining a new set of contour points. Based on the new position information of each contour point after adjustment, the new contour formed by the adjusted contour points is used as the target color selection contour.

[0069] By identifying the background of the candidate contour, the background area corresponding to the conveying device and the area corresponding to the material in the image can be accurately distinguished. This eliminates the interference of the background area, making the extracted target color sorting contour more accurate and closer to the actual contour of the material in the color sorting image. In the subsequent process of extracting color sorting images based on the target color sorting contour for identification and sorting, the influence of the background on material identification and sorting can be effectively avoided, thereby improving the accuracy of image recognition and improving the overall material sorting accuracy of the sorting equipment.

[0070] In some embodiments, step a6 above may include the following steps:

[0071] Step a61: Based on the adjusted displacement, obtain the candidate position of the candidate contour.

[0072] By adjusting the displacement, the movement direction and distance of the candidate contour can be adjusted to obtain the candidate position of the candidate contour. The candidate position can be the centroid of the candidate contour. Specifically, as shown in Figure 3, O1 in the image can be the first centroid, and the corresponding rectangle in the figure is the candidate contour corresponding to the ray contour in the color sorting image. O2 in the image can be the second centroid, where the area enclosed by the rectangle with O2 as the centroid is part of the candidate contour area, and the rectangle with O2 as the centroid is the non-background area in the candidate contour, that is, the area containing material collected in the candidate contour. This can be determined based on... Determine the adjustment displacement, including the direction and magnitude of movement of the candidate contour. This can be done... The direction of adjustment is used as the direction of displacement, and the candidate contour can be translated along this direction. The displacement adjustment can be... The first centroid can be moved based on the first centroid, the second centroid, and the adjustment displacement. This determines the third centroid O3 as the centroid corresponding to the candidate position of the candidate contour. Furthermore, assuming the contour point coordinates of the candidate contour are C, the coordinates of the contour point corresponding to the candidate position after the displacement adjustment can be calculated as follows:

[0073] Step a62: Adjust the candidate position and / or the size of the candidate contour to obtain multiple intermediate contours.

[0074] Because the material may rotate or tumble, its actual outline shape in the color sorting image may change. Therefore, the candidate outline may move, rotate, and deform relative to the actual outline of the material.

[0075] Therefore, multiple intermediate contours can be obtained by adjusting the overall size of the candidate contour, or by translating or rotating the candidate contour and adjusting its position. For example, the adjustment range of the candidate contour can be predetermined, such as setting the adjustment range of each contour point of the candidate contour to 5 pixels vertically, horizontally, and vertically. The contour points of the candidate contour can then be translated and rotated within this adjustment range, thereby adjusting the overall size and / or position of the candidate contour and determining multiple intermediate contours. Alternatively, all contour points of the candidate contour can be adjusted as a whole based on the direction and magnitude of the adjustment displacement. Furthermore, the contour points of the candidate contour can be adjusted independently to adjust the size of the candidate contour.

[0076] Step a63: Based on the area of ​​the background image within each intermediate contour, select the intermediate contour with the smallest area as the target color selection contour.

[0077] The intermediate contour may deviate from the actual material contour, resulting in some background image within the area enclosed by the intermediate contour. Therefore, by comparing the area of ​​the background image portion in each intermediate contour, the intermediate contour with the smallest background image area can be determined. The area enclosed by this intermediate contour has the largest material area, thus determining that this intermediate contour is closest to the actual initial color sorting contour of the material in the color sorting image, and this intermediate contour can be used as the target color sorting contour.

[0078] This method can significantly improve the image recognition and material sorting accuracy of material sorting equipment, thereby enhancing the accuracy and reliability of sorting.

[0079] In some embodiments, the process of determining the motion state of the material relative to the conveying device may be as follows:

[0080] Step b1: Based on the initial color sorting contour, determine the reference contour in the ray image that corresponds to the initial color sorting contour.

[0081] Since the time intervals for the X-ray device to acquire X-ray images, the time intervals for the color sorting camera to acquire color sorting images, the distance between the X-ray device and the color sorting camera, and the conveyor belt speed are all preset fixed parameters, a reference contour corresponding to the initial color sorting contour in the X-ray image can be determined based on the initial color sorting contour, the distance between the X-ray device and the color sorting camera, and the conveyor belt speed.

[0082] Step b2: Based on the reference profile and the ray profile, determine the motion state of the material relative to the conveying device.

[0083] Based on the reference profile and the ray profile, the motion state of the material relative to the conveying device can be determined according to the offset of the reference profile relative to the ray profile. Specifically, it is possible to...

[0084] Since the material may move or rotate in any direction relative to the conveying device during the movement, the deviation between the reference profile and the ray profile may include positional deviation and rotational deviation. Therefore, the relative motion state between the material and the conveying device can be determined based on the difference between the reference profile and the ray profile.

[0085] If the reference profile and the ray profile largely coincide, it indicates that there is no significant relative movement between the material and the conveying device between the acquisition of the ray image and the acquisition of the color sorting image, and the material can be considered to remain stable. If there is a significant deviation between the theoretical ray profile and the actual ray profile, it indicates that there is relative movement between the material and the conveying device.

[0086] By determining the motion state of the material relative to the conveying device, different identification methods can be adopted for the material according to different motion states. This saves the computing power of the sorting equipment that performs the material sorting method, and improves the contour matching accuracy between the X-ray image and the color sorting image in subsequent method steps. This enhances the stability and robustness of the matching between the color sorting image and the X-ray image, and improves the accuracy and reliability of material identification and sorting.

[0087] In some embodiments, the motion state of the material relative to the conveying device can be determined based on the distance between the theoretical centroid of the reference profile and the actual centroid of the ray profile. The theoretical and actual centroids can be determined directly by calculating the position information of their corresponding profile points, or they can be determined based on the reference profile or ray profile using methods such as image moments or boundary integrals. The distance between the theoretical and actual centroids can be determined based on their position information in the ray image. When the distance between the theoretical and actual centroids is greater than or equal to a distance threshold, it indicates a large offset of the material, and it can be determined that the material is moving relative to the conveying device. When the distance between the theoretical and actual centroids is less than the distance threshold, it can be considered that the material has only experienced minor disturbances, such as minor vibrations of the conveying mechanism or minor offsets due to the material's own shape. In this case, the offset of the material can be considered small, and the material can be considered stationary relative to the conveying device.

[0088] In some embodiments, the motion state of the material relative to the conveying device can be determined based on the intersection-union ratio (IU / U) of the reference profile and the ray profile. The IU / U is determined by calculating the area of ​​the intersection of the reference profile and the ray profile, and the area of ​​the union of the reference profile and the ray profile. The closer the IU / U is to 1, the higher the overlap between the reference profile and the ray profile. When the IU / U is 1, it indicates that the reference profile and the ray profile are completely identical, and it can be determined that the material has not undergone relative movement with the conveying device; therefore, the motion state of the material relative to the conveying device can be determined to be stationary. An IU / U is set as a threshold. When the IU / U is greater than or equal to the threshold, there may only be some negligible slight offset between the material and the conveying device, and the motion state of the material relative to the conveying device can be determined to be stationary. When the IU / U is less than the threshold, there is a larger offset between the material and the conveying device, and the motion state of the material relative to the conveying device can be determined to be moving.

[0089] In some embodiments, the motion state of the material relative to the conveying device can be determined based on the axis length and direction of the ellipse fitted by the reference profile or ray profile. Since some materials move at high speeds on the conveying device, the reference profile or ray profile may experience some offset and deformation. In such cases, the larger profile of the reference profile or ray profile can be fitted as an ellipse. This includes cases where the reference profile and ray profile are inclusive. The axis length and direction of the ellipse are determined. Based on a preset threshold, if the axis length of the ellipse is less than the preset threshold and the direction of the ellipse meets the preset requirements, the motion state of the material relative to the conveying device can be determined to be stationary. Conversely, if the axis length of the ellipse is greater than or equal to the preset threshold and the direction of the ellipse does not meet the preset requirements, it indicates that the material has undergone a significant offset relative to the conveying device, thus determining the motion state of the material relative to the conveying device as motion.

[0090] Furthermore, in some embodiments, if the reference profile and the ray profile do not intersect, it can be directly determined that the current material does not exist, so that the subsequent sorting device does not perform any sorting operation on the material, thereby reducing the false recognition rate and improving the accuracy of material identification and sorting.

[0091] In some embodiments, if the motion state is stationary, the contour corresponding to the ray contour can be directly determined in the color sorting image based on the ray contour and according to the distance between the ray device and the color sorting camera and the transmission speed of the conveying device, and used as the target color sorting contour of the material.

[0092] In some embodiments, as shown in FIG4, step S120 above may include the following steps:

[0093] Step S124: Based on the ray image, determine the first contour of the corresponding target.

[0094] By extracting the contour from the ray image, the first contour of the material in the ray image is determined.

[0095] Step S125: Based on the first contour, determine the second contour in the color selection image.

[0096] Since X-ray images and color sorting images are two different images of the same material acquired separately by the X-ray device and the color sorting camera, the mapping relationship between the X-ray images and the color sorting images can be determined based on information such as the time of acquisition of the X-ray images and the time of acquisition of the color sorting images, the material conveying speed of the transmission device, or the distance between the X-ray device and the color sorting camera. A first contour is mapped onto the color sorting image, and a second contour corresponding to the first contour is determined in the color sorting image.

[0097] Step S126: Based on the second contour, determine the foreground region and background region in the color-selected image.

[0098] Based on the second contour in the color sorting image, the foreground region where the material might be located in the color sorting image can be determined. This foreground region can contain the complete second contour and the area enclosed by it. Based on the second contour, its circumscribed rectangle can be determined, ensuring that the circumscribed rectangle includes the second contour and all its internal information. To ensure the completeness of the circumscribed rectangle, a region slightly larger than the second contour can be defined as a sub-image region, containing both the complete second contour and the foreground region. Since the sub-image region includes the foreground region, it is removed to obtain the background region.

[0099] Step S127: Determine the reference value for the background grayscale based on the color-selected image.

[0100] Since the color sorting image is a color space image, to more accurately determine the candidate sub-regions, the background region of the color sorting image can first be grayscale binarized, converting it into a binary image where the pixels do not differ in color space, only in grayscale values. The binarized background region is then segmented into two distinct regions with grayscale values ​​of 0 and 255, designated as the first and second sub-regions. This effectively simplifies the structural features of the background region and reduces the impact of color interference on candidate sub-region extraction.

[0101] By adaptively determining a baseline value for background grayscale from the color-sorted image and using this baseline value as the criterion for distinguishing the background area and material area of ​​the conveying device, candidate sub-regions with significantly deviated grayscale features from the background can be accurately screened from the first and second sub-regions. This provides a reliable basis for the subsequent determination of the target area and the correction of the complete outline, thereby improving the completeness and accuracy of target area recognition.

[0102] In some examples, a bounding rectangular region can be determined based on the second contour. Based on the bounding rectangular region and the color-sorted image, multiple pixels in the color-sorted image that differ from the second contour and its internal region are identified as the extraction range. Based on the extraction range, the grayscale values ​​of multiple discrete pixels within the extraction range are determined and used as a reference to distinguish the background and foreground parts of the image, i.e., to distinguish the grayscale differences between the surface of the conveyor and the material. Alternatively, the extraction range can be divided into multiple regions, and a consistent number of pixels can be selected for each region to determine the grayscale values ​​of the selected pixels in each region. Based on the extracted multiple pixel grayscale values, the pixel grayscale values ​​are averaged, median, or weighted averaged to determine a baseline value. This baseline value can serve as a reference for determining candidate sub-regions.

[0103] Step S128: Based on the baseline value, determine the candidate sub-regions in the background region.

[0104] When the grayscale value of a pixel in the color sorting image is close to the reference value, it can be considered that the grayscale value of that pixel is similar to the grayscale value of an area on the surface of the conveyor device where there is no material. Therefore, it can be determined that the current pixel may belong to the surface of the conveyor device. When the grayscale value of a pixel differs significantly from the reference value, it can be considered that the grayscale value of that pixel differs significantly from the grayscale value of the surface of the conveyor device. Therefore, it can be considered that the pixel does not belong to the surface of the conveyor device.

[0105] Therefore, the grayscale values ​​of all pixels in the first sub-region are determined, and then the average grayscale value of all pixels in the first sub-region is calculated, which is the first average grayscale value. The grayscale values ​​of all pixels in the second sub-region are determined, and then the average grayscale value of all pixels in the second sub-region is calculated, which is the second average grayscale value. If the first average grayscale value is closer to the reference value than the second average grayscale value, then the second sub-region is determined as a candidate sub-region. If the first average grayscale value is not closer to the reference value than the second average grayscale value, then the first sub-region is determined as a candidate sub-region.

[0106] Step S129: Determine the target region of the color-sorted image based on the candidate sub-regions and the foreground region.

[0107] Based on the foreground region and candidate sub-regions, the two can be merged to determine the target region of the color sorting image. The target region is the area in the color sorting image corresponding to the material. Therefore, by merging the sub-regions and the foreground region, the complete outline of the material in the color sorting image can be determined more accurately, thereby effectively improving the accuracy of subsequent material identification.

[0108] In some examples, if a candidate sub-region is adjacent to a foreground region, indicating that the candidate sub-region and the foreground region can form a continuous image, then the candidate sub-region and the foreground region can be considered to correspond to the same material, and thus the candidate sub-region and the foreground region are merged to obtain the target region. If any one or more target conditions are met, the candidate sub-region is eliminated. These target conditions may include: the candidate sub-region is not adjacent to the foreground region; the difference between the average gray value of the candidate sub-region and the reference value is less than a first approximate threshold; and the difference between the gray value of the offset position in the foreground region corresponding to the candidate sub-region and the reference value is greater than a second approximate threshold. The offset position is determined based on the first centroid position of the candidate sub-region and the second centroid position of the foreground region.

[0109] Specifically, when a candidate sub-region is not adjacent to the foreground region, it can be considered that the candidate sub-region and the foreground region do not correspond to the same material, so the candidate sub-region can be eliminated, allowing the foreground region to remain in its original state. When the difference between the average gray value of all pixels in the candidate sub-region and the reference value is less than a first approximation threshold, the candidate sub-region can be determined to belong to the background region. When the difference between the average gray value of the candidate sub-region and the reference value is greater than or equal to the first approximation threshold, the candidate sub-region can be determined not to belong to the background region, but to belong to the foreground region corresponding to the material. When the difference between the average gray value of the candidate sub-region and the reference value is less than a preset first approximation threshold, it indicates that the gray value characteristics of the region are close to the surface of the conveying device, and it can be eliminated to avoid misidentifying the background as material. Conversely, when the difference is greater than or equal to the first approximation threshold, the candidate sub-region can be considered to deviate significantly from the background gray value characteristics and belong to the foreground region where the material is located.

[0110] Because material slippage may occur during transport, the foreground region determined based on the second contour may not have a contour difference relative to the target region. Only due to the offset of the foreground region, a portion of the background region exists within it. Therefore, the target region can be determined based on the corresponding offset position between the candidate sub-region and the foreground region. Specifically, the candidate sub-region is analyzed to determine its first centroid position O1. Simultaneously, the foreground region, i.e., the centroid O2 of the second contour, can be determined. The line connecting the first centroid position O1 and the second centroid position O2 is drawn and extended; the direction of this extension is the offset direction of the foreground region relative to the target region. As shown in Figure 5, the intersection points A and B of the extended line with the edge of the foreground region can be determined, as can the distance L between A and B. Based on the first centroid position, along the offset direction, the point X, which is a distance L from the first centroid position, can be determined. Thus, point X is the offset position in the foreground region corresponding to the candidate sub-region.

[0111] Since the reference value is the grayscale value corresponding to the surface image of the conveyor device in the image, when the grayscale value of the image is close to the reference value, the image can be approximately considered to belong to the background image and the surface image of the conveyor device. Therefore, a second approximation threshold can be determined. When the difference between the grayscale value of a pixel and the reference value is greater than or equal to the second approximation threshold, it can be determined that the grayscale value of the current pixel differs significantly from the background area, and the current pixel belongs to the material area. When the material is shifted, if the candidate sub-region is the corresponding area of ​​the material, the offset position corresponding to the candidate sub-region in the color sorting image should be the background area. Thus, the grayscale value of the offset position can be determined, and the difference between the grayscale value of the offset position and the reference value can be determined. When the difference between the grayscale value of the offset position and the reference value is greater than the second approximation threshold, it can be determined that the offset position belongs to the corresponding area of ​​the material. Therefore, it can be determined that the candidate sub-region does not correspond to the offset position or does not correspond to the same material as the current foreground area, and thus the candidate sub-region can be eliminated.

[0112] Step S1210: Determine the target color sorting outline of the material in the color sorting image through the target area.

[0113] Since the target area is a local area image in the color sorting image that corresponds to the material, targeted feature extraction can be performed on this local area image to obtain the target color sorting outline of the material in the color sorting image.

[0114] By determining the target color sorting contour in the above manner, even if the material changes pose between transmission imaging and color sorting imaging, the target area in the color sorting image can be accurately and stably determined. This avoids the problem of missing material edge information or truncation of the target area caused by relying solely on contour mapping, making the target color sorting contour and color information within the target area more complete and reliable. This significantly improves the accuracy of subsequent material identification and sorting processes and reduces the probability of misjudgment and missed judgment.

[0115] The following examples will illustrate the process of determining material categories.

[0116] In some embodiments, as shown in FIG6, step S130 above may include the following steps:

[0117] Step S131: Input the color sorting image into the target recognition model to determine the multiple recognition boxes included in the color sorting image and the corresponding material types.

[0118] Because color-sorted images may contain adjacent impurities in addition to the desired material, to improve the accuracy of material identification, the color-sorted image is input into a target recognition model to identify the material types and corresponding locations that may be included in the image, and then label them with corresponding recognition boxes. For example, a correspondence between the color of the recognition box and the material type is established in advance, and then the corresponding color recognition box is used to label the material type. The target recognition model is a pre-trained recognition model that can identify not only the material type corresponding to global image features but also the material type corresponding to local image features. The output recognition result includes multiple recognition boxes and their corresponding material types. For example, as shown in Figure 7, by detecting the color-sorted image through the target recognition model, four areas containing material can be identified. Then, based on the identified material type, the corresponding recognition boxes are used for labeling, resulting in four recognition boxes in the figure (one large rectangle containing the entire image and three small rectangles in the right area of ​​the figure).

[0119] Step S132: Determine the type of material based on the contour, multiple recognition boxes, and the corresponding material type.

[0120] By analyzing the outline, the corresponding region in the color sorting image can be determined. By identifying the bounding boxes and their corresponding material types, the material type corresponding to the image region marked by each bounding box in the color sorting image can be determined. By matching the image region marked by each bounding box with the region containing the outline, the bounding boxes that best fit the outline can be selected, and the corresponding material type can be used as the material type. This effectively reduces interference from impurity areas and ensures the reliability and accuracy of material type determination.

[0121] In some examples, if a first identification box exists among multiple identification boxes, the material type is determined to be the first type. Here, the first identification box is one that encompasses the entire outline area and corresponds to the first type of material. Since the first identification box covers the entire area containing the material to be sorted, the material type corresponding to the first identification box can be directly used as the material type, thus helping to improve material identification efficiency.

[0122] If a first identification box is absent from multiple identification boxes, but a second identification box exists, it indicates that the color-sorted image contains material image features corresponding to the first type. Therefore, to avoid misidentification, the overlap between the location of the second identification box and the location of the contour is determined based on the identification area of ​​the second identification box. If the ratio of the intersection area to the contour area is greater than a target threshold, the material type is determined to be the first type; if the ratio is less than or equal to the target threshold, the material type is determined to be the second type. Here, the second identification box is an identification box that contains a local region of the contour and corresponds to the first type of material. The first type is the desired material type. The target threshold is the minimum critical ratio used to measure whether the area proportion is effective. For example, the target threshold could be 1 / 3.

[0123] If the material type corresponding to multiple recognition boxes is all of the second type, there is no ambiguity in the determination of the material type, and the material type can be directly determined to be of the second type, which helps to improve the determination efficiency.

[0124] In some embodiments, the training process of the target recognition model may include:

[0125] Step c1: Obtain multiple image samples and multiple recognition bounding boxes corresponding to each image sample;

[0126] Step c2: Based on multiple image samples and the multiple recognition box information corresponding to each image sample, train the initial recognition model to obtain the target recognition model.

[0127] When there are no obvious distinguishing features on the surface of an ore, the exposed area after an impact often reveals the true properties of the ore. For example, in some applications, as shown in Figure 8, an ore image identified solely by the area within the ray outline can easily be misidentified as coal; however, by observing the exposed area at the impact point, it can be accurately identified as gangue. Similarly, as shown in Figure 9, an ore image observed only by the area within the ray outline can easily be misidentified as gangue, but by utilizing the high brightness of the exposed area after an impact, it can be determined to be coal.

[0128] Therefore, to enable the trained target recognition model to simultaneously output multiple bounding boxes and corresponding material types, the required image samples and corresponding bounding box information are pre-constructed. Specifically, the annotation of each image sample must cover all possible material regions in the image, and the material types corresponding to the multiple bounding boxes must completely include the first type (target sorting material, such as coal) and / or the second type (non-target material, such as gangue), ensuring that the initial recognition model can learn the feature differences between the two types of materials. That is, for the same image sample, a dual-dimensional annotation method combining global and local annotations is used to ensure that the initial recognition model, during training, can learn both the complete global surface features of the material and the discriminative local surface features, making the feature information learned by the model more comprehensive, detailed, and reliable.

[0129] Ultimately, the trained target recognition model can accurately identify both global material regions and local feature regions in an image, outputting multi-dimensional recognition boxes and their corresponding types, providing sufficient and reliable recognition basis for subsequent accurate matching and judgment based on material contours.

[0130] The following examples will illustrate the process of sorting materials.

[0131] In some embodiments, as shown in FIG10, step S140 above may include the following steps:

[0132] Step S141: Determine the relative motion state between the material and the conveying device based on the X-ray image, the color sorting image, and the mapping relationship between the X-ray image and the color sorting image.

[0133] Step S142: If it is determined that the material and the conveying device are in a relatively stationary state, then the material sorting strategy is determined based on the X-ray image, the distance between the color sorting camera and the X-ray device, and the operating speed of the conveying device.

[0134] When the material and the conveying device are in a relatively stationary state, it can be determined that the material and the conveying device remain relatively stable and do not move during the process of moving from the X-ray device to the color sorting camera. Therefore, the material sorting strategy can be determined based on the X-ray image of the material, the initial distance between the color sorting camera and the X-ray device, and the operating speed of the conveying device. The material sorting strategy includes the sorting time during which the sorting device performs the sorting operation, and the position where the material will fall from the conveying device, i.e., the sorting position where the sorting device performs the sorting operation.

[0135] Step S143: If it is determined that the material and the conveying device are in relative motion, then the sorting strategy is determined based on the motion blur of the material in the color sorting image.

[0136] When the material and the conveying device are in relative motion, it can be determined that during the process of moving from the X-ray device to the color sorting camera, the material experiences acceleration relative to the conveying device. This causes the material's moving speed to differ from the conveying speed, or the material may rotate during transport. These factors result in motion blur in the color sorting image captured by the color sorting camera. Motion blur reflects the direction and speed of the relative motion between the material and the conveying device. Therefore, the material sorting strategy can be determined based on the motion blur in the color sorting image.

[0137] Step S144: Sort materials according to the sorting strategy and material category.

[0138] The sorting device can be a pusher mechanism including multiple push plates or a spraying mechanism including multiple nozzles. Based on the determined sorting strategy, the time and position of the material falling from the conveyor can be determined more accurately, so that the sorting device can respond in a timely and accurate manner, achieving accurate sorting.

[0139] Different sorting strategies are determined and sorting operations are executed based on the different relative motion states of the material and the conveying device. There is no need for manual adjustment of sorting parameters, which reduces human error and improves the automation level of material sorting equipment, making material sorting more efficient and intelligent.

[0140] In some embodiments, as shown in FIG11, step S143 above may include the following steps:

[0141] Step S1431: Determine the degree of motion blur of the material in the color sorting image.

[0142] When materials move relative to the conveying device in any direction, the color sorting camera may cause motion blur in the color sorting image of the material in its direction of movement. Therefore, edge detection, Fourier analysis, and other methods can be used to extract the motion-blurred color sorting contour information in the color sorting image, quantify the motion blur of the material, and determine the degree of motion blur. Motion blur can reflect the speed and direction of material movement.

[0143] Step S1432: If the degree of motion blur is less than the motion blur threshold, then the material sorting strategy is determined based on the ray image, the distance between the color sorting camera and the sorting mechanism, and the operating speed of the conveying device.

[0144] If the degree of motion ambiguity is less than the preset motion ambiguity threshold, the relative motion between the material and the conveying device can be considered to be relatively slow or basically stable. The relative motion between the material and the conveying device can be regarded as relatively stationary. Therefore, based on the fixed distance between the color sorting camera and the sorting device, combined with the operating speed of the conveying device, the expected transmission time of the material from the location of the color sorting camera to the sorting device can be determined as the sorting time. At the same time, the position of the material on the conveying device at the time of the color sorting camera's shooting can be determined, thereby accurately determining the falling position of the material at the end of the conveying device as the sorting position.

[0145] Step S1433: If the degree of motion ambiguity is greater than or equal to the motion ambiguity threshold, then determine the material sorting time.

[0146] If the degree of motion blur is greater than or equal to a preset motion blur threshold, it can be assumed that at the moment the color sorting camera captures the color sorting image, the material and the conveying device are still in relative motion, and the relative motion between the material and the conveying device is relatively intense. Therefore, by detecting the degree of motion blur in the color sorting image, it is possible to accurately distinguish whether the material is relatively stationary with respect to the conveying device or whether there is a significant degree of relative motion, thereby determining different sorting strategies and improving the accuracy and reliability of sorting.

[0147] For example, by using edge detection and Fourier analysis to quantify motion fuzziness, we can not only determine whether materials are in relative motion, but also further determine their direction, speed, and acceleration. This allows for more accurate determination of the material's sorting time and position. When the degree of motion fuzziness is low, the sorting time and position can be precisely calculated based on the conveyor speed and fixed distance, ensuring efficient and stable sorting. When the degree of motion fuzziness is high, the sorting time can be calculated based on the material's motion fuzziness information, thereby avoiding mis-sorting or missed sorting due to material position deviation and improving the sorting success rate.

[0148] By using precise predictions based on motion fuzzing, it can be ensured that sorting operations only apply to the correct materials, thereby helping to reduce ineffective operations and improve equipment lifespan and energy efficiency.

[0149] In some embodiments, the actual direction and speed of material movement can be determined based on the motion fuzziness of the material; the theoretical direction of material movement can be determined based on the ray image, the color sorting image, and the mapping relationship between the ray image and the color sorting image; the deviation between the actual direction of material movement and the theoretical direction of material movement can be determined; if the deviation between the actual direction of material movement and the theoretical direction of material movement is less than the motion deviation threshold, then a material sorting strategy is determined based on the ray image, the distance between the color sorting camera and the ray device, and the operating speed of the conveying device; and if the deviation between the actual direction of material movement and the theoretical direction of material movement is greater than or equal to the motion deviation threshold, then the sorting of materials based on the sorting strategy and material category is skipped.

[0150] Specifically, because motion blur causes the edges of the sorted contour in the color sorting image to appear elongated, the blur direction and length of the material can be determined based on the motion blur, and the blur direction is determined as the actual movement direction of the material. The material's movement speed is determined based on the exposure time of the color sorting camera and the blur length. Based on the ray contour in the X-ray image, the mapping relationship between the X-ray image and the color sorting image, and the sorted contour in the color sorting image, the ray contour can be mapped to the color sorting image to determine the theoretical contour of the ray contour in the color sorting image. Then, based on the deviation between the sorted contour and the theoretical contour, the theoretical movement direction of the material can be determined. The angle formed by the actual movement direction and the theoretical movement direction of the material is determined as the deviation between the two. A motion deviation threshold can be set, and the deviation between the actual and theoretical movement directions of the material is compared with the motion deviation threshold. Different sorting strategies can be adopted based on the magnitude of the deviation between the actual and theoretical movement directions of the material. If the deviation between the actual and theoretical movement directions of the material is less than the motion deviation threshold, the material sorting strategy determined based on the X-ray image, the distance between the color sorting camera and the X-ray device, and the operating speed of the conveyor device is executed. If the deviation between the actual direction of material movement and the theoretical direction of material movement is greater than or equal to the movement deviation threshold, then skip the process of sorting materials according to the sorting strategy and material category.

[0151] In some embodiments, the process of determining the material sorting time based on the motion fuzziness of the material may include the following steps: Determining the material's speed from the X-ray device to the color sorting camera based on the distance between the X-ray device and the color sorting camera, the material's speed upon reaching the color sorting camera, and the time it takes for the material to move from the X-ray device to the color sorting camera, allows for a more accurate grasp of the initial motion state of the material on the conveyor device and effectively compensates for errors in the color sorting image caused by shooting angle or lighting conditions. Determining the material's acceleration based on its speed upon reaching the color sorting camera, the time it takes for the material to move from the X-ray device to the color sorting camera, and the speed at which the material reaches the X-ray device makes the calculation of the material's motion parameters more stable and reliable. Determining the material sorting time based on the material's acceleration, its speed upon reaching the color sorting camera, and its speed at which the material reaches the X-ray device allows for accurate judgment of the material's motion trend on the conveyor device, thereby adjusting the sorting strategy and ensuring the accuracy of the sorting operation.

[0152] In some embodiments, the material sorting equipment further includes a sorting device disposed downstream of the conveying device. The process of determining the material sorting time may further include: determining the relative motion between the material and the conveying device before reaching the sorting device based on the material's acceleration, the material's velocity when it reaches the color sorting camera, and the material's velocity when it reaches the X-ray device. If the material is relatively stationary with respect to the conveying device before reaching the sorting device, the time required for the material and the conveying device to reach relative stationary, and the time required for the material to move to the sorting device after reaching relative stationary are determined as the material sorting time. If the material moves relative to the conveying device before reaching the sorting device, the material sorting time is determined based on the material's acceleration, the material's velocity when it reaches the color sorting camera, the conveying device's speed, and the distance between the color sorting camera and the sorting device. By calculating whether the material can reach a relatively stationary state with the conveying device before reaching the sorting device, the material movement pattern can be accurately determined, thereby optimizing the calculation logic for the sorting timing.

[0153] Based on the same inventive concept, this disclosure also provides a material sorting device. This material sorting device may include:

[0154] Conveying devices are used to transport materials;

[0155] The radiation device is located above the transmission device;

[0156] The color sorting camera is positioned above the conveyor and downstream of the ray device;

[0157] The material sorting device is connected to an X-ray device and a color sorting camera, respectively, and is used to determine the material category of the material by any of the material sorting methods provided in this disclosure;

[0158] The sorting device, located downstream of the conveyor, is used to sort materials based on their category.

[0159] Regarding the material sorting equipment in the above embodiments, the specific methods by which each module performs its operations have been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0160] Based on the same inventive concept, this disclosure also provides a computer-readable storage medium storing computer instructions, which are used to cause a computer to execute the material sorting method of any of the foregoing embodiments.

[0161] This disclosure uses specific terms to describe embodiments of the present disclosure. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of the present disclosure. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics in one or more embodiments of the present disclosure can be appropriately combined.

[0162] In the context of this disclosure, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0163] Similarly, it should be noted that, in order to simplify the description of this disclosure and thus aid in the understanding of one or more embodiments, the foregoing description of embodiments of this disclosure may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this disclosure method does not imply that the subject matter of this disclosure requires more features than the features claimed. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

[0164] The basic concepts have been described above. It is obvious that the above disclosure is merely illustrative and does not constitute a limitation of this disclosure. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to this disclosure by those skilled in the art. Such modifications, improvements, and corrections are suggested in this disclosure and therefore remain within the spirit and scope of the embodiments of this disclosure.

Claims

1. A material sorting method, wherein, An application is made in a material sorting device, the material sorting device comprising: a conveying device for conveying materials, an X-ray device disposed above the conveying device, and a color sorting camera disposed above the conveying device and downstream of the X-ray device, the method comprising: The X-ray image of the material is obtained through the X-ray device, and the color sorting image of the material is obtained through the color sorting camera; Based on the ray profile of the material in the ray image, the target color sorting profile of the material in the color sorting image is determined; Based on the color sorting image and the target color sorting outline, the material category of the material is determined; The materials are sorted based on the material category.

2. The material sorting method according to claim 1, wherein, Determining the target color sorting profile of the material in the color sorting image based on the ray profile of the material in the ray image includes: The ray profile of the material is determined based on the ray image, and the initial color sorting profile of the material is determined based on the color sorting image. Based on the ray profile and the initial color sorting profile, the motion state of the material relative to the conveying device is determined; If the motion state is motion, then based on the ray contour, the candidate contour corresponding to the ray contour in the color sorting image is adjusted to obtain the target color sorting contour of the material in the color sorting image.

3. The material sorting method according to claim 2, wherein adjusting the candidate contour corresponding to the ray contour in the color sorting image based on the ray contour to obtain the target color sorting contour of the material in the color sorting image includes: Based on the ray profile, the candidate profile corresponding to the ray profile in the color selection image is determined; Based on the position of the candidate contour and the background image in the color-sorted image, the background region in the candidate contour is determined; Based on the candidate contours, a first centroid is determined; Based on the background region, determine the second centroid corresponding to the non-background region in the candidate contour; The adjustment displacement is determined based on the first centroid and the second centroid; Based on the adjusted displacement, the candidate contour is adjusted to obtain the target color selection contour.

4. The material sorting method according to claim 3, wherein, The step of adjusting the candidate contour based on the adjusted displacement to obtain the target color selection contour includes: Based on the adjusted displacement, the candidate position of the candidate contour is obtained; Adjust the candidate position and / or the size of the candidate contour to obtain multiple intermediate contours; Based on the area of ​​the background image within each intermediate contour, the intermediate contour with the smallest area is taken as the target color selection contour.

5. The material sorting method according to claim 1, wherein, Determining the target color sorting profile of the material in the color sorting image based on the ray profile of the material in the ray image includes: Based on the ray image, a first contour corresponding to the target is determined; Based on the first contour, a second contour is determined in the color selection image; Based on the second contour, the foreground region and the background region are determined in the color-sorted image; Based on the color-selected image, a baseline value for the background grayscale is determined; Based on the benchmark value, candidate sub-regions in the background region are determined; Based on the candidate sub-region and the foreground region, the target region of the color sorting image is determined; The target color sorting outline of the material in the color sorting image is determined by the target region.

6. The material sorting method according to claim 5, wherein, Determining the target region of the color-sorted image based on the candidate sub-region and the foreground region includes: If the candidate sub-region is adjacent to the foreground region, then the candidate sub-region and the foreground region are merged to obtain the target region; If any one or more target conditions are met, the candidate sub-region is eliminated. The target conditions include: the candidate sub-region is not adjacent to the foreground region; the difference between the average gray value of the candidate sub-region and the reference value is less than a first approximate threshold; and the difference between the gray value of the offset position in the foreground region corresponding to the candidate sub-region and the reference value is greater than a second approximate threshold. The offset position is determined based on the first centroid position of the candidate sub-region and the second centroid position of the foreground region.

7. The material sorting method according to claim 1, wherein, Determining the material category of the material based on the color sorting image and the target color sorting outline includes: The color sorting image is input into the target recognition model to determine the multiple recognition boxes included in the color sorting image and the corresponding material types; The type of material is determined based on the outline, multiple recognition boxes, and the corresponding material type.

8. The material sorting method according to claim 7, wherein, The step of determining the type of material based on the contour, multiple recognition boxes, and the corresponding material type includes: If a first identification box exists among multiple identification boxes, then the material type is determined to be a first type, wherein the first identification box is an identification box that includes the entire area of ​​the outline and the corresponding material type is the first type; If the first identification box is not present in any of the multiple identification boxes, the type of the material is determined based on the area of ​​the second identification box, wherein the second identification box is an identification box that includes a partial outline region and the corresponding material type is the first type; If the material type corresponding to multiple identification boxes is the second type, then the material type is determined to be the second type.

9. The material sorting method according to claim 7, wherein, The training process of the target recognition model includes: Acquire multiple image samples and multiple recognition box information corresponding to each image sample, wherein the material types corresponding to the multiple recognition box information include a first type and / or a second type; Based on the multiple image samples and the multiple recognition box information corresponding to each image sample, an initial recognition model is trained to obtain the target recognition model.

10. The material sorting method according to claim 1, wherein, The sorting of materials based on the material category includes: The relative motion state between the material and the conveying device is determined based on the X-ray image, the color sorting image, and the mapping relationship between the X-ray image and the color sorting image. If it is determined that the material and the conveying device are in a relatively stationary state, then the sorting strategy of the material is determined based on the X-ray image, the distance between the color sorting camera and the X-ray device, and the operating speed of the conveying device, wherein the sorting strategy includes sorting time and sorting position; If it is determined that the material and the conveying device are in relative motion, then a sorting strategy is determined based on the motion blur of the material in the color sorting image; The materials are sorted according to the sorting strategy and the material category.

11. The material sorting method according to claim 10, wherein, The step of determining the sorting strategy based on the motion blur of the material in the color sorting image includes: Determine the degree of motion blur of the material in the color sorting image; If the degree of motion blur is less than the motion blur threshold, the material sorting strategy is determined based on the X-ray image, the distance between the color sorting camera and the sorting mechanism, and the operating speed of the conveying device. If the degree of motion ambiguity is greater than or equal to the motion ambiguity threshold, the material sorting time is determined based on the motion ambiguity of the material.

12. A material sorting device, wherein, The device includes: Conveying devices are used to transport materials; A radiation device is disposed above the conveying device; A color sorting camera is positioned above the conveying device and downstream of the ray device; A material sorting device, connected to the X-ray device and the color sorting camera respectively, is used to determine the material category of the material by the material sorting method as described in any one of claims 1-11; A sorting device is located downstream of the conveying device and is used to sort the materials based on the material category.

Images