Machine vision-based foreign matter identification system for coal mine belt conveyor

Abstract

The application belongs to the technical field of computer vision, and discloses a coal mine belt conveyor foreign matter identification system based on machine vision, which comprises the following steps: collecting a standardized image data set of a coal mine belt conveyor; performing target detection to obtain a candidate detection frame set and performing effectiveness determination to filter false detection frames to obtain an effective small target detection frame set; performing direction perception and separation reservation determination to output a target separation decision signal; performing target attribution aggregation determination to output a foreign matter entity correlation table; performing edge region determination and cross-frame trajectory correlation to output a foreign matter tracking state record; based on the foreign matter tracking state record, the attribute information of the foreign matter target is statistically confirmed, a foreign matter identification result data is generated, and the foreign matter identification result data is sent to a preset coal mine belt conveyor control terminal for application response; and the foreign matter identification result has high stability, accuracy and engineering application reliability.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a foreign object identification system for coal mine belt conveyors based on machine vision. Background Technology

[0002] Coal mine belt conveyors play a vital role in the continuous transport of coal. Since foreign objects are often mixed in with the materials in coal mines, it is of great significance to improve the ability to identify foreign objects, thereby reducing the risk of conveyor belt tearing and jamming, and ensuring continuous and safe production in coal mines. However, traditional foreign object identification systems still have many technical limitations when facing the more complex visual conditions in underground coal mines.

[0003] In practical applications, suspended dust in coal mines easily forms numerous dynamic bright spots under supplementary lighting, while reflective points on the surface of metallic foreign objects also appear as localized high-brightness areas. Traditional foreign object identification systems typically rely on the brightness or simple morphological features of images for judgment, lacking in-depth utilization of the motion patterns between consecutive image frames. This leads to easy confusion between dust bright spots and the reflective points of real foreign objects. Furthermore, slender foreign objects such as anchor bolts or reinforcing bars are often scattered in parallel within coal flow materials. Traditional foreign object identification systems often use a uniform overlap threshold for merging, failing to jointly analyze the extension direction, length overlap, and lateral spacing of slender targets. This easily leads to the mismerging of multiple independent parallel anchor bolts into a single target, resulting in distorted foreign object quantity identification results. Simultaneously, the supplementary lighting conditions for a single metallic foreign object in a coal mine further complicate the situation. Multiple spatially separated reflective points may form on the surface of a foreign object. Traditional foreign object identification systems lack correlation analysis of the motion patterns, spatial patterns, and relative positional stability among multiple small targets, and thus classify multiple reflective points on the same foreign object surface as the same foreign object entity, resulting in the same foreign object being repeatedly identified as multiple independent targets. On the other hand, when a foreign object target enters the field of view from the edge of the image, it is usually only partially visible. Traditional foreign object identification systems generally directly count the detected targets, lacking special judgment for edge targets, pending confirmation status management, and cross-frame integrity confirmation mechanisms. This leads to the repeated counting of the part of the same foreign object truncated in the edge region and the part that is completely in the image frame. This not only affects the accuracy of the foreign object identification results, but also reduces the reliability of subsequent belt conveyor operations such as alarm, deceleration, or shutdown.

[0004] In view of this, this application proposes a machine vision-based foreign object identification system for coal mine belt conveyors to solve the above problems. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, this application provides the following technical solution: a machine vision-based foreign object identification system for coal mine belt conveyors, comprising: The data acquisition module acquires a continuous video frame sequence of a coal mine belt conveyor and performs image preprocessing to output a standardized image dataset; the standardized image dataset includes the current frame image and historical frame images. The effective detection module performs target detection based on a standardized image dataset to obtain a set of candidate detection boxes, extracts inter-frame motion features from the candidate detection box set, performs validity determination based on the consistency between the inter-frame motion features and the conveyor belt motion direction, and filters out false detection boxes to obtain a set of effective small target detection boxes. The target separation module identifies elongated target detection boxes in the candidate detection box set and extracts geometric features. Based on the geometric features, it performs orientation perception, determines the separation and retention of elongated targets based on the perception results, and outputs a target separation decision signal. The foreign object association module performs fusion processing based on the target separation decision signal and the set of effective small target detection boxes, extracts motion consistency features from the fusion result, performs target affiliation aggregation determination on the motion consistency features, and outputs a foreign object entity association table. The target tracking module performs edge region judgment on all foreign object targets based on the foreign object entity association table, marks foreign object targets located in the edge region and performs cross-frame trajectory association, identifies the completeness of the target entering the field of view, updates the foreign object confirmation status, and outputs the foreign object tracking status record. The identification result generation module confirms the attribute information of the foreign object target based on the foreign object tracking status record, generates foreign object identification result data, and sends it to the preset coal mine belt conveyor control terminal to execute the application response; the various modules are connected to each other through wired and / or wireless means.

[0006] Furthermore, methods for extracting inter-frame motion features include: Set a sliding window and use it to perform a sliding scan of the current frame image, and calculate the mean gray value and the variance of gray value in the region corresponding to each sliding window. The sliding window regions with a mean grayscale value higher than the first grayscale threshold and a variance of grayscale value lower than the first variance threshold are selected as candidate highlight regions. Adjacent candidate highlight regions are connected and the smallest bounding rectangle is identified to obtain the candidate detection box of the corresponding region. Extract multiple reference frames from historical frames that have different time intervals from the current frame image. For each candidate detection box, perform an extended search along the conveyor belt movement direction in the corresponding spatial neighborhood of the multiple reference frames to obtain the matching position coordinates of the candidate detection box in each reference frame. Calculate the coordinate difference between the center point coordinates of the candidate detection box and the coordinates of each matching position, and calculate the displacement and displacement direction angle of the center point coordinates and the corresponding matching position coordinates based on the coordinate difference; Calculate the mean of all displacement direction angles and the deviation of each displacement direction angle from the mean. Combine all displacement amounts, all displacement direction angles, and deviations into the inter-frame motion features of the corresponding candidate detection boxes.

[0007] Furthermore, the methods for determining the validity of an action include: Obtain the conveyor belt running direction angle, and calculate the angle difference between each displacement direction angle and the conveyor belt running direction angle; calculate the average displacement of all displacements for each candidate detection frame, and at the same time calculate the displacement difference between the corresponding displacement of each candidate detection frame and the average displacement; The number of displacement direction angles whose included angle difference is less than the displacement included angle threshold for each candidate detection box is selected to obtain the number of times the direction is consistent; at the same time, the number of displacements whose displacement deviation value is less than the displacement deviation threshold is counted to obtain the number of times the displacement is stable. The number of times the orientation is consistent and the number of times the displacement is stable are normalized to obtain the orientation consistency ratio and the displacement stability ratio. The motion consistency index of each corresponding candidate detection box is calculated based on the orientation consistency ratio and the displacement stability ratio. Candidate detection boxes with a motion consistency index higher than a preset consistency threshold are selected as valid detection boxes, and the remaining candidate detection boxes are treated as false detection boxes and filtered to obtain a set of valid small target detection boxes.

[0008] Furthermore, the methods for performing direction awareness include: Identify the width and length of the bounding rectangle of each candidate detection box in the candidate detection box set, calculate the aspect ratio of the rectangle, and filter candidate detection boxes whose aspect ratio is higher than the preset slenderness ratio threshold as slender target detection boxes; Identify the contour boundary of the coverage area corresponding to each elongated target detection box, and calculate the coordinates of the boundary center point of the contour boundary; using the boundary center point coordinates as a reference, calculate the coordinate deviation values ​​of each boundary point on the contour boundary in the horizontal and vertical directions of the image frame respectively; The main extension direction of the corresponding slender target detection box is determined based on the coordinate deviation values ​​in the horizontal and vertical directions of the image frame, respectively. Calculate the extreme values ​​of the main projection distance and the vertical projection distance perpendicular to the main extension direction of the coverage area corresponding to the detection box of the slender target to obtain the main axis length and the secondary axis width; The main extension direction of each slender target detection box, along with the main axis length and secondary axis width in that direction, are integrated as the orientation sensing result.

[0009] Furthermore, the methods for determining separation and retention include: Extract the coordinates of the boundary center points of any two slender target detection boxes, and calculate the main projection distance and the perpendicular projection distance of the line connecting the two boundary center points in the corresponding main extension direction. Calculate the angle between the main directions of the main extension directions of any two slender target detection boxes, and select two slender target detection boxes whose main direction angle is less than a preset direction angle threshold as parallel candidate groups; The main projection overlap length of the two slender target detection boxes in each parallel candidate group is obtained based on the main projection distance of the connecting line in the main extension direction. The ratio of the main projection overlap length to the shorter main axis length is calculated to obtain the length overlap ratio. At the same time, the ratio of the vertical projection distance of the connecting line in each parallel candidate group to the larger secondary axis width is calculated to obtain the lateral spacing ratio. Filter parallel candidate groups whose length overlap ratio is higher than the preset overlap ratio and whose horizontal interval ratio is higher than the preset interval ratio, determine them as independent slender target combinations and generate separate retained fields; Parallel candidate groups with a length overlap ratio higher than a preset second overlap ratio and a horizontal interval ratio lower than a preset second interval ratio are selected as repeated combinations and a merge field is generated; a target separation decision signal corresponding to the slender target detection box is constructed based on the separation retention field and the merge field.

[0010] Furthermore, motion consistency features refer to the difference in displacement, the angle between displacement directions, and the distance between the center points of adjacent image frames for any two valid small target detection boxes in the fusion result.

[0011] Furthermore, the methods for performing target attribution aggregation determination include: Extract the displacement difference, displacement direction angle, and center point distance between any two valid small target detection boxes, and select two valid small target detection boxes whose displacement difference, displacement direction angle, and center point distance are all less than the corresponding preset threshold as the initial association candidate group; Calculate the difference in the center point distance between each initial association candidate group in adjacent image frames, and select the initial association candidate groups whose center point distance difference is less than a preset change threshold as stable association groups; Redundancy merging is performed on stable association groups with identical valid small target detection boxes to obtain corresponding foreign object entity groups; the number of valid small target detection boxes in each foreign object entity group is counted, and the area of ​​the outer region of the center point of all valid small target detection boxes in the corresponding foreign object entity group is calculated; If the number of detection boxes is not less than the preset aggregation number threshold and the area of ​​the outer region is less than the preset range threshold, then the corresponding foreign object entity group is marked as the same foreign object entity; a foreign object entity association table is constructed based on the labels of the effective small target detection boxes in each foreign object entity group and the labels of the corresponding foreign object entities.

[0012] Furthermore, the methods for performing edge region determination include: Extract the width and height of the latest current frame image, and extend them into the current frame image by a preset length based on each lateral boundary of the current frame image to form the edge region of the corresponding current frame image; Identify the center point of each foreign object target and the coordinates of the bounding rectangle of the corresponding detection area, calculate the edge distance from the center point of the foreign object to each side of the edge area; calculate the overlap length between the corresponding detection area and any side of the corresponding edge area based on the bounding rectangle coordinates. If the edge distance from the center point of the foreign object to any edge region is less than the preset edge distance threshold, or the overlap length between the detection area and any edge region is greater than the preset overlap length threshold, then the corresponding foreign object target is taken as an edge target and the corresponding edge region is marked.

[0013] Furthermore, methods for performing cross-frame trajectory association include: Add a pending confirmation status mark to edge targets with edge region markers, and identify the image frame number, foreign object center point and corresponding detection area area of ​​the first appearance of the edge target; Extract the center point sequence and area sequence of the same edge target in different image frames in consecutive image frames after the first image frame number; Calculate the displacement of the center point of the corresponding edge target in the foreign object center point sequence, identify the displacement direction of the center point displacement and determine whether the displacement direction is from the edge region to the interior of the corresponding image frame; at the same time, calculate the area change of the edge target in the area sequence and determine whether the area of ​​the corresponding detection region is in an increasing trend. If the distance from the center point of the foreign object of the edge target to the edge of all edge regions is greater than the preset complete distance threshold and the overlap length between the corresponding detection area and all edge regions is zero, then the edge target is determined to have completely entered the field of view. If the edge target fully enters the field of view, update the pending status marker of the edge target to the confirmed status marker and add a unique foreign object marker; By integrating the status marker, unique foreign object marker, corresponding continuous image frame sequence, foreign object center point sequence, and area sequence of each foreign object target, a foreign object tracking status record is obtained.

[0014] Furthermore, the methods for generating foreign object identification result data include: Foreign object targets with confirmed status markers are screened from the foreign object tracking status records. The unique foreign object marker, all associated image frame numbers, coordinates of the final foreign object center point, and maximum detection area are counted for each foreign object target. Foreign object identification result data is generated by combining and editing these data.

[0015] The technical effects and advantages of the machine vision-based foreign object identification system for coal mine belt conveyors in this application are as follows: A standardized image dataset was obtained by acquiring continuous video frames of a coal mine belt conveyor and performing image preprocessing. Compared with existing technologies, this method improves the ability to distinguish between dust bright spots and real foreign object reflective points, as well as the filtering accuracy of small target false detections, through inter-frame motion feature analysis and validity determination. Independent identification and false merging suppression of slender foreign objects such as parallel anchor bolts were achieved through directional perception and separation retention determination of slender targets. The target association accuracy and identification completeness of a single metal foreign object in multi-reflective point scenarios were improved through foreign object entity affiliation aggregation. The confirmation status update and duplicate count correction for edge-entry foreign objects were achieved through edge region judgment and cross-frame trajectory association. This ensures that the foreign object identification results have high stability, accuracy, and reliability for engineering applications. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the foreign object recognition system for coal mine belt conveyors based on machine vision, as described in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Example 1 Please see Figure 1 As shown in this embodiment, the foreign object identification system for coal mine belt conveyors based on machine vision includes: The data acquisition module acquires a continuous video frame sequence of a coal mine belt conveyor and performs image preprocessing to output a standardized image dataset; the standardized image dataset includes the current frame image and historical frame images. The effective detection module performs target detection based on a standardized image dataset to obtain a set of candidate detection boxes, extracts inter-frame motion features from the candidate detection box set, performs validity determination based on the consistency between the inter-frame motion features and the conveyor belt motion direction, and filters out false detection boxes to obtain a set of effective small target detection boxes. The target separation module identifies elongated target detection boxes in the candidate detection box set and extracts geometric features. Based on the geometric features, it performs orientation perception, determines the separation and retention of elongated targets based on the perception results, and outputs a target separation decision signal. The foreign object association module performs fusion processing based on the target separation decision signal and the set of effective small target detection boxes, extracts motion consistency features from the fusion result, performs target affiliation aggregation determination on the motion consistency features, and outputs a foreign object entity association table. The target tracking module performs edge region judgment on all foreign object targets based on the foreign object entity association table, marks foreign object targets located in the edge region and performs cross-frame trajectory association, identifies the completeness of the target entering the field of view, updates the foreign object confirmation status, and outputs the foreign object tracking status record. The identification result generation module confirms the attribute information of the foreign object target based on the foreign object tracking status record, generates foreign object identification result data, and sends it to the preset coal mine belt conveyor control terminal to execute the application response; the various modules are connected to each other through wired and / or wireless means.

[0019] In this embodiment, real-time monitoring of a belt conveyor in a coal mine scenario is used to obtain images of the conveyor belt and the materials on it, i.e., a continuous video frame sequence. Image preprocessing is achieved by sequentially performing image size unification and filtering and noise reduction on the original continuous video frame images to obtain a higher quality standardized image dataset. The standardized image dataset includes the current frame image acquired in real time at the current moment and historical frame images used for inter-frame comparison, as well as the acquisition time information corresponding to each image frame and the position information corresponding to each image frame in the direction of the conveyor belt.

[0020] Methods for extracting inter-frame motion features include: Set a sliding window and use it to perform a sliding scan of the current frame image, and calculate the mean gray value and the variance of gray value in the corresponding region of each sliding window.

[0021] The specific size of the sliding window is set based on the reflective area size of common small foreign objects in coal mine scenarios or the typical pixel range of dust bright spots. The sliding window is used to scan the current frame image region by region and calculate the gray mean and gray variance of the corresponding region of each sliding window. The gray mean is used to represent the overall brightness level of the region corresponding to the sliding window, and the gray variance is used to represent the dispersion of the pixel brightness distribution in the corresponding region.

[0022] Select the sliding window regions whose mean grayscale value is higher than the first grayscale threshold and whose variance of grayscale value is lower than the first variance threshold as candidate highlight regions.

[0023] The first grayscale threshold and the first variance threshold are set based on the brightness fluctuation range of coal mineral materials in theory, which are used to screen areas with relatively concentrated brightness changes; if the conditions of high grayscale mean and low grayscale variance are met at the same time, the area of ​​the corresponding sliding window can be used as a local candidate bright area.

[0024] Connect adjacent candidate highlighted regions and identify the smallest bounding rectangle to obtain the candidate detection box for the corresponding region.

[0025] In this embodiment, a region connectivity algorithm is used to perform region connectivity on adjacent candidate bright regions, merging spatially continuous or adjacent bright spot regions into a unified region. At the same time, the minimum bounding rectangle of the region is identified, and the candidate bright regions are converted into candidate detection boxes with position boundary and size information.

[0026] Extract multiple reference frames from historical frames that have different time intervals from the current frame image. For each candidate detection box, perform an extended search along the conveyor belt movement direction in the corresponding spatial neighborhood of the multiple reference frames to obtain the matching position coordinates of the candidate detection box in each reference frame.

[0027] The number of reference frames and the interval between different reference frames are set based on the operating speed of the coal mine belt conveyor, the video acquisition frame rate, or the normal displacement range of foreign objects in continuous image frames, in order to represent the positional changes of candidate detection boxes in continuous time periods.

[0028] Since real foreign objects move along with the conveyor belt during operation, an extended search is performed along the direction of the conveyor belt's movement. The extended search range can be set according to specific working conditions such as the conveyor belt speed, inter-frame time interval, and the conversion relationship of image spatial scale, to cover the location areas where real foreign objects may appear in adjacent time intervals. By obtaining the matching position coordinates of each candidate detection box in multiple reference frames, the displacement relationship of the candidate detection box in the time dimension is identified.

[0029] Calculate the coordinate difference between the center point coordinates of the candidate detection box and the coordinates of each matching position. Simultaneously, based on this coordinate difference, calculate the displacement and displacement direction angle between the center point coordinates and the corresponding matching position coordinates.

[0030] The matching position coordinates refer to the coordinates of the center point of the extended region in the reference frame. The formula for calculating the displacement is: ;in The x-coordinate of the center point of the candidate detection box is relative to the x-coordinate of the first detection box. The difference in the x-coordinates of the matching positions in each reference frame; The ordinate of the center point of the candidate detection box is relative to the first... The difference in the ordinate of the matching position coordinates of each reference frame; This indicates the amount of displacement.

[0031] The formula for calculating the displacement direction angle is: ;in Indicates the coordinates of the center point of the candidate detection box and the first... The displacement direction angle of the matching position coordinates of each reference frame.

[0032] Calculate the mean of all displacement direction angles and the deviation of each displacement direction angle from the mean. Combine all displacement amounts, all displacement direction angles, and deviations into the inter-frame motion features of the corresponding candidate detection boxes.

[0033] The mean of all displacement direction angles is calculated to represent the overall motion direction of the candidate detection box in multiple reference frames. The deviation of each displacement direction angle from the mean is calculated to represent the degree of dispersion of each displacement direction angle around the overall motion direction. The smaller the deviation, the more stable the motion direction of the candidate detection box is over multiple time intervals.

[0034] The methods for determining the validity of an execution include: Obtain the conveyor belt running direction angle and calculate the angle difference between each displacement direction angle and the conveyor belt running direction angle.

[0035] The conveyor belt running direction angle is used to indicate the overall direction of material movement during the operation of the coal mine belt conveyor; the included angle difference is used to indicate the degree of consistency between the movement direction of the candidate detection box and the overall running direction of the conveyor belt.

[0036] Calculate the average displacement of all displacements for each candidate detection box, and simultaneously calculate the displacement difference between the corresponding displacement of each candidate detection box and the average displacement.

[0037] The mean displacement value is used to represent the average displacement level of the candidate detection box across multiple reference frames, and the displacement difference value is used to represent the degree of fluctuation of each displacement value relative to the mean displacement level.

[0038] The number of displacement direction angles whose corresponding angle difference for each candidate detection box is less than the displacement angle threshold is selected to obtain the number of times the direction is consistent.

[0039] The displacement angle threshold is set based on the allowable normal directional offset range of the conveyor belt running direction. This threshold is used to determine whether the displacement direction of the candidate detection box in multiple reference frames is basically consistent with the conveyor belt running direction. The number of times the direction is consistent is used to indicate the degree to which the movement direction of the candidate detection box stably conforms to the conveyor belt direction.

[0040] Simultaneously, the number of times the displacement deviation value is less than the displacement deviation threshold is counted to obtain the displacement stabilization number.

[0041] The displacement deviation threshold is set based on the normal displacement fluctuation range of the foreign object in consecutive frames over a short period of time, which is used to determine whether the displacement of the candidate detection box remains stable in multiple reference frames; the number of displacement stabilization times is used to represent the degree of stability of the motion amplitude.

[0042] The number of times the direction is consistent and the number of times the displacement is stable are normalized to obtain the proportion of the direction consistency and the proportion of the displacement stability.

[0043] The difference in dimensions is eliminated by normalizing the number of times the orientation is consistent and the number of times the displacement is stable, so that the orientation consistency and displacement stability between different candidate detection boxes can be directly compared. The orientation consistency ratio is used to represent the stability of the motion direction of the candidate detection box in all reference frames, and the displacement stability ratio is used to represent the stability of the displacement amplitude.

[0044] The motion consistency index of each corresponding candidate detection box is calculated based on the orientation consistency ratio and displacement stability ratio.

[0045] The formula for calculating the motion consistency index is as follows: ;in Indicates a proportion that is consistent in direction. Indicates the proportion of stable displacement; This represents the motion consistency index.

[0046] Candidate detection boxes with a motion consistency index higher than a preset consistency threshold are selected as valid detection boxes, and the remaining candidate detection boxes are treated as false detection boxes and filtered to obtain a set of valid small target detection boxes.

[0047] The system sets a preset consistency threshold based on historical validity judgment experience to distinguish between targets that conform to the continuous motion law of the conveyor belt and targets that may be random bright spots. If the motion consistency index is higher than the threshold, it means that the motion direction and displacement amplitude of the candidate detection box in multiple reference frames are highly consistent with the motion state of the conveyor belt, so it can be judged as a valid detection box. By filtering false detection boxes, the false detection caused by dynamic bright spots in the coal mine dust environment can be effectively suppressed.

[0048] Methods of execution direction awareness include: Identify the width and length of the bounding rectangle of each candidate detection box in the candidate detection box set, calculate the aspect ratio of the rectangle, and filter candidate detection boxes whose aspect ratio is higher than the preset slenderness ratio threshold as slender target detection boxes.

[0049] Since the candidate detection box is a rectangle, the width and length of the bounding rectangle of each candidate detection box are identified, and the ratio is calculated to obtain the aspect ratio of the rectangle, which reflects the slenderness of the candidate detection box. In the scenario of coal mine belt conveyors, dangerous foreign objects such as anchor bolts, steel bars or iron bars usually appear as slender targets in the image, while blocky foreign objects such as coal blocks usually do not show slender features. Therefore, a preset slenderness ratio threshold is set based on the length and width distribution characteristics of common slender foreign objects in coal mines, which is used to preferentially filter targets with slender morphological features from the candidate detection box set.

[0050] Identify the contour boundary of the coverage area corresponding to each elongated target detection box, and calculate the coordinates of the boundary center point of the contour boundary.

[0051] In this embodiment, an edge detection algorithm is used to identify the contour of the area covered by each slender target detection box, and the center point coordinates of the contour are further identified based on geometric knowledge.

[0052] Using the coordinates of the boundary center point as a reference, calculate the coordinate deviation values ​​of each boundary point on the contour boundary in the horizontal and vertical directions of the image frame.

[0053] The coordinate deviation value of each boundary point is obtained by subtracting the x-coordinate and y-coordinate of each boundary point from the x-coordinate and y-coordinate of the boundary center point, respectively.

[0054] The main extension direction of the corresponding slender target detection box is determined based on the coordinate deviation values ​​in the horizontal and vertical directions of the image frame, respectively.

[0055] If the coordinate deviation of the boundary points on the contour boundary in the horizontal direction of the image is greater than the coordinate deviation in the vertical direction, it indicates that the slender target has a larger extension range in the horizontal direction; conversely, it indicates that the corresponding slender target has a larger extension range in the vertical direction. For the horizontal or vertical direction with a larger extension range, construct a direction vector, and form a sub-direction vector for the corresponding direction based on each coordinate deviation value on the horizontal or vertical side. Then, use all the sub-direction vectors on the corresponding side to perform vector fusion on the original direction vector to form the main extension direction, which is used to represent the direction in which the slender target is stretched in the image.

[0056] Calculate the extreme values ​​of the main projection distance and the vertical projection distance perpendicular to the main extension direction of the coverage area corresponding to the detection box of the slender target to obtain the main axis length and the secondary axis width.

[0057] The extreme value of the main projection distance refers to the maximum extension range of the coverage area of ​​the slender target in the main extension direction, and the extreme value of the vertical projection distance refers to the maximum lateral expansion range of the slender target in the direction perpendicular to the main extension direction. Therefore, the extreme value of the main projection distance is the main axis length, and the extreme value of the vertical projection distance is the secondary axis width, which is used to quantify the length scale and lateral width scale of the slender target in the main extension direction.

[0058] The main extension direction of each slender target detection box, along with the main axis length and secondary axis width along that direction, are integrated as the orientation perception result, which is used to represent the orientation and geometric scale attributes of the slender target.

[0059] The methods for determining separation and retention include: Extract the coordinates of the boundary center points of any two slender target detection boxes, and calculate the main projection distance and the perpendicular projection distance of the line connecting the two boundary center points in the corresponding main extension direction.

[0060] The line connecting the two boundary center points represents the relative positional relationship of the two slender target detection boxes in the image; the main projection distance of the line represents the relative positional offset of the two slender targets in the main extension direction; and the vertical projection distance of the line represents the lateral spacing distance of the two slender targets perpendicular to the main extension direction.

[0061] Calculate the angle between the main directions of any two slender target detection boxes, and select two slender target detection boxes whose main direction angle is less than a preset direction angle threshold as parallel candidate groups.

[0062] The main direction angle is obtained by subtracting the main extension directions of two slender target detection boxes, which is used to represent the degree of directional similarity between the two slender targets in the image. Based on the theoretical directional deviation range of slender foreign objects such as parallel anchors and inclined steel bars in coal mine conveyor belt images, a directional angle threshold is set, and slender target detection boxes that are sufficiently close in direction are selected to form a parallel candidate group.

[0063] The main projection overlap length of the two slender target detection boxes in each parallel candidate group is obtained based on the main projection distance of the connecting line, and the length overlap ratio is obtained by calculating the ratio of the main projection overlap length to the shorter main axis length.

[0064] The formula for calculating the overlap length of the principal projection is: ;in This represents the overlap length of the main projections of two elongated target detection boxes in each parallel candidate group along the main extension direction. This represents the projection range of one of the elongated target detection boxes A along the main extension direction. This represents the projection range of another elongated target detection box B in the main extension direction.

[0065] The shorter main axis length of the two slender target detection frames is used as a reference to indicate the degree to which the shorter slender target is covered by the other slender target in the main extension direction. If a longer main axis length is used, the overlap relationship in the main direction will be underestimated when the length difference between the two slender targets is large, which is not conducive to identifying repeated detection frames of the same slender object. The length overlap ratio is obtained by calculating the ratio of the main projection overlap length to the shorter main axis length of the two slender target detection frames, which is used to quantify the degree of fit between the two slender targets in the parallel candidate group in the length direction.

[0066] Simultaneously, the ratio of the vertical projection distance of the connecting lines in each parallel candidate group to the width of the larger secondary axis is calculated to obtain the lateral spacing ratio.

[0067] The lateral spacing ratio is obtained by calculating the ratio of the vertical projection distance of the connecting line to the larger secondary axis width of the two slender target detection boxes. This is to use the slender target with a larger lateral area as a reference to determine the degree of separation between two parallel candidate slender targets perpendicular to the main extension direction. If a smaller secondary axis width is used, when the lateral widths of the two slender targets differ greatly, the degree of lateral separation between them is easily overestimated, thus misjudging the duplicate detection boxes of the same slender object as independent parallel targets.

[0068] Parallel candidate groups with a length overlap ratio higher than a preset overlap ratio and a horizontal spacing ratio higher than a preset spacing ratio are selected as independent slender target combinations and separate retention fields are generated.

[0069] The system sets a preset overlap ratio and a preset interval ratio based on the theoretical overlap degree. The preset overlap ratio is used to limit the two parallel slender targets to have sufficient overlap in the main direction, so as to ensure that they belong to parallel targets within the same length range. The preset interval ratio is used to limit the two parallel slender targets to have sufficient lateral separation distance in the vertical direction, so as to exclude duplicate detection boxes of the same target. When the length overlap ratio is high and the lateral interval ratio is also high, it means that the two slender targets are in the same direction, have similar length regions, and are separated in the lateral direction. This is more in line with the scenario of multiple parallel anchor bolts distributed side by side on a coal mine conveyor belt. Therefore, a separation retention field is generated to mark the two slender targets in the parallel candidate group as independent targets and retain them at the same time.

[0070] Parallel candidate groups with a length overlap ratio higher than the preset second overlap ratio and a horizontal interval ratio lower than the preset second interval ratio are identified as duplicate combinations and a merge field is generated.

[0071] The preset second overlap ratio is used to filter two slender targets with a high degree of overlap in the main direction, and the preset second interval ratio is used to filter two slender targets with a sufficiently small interval in the vertical direction. When two slender targets simultaneously meet the requirements of a high length overlap ratio and a small lateral interval ratio, it indicates that the corresponding two slender targets are more likely to be repeated detection results of the same slender target rather than independent existence. Therefore, a merge field is added to mark that the parallel candidate group should be subject to target merging processing.

[0072] The target separation decision signal is constructed based on the separation retention field and the merging field for the corresponding slender target detection box. The target separation decision signal is obtained by setting the separation retention field or the merging field for the corresponding slender target detection box and converting it into a digital signal form.

[0073] Motion consistency features refer to the difference in displacement, the angle between displacement directions, and the distance between the center points of adjacent image frames for any two valid small target detection boxes in the fusion result.

[0074] The displacement difference represents the difference in the movement amplitude of any two effective small target detection boxes in consecutive image frames, the displacement direction angle represents the consistency between the movement directions of the two effective small target detection boxes, and the distance between the center points of adjacent image frames represents the relative proximity of the two effective small target detection boxes in spatial position.

[0075] The methods for performing target attribution aggregation determination include: Extract the displacement difference, displacement direction angle, and center point distance between any two valid small target detection boxes, and select two valid small target detection boxes whose displacement difference, displacement direction angle, and center point distance are all less than the corresponding preset thresholds as the initial association candidate group.

[0076] Based on relevant theoretical knowledge of actual coal mine working conditions, thresholds are set for the corresponding theoretical conditions of displacement difference, displacement direction angle and center point distance. If two valid small target detection boxes meet these three threshold conditions at the same time, candidate combinations that are more likely to belong to the same foreign object entity can be screened out to form the initial association candidate group.

[0077] Calculate the difference in center point distance between each initial association candidate group and adjacent image frames, and select initial association candidate groups whose center point distance difference is less than a preset change threshold as stable association groups.

[0078] The difference in center point distance refers to the degree of change in the relative spacing between two effective small target detection boxes in the initial association candidate group in consecutive image frames; the change threshold under theoretical conditions is set based on historical judgment experience to further screen out candidate combinations with stable relative positions as stable association groups.

[0079] Redundancy merging is performed on stable association groups with the same valid small target detection boxes to obtain the corresponding foreign object entity group.

[0080] Multiple stable associations with common detection boxes will be combined into the same entity set, which is called a heterogeneous entity group. For example, if the first detection box and the second detection box belong to a stable association group, and the second detection box and the third detection box also belong to a stable association group, it means that these three detection boxes may originate from the same heterogeneous entity and should be merged.

[0081] Count the number of valid small target detection boxes in each foreign object group, and calculate the area of ​​the circumscribed region of the center point of all valid small target detection boxes in the corresponding foreign object group.

[0082] The number of detection boxes in the foreign object entity group refers to the number of targets contained in the corresponding foreign object entity group, and the area of ​​the outer region represents the overall spatial distribution range of all effective small target detection boxes in the foreign object entity group.

[0083] If the number of detection boxes is not less than the preset aggregation quantity threshold and the area of ​​the outer region is less than the preset range threshold, then the corresponding foreign object entity group will be marked as the same foreign object entity.

[0084] The preset aggregation number threshold and preset range threshold are set based on the aggregation number and area size under theoretical conditions. The aggregation number threshold is used to filter the minimum number of reflective points that are usually formed on the surface of the same metal foreign object. The range threshold corresponds to the theoretical distribution area range of multiple reflective points on the same foreign object surface in the image. If the number of detection boxes reaches or exceeds the aggregation number threshold and the area of ​​the outer region is less than the range threshold, it means that the foreign object entity group contains a sufficient number of small target detection boxes and meets the local concentrated distribution characteristics. Therefore, it can be marked as the same foreign object entity.

[0085] A foreign object entity association table is constructed based on the labels of the valid small target detection boxes in each foreign object entity group and the labels of the corresponding foreign objects.

[0086] The label of the effective small target detection box in each foreign object group is the separation retention field or merging field in the target separation decision signal. It is then associated with the corresponding foreign object label and converted into a table to obtain the foreign object association table.

[0087] The methods for performing edge region determination include: Extract the width and height of the latest current frame image, and extend them into the current frame image by a preset length based on each lateral boundary of the current frame image to form the edge region of the corresponding current frame image.

[0088] The latest current frame image refers to the latest real-time acquired current frame image. The width and height of this image are used to determine the boundary range of the image. The lateral direction refers to the four sides of the current frame image. Each lateral edge region is formed by extending a certain length from the lateral boundary into the image. The preset length is set based on the theoretical cutoff range when a foreign object enters the field of view.

[0089] Identify the center point of each foreign object target and the coordinates of the bounding rectangle of the corresponding detection area, and calculate the edge distance from the center point of the foreign object to each side of the edge area.

[0090] The foreign object center point refers to the center point of the foreign object's contour, which is equivalent to the coordinates of the boundary center point of the contour boundary; the coordinates of the circumscribed rectangle are used to represent the spatial boundary of the detection area corresponding to the foreign object; the edge distance from the foreign object center point to each lateral edge region is used to determine whether the corresponding foreign object is close to the image edge.

[0091] The overlap length between the corresponding detection area and any side of the corresponding edge area is calculated based on the coordinates of the circumscribed rectangle.

[0092] The detection area refers to the area formed by the corresponding circumscribed rectangle. The overlap length between the detection area and the edge area on the corresponding side is calculated to determine whether the foreign object has partially entered the edge area.

[0093] If the edge distance from the center point of the foreign object to any edge region is less than the preset edge distance threshold, or the overlap length between the detection area and any edge region is greater than the preset overlap length threshold, then the corresponding foreign object target is taken as an edge target and the corresponding edge region is marked.

[0094] The threshold for edge distance is set based on the theoretical range of motion of the foreign object in the edge area, and the threshold for overlap length is set based on the theoretical effective overlap range when the foreign object comes into contact with the edge area. If the foreign object meets any of the threshold conditions, it can be identified as an edge object and the corresponding edge area mark is added.

[0095] Methods for performing cross-frame trajectory association include: Add a pending confirmation status mark to edge targets with edge region markers, and identify the image frame number, foreign object center point and corresponding detection area area of ​​the first appearance of the edge target.

[0096] The pending status marker indicates that the corresponding edge target has not fully entered the field of view and cannot be officially counted as a confirmed foreign object target; the first image frame number refers to the image frame in which the corresponding edge target first enters the field of view.

[0097] Extract the center point sequence and area sequence of the same edge target in different image frames in the consecutive image frames after the first image frame number.

[0098] The foreign object center point sequence represents the motion trajectory of the same edge target in consecutive image frames, and the area sequence represents the change process of the visible area of ​​the edge target in consecutive image frames.

[0099] Calculate the displacement of the center point of the corresponding edge target in the foreign object center point sequence, identify the displacement direction of the center point displacement, and determine whether the displacement direction is from the edge region to the interior of the corresponding image frame.

[0100] The formula for calculating the displacement of the center point is: ;in, This represents the displacement of the center point of the corresponding edge target; The ordinate represents the final position of the corresponding edge target in the sequence of foreign object center points. The ordinate represents the initial position of the corresponding edge target in the sequence of foreign object center points; This represents the x-coordinate of the final position of the corresponding edge target in the sequence of foreign object center points. It represents the x-coordinate of the initial position of the corresponding edge target in the sequence of foreign object center points; the displacement direction is obtained by identifying the direction from the initial position to the final position, and it is determined whether the displacement direction points into the image frame.

[0101] Simultaneously, the area change of the edge target in the area sequence is calculated to determine whether the area of ​​the corresponding detection region is in an increasing trend.

[0102] The formula for calculating the change in area is: ;in Indicates the first The area of ​​the detection region in the frame of the image is related to the first frame. The change in the area of ​​the detection region within a frame of an image; Indicates the first Area of ​​the detection region in the frame of the image. Indicates the first The area of ​​the detected region in the frame image is determined by taking the first derivative of the area sequence to determine the direction of area change, and then to determine whether the area of ​​the corresponding detected region is in an increasing trend.

[0103] If the distance from the center point of the foreign object to the edge of all edge regions is greater than the preset complete distance threshold and the overlap length between the corresponding detection area and all edge regions is zero, then the edge target is determined to have completely entered the field of view.

[0104] The complete distance threshold is set based on the theoretical safe distance maintained between the foreign object and the edge region after the foreign object has completely entered the image. This threshold is used to determine whether the center point of the foreign object has been removed from all edge-sensitive areas. If the overlap length between the detection area and all edge regions is zero, it means that the outer rectangular area of ​​the foreign object has been completely removed from the image edge region. Only when both the center point of the foreign object and the detection area meet these two conditions can it be said that the edge object has completely entered the current image field of view.

[0105] If an edge target fully enters the field of view, update the pending status marker of the edge target to the confirmed status marker and add a unique foreign object marker.

[0106] The confirmed status marker indicates that the corresponding foreign object has fully entered the field of view. The unique foreign object marker is used to distinguish different foreign objects, so that the tracking and statistics of the same foreign object in subsequent consecutive image frames all correspond to the same unique identifier.

[0107] By integrating the status marker, unique foreign object marker, corresponding continuous image frame sequence, foreign object center point sequence, and area sequence of each foreign object target, a foreign object tracking status record is obtained.

[0108] The foreign object tracking status record is used to record the status changes, unique identification, trajectory position changes, and area changes of each foreign object target in consecutive image frames.

[0109] The methods for generating foreign object identification result data include: Foreign object targets with confirmed status markers are screened from the foreign object tracking status records. The unique foreign object marker, all associated image frame numbers, coordinates of the final foreign object center point, and maximum detection area are counted for each foreign object target. Foreign object identification result data is generated by combining and editing these data.

[0110] By combining and editing the attribute information related to each foreign object target, the status information, time information, location information and scale information scattered in the foreign object tracking status record are integrated into foreign object identification result data in a unified format, which is then applied to the preset coal mine belt conveyor control terminal to perform response operations such as alarm, deceleration or shutdown.

[0111] This embodiment obtains a standardized image dataset by acquiring continuous video frames of a coal mine belt conveyor and performing image preprocessing. Compared with existing technologies, it improves the ability to distinguish between dust bright spots and real foreign object reflective points and enhances the filtering accuracy of small target false detections by performing inter-frame motion feature analysis and validity determination. By performing slender target direction perception and separation retention determination, it achieves independent identification and false merging suppression of slender foreign objects such as parallel anchor bolts. By performing foreign object entity affiliation aggregation, it improves the target association accuracy and identification completeness of a single metal foreign object in multi-reflective point scenarios. By performing edge region judgment and cross-frame trajectory association, it achieves confirmation status update and duplicate count correction for foreign objects entering from the edge. This ensures that the foreign object identification results have high stability, accuracy, and reliability for engineering applications.

[0112] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0113] All formulas in this manual are dimensionless and calculated numerically. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0114] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A foreign object identification system for coal mine belt conveyors based on machine vision, characterized in that, include: The data acquisition module acquires a continuous video frame sequence of a coal mine belt conveyor and performs image preprocessing to output a standardized image dataset; the standardized image dataset includes the current frame image and historical frame images. The effective detection module performs target detection based on a standardized image dataset to obtain a set of candidate detection boxes, extracts inter-frame motion features from the candidate detection box set, performs validity determination based on the consistency between the inter-frame motion features and the conveyor belt motion direction, and filters out false detection boxes to obtain a set of effective small target detection boxes. The target separation module identifies elongated target detection boxes in the candidate detection box set and extracts geometric features. Based on the geometric features, it performs orientation perception, determines the separation and retention of elongated targets based on the perception results, and outputs a target separation decision signal. The foreign object association module performs fusion processing based on the target separation decision signal and the set of effective small target detection boxes, extracts motion consistency features from the fusion result, performs target affiliation aggregation determination on the motion consistency features, and outputs a foreign object entity association table. The target tracking module performs edge region judgment on all foreign object targets based on the foreign object entity association table, marks foreign object targets located in the edge region and performs cross-frame trajectory association, identifies the completeness of the target entering the field of view, updates the foreign object confirmation status, and outputs the foreign object tracking status record. The identification result generation module confirms the attribute information of the foreign object target based on the foreign object tracking status record, generates foreign object identification result data, and sends it to the preset coal mine belt conveyor control terminal to execute the application response; the various modules are connected to each other through wired and / or wireless means.

2. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 1, characterized in that, The methods for extracting inter-frame motion features include: Set a sliding window and use it to perform a sliding scan of the current frame image, and calculate the mean gray value and the variance of gray value in the region corresponding to each sliding window. The sliding window regions with a mean grayscale value higher than the first grayscale threshold and a variance of grayscale value lower than the first variance threshold are selected as candidate highlight regions. Adjacent candidate highlight regions are connected and the smallest bounding rectangle is identified to obtain the candidate detection box of the corresponding region. Extract multiple reference frames from historical frames that have different time intervals from the current frame image. For each candidate detection box, perform an extended search along the conveyor belt movement direction in the corresponding spatial neighborhood of the multiple reference frames to obtain the matching position coordinates of the candidate detection box in each reference frame. Calculate the coordinate difference between the center point coordinates of the candidate detection box and the coordinates of each matching position, and calculate the displacement and displacement direction angle of the center point coordinates and the corresponding matching position coordinates based on the coordinate difference; Calculate the mean of all displacement direction angles and the deviation of each displacement direction angle from the mean. Combine all displacement amounts, all displacement direction angles, and deviations into the inter-frame motion features of the corresponding candidate detection boxes.

3. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 2, characterized in that, The methods for determining the validity of the execution include: Obtain the conveyor belt running direction angle, and calculate the angle difference between each displacement direction angle and the conveyor belt running direction angle; calculate the average displacement of all displacements for each candidate detection frame, and at the same time calculate the displacement difference between the corresponding displacement of each candidate detection frame and the average displacement; The number of displacement direction angles whose included angle difference is less than the displacement included angle threshold for each candidate detection box is selected to obtain the number of times the direction is consistent; at the same time, the number of displacements whose displacement deviation value is less than the displacement deviation threshold is counted to obtain the number of times the displacement is stable. The number of times the orientation is consistent and the number of times the displacement is stable are normalized to obtain the orientation consistency ratio and the displacement stability ratio. The motion consistency index of each corresponding candidate detection box is calculated based on the orientation consistency ratio and the displacement stability ratio. Candidate detection boxes with a motion consistency index higher than a preset consistency threshold are selected as valid detection boxes, and the remaining candidate detection boxes are treated as false detection boxes and filtered to obtain a set of valid small target detection boxes.

4. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 3, characterized in that, The methods for sensing the execution direction include: Identify the width and length of the bounding rectangle of each candidate detection box in the candidate detection box set, calculate the aspect ratio of the rectangle, and filter candidate detection boxes whose aspect ratio is higher than the preset slenderness ratio threshold as slender target detection boxes; Identify the contour boundary of the coverage area corresponding to each elongated target detection box, and calculate the coordinates of the boundary center point of the contour boundary; using the boundary center point coordinates as a reference, calculate the coordinate deviation values ​​of each boundary point on the contour boundary in the horizontal and vertical directions of the image frame respectively; The main extension direction of the corresponding slender target detection box is determined based on the coordinate deviation values ​​in the horizontal and vertical directions of the image frame, respectively. Calculate the extreme values ​​of the main projection distance and the vertical projection distance perpendicular to the main extension direction of the coverage area corresponding to the detection box of the slender target to obtain the main axis length and the secondary axis width; The main extension direction of each slender target detection box, along with the main axis length and secondary axis width in that direction, are integrated as the orientation sensing result.

5. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 4, characterized in that, The methods for determining separation and retention include: Extract the coordinates of the boundary center points of any two slender target detection boxes, and calculate the main projection distance and the perpendicular projection distance of the line connecting the two boundary center points in the corresponding main extension direction. Calculate the angle between the main directions of the main extension directions of any two slender target detection boxes, and select two slender target detection boxes whose main direction angle is less than a preset direction angle threshold as parallel candidate groups; The main projection overlap length of the two slender target detection boxes in each parallel candidate group is obtained based on the main projection distance of the connecting line in the main extension direction. The ratio of the main projection overlap length to the shorter main axis length is calculated to obtain the length overlap ratio. At the same time, the ratio of the vertical projection distance of the connecting line in each parallel candidate group to the larger secondary axis width is calculated to obtain the lateral spacing ratio. Filter parallel candidate groups whose length overlap ratio is higher than the preset overlap ratio and whose horizontal interval ratio is higher than the preset interval ratio, determine them as independent slender target combinations and generate separate retained fields; Parallel candidate groups with a length overlap ratio higher than a preset second overlap ratio and a horizontal interval ratio lower than a preset second interval ratio are selected as repeated combinations and a merge field is generated; a target separation decision signal corresponding to the slender target detection box is constructed based on the separation retention field and the merge field.

6. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 5, characterized in that, The motion consistency features refer to the difference in displacement, the angle between displacement directions, and the distance between the center points of adjacent image frames of any two valid small target detection boxes in the fusion result.

7. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 6, characterized in that, The methods for determining the aggregation of execution targets include: Extract the displacement difference, displacement direction angle, and center point distance between any two valid small target detection boxes, and select two valid small target detection boxes whose displacement difference, displacement direction angle, and center point distance are all less than the corresponding preset threshold as the initial association candidate group; Calculate the difference in the center point distance between each initial association candidate group in adjacent image frames, and select the initial association candidate groups whose center point distance difference is less than a preset change threshold as stable association groups; Redundancy merging is performed on stable association groups with identical valid small target detection boxes to obtain corresponding foreign object entity groups; the number of valid small target detection boxes in each foreign object entity group is counted, and the area of ​​the outer region of the center point of all valid small target detection boxes in the corresponding foreign object entity group is calculated; If the number of detection boxes is not less than the preset aggregation number threshold and the area of ​​the outer region is less than the preset range threshold, then the corresponding foreign object entity group is marked as the same foreign object entity; a foreign object entity association table is constructed based on the labels of the effective small target detection boxes in each foreign object entity group and the labels of the corresponding foreign object entities.

8. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 7, characterized in that, The methods for performing edge region determination include: Extract the width and height of the latest current frame image, and extend them into the current frame image by a preset length based on each lateral boundary of the current frame image to form the edge region of the corresponding current frame image; Identify the center point of each foreign object target and the coordinates of the bounding rectangle of the corresponding detection area, calculate the edge distance from the center point of the foreign object to each side of the edge area; calculate the overlap length between the corresponding detection area and any side of the corresponding edge area based on the bounding rectangle coordinates. If the edge distance from the center point of the foreign object to any edge region is less than the preset edge distance threshold, or the overlap length between the detection area and any edge region is greater than the preset overlap length threshold, then the corresponding foreign object target is taken as an edge target and the corresponding edge region is marked.

9. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 8, characterized in that, The methods for performing cross-frame trajectory association include: Add a pending confirmation status mark to edge targets with edge region markers, and identify the image frame number, foreign object center point and corresponding detection area area of ​​the first appearance of the edge target; Extract the center point sequence and area sequence of the same edge target in different image frames in consecutive image frames after the first image frame number; Calculate the displacement of the center point of the corresponding edge target in the foreign object center point sequence, identify the displacement direction of the center point displacement and determine whether the displacement direction is from the edge region to the interior of the corresponding image frame; at the same time, calculate the area change of the edge target in the area sequence and determine whether the area of ​​the corresponding detection region is in an increasing trend. If the distance from the center point of the foreign object of the edge target to the edge of all edge regions is greater than the preset complete distance threshold and the overlap length between the corresponding detection area and all edge regions is zero, then the edge target is determined to have completely entered the field of view. If the edge target fully enters the field of view, update the pending status marker of the edge target to the confirmed status marker and add a unique foreign object marker; By integrating the status marker, unique foreign object marker, corresponding continuous image frame sequence, foreign object center point sequence, and area sequence of each foreign object target, a foreign object tracking status record is obtained.

10. The foreign object identification system for coal mine belt conveyors based on machine vision according to claim 9, characterized in that, The methods for generating foreign object identification result data include: Foreign object targets with confirmed status markers are screened from the foreign object tracking status records. The unique foreign object marker, all associated image frame numbers, coordinates of the final foreign object center point, and maximum detection area are counted for each foreign object target. Foreign object identification result data is generated by combining and editing these data.

Images