Dense medium shallow slot sorting machine intelligent protection method and device based on key point detection, computer equipment and readable storage medium

Abstract

The invention discloses a dense-medium shallow slot sorting machine intelligent protection method and device based on key point detection, computer equipment and a readable storage medium, and the method comprises the steps: firstly, obtaining shallow slot scraper video data collected by a camera, preprocessing the data, inputting the data into a key point detection model to obtain scraper key points, and then calculating scraper inclination and deformation angles; and when abnormity exists, a shutdown instruction is initiated, and an alarm signal is output, so that safe and stable operation of equipment can be effectively guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent maintenance, and in particular to an intelligent protection method, device, computer equipment and readable storage medium for a heavy medium shallow trough separator based on key point detection. Background Art

[0002] In industries such as coal washing, heavy medium shallow trough separators are key equipment, and the operating status of their shallow trough scrapers directly affects production. Traditional protection technologies have defects and are difficult to monitor the scraper status in real time and accurately. For example, deformation, chain skipping, chain breaking, and other problems of the scraper may not be discovered in time, resulting in equipment damage and affecting production efficiency and safety. Summary of the invention

[0003] The object of the present invention is to provide an intelligent protection method, device, computer equipment and readable storage medium for a heavy medium shallow trough sorting machine based on key point detection.

[0004] In a first aspect, an embodiment of the present invention provides an intelligent protection method for a heavy-medium shallow trough sorter based on key point detection, which is applied to an edge device included in an intelligent protection system for a heavy-medium shallow trough sorter, wherein the intelligent protection system for a heavy-medium shallow trough sorter also includes a camera, and the method includes:

[0005] Acquiring video data of a shallow groove scraper included in the heavy medium shallow groove separator collected by the camera; the camera is located at a preset position facing the movement direction of the shallow groove scraper;

[0006] Preprocessing the video data to obtain preprocessed image data;

[0007] Inputting the pre-processed image data into a pre-trained key point detection model to obtain the key points of the shallow groove scraper;

[0008] The scraper pull angle and the scraper deformation angle of the shallow groove scraper are calculated according to the key points;

[0009] Determining the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle;

[0010] When the abnormal state is characterized by the presence of an abnormality, a shutdown command is initiated and an alarm signal is output.

[0011] In a possible implementation manner, preprocessing the video data to obtain preprocessed image data includes:

[0012] Acquire the video frame image included in the video data;

[0013] Based on the preset detection range area, each of the video frame images is cropped, filtered, denoised and histogram equalized in sequence to obtain the preprocessed image data; the shallow groove scraper is located within the preset detection range area, and the preprocessed image data is the image data of the movement direction of the shallow groove scraper.

[0014] In a possible implementation, the inputting the preprocessed image data into a pre-trained key point detection model to obtain the key points of the shallow groove scraper includes:

[0015] The preprocessed image data is input into a pre-trained key point detection model to obtain the left key point, the right key point and the middle key point of the shallow groove scraper in the preprocessed image data.

[0016] In a possible implementation, the inputting the pre-processed image data into a pre-trained key point detection model to obtain the left key point, the right key point and the middle key point of the shallow groove scraper in the pre-processed image data includes:

[0017] Inputting the preprocessed image data into a pre-trained key point detection model to obtain a plurality of first pending detection objects and a confidence level corresponding to each of the first pending detection objects;

[0018] Filtering the first pending detection objects whose confidence levels are lower than a preset confidence threshold to obtain a plurality of second pending detection objects;

[0019] Obtaining a target frame size and aspect ratio of each of the second to-be-determined detection objects;

[0020] Performing secondary filtering on the target frame size and length-width ratio of each of the second pending detection objects according to the preset distance between key points and the length-width ratio, to obtain a plurality of third pending detection objects;

[0021] The third pending detection object with a relatively large intersection is filtered out by non-maximum suppression to obtain the left key point, the right key point and the middle key point.

[0022] In a possible implementation manner, the calculating of the scraper bevel angle and the scraper deformation angle of the shallow groove scraper according to the key points includes:

[0023] The angle between the straight line formed by connecting the left key point and the right key point and the reference line is used as the scraper bevel angle; the reference line is parallel to the upper edge line of the shallow groove scraper in normal operation;

[0024] The angle between the straight line connecting the left key point and the middle key point and the straight line connecting the right key point and the middle key point is used as the scraper deformation angle.

[0025] In a possible implementation manner, determining the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle includes:

[0026] Determine whether a scraper chain jumping alarm condition or a scraper chain breaking alarm condition is met according to the scraper inclined angle;

[0027] Determining whether a scraper deformation alarm condition is met according to the scraper deformation angle;

[0028] If satisfied, then determining the abnormal state corresponding to the heavy medium shallow trough separator;

[0029] If not, it is determined that there is no abnormality in the heavy medium shallow trough separator.

[0030] In a possible implementation manner, when the abnormal state is characterized as an abnormality, initiating a shutdown instruction and outputting an alarm signal includes:

[0031] When the abnormal state is characterized by at least one of the following abnormal states: scraper chain jumping, scraper chain breaking and scraper deformation, a stop command of the heavy medium shallow trough separator is sent to the main PLC through hard wiring to control the scraper of the heavy medium shallow trough separator to stop, and the alarm signal is sent to the alarm platform through the network; the alarm signal includes the current image captured by the camera.

[0032] In a second aspect, an embodiment of the present invention provides an intelligent protection device for a heavy-medium shallow trough sorting machine based on key point detection, which is applied to an edge device included in an intelligent protection system for a heavy-medium shallow trough sorting machine, wherein the intelligent protection system for a heavy-medium shallow trough sorting machine also includes a camera, and the device includes:

[0033] An acquisition module is used to acquire video data of a shallow groove scraper included in a heavy medium shallow groove separator collected by the camera; the camera is located at a preset position facing the movement direction of the shallow groove scraper; the video data is preprocessed to obtain preprocessed image data; the preprocessed image data is input into a pre-trained key point detection model to obtain key points of the shallow groove scraper; and the scraper pull angle and the scraper deformation angle of the shallow groove scraper are calculated according to the key points;

[0034] The protection module is used to determine the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle; when the abnormal state is characterized by the existence of an abnormality, initiate a shutdown command and output an alarm signal.

[0035] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor and a non-volatile memory storing computer instructions, wherein when the computer instructions are executed by the processor, the computer device executes the method described in the first aspect.

[0036] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, wherein the readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method described in the first aspect.

[0037] Compared with the prior art, the beneficial effects provided by the present invention include: adopting an intelligent protection method, device, computer equipment and readable storage medium for a heavy medium shallow trough sorting machine based on key point detection disclosed in the present invention, by acquiring shallow trough scraper video data collected by a camera, inputting the key point detection model after preprocessing to obtain the scraper key points, and then calculating the scraper tilting and deformation angles, thereby determining the abnormal state of the sorting machine, initiating a shutdown command and outputting an alarm signal when an abnormality exists, which can effectively ensure the safe and stable operation of the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0039] Figure 1 A schematic flow chart of the steps of an intelligent protection method for a heavy medium shallow trough separator based on key point detection provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the locations of key points provided in an embodiment of the present invention;

[0041] Figure 3 A schematic block diagram of the structure of an intelligent protection device for a heavy medium shallow trough separator based on key point detection provided by an embodiment of the present invention;

[0042] Figure 4 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0044] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0045] In order to solve the technical problems in the aforementioned background technology, Figure 1 A flow chart of an intelligent protection method for a heavy-medium shallow trough sorting machine based on key point detection is provided for an embodiment of the present disclosure. The method is applied to an edge device included in an intelligent protection system for a heavy-medium shallow trough sorting machine. The intelligent protection system for a heavy-medium shallow trough sorting machine also includes a camera. The intelligent protection method for a heavy-medium shallow trough sorting machine based on key point detection is introduced in detail below.

[0046] Step S201, obtaining video data of a shallow groove scraper included in a heavy medium shallow groove separator collected by the camera; the camera is located at a preset position facing the movement direction of the shallow groove scraper;

[0047] Step S202, preprocessing the video data to obtain preprocessed image data;

[0048] Step S203, inputting the pre-processed image data into a pre-trained key point detection model to obtain the key points of the shallow groove scraper;

[0049] Step S204, calculating the scraper bevel angle and the scraper deformation angle of the shallow groove scraper according to the key points;

[0050] Step S205, determining the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle;

[0051] Step S206: When the abnormal state is characterized by the presence of an abnormality, a shutdown command is initiated and an alarm signal is output.

[0052] In the embodiment of the present invention, for example, in a typical coal washing plant, a heavy medium shallow trough separator is a key equipment for coal sorting. In order to realize intelligent protection of the equipment, a network camera is installed obliquely above the shallow trough scraper. This position is the preset position, and its viewing angle faces the direction of the scraper movement.

[0053] As the core processing unit of the entire system, the server is always ready to receive data from the camera. For example, in the daily production process of the coal washing plant, the heavy medium shallow trough separator starts to operate, and the shallow trough scraper starts to move along the predetermined track. At this time, the network camera located diagonally above starts to work, continuously collecting real-time video streams of the shallow trough scraper at a rate of 30 frames per second. These video stream data are transmitted through the switch and sent to the server in the form of data packets. The specific program module in the server is responsible for receiving these data packets and reassembling them into complete video data in preparation for subsequent processing.

[0054] After receiving the video data, the server will perform a series of preprocessing operations on it.

[0055] First, the server extracts images frame by frame from the video data. Assume that the currently received frame of video image contains not only the shallow groove scraper, but also the surrounding fences and other areas that may affect the detection effect. The server program will crop this frame of image according to the ROI (detection range) area information specified in advance. For example, the manually specified ROI area is a rectangular box, which can just frame all the scrapers in the picture and exclude other irrelevant targets, such as the fence area. After cropping, only the part related to the scraper remains in the image, which greatly reduces the false detection rate of the subsequent model.

[0056] Next, the server will filter and denoise the cropped image. Due to the complex on-site environment, there may be dim light, blurred images, etc., and the image will contain some noise points. The server uses the median filter algorithm to filter the image to remove these noise points and make the image clearer. For example, for a certain pixel in the image, the median filter algorithm will take the median value of the pixel values ​​in its surrounding neighborhood to replace the value of the pixel, thereby effectively removing isolated noise points.

[0057] In addition, in order to highlight the edge of the scraper, the server will also improve the contrast of the image through histogram equalization. For example, the grayscale distribution of the original image is uneven, resulting in unclear scraper edges. After histogram equalization processing, the grayscale distribution of the image becomes more uniform, and the contrast between the scraper edge and the background is enhanced, making the scraper outline clearer, which is convenient for subsequent key point detection.

[0058] After the server completes the preprocessing of the image data, it will be input into the pre-trained key point detection model. This model is trained based on the improved yolov8-pose algorithm.

[0059] When training the model, the staff first collected and manually annotated a data set of more than 8,000 images, which covered the status of shallow groove scrapers under various working conditions. For example, in some images, the scraper is in normal operation, and in some images, the scraper has different degrees of deformation or tilt. Then, data enhancement operations are performed on these image data, such as random rotation, flipping, scaling, etc., to increase the generality of the data. Through these data and optimization methods, a model suitable for shallow groove scraper key point detection was finally trained and deployed to the edge computing platform where the server is located.

[0060] When the preprocessed image data is input to the model, the model will analyze and reason about the image. For example, for an input frame of image, the model will search for possible scraper targets in the image and generate a target box and corresponding key points and their confidence for each target. Assuming there are multiple scrapers in the image, the model will generate corresponding target boxes and key point information for each scraper.

[0061] Next, the server will post-process the output results of the model. Specifically, it will first filter out the identified target objects with a confidence level exceeding the threshold and exclude the target objects with low confidence levels. For example, if the confidence level is set to 0.8, the server will consider the target objects with a confidence level lower than 0.8 as the result of misidentification and exclude them.

[0062] The server will then compare the target box size and its aspect ratio, and the distance between key points and their ratio with the mean, and further filter out targets that differ greatly from the mean. These means are calculated from historical data. For example, according to historical data statistics, the average aspect ratio of the target box of a normal scraper is 2:1, and the average aspect ratio of the distance between key points is 1:1. If the target box aspect ratio of a target object is 4:1, and the distance ratio between key points is 3:1, which is significantly different from the mean, the server will consider the target to be abnormal and filter it out.

[0063] Finally, the server will further filter out the recognized objects with large IOU (intersection over union) by non-maximum suppression to avoid re-recognition. For example, if there are two target boxes in the image and their IOU value is 0.9, it means that the two target boxes are highly overlapped and are likely to be repeated recognitions of the same scraper. The server will retain one of the target boxes and filter out the other according to certain rules.

[0064] After the post-processing steps, the server finally outputs the key point coordinates of each identified object, that is, the three key points on the upper edge of each scraper, namely the left key point, the middle key point, and the right key point.

[0065] After obtaining the coordinates of the key points of the shallow groove scraper, the server will calculate the scraper bevel angle and scraper deformation angle based on these key points.

[0066] At the beginning of the program, the staff will manually set a baseline, which is parallel to the upper edge of the normal scraper, and only needs to be set once. For example, in the configuration file of the server, the relevant parameters of the baseline, such as slope and intercept, are recorded.

[0067] For each scraper, the server will calculate the slope of the straight line formed by the left and right key points based on the coordinates of the two points. Then, the angle between the straight line and the baseline is calculated using the straight line slope calculation formula. This angle is the scraper bevel angle. For example, assuming that the coordinates of the left key point are (x1, y1) and the coordinates of the right key point are (x2, y2), the slope of the straight line k = (y2-y1) / (x2-x1). Then, based on the slope of the baseline k0, the scraper bevel angle θ is calculated using the angle formula.

[0068] At the same time, the server will calculate the slope k1 of the straight line connecting left and middle and the slope k2 of the straight line connecting right and middle according to the coordinates of the left, middle and right key points. Then, the angle between the two straight lines is calculated by the angle formula of the straight line slopes. This angle is the scraper deformation angle. For example, assuming that the coordinates of the middle key point are (x3, y3), then k1 = (y3-y1) / (x3-x1), k2 = (y3-y2) / (x3-x2), and the scraper deformation angle φ = arctan(|(k1-k2) / (1+k1*k2)|).

[0069] After calculating the scraper tilting angle and the scraper deformation angle, the server will determine the abnormal state of the heavy medium shallow trough separator according to the set alarm rules.

[0070] These alarm rules are formulated by expert experience. For example, based on previous production experience and equipment operation data, the scraper chain jump alarm condition is set to the scraper pull angle greater than 30°, the scraper chain break alarm condition is set to the scraper pull angle greater than 45°, and the scraper deformation alarm condition is set to the scraper deformation angle greater than 20°.

[0071] The server will compare the calculated scraper pull angle and scraper deformation angle with these alarm conditions. For example, for a scraper, if the scraper pull angle is 35°, which is greater than the scraper chain jump alarm condition of 30°, the server will think that the scraper may have a chain jump abnormality; if the scraper pull angle is 50°, which is greater than the scraper chain break alarm condition of 45°, the scraper may have a chain break abnormality; if the scraper deformation angle is 25°, which is greater than the scraper deformation alarm condition of 20°, the scraper is considered to have a deformation abnormality.

[0072] When the server determines that the abnormal state of the heavy medium shallow trough separator is characterized by an abnormality, it will immediately take corresponding measures.

[0073] First, the server will immediately send the shutdown command of the heavy medium shallow trough separator to the main PLC through hard wiring. For example, the specific control program in the server will send a specific electrical signal to the hard wiring interface, and this electrical signal will be transmitted to the main PLC along the hard wiring line. After receiving the shutdown command, the main PLC will control the scraper of the heavy medium shallow trough separator to stop running to avoid further damage to the equipment.

[0074] At the same time, the server will capture a current image as an alarm image. For example, the server will call an image acquisition program to capture a frame of image from the current video data and save it as an alarm image file.

[0075] Then, the server will send the alarm information and alarm image to the alarm platform through the network. The alarm information will include the specific type of abnormality, such as chain jump alarm, chain break alarm or deformation alarm, as well as the time and location of the abnormality. After receiving this information, the alarm platform will promptly display it to relevant personnel so that they can perform maintenance on the equipment in a timely manner. For example, after the maintenance personnel see the scraper deformation alarm information and the corresponding alarm image on the alarm platform, they can quickly understand the abnormal situation of the equipment and rush to the site for maintenance in time.

[0076] Through the above intelligent protection method of heavy medium shallow trough separator based on key point detection, the server can monitor the operating status of the shallow trough scraper in real time, detect abnormalities in time and take corresponding measures, effectively ensuring the safe and stable operation of the heavy medium shallow trough separator.

[0077] In the embodiment of the present invention, the preprocessing of the video data to obtain the preprocessed image data may be implemented through the following examples.

[0078] Acquire the video frame image included in the video data;

[0079] Based on the preset detection range area, each of the video frame images is cropped, filtered, denoised and histogram equalized in sequence to obtain the preprocessed image data; the shallow groove scraper is located within the preset detection range area, and the preprocessed image data is the image data of the movement direction of the shallow groove scraper.

[0080] In the embodiment of the present invention, for example, at the production site of a coal washing plant, a heavy medium shallow trough separator is operating in an orderly manner, and the shallow trough scraper is continuously moving on the track to sort the coal. A camera installed obliquely above the shallow trough scraper continuously collects video data of the shallow trough scraper operation at a set frame rate (for example, 30 frames per second), and transmits the video data to the server through a switch.

[0081] After receiving the video data, the server will start a special video processing program to process the data. The program will first extract the video frame images. Video data is actually composed of a series of continuous video frames, just like a movie film, each frame is a static image. The server program will extract these video frame images from the video data frame by frame in chronological order. For example, at a certain moment, the server received a 10-second video data. According to the frame rate of 30 frames per second, this video data contains 300 video frame images. The server program will extract these 300 video frame images in turn and prepare for subsequent processing.

[0082] In an actual production environment, the video frame images captured by the camera may contain a lot of information irrelevant to the shallow groove scraper, such as the surrounding fences, ground, other equipment, etc. This irrelevant information may interfere with the subsequent detection and analysis of the shallow groove scraper, so it is necessary to crop the video frame images to keep only the parts related to the shallow groove scraper.

[0083] The server pre-stores the relevant parameters of the preset detection range area. This preset detection range area is manually specified. It is a rectangular frame that can just select all the scrapers in the picture and exclude other irrelevant targets. When the server obtains a video frame image, it will crop the image according to these parameters. For example, the size of a video frame image is 1920×1080 pixels, the coordinates of the upper left corner of the preset detection range area are (200,100), and the coordinates of the lower right corner are (1700,900). The server program will crop a sub-image of 1500×800 pixels from the original video frame image based on these coordinate information. This sub-image only contains the shallow groove scraper and a small part of the surrounding related area, which greatly reduces the amount of data for subsequent processing and also reduces the false detection rate of the model.

[0084] In the complex production environment of a coal washing plant, due to the influence of light, dust and other factors, the video frames collected by the camera may contain some noise. These noises will affect the quality of the image, and then affect the subsequent detection and analysis results of the shallow groove scraper. Therefore, the server will filter the cropped video frame images to remove these noises.

[0085] The server uses the median filter algorithm to filter the image. The median filter algorithm is a nonlinear filtering method. Its basic principle is to take the median value of the pixel values ​​in the neighborhood around a certain pixel in the image to replace the value of the pixel. For example, for a pixel in the cropped video frame image, the server will take the pixel values ​​of the 9 pixels in its 3×3 neighborhood, sort these pixel values ​​from small to large, and then take the middle value as the new pixel value of the pixel. By performing this operation on each pixel in the image, the noise points in the image can be effectively removed, making the image clearer.

[0086] In addition to the filtering operation, the server will further perform denoising on the filtered video frame image to improve the image quality. In this process, the server may use some more complex denoising algorithms, such as Gaussian filtering, bilateral filtering, etc.

[0087] Taking Gaussian filtering as an example, Gaussian filtering is a linear smoothing filtering method that achieves the purpose of denoising by performing weighted averaging on each pixel in the image and the pixels in its surrounding neighborhood. The weight of the weighted average is calculated based on the Gaussian function. The closer the pixel is to the center pixel, the greater the weight, and the farther the pixel is from the center pixel, the smaller the weight. The server will perform Gaussian filtering on the filtered video frame image according to the pre-set Gaussian kernel size and standard deviation. For example, if the Gaussian kernel size is set to 5×5 and the standard deviation is set to 1.0, the server program will traverse each pixel in the image, calculate the weighted average of the pixels in the surrounding 5×5 neighborhood, and use this average as the new pixel value of the pixel. After the Gaussian filtering operation, the noise in the image is further removed and the image quality is further improved.

[0088] In the actual production environment of a coal washing plant, due to dim light and other reasons, the contrast of the video frame image may be low, and the edge of the shallow groove scraper may not be clear enough, which will bring difficulties to the subsequent key point detection. To solve this problem, the server will perform histogram equalization on the denoised video frame image to improve the contrast of the image and highlight the edge of the shallow groove scraper.

[0089] The basic principle of histogram equalization is to adjust the grayscale histogram of the image to make the grayscale distribution of the image more uniform, thereby improving the contrast of the image. The server program will first calculate the grayscale histogram of the denoised video frame image and count the number of pixels that appear at each grayscale level. Then, the cumulative distribution function of each grayscale level is calculated based on the grayscale histogram. Finally, the grayscale value of each pixel in the image is remapped according to the cumulative distribution function to obtain the image after histogram equalization. For example, in a denoised video frame image, the grayscale value is mainly concentrated in the range of 0-50. After histogram equalization, the grayscale value will be remapped to the entire range of 0-255. The contrast of the image is significantly improved, and the edge of the shallow groove scraper becomes clearer, providing better image data for subsequent key point detection.

[0090] After the above-mentioned preprocessing steps such as cropping, filtering, denoising and histogram equalization, the server finally obtains the preprocessed image data. These image data are clear and high-quality image data of the movement direction of the shallow groove scraper, which provides a reliable basis for subsequent key point detection and analysis.

[0091] In an embodiment of the present invention, the inputting of the preprocessed image data into a pre-trained key point detection model to obtain the key points of the shallow groove scraper can be implemented through the following examples.

[0092] The preprocessed image data is input into a pre-trained key point detection model to obtain the left key point, the right key point and the middle key point of the shallow groove scraper in the preprocessed image data.

[0093] In the embodiment of the present invention, for example, in the intelligent production control system of a coal washing plant, the server, as the core data processing and analysis center, plays a vital role in the intelligent protection system of a heavy medium shallow trough separator based on key point detection. The following is a detailed scenario example of the process of inputting the pre-processed image data into a pre-trained key point detection model to obtain the left key point, right key point and middle key point of the shallow trough scraper:

[0094] Before the system started running, the server had completed a series of preparations. First, the staff collected and manually annotated a large number of image data sets, reaching more than 8,000 images. These images cover the status of shallow groove scrapers under various working conditions, including normal operation, different degrees of deformation, and pulling. For example, under different lighting conditions, the appearance of the scraper may be different; in different stages of operation, the position and posture of the scraper will also change. By carefully annotating these images, accurate position information is annotated for the left key point, right key point, and middle key point of each scraper, providing a rich and accurate data basis for model training.

[0095] Then, the dataset was further expanded using these labeled data and advanced data enhancement techniques, such as random rotation, flipping, scaling, etc., to increase the generality and diversity of the data. After a series of complex training processes and the application of optimization methods, a high-precision model suitable for shallow groove scraper key point detection was finally trained and deployed on the server's edge computing platform, ready to process and analyze the input image data at any time.

[0096] When the server receives the pre-processed image data, it will immediately start the relevant program module and input the image data into the pre-trained key point detection model. For example, at a certain moment, the server receives a set of pre-processed image data, which is a true reflection of the motion state of the shallow groove scraper at the current moment. The server will transmit the image data to the input interface of the key point detection model in an orderly manner according to the established process.

[0097] Before the image data enters the model, the model will perform some preliminary processing on it. For example, the model will adjust the size of the image to meet the requirements of the model input layer. Assume that the model input layer requires the image size to be 640×640 pixels, and the size of the pre-processed image data received is 800×1000 pixels. The model will adjust the image size to 640×640 pixels through operations such as scaling to ensure that the image data can smoothly enter the model for subsequent processing.

[0098] Once the image data enters the keypoint detection model, the model begins a complex reasoning process. The model uses its internal neural network structure to extract and analyze features in the image. These neural network layers have been trained with a large amount of data and have learned to recognize various features of shallow groove scrapers and patterns of key points.

[0099] During the inference process, the model searches for possible shallow groove scraper targets in the image. For example, the model will perform convolution operations on the image through the convolution layer to extract feature information such as edges and textures in the image. Then, the pooling layer will reduce the dimension and filter these feature information to retain the most representative features. Then, the fully connected layer will classify and locate the targets in the image based on these filtered features to determine whether there is a shallow groove scraper and its approximate location.

[0100] After the model determines the location of the shallow groove scraper, it will further identify the key points of the scraper. The model will search for the left key point, the right key point, and the middle key point in the upper edge area of ​​the scraper based on the key point feature pattern it has learned. For example, the model may determine the specific locations of the three key points by analyzing the line features, color features, and relative position relationship of the upper edge of the scraper with the surrounding environment.

[0101] After a series of complex reasoning and calculations, the model will output the coordinate information of the left key point, right key point and middle key point of the shallow groove scraper in the preprocessed image data. These coordinate information are the model's accurate description of the location of the scraper key points in the image.

[0102] For example, for a preprocessed image, the model outputs the left keypoint coordinates as (100, 200), the right keypoint coordinates as (500, 200), and the middle keypoint coordinates as (300, 200). The server receives these coordinate information and stores them in a specific data structure for subsequent calculation and analysis.

[0103] After obtaining the coordinate information of these key points, the server can perform further processing based on this information, such as calculating the scraper pull angle and scraper deformation angle, so as to achieve intelligent protection and fault warning for the heavy medium shallow slot separator. Through this key point detection-based method, the server can accurately and quickly identify the key information of the shallow slot scraper, providing strong technical support for ensuring the safe and stable operation of the equipment.

[0104] In an embodiment of the present invention, the preprocessed image data is input into a pre-trained key point detection model to obtain the left key point, right key point and middle key point of the shallow groove scraper in the preprocessed image data, which can be implemented through the following example.

[0105] Inputting the preprocessed image data into a pre-trained key point detection model to obtain a plurality of first pending detection objects and a confidence level corresponding to each of the first pending detection objects;

[0106] Filtering the first pending detection objects whose confidence levels are lower than a preset confidence threshold to obtain a plurality of second pending detection objects;

[0107] Obtaining a target frame size and aspect ratio of each of the second to-be-determined detection objects;

[0108] Performing secondary filtering on the target frame size and length-width ratio of each of the second pending detection objects according to the preset distance between key points and the length-width ratio, to obtain a plurality of third pending detection objects;

[0109] The third pending detection object with a relatively large intersection is filtered out by non-maximum suppression to obtain the left key point, the right key point and the middle key point.

[0110] In the embodiment of the present invention, for example, after receiving the pre-processed shallow groove scraper image data, the server will input it into a pre-trained key point detection model. This model has been trained with a large amount of data and has the ability to identify the characteristics of the shallow groove scraper.

[0111] For example, at a certain moment, the server receives a set of pre-processed image data from the camera, which clearly shows the motion state of the shallow groove scraper. When the server inputs these image data into the key point detection model, the model will conduct a comprehensive analysis and recognition of the image. The model will search for possible shallow groove scraper targets in the image and generate a corresponding detection result for each target found, that is, the first pending detection object. At the same time, the model will assign a confidence level to each first pending detection object, which indicates the model's confidence level that the detection object is a shallow groove scraper.

[0112] Assume that in this set of images, the model identified 5 possible shallow groove scraper targets, that is, 5 first pending detection objects were obtained. The confidences of these 5 first pending detection objects are 0.9, 0.8, 0.6, 0.4 and 0.3 respectively. These confidences are calculated by the model based on the degree of match between the features of the detection object and the shallow groove scraper features in the training data.

[0113] In order to ensure the accuracy of the detection results, the server will set a preset confidence threshold. This threshold is determined based on actual application scenarios and experience. Generally speaking, detection objects with confidence levels higher than the threshold are considered to be more reliable.

[0114] Assume that the confidence threshold preset by the server is 0.7. In the above example, the confidences of the five first pending detection objects are 0.9, 0.8, 0.6, 0.4 and 0.3 respectively. The server will compare these confidences and filter out the first pending detection objects with confidences lower than 0.7. After filtering, the two first pending detection objects with confidences of 0.9 and 0.8 are retained and become the second pending detection objects. The purpose of this is to exclude those detection objects that the model is not sure about and improve the accuracy of subsequent detection results.

[0115] After obtaining the second pending detection object, the server will further obtain the target frame size and aspect ratio of each second pending detection object. The target frame is a rectangular frame generated by the model for the detection object when it is identified, and is used to mark the position and range of the detection object in the image.

[0116] For example, for the two remaining second pending detection objects, the server will obtain their target frame sizes respectively. Assuming that the target frame size of the first second pending detection object is 200 pixels long and 100 pixels wide, its aspect ratio is 2:1. The target frame size of the second second pending detection object is 180 pixels long and 90 pixels wide, and its aspect ratio is also 2:1. These target frame sizes and aspect ratio information will be used as the basis for subsequent secondary filtering.

[0117] The server pre-stores the average values ​​of the distances between preset key points and the length-to-width ratios calculated based on historical data. These average values ​​represent the characteristics of the shallow groove scraper under normal circumstances.

[0118] In this example, the preset normal shallow groove scraper target frame length-width ratio is 2:1, and the average distance ratio between key points is 1:1. The server will compare the target frame size and length-width ratio of each second pending detection object with these preset values. If the target frame size and length-width ratio of a second pending detection object differ greatly from the preset values, it means that the detection object may not be a real shallow groove scraper and will be filtered out.

[0119] For example, suppose a new second pending detection object appears in a subsequent image, and its target frame size is 300 pixels long and 50 pixels wide, with an aspect ratio of 6:1, which is quite different from the preset aspect ratio average of 2:1. The server will consider this detection object to be abnormal and filter it out. As for the first two second pending detection objects with an aspect ratio of 2:1, since their target frame size and aspect ratio are consistent with the preset values, they will be retained and become the third pending detection object.

[0120] In some cases, there may be multiple target frames of the third pending detection object overlapping each other. This may be caused by the model identifying the same shallow groove scraper multiple times or there are some interference factors. To avoid such repeated identification, the server will use the non-maximum suppression method to process it.

[0121] For example, suppose there are two target frames of the third undetermined object in the image, and their intersection-over-union (IOU) is 0.8, indicating that the two target frames highly overlap. The non-maximum suppression method will select the target frame with higher confidence and filter out the other one. Suppose the confidence corresponding to one of the target frames is 0.9 and the other is 0.85, then the server will retain the target frame with confidence 0.9.

[0122] After non-maximum suppression processing, the server finally obtains accurate detection objects, and then extracts the coordinate information of the left key points, right key points, and middle key points of the shallow trough scraper from these detection objects. For example, for the shallow trough scraper corresponding to the retained target frame, the server determines that its left key point coordinates are (100,200), the right key point coordinates are (300,200), and the middle key point coordinates are (200,200). These key point information will be used for subsequent calculations of scraper pull angles and scraper deformation angles, thereby realizing intelligent protection and fault warning for heavy medium shallow trough separators.

[0123] In the embodiment of the present invention, the scraper bevel angle and the scraper deformation angle of the shallow groove scraper are calculated according to the key points, which can be implemented through the following examples.

[0124] The angle between the straight line formed by connecting the left key point and the right key point and the reference line is used as the scraper bevel angle; the reference line is parallel to the upper edge line of the shallow groove scraper in normal operation;

[0125] The angle between the straight line connecting the left key point and the middle key point and the straight line connecting the right key point and the middle key point is used as the scraper deformation angle.

[0126] In the embodiment of the present invention, for example, in the daily production of the coal washing plant, the heavy medium shallow trough separator is continuously running, and the server processes the image data transmitted from the camera in real time. At a certain moment, the server has successfully obtained the coordinate information of the left key point and the right key point of the shallow trough scraper. For example, the coordinates of the left key point are (150, 200), and the coordinates of the right key point are (450, 200).

[0127] The server will determine the straight line formed by the two key points based on their coordinates. Imagine that on the image, these two points determine a straight line, which represents a characteristic line in the left and right directions of the scraper in the current state.

[0128] At the same time, the server stores relevant information about the baseline. The baseline is set manually by the staff at the beginning of the program based on the upper edge of the normal shallow groove scraper. This baseline is like a standard reference to measure whether the scraper has tilted.

[0129] When the server obtains the straight line connecting the left and right key points and the baseline, it will calculate the angle between the two straight lines through a specific algorithm. This angle is the scraper bevel angle. For example, in the current example, the server calculates that the scraper bevel angle is 0°, which means that the scraper is parallel to the baseline in the normal operating state in the left and right directions, and there is no bevel.

[0130] However, in the actual production process, the scraper may be tilted due to various reasons. For example, at another moment, the server obtains the new left key point coordinates to (120,210), and the right key point coordinates to (480,190). At this time, the straight line formed by the left and right key points has changed relative to the previous one. The server calculated again and found that the scraper tilt angle was 3.18°. This angle indicates that the scraper has tilted to a certain extent. The server will record this angle value for subsequent comparison with the preset alarm rules to determine whether an alarm signal needs to be issued.

[0131] After obtaining the coordinate information of the left key point, middle key point and right key point of the shallow groove scraper, the server will further calculate the scraper deformation angle. For example, at a specific production moment, the server determines that the left key point coordinate of the shallow groove scraper is (130,220), the middle key point coordinate is (300,210), and the right key point coordinate is (470,200).

[0132] The server first determines the straight line formed by the left key point and the middle key point based on their coordinates. This straight line reflects the relative position relationship between the left part and the middle part of the scraper. Similarly, the server determines another straight line based on the coordinates of the right key point and the middle key point, which reflects the relative position relationship between the right part and the middle part of the scraper.

[0133] Next, the server will calculate the angle between the two lines, which is the scraper deformation angle. In this example, the server calculated that the scraper deformation angle is 0°, which means that at this moment, the left, middle and right parts of the scraper are normal in relative position and have not deformed.

[0134] However, in actual production, the scraper may be deformed due to various factors. For example, at a certain point in the future, the coordinates of the left key point obtained by the server become (120, 230), the coordinates of the middle key point become (310, 200), and the coordinates of the right key point become (480, 180). At this time, the relative positions of the straight line connecting the left key point and the middle key point and the straight line connecting the right key point and the middle key point have changed. The server recalculates and finds that the deformation angle of the scraper is 4.5°. This angle indicates that the scraper has been deformed to a certain extent. The server will record this angle value and compare it with the preset alarm rules. If the preset threshold is exceeded, the server will determine that the equipment is abnormal, and then take corresponding measures, such as initiating a shutdown command and outputting an alarm signal to ensure the safe and stable operation of the equipment.

[0135] Through such a calculation process, the server can accurately calculate the scraper pulling angle and the scraper deformation angle according to the key point information, thereby providing strong support for timely detection and processing of abnormal conditions of the heavy medium shallow trough separator.

[0136] In the embodiment of the present invention, the determination of the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle can be implemented through the following examples.

[0137] Determine whether a scraper chain jumping alarm condition or a scraper chain breaking alarm condition is met according to the scraper inclined angle;

[0138] Determining whether a scraper deformation alarm condition is met according to the scraper deformation angle;

[0139] If satisfied, then determining the abnormal state corresponding to the heavy medium shallow trough separator;

[0140] If not, it is determined that there is no abnormality in the heavy medium shallow trough separator.

[0141] In the embodiment of the present invention, for example, at the production site of a coal washing plant, a heavy medium shallow trough separator is continuously and stably operated, and the server receives and processes the image data transmitted from the camera in real time to calculate the scraper pull angle. For example, at a certain moment, the server calculates the current shallow trough scraper pull angle to be 25° based on the image data.

[0142] The server pre-stores the scraper chain jumping alarm conditions and scraper chain breaking alarm conditions formulated by expert experience. Assume that the scraper chain jumping alarm condition is set to the scraper pull angle greater than 30°, and the scraper chain breaking alarm condition is set to the scraper pull angle greater than 45°.

[0143] The server will compare the calculated scraper pull angle with these two alarm conditions. In the current example, 25° is less than 30°, which does not meet the scraper chain jump alarm condition; at the same time, 25° is also less than 45°, which does not meet the scraper chain break alarm condition. This shows that at this moment, from the scraper pull angle, the heavy medium shallow trough separator has no abnormal chain jump or chain break.

[0144] However, as the production process continues, the situation may change. At another moment, the scraper pull angle calculated by the server becomes 38°. At this time, 38° is greater than 30°, which meets the scraper chain jump alarm condition, but is less than 45°, which does not meet the scraper chain break alarm condition. The server will record this situation, indicating that the heavy medium shallow trough separator may have a scraper chain jump abnormality at this time.

[0145] While determining whether the scraper tilt angle meets the alarm condition, the server also determines whether the scraper deformation alarm condition is met based on the calculated scraper deformation angle. For example, at a certain moment, the server calculates that the scraper deformation angle is 15°.

[0146] Similarly, the server pre-sets the scraper deformation alarm condition, assuming that the scraper deformation angle is greater than 20°. The server compares the calculated 15° with 20° and finds that 15° is less than 20°, which does not meet the scraper deformation alarm condition. This means that at this moment, from the perspective of scraper deformation, the scraper of the heavy medium shallow trough separator has no obvious deformation abnormality.

[0147] However, in the subsequent production process, due to various factors, the deformation angle of the scraper may change. For example, at a certain moment, the scraper deformation angle calculated by the server becomes 23°. At this time, 23° is greater than 20°, which meets the scraper deformation alarm condition. Based on this, the server will determine that the heavy medium shallow tank separator may have an abnormal scraper deformation.

[0148] When the server determines that the corresponding alarm conditions are met based on the scraper tilting angle or the scraper deformation angle, the corresponding abnormal state of the heavy medium shallow trough separator will be determined.

[0149] For example, at a certain moment, the server calculated that the scraper pull angle was 42° and the scraper deformation angle was 21°. By comparing with the pre-set alarm conditions, 42° is greater than 30° and less than 45°, which meets the scraper chain jump alarm condition; 21° is greater than 20°, which meets the scraper deformation alarm condition. The server will combine these two results and determine that the abnormal state of the heavy medium shallow trough separator at this time may have both scraper chain jump problems and scraper deformation problems.

[0150] The server will record the abnormal state and prepare to take corresponding measures. For example, after determining the abnormal state, the server will immediately generate a detailed abnormal report, including the time when the abnormality occurred, the scraper tilt angle, the scraper deformation angle, and the corresponding abnormality type, so as to facilitate subsequent analysis and processing.

[0151] When the server determines that the corresponding alarm conditions are not met based on the scraper tilting angle and the scraper deformation angle, it will be determined that there is no abnormality in the heavy medium shallow trough separator.

[0152] For example, at a certain moment, the server calculated that the scraper pull angle was 22° and the scraper deformation angle was 18°. 22° is less than 30°, which does not meet the scraper chain jump alarm condition and the scraper chain break alarm condition; 18° is less than 20°, which does not meet the scraper deformation alarm condition. Based on these comparison results, the server determines that the heavy medium shallow trough separator is currently in normal operation and no abnormal situation has occurred.

[0153] The server will regularly check and record the status of the heavy medium shallow trough separator. Even if it is judged that there is no abnormality, the relevant data will be stored for subsequent analysis and statistics of the equipment's operating status. For example, the server will store information such as the time of each inspection, the scraper pull angle, and the scraper deformation angle in the database to form a historical record of the equipment's operating status. In this way, the staff can understand the long-term operating trend of the equipment by querying these historical records, and promptly discover potential problems and perform maintenance.

[0154] Through the above detailed judgment process, the server can accurately determine the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle, providing strong support for ensuring the safe and stable operation of the equipment.

[0155] In the embodiment of the present invention, when the abnormal state is characterized by the presence of an abnormality, initiating a shutdown instruction and outputting an alarm signal can be implemented through the following examples.

[0156] When the abnormal state is characterized by at least one of the following abnormal states: scraper chain jumping, scraper chain breaking and scraper deformation, a stop command of the heavy medium shallow trough separator is sent to the main PLC through hard wiring to control the scraper of the heavy medium shallow trough separator to stop, and the alarm signal is sent to the alarm platform through the network; the alarm signal includes the current image captured by the camera.

[0157] In the embodiment of the present invention, the server monitors the operation status of the equipment in real time. At a certain moment, the server determines that the current heavy medium shallow trough separator is in an abnormal state based on the analysis results of the scraper tilting angle and the scraper deformation angle, which is specifically manifested as abnormal scraper chain jumping.

[0158] There is a dedicated module inside the server to handle abnormal situations. When a scraper chain jump abnormality is detected, this module will immediately trigger the corresponding shutdown command generation program. The program will communicate with the hard-wired interface of the server and send a shutdown command to the main PLC through the hard wiring.

[0159] Hard wiring is a reliable physical connection method that ensures fast and accurate transmission of instructions. On the server side, the hard wiring interface is connected to a specific circuit. When the shutdown instruction generation program sends electrical signals to the hard wiring interface, these electrical signals are quickly transmitted to the main PLC along the hard wiring line.

[0160] As the core control unit of the heavy medium shallow trough separator, the main PLC is always ready to receive instructions from the server. When the main PLC receives a stop command, it will immediately respond and perform the corresponding operation. The main PLC will send a stop signal to the motor controller that drives the scraper. After receiving the stop signal, the motor controller will cut off the power supply of the motor, thereby stopping the motor. As the motor stops, the scraper of the heavy medium shallow trough separator will also stop moving.

[0161] For example, in another scenario, the server detected that the heavy medium shallow trough separator had two abnormal conditions: broken scraper chain and deformed scraper. The server's abnormality handling module would also trigger the shutdown command generation program, and send the shutdown command to the main PLC through hard wiring. After receiving the command, the main PLC took prompt action to control the scraper to stop moving, avoiding the equipment from continuing to operate in an abnormal state and causing more serious damage.

[0162] When the server sends a shutdown command to the main PLC through hard wiring, it also sends an alarm signal to the alarm platform so that relevant personnel can understand the abnormal situation of the equipment in a timely manner.

[0163] When the server determines that the heavy medium shallow trough separator is in an abnormal state, it will immediately send a capture command to the camera. After receiving the capture command, the camera will quickly capture a current image. This image can truly reflect the specific situation of the equipment in the abnormal state, including the position and posture of the scraper and the surrounding environment.

[0164] The camera transmits the captured image data back to the server through the network. After receiving the image data, the server will integrate it into the alarm signal. In addition to the captured current image, the alarm signal also contains detailed information about the abnormal state, such as the abnormal type (scraper chain jumping, scraper chain breaking or scraper deformation), occurrence time, scraper pull angle, scraper deformation angle, etc.

[0165] The server sends the alarm signal to the alarm platform through the network. The alarm platform is a system dedicated to receiving and processing equipment alarm information. It usually has an intuitive user interface to facilitate relevant personnel to view and process alarm information.

[0166] For example, when the server detects abnormal deformation of the scraper and sends an alarm signal to the alarm platform, the alarm platform will immediately pop up an alarm window to display relevant information about the abnormality. After opening the alarm window, the relevant personnel will first see the current image captured, through which they can visually observe the deformation of the scraper. At the same time, the alarm window will also display detailed information such as the abnormality type "scraper deformation", the occurrence time "XX year XX month XX day XX hour XX minute", and the scraper deformation angle "XX°".

[0167] For another example, when the server detects the abnormal situation of scraper chain jumping and scraper chain breaking at the same time, the alarm signal sent will contain the corresponding detailed information and snapshot images. After receiving the alarm signal, the alarm platform will highlight this serious abnormal situation and remind relevant personnel to take immediate measures to deal with it. Relevant personnel can quickly determine the cause of the equipment failure based on the information in the alarm signal, and organize maintenance personnel to carry out emergency repairs to restore the normal operation of the equipment as soon as possible.

[0168] In this way, after detecting the abnormal state of the heavy medium shallow trough sorting machine, the server can promptly initiate a shutdown command and output an alarm signal, effectively ensuring the safe operation of the equipment and reducing production losses that may be caused by equipment failure.

[0169] In order to clearly express the solution provided by the embodiment of the present invention, a more complete implementation method is provided below.

[0170] A network camera is installed diagonally above the shallow groove scraper, with the view facing the direction of the scraper's movement. The video stream signal is transmitted to the edge computing device through the switch. When the edge computing device detects the video frame image, it first cuts out the ROI (detection range) area in the image, filters out the surrounding fence area that may affect the detection effect, and reduces the false detection rate of the model. This ROI area is manually specified and can select all scrapers in the picture while excluding other irrelevant targets. Then, the image of the ROI area is input into the yolov8-pose algorithm model. After the model is inferred, each target frame and the corresponding key points and their confidence are output. Then, the real detection target is selected through a series of post-processing steps. The post-processing steps are as follows: 1. Filter out the identified target objects whose confidence exceeds the threshold, and exclude the target objects with low confidence; 2. Compare the target frame size and its aspect ratio, and the distance between key points and their ratio with the mean of the identified target object, and further filter out the targets with large differences from the mean. The target frame size and its aspect ratio, and the mean of the distance between key points and their ratio are calculated from historical data; 3. The non-maximum suppression method is used to further filter out the identified objects with large IOU (intersection-over-union ratio) to avoid re-identification. Finally, the key point coordinates of each identified object processed by the post-processing steps are output, that is, the three key points on the upper edge of each scraper (such as Figure 2 As shown in the figure, the three points are the left key point (left key point), the middle key point (middle key point), and the right key point (right key point). The angle between the straight line formed by connecting left and right and the reference line (the line horizontal to the upper edge direction of the normal scraper) is used as the scraper bevel angle. The angle between the straight line connecting left and middle and the straight line connecting right and middle is used as the scraper deformation angle. After calculating the two angles, it is judged whether the scraper bevel angle meets the scraper chain jumping alarm condition or the scraper chain breaking alarm condition, and whether the scraper deformation angle meets the scraper deformation alarm condition. These conditions are determined by expert experience. After the alarm condition is met, the edge computing device immediately sends the heavy medium shallow trough separator shutdown command to the main PLC through hard wiring, controls the heavy medium shallow trough separator scraper to stop, and sends an alarm signal to the alarm platform through the network.

[0171] It mainly includes the following three steps:

[0172] The scraper key point detection model is used to detect the three key points on the upper edge of the scraper in the video stream image.

[0173] The scraper pulling angle and scraper deformation angle are calculated, and abnormal judgment is made using the set alarm rules.

[0174] When the judgment result meets the alarm condition, a shutdown command is issued, and an alarm message and an alarm image are issued at the same time.

[0175] In step 1: training the scraper key point detection model and obtaining the scraper key points includes the following steps:

[0176] Collect and manually annotate more than 8,000 image data sets, perform data enhancement on the image data to increase the generality of the data, and then train the final model through this data and optimization methods.

[0177] The final model is deployed to the edge computing platform. When an image frame is transmitted from the network camera to the platform, the program first performs ROI cropping, filtering, and denoising on the image. Because the light is dim and the picture is blurry, the image contrast is also improved through histogram equalization to highlight the edge of the scraper. The preprocessed image is then input into the model, and the model outputs recognition information. Finally, effective key point location information is extracted from the output information of the model through post-processing.

[0178] In step 2: calculate the scraper pulling angle and the scraper deformation angle, and use the set alarm rules to make judgments.

[0179] When the program is initialized, a reference line is manually set. The reference line is parallel to the upper edge line of the normally operating scraper and only needs to be set once.

[0180] The angle between the straight line formed by connecting left and right and the reference line is used as the scraper bevel angle. The angle between the straight line connecting left and middle and the straight line connecting right and middle is used as the scraper deformation angle. After calculating the two angles, determine whether the scraper bevel angle meets the scraper chain jump alarm condition or the scraper chain break alarm condition, and determine whether the scraper deformation angle meets the scraper deformation alarm condition.

[0181] In step 3: when the judgment result meets the alarm condition, an alarm signal is sent out while sending out alarm information and an alarm image.

[0182] (1) When any alarm condition is met, the edge computing device immediately sends a shutdown command of the heavy medium shallow trough sorting machine to the main PLC through hard wiring, and at the same time captures a current image and sends the alarm information and alarm image to the alarm platform through the network.

[0183] In summary, the heavy medium shallow trough separator protection technology based on key point detection provided by the embodiment of the present invention can make up for the defects of the existing shallow trough scraper protection technology. A network camera is used to obtain a real-time video stream of the shallow trough scraper during operation, and the key points of the scraper are detected based on the improved yolov8-pose algorithm. When it is detected that the deformation angle of the scraper is too large, the edge computing device immediately sends a shutdown command of the heavy medium shallow trough separator to the main PLC through hard wiring, controls the equipment to stop, and simultaneously sends alarm information and alarm images to the alarm platform. So that relevant personnel can perform maintenance on the equipment in time. The device of the present invention has a simple and clear structure and is easy to install and maintain.

[0184] Please refer to Figure 3 , Figure 3 An intelligent protection device 110 for a heavy-medium shallow trough sorting machine based on key point detection provided in an embodiment of the present invention is applied to an edge device included in an intelligent protection system for a heavy-medium shallow trough sorting machine, wherein the intelligent protection system for a heavy-medium shallow trough sorting machine also includes a camera. The device includes:

[0185] The acquisition module 1101 is used to acquire the video data of the shallow groove scraper included in the heavy medium shallow groove separator collected by the camera; the camera is located at a preset position facing the movement direction of the shallow groove scraper; the video data is preprocessed to obtain preprocessed image data; the preprocessed image data is input into a pre-trained key point detection model to obtain the key points of the shallow groove scraper; the scraper pull angle and the scraper deformation angle of the shallow groove scraper are calculated according to the key points;

[0186] The protection module 1102 is used to determine the abnormal state of the heavy medium shallow tank separator according to the scraper tilting angle and the scraper deformation angle; when the abnormal state is characterized by the existence of an abnormality, initiate a shutdown command and output an alarm signal.

[0187] It should be noted that the implementation principle of the aforementioned intelligent protection device 110 for heavy-medium shallow trough sorting machine based on key point detection can refer to the implementation principle of the aforementioned intelligent protection method for heavy-medium shallow trough sorting machine based on key point detection, which will not be repeated here. It should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the intelligent protection device 110 for heavy-medium shallow trough sorting machine based on key point detection can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the intelligent protection device 110 for heavy-medium shallow trough sorting machine based on key point detection. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in a processor element or an instruction in software form.

[0188] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0189] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned intelligent protection device 110 for heavy medium shallow trough separator based on key point detection. Figure 4 As shown, Figure 4The computer device 100 provided in the embodiment of the present invention is a structural block diagram. The computer device 100 includes an intelligent protection device 110 for a heavy medium shallow tank separator based on key point detection, a memory 111, a processor 112 and a communication unit 113.

[0190] In order to realize data transmission or interaction, the memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly. For example, the electrical connection between these elements can be realized through one or more communication buses or signal lines. The intelligent protection device 110 for heavy-medium shallow trough sorting machine based on key point detection includes at least one software function module that can be stored in the memory 111 in the form of software or firmware or solidified in the operating system (OS) of the computer device 100. The processor 112 is used to execute the intelligent protection device 110 for heavy-medium shallow trough sorting machine based on key point detection stored in the memory 111, such as the software function modules and computer programs included in the intelligent protection device 110 for heavy-medium shallow trough sorting machine based on key point detection.

[0191] An embodiment of the present invention provides a readable storage medium, which includes a computer program. When the computer program is running, it controls the computer device where the readable storage medium is located to execute the aforementioned intelligent protection device 110 for the heavy medium shallow trough sorting machine based on key point detection.

[0192] For illustrative purposes, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Numerous modifications and variations are possible in accordance with the above teachings. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application.

Claims

1. An intelligent protection method for a heavy-medium shallow trough sorting machine based on key point detection, applied to an edge device included in an intelligent protection system for a heavy-medium shallow trough sorting machine, wherein the intelligent protection system for a heavy-medium shallow trough sorting machine also includes a camera, characterized in that: The method comprises: Acquiring video data of a shallow groove scraper included in the heavy medium shallow groove separator collected by the camera; the camera is located at a preset position facing the movement direction of the shallow groove scraper; Preprocessing the video data to obtain preprocessed image data; Inputting the preprocessed image data into a pre-trained key point detection model to obtain the key points of the shallow groove scraper; The scraper bevel angle and the scraper deformation angle of the shallow groove scraper are calculated according to the key points; Determining the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle; When the abnormal state is characterized by the presence of an abnormality, a shutdown command is initiated and an alarm signal is output.

2. The method according to claim 1, characterized in that The preprocessing of the video data to obtain preprocessed image data includes: Acquire the video frame image included in the video data; Based on the preset detection range area, each of the video frame images is cropped, filtered, denoised and histogram equalized in sequence to obtain the preprocessed image data; the shallow groove scraper is located within the preset detection range area, and the preprocessed image data is the image data of the movement direction of the shallow groove scraper.

3. The method according to claim 1, characterized in that The step of inputting the pre-processed image data into a pre-trained key point detection model to obtain the key points of the shallow groove scraper comprises: The preprocessed image data is input into a pre-trained key point detection model to obtain the left key point, the right key point and the middle key point of the shallow groove scraper in the preprocessed image data.

4. The method according to claim 3, characterized in that The method of inputting the pre-processed image data into a pre-trained key point detection model to obtain the left key point, the right key point and the middle key point of the shallow groove scraper in the pre-processed image data comprises: Inputting the preprocessed image data into a pre-trained key point detection model to obtain a plurality of first pending detection objects and a confidence level corresponding to each of the first pending detection objects; Filtering the first pending detection objects whose confidence levels are lower than a preset confidence threshold to obtain a plurality of second pending detection objects; Obtaining a target frame size and aspect ratio of each of the second to-be-determined detection objects; Performing secondary filtering on the target frame size and length-width ratio of each of the second pending detection objects according to the preset distance between key points and the length-width ratio, to obtain a plurality of third pending detection objects; The third pending detection object with a relatively large intersection is filtered out by non-maximum suppression to obtain the left key point, the right key point and the middle key point.

5. The method according to claim 3, characterized in that: The scraper pull angle and the scraper deformation angle of the shallow groove scraper are calculated according to the key points, including: The angle between the straight line formed by connecting the left key point and the right key point and the reference line is used as the scraper bevel angle; the reference line is parallel to the upper edge line of the shallow groove scraper in normal operation; The angle between the straight line connecting the left key point and the middle key point and the straight line connecting the right key point and the middle key point is used as the scraper deformation angle.

6. The method according to claim 1, characterized in that The method of determining the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle includes: Determine whether a scraper chain jumping alarm condition or a scraper chain breaking alarm condition is met according to the scraper inclined angle; Determining whether a scraper deformation alarm condition is met according to the scraper deformation angle; If satisfied, then determining the abnormal state corresponding to the heavy medium shallow trough separator; If not, it is determined that there is no abnormality in the heavy medium shallow trough separator.

7. The method according to claim 6, characterized in that Initiating a shutdown command and outputting an alarm signal when the abnormal state is characterized as an abnormality includes: When the abnormal state is characterized by at least one of the following abnormal states: scraper chain jumping, scraper chain breaking and scraper deformation, a stop command of the heavy medium shallow trough separator is sent to the main PLC through hard wiring to control the scraper of the heavy medium shallow trough separator to stop, and the alarm signal is sent to the alarm platform through the network; the alarm signal includes the current image captured by the camera.

8. An intelligent protection device for a heavy medium shallow trough sorting machine based on key point detection, applied to an edge device included in an intelligent protection system for a heavy medium shallow trough sorting machine, wherein the intelligent protection system for a heavy medium shallow trough sorting machine also includes a camera, characterized in that: The device comprises: An acquisition module is used to acquire video data of a shallow groove scraper included in a heavy medium shallow groove separator collected by the camera; the camera is located at a preset position facing the movement direction of the shallow groove scraper; the video data is preprocessed to obtain preprocessed image data; the preprocessed image data is input into a pre-trained key point detection model to obtain key points of the shallow groove scraper; and the scraper pull angle and the scraper deformation angle of the shallow groove scraper are calculated according to the key points; The protection module is used to determine the abnormal state of the heavy medium shallow trough separator according to the scraper tilting angle and the scraper deformation angle; when the abnormal state is characterized by the existence of an abnormality, initiate a shutdown command and output an alarm signal.

9. A computer device, characterized in that: The computer device comprises a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device executes the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images